View source for ReleaseEngineering/How To/Upload to internal Pypi

__TOC__

== Overview ==
Mozharness no longer uses packages from the PuppetAgain repositories! Instead, it uses http://pypi.pub.build.mozilla.org/pub and http://pypi.pvt.build.mozilla.org/pub, both served from the same directory.

== Steps ==
In order to upload to internal Pypi a package you simply need to run the script bellow :
* https://hg.mozilla.org/build/braindump/file/default/utils/publish_package_our_pypi.sh

=== Downloading or Updating the repo ===
'''publish_package_our_pypi.sh''' is available on the [https://hg.mozilla.org/build/braindump/file/default/utils/publish_package_our_pypi.sh braindump repository] which can be clone using:
<pre>hg clone https://hg.mozilla.org/build/braindump/</pre>

''Note:'' Make sure that you are using the latest revision of the script. You can update your repo with '''hg pull''' from within the local repository folder.


=== Obtaining Package File(s) ===
Next you'll need to obtain the package file(s) you intend to upload. Sometimes the requester will provide the desired urls directly, in which case you can wget them and verify their md5 hash.

If a package version was requested, you can use the 'utils/download_from_pypi' script, e.g:
<pre> ./download_from_pypi scipy==1.2.3</pre>

This will download all release files for the specified version of the package and verify their sha256 hash. Multiple packages can also be specified:
<pre>./download_from_pypi scipy==1.2.3 Pillow==6.1.0</pre>

This might result in more files than you need to upload (e.g, maybe you don't care about Python 2.7 wheels). You can use shell expansion to delete any packages you don't need:
<pre>rm *cp27* *.egg *.exe</pre>

The downloaded files can now be fed into the upload script.

=== Uploading to PyPi ===

First ensure your repository is up-to-date.

The upload script needs to use your LDAP username to upload. If your machine user is the same as your LDAP, all you have to do is run the following command:
<pre>./publish_package_our_pypi.sh <path/to/file1> <path/to/file2> ...</pre>

If your machine user is different than your LDAP, you can set it with the $USER environment variable:
<pre>USER=<ldap username> ./publish_package_our_pypi.sh <path/to/file1> <path/to/file2> ...</pre>

The script will practically copy the file to relengwebadmin and move it where the files are located for pypi internal (/mnt/netapp/relengwebadmin/pypi/pub) then will change the rights and upload the file to 
http://pypi.pub.build.mozilla.org/pub.

If everything worked fine, you should now have a link that looks like this:
'''http://pypi.pub.build.mozilla.org/pub/FILE-NAME-HERE'''

== See also ==
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1286278 bug 1286278]