ReleaseEngineering/How To/Upload to internal Pypi: Difference between revisions
(Updated the page to contain a few more information.) |
No edit summary |
||
| Line 14: | Line 14: | ||
''Note:'' Make sure that you are using the latest revision of the script. You can update your repo with '''hg pull''' from within the local repository folder. | ''Note:'' Make sure that you are using the latest revision of the script. You can update your repo with '''hg pull''' from within the local repository folder. | ||
=== | |||
=== Obtaining Package File(s) === | |||
Next you'll need to obtain the package file(s) you intend to upload. Sometimes the requester will provide the desired urls directly, in which case you can wget them and verify their md5 hash. | |||
If a package version was requested, you can use the 'utils/download_from_pypi' script, e.g: | |||
<pre> ./download_from_pypi scipy==1.2.3</pre> | |||
This will download all release files for the specified version of the package and verify their sha256 hash. Multiple packages can also be specified: | |||
<pre>./download_from_pypi scipy==1.2.3 Pillow==6.1.0</pre> | |||
This might result in more files than you need to upload (e.g, maybe you don't care about Python 2.7 wheels). You can use shell expansion to delete any packages you don't need: | |||
<pre>rm *cp27* *.egg *.exe</pre> | |||
The downloaded files can now be fed into the upload script. | |||
=== Uploading to PyPi === | |||
The upload script expects that your machine/vm will have the same username as your Mozilla LDAP. If you aren't sure how to check this, open a new terminal and type: | The upload script expects that your machine/vm will have the same username as your Mozilla LDAP. If you aren't sure how to check this, open a new terminal and type: | ||
<pre>echo "Machine username is: $(whoami)"</pre> | <pre>echo "Machine username is: $(whoami)"</pre> | ||
| Line 20: | Line 35: | ||
If the machine username is not the same as your Mozilla LDAP, we recommend you to either create a new username with your Mozilla LDAP username or change '''$(whoami)''' inside the script to be equal to your LDAP username. | If the machine username is not the same as your Mozilla LDAP, we recommend you to either create a new username with your Mozilla LDAP username or change '''$(whoami)''' inside the script to be equal to your LDAP username. | ||
If your repository is up-to-date and your machine user is the same with your LDAP, all you have to do now is run the following command: | |||
If your repository is up-to-date and your machine user is the same with your LDAP, all you have to do now is | <pre>./publish_package_our_pypi.sh <path/to/file1> <path/to/file2> ...</pre> | ||
<pre> | |||
The script will practically copy the file to relengwebadmin and move it where the files are located for pypi internal (/mnt/netapp/relengwebadmin/pypi/pub) then will change the rights and upload the file to | The script will practically copy the file to relengwebadmin and move it where the files are located for pypi internal (/mnt/netapp/relengwebadmin/pypi/pub) then will change the rights and upload the file to | ||
| Line 29: | Line 43: | ||
If everything worked fine, you should now have a link that looks like this: | If everything worked fine, you should now have a link that looks like this: | ||
'''http://pypi.pub.build.mozilla.org/pub/FILE-NAME-HERE''' | '''http://pypi.pub.build.mozilla.org/pub/FILE-NAME-HERE''' | ||
== See also == | == See also == | ||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1286278 bug 1286278] | * [https://bugzilla.mozilla.org/show_bug.cgi?id=1286278 bug 1286278] | ||
Revision as of 20:15, 18 March 2021
Overview
Mozharness no longer uses packages from the PuppetAgain repositories! Instead, it uses http://pypi.pub.build.mozilla.org/pub and http://pypi.pvt.build.mozilla.org/pub, both served from the same directory.
Steps
In order to upload to internal Pypi a package you simply need to run the script bellow :
Downloading or Updating the repo
publish_package_our_pypi.sh is available on the braindump repository which can be clone using:
hg clone https://hg.mozilla.org/build/braindump/
Note: Make sure that you are using the latest revision of the script. You can update your repo with hg pull from within the local repository folder.
Obtaining Package File(s)
Next you'll need to obtain the package file(s) you intend to upload. Sometimes the requester will provide the desired urls directly, in which case you can wget them and verify their md5 hash.
If a package version was requested, you can use the 'utils/download_from_pypi' script, e.g:
./download_from_pypi scipy==1.2.3
This will download all release files for the specified version of the package and verify their sha256 hash. Multiple packages can also be specified:
./download_from_pypi scipy==1.2.3 Pillow==6.1.0
This might result in more files than you need to upload (e.g, maybe you don't care about Python 2.7 wheels). You can use shell expansion to delete any packages you don't need:
rm *cp27* *.egg *.exe
The downloaded files can now be fed into the upload script.
Uploading to PyPi
The upload script expects that your machine/vm will have the same username as your Mozilla LDAP. If you aren't sure how to check this, open a new terminal and type:
echo "Machine username is: $(whoami)"
If the machine username is not the same as your Mozilla LDAP, we recommend you to either create a new username with your Mozilla LDAP username or change $(whoami) inside the script to be equal to your LDAP username.
If your repository is up-to-date and your machine user is the same with your LDAP, all you have to do now is run the following command:
./publish_package_our_pypi.sh <path/to/file1> <path/to/file2> ...
The script will practically copy the file to relengwebadmin and move it where the files are located for pypi internal (/mnt/netapp/relengwebadmin/pypi/pub) then will change the rights and upload the file to http://pypi.pub.build.mozilla.org/pub.
If everything worked fine, you should now have a link that looks like this: http://pypi.pub.build.mozilla.org/pub/FILE-NAME-HERE