On 27 October 2016 at 22:50, James Pic <jamespic@gmail.com> wrote:
Hi all !
Ive heard some people saying it was rude to post on a mailing list without introducing yourself so here goes something: my name is James Pic and I've been developing and deploying a wide variety fof Python projects Python for the last 8 years, I love to learn and share and writing documentation amongst other things such as selling liquor.
The way I've been deploying Python projects so far is probably similar to what a lot of people do and it almost always includes building runtime dependencies on the production server. So, nobody is going to congratulate me for that for sure but I think a lot of us have been doing so.
You're right that this is a common problem, but it also isn't a language level problem - it's a software publication and distribution one, and for the Python community, the folks most actively involved in driving and/or popularising improvements in that space are those running packaging.python.org. While there's a fair bit of overlap between the two lists, the main home for those discussions is over on distutils-sig: https://mail.python.org/mailman/listinfo/distutils-sig (so called due to the standard library module that provided Python's original project-agnostic interface for building extension modules)
Now I'm fully aware of distribution specific packaging solutions like dh-virtualenv shared by Spotify but here's my mental problem: I love to learn and to hack. I'm always trying now distributions and I rarely run the one that's in production in my company and when I'm deploying personal projects I like funny distributions like arch, Alpine Linux, or interesting paas solutions such as cloudfoundry, openshift, rancher and many others.
And I'm always facing the same problem: I have to either build runtime dependencies on the server, either package my thing in the platform specific way. I feel like I've spent a really huge amount of time doing this king of thing. But the java people, they have jars, and they have smooth deployments no matter where they deploy it.
If you're not using C extensions (the closest Python equivalent to the typical jar use case), then ``zipapp`` should have you covered: https://docs.python.org/3/library/zipapp.html While the zipapp module itself is relatively new, the underlying interpreter and import system capabilities that it relies on have been around since Python 2.6.
So that's the idea I'm trying to share: I'd like to b able to build a file with my dependencies and my project in it. I'm not sure packaging only Python bytecode would work here because of c modules.
For extension modules, you're facing a much harder problem than doing the same for pure Python code (where you can just use zipapp). However, engineering folks at Twitter put together pex, which gives you a full virtual environment to play with on the target system, and hence can handle extension modules as well: https://pex.readthedocs.io/en/stable/ The only runtime dependency pex places on the target system is having a Python runtime available. More generally, one of the major problems we have in this area at the moment is that a lot of the relevant information is just plain hard for people to find, so if this is an area you're interested in, then https://github.com/pypa/python-packaging-user-guide/issues/267 is aiming to pull together some of the currently available information into a more readily consumable form and is mainly waiting on a draft PR that attempts to make the existing content more discoverable. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia