On Thu, Oct 27, 2016 at 8:50 AM, James Pic <jamespic@gmail.com> wrote:

Hi all !

Ive heard some people saying it was rude to post on a mailing list without introducing yourself so here goes something: my name is James Pic and I've been developing and deploying a wide variety fof Python projects Python for the last 8 years, I love to learn and share and writing documentation amongst other things such as selling liquor.

The way I've been deploying Python projects so far is probably similar to what a lot of people do and it almost always includes building runtime dependencies on the production server. So, nobody is going to congratulate me for that for sure but I think a lot of us have been doing so.

Now I'm fully aware of distribution specific packaging solutions like dh-virtualenv shared by Spotify but here's my mental problem: I love to learn and to hack. I'm always trying now distributions and I rarely run the one that's in production in my company and when I'm deploying personal projects I like funny distributions like arch, Alpine Linux,  or interesting paas solutions such as cloudfoundry, openshift, rancher and many others.

And I'm always facing the same problem: I have to either build runtime dependencies on the server, either package my thing in the platform specific way. I feel like I've spent a really huge amount of time doing this king of thing. But the java people, they have jars, and they have smooth deployments no matter where they deploy it.

So that's the idea I'm trying to share: I'd like to b able to build a file with my dependencies and my project in it. I'm not sure packaging only Python bytecode would work here because of c modules. Also, I'm always developing against a different Python version because I'm using different distributions because it's part of my passions in life, as ridiculous as it could sound to most people, I'm expecting at least some understanding from this list :)

So I wonder, do you think the best solution for me would be to build an elf binary with my Python and dependencies that I could just run on any distribution given its on the right architecture ? Note that I like to use Arm too, so I know I'd need to be able to cross compile too.

Thanks a lot for reading and if you can to take some time to share your thoughts and even better : point me in a direction, if that idea is the right solution and I'm going to be the only one interested I don't care if it's going to take years for me to achieve this.

Thanks a heap !

Beat regards

PS: I'm currently at the openstack summit in Barcelona if anybody there would like to talk about it in person, in which case I'll buy you the drinks ;)


Are you sure this is really what you need to do?  With dependency handling, you can define the dependencies of your project and they will automatically get installed from pypi when the user tries to install the package (if they aren't already installed).  manylinux wheels [1] allow you to distribute your own code in a manner that is compatible with most linux distributions, and many c-based projects now offer such wheels. 

Assuming your dependencies have version agnostic wheels (either manylinux or pure python), what would be the advantage to you of putting everything together in a single file?

That being said, I suppose it would be possible to create your own manylinux wheels that include all the necessary dependencies, but that would make building more difficult and opens up the possibility that the installed modules will conflict with users' existing installed packages.

Another possibility would be to use docker to create a container [2] that includes everything you need to run the code in an isolated environment that won't conflict

[1] https://github.com/pypa/manylinux
[2] https://www.digitalocean.com/community/tutorials/docker-explained-how-to-containerize-python-web-applications