[Python-ideas] Distribution agnostic Python project packaging

[Chris Angelico]

On Thu, Oct 27, 2016 at 11:50 PM, James Pic <jamespic at gmail.com> wrote:
> So that's the idea I'm trying to share: I'd like to b able to build a file
> with my dependencies and my project in it. I'm not sure packaging only
> Python bytecode would work here because of c modules. Also, I'm always
> developing against a different Python version because I'm using different
> distributions because it's part of my passions in life, as ridiculous as it
> could sound to most people, I'm expecting at least some understanding from
> this list :)
>
> So I wonder, do you think the best solution for me would be to build an elf
> binary with my Python and dependencies that I could just run on any
> distribution given its on the right architecture ? Note that I like to use
> Arm too, so I know I'd need to be able to cross compile too.

In theory, you could do that. You'd have to include *all* of Python,
and all of everything else you might depend on, because you can't be
sure what is and isn't available, so you might as well ship your app
as a VM image or something, with an entire operating system.

In practice, you're probably going to need to deal with some sort of
package manager, and that's where the difficulties start. You can
probably cover most of the Debian-based Linuxes by targeting either
Debian Stable or Ubuntu LTS and creating a .deb file that specifies
what versions of various libraries you need. There's probably a way to
aim an RPM build that will work on RHEL, Fedora, SUSE, etc, but I'm
not familiar with that family tree and where their library versions
tend to sit. The trouble is that as soon as you land on an OS version
that's too far distant from the one you built on, stuff will break.
Between the bleeding-edge rolling distros and the super-stable ones
could be over a decade of development (RHEL 4, released in 2005, is
still supported).

What you can probably do is ignore the absolute bleeding edge (anyone
who's working on Debian Unstable or Arch or Crunchbang is probably
aware of the issues and can solve them), and then decide how far back
you support by looking at what you depend on, probably losing the very
oldest of distributions. It should be possible to hit 95% of Linuxes
out there by providing one .deb and one .rpm (per architecture, if you
support multiple), but don't quote me on that figure.

Unfortunately, the problem you're facing is virtually unsolvable,
simply because the freedom of open systems means there is a LOT of
variation out there. But most people on the outskirts are accustomed
to doing their own dependency management (like when I used to work
primarily on OS/2 - nobody supports it much, so you support it
yourself).

With all sincerity I say to you, good luck. Try not to lose the
enthusiasm that I'm hearing from you at the moment!

ChrisA