[Distutils] Idea: allow PyPI projects to link to DockerHub container images

[Nick Coghlan]

One of the recurring problems folks mention here is how to deal with
the complexities of handling Linux ABI compatibility issues.

That's a genuinely hard problem, and not one that *anyone* has solved
well - it's one of the reasons being an independent software vendor
for Linux in general (rather than just certifying with the major
commercial distros) is a pain. When folks do it, they tend to take the
"bundle everything you need and drop it somewhere in /opt" approach
which (quite rightly) makes professional system administrators very
unhappy.

On the distro side, this is one of the big factors driving the
popularity of the "bundle all the things" container image model: it
does the bundling in such a way that it's amenable to programmatic
introspection, and it still reduces the runtime ABI compatibility
question to just the kernel ABI. This tends to work really when in the
case of dynamic languages like Python, as the language runtime is
likely to deal with most kernel compatibility issues for you. (ABI
incompatibilities can still bite you if you're using system libraries
inside the container and your base image doesn't match your runtime
kernel, but the bug surface is still much smaller than when you use
the end user's system libraries directly)

It seems to me that, at least for web services published via PyPI
(like Kallithea), "use our recommended container", is likely to be the
easiest way to get folks on Linux up and running quickly with the
service. Folks may still want to take the image apart later and roll
their own (e.g. to switch to running on a different web server or a
different base image), but they wouldn't have to do their own
integration work just to get started.

The other advantage of nudging folks in the direction of Linux
containers to address their ABI compatibility woes is that this is
tech that already (mostly) works, and has a broader management
ecosystem growing around it (including both the major open source
platform-as-a-service offerings in OpenShift and Cloud Foundry).
Inventing our own way of abstracting away the Linux ABI compatibility
problem would be an awful lot of work, and likely leave us with an end
result that isn't pre-integrated with anything else.

Regards,
Nick.

P.S. Full disclosure: for Fedora's developer experience work for web
service developers, we're heading heavily in the direction of
containers+Vagrant for local dev workstations, to allow common dev
workflows across Linux, Mac OS X and Windows, and then pushing the
containers through Linux based CI and independent QE workflows, into
container based production Linux environments, including the Google &
Red Hat backed Kubernetes container orchestration framework and
OpenStack's Project Solum. In my day job, this is also the direction
we're taking Red Hat's internal infrastructure since it systematically
solves a variety of problems for us (like how to most effectively
allow folks to develop on Fedora while deploying on RHEL).

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia