[Python-Dev] Single-file Python executables (was: Computed Goto dispatch for Python 2)

[Donald Stufft]

On May 29, 2015 at 4:37:37 AM, Steven D'Aprano (steve at pearwood.info) wrote:
> On Thu, May 28, 2015 at 01:20:06PM -0400, Donald Stufft wrote:
>  
> > I think it’s an issue for all platforms, even when there is a system Python
> > that can be used.
> >
> > Here’s why:
> >
> > * Even on Linux systems Python isn’t always a guaranteed thing to be installed,
> > for instance Debian works just fine without any Python installed.
>  
> Donald, are you a Linux user? If so, which distro? Because in the Linux
> world that I'm familiar with, this (and the points you make below) are
> absolutely not an issue. You let the package manager worry about
> dependencies:
>  
> yum install myapp # Red Hat based distros
> apt-get install myapp # Debian based distros
>  
> will ensure that the right version of Python is installed.
>  
> In the circles I move in, installing anything which does not go through
> the package manager is considered to be quite dubious. In order of
> preference (best to worst):
>  
> - install from official distro repositories;
> - install from a third-party repo;
> - install from source;
> - find an alternative application;
> - do without;
> - install from some binary format (also known as "would you like
> a root kit with that?").
>  
>  
> [...]
> > * Even if you have Python installed already, is it the right one? What if it’s
> > an ancient RHEL box that has 2.6 or (heaven forbid) 2.4? What if it’s a not
> > ancient box that has Python 2.7 but you want to deploy your app in Python 3?
>  
> Most recent distros have both a python2 and python3 package, and when
> building your rpm or deb file, you specify which is your dependency in
> the normal fashion.
>  
>  
> > * What if you have Python installed already, but it’s been patched by the place
> > you got it from and now the behavior is different than what you expected?
>  
> Normally you would write for the version of Python provided by the
> distros you wish to support. In practice that might mean writing hybrid
> code targetting (say) 2.6 and 2.7, which covers most recent Red Hat and
> Debian based systems, and anything else, you provide the source code and
> let the user work it out.
>  
> I suppose that in principle you could include whatever version of Python
> you like *in your application*, but that would be considered an unusual
> thing to do. I believe that LibreOffice and OpenOffice do that, so they
> can support Python as a scripting language, but they're generally
> considered (1) a special case because they're cross-platform, and (2)
> not "proper" Unix or Linux apps anyway.
>  
> The point is, in the Linux circles I move in, this idea of single file
> installation would be about as popular as a police raid at a rave club.
> Maybe you move in different circles (perhaps more enterprisey?), but I
> can already imagine the sort of derogatory comments the sys admins I
> work with would make about this idea.
>  

I use Linux for servers yes, I don't stick with a single Distribution and right
now I manage services that are running on CentOS, Ubuntu, Debian, Alpine, and
FreeBSD (not a Linux, but w/e). 

Here's the thing though, when you make software that is designed for other
people to consume you have two choices. Either you have to try to anticipate
every single environment that they might possibly run it in and what is
available there so that your software either runs there or you can provide
instructions on how to run it there, or you need to depend on as little from
the OS as possible.

An example of a product that does this is Chef, they install their own Ruby
and everything but libc into /opt/chef to completely isolate themselves from
the host system. I’m told this made things *much* easier for them as they
don't really have to worry at all about what's available on the host system,
Chef pretty much just works.

Another example is one that I personally worked on recently, where the company
I worked for wanted to distribute a CLI to our customers which would
"just work" that they could use to interact with the service we provided. We
were writing this client in Python and it was very painful to satisfy the
requirement that it work without Python installed as a single file executable.
That was a hard requirement as anything else would be more difficult to
distribute to end users and be more likely to break. The entire time I was
working on that particular piece I was feeling like I should be suggesting to
my manager that we throw away the Python code and just write it in Go.

As folks may or may not know, I'm heavily involved in pip which is probably one
of the most widely used CLIs written in Python. A single file executable won't
help pip, however through my experience there I can tell you that a significant
portion of our issues come from random weird differences in how a particular
person's Python environment is setup. If someone is distributing a CLI app
written in Python, being able to remain independent from the OS eliminates a
whole class of problems.

---  
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA