[Python-Dev] Single-file Python executables (was: Computed Goto dispatch for Python 2)

[Barry Warsaw]

On May 28, 2015, at 11:39 AM, Donald Stufft wrote:

>You don’t need a "fully functioning Python" for a single file binary, you
>only need enough to actually run your application. For example, if you're
>making an application that can download files over HTTP, you don't need to
>include parts of the stdlib like xmlrpc, pickle, shelve, marshall, sqlite,
>csv, email, mailcap, mailbox, imaplib, nntplib, etc.

There are actually two related but different use cases to "single file
executables".

The first is nicely solved by tools like pex, where you don't need to include
a fully functional Python at the head of the zip file because the environment
you're deploying it into will have enough Python to make the zip work.  This
can certainly result in smaller zip files.  This is the approach I took with
Snappy Ubuntu Core support for Python 3, based on the current situation that
the atomic upgrade client is written in Python 3.  If that changes and Python
3 is removed from the image, then this approach won't work.

pex (and others) does a great job at this, so unless there are things better
refactored into upstream Python, I don't think we need to do much here.

The second use case is as you describe: put a complete functional Python
environment at the head of the zip file so you don't need anything in the
target deployment environment.  "Complete" can easily mean the entire stdlib,
and although that would usually be more bloated than you normally need, hey,
it's just some extra unused bits so who cares? <wink>.  I think this would be
an excellent starting point which can be optimized to trim unnecessary bits
later, maybe by third party tools.

>Of course deciding which pieces you include in the zip file you're appending
>to the end of Python is up to whatever tool builds this executable which
>doesn't need to be part of Python itself. If Python itself gained the ability
>to operate in that manner than third party tools could handle trying to do
>the optimizations where it only includes the things it actually needs in the
>stdlib and excludes things it doesn't. The key thing here is that since
>you're doing a single file binary, you don't need to have a Python which is
>suitable to execute random Python code, you only need one that is suitable to
>execute this particular code so you can specialize what that includes.

I'd love to see Python itself gain such a tool, but if it had the critical
pieces to execute in this way, that would enable a common approach to
supporting this in third party tools, on a variety of platforms.

I do think single-file executables are an important piece to Python's
long-term competitiveness.

Cheers,
-Barry