[Distutils] distlib - installer support vs runtime support

[PJ Eby]

On Mon, Mar 11, 2013 at 11:36 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
> On Sat, Mar 9, 2013 at 9:33 PM, PJ Eby <pje at telecommunity.com> wrote:
>> On Sat, Mar 9, 2013 at 8:14 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Longer term, something like the import engine PEP may let us implement a
>>> cleaner solution.
>>
>> I've been giving the bootstrapping issue a bit more thought, though,
>> and I think there's a way to take the pain out of multi-file
>> bootstraps of things like pip and setuptools and whatnot.
>>
>> Suppose you make a .py file that contains (much like ez_setup.py) a
>> list of files to download, along with their checksums.  And suppose
>> this .py file basically just has some code that checks a local cache
>> directory for those files, adds them to sys.path if they're found,
>> downloads them if they're not (validating the checksum), and then
>> proceeds to import a specific entry point and run it?
>
> I've been thinking about something like this lately too. A simple
> script like your proposed pystart could be used to generate these
> files.  I've gone one further and considered a format that includes
> dependencies (or at the very least install-time dependencies right in
> the .py file itself as a base64 string.  This would work fine at least
> for small dependencies--no downloads necessary and you're guaranteed
> to get the right version that actually installs the damn package.

You could, but it makes the initial download bigger and adds more
moving parts.  If you have to download some of them, might as well do
all of them.  (Besides, the approach I outlined allows for sharing
distributions and avoiding repeated downloads.)


> For packages using setuptools it's also possible to just include a
> dependency as an egg or tar.gz right in your source dist and add
> something like "[easy_install]\nfind_links = ." to your setup.cfg and
> it'll go.  But I'm not sure that's exactly the use case you're after
> here.

No.  A particular goal behind this idea is that a pystart script
should be cross-platform and ideally cross-Python-version as well,
using environment markers to determine what should be downloaded.  So,
 syntactically, the script would be written so that it runs on as many
Python versions as possible, which shouldn't be too hard since it will
basically be just looking for a series of files, downloading and
hashing them.  You'd have one function that does a "check for this
file, download and extract if needed (w/hash validation), add to
sys.path", and a bunch of if/then blocks calling that function with
different name/url/hash triplets according to the current
platform/python version markers.  Then the bottom of the script would
import the script entry point and run it.  (And the top would have a
preamble to import stuff from the right places based on Python
version.)

Alternatively, for a simpler code generation model, it could just
stick everything in a giant data structure at the top, and append
standard code that checks the environment markers.  But I'd rather not
have to include full environment marker interpretation, so translating
the markers to Python seems advisable, even if it's as lambdas in the
data structure.