[Python-ideas] python_modules as default directory for dependencies in distutils

[Jim Jewett]

On 11/20/12, Daniel Holth <dholth at gmail.com> wrote:
> On Tue, Nov 20, 2012 at 1:21 PM, Philipp Hagemeister
> <phihag at phihag.de>wrote:

>> Currently, there are the following methods for installing dependencies:
...
>> Therefore, I'd like to suggest a new alternative location (--here =
>> --root "./python_modules", intended to become default in Python 5),
>> modeled after node's packaging system (http://goo.gl/dMRTC).

If I'm understanding correctly,  you just mean "install dependencies
in the same place as the application that asked for them", or maybe in
a magically named subdirectory.  That does sound like a reasonable
policy -- similar to the windows or java solution of packing
everything into a single bundle.

>> Aside from the new flag in distutils, the site module should
>> automatically look into ./python_modules , as if it were a second
>> USER_SITE.

As opposed to just putting them a layer up, and looking into the
application package's own directory for relative imports?

> You wouldn't need stdlib support to do this. I believe setuptools'
> pkg_resources can look in a directory full of eggs, adding the required
> ones to PYTHONPATH based on requirements as specified in a wrapper script.
> Gem uses a directory full of versioned packages like
> ~/.gem/ruby/1.8/gems/sinatra-1.3.3/.

If I understand correctly, that just provides a way to include the
version number when choosing the system-wide package location (and
later, when importing).  Also useful, but different from bundling the
dependencies inside each application that requires them.

Most notably, the bundle-inside solution will* find exactly the module
it shipped with, including custom patches.  The versioned-packages
solution will have conflicts when more than one application provides
for the same dependency, but will better support independent
maintenance (or at least security patches) for the 4th-party modules.

* Err, unless the module was loaded before the application, or
modified locally, or something odd happened with import, or ...

-jJ