[Python-ideas] npm-style venv-aware launcher

[Donald Stufft]

On May 31, 2015 at 11:05:24 AM, Nick Coghlan (ncoghlan at gmail.com) wrote:
> On 31 May 2015 at 23:10, Stephen J. Turnbull wrote:
> > Steven D'Aprano writes:
> >
> > > I don't think this is the right place to discuss either of those ideas.
> >
> > I think you're missing the point -- this is part of the larger
> > discussion on packaging, as Alexander recognized ("shoot this over to
> > distutils-sig", he said). While technically it may belong elsewhere
> > (distutils, for example), the amount of attention it's attracting from
> > core committers right now suggests that it's a real pain point, and
> > should get discussion from the wider community while requirements are
> > still unclear.
> >
> > While I'm not one for suggesting that TOOWTDI is obvious in advance
> > (and not even if you're Dutch), surely it's worth narrowing down the
> > field by looking at a lot of ideas.
>  
> There are a plethora of environment management options out there, and
> https://github.com/pypa/python-packaging-user-guide/issues/118
> discusses some of them (focusing specifically on the ad hoc
> environment management side of things rather than VCS linked
> environment management, though).
>  
> The npm model in particular unfortunately gets a lot of its
> "simplicity" by isolating all the dependencies from each other during
> component development (including freely permitting duplicates and even
> different versions of the same component), so you get the excitement
> of live integration at runtime instead of rationalising your
> dependency set as part of your design and development process (see
> https://speakerdeck.com/nzpug/francois-marier-external-dependencies-in-web-apps-system-libs-are-not-that-scary?slide=9  
> ). As developers, we can make our lives *very* easy if we're happy to
> discount the interests of other folks that are actually tasked with
> deploying and maintaining our code (either an operations team if we
> have one, or at the very least future maintainers if we don't).
>  
> So while there are still useful user experience lessons to be learned
> from npm, they require careful filtering to ensure they actually *are*
> a simplification of the overall user experience, rather than cases
> where the designers of the system have made things easier for
> developers working on the project itself at the expense of making them
> harder for operators and end users that just want to install it
> (potentially as part of a larger integrated system).
>  
> Cheers,
> Nick.
>  
> P.S. I've unfortunately never found the time to write up my own
> packaging system research properly, but
> https://bitbucket.org/ncoghlan/misc/src/default/talks/2013-07-pyconau/packaging/brispy-talk.md  
> has some rough notes from a couple of years ago, while
> https://fedoraproject.org/wiki/Env_and_Stacks/Projects/UserLevelPackageManagement  
> looks at the general problem space from an operating system developer
> experience design perspective.
>  
> --
> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>  


One of the things that make NPM a lot simpler is that their “virtualenv”
is implicit and the default, and you have to go out of your way to get
a “global” install. It would be possible to add this to Python by doing
something like ``sys.path.append(“./.python-modules/“)`` (but it also
needs to recurse upwards) to the Python startup (and possibly some file
you can put in that folder so that it doesn’t add the typical site-packages
or user-packages to the sys.path.

This makes it easier to have isolation being the default, however it
comes with it’s own problems. It becomes a lot harder to determine
what’s going to happen when you type ``python`` since you have to inspect
the entire directory hierarchy above you looking for a .python_modules
file. There’s also the problem that binary scripts tend to get installed
into something like .python-modules/bin/ or so in that layout, but that’s
rarely what people want. The npm community “solved” this by having the
actual CLI command be installable on it’s own that will call into the
main program that you have installed per project.

---  
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA