[Python-ideas] npm-style venv-aware launcher

[Andrew Barnert]

On May 31, 2015, at 09:19, David Townshend <aquavitae69 at gmail.com> wrote:
>> 
>> The default for npm is that your package dir is attached directly to the project. You can get more flexibility by setting an environment variable or creating a symlink, but normally you don't. It has about the same flexibility as virtualenvwrapper, with about the same amount of effort. So if virtualenvwrapper isn't flexible enough for you, my guess is that your take on npm won't be flexible enough either, it'll just come preconfigured for your own idiosyncratic use and everyone else will have to adjust...
> 
> You have a point.  Maybe lack of flexibility is not actually the issue - it's too much flexibility.

I think Python needs that kind of flexibility, because it's used in a much wider range of use cases, from binary end-user applications to OS components to "just run this script against your system environment" to conda packages, not just web apps managed by a deployment team and other things that fall into the same model. And it needs to be backward compatible with the different ways people have come up with for handling all those models.

While it's possible to rebuild all of those models around the npm model, and the node community is gradually coming up with ways of doing so (although notice that much of the node community is instead relying on docker or VMs...), you'd have to be able to transparently replace all of the current Python use cases today if you wanted to change Python today.

Also, as Nick pointed out, making things easier for the developer comes at the cost of making things harder for the user--which is acceptable when the user is the developer himself or a deployment team that sits at the next set of cubicles, but may not be acceptable when the user is someone who just wants to run a script he found online. Again, the Node community is coming to terms with this, but they haven't got to the same level as the Python community, and, even if they had, it still wouldn't work as a drop-in replacement without a lot of work.

What someone _could_ do is make it easier to set up a dev-friendly environment based on virtualenvwrapper and virtualenvwrapperhelper. Currently, you have to know what you're looking for and find a blog page somewhere that tells you how to install and configure all the tools and follow three or four steps. That's obvious less than ideal. It would be nice if there were a single "pip install envstuff" that got you ready out of the box (including working for Windows cmd and PowerShell), and if links to that were included in the basic Python docs. It would also be nice if there were a way to transfer your own custom setup to a new machine. But I don't see why that can't all be built as improvements on the existing tools (and a new package that just included requirements and configuration and no new tools).

> The problem that I have with virtualenv is that it requires quite a bit of configuration and a great deal of awareness by the user of what is going on and how things are configured. As stated on it's home page While there is nothing specifically wrong with this, I usually just want a way to do something in a venv without thinking too much about where it is or when or how to activate it.

But again, if that's what you want, that's what you have with virtualenvwrapper or autoenv. You just cd into the directory (whether a new one you just created with the wrapper or an old one you just pulled from git) and it's set up for you. And setting up a new environment or cloning an existing one is just a single command, too. Sure, you can make your configuration more complicated than that, but if you don't want to, you don't have to.

> If you've had a look at the details of the sort of tool I'm proposing, it is completely transparent.  Perhaps the preconfiguration is just to my own idiosyncrasies, but if it serves its use 90% of the time then maybe that is good enough.
> 
> Some of what I'm proposing could be incorporated in to pip (i.e. better requirements) and some could possibly be incorporated into virtualenvwrapper (although I still think that my proposal for handling venvs is just too different from that of virtualenvwrapper to be worth pursuing that course), but one of the main aims is to merge it all into one tool that manages both the venv and the requirements.

There are major advantages in not splitting the Python community between two different sets of tools. We've only recently gotten past easy_install vs. pip and distribute vs. setuptools, which has finally enabled a clean story for everyone who wants to distribute packages to get it right, which has finally started to happen (although there are people still finding and following blog posts that tell them to install distribute or not to use virtualenv because it doesn't play nice with py2app or whatever).

> I'm quite sure that this proposal is not going to accepted without a trial period on pypi, so maybe that will be the test of whether this is useful.
> 
> Is this the right place for this, or would distutils-sig be better?

Other people have made the case for both sides of that earlier in the thread and I'm not sure which one is more compelling...

Also, the pure pip enhancement of coming up with something better than freeze/-r may belong on distutils-sig while the environment-aware launcher and/or environment-managing tools may belong here. (Notice that Python includes venv and the py launcher, but doesn't include setuptools or pip...)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150531/b4acf0e4/attachment-0001.html>