[Python-ideas] npm-style venv-aware launcher

[David Townshend]

On Sun, May 31, 2015 at 9:00 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On May 31, 2015, at 09:19, David Townshend <aquavitae69 at gmail.com> wrote:
>
>
>> The default for npm is that your package dir is attached directly to the
>> project. You can get more flexibility by setting an environment variable or
>> creating a symlink, but normally you don't. It has about the same
>> flexibility as virtualenvwrapper, with about the same amount of effort. So
>> if virtualenvwrapper isn't flexible enough for you, my guess is that your
>> take on npm won't be flexible enough either, it'll just come preconfigured
>> for your own idiosyncratic use and everyone else will have to adjust...
>>
>
> You have a point.  Maybe lack of flexibility is not actually the issue -
> it's too much flexibility.
>
>
> I think Python needs that kind of flexibility, because it's used in a much
> wider range of use cases, from binary end-user applications to OS
> components to "just run this script against your system environment" to
> conda packages, not just web apps managed by a deployment team and other
> things that fall into the same model. And it needs to be backward
> compatible with the different ways people have come up with for handling
> all those models.
>
> While it's possible to rebuild all of those models around the npm model,
> and the node community is gradually coming up with ways of doing so
> (although notice that much of the node community is instead relying on
> docker or VMs...), you'd have to be able to transparently replace all of
> the current Python use cases today if you wanted to change Python today.
>
> Also, as Nick pointed out, making things easier for the developer comes at
> the cost of making things harder for the user--which is acceptable when the
> user is the developer himself or a deployment team that sits at the next
> set of cubicles, but may not be acceptable when the user is someone who
> just wants to run a script he found online. Again, the Node community is
> coming to terms with this, but they haven't got to the same level as the
> Python community, and, even if they had, it still wouldn't work as a
> drop-in replacement without a lot of work.
>
> What someone _could_ do is make it easier to set up a dev-friendly
> environment based on virtualenvwrapper and virtualenvwrapperhelper.
> Currently, you have to know what you're looking for and find a blog page
> somewhere that tells you how to install and configure all the tools and
> follow three or four steps. That's obvious less than ideal. It would be
> nice if there were a single "pip install envstuff" that got you ready out
> of the box (including working for Windows cmd and PowerShell), and if links
> to that were included in the basic Python docs. It would also be nice if
> there were a way to transfer your own custom setup to a new machine. But I
> don't see why that can't all be built as improvements on the existing tools
> (and a new package that just included requirements and configuration and no
> new tools).
>
> The problem that I have with virtualenv is that it requires quite a bit of
> configuration and a great deal of awareness by the user of what is going on
> and how things are configured. As stated on it's home page While there is
> nothing specifically wrong with this, I usually just want a way to do
> something in a venv without thinking too much about where it is or when or
> how to activate it.
>
>
> But again, if that's what you want, that's what you have with
> virtualenvwrapper or autoenv. You just cd into the directory (whether a new
> one you just created with the wrapper or an old one you just pulled from
> git) and it's set up for you. And setting up a new environment or cloning
> an existing one is just a single command, too. Sure, you can make your
> configuration more complicated than that, but if you don't want to, you
> don't have to.
>
> If you've had a look at the details of the sort of tool I'm proposing, it
> is completely transparent.  Perhaps the preconfiguration is just to my own
> idiosyncrasies, but if it serves its use 90% of the time then maybe that is
> good enough.
>
>
> Some of what I'm proposing could be incorporated in to pip (i.e. better
> requirements) and some could possibly be incorporated into
> virtualenvwrapper (although I still think that my proposal for handling
> venvs is just too different from that of virtualenvwrapper to be worth
> pursuing that course), but one of the main aims is to merge it all into one
> tool that manages both the venv and the requirements.
>
>
> There are major advantages in not splitting the Python community between
> two different sets of tools. We've only recently gotten past easy_install
> vs. pip and distribute vs. setuptools, which has finally enabled a clean
> story for everyone who wants to distribute packages to get it right, which
> has finally started to happen (although there are people still finding and
> following blog posts that tell them to install distribute or not to use
> virtualenv because it doesn't play nice with py2app or whatever).
>
> I'm quite sure that this proposal is not going to accepted without a trial
> period on pypi, so maybe that will be the test of whether this is useful.
>
> Is this the right place for this, or would distutils-sig be better?
>
>
> Other people have made the case for both sides of that earlier in the
> thread and I'm not sure which one is more compelling...
>
> Also, the pure pip enhancement of coming up with something better than
> freeze/-r may belong on distutils-sig while the environment-aware launcher
> and/or environment-managing tools may belong here. (Notice that Python
> includes venv and the py launcher, but doesn't include setuptools or pip...)
>

Just to be clear, I'm not suggesting changing the python executable itself,
or any of the other tools already in existence.  My proposal is a separate
wrapper around existing python, pip and venv which would not change
anything about the way it works currently.  A dev environment set up using
it could still be deployed in the same way it would be now, and there would
still be the option of using virtualenvwrapper, or something else for those
that want to.  It is obviously way too early to try to get it included in
the next python release (apart form anything else, pip would need to be
added first), so really this proposal is meant more to gauge interest in
the concept so that if it is popular I can carry on developing it and
preparing it for inclusion in the stdlib, or at least a serious discussion
about including it, once it is mature.

That said, Andrew's arguments have convinced me that much could be done to
improve existing tools before creating a new one, although I still don't
believe virtualenvwrapper can be squashed into the shape I'm aiming for
without fundamental changes.  Also, from the other responses so far it
seems that the general feeling is that handling of requirements could
definitely be improved, but that anything too prescriptive with venvs would
be problematic.  Unfortunately for my proposal, if something like what I'm
suggesting were officially supported via inclusion in the stdlib it would
quickly become, at best, the "strongly recommended" way of working and at
worst the One Obvious Way.  With all this in mind, I'll withdraw my
proposal, but continue development on my version and see if it goes
anywhere.  I'll also see how much of it's functionality I can put into
other tools (specifically pip's requirements handling) instead.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150531/82262f5b/attachment.html>