Re: [Python-Dev] Python install layout and the PATH on win32 (Rationale part 1: Regularizing the layout)
On Mar 23, 2012 3:53 PM, "Carl Meyer"
Hi PJ,
On 03/23/2012 12:35 PM, PJ Eby wrote:
AFAICT, virtualenvs are overkill for most development anyway. If you're not using distutils except to install dependencies, then configure distutils to install scripts and libraries to the same directory, and then do all your development in that directory. Presto! You now have a cross-platform "virtualenv". Want the scripts on your path? Add that directory to your path... or if on Windows, don't bother, since the current directory is usually on the path. (In fact, if you're only using easy_install to install your dependencies, you don't even need to edit the distutils configuration, just use "-md targetdir".)
Creating and using a virtualenv is, in practice, _easier_ than any of those alternatives,
Really? As I said, I've never seen the need to try, since just installing stuff to a directory on PYTHONPATH seems quite easy enough for me.
that the "isolation from system site-packages" feature is quite popular (the outpouring of gratitude when virtualenv went isolated-by-default a few months ago was astonishing), and AFAIK none of your alternative proposals support that at all.
What is this isolation for, exactly? If you don't want site-packages on your path, why not use python -S? (Sure, nobody knows about these things, but surely that's a documentation problem, not a tooling problem.) Don't get me wrong, I don't have any deep objection to virtualenvs, I've just never seen the *point* (outside of the scenarios I mentioned), and thus don't see what great advantage will be had by rearranging layouts to make them shareable across platforms, when "throw stuff in a directory" seems perfectly serviceable for that use case already. Tools that *don't* support "just throw it in a directory" as a deployment option are IMO unpythonic -- practicality beats purity, after all. ;-)
On 03/23/2012 09:22 PM, PJ Eby wrote:
On Mar 23, 2012 3:53 PM, "Carl Meyer"
On 03/23/2012 12:35 PM, PJ Eby wrote:
AFAICT, virtualenvs are overkill for most development anyway. If you're not using distutils except to install dependencies, then configure distutils to install scripts and libraries to the same directory, and then do all your development in that directory. Presto! You now have a cross-platform "virtualenv". Want the scripts on your path? Add that directory to your path... or if on Windows, don't bother, since the current directory is usually on the path. (In fact, if you're only using easy_install to install your dependencies, you don't even need to edit the distutils configuration, just use "-md targetdir".)
Creating and using a virtualenv is, in practice, _easier_ than any of those alternatives,
Really? As I said, I've never seen the need to try, since just installing stuff to a directory on PYTHONPATH seems quite easy enough for me.
that the "isolation from system site-packages" feature is quite popular (the outpouring of gratitude when virtualenv went isolated-by-default a few months ago was astonishing), and AFAIK none of your alternative proposals support that at all.
What is this isolation for, exactly? If you don't want site-packages on your path, why not use python -S?
(Sure, nobody knows about these things, but surely that's a documentation problem, not a tooling problem.)
Don't get me wrong, I don't have any deep objection to virtualenvs, I've just never seen the *point* (outside of the scenarios I mentioned),
No problem. I was just responding to the assertion that people only use virtualenvs because they aren't aware of the alternatives, which I don't believe is true. It's likely many people aren't aware of python -S, or of everything that's possible via distutils.cfg. But even if they were, for the cases where I commonly see virtualenv in use, it solves the same problems more easily and with much less fiddling with config files and environment variables. Case in point: libraries that also install scripts for use in development or build processes. If you're DIY, you have to figure out where to put these, too, and make sure it's on your PATH. And if you want isolation, not only do you have to remember to run python -S every time, you also have to edit every script wrapper to put -S in the shebang line. With virtualenv+easy_install/pip, all of these things Just Simply Work, and (mostly) in an intuitive way. That's why people use it.
thus don't see what great advantage will be had by rearranging layouts to make them shareable across platforms, when "throw stuff in a directory" seems perfectly serviceable for that use case already. Tools that *don't* support "just throw it in a directory" as a deployment option are IMO unpythonic -- practicality beats purity, after all. ;-)
No disagreement here. I think virtualenv's sweet spot is as a convenient tool for development environments (used in virtualenvwrapper fashion, where the file structure of the virtualenv itself is hidden away and you never see it at all). I think it's fine to deploy _into_ a virtualenv, if you find that convenient too (though I think there are real advantages to deploying just a big ball of code with no need for installers). But I see little reason to make virtualenvs relocatable or sharable across platforms. I don't think virtualenvs as on on-disk file structure make a good distribution/deployment mechanism at all. IOW, I hijacked this thread (sorry) to respond to a specific denigration of the value of virtualenv that I disagree with. I don't care about making virtualenvs consistent across platforms. Carl
On Mon, Mar 26, 2012 at 12:58 PM, Carl Meyer
No disagreement here. I think virtualenv's sweet spot is as a convenient tool for development environments (used in virtualenvwrapper fashion, where the file structure of the virtualenv itself is hidden away and you never see it at all). I think it's fine to deploy _into_ a virtualenv, if you find that convenient too (though I think there are real advantages to deploying just a big ball of code with no need for installers). But I see little reason to make virtualenvs relocatable or sharable across platforms. I don't think virtualenvs as on on-disk file structure make a good distribution/deployment mechanism at all.
IOW, I hijacked this thread (sorry) to respond to a specific denigration of the value of virtualenv that I disagree with. I don't care about making virtualenvs consistent across platforms.
Well, if you're the virtualenv maintainer (or at least the PEP author), and you're basically shooting down the principal rationale for reorganizing the Windows directory layout, then it's not really much of a hijack - it's pretty darn central to the thread!
On 3/26/2012 12:27 PM, PJ Eby wrote:
On Mon, Mar 26, 2012 at 12:58 PM, Carl Meyer
mailto:carl@oddbird.net> wrote: No disagreement here. I think virtualenv's sweet spot is as a convenient tool for development environments (used in virtualenvwrapper fashion, where the file structure of the virtualenv itself is hidden away and you never see it at all). I think it's fine to deploy _into_ a virtualenv, if you find that convenient too (though I think there are real advantages to deploying just a big ball of code with no need for installers). But I see little reason to make virtualenvs relocatable or sharable across platforms. I don't think virtualenvs as on on-disk file structure make a good distribution/deployment mechanism at all.
IOW, I hijacked this thread (sorry) to respond to a specific denigration of the value of virtualenv that I disagree with. I don't care about making virtualenvs consistent across platforms.
Well, if you're the virtualenv maintainer (or at least the PEP author), and you're basically shooting down the principal rationale for reorganizing the Windows directory layout, then it's not really much of a hijack - it's pretty darn central to the thread!
What I read here is a bit different than Mr Eby read, it seems. I read Carl as suggesting that keeping deployment copies of virtualenvs as foolish, but thinking it is fine to deploy into a virtualenv file structure (although preferring to deplay a big ball of code, himself). Personally, I see application deployment as a big ball of code the preferred technique also, but library/module deployment is harder to do that way... it sort of defeats the ability to then bundle the library/module into the big ball of code for the application. But if the goal is to deploy a big ball of code, that would run on top of an installed Python or virtualenv Python, then that is a lot easier if the only modules used are Python modules (no C extensions). Such can be bundled into a zip file, with little support, such that a relative Python novice as myself can figure it out and implement it quickly. C extensions cannot be run from a zip file, so then one needs support code to unzip the C binaries dynamically, and (possibly) delete them when done. Or am I missing something? Hmm. And here's something else that might be missing: integration of the launcher with .py files that are actually ZIP archives... where does it find the #! line? (probably it can't, currently -- I couldn't figure out how to make it do it). Is it possible to add a #! line at the beginning of a ZIP archive for the launcher to use, and still have Python recognize the result as a ZIP archive? I know self-extracting archives put an executable program in front of a ZIP file, and the result is still recognized by most ZIP archivers, but I tried just putting a #! line followed by a ZIP archive, and Python gave me SyntaxError: unknown decode error.
On 3/26/2012 1:21 PM, Glenn Linderman wrote:
Hmm. And here's something else that might be missing: integration of the launcher with .py files that are actually ZIP archives... where does it find the #! line? (probably it can't, currently -- I couldn't figure out how to make it do it). Is it possible to add a #! line at the beginning of a ZIP archive for the launcher to use, and still have Python recognize the result as a ZIP archive? I know self-extracting archives put an executable program in front of a ZIP file, and the result is still recognized by most ZIP archivers, but I tried just putting a #! line followed by a ZIP archive, and Python gave me SyntaxError: unknown decode error.
OK, my first try there, I forgot the stupid Windows /b switch on copy, so apparently the ZIP archive got mangled. When I use ropy /b to join #!/usr/bin/python3.2 and a zip file, it now works. Sorry for the noise.
participants (3)
-
Carl Meyer
-
Glenn Linderman
-
PJ Eby