[Python-Dev] Python install layout and the PATH on win32 (Rationale part 1: Regularizing the layout)

[M.-A. Lemburg]

VanL wrote:
> As this has been brought up a couple times in this subthread, I figured that I would lay out the
> rationale here.
> 
> There are two proposals on the table: 1) Regularize the install layout, and 2) move the python
> binary to the binaries directory. This email will deal with the first, and a second email will deal
> with the second.
> 
> 1) Regularizing the install layout:
> 
> One of Python's strengths is its cross-platform appeal. Carefully-
> written Python programs are frequently portable between operating
> systems and Python implementations with very few changes. Over the
> years, substantial effort has been put into maintaining platform
> parity and providing consistent interfaces to available functionality,
> even when different underlying implementations are necessary (such
> as with ntpath and posixpath).
> 
> One place where Python is unnecessarily different, however, is in
> the layout and organization of the Python environment. This is most
> visible in the name of the directory for binaries on the Windows platform ("Scripts") versus the
> name of the directory for binaries on every other platform ("bin"), but a full listing of the
> layouts shows
> substantial differences in layout and capitalization across platforms.
> Sometimes the include is capitalized ("Include"), and sometimes not; and
> the python version may or may not be included in the path to the
> standard library or not.
> 
> This may seem like a harmless inconsistency, and if that were all it was, I wouldn't care. (That
> said, cross-platform consistency is its own good). But it becomes a real pain when combined with
> tools like virtualenv or the new pyvenv to create cross-platform development environments.
> 
> In particular, I regularly do development on both Windows and a Mac, and then deploy on Linux. I do
> this in virtualenvs, so that I have a controlled and regular environment. I keep them in sync using
> source control.
> 
> The problem comes when I have executable scripts that I want to include in my dvcs - I can't have it
> in the obvious place - the binaries directory - because *the name of the directory changes when you
> move between platforms.* More concretely, I can't hg add "Scripts/runner.py? on my windows
> environment (where it is put in the PATH by virtualenv) and thendo a pull on Mac or Linux and have
> it end up properly in "bin/runner.py" which is the correct PATH for those platforms.
> 
> This applies anytime there are executable scripts that you want to manage using source control
> across platforms. Django projects regularly have these, and I suspect we will be seeing more of this
> with the new "project" support in virtualenvwrapper.
> 
> While a few people have wondered why I would want this -- hopefully answered above -- I have not
> heard any opposition to this part of the proposal.
> 
> This first proposal is just to make the names of the directories match across platforms. There are
> six keys defined in the installer files (sysconfig.cfg and distutils.command.install): 'stdlib',
> 'purelib', 'platlib', 'headers', 'scripts',  and 'data'.
> 
> Currently on Windows, there are two different layouts defined:
> 
>   'nt': {
>     'stdlib': '{base}/Lib',
>     'platstdlib': '{base}/Lib',
>     'purelib': '{base}/Lib/site-packages',
>     'platlib': '{base}/Lib/site-packages',
>     'include': '{base}/Include',
>     'platinclude': '{base}/Include',
>     'scripts': '{base}/Scripts',
>     'data'   : '{base}',
>     },
> 
>   'nt_user': {
>     'stdlib': '{userbase}/Python{py_version_nodot}',
>     'platstdlib': '{userbase}/Python{py_version_nodot}',
>     'purelib': '{userbase}/Python{py_version_nodot}/site-packages',
>     'platlib': '{userbase}/Python{py_version_nodot}/site-packages',
>     'include': '{userbase}/Python{py_version_nodot}/Include',
>     'scripts': '{userbase}/Scripts',
>     'data'   : '{userbase}',
>     },
> 
> 
> The proposal is to make all the layouts change to:
> 
>   'nt': {
>     'stdlib': '{base}/lib',
>     'platstdlib': '{base}/lib',
>     'purelib': '{base}/lib/site-packages',
>     'platlib': '{base}/lib/site-packages',
>     'include': '{base}/include',
>     'platinclude': '{base}/include',
>     'scripts': '{base}/bin',
>     'data'   : '{base}',
>     },
> 
> The change here is that 'Scripts' will change to 'bin' and the capitalization will be removed. Also,
> "user installs" of Python will have the same internal layout as "system installs" of Python. This
> will also, not coincidentally, match the install layout for posix, at least with regard to the
> 'bin', 'lib', and 'include' directories.
> 
> Again, I have not heard *anyone* objecting to this part of the proposal as it is laid out here.
> (Paul had a concern with the lib directory earlier, but he said he was ok with the above).
> 
> Please let me know if you have any problems or concerns with this part 1.

Since userbase will usually be a single directory in the home
dir of a user, the above would loose the possibility to support
multiple Python versions in that directory.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 23 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-04-03: Python Meeting Duesseldorf                     11 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/