[Python-Dev] Python environment registration in the Windows Registry

[Steve Dower]

On 03Feb2016 0321, Paul Moore wrote:
> Some issues with this proposal:
> 
> 1. I don't like the way it states that Python distributions "must" set
> keys. I'd rather that it were explicitly stated that a distribution
> which sets no registry keys is entirely valid (with the obvious
> implication that tools which scan the registry looking for installed
> Python distributions won't be able to see such an installation).

Good point, I never meant to imply that. If you don't want your Python install/env found then you don't have to register anything.

(Of course, when users come to us IDE developers and say "your tool can't find Python XYZ", we'll all just go to Python XYZ and say "your users need you to register when you install, see this PEP" :) )

> 2. It's not clear to me how alternative distributions will specify
> their registry keys (or the fact that they don't use them). The
> registry keys are built into the Python interpreter - this proposal
> seems to imply that distributions that simply repackage the python.org
> build will no longer be valid or supported, and instead anyone who
> wants to produce a custom Python distribution (even private in-house
> repackagings) will need to change the Python source and rebuild.
> That's a pretty major change, and if that *is* the intent, then as a
> minimum I'd say we need to provide compiler flags to let
> redistributors specify their Company/Tag values (or say that they want
> to omit registry use). And I'm still not happy that "repackage the
> python.org binaries" has been removed as an option.

There's only one place where the registry key is used within the interpreter itself, which is PC/getpathp.c. Essentially the process is this:

sys.path = []
sys.path.append('')
sys.path.extend(os.getenv('PYTHONPATH').split(';'))
sys.path.extend(read_subkeys(fr'HKCU\Software\Python\PythonCore\{sys.winver}\PythonPath\**'))
sys.path.extend(read_subkeys(fr'HKLM\Software\Python\PythonCore\{sys.winver}\PythonPath\**'))

home = os.getenv('PYTHONHOME')
if not home:
    if os.path.exists(os.path.join(sys.argv[0], '..', 'Lib', 'os.py')):
        home = os.path.dirname(sys.argv[0])
if not home:
    paths = read_value(fr'HKCU\Software\Python\PythonCore\{sys.winver}\PythonPath')
    if not paths:
        paths = read_value(fr'HKLM\Software\Python\PythonCore\{sys.winver}\PythonPath')
    if paths:
        sys.path.extend(paths.split(';'))
    else:
        sys.path.append(r'.\Lib')
        # more well-known subdirs
else:
    sys.path.append(os.path.join(home, 'Lib'))
    # more well-known subdirs ...

So a few high-level observations:

* any program can install anywhere on the machine and make its libraries available to a specific version of Python by creating a subkey under 'PythonCore\x.y\PythonPath'
* any environment lacking 'Lib\os.py' (e.g. venv) relies on the registry to locate enough stdlib to import site
* this is too complicated, but guaranteed we will break users in production if we change it now

So if repackagers follow a few rules (that I documented in https://docs.python.org/3.5/using/windows.html - I see the process above is also documented there, which I wish I remembered before writing all that out), they'll be fine. Unfortunately, following those rules means that you don't register anywhere that separate tools can find you, and so users complain and you "fix" it by doing the wrong thing.

This PEP offers a right way to fix it.

> It's possible that the reason the above two points have been missed is
> because the proposal focuses purely on "informational" registry data.
> But Python also modifies sys.path based on the registry entries - and
> possibly has other behavioural changes as well. The pywin32 package,
> in particular, makes use of this (it's a feature of pywin32 that I
> disagree with and I wish it didn't do that, but it does, and it's a
> very widely used package on Windows). So ignoring this aspect of
> Python's behaviour is a big problem. (Also, what changes will pywin32
> need to make to correctly support being installed into non-python.org
> distributions when this proposal is implemented?)

I haven't looked into pywin32's use of this recently - I tend to only use Christoph Gohlke's  wheels that don't register anything. But it is certainly a valid concern. Hopefully Mark Hammond is watching :)

Cheers,
Steve

> 
> Paul