[Python-Dev] PEP 514: Python registration in the Windows registry
steve.dower at python.org
Mon Jul 18 12:33:22 EDT 2016
On 16Jul2016 1254, Paul Moore wrote:
> On 15 July 2016 at 23:20, Steve Dower <steve.dower at python.org> wrote:
>> Hi all
>> I'd like to get this PEP approved (status changed to Active, IIUC).
> Some comments below.
Awesome, thanks! Posted a pull request at
https://github.com/python/peps/pull/59 for ease of diff reading, and
some commentary below (with aggressive snipping).
>> When installed on Windows, the official Python installer creates a registry
>> key for discovery and detection by other applications. This allows tools such
>> as installers or IDEs to automatically detect and display a user's Python
> The PEP seems quite strongly focused on GUI tools ... I'd like to avoid tool
> developers reading this section and thinking "it only applies to GUI tools or
> OS integration, not to me".
Agreed. I tried to avoid any console/GUI-specific terms, but I can
probably be more explicit about it being useful to both.
> For example, virtualenv introspects the available Python installations
> - see https://github.com/pypa/virtualenv/blob/master/virtualenv.py#L86
> - to support the "-p <interpreter>" flag. To handle this well, it
> would be useful to allow distributions to register a "short tag", so
> that as well as "-p 3.5" or "-p 2", Virtualenv could support (say) "-p
> conda3.4" or "-p pypy2". (The short tag should be at the Company
> level, so "conda" or "pypy", and the version gets added to that).
> Another place where this might be useful is the py.exe launcher (it's
> not in scope for this PEP, but having the data needed to allow the
> launcher to invoke any available installation could be useful for
> future enhancements).
virtualenv would be a great example to use. My thinking was that the Tag
should be appropriate here (perhaps with the Company to disambiguate
when necessary), and that is now explicit.
Anaconda currently has "Anaconda_4.1.1_64-bit" as their tag, which would
not be convenient, so an explicit suggestion here would help ensure this
> Another key motivation for me would be to define clearly what
> information tools can rely on being able to get from the available
> registry entries describing what's installed. Whenever I've needed to
> scan the registry, the things I've needed to find out are where I find
> the Python interpreter, what Python version it is, and whether it's
> 32-bit or 64-bit. The first so that I can run Python, and the latter
> two so that I can tell if this is a version I support *without*
> needing to run the interpreter. For me, everything else in this PEP is
> about UI, but those 3 items plus the "short tag" idea are more about
> what capabilities I can provide.
Good points. I discussed architecture with a colleague at one point and
I'm not entirely sure it's universally useful (what architecture is
IronPython when built for Any CPU? what architecture is Jython?), but
maybe something like the contents of importlib.machinery.IMPORT_SUFFIXES
>> On 64-bit Windows, ``HKEY_LOCAL_MACHINE\Software\Wow6432Node`` is a special
>> key that 32-bit processes transparently read and write to rather than
>> accessing the ``Software`` key directly.
> It might be worth being more explicit here that 32-bit and 64-bit
> processes see the registry keys slightly differently. More on this
I considered this and thought I had a link to the official docs about
it. I don't want this PEP to be mistaken for documentation on how
registry redirection works :)
>> Backwards Compatibility
> Also, Python 3.5 doesn't appear to include the architecture in
> sys.winver either.
> (Unless it adds -32 for 32-bit, and reserves the bare version for
> 64-bit. I've skimmed the CPython source but can't confirm that). The
> documentation of sys.winver makes no mention of whether it
> distinguishes 32- and 64-bit builds. In fact, it states "The value is
> normally the first three characters of version". If we're relying on
> sys.winver being unique by version/architecture, the docs need to say
> so (so that future changes don't accidentally violate that).
I'll update the docs, but your guess is correct. I changed sys.winver on
32-bit to be "3.5-32" since that matches what py.exe was already using
to refer to it. I didn't want to invent
yet-another-way-to-tag-architectures. (I also updated py.exe to match
tags directly under PythonCore, so 3.5-32 is matched without scanning
the binary type.)
Also, sys.winver is defined in PCBuild/python.props, which is how we
accidentally backported the suffix to 2.7.11 :(
>> It is not possible to detect side-by-side installations of both 64-bit and
>> 32-bit versions of Python prior to 3.5 when they have been installed for the
>> current user. Python 3.5 and later always uses different Tags for 64-bit and
>> 32-bit versions.
> From what I can see, this latter isn't true. I presume that 64-bit
> uses no suffix, but 32-bit uses a "-32" suffix? This should probably
> be made explicit. At a minimum, if I were writing a tool to list all
> installed Python versions, with only what I have available to go on
> (the PEP and a 64-bit Python 3.5) I wouldn't be able to write correct
> code, as I don't have all the information I need.
> Also, if we expect to be able to distinguish 32 and 64 bit
> implementations in this way, that's putting a new restriction on
> sys.winver, that it returns a different value for 32-bit and 64-bit
> builds. If that's the case, I'd rather see that explicitly documented,
> both here and in the sys.winver documentation.
> I'd actually prefer a more explicit mechanism going forward, but as
> this is a "backward compatibility" section I'll save that for later.
I don't want to lock in the actual scheme of the tags used by CPython.
Granted, without SysArchitecture in the key you can't identify the
architecture from the information you have, but Tags are supposed to be
treated as opaque with the exception of those that were released prior
to 3.5 (also not explicit, so I'll fix that).
>> The Company part of the key is intended to group related environments and to
>> ensure that Tags are namespaced appropriately. The key name should be
>> alphanumeric without spaces and likely to be unique. For example, a
>> name, a UUID, or a hostname would be appropriate::
> I'd suggest adding "Human-readable Company values are preferred".
> UUIDs seem like a horrible idea in practice.
Maybe, but name-squatting seems equally bad (and I've seen enough
collaborative registration systems like this to see the value in
guaranteed uniqueness). I'll recommend trademarked names though.
> It's also worth noting that "Display Name" isn't actually as useful as
> it sounds, in practice. A tool that relies on it would report the
> python.org installers as being provided by "PythonCore", which isn't
> particularly user friendly. Maybe we need something in the "Backward
> Compatibility" section going into a bit more detail as to how tools
> should deal with that, and maybe we need to add a "DisplayName" in
I'll specify defaults for PythonCore on each of them and add them to 3.6
(once the PEP is approved and we agree on the values). It certainly
doesn't harm the usefulness, but we do want to make sure that tools are
handling old PythonCore entries in a consistent way.
>> If a string value named ``DisplayName`` exists, it should be used to
>> identify the environment to users. Otherwise, the name of the key should be used.
> To an extent there's the same comment here as for DisplayName for
> Company - it needs to be defined with consideration for how it will be
> used. This is, of course, more of a "quality of implementation" matter
> than a standards one. But the PEP might benefit from an example of
> use, maybe showing the output from a hypothetical command line tool
> that lists all installations on the machine.
It's defined as being used to "identify ... to users". Equally, the
SupportUrl "may be displayed or otherwise used to direct users to
[support]". I feel that these are strong enough definitions, and that
showing an hypothetical command line tool output might be seen as too
prescriptive (or alternatively, swinging the pendulum too far away from
>> If a string value named ``Version`` exists, it should be used to identify the
>> version of the environment. This is independent from the version of Python
>> implemented by the environment.
>> If a string value named ``SysVersion`` exists, it must be in ``x.y`` or
>> ``x.y.z`` format matching the version returned by ``sys.version_info`` in the
>> interpreter. Otherwise, if the Tag matches this format it is used. If not,
>> the Python version is unknown.
> I'm not too happy with this. [...]
>> Note that each of these values is recommended, but optional.
> SysVersion and SysArchitecture (or a Tag that works as a fallback)
> should be mandatory. Otherwise I'm OK with this statement.
Snipped most of the details because I agree it's unsatisfying right now,
but I disagree with enough of the counterproposal that it was getting to
be messy commenting on each bit.
Basically, I added SysArchitecture (to match platform.architecture(),
typically '32bit' or '64bit' but extensible without having to define all
potential values in the PEP) with a note that for PythonCore it should
be inferred from the registry path.
SysVersion no longer allows inferring it from the Tag, except for
PythonCore and only when SysVersion is missing.
I'm very keen to not force any of this information to be required as it
is very difficult to know how to deal with interpreters that don't
include it. Does virtualenv refuse to list/use it, even if the install
path is valid, just because SysArchitecture was omitted? What if the
registry becomes corrupt - should Visual Studio refuse to show what
information it can obtain?
I already stated that all information is recommended. The change I've
made now is that tools shouldn't work too hard to guess - if SysVersion
is missing, they can simply say they don't know what version the
interpreter is. If knowing the version is critically important, they can
refuse to use it, but for many applications this is not going to be the
And Python 3.6 will specify all of the keys. Adding it to Python 3.5 is
only likely to cause issues now with people who test against 3.5.3 and
not 3.5.2, which didn't have the keys.
>> Beneath the environment key, an ``InstallPath`` key must be created. This key
>> is always named ``InstallPath``, and the default value must match
>> (Default) = "C:\ExampleCorpPy36"
>> If a string value named ``ExecutablePath`` exists, it must be a path to the
>> ``python.exe`` (or equivalent) executable. Otherwise, the interpreter
>> executable is assumed to be called ``python.exe`` and exist in the directory
>> referenced by the default value.
>> If a string value named ``WindowedExecutablePath`` exists, it must be a path
>> to the ``pythonw.exe`` (or equivalent) executable. Otherwise, the windowed
>> interpreter executable is assumed to be called ``pythonw.exe`` and exist in
>> the directory referenced by the default value.
> These two items assume implicitly that a Python installation must
> provide python.exe and pythonw.exe. I'm inclined to make this
> explicit. Specifically, I think it's crucial that tools can read the
> (console or windowed) executable path as described here, and run that
> executable with standard Python command line arguments, and expect it
> to work. Otherwise there's little point in the installation
> registering its existence.
Again, there's a backwards compatibility argument here in that Python
3.4 and earlier did not create full paths to the executables. But that
can be called out separately.
When you say "assume implicitly ... python.exe and pythonw.exe", do you
mean executables by those names (counterexample - IronPython includes
ipy.exe and ipyw.exe, which you either know specially or would discover
from these keys)? Or BOTH executables (e.g. python.exe AND pythonw.exe)?
Or executables with equivalent behaviour?
I'd argue that the whole PEP only applies to Python interpreters, and if
you don't support standard command line arguments you aren't really a
Python interpreter and shouldn't be registering as one. But I hesitate
to try and define a hard rule that captures all the possible nuances
here - I'd rather deal with it by having users file bugs against
offending interpreters for not working correctly.
> I can see an argument for a distribution providing just python.exe and
> omitting pythonw.exe (or even the other way around). But I can't see
> how I could write generic code to work with such a distribution. So
> let's disallow that possibility until someone comes up with a concrete
> use case
I think in this case, you'd either specify both keys with the same path
(so tools that want the windowed executable are going to get a console
window) or omit the key and make sure you don't have a "pythonw.exe" in
your install directory (which seems unlikely :) ). These keys are mainly
about the possibility of renaming the executables, as shown in the example.
But I've added a note reminding tools developers that the executable may
not exist, and attempting to launch it may fail (i.e. business as usual).
>> Other Keys
>> Some other registry keys are used for defining or inferring search paths
>> under certain conditions. A third-party installation is permitted to define
>> these keys under their Company-Tag key, however, the interpreter must be
>> modified and rebuilt in order to read these values. Alternatively, the
>> interpreter may be modified to not use any registry keys for determining
>> search paths. Making such changes is a decision for the third party; this PEP
>> makes no recommendation either way.
> I think we need to be clearer here. ...
Great suggestion. I've revised this section.
(I have vague plans to make the PythonPath subkey redundant in more
cases for 3.6, and I *think* we can probably drop the Modules key
completely, but I'm not entirely sure it's a good idea. Still thinking
about it :) )
More information about the Python-Dev