Adding Python scripts to PATHEXT on Windows
With the addition of the Python launcher (PEP 397), Python scripts on Windows are executable in much the same way as on Unix. However, "out of the box" it is still not quite possible to use Python scripts. This is because the .py extension is not registered with Windows in the PATHEXT environment variable (which holds all file extensions which should be looked for on PATH at the command line). There are two main consequences to this: 1. Under powershell, ".py" files on PATH are not found at all (this is not the case with cmd.exe, and may be technically a bug in powershell - but it's unlikely to change nevertheless). 2. Under both powershell and cmd, users have to explicitly include the extension when invoking the command. I would like to propose that the .py extension be added to PATHEXT as part of the "Register Python Extensions" option in the Python MSI installer. This would mean that Python users had a clean way of writing cross-platform "wrapper scripts" and command-line applications out of the box, just by writing a normal Python script with a #! line at the top. Does anyone have any objections to this? I could try to write a patch, but I know next to nothing about building MSIs, so if any of the installer experts could help that would be fantastic. Paul PS There would still be one difference, in that scripts intended to be used as commands would be named with '.py' on Windows and with no extension on Unix, but that's a pretty trivial difference, and reflects fundamental platform differences rather than configuration. So I don't propose worrying about this.
+1 BTW: I use lots of "*.py-executables" on Linux and OSX, so different extension don't bother me. Gerald Am 15.07.2013 um 10:26 schrieb Paul Moore <p.f.moore@gmail.com>:
With the addition of the Python launcher (PEP 397), Python scripts on Windows are executable in much the same way as on Unix. However, "out of the box" it is still not quite possible to use Python scripts. This is because the .py extension is not registered with Windows in the PATHEXT environment variable (which holds all file extensions which should be looked for on PATH at the command line).
There are two main consequences to this:
1. Under powershell, ".py" files on PATH are not found at all (this is not the case with cmd.exe, and may be technically a bug in powershell - but it's unlikely to change nevertheless). 2. Under both powershell and cmd, users have to explicitly include the extension when invoking the command.
I would like to propose that the .py extension be added to PATHEXT as part of the "Register Python Extensions" option in the Python MSI installer. This would mean that Python users had a clean way of writing cross-platform "wrapper scripts" and command-line applications out of the box, just by writing a normal Python script with a #! line at the top.
Does anyone have any objections to this? I could try to write a patch, but I know next to nothing about building MSIs, so if any of the installer experts could help that would be fantastic.
Paul
PS There would still be one difference, in that scripts intended to be used as commands would be named with '.py' on Windows and with no extension on Unix, but that's a pretty trivial difference, and reflects fundamental platform differences rather than configuration. So I don't propose worrying about this. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python.00%40klix.ch
Sounds good to me - in the long run it may allow us to drop the Windows exe wrappers when installing scripts on Windows. Cheers, Nick.
On 15 July 2013 09:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
Sounds good to me - in the long run it may allow us to drop the Windows exe wrappers when installing scripts on Windows.
That's my motivation. In particular removing pip's exe wrappers to ease some of the issues being flagged on distutils-sig. Paul
On 15 July 2013 18:42, Paul Moore <p.f.moore@gmail.com> wrote:
On 15 July 2013 09:40, Nick Coghlan <ncoghlan@gmail.com> wrote:
Sounds good to me - in the long run it may allow us to drop the Windows exe wrappers when installing scripts on Windows.
That's my motivation. In particular removing pip's exe wrappers to ease some of the issues being flagged on distutils-sig.
Ah, true. For some reason I was thinking we kept __main__.__file__ open, but we don't - we close it as soon as we have read and compiled it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 15.07.13 10:26, schrieb Paul Moore:
Does anyone have any objections to this? I could try to write a patch, but I know next to nothing about building MSIs, so if any of the installer experts could help that would be fantastic.
It's fine with me. I could write the patch, but will likely forget - so please remind me with every alpha that misses this. Also, contributions are welcome. Regards, Martin
On 23 July 2013 17:11, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Am 15.07.13 10:26, schrieb Paul Moore:
Does anyone have any objections to this? I could try to write a patch, but I know next to nothing about building MSIs, so if any of the installer experts could help that would be fantastic.
It's fine with me. I could write the patch, but will likely forget - so please remind me with every alpha that misses this. Also, contributions are welcome.
Thanks. I think I have the basic idea of what's needed, so I'll write an initial patch. If you can check it that'd be great. Paul
On 23 July 2013 17:33, Paul Moore <p.f.moore@gmail.com> wrote:
On 23 July 2013 17:11, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Am 15.07.13 10:26, schrieb Paul Moore:
Does anyone have any objections to this? I could try to write a patch, but I know next to nothing about building MSIs, so if any of the installer experts could help that would be fantastic.
It's fine with me. I could write the patch, but will likely forget - so please remind me with every alpha that misses this. Also, contributions are welcome.
Thanks.
I think I have the basic idea of what's needed, so I'll write an initial patch. If you can check it that'd be great.
OK, see http://bugs.python.org/issue18569 As I mention in the patch, I have yet to successfully manage to build an MSI installer (I tried a while ago, and the instructions didn't work - I'm not 100% sure why and I'm afraid I don't have the time right now to try again) so the patch will definitely need checking by someone who can build the MSI. When I get some time (probably not for a few weeks) I'll try the "how to build the installer" instructions again and see if I can work out what went wrong and either fix the issue or report back. Paul
Any chance of this being made optional when installing? It provides no benefit for people who prefer to associate scripts with an editor and may be a source of confusion/complaints. Env var changes typically require a reboot to take full effect, which I've also seen cause confusion. Also, it should include .pyw, .pyz and whatever was decided instead of .pywz (.pwz, I think?). Steve (Apologies for the top post.) Sent from my Windows Phone ________________________________ From: Paul Moore<mailto:p.f.moore@gmail.com> Sent: 7/27/2013 1:13 To: Martin v. Löwis<mailto:martin@v.loewis.de> Cc: Python Dev<mailto:python-dev@python.org> Subject: Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows On 23 July 2013 17:33, Paul Moore <p.f.moore@gmail.com<mailto:p.f.moore@gmail.com>> wrote: On 23 July 2013 17:11, "Martin v. Löwis" <martin@v.loewis.de<mailto:martin@v.loewis.de>> wrote: Am 15.07.13 10:26, schrieb Paul Moore:
Does anyone have any objections to this? I could try to write a patch, but I know next to nothing about building MSIs, so if any of the installer experts could help that would be fantastic.
It's fine with me. I could write the patch, but will likely forget - so please remind me with every alpha that misses this. Also, contributions are welcome. Thanks. I think I have the basic idea of what's needed, so I'll write an initial patch. If you can check it that'd be great. OK, see http://bugs.python.org/issue18569 As I mention in the patch, I have yet to successfully manage to build an MSI installer (I tried a while ago, and the instructions didn't work - I'm not 100% sure why and I'm afraid I don't have the time right now to try again) so the patch will definitely need checking by someone who can build the MSI. When I get some time (probably not for a few weeks) I'll try the "how to build the installer" instructions again and see if I can work out what went wrong and either fix the issue or report back. Paul
On 27 July 2013 21:14, Steve Dower <Steve.Dower@microsoft.com> wrote:
Any chance of this being made optional when installing? It provides no benefit for people who prefer to associate scripts with an editor and may be a source of confusion/complaints.
Personally, I don't know how to do this so someone else would have to - but if someone does, I won't particularly object. I'd prefer it to be on by default, though (just like associating py.exe with scripts is) because otherwise it's not reasonable to assume that a .py script will work as a command in the same way that a bat file does - which is really the point of wanting this in the first place. I'm a bit confused by your comment about "people who prefer to associate scripts with an editor", though. This is only done in precisely those situations when the installer associates .py scripts with py.exe (the launcher). So if you switch that off, you don't get PATHEXT either. And if you change the association after the fact, you're presumably just as capable of changing PATHEXT. The key point here is that I want the *default* behaviour of Python when installed using the standard msi to be that if I put a script foo.py on PATH somewhere, then typing "foo" at the command line will run that script. For that to be the case, PATHEXT needs to be set, and I view that as a (minor) omission which was made when the association with py.exe was implemented. (Which reminds me - it would be nice if the standalone installer for the py launcher included setting PATHEXT as well).
Env var changes typically require a reboot to take full effect, which I've also seen cause confusion.
Really? If I add python.exe to PATH in the installer do I need to reboot? (I never do this, so I don't know - it's a genuine question). Also, it should include .pyw, .pyz and whatever was decided instead of
.pywz (.pwz, I think?).
It's basically a command line thing - I don't see why you'd want to launch a GUI script from the command line in the same way. But again, if someone wants to do it, I don't really mind (in this case I know what to do, so if there's consensus that it's needed, I can add it to the patch myself). The PEP for pyz and pwz hasn't been accepted yet. If it does, then yes pyz should be treated the same way (and we can have the same debate about pwz as we do about pyw :-)) Paul
From: Paul Moore <p.f.moore@gmail.com>
I'm a bit confused by your comment about "people who prefer to associate scripts with an editor", though. This is only done in precisely those situations when the installer associates .py scripts with py.exe (the launcher). So if you switch that off, you don't get PATHEXT either.
That's optional enough for me - I always disable that (because I typically have 4-10 different versions installed at once).
And if you change the association after the fact, you're presumably just as capable of changing PATHEXT.
Not if the association is changed by another installer (presumably with the user's explicit permission). It would be very easy for people to go a long time without noticing this until some program does ShellExecute(...) on a name that happens to be a Python script and Visual Studio pops open...
Env var changes typically require a reboot to take full effect, which I've also seen cause confusion. Really? If I add python.exe to PATH in the installer do I need to reboot? (I never do this, so I don't know - it's a genuine question).
Env vars are inherited by child processes, so until you create a process without inheriting them, you won't see the new value. Depending on the shortcut used, cmd.exe is likely to inherit the variables from explorer.exe, which won't see the new ones until you log off/on again. Certainly any running prompts will need to be closed and restarted after installing. That said, on my Win8 machine, cmd.exe seems to be okay after changing the variables. However, PowerShell did not see the change immediately, so cmd.exe may be getting special treatment (or just ignoring the default environment).
Also, it should include .pyw, .pyz and whatever was decided instead of .pywz (.pwz, I think?).
It's basically a command line thing - I don't see why you'd want to launch a GUI script from the command line in the same way.
That's fair enough, though it may be seen as inconsistent. TBH I've never actually seen anything in the wild other than .py. My suspicion is that the .exe wrappers would remain for tools like pip anyway, rather than breaking people who choose to deselect that option. Of course, it's easy enough to replace them. Another issue to consider is that the modification to PATHEXT can't be undone when Python is uninstalled, unless each installation adds another ".PY" and each uninstall removes only one (so my PATHEXT would look like ...;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY). I haven't checked, but this could seriously hurt performance of anything that uses PATHEXT to search PATH for anything ending in this extension. I hope that cmd.exe and PowerShell check for duplicate extensions, but I wouldn't be betting any money that they do. Steve
On 28 July 2013 00:30, Steve Dower <Steve.Dower@microsoft.com> wrote:
Another issue to consider is that the modification to PATHEXT can't be undone when Python is uninstalled, unless each installation adds another ".PY" and each uninstall removes only one (so my PATHEXT would look like ...;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY;.PY). I haven't checked, but this could seriously hurt performance of anything that uses PATHEXT to search PATH for anything ending in this extension. I hope that cmd.exe and PowerShell check for duplicate extensions, but I wouldn't be betting any money that they do.
I used the same mechanism in msi.py (which is MSI functionality, I believe) that is used to add Python's directory to PATH - this adds on install and removes on ininstall. So it should work just as well as that does. Paul
On 28 July 2013 00:30, Steve Dower <Steve.Dower@microsoft.com> wrote:
And if you change the association after the fact, you're presumably just as capable of changing PATHEXT.
Not if the association is changed by another installer (presumably with the user's explicit permission). It would be very easy for people to go a long time without noticing this until some program does ShellExecute(...) on a name that happens to be a Python script and Visual Studio pops open...
Mph. I'd argue that as Python "owns" the .py extension, any other application that changes the default action of .py scripts without both making that behaviour optional and making it off by default, is at least unfriendly and arguably even broken. But that's just a matter of opinion... As regards ShellExecute, I don't know how that is affected by PATHEXT. I looked quite hard and couldn't find complete "official" documentation on how PATHEXT is supposed to work, so I'm working a bit on experiment and best guesses here :-( I'll go and check what the ShellExecute documentation says on the matter (but I'm surprised - I got the impression that PATHEXT was purely a console mechanism, so I don't quite understand why ShellExecute would use it). Paul
Windows fully supports *user* customization of file associations and there is no concept of ownership. The first program installed that sets up an association will normally get it, and as of Win8 the next application will be blocked (and suggested to the user next time they open a file). Whether the other application stole the extension is irrelevant: a user is allowed to associate .py with something other than py.exe. I'm also working off experience with Windows here, I haven't done any special research. It may be that the APIs will assume .exe if no extension is specified, but I'd be surprised if backwards compatibility with with DOS, which I assume included PATHEXT, hadn't been a consideration. The Run dialog uses ShellExecute, so you can test things there easily. Sent from my Windows Phone ________________________________ From: Paul Moore<mailto:p.f.moore@gmail.com> Sent: 7/28/2013 1:00 To: Steve Dower<mailto:Steve.Dower@microsoft.com> Cc: Martin v. Löwis<mailto:martin@v.loewis.de>; Python Dev<mailto:python-dev@python.org> Subject: Re: [Python-Dev] Adding Python scripts to PATHEXT on Windows On 28 July 2013 00:30, Steve Dower <Steve.Dower@microsoft.com<mailto:Steve.Dower@microsoft.com>> wrote:
And if you change the association after the fact, you're presumably just as capable of changing PATHEXT.
Not if the association is changed by another installer (presumably with the user's explicit permission). It would be very easy for people to go a long time without noticing this until some program does ShellExecute(...) on a name that happens to be a Python script and Visual Studio pops open... Mph. I'd argue that as Python "owns" the .py extension, any other application that changes the default action of .py scripts without both making that behaviour optional and making it off by default, is at least unfriendly and arguably even broken. But that's just a matter of opinion... As regards ShellExecute, I don't know how that is affected by PATHEXT. I looked quite hard and couldn't find complete "official" documentation on how PATHEXT is supposed to work, so I'm working a bit on experiment and best guesses here :-( I'll go and check what the ShellExecute documentation says on the matter (but I'm surprised - I got the impression that PATHEXT was purely a console mechanism, so I don't quite understand why ShellExecute would use it). Paul
On 28 July 2013 08:36, Paul Moore <p.f.moore@gmail.com> wrote:
The PEP for pyz and pwz hasn't been accepted yet. If it does, then yes pyz should be treated the same way (and we can have the same debate about pwz as we do about pyw :-))
Oops, thanks for the reminder. I'll nudge Daniel about that :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 28.07.13 00:36, schrieb Paul Moore:
On 27 July 2013 21:14, Steve Dower <Steve.Dower@microsoft.com <mailto:Steve.Dower@microsoft.com>> wrote:
Any chance of this being made optional when installing? It provides no benefit for people who prefer to associate scripts with an editor and may be a source of confusion/complaints.
Personally, I don't know how to do this so someone else would have to -
It seems that this was settled as fine as-is; if you ever wanted to allow this to be specifically enabled, you'd have to do three things 1. create a new "feature" in the feature tree. Your patch currently uses the "Extensions" feature (child of the default feature); you could make your feature a subfeature of Extensions. 2. create a new "component". This should work similar to REGISTRY.path. 3. Add a "FeatureComponents" entry saying that the "REGISTRY.pathext" component (say) belongs to the "PathExt" feature (say). Instead of hooking the environment variable to the REGISTRY.def component, it should then go to the REGISTRY.pathext component. The nice thing about this feature tree is that it allows convenient user configuration yet powerful scripted installation, see http://msdn.microsoft.com/en-us/library/windows/desktop/aa368585(v=vs.85).as... For example, you could set the msidbFeatureAttributesFollowParent flag, which would make it be selected/unselected if the parent feature is, yet still allow it to be independent of the parent for people who really want to. Regards, Martin
On 30 July 2013 18:24, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Personally, I don't know how to do this so someone else would have to -
It seems that this was settled as fine as-is; if you ever wanted to allow this to be specifically enabled, you'd have to do three things
Thanks, Martin - MSI is a mystery to me, so this is very useful. Saved for future reference. Paul
participants (5)
-
"Martin v. Löwis" -
Gerald Klix -
Nick Coghlan -
Paul Moore -
Steve Dower