PEP 370 - per-user scripts directory on Windows
tl;dr - Having a shared per-user scripts directory on Windows (%APPDATA%/Python/Scripts) causes a number of potential issues when users install the same package in multiple Python versions. I'd like to suggest that this be changed to a versioned directory (%APPDATA%/PythonXY/Scripts) to match all other Python directories on Windows. This issue originally started as a discussion on the pip tracker around making --user the default for pip. See https://github.com/pypa/pip/issues/1668 for details. However, because the user scripts directory is shared across all Python versions, this poses a number of issues: 1. If we "pip install" something like pytest, which version gets the name "py.test.exe" works on a "last install wins" basis, which is not ideal. 2. The above behaviour (and more generally the shared site scripts directory) does not respect the user's "Make this my default Python" choice from the installer, in potentially surprising ways. 3. Uninstalling a package which has been installed in 2 pythons will likely fail, because either the checksum of an installed script does not match what's in the RECORD file, or because a file that should be present has already been uninstalled via a different Python version [1]. I suggest that the per-user scripts directory should be versioned, in exactly the same way as every other Python-related directory on Windows. So we would have %APPDATA%\PythonXY\Scripts alongside %APPDATA%\PythonXY\site-packages. This is a familiar pattern for Windows users, and should not cause significant confusion. It would be good to get this added for Python 3.5, *before* making user installs the default (needed for 3.5 as the default install location for 3.5 will need admin privileges to write). Some additional points: 1. This is a Windows-specific proposal, because Unix has always had scripts installed in a common location (e.g., /usr/local/bin) and commonly-understood solutions already exist on that platform (versioned command names, python -m, system package management, etc). However, these solutions would be a significant change for Windows users. 2. Packages which recommend a user install specifically so that their commands are available regardless of which Python is on PATH would have to find an alternative solution. I doubt there are many such packages, but this should be considered. 3. This proposal does not affect the Python launcher for Windows. People using the launcher exclusively may be currently using "py -m pip" (or whatever) to launch applications, in which case the change of user script directory won't affect them. 4. I am not aware of any tools that rely on the location of the user scripts directory. Such scripts should be using sysconfig in any case. 5. The problems mentioned exist in current Python versions, but haven't been flagged up, presumably because very few people use the user install location - precisely because it's not the default in pip. Changing the pip default would cause more people to encounter the issues, but I don't propose backporting this change, as it's not a bug fix. If it becomes a major issue for pip users on older versions, we can either reconsider this, or pip can decide on its own workaround. Comments? If this is acceptable, I would be willing to prepare a patch for Python 3.5 implementing this. Paul [1] Checking of checksums is not currently implemented, AFAIK, but it is the purpose of the checksums in the RECORD file, and could be implemented in future versions of pip.
On 10 Feb 2015 22:40, "Paul Moore" <p.f.moore@gmail.com> wrote:
tl;dr - Having a shared per-user scripts directory on Windows (%APPDATA%/Python/Scripts) causes a number of potential issues when users install the same package in multiple Python versions. I'd like to suggest that this be changed to a versioned directory (%APPDATA%/PythonXY/Scripts) to match all other Python directories on Windows.
This issue originally started as a discussion on the pip tracker around making --user the default for pip. See https://github.com/pypa/pip/issues/1668 for details.
For the reasons Paul gives, I'm personally +1 on making the change. I was actually surprised it didn't already work that way, given that almost everything else on Windows is version specific. Cheers, Nick.
On 10 February 2015 at 13:34, Nick Coghlan <ncoghlan@gmail.com> wrote:
I was actually surprised it didn't already work that way, given that almost everything else on Windows is version specific.
That's actually a point I missed making. While I wouldn't class myself as "typical", when I went looking for the user-scripts directory on my machine, I automatically looked in %APPDATA%\PythonXY. I suspect that many people would assume this was how it worked anyway, and be surprised by the current behaviour. Paul
I was also surprised as I was working on the installer, so +1 on changing it. The installer will also need some changes to add it to PATH, which should be fairly straightforward, but ping me if you need/want pointers. (It's checked in now, so I consider it fair game for anyone who wants to change it.) Cheers, Steve Top-posted from my Windows Phone ________________________________ From: Paul Moore<mailto:p.f.moore@gmail.com> Sent: 2/10/2015 5:40 To: Nick Coghlan<mailto:ncoghlan@gmail.com> Cc: Python Dev<mailto:python-dev@python.org> Subject: Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows On 10 February 2015 at 13:34, Nick Coghlan <ncoghlan@gmail.com> wrote:
I was actually surprised it didn't already work that way, given that almost everything else on Windows is version specific.
That's actually a point I missed making. While I wouldn't class myself as "typical", when I went looking for the user-scripts directory on my machine, I automatically looked in %APPDATA%\PythonXY. I suspect that many people would assume this was how it worked anyway, and be surprised by the current behaviour. Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.c...
On 10 February 2015 at 14:13, Steve Dower <Steve.Dower@microsoft.com> wrote:
The installer will also need some changes to add it to PATH, which should be fairly straightforward, but ping me if you need/want pointers. (It's checked in now, so I consider it fair game for anyone who wants to change it.)
I was sort of hoping you'd be doing that independently, but never mind :-) I'll give it a go and shout if I get stuck. Paul
One of my main engineering concerns is lack of shared knowledge a.k.a. the truck factor (not CPython specific, btw, just every project I work on). So I'm always keen to get multiple people working on new code asap. I also prefer installers to be treated as part of the product and updated with whatever changes affect it, though that basically never happens in my experience :) I assume the PEP will need updating if it specifies the exact path, and the installer readme may have a reference too. Presumably it'll only affect 3.5, as previous versions wouldn't use the directory for anything other than installing scripts. Cheers, Steve Top-posted from my Windows Phone ________________________________ From: Paul Moore<mailto:p.f.moore@gmail.com> Sent: 2/10/2015 6:41 To: Steve Dower<mailto:Steve.Dower@microsoft.com> Cc: Nick Coghlan<mailto:ncoghlan@gmail.com>; Python Dev<mailto:python-dev@python.org> Subject: Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows On 10 February 2015 at 14:13, Steve Dower <Steve.Dower@microsoft.com> wrote:
The installer will also need some changes to add it to PATH, which should be fairly straightforward, but ping me if you need/want pointers. (It's checked in now, so I consider it fair game for anyone who wants to change it.)
I was sort of hoping you'd be doing that independently, but never mind :-) I'll give it a go and shout if I get stuck. Paul
Am 10.02.15 um 16:41 schrieb Steve Dower:
One of my main engineering concerns is lack of shared knowledge a.k.a. the truck factor (not CPython specific, btw, just every project I work on).
Wrt. the installer, I think this is a lost cause. IIUC, you aren't really getting familiar with msi.py, and you are the one who has the highest chances - so I was the only real maintainer of it. Likewise, the Wise installer was only maintained by Tim Peters. So chances are really high that you will be the only one making substantial changes to the WiX files, although some people might learn how to build the installer. Regards, Martin
Martin v. Löwis wrote:
Am 10.02.15 um 16:41 schrieb Steve Dower:
One of my main engineering concerns is lack of shared knowledge a.k.a. the truck factor (not CPython specific, btw, just every project I work on).
Wrt. the installer, I think this is a lost cause. IIUC, you aren't really getting familiar with msi.py, and you are the one who has the highest chances - so I was the only real maintainer of it. Likewise, the Wise installer was only maintained by Tim Peters.
So chances are really high that you will be the only one making substantial changes to the WiX files, although some people might learn how to build the installer.
Aww, come on. Let me have a little bit of hope ;)
Regards, Martin
On 10 February 2015 at 14:13, Steve Dower <Steve.Dower@microsoft.com> wrote:
I was also surprised as I was working on the installer, so +1 on changing it.
On a procedural note, does this require a change to the PEP (assuming it's generally acceptable)? Or is a patch to the code and docs sufficient? Paul
Am 10.02.15 um 15:43 schrieb Paul Moore:
On 10 February 2015 at 14:13, Steve Dower <Steve.Dower@microsoft.com> wrote:
I was also surprised as I was working on the installer, so +1 on changing it.
On a procedural note, does this require a change to the PEP (assuming it's generally acceptable)? Or is a patch to the code and docs sufficient?
There should be a PEP if the change is likely controversial. In any case, there should be documentation of the rationale for the change, and the docs itself is the wrong place for it. The commit message could be the right place, but if the rationale is a bit more verbose, a bug tracker item would be right. All IMO, of course. Martin
On 10 February 2015 at 12:38, Paul Moore <p.f.moore@gmail.com> wrote:
tl;dr - Having a shared per-user scripts directory on Windows (%APPDATA%/Python/Scripts) causes a number of potential issues when users install the same package in multiple Python versions. I'd like to suggest that this be changed to a versioned directory (%APPDATA%/PythonXY/Scripts) to match all other Python directories on Windows.
Making the full detail explicit (based on a comment made on the pip issue), the default PATH should match the behaviour of sys.path (so that "pip" and "python -m pip" always execute the same thing). So we have C:\PythonXY before %APPDATA%\PythonXY\Scripts before C:\PythonXY\Scripts (matching stdlib before user site-packages before system site-packages). And user site-packages is versioned because everything else is. The PATH setting is purely an installer change, of course - only versioning the user scripts directory affects the core. Paul
On 02/10/2015 04:38 AM, Paul Moore wrote:
tl;dr - Having a shared per-user scripts directory on Windows (%APPDATA%/Python/Scripts) causes a number of potential issues when users install the same package in multiple Python versions. I'd like to suggest that this be changed to a versioned directory (%APPDATA%/PythonXY/Scripts) to match all other Python directories on Windows.
+1 -- ~Ethan~
So I've enumerated the problems with PATH on Windows before (and my subsequent dislike for modifying it through the installer), but here's the dot-point summary for those who haven't seen it. * System-wide values always precede per-user values * Most-recently-installed values always precede existing values (within the system/user separation above) * It's impossible to specify per-user values from a system-wide installer * The system-wide value has a length limit that silently breaks PATH if you exceed it * It's difficult for users to modify PATH themselves (mostly due to bad UI in Windows) The py.exe launcher solves many of these problems, but does not help with installed scripts. It's trivially easy to end up in a place where this would fail: pip install --user spam spam The *only* case where this will work reliably (without manual user configuration) is where Python 3.5 is the only installed version on the machine, and it was installed by the current user just for them. As we've seen from earlier discussions, the main beneficiaries of having Python on PATH are those using the command-line. Most scripts are going to make assumptions or work unnecessarily hard to find the actual location of the Python version they need. My best idea for dealing with this is to add another script alongside the py.exe launcher called `activate-py`.[1] This script would take the same command-line version argument as py.exe and use it to update the current prompt's PATH correctly. Specifically, I expect the script would actually use py.exe to invoke the requested Python and get the paths directly from sysconfig, which will ensure that the order of precedence always matches. Users would need to run `activate-py`, `activate-py -2` or `activate-py -3.5` (etc.) when they start their session.[2] This parallels venv's `activate` command, though because this would be a global command I believe the `-py` suffix is necessary. The example above becomes: activate-py -3.5 pip install --user spam spam A `deactivate-py` command would also be added to revert the changes, though I don't expect that to be part of most workflows. Another benefit is that shell scripts could call `activate-py` rather than trying to detect whether Python is already on the path or requiring a PYTHON variable to be set. (I seem to write these scripts all the time and know that I'd benefit from it.) This is yet another attempt to try and change user behaviour, which I'm not thrilled about, but I'm really starting to feel that this is the best way out of a bad situation. It differs from the solutions used on Linux, though there may be some value in this approach there too. Thought/votes/suggestions? Cheers, Steve [1]: Actually multiple scripts to handle cmd.exe/PowerShell, but this is an implementation detail. Users would simply type `activate-py` and the shells know which one to use. [2]: We can easily add Start Menu shortcuts (e.g. "Python 3.5 (32-bit) Command Prompt") and run it for them, though that is not a critical part of this proposal.
On 10 February 2015 at 17:45, Steve Dower <Steve.Dower@microsoft.com> wrote:
This is yet another attempt to try and change user behaviour, which I'm not thrilled about, but I'm really starting to feel that this is the best way out of a bad situation. It differs from the solutions used on Linux, though there may be some value in this approach there too. Thought/votes/suggestions?
I have to admit, I agree with you. But before the "Add Python to PATH" option was added to the installer, we repeatedly got requests for it and it was apparently a constant source of confusion for beginners. If we're going to revert that change, we'll need to be very careful how we present it.
On 10 February 2015 at 20:50, Paul Moore <p.f.moore@gmail.com> wrote:
On 10 February 2015 at 17:45, Steve Dower <Steve.Dower@microsoft.com> wrote:
This is yet another attempt to try and change user behaviour, which I'm not thrilled about, but I'm really starting to feel that this is the best way out of a bad situation. It differs from the solutions used on Linux, though there may be some value in this approach there too. Thought/votes/suggestions?
I have to admit, I agree with you. But before the "Add Python to PATH" option was added to the installer, we repeatedly got requests for it and it was apparently a constant source of confusion for beginners. If we're going to revert that change, we'll need to be very careful how we present it.
Drat - hit "Send" too soon. Also, the proposed scripts won't be on PATH, so users will have to run them using explicit paths, which is both inconvenient and poses a discoverability issue (Tools/scripts/win_add2path.py has been round for ages, but AFAIK hardly anyone uses it). I wonder if there's a way to get the launcher (py.exe) to help? For powershell, we could have "py --setpath" write the appropriate path setting command to stdout, and users could use "Invoke-Expression $(py --setpath)", for example. That's a bit clumsy, and doesn't work for cmd.exe, but maybe there's possibilities... Essentially, py.exe is the only command guaranteed to be on PATH for any install (or is it? for a user install, is it stuck somewhere in the user's APPDATA?) It's fair enough to say that installers shouldn't be in the business of messing with PATH, but nevertheless, people expect to run an installer and have the installed programs available for use. Command line utilities like Mercurial do it, so people will expect Python to. Unfortunately. Sorry, all problems, no real answers here. Paul
Paul Moore wrote:
On 10 February 2015 at 20:50, Paul Moore <p.f.moore@gmail.com> wrote:
On 10 February 2015 at 17:45, Steve Dower <Steve.Dower@microsoft.com> wrote:
This is yet another attempt to try and change user behaviour, which I'm not thrilled about, but I'm really starting to feel that this is the best way out of a bad situation. It differs from the solutions used on Linux, though there may be some value in this approach there too. Thought/votes/suggestions?
I have to admit, I agree with you. But before the "Add Python to PATH" option was added to the installer, we repeatedly got requests for it and it was apparently a constant source of confusion for beginners. If we're going to revert that change, we'll need to be very careful how we present it.
Drat - hit "Send" too soon.
Also, the proposed scripts won't be on PATH, so users will have to run them using explicit paths, which is both inconvenient and poses a discoverability issue (Tools/scripts/win_add2path.py has been round for ages, but AFAIK hardly anyone uses it). I wonder if there's a way to get the launcher (py.exe) to help? For powershell, we could have "py --setpath" write the appropriate path setting command to stdout, and users could use "Invoke-Expression $(py --setpath)", for example. That's a bit clumsy, and doesn't work for cmd.exe, but maybe there's possibilities... Essentially, py.exe is the only command guaranteed to be on PATH for any install (or is it? for a user install, is it stuck somewhere in the user's APPDATA?)
(For a user install, I've guaranteed that the launcher will be added to the user's PATH. But a system-wide launcher will win.) win_add2path.py does the same thing as the installer, so it isn't a useful solution. The `py --setpath` is basically my idea with `activate-py`, though with the "Invoke-Expression" part handled in the script (and not actually using Invoke-Expression, but that's implementation details). If these are part of the launcher, then they'll always be on PATH, and if they're just thin wrappers around py.exe then they can support all the same versions of Python (in effect, all of them, though with the 32/64-bit per-user collisions that existed pre-3.5).
It's fair enough to say that installers shouldn't be in the business of messing with PATH, but nevertheless, people expect to run an installer and have the installed programs available for use. Command line utilities like Mercurial do it, so people will expect Python to. Unfortunately.
And Python does, as long as that installed program is "py". The problem is that people expect "python" to be there and be the correct one (for a totally arbitrary definition of "correct") as well as pip, easy_install and anything installed by those two tools, whether they were installed for all users or just the current one. It's far more complex than a single app (especially when you realise that a system-wide install is effectively trying to make Python look like part of the OS - we only get that about 50% right now...).
Sorry, all problems, no real answers here. Paul
Yeah, same. `activate-py` is the best I've got. Cheers, Steve
On 11 Feb 2015 03:47, "Steve Dower" <Steve.Dower@microsoft.com> wrote:
This is yet another attempt to try and change user behaviour, which I'm
not thrilled about, but I'm really starting to feel that this is the best way out of a bad situation. It differs from the solutions used on Linux, though there may be some value in this approach there too. Thought/votes/suggestions? I don't think we have this problem solved on Linux either, we just have a variety of workarounds in place of varying degrees of clunkiness :) The thing I find attractive about this specific proposal is that we could potentially expand it to Linux as well through appropriate use of environment modules, and I expect that would also cover other POSIX systems. So if you wanted to draft up an initial PEP with a concrete proposal covering Windows systems, I'd be happy to tackle the task of getting the same design to work on Linux (and hopefully POSIX in general) using environment modules. Cheers, Nick.
Cheers, Steve
[1]: Actually multiple scripts to handle cmd.exe/PowerShell, but this is
an implementation detail. Users would simply type `activate-py` and the shells know which one to use.
[2]: We can easily add Start Menu shortcuts (e.g. "Python 3.5 (32-bit)
Command Prompt") and run it for them, though that is not a critical part of this proposal.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
Am 10.02.2015 um 18:45 schrieb Steve Dower:
As we've seen from earlier discussions, the main beneficiaries of having Python on PATH are those using the command-line. Most scripts are going to make assumptions or work unnecessarily hard to find the actual location of the Python version they need.
Maybe I'm more or less alone with the way I work, but I don't like python.exe on my PATH (and py.exe alloes me to do this). I start python scripts from the command line either with 'script.py' or 'py -3.4 script.py' or 'py -2.7 script.py'. I even like running scripts in this way: 'py -2.7 -m pip install whatever'. However, this only works when the script also can be started as package. Some packages allow this, some don't. Could not py.exe be extended so that it allows starting scripts in a somewhat similar way? 'py-script -2.7 myscript foo bar baz' ??? Which would execute the script myscript.exe, myscript.bat, myscript.py, myscript.cmd or whatever is in the Scripts directory on the Python 2.7 installation, without changing PATH presistently? Thanks, Thomas
You're not alone, that's exactly how I do it too. I'm writing up a PEP for the activate-py command I suggested earlier that will temporarily set your PATH. If that goes well then we'll be able to ship that with the py.exe launcher (I'm particularly excited for batch files to be able to use it). Cheers, Steve Top-posted from my Windows Phone ________________________________ From: Thomas Heller<mailto:theller@ctypes.org> Sent: 2/12/2015 0:03 To: python-dev@python.org<mailto:python-dev@python.org> Subject: Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows Am 10.02.2015 um 18:45 schrieb Steve Dower:
As we've seen from earlier discussions, the main beneficiaries of having Python on PATH are those using the command-line. Most scripts are going to make assumptions or work unnecessarily hard to find the actual location of the Python version they need.
Maybe I'm more or less alone with the way I work, but I don't like python.exe on my PATH (and py.exe alloes me to do this). I start python scripts from the command line either with 'script.py' or 'py -3.4 script.py' or 'py -2.7 script.py'. I even like running scripts in this way: 'py -2.7 -m pip install whatever'. However, this only works when the script also can be started as package. Some packages allow this, some don't. Could not py.exe be extended so that it allows starting scripts in a somewhat similar way? 'py-script -2.7 myscript foo bar baz' ??? Which would execute the script myscript.exe, myscript.bat, myscript.py, myscript.cmd or whatever is in the Scripts directory on the Python 2.7 installation, without changing PATH presistently? Thanks, Thomas _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.c...
On 12 February 2015 at 08:05, Thomas Heller <theller@ctypes.org> wrote:
Maybe I'm more or less alone with the way I work, but I don't like python.exe on my PATH (and py.exe alloes me to do this). I start python scripts from the command line either with 'script.py' or 'py -3.4 script.py' or 'py -2.7 script.py'.
This is how I worked for a long time after py.exe became available. But then I discovered virtualenv, and started using that, and needing to use 2 completely different command sets for "when I'm in a virtualenv" vs "when I'm not" became completely unusable. If py.exe detected when the environment variable VIRTUAL_ENV was set, and used that virtualenv as the default Python rather than the "system" python it normally used, this would be perfect. I think I'll write a PEP for this. Do people think it's a reasonable idea? (On that note, is there any relationship between the py.exe shipped with Python and the standalone version Vinay used to maintain but which now appears to be hosted at https://bitbucket.org/pypa/pylauncher? The two seem to be different, for example the pypa version registers pyz/pyzw extensions). Paul
If venv's activate script sets it, I say go ahead and write it up. If it's just virtualenv, I'd rather not explicitly depend on it, especially since PEP 397's stated aim is file associations and not command line. I've been making changes to py.exe in hg.p.o, so I hope the standalone one is tracking. The msi for it as part of the official build can also standalone, so maybe we should merge the two? Top-posted from my Windows Phone ________________________________ From: Paul Moore<mailto:p.f.moore@gmail.com> Sent: 2/12/2015 6:48 To: Thomas Heller<mailto:theller@ctypes.org> Cc: Python Dev<mailto:python-dev@python.org> Subject: Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows On 12 February 2015 at 08:05, Thomas Heller <theller@ctypes.org> wrote:
Maybe I'm more or less alone with the way I work, but I don't like python.exe on my PATH (and py.exe alloes me to do this). I start python scripts from the command line either with 'script.py' or 'py -3.4 script.py' or 'py -2.7 script.py'.
This is how I worked for a long time after py.exe became available. But then I discovered virtualenv, and started using that, and needing to use 2 completely different command sets for "when I'm in a virtualenv" vs "when I'm not" became completely unusable. If py.exe detected when the environment variable VIRTUAL_ENV was set, and used that virtualenv as the default Python rather than the "system" python it normally used, this would be perfect. I think I'll write a PEP for this. Do people think it's a reasonable idea? (On that note, is there any relationship between the py.exe shipped with Python and the standalone version Vinay used to maintain but which now appears to be hosted at https://bitbucket.org/pypa/pylauncher? The two seem to be different, for example the pypa version registers pyz/pyzw extensions). Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.c...
On 12 February 2015 at 15:37, Steve Dower <Steve.Dower@microsoft.com> wrote:
If venv's activate script sets it, I say go ahead and write it up. If it's just virtualenv, I'd rather not explicitly depend on it, especially since PEP 397's stated aim is file associations and not command line.
Yep, venv uses it too (see https://hg.python.org/cpython/file/9e10c4255277/Lib/venv/scripts/nt/activate...). As people like Thomas (and me, until this issue stopped me :-)) are using the launcher for command line use, I think it's fair to broaden the scope to make command line usage more convenient. I agree that PEP 397 was originally focused mainly on file associations, but I think it's worth looking beyond that now.
I've been making changes to py.exe in hg.p.o, so I hope the standalone one is tracking. The msi for it as part of the official build can also standalone, so maybe we should merge the two?
Hmm, sadly I don't think it is. Originally the standalone one was (I believe) provided by Vinay for people using Pythons that didn't have it bundled, and to add features (such as the new file extensions) on a quicker timescale than Python releases. But since he passed it to the pypa umbrella I don't think he's been keeping the two in sync. I've copied him in case I'm completely wrong on this. Personally if we now have a standalone launcher MSI, I'd like to discontinue the external one completely, and officially publish the standalone launcher MSI on python.org as a service for users of older Pythons. There seems little reason to maintain 2 repos if we don't have to. We could merge in changes from the external repo before discontinuing it, but I'm not sure how controversial that would be (for example, would it need a PEP to include the 2 new extensions?) Paul
Paul Moore wrote:
If venv's activate script sets it, I say go ahead and write it up. If it's just virtualenv, I'd rather not explicitly depend on it, especially since PEP 397's stated aim is file associations and not command
On 12 February 2015 at 15:37, Steve Dower <Steve.Dower@microsoft.com> wrote: line.
Yep, venv uses it too (see https://hg.python.org/cpython/file/9e10c4255277/Lib/venv/scripts/nt/activate...).
As people like Thomas (and me, until this issue stopped me :-)) are using the launcher for command line use, I think it's fair to broaden the scope to make command line usage more convenient. I agree that PEP 397 was originally focused mainly on file associations, but I think it's worth looking beyond that now.
That's roughly the premise behind my activate-py proposal. Use `activate` to make a virtualenv/venv the default, and `activate-py [-x[.y[-32]]]` to make a non-venv the default. I have proof of concept implementations already for cmd.exe and PowerShell and it's fairly simple, but writing up the rationale is time consuming (and also the main point - it's worth having all the problems with PATH documented in one place).
I've been making changes to py.exe in hg.p.o, so I hope the standalone one is tracking. The msi for it as part of the official build can also standalone, so maybe we should merge the two?
Hmm, sadly I don't think it is. Originally the standalone one was (I believe) provided by Vinay for people using Pythons that didn't have it bundled, and to add features (such as the new file extensions) on a quicker timescale than Python releases. But since he passed it to the pypa umbrella I don't think he's been keeping the two in sync. I've copied him in case I'm completely wrong on this.
Personally if we now have a standalone launcher MSI, I'd like to discontinue the external one completely, and officially publish the standalone launcher MSI on python.org as a service for users of older Pythons. There seems little reason to maintain 2 repos if we don't have to. We could merge in changes from the external repo before discontinuing it, but I'm not sure how controversial that would be (for example, would it need a PEP to include the 2 new extensions?)
None of my installer changes so far have had a PEP, and only a few people have complained about that :) (it does have more documentation than I've ever written for an installer before though) IIRC, there was a PEP for executing ZIP files directly (2.6-era?), which I believe are the purpose of those extensions. If "py.exe spam.pyz" already works, I don't see any need for a PEP to add the association in the installer. Cheers, Steve
Paul
On 12 February 2015 at 16:42, Steve Dower <Steve.Dower@microsoft.com> wrote:
None of my installer changes so far have had a PEP, and only a few people have complained about that :) (it does have more documentation than I've ever written for an installer before though)
:-) You shouldn't bet on my judgement of what needs a PEP, I usually get it wrong...
IIRC, there was a PEP for executing ZIP files directly (2.6-era?), which I believe are the purpose of those extensions. If "py.exe spam.pyz" already works, I don't see any need for a PEP to add the association in the installer.
Yes, "py spam.pyz" works fine. +1 on having the associations. Actually, I've just remembered, it's already in PEP 441, which hasn't been approved but which goes further and includes a stdlib tool to create pyz files. I'm not sure if that changes things at all... Paul
Actually, I've just remembered, it's already in PEP 441, which hasn't been approved but which goes further and includes a stdlib tool to create pyz files. I'm not sure if that changes things at all...
Actually, I developed the pyzzer tool (which Paul may remember - he made some helpful suggestions to improve it) which does a similar job to PEP 441's pyzaa and provides improved Windows support (some of which might be supplanted by py.exe support for .pyz/.pyzw, but I have cases where I can't rely on py.exe being present). Regards, Vinay Sajip
On 13 February 2015 at 05:50, Paul Moore <p.f.moore@gmail.com> wrote:
On 12 February 2015 at 16:42, Steve Dower <Steve.Dower@microsoft.com> wrote:
IIRC, there was a PEP for executing ZIP files directly (2.6-era?), which I believe are the purpose of those extensions. If "py.exe spam.pyz" already works, I don't see any need for a PEP to add the association in the installer.
Yes, "py spam.pyz" works fine. +1 on having the associations. Actually, I've just remembered, it's already in PEP 441, which hasn't been approved but which goes further and includes a stdlib tool to create pyz files. I'm not sure if that changes things at all...
Zip archive & directory execution support was added in 2.6 by way of a tracker issue rather than a full PEP, but we didn't realise we left it out of the What's New docs until a few years later. That meant a lot of folks missed the addition of the feature. PEP 441 was aimed at giving the feature more visibility, in addition to making appropriately designed archives easier to create (IIRC, my main request for that PEP was to change the proposed module name to the more prosaic, but also more self-explanatory, "pyzapp", although it looks like I may have suggested that to Daniel offline rather than through the mailing list). Regardless, the PEP's not controversial as far as I am aware, so it would be nice if someone had the time to get it cleaned up and formally proposed for 3.5 (assuming Daniel was OK with that). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 13 February 2015 at 07:25, Nick Coghlan <ncoghlan@gmail.com> wrote:
PEP 441 was aimed at giving the feature more visibility, in addition to making appropriately designed archives easier to create (IIRC, my main request for that PEP was to change the proposed module name to the more prosaic, but also more self-explanatory, "pyzapp", although it looks like I may have suggested that to Daniel offline rather than through the mailing list).
Regardless, the PEP's not controversial as far as I am aware, so it would be nice if someone had the time to get it cleaned up and formally proposed for 3.5 (assuming Daniel was OK with that).
I'd be willing to give that a go. The PEPs URL is giving me a "Bad Gateway" error at the moment, but I think the only coding needed in there is writing the "pyzaa/pyzapp" module. (If Steve would be willing to add the file associations - I'm not sure I want to dive into the new installer technology along with all the *other* things I'm volunteering for right now :-)) Paul
----- Original Message ----- From: Steve Dower <Steve.Dower@microsoft.com>
I believe are the purpose of those extensions. If "py.exe spam.pyz"
already works, I don't see any need for a PEP to add the association in the installer.
That would be my view, too. Regards, Vinay Sajip
I've been making changes to py.exe in hg.p.o, so I hope the standalone one is tracking. The msi for it as part of the official build can also standalone, so maybe we should merge the two?
Hmm, sadly I don't think it is. Originally the standalone one was (I believe) provided by Vinay for people using Pythons that didn't have it bundled, and to add features (such as the new file extensions) on a quicker timescale than Python releases. But since he passed it to the pypa umbrella I don't think he's been keeping the two in sync. I've copied him in case I'm completely wrong on this.
Thanks for copying me in - I haven't been tracking changes to the launcher in hg.p.o purely because I have been really busy with consultancy work for the past few months, leading to reduced time available for open-source contributions. I intend to try and keep things synced up where feasible and when time permits, and your email has given me the heads-up to look and see what's changed in the hg.p.o version.
Personally if we now have a standalone launcher MSI, I'd like to discontinue the external one completely, and officially publish the standalone launcher MSI on python.org as a service for users of older
Pythons.
The standalone repo/installer is just there for older Pythons where it's not bundled (and also to allow trying out features independently of the Python release cycle, as you mentioned). If python.org will offer an unbundled MSI just to install the launcher on Pythons < 3.3, then I suppose my installer could be retired. I don't know if the hg.p.o. installer would be equivalent to the standalone one (e.g. when the standalone launcher is uninstalled, the user gets a chance to associate file extensions with one of the installed Pythons found on the system). Off to read the whole thread on python-dev now ... Regards, Vinay Sajip
----- Original Message ----- From: Paul Moore <p.f.moore@gmail.com>
I'd also appreciate your views on the spin-off thread and PEP 486 ("Make the Python Launcher aware of virtual environments").
On an initial reading it seems quite reasonable. I'll sleep on it and see if any other things spring to mind. By the way, although I turned over the standalone launcher to pypa, I forked that repo, and do development on my fork [1]. I remembered that I added a feature (with the help of Pawel Jasinski) to allow e.g. py -ipy to pick a command 'ipy' configured in the .ini file (this is used to launch IronPython, but it could be used for other things too). In addition, my version has some bugfixes (e.g. use of iswspace instead of isspace, which is wrong). Steve's changes seem to just be related to adding IP_VERSION - I'm not sure what that's about and I haven't looked into it yet; I just skimmed the differences, and didn't see anything major in launcher.c (I compared with the version in the default branch of cpython). Regards, Vinay Sajip [1] https://bitbucket.org/vinay.sajip/pylauncher
Vinay Sajip wrote:
By the way, although I turned over the standalone launcher to pypa, I forked that repo, and do development on my fork [1]. I remembered that I added a feature (with the help of Pawel Jasinski) to allow e.g. py -ipy to pick a command 'ipy' configured in the .ini file (this is used to launch IronPython, but it could be used for other things too). In addition, my version has some bugfixes (e.g. use of iswspace instead of isspace, which is wrong). Steve's changes seem to just be related to adding IP_VERSION - I'm not sure what that's about and I haven't looked into it yet; I just skimmed the differences, and didn't see anything major in launcher.c (I compared with the version in the default branch of cpython).
I thought I did more than that, but maybe that was attempting to make things work :) The default branch is the most up to date. The big change is that the registry key for 32-bit Python now includes the "-32" suffix, so that you can install 32-bit and 64-bit versions per-user at the same time and we don't have a collision. The launcher needed some updates to handle that, since it was assuming that registry key names would never be more than three characters long. Probably everything else I did was elsewhere in Python or in the installer. (Btw, it sounds like your installer will work better as a standalone tool, though my MSI has the advantage that it won't collide with a Python 3.5+ installation. I'm not fussed about whose 'wins' though.) Cheers, Steve
Regards,
Vinay Sajip
On 13 February 2015 at 00:15, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
I remembered that I added a feature (with the help of Pawel Jasinski) to allow e.g. py -ipy to pick a command 'ipy' configured in the .ini file (this is used to launch IronPython, but it could be used for other things too).
At the time the launcher was first released I *really* wanted that feature, but couldn't figure out how to cleanly add it. I'd love to see that go into the main repo :-) Paul
----- Original Message ----- From: Paul Moore <p.f.moore@gmail.com>
I'd also appreciate your views on the spin-off thread and PEP 486 ("Make the Python Launcher aware of virtual environments").
I just thought of something ... as far as I know, we've enabled searching for a command on the path - since when a venv is active in the sense of VIRTUAL_ENV being defined, PATH is also set up so that the venv's Scripts folder is on it - so we would just need to find "python.exe" on the PATH and invoke that - no need to even use the value of VIRTUAL_ENV, making implementation simpler. What do you think? Regards, Vinay
On 13 February 2015 at 09:47, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
I just thought of something ... as far as I know, we've enabled searching for a command on the path - since when a venv is active in the sense of VIRTUAL_ENV being defined, PATH is also set up so that the venv's Scripts folder is on it - so we would just need to find "python.exe" on the PATH and invoke that - no need to even use the value of VIRTUAL_ENV, making implementation simpler. What do you think?
Two comments: 1. I'm not sure (but I've not dug into the implementation yet) that there's much practical difference between locating python.exe via a PATH search, or locating it via concatenating VIRTUAL_ENV and the relative path. 2. It would be a nice, although extremely obscure, feature to be able to allow people who want to keep Python off their path to set VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing but there's *still* no python.exe on PATH. Limit the dependency on a single environment variable rather than two, in other words. Paul
On 02/13/2015 01:57 AM, Paul Moore wrote:
2. It would be a nice, although extremely obscure, feature to be able to allow people who want to keep Python off their path to set VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing but there's *still* no python.exe on PATH. Limit the dependency on a single environment variable rather than two, in other words.
+1 -- ~Ethan~
Paul Moore <p.f.moore <at> gmail.com> writes:
2. It would be a nice, although extremely obscure, feature to be able to allow people who want to keep Python off their path to set VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing but there's *still* no python.exe on PATH. Limit the dependency on a single environment variable rather than two, in other words.
If this is the case, then your patch may not work if PATH isn't set. If a shebang is found in a script, it is processed in maybe_handle_shebang(), which means that your code to look in a venv won't be hit (though if you rely on the PATH being correctly set, then it will work as expected because the handling for e.g. /usr/bin/env python searches the PATH). Regards, Vinay Sajip
On 15 February 2015 at 19:23, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Paul Moore <p.f.moore <at> gmail.com> writes:
2. It would be a nice, although extremely obscure, feature to be able to allow people who want to keep Python off their path to set VIRTUAL_ENV but *not* set PATH, so that py.exe does the right thing but there's *still* no python.exe on PATH. Limit the dependency on a single environment variable rather than two, in other words.
If this is the case, then your patch may not work if PATH isn't set. If a shebang is found in a script, it is processed in maybe_handle_shebang(), which means that your code to look in a venv won't be hit (though if you rely on the PATH being correctly set, then it will work as expected because the handling for e.g. /usr/bin/env python searches the PATH).
Yes, when I was actually coding this, I backed down on this (I updated the PEP accordingly). Now, running py.exe with no script will pick up VIRTUAL_ENV, but when you run a script, you need the virtualenv to be activated for this to work. The original PEP said nothing about a PATH search for #!/usr/bin/env python, so I mistakenly assumed there was no way currently to write a script that worked with an activated virtualenv. When I found that the code did this, I decided it was unnecessary to add extra tests and complexity just to cater for a case that nobody was ever likely to need. Paul
----- Original Message ----- From: Paul Moore <p.f.moore@gmail.com>
The original PEP said nothing about a PATH search for #!/usr/bin/env python, so I mistakenly assumed there was no way currently to write a script that worked with an activated virtualenv.
Actually the path search was added later, by you! See [1] and [2]. Regards, Vinay Sajip [1] http://bugs.python.org/issue17903 [2] https://bitbucket.org/pmoore/pylauncher/commits/87d41c5a8f0fef5ef22f09570903...
On 15 February 2015 at 21:00, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
The original PEP said nothing about a PATH search for #!/usr/bin/env python, so I mistakenly assumed there was no way currently to write a script that worked with an activated virtualenv.
Actually the path search was added later, by you! See [1] and [2].
Coo, that was clever of me :-) I guess I'll have to admit to going senile now... Paul
Am 12.02.2015 um 15:46 schrieb Paul Moore:
On 12 February 2015 at 08:05, Thomas Heller <theller@ctypes.org> wrote:
Maybe I'm more or less alone with the way I work, but I don't like python.exe on my PATH (and py.exe alloes me to do this). I start python scripts from the command line either with 'script.py' or 'py -3.4 script.py' or 'py -2.7 script.py'.
This is how I worked for a long time after py.exe became available. But then I discovered virtualenv, and started using that, and needing to use 2 completely different command sets for "when I'm in a virtualenv" vs "when I'm not" became completely unusable.
I'm normally not using virtual environments, so my understanding might be wrong. Isn't it the case that in virtual envs you don't have these problems at all? The 'correct' python.exe is on the PATH as well as the Scripts directory: this is what activate.bat does. Thomas
On 12 February 2015 at 17:13, Thomas Heller <theller@ctypes.org> wrote:
This is how I worked for a long time after py.exe became available. But then I discovered virtualenv, and started using that, and needing to use 2 completely different command sets for "when I'm in a virtualenv" vs "when I'm not" became completely unusable.
I'm normally not using virtual environments, so my understanding might be wrong. Isn't it the case that in virtual envs you don't have these problems at all? The 'correct' python.exe is on the PATH as well as the Scripts directory: this is what activate.bat does.
Yes, but that's essentially the problem. I don't want to use "py" to start Python, but when I'm in a virtualenv, have to switch to using "python". That was what I used to do, and I found that I was forever running "py" in a virtualenv and wondering why the packages I'd installed weren't available... Paul
On 02/12/2015 12:05 AM, Thomas Heller wrote:
Could not py.exe be extended so that it allows starting scripts in a somewhat similar way? 'py-script -2.7 myscript foo bar baz' ??? Which would execute the script myscript.exe, myscript.bat, myscript.py, myscript.cmd or whatever is in the Scripts directory on the Python 2.7 installation, without changing PATH presistently?
I'm pretty sure py.exe should only start .py files, not .bat, .cmd, or .anything-else. -- ~Ethan~
On 2/12/2015 9:39 AM, Ethan Furman wrote:
On 02/12/2015 12:05 AM, Thomas Heller wrote:
Could not py.exe be extended so that it allows starting scripts in a somewhat similar way? 'py-script -2.7 myscript foo bar baz' ??? Which would execute the script myscript.exe, myscript.bat, myscript.py, myscript.cmd or whatever is in the Scripts directory on the Python 2.7 installation, without changing PATH presistently? I'm pretty sure py.exe should only start .py files, not .bat, .cmd, or .anything-else.
IIRC, it already can be configured to start other things. I think I even used it for that early on in Vinay's development cycle, just to prove it, although the things I used it for then were not things I'm still doing.
On 02/12/2015 11:44 AM, Glenn Linderman wrote:
On 2/12/2015 9:39 AM, Ethan Furman wrote:
On 02/12/2015 12:05 AM, Thomas Heller wrote:
Could not py.exe be extended so that it allows starting scripts in a somewhat similar way? 'py-script -2.7 myscript foo bar baz' ??? Which would execute the script myscript.exe, myscript.bat, myscript.py, myscript.cmd or whatever is in the Scripts directory on the Python 2.7 installation, without changing PATH presistently? I'm pretty sure py.exe should only start .py files, not .bat, .cmd, or .anything-else.
IIRC, it already can be configured to start other things. I think I even used it for that early on in Vinay's development cycle, just to prove it, although the things I used it for then were not things I'm still doing.
I believe you are correct; however, as the PEP for the launcher stated [1] "use as a general-purpose (aka non-python) launcher is explicitly not supported". -- ~Ethan~ [1] or something to that effect -- basically, it may work, but you're on your own if you use it that way.
On 12 February 2015 at 20:11, Ethan Furman <ethan@stoneleaf.us> wrote:
I believe you are correct; however, as the PEP for the launcher stated [1] "use as a general-purpose (aka non-python) launcher is explicitly not supported".
Yes, I once used it to start Perl scripts, just because it appealed to the perverse instinct in me. Part of me wishes there had been a problem, so that I could be the one who raised a legitimate bug on bugs.python.org saying "Python encounters an error when running a Perl script" :-) Paul
On 12/02/2015 20:23, Paul Moore wrote:
On 12 February 2015 at 20:11, Ethan Furman <ethan@stoneleaf.us> wrote:
I believe you are correct; however, as the PEP for the launcher stated [1] "use as a general-purpose (aka non-python) launcher is explicitly not supported".
Yes, I once used it to start Perl scripts, just because it appealed to the perverse instinct in me. Part of me wishes there had been a problem, so that I could be the one who raised a legitimate bug on bugs.python.org saying "Python encounters an error when running a Perl script" :-)
Paul
Get thee behind me, Satan. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
Am 12.02.2015 um 18:39 schrieb Ethan Furman:
On 02/12/2015 12:05 AM, Thomas Heller wrote:
Could not py.exe be extended so that it allows starting scripts in a somewhat similar way? 'py-script -2.7 myscript foo bar baz' ??? Which would execute the script myscript.exe, myscript.bat, myscript.py, myscript.cmd or whatever is in the Scripts directory on the Python 2.7 installation, without changing PATH presistently?
I'm pretty sure py.exe should only start .py files, not .bat, .cmd, or .anything-else.
To make it clear: My suggestion is (or was, maybe it isn't a good idea) to have some way to start 'something' that is in the Scripts directory of a Python installation (which is usually a python script or an exe-wrapper for it), without typing in the full path. And without changing the PATH envrionment variable. Thomas
On 13 February 2015 at 06:59, Thomas Heller <theller@ctypes.org> wrote:
To make it clear: My suggestion is (or was, maybe it isn't a good idea) to have some way to start 'something' that is in the Scripts directory of a Python installation (which is usually a python script or an exe-wrapper for it), without typing in the full path. And without changing the PATH envrionment variable.
People using the old-style "scripts" keyword to setup() can put pretty much anything into the Scripts directory - I've seen .py files, Python files with no suffix, bat files, even Unix shell files (not much use on Windows...) in there. I don't think it's reasonable for the launcher to try to run all of those things. Maybe restricting it to running ".py" files or ".exe" files would be reasonable. That covers entry points (which should be the norm for newer projects) and Python scripts (which are the most likely things to be portable). Paul
On 2/13/2015 1:13 AM, Paul Moore wrote:
On 13 February 2015 at 06:59, Thomas Heller <theller@ctypes.org> wrote:
To make it clear: My suggestion is (or was, maybe it isn't a good idea) to have some way to start 'something' that is in the Scripts directory of a Python installation (which is usually a python script or an exe-wrapper for it), without typing in the full path. And without changing the PATH envrionment variable. People using the old-style "scripts" keyword to setup() can put pretty much anything into the Scripts directory - I've seen .py files, Python files with no suffix, bat files, even Unix shell files (not much use on Windows...) in there. I don't think it's reasonable for the launcher to try to run all of those things.
Maybe restricting it to running ".py" files or ".exe" files would be reasonable. That covers entry points (which should be the norm for newer projects) and Python scripts (which are the most likely things to be portable). The WINDOWS py launcher hardly needs to be portable. Batch/CMD files also seem useful on WINDOWS. And Powershell?
If a launcher is developed for or enhanced for other systems, shell scripts may be useful, and Batch/CMD not.
On 13 February 2015 at 10:15, Glenn Linderman <v+python@g.nevcal.com> wrote:
Maybe restricting it to running ".py" files or ".exe" files would be reasonable. That covers entry points (which should be the norm for newer projects) and Python scripts (which are the most likely things to be portable).
The WINDOWS py launcher hardly needs to be portable. Batch/CMD files also seem useful on WINDOWS. And Powershell?
If a launcher is developed for or enhanced for other systems, shell scripts may be useful, and Batch/CMD not.
By "portable" in this sense I meant "not written by a Unix developer who hadn't thought about Windows much". I was comparing Python files with a .py extension and extensionless Python files, specifically (my wording didn't make that clear, sorry). Yes, files with extensions of .bat or .cmd or .ps1 are probably written for Windows only. But I think it's getting out of scope of the launcher to decide to use cmd.exe to launch .bat and .cmd files, and powershell to launch .ps1 files. Regardless, my real point was that there can be all sorts of stuff in the Scripts directory, and I'm not convinced that this is a direction the launcher should be going in. Someone *could* make a case for launching certain types of file that might be in there, but I'm not going to be that person. One thought - it's not hard using pkg_resources to enumerate all available console_script entry points, pick one and run it. So you could pretty simply write a script (that itself could be run with the launcher) to run anything in Scripts that's built via entry points (which is the recommended way these days). # launch.py import sys import pkg_resources # Get the script name script = sys.argv[1] # Remove our name from sys.argv del sys.argv[0] # There's probably a better API you can use for ep in pkg_resources.working_set.iter_entry_points('console_scripts'): if ep.name == script: fn = ep.load() sys.exit(fn()) # Couldn't find the entry point print("No entry point called {} available".format(script), file=sys.stderr) sys.exit(1) Now you can do "py launch.py pip --help" and it will run pip for you. If a project isn't exposing its scripts as entry points, ask them to :-) Paul Paul
Launching non-Python scripts from py.exe is not going to work as well as you may hope. There will be numerous subtle bugs due to the cmd->py.exe->cmd wrapping (Powershell users would have seen this with some console apps or batch files). I think having a global command to add a particular Python to PATH for the current session will work best here, and yes, I'm working on it. If py.exe starts defaulting to whatever is on PATH then I don't see the point of it. Knowing that you'll get the latest 2.x version by default is quite handy (I'd be quite okay changing that to 3.x, though it is specified in the original PEP). For me, the point of py.exe is to be able to ignore PATH completely. Adding more commands to the launcher args (eg. -ipy) will make the parsing far more complex, especially when deciding which args to consume and which to forward. I think you can make a copy of the launcher called ipy.exe and add ipy.ini with the path to IronPython in it as default. Cheers, Steve Top-posted from my Windows Phone ________________________________ From: Paul Moore<mailto:p.f.moore@gmail.com> Sent: 2/13/2015 3:20 To: Glenn Linderman<mailto:v+python@g.nevcal.com> Cc: Python Dev<mailto:python-dev@python.org> Subject: Re: [Python-Dev] PEP 370 - per-user scripts directory on Windows On 13 February 2015 at 10:15, Glenn Linderman <v+python@g.nevcal.com> wrote:
Maybe restricting it to running ".py" files or ".exe" files would be reasonable. That covers entry points (which should be the norm for newer projects) and Python scripts (which are the most likely things to be portable).
The WINDOWS py launcher hardly needs to be portable. Batch/CMD files also seem useful on WINDOWS. And Powershell?
If a launcher is developed for or enhanced for other systems, shell scripts may be useful, and Batch/CMD not.
By "portable" in this sense I meant "not written by a Unix developer who hadn't thought about Windows much". I was comparing Python files with a .py extension and extensionless Python files, specifically (my wording didn't make that clear, sorry). Yes, files with extensions of .bat or .cmd or .ps1 are probably written for Windows only. But I think it's getting out of scope of the launcher to decide to use cmd.exe to launch .bat and .cmd files, and powershell to launch .ps1 files. Regardless, my real point was that there can be all sorts of stuff in the Scripts directory, and I'm not convinced that this is a direction the launcher should be going in. Someone *could* make a case for launching certain types of file that might be in there, but I'm not going to be that person. One thought - it's not hard using pkg_resources to enumerate all available console_script entry points, pick one and run it. So you could pretty simply write a script (that itself could be run with the launcher) to run anything in Scripts that's built via entry points (which is the recommended way these days). # launch.py import sys import pkg_resources # Get the script name script = sys.argv[1] # Remove our name from sys.argv del sys.argv[0] # There's probably a better API you can use for ep in pkg_resources.working_set.iter_entry_points('console_scripts'): if ep.name == script: fn = ep.load() sys.exit(fn()) # Couldn't find the entry point print("No entry point called {} available".format(script), file=sys.stderr) sys.exit(1) Now you can do "py launch.py pip --help" and it will run pip for you. If a project isn't exposing its scripts as entry points, ask them to :-) Paul Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.c...
On 13 February 2015 at 14:27, Steve Dower <Steve.Dower@microsoft.com> wrote:
If py.exe starts defaulting to whatever is on PATH then I don't see the point of it. Knowing that you'll get the latest 2.x version by default is quite handy (I'd be quite okay changing that to 3.x, though it is specified in the original PEP). For me, the point of py.exe is to be able to ignore PATH completely.
It does seem a bit bizarre to me that a launcher shipped with Python 3 defaults to using Python 2 if it's available. I'd like to see that change, but as it's specified in the original PEP I guess it might meet with some pushback :-(
Adding more commands to the launcher args (eg. -ipy) will make the parsing far more complex, especially when deciding which args to consume and which to forward. I think you can make a copy of the launcher called ipy.exe and add ipy.ini with the path to IronPython in it as default.
Well, Vinay has this functionality already available in his fork, so it's not impossible. But whether or not it's added I do think that all distributed versions of py.exe should have the same functionality, so either the CPython version should add it or Vinay should remove it (or rename his version and describe it as an *enhanced* version of the py.exe launcher, rather than as a *standalone* version...) I'm +0 on the functionality personally - I quite like the idea, but I've no real need for it and I've lived without it long enough now. Paul
On 14 Feb 2015 01:47, "Paul Moore" <p.f.moore@gmail.com> wrote:
On 13 February 2015 at 14:27, Steve Dower <Steve.Dower@microsoft.com>
wrote:
If py.exe starts defaulting to whatever is on PATH then I don't see the point of it. Knowing that you'll get the latest 2.x version by default is quite handy (I'd be quite okay changing that to 3.x, though it is specified in the original PEP). For me, the point of py.exe is to be able to ignore PATH completely.
It does seem a bit bizarre to me that a launcher shipped with Python 3 defaults to using Python 2 if it's available. I'd like to see that change, but as it's specified in the original PEP I guess it might meet with some pushback :-(
If there's no Python 2 on the system it doesn't matter, and if they're both there, we opted to remain consistent with the Linux distro world. OTOH, it may be time to reconsider our recommendation to distros as well, suggesting that for Python 3.5+, we will consider it appropriate to have the "python" symlink refer to "python3". Cheers, Nick.
On Sat, Feb 14, 2015 at 11:07 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
OTOH, it may be time to reconsider our recommendation to distros as well, suggesting that for Python 3.5+, we will consider it appropriate to have the "python" symlink refer to "python3".
If *all* distros provide a "python2" symlink, then the recommendation can become "use python if it's 2/3 compatible, or python2/python3 to choose", and then it won't hurt to switch python to be python3. Are there known distros in which current versions (or, those that will be current when 3.5 is released) aren't providing "python2"? ChrisA
On Mon, Feb 16, 2015 at 12:00 AM, Chris Angelico <rosuav@gmail.com> wrote:
On Sat, Feb 14, 2015 at 11:07 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
OTOH, it may be time to reconsider our recommendation to distros as well, suggesting that for Python 3.5+, we will consider it appropriate to have the "python" symlink refer to "python3".
If *all* distros provide a "python2" symlink, then the recommendation can become "use python if it's 2/3 compatible, or python2/python3 to choose", and then it won't hurt to switch python to be python3. Are there known distros in which current versions (or, those that will be current when 3.5 is released) aren't providing "python2"?
That wouldn't be a good recommendation for the long term. Fedora probably won't drop python2 from the default installation before 3.5 is released, but hopes are high that it'll happen when 3.5 is still current. Not sure about others.
On Mon, Feb 16, 2015 at 10:47 PM, Petr Viktorin <encukou@gmail.com> wrote:
If *all* distros provide a "python2" symlink, then the recommendation can become "use python if it's 2/3 compatible, or python2/python3 to choose", and then it won't hurt to switch python to be python3. Are there known distros in which current versions (or, those that will be current when 3.5 is released) aren't providing "python2"?
That wouldn't be a good recommendation for the long term. Fedora probably won't drop python2 from the default installation before 3.5 is released, but hopes are high that it'll happen when 3.5 is still current. Not sure about others.
Clarification: If all distros *that provide Python 2.x* provide a "python2" symlink, etc, etc. ChrisA
Am 10.02.15 um 18:45 schrieb Steve Dower:
So I've enumerated the problems with PATH on Windows before (and my subsequent dislike for modifying it through the installer)
It's quite comforting to hear this - I was arguing against that addition for years (to the point of refusing to implement it). So you are certainly not alone here, either. Regards, Martin
On 10 February 2015 at 12:38, Paul Moore <p.f.moore@gmail.com> wrote:
Comments? If this is acceptable, I would be willing to prepare a patch for Python 3.5 implementing this.
See http://bugs.python.org/issue23437 As yet untested, as I just realised I need to get Visual Studio 2015 installed to be able to build Python 3.5. I'll try to get that sorted out, but I thought it would be worth putting the patch up anyway - it's pretty simple. Paul
On 10/02/2015 20:47, Paul Moore wrote:
On 10 February 2015 at 12:38, Paul Moore <p.f.moore@gmail.com> wrote:
Comments? If this is acceptable, I would be willing to prepare a patch for Python 3.5 implementing this.
See http://bugs.python.org/issue23437
As yet untested, as I just realised I need to get Visual Studio 2015 installed to be able to build Python 3.5. I'll try to get that sorted out, but I thought it would be worth putting the patch up anyway - it's pretty simple.
Paul
Visual Studio 2013 is fine for building 3.5. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
participants (11)
-
"Martin v. Löwis" -
Chris Angelico -
Ethan Furman -
Glenn Linderman -
Mark Lawrence -
Nick Coghlan -
Paul Moore -
Petr Viktorin -
Steve Dower -
Thomas Heller -
Vinay Sajip