Hi, These are two emails I sent a short while ago about finalizing PEP 394. There was no response, so in case the messages didn't go through, I'm resending them.
Thanks, Kerrick Staley
---------- Forwarded message ---------- From: Kerrick Staley mail@kerrickstaley.com Date: Sat, Jul 9, 2011 at 7:45 PM Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream To: python-dev@python.org
Sorry that I dropped the ball on this. I'd still like to see this get implemented, but I got distracted with school and forgot about it. Updates I have made to the PEP will be sent as a patch immediately after this email.
Here's a summary of what was happenening when we left off: * The draft SVN version from March 4 was pretty much complete.
* Some were concerned about addressing Windows, but it was generally agreed to leave the Windows issue to another PEP. PEP 397 was started on March 15 to address the Windows side of the issue. PEP 397 recommends that the Windows Python launcher read the shebang and use it to determine which Python version to use; this allows one syntax for both operating systems that is compatible with the current PEP 394 recommendation.
* There were concerns from Ned Deily about the naming of other binaries such as idle, pydoc, and python-config; these need to be created as idle2, pydoc2, and python2-config, with links created at the locations of the original binaries.
* There were concerns from Glenn Linderman that the shebang line doesn't encode enough information to flexibly handle Windows launching (or even launching in general).
==== Here are my thoughts: * For Ned's comments, I agree. Although the issue isn't as large with these programs, there's no reason we can't handle them in the same way. I updated the PEP.
* For Glenn's comments, I think the method you propose adds too much complexity. Regardless, if the #@ syntax is implemented, it can be described in PEP 397; it won't have an impact on the contents of this PEP. I think, though, that having multiple syntaxes may cause many scripts to be incompatible with multiple platforms when they don't have to be, since Unix coders will rarely add a #@ line, and Windows coders will likely forget the #! line.
Also, #@ really ignores the purpose of a shebang: shebangs simply indicate an interpreter that works with the script; the shebang allows users to run arbitrary scripts without worrying about which interpreter they should specify. There's no reason that a script should use one interpreter on Unix, but be incompatible with that interpreter on Windows yet compatible with another. The name of the Unix binary is enough to determine the implementation and version of the interpreter to be used on Unix, and a Windows launcher should always invoke the same implementation/version on Windows (and this won't require hard-coding anything into the launcher if done right). If you want the script to run with a specific interpreter and version, possibly contingent on which operating system you're running the script under, then you can just invoke the interpreter by name with the script as an argument (e.g. python3 myprogram.py).
TL;DR: shebangs encode a default implementation/version, and if you need something special, you can just manually run python3 myprogram.py or use a .bat file.
==== Also, I updated the PEP with the clarification that commands like python3 should be hard links (because they'll be invoked from code and are more efficient; also, hard links are just as flexible as symlinks here), while commands like python should be soft links (because this makes it clear to sysadmins that they can be "switched", and it's needed for flexibility if python3 changes). This really doesn't matter, but can we keep it this way unless there are serious objections?
Regards, Kerrick Staley
---------- Forwarded message ---------- From: Kerrick Staley mail@kerrickstaley.com Date: Sat, Jul 9, 2011 at 7:46 PM Subject: Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream To: python-dev@python.org
$ svn diff Index: pep-0394.txt =================================================================== --- pep-0394.txt (revision 88860) +++ pep-0394.txt (working copy) @@ -1,5 +1,5 @@ PEP: 394 -Title: The "python" command on Unix-Like Systems +Title: The "python" Command on Unix-Like Systems Version: $Revision$ Last-Modified: $Date$ Author: Kerrick Staley mail@kerrickstaley.com, @@ -53,10 +53,14 @@ * When reinvoking the interpreter from a Python script, querying ``sys.executable`` to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach. +* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0 +should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, +with the original commands invoking these binaries by default, but possibly +invoking the Python 3.0 versions instead.
These recommendations are the outcome of the relevant python-dev discussion in -March 2011 [1] (NOTE: More accurately, they will be such once that "Draft" -status disappears from the PEP header, it has been moved into the "Other +March to July 2011 [1] (NOTE: More accurately, they will be such once that +"Draft" status disappears from the PEP header, it has been moved into the "Other Informational PEP" section in PEP 0 and this note has been deleted)
@@ -144,11 +148,16 @@
While technically a new feature, the ``make install`` command and the Mac OS X installer in the 2.7 version of CPython will be adjusted to create the -new ``python2`` command in addition to the existing ``python`` and -``python2.7`` commands. This feature will first appear in CPython 2.7.2. +``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries, +with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as hard links +to the respective binaries, and ``python``, ``idle``, ``pydoc``, and +``python-config`` as symbolic links to the respective hard links. This feature +will first appear in CPython 2.7.2.
-The ``make install`` command in the CPython 3.x series will continue to -install only the ``python3`` symlink for the foreseeable future. +The ``make install`` command in the CPython 3.x series will similarly install +the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries +(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and +``python3-config`` as hard links.
Impact on PYTHON* Environment Variables @@ -166,27 +175,12 @@ Exclusions of MS Windows ========================
-This PEP deliberately excludes any proposals relating to Microsoft Windows. -The use of parallel installs on Windows suffers from numerous issues, -including the "last installed wins" behaviour for handling of file -associations, a lack of universal robust symlink support for easy aliasing of -commands, the fact that the Python executable is not available on ``PATH`` by -default, the fact that the ``python.exe`` and ``pythonw.exe`` names are -used for both Python 2 and Python 3 binaries and the lack of distinction -between the different Python versions when the Start menu shortcuts are -divorced from their containing folders (e.g. when they appear in the -"Recently Used" list. +This PEP deliberately excludes any proposals relating to Microsoft Windows: +devising an equivalent solution for Windows was deemed too complex to handle +here. PEP 397 and the related discussion on the python-dev mailing list address +this issue.
-While these questions are well worth addressing, they do not have easy -answers. The authors of this particular PEP aren't inclined to even begin -trying to answer them, but anyone that wants to tackle them should feel free -to start working on their own PEP.
-Note that, while the topic has been excluded from this PEP, there is plenty of -material in the linked python-dev discussion that may be useful in the design -and implementation of a Windows-specific solution. - - References ==========
In article CANaWP3zBo8cNWNHN=jxx_m3tUBk3k+vn+LYgqB+yimdTrzVxwA@mail.gmail.com, Kerrick Staley mail@kerrickstaley.com wrote:
Here are my thoughts:
- For Ned's comments, I agree. Although the issue isn't as large with
these programs, there's no reason we can't handle them in the same way. I updated the PEP.
Thanks.
Also, I updated the PEP with the clarification that commands like python3 should be hard links (because they'll be invoked from code and are more efficient; also, hard links are just as flexible as symlinks here), while commands like python should be soft links (because this makes it clear to sysadmins that they can be "switched", and it's needed for flexibility if python3 changes). This really doesn't matter, but can we keep it this way unless there are serious objections?
I think adding the requirement to mandate hard link vs soft link usage is an unnecessary and unwarranted attempt at optimization. For instance, IIRC, the OS X installers don't use any hard links: that may complicate the install, plus hard links on OS X HFS* file systems are a bit of a kludge and not necessarily more efficient than symlinks. It's not a big deal but perhaps the wording should be changed to make a suggestion about hard links vs syminks rather than mandate which should be used.
On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily nad@acm.org wrote:
I think adding the requirement to mandate hard link vs soft link usage is an unnecessary and unwarranted attempt at optimization. For instance, IIRC, the OS X installers don't use any hard links: that may complicate the install, plus hard links on OS X HFS* file systems are a bit of a kludge and not necessarily more efficient than symlinks. It's not a big deal but perhaps the wording should be changed to make a suggestion about hard links vs syminks rather than mandate which should be used.
Ah, OK. The wording's been changed so that symbolic links will be installed on Mac OS X and hard links elsewhere (although maybe symbolic links are also better on certain other platforms; I'm not sure).
I do think that specific instructions must be given (rather than just a suggestion) because it's indicating what must be done to CPython. The instructions *should* be as close as possible to what the installer already does, but I'm not entirely sure what the installer does by default, and the hard-link recommendation was based off a cursory inspection of my own system, so further input from yourself and the rest of python-dev would be appreciated.
Nick, can you please apply the patch (will be sent in the following email) to the PEP SVN as soon as we get the hard-link issue is figured out? Alternatively, could you provide me write access to just the pep-0394.txt file so I can update it myself? Thanks.
-Kerrick Staley
$ svn diff Index: pep-0394.txt =================================================================== --- pep-0394.txt (revision 88866) +++ pep-0394.txt (working copy) @@ -1,5 +1,5 @@ PEP: 394 -Title: The "python" command on Unix-Like Systems +Title: The "python" Command on Unix-Like Systems Version: $Revision$ Last-Modified: $Date$ Author: Kerrick Staley mail@kerrickstaley.com, @@ -53,10 +53,14 @@ * When reinvoking the interpreter from a Python script, querying ``sys.executable`` to avoid hardcoded assumptions regarding the interpreter location remains the preferred approach. +* The ``idle``, ``pydoc``, and ``python-config`` binaries from Python 2.0 +should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, +with the original commands invoking these binaries by default, but possibly +invoking the Python 3.0 versions instead.
These recommendations are the outcome of the relevant python-dev discussion in -March 2011 [1] (NOTE: More accurately, they will be such once that "Draft" -status disappears from the PEP header, it has been moved into the "Other +March to July 2011 [1] (NOTE: More accurately, they will be such once that +"Draft" status disappears from the PEP header, it has been moved into the "Other Informational PEP" section in PEP 0 and this note has been deleted)
@@ -142,13 +146,21 @@ Application to the CPython Reference Interpreter ================================================
-While technically a new feature, the ``make install`` command and the Mac OS -X installer in the 2.7 version of CPython will be adjusted to create the -new ``python2`` command in addition to the existing ``python`` and -``python2.7`` commands. This feature will first appear in CPython 2.7.2. +While technically a new feature, the ``make install`` command and the Mac +OS X installer in the 2.7 version of CPython will be adjusted to create the +``python2.7``, ``idle2.7``, ``pydoc2.7``, and ``python2.7-config`` binaries, +with ``python2``, ``idle2``, ``pydoc2``, and ``python2-config`` as links to +the respective binaries. On Mac OS X, these will be symbolic links; on other +platforms, they will be hard links, because hard links improve performance +on filesystems other than Mac OS X's HFS. ``python``, ``idle``, ``pydoc``, +and ``python-config`` will be created as symbolic links to these links. +This feature will first appear in CPython 2.7.2.
-The ``make install`` command in the CPython 3.x series will continue to -install only the ``python3`` symlink for the foreseeable future. +The ``make install`` command in the CPython 3.x series will similarly install +the ``python3.x``, ``idle3.x``, ``pydoc3.x``, and ``python3.x-config`` binaries +(with appropriate ``x``), and ``python3``, ``idle3``, ``pydoc3``, and +``python3-config`` as symbolic links on Mac OS X and as hard links on other +platforms.
Impact on PYTHON* Environment Variables @@ -166,27 +178,12 @@ Exclusions of MS Windows ========================
-This PEP deliberately excludes any proposals relating to Microsoft Windows. -The use of parallel installs on Windows suffers from numerous issues, -including the "last installed wins" behaviour for handling of file -associations, a lack of universal robust symlink support for easy aliasing of -commands, the fact that the Python executable is not available on ``PATH`` by -default, the fact that the ``python.exe`` and ``pythonw.exe`` names are -used for both Python 2 and Python 3 binaries and the lack of distinction -between the different Python versions when the Start menu shortcuts are -divorced from their containing folders (e.g. when they appear in the -"Recently Used" list. +This PEP deliberately excludes any proposals relating to Microsoft Windows: +devising an equivalent solution for Windows was deemed too complex to handle +here. PEP 397 and the related discussion on the python-dev mailing list address +this issue.
-While these questions are well worth addressing, they do not have easy -answers. The authors of this particular PEP aren't inclined to even begin -trying to answer them, but anyone that wants to tackle them should feel free -to start working on their own PEP.
-Note that, while the topic has been excluded from this PEP, there is plenty of -material in the linked python-dev discussion that may be useful in the design -and implementation of a Windows-specific solution. - - References ==========
In article CANaWP3zfhpaAGdgnuHd4diFFja2qMNkF+7WKW7qBY_F2vbyqhw@mail.gmail.com, Kerrick Staley mail@kerrickstaley.com wrote:
On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily nad@acm.org wrote:
I think adding the requirement to mandate hard link vs soft link usage is an unnecessary and unwarranted attempt at optimization. For instance, IIRC, the OS X installers don't use any hard links: that may complicate the install, plus hard links on OS X HFS* file systems are a bit of a kludge and not necessarily more efficient than symlinks. It's not a big deal but perhaps the wording should be changed to make a suggestion about hard links vs syminks rather than mandate which should be used.
Ah, OK. The wording's been changed so that symbolic links will be installed on Mac OS X and hard links elsewhere (although maybe symbolic links are also better on certain other platforms; I'm not sure).
I do think that specific instructions must be given (rather than just a suggestion) because it's indicating what must be done to CPython. The instructions *should* be as close as possible to what the installer already does, but I'm not entirely sure what the installer does by default, and the hard-link recommendation was based off a cursory inspection of my own system, so further input from yourself and the rest of python-dev would be appreciated.
Thanks for addressing the comments. Unfortunately, I was relying too much on memory and I should have checked more carefully. I see now that, within the framework bin directory for Python 3, the python3 -> python3.x link is, in fact, a hard link because it is being produced by the standard Makefile target "bininstall". I was thinking more of the optional /usr/local/bin links which are all symlinks. So, if the Python 3 installer does is, there's no reason why Python 2 can't do it, I suppose.
But if you look at the Python 3 "bininstall" target (Makefile.pre.in starting around line 870 or so), you'll see that, for Python 3, symlinks are made for all the versioned files except "python3". I'm not sure that there is a particular reason why a distinction was made; IIRC, the other versioned links were added later in the cycle. The other Python 3 versioned links could probably be changed to hard links as well. In the end, I don't think it makes a lot of difference. But it would be better if Python 2 and Python 3 were consistent here.
My apologies for the confusion and extra work.
On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley mail@kerrickstaley.com wrote:
Nick, can you please apply the patch (will be sent in the following email) to the PEP SVN as soon as we get the hard-link issue is figured out? Alternatively, could you provide me write access to just the pep-0394.txt file so I can update it myself? Thanks.
Done: http://www.python.org/dev/peps/pep-0394/
Main thing needed now is a tracker issue to reference (ideally with a first cut at a patch) and python-dev's collective blessing to take it out of Draft status.
Cheers, Nick.
On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan ncoghlan@gmail.com wrote:
Quick question: When I do "svn up", it doesn't show any changes. Has it been switched over to Mercurial recently?
Thanks, Kerrick Staley
On Wed, Jul 20, 2011 at 12:03 PM, Kerrick Staley mail@kerrickstaley.com wrote:
On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan ncoghlan@gmail.com wrote:
Quick question: When I do "svn up", it doesn't show any changes. Has it been switched over to Mercurial recently?
-eric
Thanks, Kerrick Staley _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail....
I put up a tracker issue at http://bugs.python.org/issue12627
There are patches for 2.7 as well as tip, but they only fix the Makefiles; no changes are done to documentation.
Also, Ned, it appears that Python 2.7 doesn't install the Idle or PyDoc binaries (grep the 2.7 Makefile to see what I mean), so I didn't make any changes related to idle2 or pydoc2.
-Kerrick Staley
On Wed, Jul 20, 2011 at 3:28 AM, Nick Coghlan ncoghlan@gmail.com wrote:
On Wed, Jul 20, 2011 at 4:53 PM, Kerrick Staley mail@kerrickstaley.com wrote:
Nick, can you please apply the patch (will be sent in the following email) to the PEP SVN as soon as we get the hard-link issue is figured out? Alternatively, could you provide me write access to just the pep-0394.txt file so I can update it myself? Thanks.
Done: http://www.python.org/dev/peps/pep-0394/
Main thing needed now is a tracker issue to reference (ideally with a first cut at a patch) and python-dev's collective blessing to take it out of Draft status.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, 20 Jul 2011 01:53:09 -0500 Kerrick Staley mail@kerrickstaley.com wrote:
On Mon, Jul 18, 2011 at 3:03 AM, Ned Deily nad@acm.org wrote:
I think adding the requirement to mandate hard link vs soft link usage is an unnecessary and unwarranted attempt at optimization. For instance, IIRC, the OS X installers don't use any hard links: that may complicate the install, plus hard links on OS X HFS* file systems are a bit of a kludge and not necessarily more efficient than symlinks. It's not a big deal but perhaps the wording should be changed to make a suggestion about hard links vs syminks rather than mandate which should be used.
Ah, OK. The wording's been changed so that symbolic links will be installed on Mac OS X and hard links elsewhere (although maybe symbolic links are also better on certain other platforms; I'm not sure).
I do think that specific instructions must be given (rather than just a suggestion) because it's indicating what must be done to CPython. The instructions *should* be as close as possible to what the installer already does, but I'm not entirely sure what the installer does by default, and the hard-link recommendation was based off a cursory inspection of my own system, so further input from yourself and the rest of python-dev would be appreciated.
I think the recommendation should be symbolic links for all systems. Hard links are generally harder to discover, while it is trivial to find out that a given file is a symbolink link, and to which other file. The optimization is probably not useful in the real world (our executables are relatively small, and people worried about a couple of megabytes can always go for the shared library option).
Regards
Antoine.
On Sun, 24 Jul 2011 23:16:55 +0200 Antoine Pitrou solipsis@pitrou.net wrote:
I think the recommendation should be symbolic links for all systems. Hard links are generally harder to discover, while it is trivial to find out that a given file is a symbolink link, and to which other file. The optimization is probably not useful in the real world (our executables are relatively small, and people worried about a couple of megabytes can always go for the shared library option).
Oh, I don't even know why I thought hard links would be a size optimization anyway. I guess I thought "file copy" instead of "symbolic link".
Regards
Antoine.
I'm indifferent either way. python3 is a hard link to python3.2, so I thought we'd make everything that way for consistency. Higher-level links (python/idle/pydoc/python-config) have to be soft links so that if, e.g., python points to python3, and python3 is then pointed to another location, python also gets changed.
-Kerrick Staley
Le mardi 26 juillet 2011 à 10:56 -0500, Kerrick Staley a écrit :
I'm indifferent either way. python3 is a hard link to python3.2, so I thought we'd make everything that way for consistency.
Is it? Yikes, I didn't know about that.
Regards
Antoine.
Le 26/07/2011 18:05, Antoine Pitrou a écrit :
Le mardi 26 juillet 2011 à 10:56 -0500, Kerrick Staley a écrit :
I'm indifferent either way. python3 is a hard link to python3.2, so I thought we'd make everything that way for consistency.
Is it? Yikes, I didn't know about that.
Yikes for me too. I’ve had a quick look at the Makefile (look for $(LN)) and found that all scripts use symbolic links, but the python3 to python3.y link is hard. I wonder why this is.
FTR, downstream packagers may change this. Example on Debian:
/usr/bin/python3: symbolic link to `python3.2' /usr/bin/python3.2: symbolic link to `python3.2mu'
Cheers
In article 4E2EE813.1080407@netwok.org, Éric Araujo merwok@netwok.org wrote:
Le 26/07/2011 18:05, Antoine Pitrou a écrit :
Le mardi 26 juillet 2011 à 10:56 -0500, Kerrick Staley a écrit :
I'm indifferent either way. python3 is a hard link to python3.2, so I thought we'd make everything that way for consistency.
Is it? Yikes, I didn't know about that.
Yikes for me too. I’ve had a quick look at the Makefile (look for $(LN)) and found that all scripts use symbolic links, but the python3 to python3.y link is hard. I wonder why this is.
I pointed that out earlier in the thread:
"But if you look at the Python 3 "bininstall" target (Makefile.pre.in starting around line 870 or so), you'll see that, for Python 3, symlinks are made for all the versioned files except "python3". I'm not sure that there is a particular reason why a distinction was made; IIRC, the other versioned links were added later in the cycle. The other Python 3 versioned links could probably be changed to hard links as well. In the end, I don't think it makes a lot of difference. But it would be better if Python 2 and Python 3 were consistent here."
I don't think it makes all that much difference one way or the other. But it would be better for them all to be one kind or the other.
Ok, apparently the decision to make hard links for executables dates at least back to:
changeset: 16221:588691f806f4 branch: legacy-trunk user: Neil Schemenauer nascheme@enme.ucalgary.ca date: Wed Jan 24 17:11:43 2001 +0000 files: Makefile.pre.in description: Flat makefile based on toplevel Makefile.in and makefiles in build subdirectories. Those other makefiles will go away eventually.
[...]
+# Install the interpreter (by creating a hard link to python$(VERSION)) +bininstall: altbininstall + -if test -f $(BINDIR)/$(PYTHON); \ + then rm -f $(BINDIR)/$(PYTHON); \ + else true; \ + fi + (cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT)) +
Regards
Antoine.
On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:
Ok, apparently the decision to make hard links for executables dates at least back to:
That still doesn't explain *why* hardlinks were originally chosen instead of symlinks. In the absence of any other compelling argument against it, I think they should all consistently be symlinks. I don't see any Ubuntu or Debian (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and I haven't experienced any issues with it personally.
changeset: 16221:588691f806f4 branch: legacy-trunk user: Neil Schemenauer nascheme@enme.ucalgary.ca date: Wed Jan 24 17:11:43 2001 +0000 files: Makefile.pre.in description: Flat makefile based on toplevel Makefile.in and makefiles in build subdirectories. Those other makefiles will go away eventually.
[...]
+# Install the interpreter (by creating a hard link to python$(VERSION)) +bininstall: altbininstall
-if test -f $(BINDIR)/$(PYTHON); \then rm -f $(BINDIR)/$(PYTHON); \else true; \fi(cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT))
-Barry
It's been suggested that *ix has hardlinks because someone thought up hardlinks before someone thought up symlinks - IOW, there are those who suggest that if people had added symlinks first, no one would've bothered adding hardlinks.
Symlinks are almost always more flexible, and almost always more clear.
The main counterexample seems to be rsync-based backup systems, that will hardlink identical files of a given pathname. But that seems to be a bit of a mess when it comes time to transfer such a backup from one filesystem to another.
On Tue, Jul 26, 2011 at 3:32 PM, Barry Warsaw barry@python.org wrote:
On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:
Ok, apparently the decision to make hard links for executables dates at least back to:
That still doesn't explain *why* hardlinks were originally chosen instead of symlinks. In the absence of any other compelling argument against it, I think they should all consistently be symlinks. I don't see any Ubuntu or Debian (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and I haven't experienced any issues with it personally.
changeset: 16221:588691f806f4 branch: legacy-trunk user: Neil Schemenauer nascheme@enme.ucalgary.ca date: Wed Jan 24 17:11:43 2001 +0000 files: Makefile.pre.in description: Flat makefile based on toplevel Makefile.in and makefiles in build subdirectories. Those other makefiles will go away eventually.
[...]
+# Install the interpreter (by creating a hard link to python$(VERSION)) +bininstall: altbininstall
-if test -f $(BINDIR)/$(PYTHON); \then rm -f $(BINDIR)/$(PYTHON); \else true; \fi(cd $(BINDIR); $(LN) python$(VERSION)$(EXEEXT) python$(EXEEXT))-Barry
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com
Dan Stromberg drsalists@gmail.com writes:
It's been suggested that *ix has hardlinks because someone thought up hardlinks before someone thought up symlinks - IOW, there are those who suggest that if people had added symlinks first, no one would've bothered adding hardlinks.
Well, that suggestion is faulty. It ignores the fact that *all* ordinary files on Unix are hard links. Any ordinary file entry in a directory is a hard link to the file's data.
The “hard links” capability, therefore, isn't something that was added; it's fundamental to Unix filesystems from their inception.
The ‘ln’ command adds *additional* hard links to an existing file's data; but, once added, they're exactly the same as any other ordinary file entry.
Symlinks are almost always more flexible, and almost always more clear.
Yet many tools don't work as expected with symbolic links which will work with an ordinary file (a “hard link”). One can argue that such tools are to that extent buggy, but symbolic links produce deliberately different behaviour which is sometimes not what one needs.
On Wed, Jul 27, 2011 at 2:37 PM, Ben Finney ben+python@benfinney.id.auwrote:
Dan Stromberg drsalists@gmail.com writes:
It's been suggested that *ix has hardlinks because someone thought up hardlinks before someone thought up symlinks - IOW, there are those who suggest that if people had added symlinks first, no one would've bothered adding hardlinks.
Well, that suggestion is faulty. It ignores the fact that *all* ordinary files on Unix are hard links. Any ordinary file entry in a directory is a hard link to the file's data.
Not really. Whether hard links is supported is mostly a matter of what filesystem you are using - in modern times. It's true that filesystems with complete POSIX semantics probably all support hardlinks, but that's far from every file on any given *ix. And of course, POSIX doesn't appear to have been created until the late 1990's.
The “hard links” capability, therefore, isn't something that was added; it's fundamental to Unix filesystems from their inception.
Hard linking was reportedly in Unix Version 1, but I see nothing indicating it was in the original Unics of 1969. Then again, I don't see much of anything on the net about what was and wasn't in Unics.
The ‘ln’ command adds *additional* hard links to an existing file's data; but, once added, they're exactly the same as any other ordinary file entry.
Well, if you're in a filesystem that supports hardlinking anyway. Supporting that isn't inherent. It requires some sort of on-disk representation for persistence, and not all filesystems support that.
Symlinks are almost always more flexible, and almost always more clear.
Yet many tools don't work as expected with symbolic links which will work with an ordinary file (a “hard link”). One can argue that such tools are to that extent buggy, but symbolic links produce deliberately different behaviour which is sometimes not what one needs.
Please recall that I was paraphrasing someone saying that hardlinks were seldom better, not never better. I don't know that there's anything in your post that addresses that.
It's much easier to imagine a system with no hardlinks, than to imagine a system with no symlinks.
Dan Stromberg drsalists@gmail.com writes:
On Wed, Jul 27, 2011 at 2:37 PM, Ben Finney ben+python@benfinney.id.auwrote:
Dan Stromberg drsalists@gmail.com writes:
It's been suggested that […] if people had added symlinks first, no one would've bothered adding hardlinks.
Well, that suggestion is faulty. It ignores the fact that *all* ordinary files on Unix are hard links. Any ordinary file entry in a directory is a hard link to the file's data.
Not really. Whether hard links is supported is mostly a matter of what filesystem you are using - in modern times. It's true that filesystems with complete POSIX semantics probably all support hardlinks, but that's far from every file on any given *ix. And of course, POSIX doesn't appear to have been created until the late 1990's.
POSIX didn't create those semantics, though; it formalised semantics that were already long-time convention.
My only point was that, unlike symbolic links, hard linking isn't some added feature, it's a property of the way Unix filesystems represent normal files. And you're right that I mean “on filesystems with POSIX semantics”.
It's much easier to imagine a system with no hardlinks, than to imagine a system with no symlinks.
Why imagine? There have been many of both.
On Tue, Jul 26, 2011 at 3:32 PM, Barry Warsaw barry@python.org wrote:
On Jul 27, 2011, at 12:19 AM, Antoine Pitrou wrote:
Ok, apparently the decision to make hard links for executables dates at least back to:
That still doesn't explain *why* hardlinks were originally chosen instead of symlinks. In the absence of any other compelling argument against it, I think they should all consistently be symlinks. I don't see any Ubuntu or Debian (where /usr/bin/python3 -> python3.2) bug reports indicating any problems, and I haven't experienced any issues with it personally.
I ought to remember why because I remember I was involved. (And I have a feeling that the change Antoine dug up was just a refactoring, moving an existing hard-link target to a different file. Because in my memory the hard link was my idea and I implemented it. But that change is Neil Schemenauer's.)
But I can't. :-(
The best I can come up with is that when it's a hard link, you can replace either "python" or "python2.1" with another version without disturbing the other. And I think it has something to do with altinstall vs. install. So it would mean that after first installing 2.1 (so python and python2.1 are the same file), and then alt-installing 2.1.1, the original 2.1 binary is still accessible as "python". But I don't know why you'd want that.
Guido van Rossum guido@python.org wrote:
I ought to remember why because I remember I was involved. (And I have a feeling that the change Antoine dug up was just a refactoring,
You are correct. I checked Python 1.5.2 and it also creates hard links (prior to Makefile overhaul).
The best I can come up with is that when it's a hard link, you can replace either "python" or "python2.1" with another version without disturbing the other.
So, I don't think anyone has a good argument for the status quo. A symlink would make more sense to me.
Neil
Hi,
I’ve read the latest version of this PEP, as updated by Nick Coghlan in the Mercurial repo on July, 20th. Excuse me if I repeat old arguments, I did not reread all the threads.
In summary, I don’t think the PEP is useful right now, nor that it will set a good practice for the future.
- Unix-like software distributions (including systems like Mac OS X and
Minor: I call these “operating systems”.
- The Python 2.x ``idle``, ``pydoc``, and ``python-config`` commands should likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``, with the original commands invoking these versions by default, but possibly invoking the Python 3.x versions instead if configured to do so by the system administrator.
This item ignores that on some OSes, defining the default Python version is not a decision made by the sysadmin. The example I know is Debian (and derivatives): despite what one can read on the Web, it is not a good idea to change /usr/bin/python to point to the version you want; the decision affects all scripts used by the system itself, and is thus the call of the Debian Python maintainers. (FTR, Debian developers discussed adding /usr/bin/python2 at the latest DebConf and rejected it; I don’t know if the arguments raised are the same as mine, but maybe Piotr or someone else will chime in in this thread.)
This is needed as, even though the majority of distributions still alias the ``python`` command to Python 2, some now alias it to Python 3. Some of the former also do not provide a ``python2`` command; hence, there is currently no way for Python 2 code (or any code that invokes the Python 2 interpreter directly rather than via ``sys.executable``) to reliably run on all Unix-like systems without modification, as the ``python`` command will invoke the wrong interpreter version on some systems, and the ``python2`` command will fail completely on others. The recommendations in this PEP provide a very simple mechanism to restore cross-platform support, with minimal additional work required on the part of distribution maintainers.
I would like more data about this. How many OSes have moved their python executable to python2? How much people does that impact? Right now I think that there’s only Arch and Gentoo, which I would call minority platforms. (I’m aware that all UNIX-like free operating systems could be considered a minority OS all together, but we’re talking about UNIX-like OSes here :) Doing what the majority does is not always a good thing, but for this PEP I think that numbers can help us assess whether the trouble/benefit ratio is worth it.
In my opinion, the current situation is clear: python is some python2.y, python3 is a python3.y, this is not ambiguous and will still work in ten years when we get python4. Thus, the previous decision of python-dev to use python3 indefinitely seems good to me. As a script/program author, if I use python2 in my shebangs now to support what appears to be minority platforms, I’m breaking compatibility with a huge number of systems. Therefore, I don’t see how this PEP makes the situation better. If one OS wants to change the meaning of the python command, then its packaging tools should adapt shebangs, and its users should be aware that the majority of existing Python 3 scripts will break. Therefore, I’m strongly -1 on this PEP: changing the meaning of python brings much trouble for little or no benefit, and adding python2 adds another compatibility trouble.
It would be interesting to have feedback from people who lived the transition to Python 2.
- The ``pythonX.X`` (e.g. ``python2.6``) commands exist on some systems, on which they invoke specific minor versions of the Python interpreter. It can be useful for distribution-specific packages to take advantage of these utilities if they exist, since it will prevent code breakage if the default minor version of a given major version is changed. However, scripts intending to be cross-platform should not rely on the presence of these utilities, but rather should be tested on several recent minor versions of the target major version, compensating, if necessary, for the small differences that exist between minor versions. This prevents the need for sysadmins to install many very similar versions of the interpreter.
Here again I would be interested in more numbers. Pythons that people manually download and install using the provided makefile do have these pythonx.y executables, so I thought that all OSes did likewise.
Moreover, I disagree about the implied assertion that the minor number hardly matters (I’m paraphrasing): Python 2.6 and 2.7 *are* different, not “very similar”. I don’t know very well the usages of the community, but in my experience moving from 2.x to 2.x+1, or even just checking that your code still works, is a Big Deal. I’d like this whole bullet item to be removed.
Impact on PYTHON* Environment Variables
I think this section should be named PYTHONPATH, as it is the only envvar that it talks about. Another minor edit: s/folder/directory/
Regards
I think you missed the point of the PEP. The point is to create a new, python-dev-blessed standard that the distros will follow. The primary goal is so that a script can specify python2 or python3 in the #! line and expect that to work on all compliant linux systems, which we hope will be all of them. Everything else is just details. And yes, that distinction is much more important than the distinction between minor version numbers. That's the whole point of python3, after all.
-- R. David Murray http://www.bitdance.com
Hi Devid,
I think you missed the point of the PEP. The point is to create a new, python-dev-blessed standard that the distros will follow. The primary goal is so that a script can specify python2 or python3 in the #! line and expect that to work on all compliant linux systems, which we hope will be all of them. Everything else is just details.
I’m sorry if my opinion on that main point was lost among remarks on details. To rephrase one part of my reply: Right now, the de facto standard is that shebangs can use python to mean python2 and python3 to mean python3. Adding python2 to that and supporting making python ambiguous seems harmful to me.
Regards
On Thu, 11 Aug 2011 17:12:22 +0200, =?UTF-8?B?w4lyaWMgQXJhdWpv?= merwok@netwok.org wrote:
I’m sorry if my opinion on that main point was lost among remarks on details. To rephrase one part of my reply: Right now, the de facto standard is that shebangs can use python to mean python2 and python3 to mean python3. Adding python2 to that and supporting making python ambiguous seems harmful to me.
OK. So you are -1 on the PEP.
I'm a big +1.
To address your argument briefly, *now* a minority of distros have python pointing to python2. We expect this to change. It may not happen for 5 years, but someday it will. So this PEP is about preparing for the future.
Given that, I fail to see what harm having an additional symlink named python2 will do.
And yes this was argued about earlier and should (in theory at least) be addressed by the PEP, which is why I'm concluding that you are -1 on the PEP :).
-- R. David Murray http://www.bitdance.com
On Fri, Aug 12, 2011 at 1:12 AM, Éric Araujo merwok@netwok.org wrote:
I’m sorry if my opinion on that main point was lost among remarks on details. To rephrase one part of my reply: Right now, the de facto standard is that shebangs can use python to mean python2 and python3 to mean python3. Adding python2 to that and supporting making python ambiguous seems harmful to me.
This PEP comes mainly out of the fact that we collectively think Arch (the case that prompted the original discussion) are making a mistake that will hurt their users in switching the default Python *right now*, so the PEP is first and foremost designed to record that consensus. However, their actions do mean that the 'python' name is *already* ambiguous, no matter what the mainstream distros think. The Debian maintainers may not care about that, but *I* do, as does anyone wanting to write distro-agnostic shebang lines.
Given that some distros (large or small), along with some system administrators, are going to want to have python refer to python3, either now or at some point in the future, there are really only two options available to us here:
1. Accept the reality of that situation, and propose a mechanism that minimises the impact of the resulting ambiguity on end users of Python by allowing developers to be explicit about their target language. This is the approach advocated in PEP 394.
2. Tell the Arch developers (and anyone else inclined to point the python name at python3) that they're wrong, and the python symlink should, now and forever, always refer to a version of Python 2.x.
It's worth noting that there has never been any previous python-dev consensus to use 'python3' indefinitely - the status quo came about because it makes sense for the moment even to those of us that *want* 'python' to eventually refer to 'python3', so there was no previous need for an explicit choice between the two alternatives. By migrating, Arch has forced us to choose between either supporting their action or else telling Python users "Don't blame us, blame the distros that pointed python at python3" when things break.
I flat out disagree with the second approach - having to type 'python3' when a 3.x variant is the only version of Python installed would just be dumb, and I also think playing that kind of blame game in the face of inevitable cross-distro compatibility problems is disrespectful to our users. If you want to get Zen about it, 'practicality beats purity', 'explicit is better than implicit' and 'In the face of ambiguity, refuse the temptation to guess' all come down in favour of the approach in PEP 394.
If I haven't persuaded you to adjust your view up to at least a -0 (i.e. don't entirely agree, but won't object to others moving forward with it) and you still wish to advocate for the second approach, then I suggest creating a competing PEP in order to provide a clear alternative proposal (with Guido or his appointed delegate having the final say, as usual) that explains the alternative recommendation for: - distros that have already switched their 'python' links to refer to python3 - Python developers wishing to write shebang lines that work on multiple 2.x versions and support both platforms that define 'python2' and those that only define 'python'
FWIW, the closest historical precedent I can recall is Red Hat's issues when users switched the system Python from 1.5 to 2.2+, and the lesson learned from that exercise was that distro installed utilities should always reference a specific Python version rather than relying on the system administrator leaving the 'python' link alone. It sounds like Debian chose not to heed that lesson, which is unfortunate (although, to be honest, I'm not sure how well Fedora/Red Hat heed it, either). However, the commentary in PEP 394 based on that history (i.e. that distros really shouldn't care where the python name points) will remain in place.
Regards, Nick.
On Aug 12, 2011, at 01:10 PM, Nick Coghlan wrote:
- Accept the reality of that situation, and propose a mechanism that
minimises the impact of the resulting ambiguity on end users of Python by allowing developers to be explicit about their target language. This is the approach advocated in PEP 394.
- Tell the Arch developers (and anyone else inclined to point the
python name at python3) that they're wrong, and the python symlink should, now and forever, always refer to a version of Python 2.x.
FWIW, although I generally support the PEP, I also think that distros themselves have a responsibility to ensure their #! lines are correct, for scripts they install. Meaning, if it requires rewriting the #! line on OS package install, so be it.
-Barry
On Fri, 12 Aug 2011 12:19:23 -0400, Barry Warsaw barry@python.org wrote:
On Aug 12, 2011, at 01:10 PM, Nick Coghlan wrote:
- Accept the reality of that situation, and propose a mechanism that
minimises the impact of the resulting ambiguity on end users of Python by allowing developers to be explicit about their target language. This is the approach advocated in PEP 394.
- Tell the Arch developers (and anyone else inclined to point the
python name at python3) that they're wrong, and the python symlink should, now and forever, always refer to a version of Python 2.x.
FWIW, although I generally support the PEP, I also think that distros themselves have a responsibility to ensure their #! lines are correct, for scripts they install. Meaning, if it requires rewriting the #! line on OS package install, so be it.
True, but I think that is orthogonal to the purposes of the PEP, which is about supporting writing of system independent scripts that are *not* provided by the distribution (or installed via packaging). And PEP 397 aims to extend that to Windows, as well.
-- R. David Murray http://www.bitdance.com
On Aug 12, 2011, at 01:34 PM, R. David Murray wrote:
True, but I think that is orthogonal to the purposes of the PEP, which is about supporting writing of system independent scripts that are *not* provided by the distribution (or installed via packaging). And PEP 397 aims to extend that to Windows, as well.
Yep, agreed. It probably should also inform #! transformations that pysetup could do.
-Barry
On Fri, Aug 12, 2011 at 12:19:23PM -0400, Barry Warsaw wrote:
On Aug 12, 2011, at 01:10 PM, Nick Coghlan wrote:
- Accept the reality of that situation, and propose a mechanism that
minimises the impact of the resulting ambiguity on end users of Python by allowing developers to be explicit about their target language. This is the approach advocated in PEP 394.
- Tell the Arch developers (and anyone else inclined to point the
python name at python3) that they're wrong, and the python symlink should, now and forever, always refer to a version of Python 2.x.
FWIW, although I generally support the PEP, I also think that distros themselves have a responsibility to ensure their #! lines are correct, for scripts they install. Meaning, if it requires rewriting the #! line on OS package install, so be it.
+1 with the one caveat... it's nice to upstream fixes. If there's a simple thing like python == python-2 and python3 == python-3 everywhere, this is possible. If there's something like python2 == python-2 and python-3 == python3 everywhere, this is also possible. The problem is that: the latter is not the case (python from python.org itself doesn't produce a python2 symlink on install) and historically the former was the case but since python-dev rejected the notion that python == python-2 that is no long true.
As long as it's just Arch, there's still time to go with #2. #1 is not a complete solution (especially because /usr/bin/python2 will never exist on some historical systems [not ones I run though, so someone else will need to beat that horse :-)]) but is better than where we are now where there is no guidance on what's right and wrong at all.
-Toshio
On 8/11/2011 10:36 AM, Éric Araujo wrote:
It would be interesting to have feedback from people who lived the transition to Python 2.
There was no comparable transition. Python 2.0 was basically 1.6 renamed for a different distributor. I regard Python 2.2, which introduced new-style, as the beginning of Python 2 as something significantly different from Python 1. I suppose one could also point to the earlier intro of unicode. The new iterator protocol was also a major change. In any case, back compatibility was kept in all three respects (and others) until Python 3.
On Thu, Aug 11, 2011 at 6:05 PM, Terry Reedy tjreedy@udel.edu wrote:
There was no comparable transition. Python 2.0 was basically 1.6 renamed for a different distributor.
No that's not true. If you compare the "what's new" sections there is quite a large difference between 1.6 and 2.0, despite being released simultaneously.
I regard Python 2.2, which introduced new-style, as the beginning of Python 2 as something significantly different from Python
Just compare:
http://www.python.org/download/releases/2.0/ http://www.python.org/download/releases/1.6/
No argument that 2.2 was a big jump for the type system -- but not for Unicode.
I suppose one could also point to the earlier intro of unicode.
In 1.6. (But internally we called it the "contractual obligation release", a Monty Python reference.)
The new iterator protocol was also a major change. In any case, back compatibility was kept in all three respects (and others) until Python 3.
(I gotta go, but I don't think it was such a big deal -- it was very carefully made backwards compatible.)
-
Antoine Pitrou -
Barry Warsaw -
Ben Finney -
Dan Stromberg -
Eric Snow -
Guido van Rossum -
Kerrick Staley -
Ned Deily -
Neil Schemenauer -
Nick Coghlan -
R. David Murray -
Terry Reedy -
Toshio Kuratomi -
Éric Araujo