Mailman 3 Scripts/Entry Points Python-Version Naming - Distutils-SIG

Scripts/Entry Points Python-Version Naming

Bohuslav Kabrda

19 May 2014 19 May '14

7:16 a.m.

Hi all, (I hope that this hasn't been discussed previously) so I've been trying to find out whether there's an explicit recommendation for creating and naming scripts/entry points depending on Python version that they're built with, but I didn't find any. As an example, setuptools' easy_install uses "easy_install-MAJOR.MINOR" (with dash), while pip uses "pipMAJOR.MINOR" (without a dash). Also, some projects only create "foo-MAJOR.MINOR", while others also create "foo-MAJOR" (and most also create "foo" without any versions). It may seem like an overkill, but wouldn't it be best to standardize: - which version is preferred (with or without dash) - which of these three variants (foo-MAJOR.MINOR, foo-MAJOR, foo) should be created by default Or better yet, I think it'd make sense to provide setuptools facilites to create these variants in a sensible default way and provide installation flags to alter this behaviour. Right now, it seems to me that every project is doing this on its own, which is not only inconsistent, but it also duplicates lots of efforts and is more error prone than providing one centralized solution (e.g. a function in distutils/setuptools). Thoughts/comments? Thanks! -- Regards, Bohuslav "Slavek" Kabrda.

Show replies by date

Donald Stufft

19 May 19 May

8:28 a.m.

On May 19, 2014, at 8:16 AM, Bohuslav Kabrda wrote:

...

Hi all, (I hope that this hasn't been discussed previously) so I've been trying to find out whether there's an explicit recommendation for creating and naming scripts/entry points depending on Python version that they're built with, but I didn't find any. As an example, setuptools' easy_install uses "easy_install-MAJOR.MINOR" (with dash), while pip uses "pipMAJOR.MINOR" (without a dash). Also, some projects only create "foo-MAJOR.MINOR", while others also create "foo-MAJOR" (and most also create "foo" without any versions). It may seem like an overkill, but wouldn't it be best to standardize: - which version is preferred (with or without dash) - which of these three variants (foo-MAJOR.MINOR, foo-MAJOR, foo) should be created by default

Or better yet, I think it'd make sense to provide setuptools facilites to create these variants in a sensible default way and provide installation flags to alter this behaviour. Right now, it seems to me that every project is doing this on its own, which is not only inconsistent, but it also duplicates lots of efforts and is more error prone than providing one centralized solution (e.g. a function in distutils/setuptools).

Thoughts/comments? Thanks!

I completely agree and this was something that has been on my radar for awhile. This also enables universal Wheels for projects that want/require having the version in the script name because the current way of hacking it yourself creates a command which is accurate for only one Python version. This is actually going to "do the wrong thing" in pure python Wheels because a wheel created with a script like that in Python 3.4 is also valid for Python 3.5 but the script wrappers will still have "3.4" in them. Personally I feel that foo, fooX, fooX.Y is a reasonable standard and it matches what Python itself does. Being consistent seems like a reasonable goal to have with it. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Daniel Holth

9:38 a.m.

On Mon, May 19, 2014 at 9:28 AM, Donald Stufft wrote:

...

On May 19, 2014, at 8:16 AM, Bohuslav Kabrda wrote:

...
Hi all, (I hope that this hasn't been discussed previously) so I've been trying to find out whether there's an explicit recommendation for creating and naming scripts/entry points depending on Python version that they're built with, but I didn't find any. As an example, setuptools' easy_install uses "easy_install-MAJOR.MINOR" (with dash), while pip uses "pipMAJOR.MINOR" (without a dash). Also, some projects only create "foo-MAJOR.MINOR", while others also create "foo-MAJOR" (and most also create "foo" without any versions). It may seem like an overkill, but wouldn't it be best to standardize: - which version is preferred (with or without dash) - which of these three variants (foo-MAJOR.MINOR, foo-MAJOR, foo) should be created by default

Or better yet, I think it'd make sense to provide setuptools facilites to create these variants in a sensible default way and provide installation flags to alter this behaviour. Right now, it seems to me that every project is doing this on its own, which is not only inconsistent, but it also duplicates lots of efforts and is more error prone than providing one centralized solution (e.g. a function in distutils/setuptools).

Thoughts/comments? Thanks!

I completely agree and this was something that has been on my radar for awhile. This also enables universal Wheels for projects that want/require having the version in the script name because the current way of hacking it yourself creates a command which is accurate for only one Python version. This is actually going to "do the wrong thing" in pure python Wheels because a wheel created with a script like that in Python 3.4 is also valid for Python 3.5 but the script wrappers will still have "3.4" in them.

Personally I feel that foo, fooX, fooX.Y is a reasonable standard and it matches what Python itself does. Being consistent seems like a reasonable goal to have with it.

Although I would suggest extending the idea. The version number suffix "3" or "3.4" is a shorthand for referring to the Python environment in which the script should run, including both the version of Python and the set of packages which are importable in that environment, so that for example copies of the "pip" installer for multiple Python environments can exist on the same path. Since multiple Python 3.4's can exist on the same system, this scheme is far too limiting. Instead, to handle virtualenvs, scripts should be suffixed with the hash of the absolute path to the interpreter and site-packages directory. So instead of pip-3.4 you get pip-5c280763736b1c7ff400b1ffc87ad5c5e581c0cf32dbead89cbc6a2823a4c4e3 ; the same for e.g. Mercurial hg-5c280763736b1c7ff400b1ffc87ad5c5e581c0cf32dbead89cbc6a2823a4c4e3. This scheme should be able to handle any number of Python versions & environments without confusion.

Paul Moore

10:36 a.m.

On 19 May 2014 15:38, Daniel Holth wrote:

...

On Mon, May 19, 2014 at 9:28 AM, Donald Stufft wrote:

...
...
It may seem like an overkill, but wouldn't it be best to standardize: - which version is preferred (with or without dash) - which of these three variants (foo-MAJOR.MINOR, foo-MAJOR, foo) should be created by default [...] I completely agree and this was something that has been on my radar for awhile. This also enables universal Wheels for projects that want/require having the version in the script name because the current way of hacking it yourself creates a command which is accurate for only one Python version. This is actually going to "do the wrong thing" in pure python Wheels because a wheel created with a script like that in Python 3.4 is also valid for Python 3.5 but

On May 19, 2014, at 8:16 AM, Bohuslav Kabrda wrote: [...] the script wrappers will still have "3.4" in them.

Personally I feel that foo, fooX, fooX.Y is a reasonable standard and it matches what Python itself does. Being consistent seems like a reasonable goal to have with it.

Although I would suggest extending the idea. The version number suffix "3" or "3.4" is a shorthand for referring to the Python environment in which the script should run, including both the version of Python and the set of packages which are importable in that environment, so that for example copies of the "pip" installer for multiple Python environments can exist on the same path. Since multiple Python 3.4's can exist on the same system, this scheme is far too limiting.

Instead, to handle virtualenvs, scripts should be suffixed with the hash of the absolute path to the interpreter and site-packages directory. So instead of pip-3.4 you get pip-5c280763736b1c7ff400b1ffc87ad5c5e581c0cf32dbead89cbc6a2823a4c4e3 ; the same for e.g. Mercurial hg-5c280763736b1c7ff400b1ffc87ad5c5e581c0cf32dbead89cbc6a2823a4c4e3. This scheme should be able to handle any number of Python versions & environments without confusion.

:-) Unless you have multiple versions of Python on your path and expect to be able to use them simultaneously, the unadorned script name ("foo" on Unix, "foo.exe" on Windows) should be entirely sufficient. However, unfortunately, some people do expect versioned executables to work, and have multiple Python executables on their path (it's standard to do so for Python 2 and 3 on Unix, after all...) and we should offer some support for those cases. It seems to me that the only sane approach would be to follow the behaviour of Python itself (pythonX and pythonX.Y). And update pip and setuptools to automatically generate those forms on install. I'd almost argue that on Windows the versioned names should not be created, because Python doesn't have versioned names there, but that's probably a step too far. This is a compatibility-breaking change, though, and would require executable names changing for some projects (after all, pip and easy_install themselves use different conventions - we'd need to get our own house in order before imposing a standard on others!) So it would need to be handled carefully, to make sure projects have a chance to deal with the impact. At a minimum, all projects would need to buy into the idea that versioning executables is no longer under their control, but is handled by the tools - projects that object to this could make a real mess by adding their own version numbers, or by using the old "scripts" approach and abandoning entry points. Paul.

Donald Stufft

10:37 a.m.

On May 19, 2014, at 11:36 AM, Paul Moore wrote:

...

On 19 May 2014 15:38, Daniel Holth wrote:

...
On Mon, May 19, 2014 at 9:28 AM, Donald Stufft wrote:

...
...
It may seem like an overkill, but wouldn't it be best to standardize: - which version is preferred (with or without dash) - which of these three variants (foo-MAJOR.MINOR, foo-MAJOR, foo) should be created by default [...] I completely agree and this was something that has been on my radar for awhile. This also enables universal Wheels for projects that want/require having the version in the script name because the current way of hacking it yourself creates a command which is accurate for only one Python version. This is actually going to "do the wrong thing" in pure python Wheels because a wheel created with a script like that in Python 3.4 is also valid for Python 3.5 but

On May 19, 2014, at 8:16 AM, Bohuslav Kabrda wrote: [...] the script wrappers will still have "3.4" in them.

Personally I feel that foo, fooX, fooX.Y is a reasonable standard and it matches what Python itself does. Being consistent seems like a reasonable goal to have with it.

Although I would suggest extending the idea. The version number suffix "3" or "3.4" is a shorthand for referring to the Python environment in which the script should run, including both the version of Python and the set of packages which are importable in that environment, so that for example copies of the "pip" installer for multiple Python environments can exist on the same path. Since multiple Python 3.4's can exist on the same system, this scheme is far too limiting.

Instead, to handle virtualenvs, scripts should be suffixed with the hash of the absolute path to the interpreter and site-packages directory. So instead of pip-3.4 you get pip-5c280763736b1c7ff400b1ffc87ad5c5e581c0cf32dbead89cbc6a2823a4c4e3 ; the same for e.g. Mercurial hg-5c280763736b1c7ff400b1ffc87ad5c5e581c0cf32dbead89cbc6a2823a4c4e3. This scheme should be able to handle any number of Python versions & environments without confusion.

:-)

Unless you have multiple versions of Python on your path and expect to be able to use them simultaneously, the unadorned script name ("foo" on Unix, "foo.exe" on Windows) should be entirely sufficient. However, unfortunately, some people do expect versioned executables to work, and have multiple Python executables on their path (it's standard to do so for Python 2 and 3 on Unix, after all...) and we should offer some support for those cases.

It seems to me that the only sane approach would be to follow the behaviour of Python itself (pythonX and pythonX.Y). And update pip and setuptools to automatically generate those forms on install. I'd almost argue that on Windows the versioned names should not be created, because Python doesn't have versioned names there, but that's probably a step too far.

This is a compatibility-breaking change, though, and would require executable names changing for some projects (after all, pip and easy_install themselves use different conventions - we'd need to get our own house in order before imposing a standard on others!) So it would need to be handled carefully, to make sure projects have a chance to deal with the impact. At a minimum, all projects would need to buy into the idea that versioning executables is no longer under their control, but is handled by the tools - projects that object to this could make a real mess by adding their own version numbers, or by using the old "scripts" approach and abandoning entry points.

Paul.

If we do standardize we should also likely standardize on how we handle alternative interpreters. Things like PyPy, Jython etc. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Nick Coghlan

6:40 p.m.

On 20 May 2014 01:38, "Donald Stufft" wrote:

...

If we do standardize we should also likely standardize on how we handle alternative interpreters. Things like PyPy, Jython etc.

The idea of a "py" launcher equivalent for POSIX systems likely has a place in that discussion. As far as the original post goes, the main place where I believe this is currently a common issue is on POSIX systems with parallel Python 2 & 3 stacks. On Windows, there is only "python", and PATH controls which version you get. You use the "py" command line options to nominate a specific interpreter. On POSIX, there is generally "python", "python2", "python2.x", "python3" and "python3.y", with "python" referring to the default Python 2 install (except on Arch). CPython provided scripts that exist in both (like pydoc) have a similar naming scheme, while Python 3 only scripts (like pyvenv) omit the Python 2 variants, and the unqualified names refer to the Python 3 version. At least pip 1.5+ follows the same naming conventions (I'm not sure about earlier versions). So, I think there are two problems here: 1. The dual Python 2/3 stacks that are common on POSIX systems 2. The more general problem of installing packages for multiple Python interpreters without naming conflicts in POSIX binary directories I think we actually solved problem 1 pretty well when implementing ensurepip - the current UI for enabling it is a horrible internal-only hack, but the *behaviour* available when pip is installing itself under ensurepip is exactly what I would like to see standardised on that front (the python.commands extension in PEP 459 already distinguishes between entry points with wrappers to be generated at install time and pre-built scripts). For the more general case, I don't believe we even have a behavioural precedent to consider following at this point. Cheers, Nick.

...

----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372

DCFA

...

_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

Daniel Holth

6:48 p.m.

On Mon, May 19, 2014 at 7:40 PM, Nick Coghlan wrote:

...

On 20 May 2014 01:38, "Donald Stufft" wrote:

...
If we do standardize we should also likely standardize on how we handle alternative interpreters. Things like PyPy, Jython etc.

The idea of a "py" launcher equivalent for POSIX systems likely has a place in that discussion.

As far as the original post goes, the main place where I believe this is currently a common issue is on POSIX systems with parallel Python 2 & 3 stacks.

On Windows, there is only "python", and PATH controls which version you get. You use the "py" command line options to nominate a specific interpreter.

On POSIX, there is generally "python", "python2", "python2.x", "python3" and "python3.y", with "python" referring to the default Python 2 install (except on Arch). CPython provided scripts that exist in both (like pydoc) have a similar naming scheme, while Python 3 only scripts (like pyvenv) omit the Python 2 variants, and the unqualified names refer to the Python 3 version.

At least pip 1.5+ follows the same naming conventions (I'm not sure about earlier versions).

So, I think there are two problems here:

1. The dual Python 2/3 stacks that are common on POSIX systems 2. The more general problem of installing packages for multiple Python interpreters without naming conflicts in POSIX binary directories

I think we actually solved problem 1 pretty well when implementing ensurepip - the current UI for enabling it is a horrible internal-only hack, but the *behaviour* available when pip is installing itself under ensurepip is exactly what I would like to see standardised on that front (the python.commands extension in PEP 459 already distinguishes between entry points with wrappers to be generated at install time and pre-built scripts).

For the more general case, I don't believe we even have a behavioural precedent to consider following at this point.

Cheers, Nick.

I think of the problem as having two classes of programs: programs that help you use Python itself (pip, ipython), and applications that are useful on their own (Mercurial). Only the first kind should have the Python version suffix. I find the second kind very interesting, mature applications for which the implementation language is not a nuisance for the user.

Donald Stufft

7:25 p.m.

On May 19, 2014, at 7:48 PM, Daniel Holth wrote:

...

On Mon, May 19, 2014 at 7:40 PM, Nick Coghlan wrote:

...
On 20 May 2014 01:38, "Donald Stufft" wrote:

...
If we do standardize we should also likely standardize on how we handle alternative interpreters. Things like PyPy, Jython etc.

The idea of a "py" launcher equivalent for POSIX systems likely has a place in that discussion.

As far as the original post goes, the main place where I believe this is currently a common issue is on POSIX systems with parallel Python 2 & 3 stacks.

On Windows, there is only "python", and PATH controls which version you get. You use the "py" command line options to nominate a specific interpreter.

On POSIX, there is generally "python", "python2", "python2.x", "python3" and "python3.y", with "python" referring to the default Python 2 install (except on Arch). CPython provided scripts that exist in both (like pydoc) have a similar naming scheme, while Python 3 only scripts (like pyvenv) omit the Python 2 variants, and the unqualified names refer to the Python 3 version.

At least pip 1.5+ follows the same naming conventions (I'm not sure about earlier versions).

So, I think there are two problems here:

1. The dual Python 2/3 stacks that are common on POSIX systems 2. The more general problem of installing packages for multiple Python interpreters without naming conflicts in POSIX binary directories

I think we actually solved problem 1 pretty well when implementing ensurepip - the current UI for enabling it is a horrible internal-only hack, but the *behaviour* available when pip is installing itself under ensurepip is exactly what I would like to see standardised on that front (the python.commands extension in PEP 459 already distinguishes between entry points with wrappers to be generated at install time and pre-built scripts).

For the more general case, I don't believe we even have a behavioural precedent to consider following at this point.

Cheers, Nick.

I think of the problem as having two classes of programs: programs that help you use Python itself (pip, ipython), and applications that are useful on their own (Mercurial). Only the first kind should have the Python version suffix. I find the second kind very interesting, mature applications for which the implementation language is not a nuisance for the user.

I agree with these two classifications, however I'm also not sure if it actually matters or if something like Mercurial could just get the versioned suffixes and 99% of people just ignore them. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Daniel Holth

8:26 p.m.

Then I vote for just another key that is a list of the previously-defined scripts that should also have Python version suffixes. Then pip's metadata would contain: py_versioned_scripts = [ 'pip' ] The installer would know to create additional names for the console script called pip. On Mon, May 19, 2014 at 8:25 PM, Donald Stufft wrote:

...

On May 19, 2014, at 7:48 PM, Daniel Holth wrote:

...
On Mon, May 19, 2014 at 7:40 PM, Nick Coghlan wrote:

...
On 20 May 2014 01:38, "Donald Stufft" wrote:

...
If we do standardize we should also likely standardize on how we handle alternative interpreters. Things like PyPy, Jython etc.

The idea of a "py" launcher equivalent for POSIX systems likely has a place in that discussion.

As far as the original post goes, the main place where I believe this is currently a common issue is on POSIX systems with parallel Python 2 & 3 stacks.

On Windows, there is only "python", and PATH controls which version you get. You use the "py" command line options to nominate a specific interpreter.

On POSIX, there is generally "python", "python2", "python2.x", "python3" and "python3.y", with "python" referring to the default Python 2 install (except on Arch). CPython provided scripts that exist in both (like pydoc) have a similar naming scheme, while Python 3 only scripts (like pyvenv) omit the Python 2 variants, and the unqualified names refer to the Python 3 version.

At least pip 1.5+ follows the same naming conventions (I'm not sure about earlier versions).

So, I think there are two problems here:

1. The dual Python 2/3 stacks that are common on POSIX systems 2. The more general problem of installing packages for multiple Python interpreters without naming conflicts in POSIX binary directories

I think we actually solved problem 1 pretty well when implementing ensurepip - the current UI for enabling it is a horrible internal-only hack, but the *behaviour* available when pip is installing itself under ensurepip is exactly what I would like to see standardised on that front (the python.commands extension in PEP 459 already distinguishes between entry points with wrappers to be generated at install time and pre-built scripts).

For the more general case, I don't believe we even have a behavioural precedent to consider following at this point.

Cheers, Nick.

I think of the problem as having two classes of programs: programs that help you use Python itself (pip, ipython), and applications that are useful on their own (Mercurial). Only the first kind should have the Python version suffix. I find the second kind very interesting, mature applications for which the implementation language is not a nuisance for the user.

I agree with these two classifications, however I'm also not sure if it actually matters or if something like Mercurial could just get the versioned suffixes and 99% of people just ignore them.

----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Paul Moore

20 May 20 May

1:59 a.m.

On 20 May 2014 02:26, Daniel Holth wrote:

...

Then I vote for just another key that is a list of the previously-defined scripts that should also have Python version suffixes. Then pip's metadata would contain:

py_versioned_scripts = [ 'pip' ]

The installer would know to create additional names for the console script called pip.

Having the package define *whether* versioned scripts are produced, but letting the installer choose *which* are installed, and the name format, seems sensible. Paul

Donald Stufft

19 May 19 May

7:21 p.m.

On May 19, 2014, at 7:40 PM, Nick Coghlan wrote:

...

On 20 May 2014 01:38, "Donald Stufft" wrote:

...
If we do standardize we should also likely standardize on how we handle alternative interpreters. Things like PyPy, Jython etc.

The idea of a "py" launcher equivalent for POSIX systems likely has a place in that discussion.

As far as the original post goes, the main place where I believe this is currently a common issue is on POSIX systems with parallel Python 2 & 3 stacks.

On Windows, there is only "python", and PATH controls which version you get. You use the "py" command line options to nominate a specific interpreter.

I know Paul would be for it, but I have no problem with defining the versioned binaries to not be created on Windows if that makes more sense there.

...

On POSIX, there is generally "python", "python2", "python2.x", "python3" and "python3.y", with "python" referring to the default Python 2 install (except on Arch). CPython provided scripts that exist in both (like pydoc) have a similar naming scheme, while Python 3 only scripts (like pyvenv) omit the Python 2 variants, and the unqualified names refer to the Python 3 version.

At least pip 1.5+ follows the same naming conventions (I'm not sure about earlier versions).

Earlier versions had pip and pip-X.Y I believe… except on some Linux platforms where they helpfully removes the pip-X.Y and replaced it with pipX.

...

So, I think there are two problems here:

1. The dual Python 2/3 stacks that are common on POSIX systems 2. The more general problem of installing packages for multiple Python interpreters without naming conflicts in POSIX binary directories

I think we actually solved problem 1 pretty well when implementing ensurepip - the current UI for enabling it is a horrible internal-only hack, but the *behaviour* available when pip is installing itself under ensurepip is exactly what I would like to see standardised on that front (the python.commands extension in PEP 459 already distinguishes between entry points with wrappers to be generated at install time and pre-built scripts).

Yes, when we implemented that my hope (goal?) was that we’d eventually remove the hacks and end up with something based on a standard.

...

For the more general case, I don't believe we even have a behavioural precedent to consider following at this point.

Cheers, Nick.

...
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Paul Moore

20 May 20 May

2:03 a.m.

On 20 May 2014 01:21, Donald Stufft wrote:

...

On May 19, 2014, at 7:40 PM, Nick Coghlan wrote:

...
On Windows, there is only "python", and PATH controls which version you get. You use the "py" command line options to nominate a specific interpreter.

I know Paul would be for it, but I have no problem with defining the versioned binaries to not be created on Windows if that makes more sense there.

My only reservation is that there have been discussions about adding versioned Python executables on Windows. I'd hate to have Python 3.5 (say) add them and us have to revisit this discussion with the added complexity of "should we do something different depending on the version of Python"...

...

On POSIX, there is generally "python", "python2", "python2.x", "python3" and "python3.y", with "python" referring to the default Python 2 install (except on Arch). CPython provided scripts that exist in both (like pydoc) have a similar naming scheme, while Python 3 only scripts (like pyvenv) omit the Python 2 variants, and the unqualified names refer to the Python 3 version.

At least pip 1.5+ follows the same naming conventions (I'm not sure about earlier versions).

IIRC, easy_install doesn't (it has a dash before the version). We should at least standardise all PyPA projects before looking to impose a standard on other projects. Paul

3622

Age (days ago)

3623

Last active (days ago)

List overview

Download

11 comments

5 participants

participants (5)

Bohuslav Kabrda
Daniel Holth
Donald Stufft
Nick Coghlan
Paul Moore

Scripts/Entry Points Python-Version Naming

tags

participants (5)