Proposal to change Python version release cycle
Hello, one of my long standing ideas to improve Python is to adjust the release cycle and version number handling. In short, to simplify it. This is the first draft of the idea: Proposal to change Python version release cycle =============================================== Goal ---- Increment the major version number more frequently for every new language feature release. Change Python feature version counting from "major.minor" to "major". This is simpler, better to handle compatibility and allows adding new library features more frequently. The current Python version scheme is Major.Minor.Patch this will be kept but the major version number will be incremented faster and for every language feature release as it is now for the minor number. For example today the minor version number is increased for every big release done in 1,5 year cycle. In this new language features with new keywords can appear, new libraries can be included and other C level API changes can happen. There backward compatible feature improvements and new features also backward incompatible can happen. I suggest to change this to increment the major version for every new release of the 1,5 year cycle. And allow new Python standard library backward compatible changes for every minor release cycle every 6 months. Start with this from Python version 4 on. Benefit ------- Clear separation of new language features and C API changes and new backward compatible feature additions. So for a major version on Python level new keywords can be added, build ins can be added or removed and the C API can be changed (new features or other incompatible changes). For a minor version new standard libraries can be added and API can be improved in a backward compatible way. This can be done more frequently for example in a 0,5 year cycle. Allowing the Python library part to improve in a higher rate. For a patch version only none breaking bug and security fixes are allowed. (Same as it is now, or even a little bit stricter) With this change there is a clear separation of new language Features which are also tracked by other Python implementations and new library features. This can be seen very easy by only looking at the Python major version. The C API level is included in this standard and is not allowed to change in a minor version as it is now. This is because booth the Python language and the C API of CPython implementation is a standard and tracked by other implementations. For them it is easy to say I am compatible to Version 4. For everyone in computer business it is clear what is meant. The Python standard library can be improved in a higher rate because backward compatible feature changes are allowed in a higher rate for minor versions. Also provisional libraries can be improved in a faster rate for every minor version. (for example every 6 months instead of 1,5 years) For Unix in the Python executable the major version can be appended and this specifies the Python language standard used. This is done already but most of the time only EXECUTABLEmajor.minor is useful. Also more complicated to handle as a simple Number. Can be problematic on some platforms because of the dot ".". Python major version every 1,5 years (same as it is now for major.minor). Python minor version every 6 months (to allow faster standard library changes). Python patch version as needed (to fix only bugs and security related stuff) This can solve the problem of adding something as a standard library and then this is the dead of the library because it cannot be improved any further in a higher update rate. The Python standard library should not be a dead end for a library after inclusion. This allows Python code to improve faster than the C code. This is needed to include new features faster and because we have more people capable in Python programming than for C in the Python community. On C level the compatibility distinction is simple only track the major version and no longer a mixture between major and major.minor compatibility. Risk ---- Minimal because to track features the major.minor version already has to be tracked. It is a change some assumptions about Python versions are at this point no longer valid. On C level external libraries and wrappers must adopt it and changes are needed. Open questions -------------- The exact release cycle for minor, can also be every 4 months if needed. Clear separation what is part of the language standard and what is part of the C API. Some modules, for example "sys" can also be part of the language Standard and should not change between minor versions. The difference between the stable C API and that for every minor version can be removed. There is only one C API guarantee for a major version and it does not change for a minor version. None goals ---------- Change the release cycle of 1,5 years for new big releases. Change the versioning scheme to something other than "major.minor.patch". New big ground breaking changes to everything. To complicate something. Regards, Wolfgang
On Sat, Nov 4, 2017 at 10:25 PM, <tds333@mailbox.org> wrote:
The usual implication of a major version bump is that there is significant incompatibility. That's something that should happen roughly once a decade, not every couple of years. Massive -1.0 from me on this. ChrisA
On 04.11.2017 12:35, Chris Angelico wrote:
Yes that implication is a possible risk. I will add it to the section if needed. As noted by Guido a ground breaking change as it happened from version 2 to 3 should never happen in Python land again and a possible version 4 will simply be after 3.9. Other languages increment their major version already faster and track the language standard simply with the major version. The major version should be incremented for a ground breaking change but an increment requires not a ground breaking change to do this. For me even adding new keywords to a language is a major change. (Such as async and await) Or changes in the C API. And talking about language standard version 4 is simpler than a more complicated scheme of major.minor. Also other implementations such as pypy can use this first version number in their name to show compatibility. (pypy4 compatible to Python ) I think if the changes happens it will take some time for people to adjust but then life will be simpler. :-) Thank you for your feedback. Regards, Wolfgang
On Sat, Nov 4, 2017 at 11:00 PM, Wolfgang <tds333@mailbox.org> wrote:
If I write code for Python 3.4 and try to run it on Python 3.7, there is a small probability that it will break (eg if it tries to use "await" as a function name - I had a program like that, and once the new keywords were pushed through, I renamed it to "wait" to ensure compatibility). But it's a fairly small chance, and it's usually pretty easy to write code that's compatible with several minor versions (by targeting the oldest and then testing on the newer ones). With major version changes, it's much harder to be compatible with both, and you often end up with compatibility shims (eg "try: input = raw_input; except NameError: pass"). This applies to code, to documentation, to answers on Stack Overflow, to blogs, and to everything else that discusses Python in any way. At work, we have to use certain JavaScript packages that have had, I kid you not, more than one major version bump per year since initial release. One time we only discovered an issue after a student installed the latest version of something, and the setup instructions didn't work. Oh, great, version 10 breaks things compared to version 9. I do NOT enjoy dealing with that kind of incompatibility, and permitting it every 1.5 years is a bad thing. And if compatibility is to be guaranteed, why bump the major version number? That's the whole point of the three-part version. ChrisA
Hello Wolfgang, On Sat, 4 Nov 2017 12:25:57 +0100 (CET) tds333@mailbox.org wrote:
There has been ample discussion in the past about changing our release cycle one way or another. In short, the meta-problem is there are many contradicting interests which would each favour a different solution to the problem. See for example this PEP (ultimately rejected): https://www.python.org/dev/peps/pep-0407/ and the discussion that ensued: https://mail.python.org/pipermail/python-dev/2012-January/thread.html#115591 I haven't read your proposal in detail, but I suspect that it may be vulnerable to some of the same objections. The big objection being that a significant part of our ecosystem (that is, to put it roughly, the more corporate-minded segment, though I believe it is a simplification and may include other segments, such as Debian stable users and maintainers) doesn't want to deal more frequent feature releases. Regards Antoine.
On 04.11.2017 14:29, Antoine Pitrou wrote:
I read this PEP and some of the discussion. The difference to my idea is not to propose a LTS version and feature preview releases. The simplest form of my change is to switch from today major.minor to simply major for the feature release cycle. Additional minor feature releases for the standard library can also be optional or limited to provisional parts of the standard library. In all cases they should be backward compatible. But have the option to change the Python parts faster than the core. My idea came up because I have seen the need to improve the standard library in a faster pace than the core language. And to show this also in the version number. Hopefully ending the if it is included into the Python standard library it will be dead. Resulting in more releases but for the core with the same release cycle as of now. The amount of minor releases is not high it will be kept lower as it is now. Another point is to show by a higher major version number how mature Python is. (marketing) ;-) For Linux distributions there is still the option to include a specific Python Version with a feature set and recommend it. If someone wants to be conservative it is save to rely on features only for the first major version. This will be the same as of now for major.minor. Regards, Wolfgang
On 5 November 2017 at 00:40, Wolfgang <tds333@mailbox.org> wrote:
We're currently more likely to go the other direction, and stick with the 3.x numbering for an extended period (potentially reaching 3.10, 3.11, 3.12, etc), so that the ongoing disruption caused by the 2.x -> 3.x transition has had a chance to settle down before we ask anyone to start calling anything "Python 4". While the problems Linux distros are facing with whether "python" should refer to "python2" or "python3" are at least arguably self-inflicted (especially for those that already dealt with the Python 1.5.2 -> Python 2.0 migration back in the early 2000s), nobody is really looking forward to having to figure out how to adjust to a potential future Python 4.0 that invalidates all the current "python3" based naming schemes that have been introduced to work around the fact that Python 3.x needed to be parallel installable with Python 2.x without breaking any existing Python 2.x applications. We're not even sure yet when we're going to update PEP 394 to say that we think it's reasonable for distros to start using "python" to mean "python3" (see https://www.python.org/dev/peps/pep-0394/ and https://github.com/python/redistributor-guide/issues/1 for more discussion on the latter point). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Nick, On 04.11.2017 16:01, Nick Coghlan wrote:
A good point but two digits minor version numbers have the possibility to break a lot code. There is a lot of stuff out where a single digit major version is assumed. Even the official Python build for windows with python27.dll, python36.dll can be problematic because the dot is omitted between numbers. Other do the same for compilation they concatenate only majorminor for a name. Then version 3.10 is the same as 31.0. Ok I know this will take a while. But for the first 2.7.10 version even there some other library code was broken because of such assumptions.
I know on some build computers I have the same problem. But solved it for my scripts by using the major number as compatibility specifier and it works fine for me. Also knowing it is not upward compatible. The whole problem can only be solved by something used for Windows where the Python launcher deals with this. A Python launcher (py) for Linux can solve this there. Allowing also distributions to configure a fall back. For example the user specified python3 but I know it will be compatible with python4 so the launcher can do something like python3 is not found then I use python4 if available. ex.: #!/usr/bin/env py -3 And handle it as configurable version hint. Or even allowing alternative implementation like pypy if compatible. Regards, Wolfgang
On Sun, Nov 5, 2017 at 2:29 AM, Wolfgang <tds333@mailbox.org> wrote:
Python 1.0 was released in 1994. Then 2.0 came out in 2000 (six years), and 3.0 in 2008 (eight years). So far, we've been nine years into 3.0 and aren't looking at 4.0 yet, so it's going to be at least ten. If version 5.0 is another twelve years after that, and version 6.0 is fourteen later, we'll be looking for version 31.0 some time around the year 3000. I think we can let a future generation worry about pathing problems with DLL names. Seriously, I don't think there's any need to stress about version 31 of something as stable as Python. No matter how long it *actually* is between versions, it's going to be a lot longer than we can really plan for right now. In that much time, *anything* could change. ChrisA
Isn't this recent bit of discussion an argument in favor of a new stdlib module `version`? That would contain things like resolving a version tuple to an executable name (or a cross-platform piece of an executable name)? Obviously this doesn't actually answer the question of how naming should be done in the future, but it does mean that (new) code will have the actual choice abstracted away. --Josh On Mon, Nov 6, 2017 at 4:20 AM Stephan Houben <stephanh42@gmail.com> wrote:
On 6 November 2017 at 22:19, Stephan Houben <stephanh42@gmail.com> wrote:
Ah, you're right, I forgot about that option (I think Ezio Melotti suggested it previously). Yes, going with "3.10", but encoding it as "3A" in lexically ambiguous contexts is another option that would let us get as far as 3.35 (aka "3Z") before encountering ambiguity problems. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 5 November 2017 at 01:29, Wolfgang <tds333@mailbox.org> wrote:
Aye, we're aware :) We're not in a situation where we'll have any *good* options available, so the question we're considering is "What do we think will be the least bad approach?". At our current release cadence, 3.7 will be in mid 2018, 3.8 in late 2019 or early 2020, and then 3.9 in mid 2021. The open question will be what we call the release after that (in late 2022), with the main leading contenders being "4.0" (on the grounds that so many changes will have accumulated since 2008's 3.0 release by then that it makes sense to call them different major versions, similar to the rationale the Linux kernel now uses for major version updates), and "3.10" (on the grounds that the release won't be any more different from 3.9 than 3.9 will be from 3.8). Other variants (like making the major release number "40" or "2022", and using the minor version field for some new purpose other than indicating compatibility breaks in the compiler AST, interpreter opcode format, code caching tags, and C ABI), tend to introduce the same pragmatic questions that will already need to be considered in choosing between the 3.10 and 4.0 alternatives. It's not just the version numbering aesthetics that need to be considered either - in addition to the point you raise around how the Python version gets embedded in file path names (such that the 3-digit form is technically ambiguous without a separator, and may break parsers that are expecting exactly two digits), there are other backwards compatibility concerns around how folks do version compatibility checks based on sys.version and sys.version_info. Python and CPython will be more than 3 decades old by 2022, and an ecosystem can build up a *lot* of implicit assumptions regarding how a platform's versioning scheme works in that kind of time frame :) Personally, I doubt we'll make a firm decision on how we're going to handle this until some time after 2.7 goes EOL in 2020, as by then we'll necessarily have a better idea of how the various Linux distros (including the LTS commercial ones) ended up handling the Python 2 -> Python 3 switch, and the fact that the next release at that point will be 3.9 making it eminently clear that we've run out of time to continue postponing the question. We'll hopefully also have more experience with folks relying on the stable runtime ABI, and hence whether or not it might be beneficial to define new "py4" and "abi4" compatibility tags for the binary wheel distribution format. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
I predict by then that Larry will have finished the gilectomy, and while it will require backward incompatible changes to the C API, Python will run so blindingly fast in the quantum computing machine learning blockchain in the IOT cloud, bumping to 4.0 will be a no brainer[*]. Cheers, -Barry [*] Plus, our AI overlords will have made the decision for us. P.S. I suck at predictions.
2017-11-05 21:02 GMT+01:00 Serhiy Storchaka <storchaka@gmail.com>:
But len(os.fsencode("python3⑽.dll")) != len(os.fsencode("python39.dll")).
I think it is on Windows. os.fsencode should use UTF-16 there. ⑽ is in the BMP Presumably, non-Windows platforms wouldn't be interested in .dll files anyway... Stephan
Just to clarify: Python 2.0 was called 2.0 because the BeOpen marketing department thought it was good idea, not because there were major incompatible changes going into that release. Porting code from Python 1.5.2 to 2.0 was relatively straight forward and not much different from other minor releases. The first ever major backwards incompatibility release switch we had in Python after the great renaming of the C APIs between Python 1.1 and 1.2 (which was only visible to C extensions and relatively easy to fix using a compatibility header file), was the transition from Python 2.x to Python 3.x. IMO, the only reason to do this again would be the removal of the GIL, but only if there's absolutely no other way to get around such breakage. I would very much appreciate Python's releases not to introduce such major changes anymore. A continuous gradual change is a lot better to handle than non-continuous breaks and I'm pretty sure both will get us to the same level functionality eventually, perhaps even avoiding a few new mistakes along the way. With that approach, the versioning scheme would just be an implementation detail, which is good. Whether we call it Python 4.2 or Python 42 is really not all that important. Python is mature enough to not have to pull marketing tricks anymore. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 04 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 04.11.2017 16:44, M.-A. Lemburg wrote:
Thanks for the interesting details. Have to admit I entered the Python land with version 1.4 and missed the C API change from 1.1 to 1.2 completely. :-) Getting more opinions about the idea confirms me slowly to something like, good to discuss but not worth the change. Because some arguments confirmed me that it is not worth to change this. Also the marketing argument is weak and you are right that it should not care us. The only remaining argument is to have shorter release cycles but I think they don't depend on the versioning scheme nor a change in it improves this. Looking forward to 2020 when 2.7 is out of business and the maintenance cost can be invested in Python 4 and a faster release schedule. :-) Regards, Wolfgang
On Sat, Nov 4, 2017 at 10:44 AM, M.-A. Lemburg <mal@egenix.com> wrote:
Alternative history question: if it was just 1.6, then would Python 3000, probably called "Python 2000/2.x" in that world, with all the fixes have happened sooner because you would have "ran out" (yes, v1.10 > v1.9) of numbers leading up to 2?
But of course we'll skip Python 9, just like iPhones and Windowses ;)
On 04.11.2017 20:33, Nick Timkovich wrote:
There was a Python 1.6, but this was cut during the Python 2.0 release cycle to make CNRI happy. It included everything the CNRI team had developed after Python 1.5.2 before moving on to BeOpen to form the "PythonLabs": https://www.python.org/download/releases/1.6/ See "Overview of Changes Since 1.6" in https://www.python.org/download/releases/2.0/ for a list of changes between 1.6 and 2.0. The Python 1.6 never went into wide distribution, so for all practical purposes the move was from Python 1.5.2 to Python 2.0. More on all this is available in the "What's New in Python 2.0" document: https://docs.python.org/3/whatsnew/2.0.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 05 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Nick Timkovich wrote:
Alternative history question: if it was just 1.6
Guido's time machine strikes again. There was both a Python 1.6 and a 1.6.1. https://www.python.org/download/releases/1.6/ https://www.python.org/download/releases/1.6.1/ The "Contractual Obligation" releases. (And another Monty Python's Flying Circus reference.) These resolved licensing and release issues related to the BeOpen adventure and GPL-compatibility. Cheers, -Barry
M.-A. Lemburg wrote:
Fond memories of some of my first contributions to Python: http://python-history.blogspot.com/2009/03/great-or-grand-renaming.html Cheers, -Barry
On 4 November 2017 at 23:29, Antoine Pitrou <solipsis@pitrou.net> wrote:
PEP 413 was another competing proposal from around the same time: https://www.python.org/dev/peps/pep-0413/ Technically neither proposal has been formally Rejected, but they're also Deferred/Withdrawn because we knew that if we *had* asked for an explicit determination, the answer would have been "No, too high a cost, for not enough of a demonstrated benefit" (and that hasn't really changed in the past 5 years).
It's not just redistributors that get affected - it's the compatibility testing matrices for community projects as well. With our current release cycles (and excluding the 2.7 LTS release from consideration) we tend to have the following situation for the 3.x branches: - 1 actively maintained version - 2 or 3 security-fix only branches This stems from the default commitment of security fixes ending around 5 years after an X.Y.0 release, and releases happening 18-24 months apart: * Month 0: X.Y.0 released * Month 18: X.(Y+1).0 released * Month 36: X.(Y+2).0 released * Month 54: X.(Y+3).0 released * Month 60: X.Y.Z goes end-of-life * Month 72: X.(Y+4).0 released That same pattern also held for the 2.x series until the 2.7 release in 2010 (and it only changed then because that was the last 2.x feature release). The current year or so where the number of supported branches drops down to only 3 means we have some scope to speed up the rate of feature releases a bit (i.e. a release cadence of every 15 months instead of every 18 still means the community compatibility testing matrix consistently stays around 4 versions, or 5 if you're also testing against the dev release), but going to 6 monthly version updates means folks will either keep their test matrices the same as they are now (increasing the odds of the standard-library-only updates breaking things and that not being detected for some time), or else feeling obliged to expand their test matrices to cover even more concurrently supported versions. There are potential ways around both of those problems, but they require making the defined support periods for minor releases significantly shorter, unless they're the last minor release before the next major release (which means that instead of being simpler, these kinds of arrangements tend to end up being *more* complicated than the "every feature release receives security updates for 5 years" status quo). So rather than making fundamental changes to the way Python is versioned just to make standard library updates more readily available on older base versions of Python, we've instead gone with a more needs driven approach: making backport modules available for older versions where we find it is beneficial to do so (e.g. contextlib2, unittest2, importlib2). While Python 2.7 is a major driver for that change in practice, the benefits apply to older 3.x releases as well: - the updated modules are available for all Python versions and implementations where the packaging tools work (which is most of them) - users can decide on a module-by-module basis whether they want the baseline version or fast track updates - those decisions are explicitly tracked in the project's dependency metadata, rather than being implicit in the choice of deployment platform Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Nick, On 04.11.2017 15:48, Nick Coghlan wrote:
This is a valid concern. So more pressure by more releases can be a problem. For my idea the only possibility to limit this is to have minor versions only for the provisional parts to allow improvements and to have the major version track the core features and the bug fixing only for the first major version. Resulting in something like 4.0.0, 4.0.1 and 4.1.0 for something like a feature update for provisional but not supported with bug fixing and not for compatibility testing. This is then something like PEP-407 which is rejected. And also makes things little more complicated. Another possibility is to change only the versioning to major.minor instead of major.minor.patch. Then having a simpler versioning scheme for other Python implementations as only benefit (and the simplification to spell compatibility). Thus major is for the compatibility and minor counts the bug fix releases. Thinking if a change then is needed. :-) Marketing benefits? Don't know then. Regards, Wolfgang
On Sat, 4 Nov 2017 16:10:32 +0100 Wolfgang <tds333@mailbox.org> wrote:
I think the versioning scheme is a red herring here. The important topic is when Python releases are cut and what kinds of changes they are made of.
Thinking if a change then is needed. :-) Marketing benefits? Don't know then.
I don't think there are any marketing benefits. That might have been true 10 years ago, but nowadays people are so used to staggering increases in version numbers that they are not impressed anymore. Besides, IMHO Python is well past the point where it needed a concerted marketing effort to promote itself. Regards Antoine.
On 04/11/2017 13:29, Antoine Pitrou wrote:
Another minus point from my prospective - I work for a huge corporate organisation, (~300k employees), and we have to seek approval from the legal departments, on a per division basis, before we can use any Open Source software, (I and several others know that the per division bit is a ridiculous overhead and we are trying to get it addressed). The approvers will let us get away with saying that we need approval to use version major.X, i.e. Python 3.x has been approved, but will not let us get away with requesting to use a tool without specifying the major version number. I suspect that the reasoning behind this is that most packages rightly consider a licence change a major revision. So for us this suggestion would result in us having to re-seek approval every 12-18 months. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com
On Sat, Nov 4, 2017 at 10:25 PM, <tds333@mailbox.org> wrote:
The usual implication of a major version bump is that there is significant incompatibility. That's something that should happen roughly once a decade, not every couple of years. Massive -1.0 from me on this. ChrisA
On 04.11.2017 12:35, Chris Angelico wrote:
Yes that implication is a possible risk. I will add it to the section if needed. As noted by Guido a ground breaking change as it happened from version 2 to 3 should never happen in Python land again and a possible version 4 will simply be after 3.9. Other languages increment their major version already faster and track the language standard simply with the major version. The major version should be incremented for a ground breaking change but an increment requires not a ground breaking change to do this. For me even adding new keywords to a language is a major change. (Such as async and await) Or changes in the C API. And talking about language standard version 4 is simpler than a more complicated scheme of major.minor. Also other implementations such as pypy can use this first version number in their name to show compatibility. (pypy4 compatible to Python ) I think if the changes happens it will take some time for people to adjust but then life will be simpler. :-) Thank you for your feedback. Regards, Wolfgang
On Sat, Nov 4, 2017 at 11:00 PM, Wolfgang <tds333@mailbox.org> wrote:
If I write code for Python 3.4 and try to run it on Python 3.7, there is a small probability that it will break (eg if it tries to use "await" as a function name - I had a program like that, and once the new keywords were pushed through, I renamed it to "wait" to ensure compatibility). But it's a fairly small chance, and it's usually pretty easy to write code that's compatible with several minor versions (by targeting the oldest and then testing on the newer ones). With major version changes, it's much harder to be compatible with both, and you often end up with compatibility shims (eg "try: input = raw_input; except NameError: pass"). This applies to code, to documentation, to answers on Stack Overflow, to blogs, and to everything else that discusses Python in any way. At work, we have to use certain JavaScript packages that have had, I kid you not, more than one major version bump per year since initial release. One time we only discovered an issue after a student installed the latest version of something, and the setup instructions didn't work. Oh, great, version 10 breaks things compared to version 9. I do NOT enjoy dealing with that kind of incompatibility, and permitting it every 1.5 years is a bad thing. And if compatibility is to be guaranteed, why bump the major version number? That's the whole point of the three-part version. ChrisA
Hello Wolfgang, On Sat, 4 Nov 2017 12:25:57 +0100 (CET) tds333@mailbox.org wrote:
There has been ample discussion in the past about changing our release cycle one way or another. In short, the meta-problem is there are many contradicting interests which would each favour a different solution to the problem. See for example this PEP (ultimately rejected): https://www.python.org/dev/peps/pep-0407/ and the discussion that ensued: https://mail.python.org/pipermail/python-dev/2012-January/thread.html#115591 I haven't read your proposal in detail, but I suspect that it may be vulnerable to some of the same objections. The big objection being that a significant part of our ecosystem (that is, to put it roughly, the more corporate-minded segment, though I believe it is a simplification and may include other segments, such as Debian stable users and maintainers) doesn't want to deal more frequent feature releases. Regards Antoine.
On 04.11.2017 14:29, Antoine Pitrou wrote:
I read this PEP and some of the discussion. The difference to my idea is not to propose a LTS version and feature preview releases. The simplest form of my change is to switch from today major.minor to simply major for the feature release cycle. Additional minor feature releases for the standard library can also be optional or limited to provisional parts of the standard library. In all cases they should be backward compatible. But have the option to change the Python parts faster than the core. My idea came up because I have seen the need to improve the standard library in a faster pace than the core language. And to show this also in the version number. Hopefully ending the if it is included into the Python standard library it will be dead. Resulting in more releases but for the core with the same release cycle as of now. The amount of minor releases is not high it will be kept lower as it is now. Another point is to show by a higher major version number how mature Python is. (marketing) ;-) For Linux distributions there is still the option to include a specific Python Version with a feature set and recommend it. If someone wants to be conservative it is save to rely on features only for the first major version. This will be the same as of now for major.minor. Regards, Wolfgang
On 5 November 2017 at 00:40, Wolfgang <tds333@mailbox.org> wrote:
We're currently more likely to go the other direction, and stick with the 3.x numbering for an extended period (potentially reaching 3.10, 3.11, 3.12, etc), so that the ongoing disruption caused by the 2.x -> 3.x transition has had a chance to settle down before we ask anyone to start calling anything "Python 4". While the problems Linux distros are facing with whether "python" should refer to "python2" or "python3" are at least arguably self-inflicted (especially for those that already dealt with the Python 1.5.2 -> Python 2.0 migration back in the early 2000s), nobody is really looking forward to having to figure out how to adjust to a potential future Python 4.0 that invalidates all the current "python3" based naming schemes that have been introduced to work around the fact that Python 3.x needed to be parallel installable with Python 2.x without breaking any existing Python 2.x applications. We're not even sure yet when we're going to update PEP 394 to say that we think it's reasonable for distros to start using "python" to mean "python3" (see https://www.python.org/dev/peps/pep-0394/ and https://github.com/python/redistributor-guide/issues/1 for more discussion on the latter point). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Nick, On 04.11.2017 16:01, Nick Coghlan wrote:
A good point but two digits minor version numbers have the possibility to break a lot code. There is a lot of stuff out where a single digit major version is assumed. Even the official Python build for windows with python27.dll, python36.dll can be problematic because the dot is omitted between numbers. Other do the same for compilation they concatenate only majorminor for a name. Then version 3.10 is the same as 31.0. Ok I know this will take a while. But for the first 2.7.10 version even there some other library code was broken because of such assumptions.
I know on some build computers I have the same problem. But solved it for my scripts by using the major number as compatibility specifier and it works fine for me. Also knowing it is not upward compatible. The whole problem can only be solved by something used for Windows where the Python launcher deals with this. A Python launcher (py) for Linux can solve this there. Allowing also distributions to configure a fall back. For example the user specified python3 but I know it will be compatible with python4 so the launcher can do something like python3 is not found then I use python4 if available. ex.: #!/usr/bin/env py -3 And handle it as configurable version hint. Or even allowing alternative implementation like pypy if compatible. Regards, Wolfgang
On Sun, Nov 5, 2017 at 2:29 AM, Wolfgang <tds333@mailbox.org> wrote:
Python 1.0 was released in 1994. Then 2.0 came out in 2000 (six years), and 3.0 in 2008 (eight years). So far, we've been nine years into 3.0 and aren't looking at 4.0 yet, so it's going to be at least ten. If version 5.0 is another twelve years after that, and version 6.0 is fourteen later, we'll be looking for version 31.0 some time around the year 3000. I think we can let a future generation worry about pathing problems with DLL names. Seriously, I don't think there's any need to stress about version 31 of something as stable as Python. No matter how long it *actually* is between versions, it's going to be a lot longer than we can really plan for right now. In that much time, *anything* could change. ChrisA
Isn't this recent bit of discussion an argument in favor of a new stdlib module `version`? That would contain things like resolving a version tuple to an executable name (or a cross-platform piece of an executable name)? Obviously this doesn't actually answer the question of how naming should be done in the future, but it does mean that (new) code will have the actual choice abstracted away. --Josh On Mon, Nov 6, 2017 at 4:20 AM Stephan Houben <stephanh42@gmail.com> wrote:
On 6 November 2017 at 22:19, Stephan Houben <stephanh42@gmail.com> wrote:
Ah, you're right, I forgot about that option (I think Ezio Melotti suggested it previously). Yes, going with "3.10", but encoding it as "3A" in lexically ambiguous contexts is another option that would let us get as far as 3.35 (aka "3Z") before encountering ambiguity problems. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 5 November 2017 at 01:29, Wolfgang <tds333@mailbox.org> wrote:
Aye, we're aware :) We're not in a situation where we'll have any *good* options available, so the question we're considering is "What do we think will be the least bad approach?". At our current release cadence, 3.7 will be in mid 2018, 3.8 in late 2019 or early 2020, and then 3.9 in mid 2021. The open question will be what we call the release after that (in late 2022), with the main leading contenders being "4.0" (on the grounds that so many changes will have accumulated since 2008's 3.0 release by then that it makes sense to call them different major versions, similar to the rationale the Linux kernel now uses for major version updates), and "3.10" (on the grounds that the release won't be any more different from 3.9 than 3.9 will be from 3.8). Other variants (like making the major release number "40" or "2022", and using the minor version field for some new purpose other than indicating compatibility breaks in the compiler AST, interpreter opcode format, code caching tags, and C ABI), tend to introduce the same pragmatic questions that will already need to be considered in choosing between the 3.10 and 4.0 alternatives. It's not just the version numbering aesthetics that need to be considered either - in addition to the point you raise around how the Python version gets embedded in file path names (such that the 3-digit form is technically ambiguous without a separator, and may break parsers that are expecting exactly two digits), there are other backwards compatibility concerns around how folks do version compatibility checks based on sys.version and sys.version_info. Python and CPython will be more than 3 decades old by 2022, and an ecosystem can build up a *lot* of implicit assumptions regarding how a platform's versioning scheme works in that kind of time frame :) Personally, I doubt we'll make a firm decision on how we're going to handle this until some time after 2.7 goes EOL in 2020, as by then we'll necessarily have a better idea of how the various Linux distros (including the LTS commercial ones) ended up handling the Python 2 -> Python 3 switch, and the fact that the next release at that point will be 3.9 making it eminently clear that we've run out of time to continue postponing the question. We'll hopefully also have more experience with folks relying on the stable runtime ABI, and hence whether or not it might be beneficial to define new "py4" and "abi4" compatibility tags for the binary wheel distribution format. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
I predict by then that Larry will have finished the gilectomy, and while it will require backward incompatible changes to the C API, Python will run so blindingly fast in the quantum computing machine learning blockchain in the IOT cloud, bumping to 4.0 will be a no brainer[*]. Cheers, -Barry [*] Plus, our AI overlords will have made the decision for us. P.S. I suck at predictions.
2017-11-05 21:02 GMT+01:00 Serhiy Storchaka <storchaka@gmail.com>:
But len(os.fsencode("python3⑽.dll")) != len(os.fsencode("python39.dll")).
I think it is on Windows. os.fsencode should use UTF-16 there. ⑽ is in the BMP Presumably, non-Windows platforms wouldn't be interested in .dll files anyway... Stephan
Just to clarify: Python 2.0 was called 2.0 because the BeOpen marketing department thought it was good idea, not because there were major incompatible changes going into that release. Porting code from Python 1.5.2 to 2.0 was relatively straight forward and not much different from other minor releases. The first ever major backwards incompatibility release switch we had in Python after the great renaming of the C APIs between Python 1.1 and 1.2 (which was only visible to C extensions and relatively easy to fix using a compatibility header file), was the transition from Python 2.x to Python 3.x. IMO, the only reason to do this again would be the removal of the GIL, but only if there's absolutely no other way to get around such breakage. I would very much appreciate Python's releases not to introduce such major changes anymore. A continuous gradual change is a lot better to handle than non-continuous breaks and I'm pretty sure both will get us to the same level functionality eventually, perhaps even avoiding a few new mistakes along the way. With that approach, the versioning scheme would just be an implementation detail, which is good. Whether we call it Python 4.2 or Python 42 is really not all that important. Python is mature enough to not have to pull marketing tricks anymore. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 04 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 04.11.2017 16:44, M.-A. Lemburg wrote:
Thanks for the interesting details. Have to admit I entered the Python land with version 1.4 and missed the C API change from 1.1 to 1.2 completely. :-) Getting more opinions about the idea confirms me slowly to something like, good to discuss but not worth the change. Because some arguments confirmed me that it is not worth to change this. Also the marketing argument is weak and you are right that it should not care us. The only remaining argument is to have shorter release cycles but I think they don't depend on the versioning scheme nor a change in it improves this. Looking forward to 2020 when 2.7 is out of business and the maintenance cost can be invested in Python 4 and a faster release schedule. :-) Regards, Wolfgang
On Sat, Nov 4, 2017 at 10:44 AM, M.-A. Lemburg <mal@egenix.com> wrote:
Alternative history question: if it was just 1.6, then would Python 3000, probably called "Python 2000/2.x" in that world, with all the fixes have happened sooner because you would have "ran out" (yes, v1.10 > v1.9) of numbers leading up to 2?
But of course we'll skip Python 9, just like iPhones and Windowses ;)
On 04.11.2017 20:33, Nick Timkovich wrote:
There was a Python 1.6, but this was cut during the Python 2.0 release cycle to make CNRI happy. It included everything the CNRI team had developed after Python 1.5.2 before moving on to BeOpen to form the "PythonLabs": https://www.python.org/download/releases/1.6/ See "Overview of Changes Since 1.6" in https://www.python.org/download/releases/2.0/ for a list of changes between 1.6 and 2.0. The Python 1.6 never went into wide distribution, so for all practical purposes the move was from Python 1.5.2 to Python 2.0. More on all this is available in the "What's New in Python 2.0" document: https://docs.python.org/3/whatsnew/2.0.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 05 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Nick Timkovich wrote:
Alternative history question: if it was just 1.6
Guido's time machine strikes again. There was both a Python 1.6 and a 1.6.1. https://www.python.org/download/releases/1.6/ https://www.python.org/download/releases/1.6.1/ The "Contractual Obligation" releases. (And another Monty Python's Flying Circus reference.) These resolved licensing and release issues related to the BeOpen adventure and GPL-compatibility. Cheers, -Barry
M.-A. Lemburg wrote:
Fond memories of some of my first contributions to Python: http://python-history.blogspot.com/2009/03/great-or-grand-renaming.html Cheers, -Barry
On 4 November 2017 at 23:29, Antoine Pitrou <solipsis@pitrou.net> wrote:
PEP 413 was another competing proposal from around the same time: https://www.python.org/dev/peps/pep-0413/ Technically neither proposal has been formally Rejected, but they're also Deferred/Withdrawn because we knew that if we *had* asked for an explicit determination, the answer would have been "No, too high a cost, for not enough of a demonstrated benefit" (and that hasn't really changed in the past 5 years).
It's not just redistributors that get affected - it's the compatibility testing matrices for community projects as well. With our current release cycles (and excluding the 2.7 LTS release from consideration) we tend to have the following situation for the 3.x branches: - 1 actively maintained version - 2 or 3 security-fix only branches This stems from the default commitment of security fixes ending around 5 years after an X.Y.0 release, and releases happening 18-24 months apart: * Month 0: X.Y.0 released * Month 18: X.(Y+1).0 released * Month 36: X.(Y+2).0 released * Month 54: X.(Y+3).0 released * Month 60: X.Y.Z goes end-of-life * Month 72: X.(Y+4).0 released That same pattern also held for the 2.x series until the 2.7 release in 2010 (and it only changed then because that was the last 2.x feature release). The current year or so where the number of supported branches drops down to only 3 means we have some scope to speed up the rate of feature releases a bit (i.e. a release cadence of every 15 months instead of every 18 still means the community compatibility testing matrix consistently stays around 4 versions, or 5 if you're also testing against the dev release), but going to 6 monthly version updates means folks will either keep their test matrices the same as they are now (increasing the odds of the standard-library-only updates breaking things and that not being detected for some time), or else feeling obliged to expand their test matrices to cover even more concurrently supported versions. There are potential ways around both of those problems, but they require making the defined support periods for minor releases significantly shorter, unless they're the last minor release before the next major release (which means that instead of being simpler, these kinds of arrangements tend to end up being *more* complicated than the "every feature release receives security updates for 5 years" status quo). So rather than making fundamental changes to the way Python is versioned just to make standard library updates more readily available on older base versions of Python, we've instead gone with a more needs driven approach: making backport modules available for older versions where we find it is beneficial to do so (e.g. contextlib2, unittest2, importlib2). While Python 2.7 is a major driver for that change in practice, the benefits apply to older 3.x releases as well: - the updated modules are available for all Python versions and implementations where the packaging tools work (which is most of them) - users can decide on a module-by-module basis whether they want the baseline version or fast track updates - those decisions are explicitly tracked in the project's dependency metadata, rather than being implicit in the choice of deployment platform Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Nick, On 04.11.2017 15:48, Nick Coghlan wrote:
This is a valid concern. So more pressure by more releases can be a problem. For my idea the only possibility to limit this is to have minor versions only for the provisional parts to allow improvements and to have the major version track the core features and the bug fixing only for the first major version. Resulting in something like 4.0.0, 4.0.1 and 4.1.0 for something like a feature update for provisional but not supported with bug fixing and not for compatibility testing. This is then something like PEP-407 which is rejected. And also makes things little more complicated. Another possibility is to change only the versioning to major.minor instead of major.minor.patch. Then having a simpler versioning scheme for other Python implementations as only benefit (and the simplification to spell compatibility). Thus major is for the compatibility and minor counts the bug fix releases. Thinking if a change then is needed. :-) Marketing benefits? Don't know then. Regards, Wolfgang
On Sat, 4 Nov 2017 16:10:32 +0100 Wolfgang <tds333@mailbox.org> wrote:
I think the versioning scheme is a red herring here. The important topic is when Python releases are cut and what kinds of changes they are made of.
Thinking if a change then is needed. :-) Marketing benefits? Don't know then.
I don't think there are any marketing benefits. That might have been true 10 years ago, but nowadays people are so used to staggering increases in version numbers that they are not impressed anymore. Besides, IMHO Python is well past the point where it needed a concerted marketing effort to promote itself. Regards Antoine.
On 04/11/2017 13:29, Antoine Pitrou wrote:
Another minus point from my prospective - I work for a huge corporate organisation, (~300k employees), and we have to seek approval from the legal departments, on a per division basis, before we can use any Open Source software, (I and several others know that the per division bit is a ridiculous overhead and we are trying to get it addressed). The approvers will let us get away with saying that we need approval to use version major.X, i.e. Python 3.x has been approved, but will not let us get away with requesting to use a tool without specifying the major version number. I suspect that the reasoning behind this is that most packages rightly consider a licence change a major revision. So for us this suggestion would result in us having to re-seek approval every 12-18 months. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com
participants (16)
-
Antoine Pitrou -
Barry Warsaw -
Brice Parent -
Chris Angelico -
Guido van Rossum -
Joshua Morton -
M.-A. Lemburg -
MRAB -
Nick Coghlan -
Nick Timkovich -
Random832 -
Serhiy Storchaka -
Stephan Houben -
Steve Barnes -
tds333＠mailbox.org -
Wolfgang