[Python-Dev] PEP 407 / splitting the stdlib

[Brett Cannon]

On Wed, Jan 18, 2012 at 09:08, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Wed, Jan 18, 2012 at 10:30 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > Splitting the stdlib:
> > - requires someone to do the splitting (highly non-trivial given the
> > interactions of some modules with interpreter details or low-level C
> > code)
> > - requires setting up separate resources (continuous integration with N
> > stdlib versions and M interpreter versions, for example)
> > - requires separate maintenance and releases for the stdlib (but with
> > non-trivial interaction with interpreter maintenance, since they will
> > affect each other and must be synchronized for Python to be usable at
> > all)
> > - requires more attention by users since there are now *two* release
> > schedules and independent version numbers to track
>
> Did you read what I actually proposed? I specifically *didn't* propose
> separate stdlib releases (for all the reasons you point out), only
> separate date based stdlib *versioning*. Distribution of the CPython
> interpreter + stdlib would remain monolithic, as it is today. Any
> given stdlib release would only be supported for the most recent
> language release. The only difference is that between language
> releases, where we currently only release maintenance builds, we'd
> *also* release a second version of each maintenance build with an
> updated standard library, along with an alpha release of the next
> language version (with the last part being entirely optional, but I
> figured I may as well make the suggestion since I like the idea to
> encourage getting syntax updates and the like out for earlier
> experimentation).


> When you initially pitched the proposal via email, you didn't include
> the "language moratarium applies to interim releases" idea. That one
> additional suggestion makes the whole concept *much* more appealing to
> me, but I only like it on the condition that we decouple the stdlib
> versioning from the language definition versioning (even though I
> recommend we only officially support very specific combinations of the
> two). My suggestion is really just a concrete proposal for
> implementing Ezio's idea of only bumping the Python version for
> releases with long term support, and using some other mechanism to
> distinguish the interim releases.
>

IOW we would have a language moratorium every 2 years (i.e. between LTS
releases) while switching to a 6 month release cycle for language/VM
bugfixes and full stdlib releases? I would support that as it has several
benefits from several angles.

>From a VM perspective, it gives other VMs 2 years to catch up to the next
release instead of 18 months; not a big switch, but still better than
shortening it.

It also makes disruptive language changes less frequent so people have more
time to catch up, update books/docs, etc. We can also let them bake longer
and we all get more experience with them.

Doing a release every 6 months that includes updates to the stdlib and
bugfixes to the language/VM also benefits other VMs by getting
compatibility fixes in faster. All of the other VM maintainers have told me
that keeping the stdlib non-CPython compliant is the biggest hurdle. This
kind of switch means they could release a VM that supports a release 6
months or a year after a language change release (e.g. 1 to 2 releases in)
so as to get changes in faster and lower the need to keep their own fork.

It should also increase the chances of external developers of projects
being willing to become core developers and contributing their project to
Python. If they get to keep a 6 month release cycle we could consider
pulling in project like httplib2 and others that have resisted inclusion in
the stdlib because painfully long (for them) wait between releases.


>
> So, assuming a 2 year LTS cycle, the released versions up to February
> 2015 with my suggestion would end up being:
>
> >From the default branch:
> Python 3.3.0 + stdlib 12.08.0  (~August 2012)
> Python 3.4.0a1 + stdlib 14.08.0a1  (~February 2013)
> Python 3.4.0a2 + stdlib 14.08.0a2 (~August 2013)
> Python 3.4.0a3 + stdlib 14.08.0a3  (~February 2014)
> Python 3.4.0a4 + stdlib 14.08.0a4  (~2014)
> Python 3.4.0b1 + stdlib 14.08.0b1  (~2014)
> Python 3.4.0b2 + stdlib 14.08.0b2  (~2014)
> Python 3.4.0c1 + stdlib 14.08.0c1  (~2014)
> Python 3.4.0 + stdlib 14.08  (~August 2014)
> Python 3.5.0a1 + stdlib 16.08.0a1  (~February 2015)
>
> >From the 3.3 maintenance branch (these are maintenance updates to the
> "LTS" release):
> Python 3.3.1 + stdlib 12.08.1  (~February 2013)
> Python 3.3.2 + stdlib 12.08.2  (~August 2013)
> Python 3.3.3 + stdlib 12.08.3  (~February 2014)
> Python 3.3.4 + stdlib 12.08.4  (~August 2014) (and 3.3 branch enters
> security patch only mode)
>
> >From the 3.4 maintenance branch (these are maintenance updates to the
> "LTS" release):
> Python 3.4.1 + stdlib 14.08.1  (~February 2015)
>
> >From the stdlib feature development branch (these are the new interim
> releases with standard library updates only as proposed by PEP 407):
> Python 3.3.1 + stdlib 13.02.0  (~February 2013)
> Python 3.3.2 + stdlib 13.08.0  (~August 2013)
> Python 3.3.3 + stdlib 14.02.0  (~February 2014) (only upgrade path
> from here is to make the jump to 3.4.0)
> -- 3.4.0 + 12.08.0 is released from default branch --
> Python 3.4.1 + stdlib 15.02.0  (~February 2015)
>
> If we have to make "brown paper bag" releases for the maintenance or
> stdlib branches then the micro versions get bumped - the date based
> version of the standard library versions relates to when that
> particular *API* was realised, not when bugs were last fixed in it. If
> a target release date slips, then the stdlib version would be
> increased accordingly (cf. Ubuntu 6.06).
>
> Yes, we'd have an extra set of active buildbots to handle the stdlib
> branch, but a) that's no harder than creating the buildbots for a new
> maintenance branch and b) the interim release proposal will need to
> separate language level changes from stdlib level changes *anyway*.
>
> As far as how sys.version checks would be updated, I would propose a
> simple API addition to track the new date-based standard lib
> versioning: sys.stdlib_version. People could choose to just depend on
> a specific Python version (implicitly depending on the stdlib version
> that was originally shipped with that version of CPython), or they may
> instead decide to depend on a specific stdlib version (implicitly
> depending on the first Python version that was shipped with that
> stdlib).
>
> The reason I like this scheme is that it allows us (and users) to
> precisely track the things that can vary at the two different rates.
> At least the following would still be governed by changes in the first
> two fields of sys.version (i.e. the major Python version):
>    - deprecation policy
>    - language syntax
>    - compiler AST
>    - C ABI stability
>    - Windows compilation suite and C runtime version
>    - anything else we decide to link with the Python language version
> (e.g. default pickle protocol)
>
> However, the addition of date based stdlib versioning would allow us
> to clearly identify the new interim releases proposed by PEP 407
> *without* mucking up all those things that are currently linked to
> sys.version and really *shouldn't* be getting updated every 6 months.
> Users get a clear guarantee that if they follow the stdlib updates
> instead of the regular maintenance releases, they'll get nice new
> features along with their bug fixes, but no new deprecations or
> backwards incompatible API changes. However, they're also going to be
> obliged to transition to each new language release as it comes out if
> they want to continue getting security updates.
>
> Basically, what it boils down to is that I'm now +1 on the general
> proposal in the PEP, *so long as*:
> 1. We get a separate Hg branch for "stdlib only" changes and default
> becomes the destination specifically for "language update" changes
> (with the latter being a superset of the former)
> 2. The proposed "interim releases" are denoted by a new date-based
> sys.stdlib_version field and sys.version retains its current meaning
> (and slow rate of change)
>
>
I don't think we need to do a new versioning scheme. Why can't we just say
which releases are covered by a language moratorium? The community seemed
to pick up on that rather well when we did it for Python 3 and I didn't see
anyone having difficulty explaining it when someone didn't know what was
going on. As long as we are clear which releases are under a language
moratorium and which one's aren't we shouldn't need to switch to language +
stdlib versioning scheme. This will lead to use reaching Python 4 faster
(in about 4 years), but even that doesn't need to be a big deal. Linux
jumped from 2 to 3 w/o issue. Once again, as long as we are clear on which
new versions have language changes it should be clear as to what to expect.

Otherwise I say we just  bump the major version when we do a
language-changing release (i.e. every 2 years) and just to a minor/feature
number bump (i.e. every 6 months) when we add/change stuff to the stdlib.
People can then be told "learn Python 4" which is easy to point out on
docs, e.g. you won't have to go digging for what minor/feature release a
book covers, just what major release which will probably be emblazoned on
the cover. And with the faster stdlib release schedule other VMs can aim
for X.N versions when they have all the language features *and* all of
their compatibility fixes into the stdlib. And then once they hit that they
can just continue to support that major version by just keeping up with
minor releases with compatibility fixes (which buildbots can help
guarantee).

And honestly, if we don't go with this I'm with Georg's comment in another
email of beginning to consider stripping the stdlib down to core libraries
to help stop with the bitrot (sorry, Paul). If we can't attract new
replacements for modules we can't ditch because of backwards compatibility
I start to wonder if I should even care about improving the stdlib outside
of core code required to make Python simply function.

-Brett



> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120118/54700086/attachment.html>