Add from __experimental__ import bla [was: Should we move to replace re with regex?]

In the thread about replacing re with regex someone mentioned adding to __future__ which isnt a great idea as future APIs are already solidified, they just live there to give developer time to adapt their code. The idea of a __experimental__ area is good for any pep's or stliib additions that are somewhat controversial (API isnt agreed on, code may take a while to integrate properly, developer wants some time to hash out any edge case bugs or API clarifications that may come up in large scale testing, etc). __experimental__ should emit a warning on import that says anything in here may change or be removed at any time and should not be used in stable code. __experimental__ features should behave the same as __future__ in that they can add new keywords or semantics to the existing language __experimental__ features can move directly to the stlib or builtins if they do not add new keywords and/or are backwards compatible with the feature they are replacing. Otherwise they move into __future__ for how ever many releases are deemed reasonable time for developers to adapt their code.

On Sat, Aug 27, 2011 at 3:57 PM, Dj Gilcrease <digitalxero@gmail.com> wrote:
If something's still experimental, why ship it as stdlib? Why not just keep it third party until integration? No reason to tempt people to do anything that needs a warning. If they want some software, they can install it. Mike

On Sat, Aug 27, 2011 at 7:50 PM, Mike Graham <mikegraham@gmail.com> wrote:
Putting it in the stdlib labeled as experimental would send a signal that it's slated for stdlib inclusion (in this case to replace the re module) and makes it available to everyone with the right Python version. Keeping it third-party means many people will be reluctant to add it as a dependency to any code they put out. That said, I'm not all that keen on my __experimental__ idea. The point of __future__ was that it would be recognized by the parser as it was parsing the file, so it could then modify its parsing tables on the fly. That's not needed for an experimental module. Telling people to use "import regex as re" is probably good enough, as we go that route. But personally, I'd much rather see either the existing re module upgraded, or the regex module replace the re module as of Python 3.3. Either of those sounds like a better solution for users of Python 3.3. But I realize it's more work for the core developers. -- --Guido van Rossum (python.org/~guido)

On Sun, Aug 28, 2011 at 2:35 PM, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
Indeed. And using a DVCS means it is easier these days for people to get hold of experimental code (e.g. anyone that wants to play with the yield from expression can grab it from bitbucket and build it locally: https://bitbucket.org/ncoghlan/cpython_sandbox#pep380) Documenting that packages are up for standard lib inclusion is good (specifically regex right now, but likely a couple of others before 3.3), but I don't think that means actually *shipping* them in the stdlib is a good idea. A meta-package on PyPI for "stdlib-experimental" might be a way to let people grab candidates all at the same time if it's considered worthwhile, but the stdlib itself should only get stuff we plan to keep around. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Sun, Aug 28, 2011 at 12:50 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I still see a huge difference between something in PyPI and something marked as experimental in the stdlib. Something in PyPI may or may not be promoted to stdlib status, but realistically that rarely happens (I think the last time it happened it was a JSON thing?). Something marked as experimental in the stdlib has a much higher likelihood to be promoted to regular stdlib status -- in fact that's the most likely outcome. The main difference between experimental and regular status in the stdlib is that for experimental modules we reserve the right to make incompatible changes in subsequent versions (possibly even in bugfix releases, although that's worth a separate discussion). An experimental feature in the stdlib also signals a certain commitment from the core developers. We shouldn't use experimental as a dumping ground for random stuff; that's what PyPI is for. We should use experimental within the context of the stdlib. Another thing to keep in mind is that not everybody has the same attitude towards installing 3rd party dependencies from PyPI. E.g. at Google, that requires someone to import the 3rd party library into our own code repo first, and that in turn requires a (light-weight) review. The net effect is that a Googler who happens to be the first to want to use a certain 3rd party package has to do a lot of work; it's *not* just a matter of downloading and installing the package. I imagine it's the same within many other organizations -- some better, some worse. (I was going to write something about many package developers being reluctant to require dependencies on other packages, but that's a pretty weak argument, since depending on an experimental stdlib package would limit them to a specific Python version, or dynamic presence-checking which isn't much better than having optional PyPI dependencies.) The power of "batteries included" is still with us, and experimental batteries are still batteries! -- --Guido van Rossum (python.org/~guido)

On Sun, 2011-08-28 at 10:10 -0700, Guido van Rossum wrote:
It looks to me, that Importing from __future__ can do things that a regular import can't do, like add or change core syntax. So they tend to be python core features rather than library features. The term __experimental__ may have differing expectations to different people and may be a little too broad. I like __future__ for *early* core changes, and __lib_future__ or __sdtlib_future__, for early stdlib features. I describe them as "early" because they either aren't quite ready for the current release cycle or they are waiting for something else to be depreciated or done before they can included. I think It makes sense to split core and library future features this way because usually, future core features are built into the core, while a future standard library module can be imported from the disk. Cheers, Ron

On Mon, Aug 29, 2011 at 8:05 AM, ron3200 <ron3200@gmail.com> wrote:
If we want to, we can make __experimental__ just as magical as __future__. That's orthogonal to its semantics in the library case, which is the point of interest at the moment. If people fear relying on experimental features, that's OK - if this idea happens at all, allowing people that aren't prepared to cope with the risk of breakage to easily avoid it would be a key benefit. The essential idea here is to be able to add a feature, but flag the API as potentially unstable for an entire release cycle before promising to maintain the API in perpetuity. Early adopters would get *most* of python-dev's usual guarantees (which most PyPI packages don't offer), while more conservative types can use more stable alternatives. The benefit on our side is that we'd get an full 18-24 months of feedback on a feature from a broad audience before finally locking the API down in the subsequent release. The more I think about this idea, the more I think it could help alleviate some of the concerns that can currently hold up the evolution of the standard library. (The costs of making a mistake with standard library API design are rather high due to the backwards compatibility promise, which means iterative design mostly needs to happen *before* the module is included. PyPI and the PEP process are both useful for that, but they don't really compare to the feedback available from being part of a full standard lib release). We'd still need to be careful not to throw any old rubbish in there, but as a phased introduction for things like "from __experimental__ import ipaddr" (PEP 3144) and "from __experimental__ import re" (re replacement with regex, assuming someone steps up to write the PEP and eventual drop-in replacement looks feasible), the idea definitely has potential.
No, that's *not* what __future__ means. __future__ changes are exactly what will become default behaviour in a future version - they aren't going to change, so code that uses them won't need to change. What may need changing in the case of __future__ is *old* code - the only reason we use a __future__ flag is when old code might break. The semantics of the new marker package would be to indicate that stuff is a little undercooked, but we've decided that it won't get enough exposure through PyPI (either through being too small to overcome NIH syndrome or else too closely coupled to a specific version of the interpreter). That's a completely different meaning, so we shouldn't reuse the same word. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Mon, 2011-08-29 at 21:14 +1000, Nick Coghlan wrote:
How often do you think things in __experimental__ will be aborted or canceled? Another way to think about "experimental" items is to use it in the way that science does. ie... An experiment designed to discover information, or test a very specific idea or theory. In that context, depending on the results of the experiment, a particular feature my be changed to take the results of the experiment into account. That's probably a bit too formal and restrictive, and it will be difficult to do in a very controlled way. The python community, and Python itself, tends to thrive in a more relaxed and informal atmosphere. But it may not hurt to have some guide lines on how to do a python "__experimental__" experiment so that it doesn't end up being a series of trial and error attempts at something half baked. Which do you think fits better with what you have in mind? __experimental__ __pre-view__ Cheers, Ron

On Mon, Aug 29, 2011 at 4:14 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
No, that's *not* what __future__ means. __future__ changes are exactly
Unlike __future__ the API is unstable and therefore code that works with today's __experimental__ may fail tomorrow. If the API changes in an incompatible way, we would probably prefer that the import fail rather than succeed with random results. Therefore, I would propose a mechanism to support this, perhaps something like from __experimental__ import regex {2,3,4} as re which means import the regex module version 2, 3 or 4. A later version will be imported only if it is an expansion, i.e., compatible with all previous APIs and differs only in containing new features. If the version set is omitted from the import it's treated as {1} so that dealing with this is only necessary for those modules which do change in incompatible ways. In general, checking for specific capabilities rather than version numbers is more robust but in this case, perhaps multiple versions of experimental APIs will be discouraged making that less necessary. But we could allow strings specifying required features in the version set. Note that the reason for allowing import with more than one version number is to handle the case when a new version is available but the differences are in parts of the API that don't impact the application. --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com

On Mon, 2011-08-29 at 18:22 -0700, Bruce Leban wrote:
Encouraging limited use of __experimental__ may be good. If it looks like there are going to be more than two three different combinations, then it is likely too soon to even be in __experimental__. It probably needs more thought still. One possibility would be to have experimental features be of the form, which is better? (a or b). Then you could try things out and save timings and test results in a fairly straight forward way. results = {} for x in 'ab': from __experimental__[x] import regex as re <do stuff with re and save some data about it> for x in 'ab': print(results[x]) I can think of a few issues with this. Like how long do things live in __experimental__? Can the letters be reused, or should new letters be used each time? Is "from __experimental__[x] import feature" better or worse than "from __experimental__ import feature[x]"? And do subscripts even work in import statements? Then again, maybe version #'s would work better? Having a way to ask __experimental__ what is has and use that to do the importing could make doing some things easier. Cheers, Ron

I'm sorry, the idea was never to add extra syntax to "from __experimental__". It was supposed to be just a plain import -- in fact plainer than "from __future__". There is hardly a need to use the double underscores; "from experimental" should work just as well. Why? It wasn't meant to be a language change. Just a convention to set aside an experimental namespace in the stdlib. If we're going to attempt to solve the problem of versioned dependencies, I recommend doing it outside the language syntax, at the package installer level, sort of the way setuptools / distribute do it. If you want to explore that idea, please read up on how it's done there and if you have ideas for improvement, start a new thread here. -- --Guido van Rossum (python.org/~guido)

On Tue, 2011-08-30 at 09:47 -0700, Guido van Rossum wrote:
Why? It wasn't meant to be a language change. Just a convention to set aside an experimental namespace in the stdlib.
That sounds fine. I think if the release manager has final say on the content of experimental, (along with you of course), the rest will probably take care of itself.
Hopefully it won't ever get so large or complicated that that is needed. Cheers, Ron

On Tue, Aug 30, 2011 at 11:22 AM, Bruce Leban <bruce@leapyear.org> wrote:
I'd advise against trying to overengineer this. If we decide to go down the path of breaking imports, then we'd probably just do it via naming conventions (e.g. give all experimental modules a suffix based on the Python version where they were first included). However, if you're concerned about subtle breakage due to backwards incompatibilities, then that's a good sign that depending on explicitly experimental modules is a *bad idea*. Adding a package specifically for candidate standard library modules with APIs that aren't yet locked in would be easy and low impact. Messing with the compiler or interpreter to provide additional features would be much higher impact, and consequently a much harder sell. However, you've highlighted the major problem with the idea: just because we *say* that the package is experimental and APIs may change without (programmatic) warning, doesn't mean people will pay attention. There are also a whole host of serialisation problems that arise when dealing with relocated modules, which could cause issues with the eventual migration to the "real" module location. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

This feature reminds me of "staging" in the Linux kernel. I often hear complaints that Python's bringing too many features on board, trying to explain __experimental__, {major, minor, patch} does not seem desirable. Then there's the consideration that any and all stuff floating around in __experimental__ will be overly encouraged, because honestly how many modules won't end up making the cut? Why can't a special tag/section be added in PyPi to indicate that a module is being considered for inclusion in future versions of Python, after all, we're all friends here. This might also give the oppurtunity to showcase some of the newer packaging infrastructure, and how easy it makes it to install third party modules. (Disclaimer: I haven't looked at the new one and I detest the old one.) -1 for adding yet more clutter Also -1 for renaming re: Some people call regex, regexp, you'll only be offending their sensibilities and adding to the confusion. Furthermore, a lot of Python's competitors have builtin regex support, such as =~ in Perl and bash. The terser the syntax for regex, the more at home and less cluttered it will appear to those users. I recommend that if a new regex module is created, that the name regex be used then, and the old be left alone. On Tue, Aug 30, 2011 at 1:03 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

On Wed, Aug 31, 2011 at 12:03 PM, alex23 <wuwei23@gmail.com> wrote:
The key benefit lies in ensuring that anything in the mooted experimental namespace is clearly under python-dev's aegis from at least the following perspectives: - licensing (i.e. redistributed by the PSF under a Contributor Licensing Agreement) - testing (i.e. the module test suites are run on the python.org buildbot fleet and results published via http://www.python.org/dev/buildbot) - issue management (i.e. bugs and feature requests are handled on http://bugs.python.org) - source control (i.e. the master repository for the software is published on http://hg.python.org) Those are the things that will allow the experimental modules to be used under existing legal approvals that allow the use of Python itself (e.g. in a corporate or governmental environment). Handling the actual packaging for distribution is a somewhat orthogonal question. Whether we offer 3 installers (CPython core, standard library, experimental), 2 installers (CPython core + stdlib, experimental) or 1 monolithic installer, or a combination of those approaches, doesn't actually change the surrounding legal framework all that much. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Wed, Aug 31, 2011 at 8:05 AM, Barry Warsaw <barry@python.org> wrote:
I'm sorry, I don't follow. The experimental package would only contain code distributed as part of the stdlib, and the code put in the stdlib's experimental package would get the same care from the core developers as the rest of the stdlib. The only difference would be that we'd drop the guarantee that the APIs offered would still be present in the next release (i.e. from 3.3 -> 3.4; the guarantees would hold from 3.3.1 -> 3.3.2). I hope I'm not contradicting myself? -- --Guido van Rossum (python.org/~guido)

On Aug 31, 2011, at 09:29 AM, Guido van Rossum wrote:
As long as the "guarantees" only cover code distributed in the experimental package with Python, that's okay. Maybe I was reading too much into Nick's comment, but when third party code can situate itself under 'experimental', we obviously can't make those same guarantees of the code we don't distribute. -Barry

On Thu, Sep 1, 2011 at 11:26 AM, Barry Warsaw <barry@python.org> wrote:
That certainly wasn't part of *my* plan any more than letting 3rd parties install themselves anywhere else inside stdlib packages. On the other hand it probably wouldn't matter because the review process doesn't look at what you import -- it only looks at dependencies (i.e. stuff that needs to be downloaded). Whether it goes under experimental would be orthogonal -- the litmus test for "is this covered by the PSF" would be "do I have to install it separately" (when using a Python install built from source, so there are no packages pre-installed by the distro). -- --Guido van Rossum (python.org/~guido)

On Thu, Sep 1, 2011 at 1:05 AM, Barry Warsaw <barry@python.org> wrote:
PEP 402 doesn't really change anything regarding corporate checks of code provenance - those relate to how the code is acquired and installed in the first place, not how you use it at runtime. If engineers are doing end runs around corporate policies, that's not a problem that technology can realistically address :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

alex23 writes:
"Although practicality beats purity." As already mentioned several times in this thread, PyPI, Bitbucket, and hg.python.org sandboxes as-is provide a plenty good technical solution for distribution of experimental code which is almost certain to be included in the core distribution at some point, but currently the API bikeshed is being painted (well, it will be if we can only come to consensus on color!) The problem to be solved is on the other side of the network connection, internal to the *using organizations*. Some of them have much stricter rules for adding new "approved" packages than for upgrading existing ones. In that case, developers "inside" get much freer access to "official experimental" modules, and can participate in development (including objecting to API changes they consider gratuitous :-) in a way that is hard for them to justify if *before* dealing with any API changes they have to run an internal obstacle course just to be allowed to use the code. As I understand Guido's position, "experimental" is a non-starter unless it can be expected to significantly increase beta tests of such modules by developers inside organizations that strictly control their internal code libraries. This requires that the "experimental" modules be distributed with the interpreter. Of course, if the stdlib was separated out, and the current stdlib simply bundled with the interpreter at distribution time, the experimental package could be given the same treatment. But the stdlib hasn't been done yet, and I don't think something labelled "experimental" would have the same credibility with IT Security/QA as the rest of core Python. This last might kill the whole idea, as QA might take the position that "yes, you're just upgrading Python 2.7 from 2.7.4 to 2.7.5, but we have no idea what might be in experimental, so you're going to have to make a separate request for that." (I have never worked in such an organization so I don't know if that's a realistic worry or not.)

On Tue, Aug 30, 2011 at 8:23 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I wasn't actually proposing to add to (or even change the API of modules in) the experimental package during such micro-version updates. TBH the best use case I can think of would actually be the ipaddr package, which is somewhat controversial but not overly so, and seems to lack a push to ever get it accepted. Putting it in experimental for 3.3 would let us distribute it with Python without committing 100% to the solution it offers over its nearest competitor. However the downside is that that's a very long wait, still provides a pretty strong bias (unless we were to include both competing packages), and still doesn't look like it might get enough beta testing, unless the uptake of 3.3 is huge. So maybe we should just count PyPI downloads and decide that way. "50,000,000 Elvis fans can't be wrong." (An interesting meme by itself. :-) --Guido van Rossum (python.org/~guido)

On Aug 30, 2011, at 08:53 PM, Guido van Rossum wrote:
I think an experimental namespace is actually attacking the wrong problem, or maybe the right problem in the wrong way. ;) I'd much prefer to see some brainstorming on multi-version support, which is something that Robert Collins is always bugging me about. Something like pkg_resource's require() function seems like a good place to start. If we had this built-in, then including ipaddr in the stdlib as it currently stands would be a no-brainer, even with a suboptimal API. It would get lots of testing, and a completely different API could be designed for Python 3.4 without break packages that relied on the old API. Python's deprecation policy could be adjusted to include diminishing support for older versions of a module in the stdlib, and we'd avoid ugliness like unittest2 and such. -Barry

On Wed, Aug 31, 2011 at 8:19 AM, Barry Warsaw <barry@python.org> wrote:
It's a great idea to brainstorm about, just not in this thread. I see the two issues completely orthogonal.
You sound like you have a solution for using multiple versions of an API in the same program. Do you? Then out with it! Otherwise, no, it wouldn't be a no-brainer. -- --Guido van Rossum (python.org/~guido)

On Wed, Aug 31, 2011 at 9:56 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Yes, to some extent. My specific proposal is that experimental (by whatever name) would not change APIs in bugfix release, but would (potentially) change APIs in "major" releases (e.g. 3.3 -> 3.4(*)). (*) Can we pick a terminology so we all agree that "3.3.3" is a "minor release", "3.3" is a "major release", and "3" an "earthshattering release"? Or other terms -- but something that is both agreed upon and clear enough without explanation. I'm tired of having to clarify minor and major every time I use them out of fear they'll be mistaken for "3" and "3.3". -- --Guido van Rossum (python.org/~guido)

On 31 August 2011 18:33, Antoine Pitrou <solipsis@pitrou.net> wrote:
How about 3.3.3 -> 3.3.4 is a "minor" release, 3.3 -> 3.4 is a "feature" release and 3 -> 4 is not something we generally talk about (or "compatibility-breaking" or something like that). I suspect that "minor" for changing the last digit is pretty comprehensible, it's using "major" for 3.3 that confuses people, so let's avoid that term... Paul.

On Wed, Aug 31, 2011 at 10:51, Guido van Rossum <guido@python.org> wrote:
sys.version_info has already made the declaration of what each of the digits represent when it became a named tuple: major, minor, micro (http://hg.python.org/cpython/file/4dcbae65df3f/Python/sysmodule.c#l1273). And if you don't like the naming, then blame me; http://bugs.python.org/issue4285 was the bug that led to the names and they are what I have always used for all software. After that blame Eric Smith for committing the patch. =)

Another approach to __experimental__ could be a semi-official "python-experimental" package on pypi which contains all the stuff that's already been talked about. It's just one extra package to depend on and could follow its own release cycle. I don't know if this is better than including it in the stdlib (I like the idea of __experimental__ myself), but maybe its another idea to consider. David On Sep 1, 2011 8:40 PM, "Barry Warsaw" <barry@python.org> wrote:

The issue with that is that the "y" in 'Python x.y.z' is usually called the minor release (see: http://docs.python.org/devguide/devcycle.html ). Switching which digit the minor release is would be confusing when reading older documents. It might be better to instead keep the old name, "bugfix", since that isn't confusing. Then we would be left with "compatibility-breaking.feature.bugfix", which isn't really bad in any sense I can see (aside perhaps from being new and unusual?) Devin On Wed, Aug 31, 2011 at 1:45 PM, Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, Aug 31, 2011 at 10:23 AM, Guido van Rossum <guido@python.org> wrote:
Given that there's no general industry agreement on what those things mean, "clear enough without explanation" is probably unrealistic. On the other hand, a PEP or some similar document that lays out the terminology used for python releases (and encouraged for python libraries) would let you assume that python people had read that explanation, and make a reference to it easy (i.e."see PEP XXXX"). <mike

On Thu, Sep 1, 2011 at 3:36 AM, Mike Meyer <mwm@mired.org> wrote:
That PEP is PEP 101 (at least as far as Python releases go), and it is currently a little inconsistent (using major.minor.micro in some sections, as the devguide and sys.version_info apparently also do, but also referring to feature releases as major releases on at least one occasion). PEP 386 (which aims to standardise version numbering within the wider Python ecosystem) doesn't mandate a naming scheme (only a numbering one), but does refer to the major.minor.micro names in the explanatory text. Personally, I agree with others that "feature release" and "bug fix release" are as close as we're likely to get to unambiguous names for the way Python itself uses x.y and x.y.z release numbering. Creating a new version of the language itself is rare enough that I don't think it really needs a generic name. Although with the accumulated baggage that 3.0 swept away, "spring cleaning release" might be appropriate :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Guido van Rossum writes:
I'm not sure "earthshattering" is Pythonic enough.<duck /> If you're willing to go with something pedestrian, at least the Debian types, and I think Gentoo, use "epoch" to mean that upstream decided to redo its version numbering. In a sense, I think that applies to Python 3 (which has to be judged a success, since it induced Linus to go to Linux 3!) and Linux 3 for that matter. How about the leading digit being the "Pythonic epoch", and "Python 3 was an epochal release"? Or "epoch-making release"?

On Sep 01, 2011, at 05:43 PM, Stephen J. Turnbull wrote:
How about the leading digit being the "Pythonic epoch", and "Python 3 was an epochal release"? Or "epoch-making release"?
Or maybe: snake reference: skin shedding release funny brit reference: spanish inquisition release -Barry

How about irreversible-release.

On Thu, Sep 1, 2011 at 1:16 PM, Christopher King <g.nius.ck@gmail.com> wrote:
How about irreversible-release.
"Killer release". (An oblique reference to Monty Python's killer joke.) -- --Guido van Rossum (python.org/~guido)

On Aug 31, 2011, at 09:31 AM, Guido van Rossum wrote:
It seems related though. It seems like the primary use case for experimental, especially in reference to the ipaddr module, is "we're including this battery, but it's shape might change, so don't count on it fitting into your socket next release". With multi-version libraries, we don't have to be so wishy-washy. We could say "here's version 0.5 of ipaddr" and the next release could safely include both that version and a stable, completely redesigned API of ipaddr 1.0. The two could co-exist (for a while), supporting both the legacy API, and the new whizzy one we actually like. ;)
I wish I did! -Barry

On Thu, Sep 1, 2011 at 11:34 AM, Barry Warsaw <barry@python.org> wrote:
It still sounds like they are (nearly) orthogonal though. Even if we had such a feature we'd still want to put ipaddr 0.5 into experimental, but most other new stdlib modules would not have to go there. And even if we didn't have such a feature, I'd still consider putting things in experimental, and just telling people to deal with it. Especially since we *don't* have the feature. :-)
That's what I'm talking about! :-) -- --Guido van Rossum (python.org/~guido)

On Wed, Aug 31, 2011 at 1:23 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Yep, that's a nice explanation of the motivation here. People that can just grab a module from PyPI and run it already have plenty of opportunities to provide feedback on in-development APIs. The idea of adding a new namespace within the standard library would be to have an 18-24 month window to gather similar feedback from folks in environments where they *can't* just grab packages from PyPI to try out, but *can* try out the latest version of CPython itself. (I suspect we'd also have an easier time getting feedback from folks that *could* retrieve modules PyPI if they wanted to, but in practice *don't*).
It depends on the organization, but the main hurdle I experienced at my previous employer was the initial licensing and "development pedigree" approval process (to make sure that we were on solid legal ground in redistributing the component to our own customers and that we were prepared to shoulder the risk of latent defects - both bars were significantly lower if we added "for internal use only" to the review request). Due to the way the PSF and python-dev operate, CPython and the standard library check a lot of boxes in that kind of review that smaller projects often miss. Once a component was on the approved list, as long as the licensing didn't change, decisions about upgrades were then in the engineers' hands without needing to get the lawyers involved again. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Guido van Rossum writes:
I would think that most of the discussants already have this in mind. It comes up every time a module is proposed for inclusion in the stdlib, and presumably is the primary reason for supporting __experimental__.
But the friction imposed by this required is presumably considered a net good thing by Google, no? Some benefit is received, or some cost avoided, right? So does __experimental__ just reduce friction for carefully selected proposed modules, and so increase net benefit, or does reduced friction have adverse impacts (ie, side effects of too- easily imported modules on the rest of the organization) that might outweigh the reduction of friction, too?

On Mon, Aug 29, 2011 at 11:29 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I'd say if we went down this path, a key motivator would be specifically to make it easier for downstream organisations to point to it and say "this is covered by our existing license and packaging review for the Python standard library". ActiveState/Enthought/etc would ship it with their installers, Debian/Fedora/etc would package it, and so on and so forth. That's a harder case to make for arbitrary PyPI packages, since there's far more variation in the licensing and the code isn't tested on our buildbot suite or covered by our source control and review processes. "We vetted the licensing for valid PSF redistributions rights, combined its test suite with ours, and regularly run those tests on our buildbots" really *is* a significantly stronger statement on our part than "we're considering including this module, you can get it from PyPI". I'm actually coming around to the idea, especially for cases like the ones Guido mentioned where we think the functionality is definitely desirable but aren't sure we have the API right yet. Currently, in those cases we have to choose between publishing on PyPI and locking in (potentially for a very long time) an API design that we're not entirely sure is right. (Possible case in point: we ignored some of Glyph's advice in the design of the concurrent.futures API by having the executors directly manipulate the future objects instead of introducing an additional level of indirection so that the object the executors were playing with wasn't the one clients had a direct reference to. Even if this turns out to have been a mistake, the API is unlikely to change within the 3.x series). An explicit "__experimental__" namespace would be a way to promise "this functionality will exist in *some* form in future versions of Python, but the API may change in backwards incompatible ways in the process of getting there". It reminds me in some ways of the journey of the set type from its initial addition as "sets.Set" to the final version as the "set" builtin. ImportEngine (PEP published to import-sig, I'll push it to python.org soonish), for example, would be a prime candidate for experimental status in 3.3 - it's too closely coupled to the interpreter details to make sense to distribute separately, so it would be good to have a mechanism to distribute it in a form that would allow feedback to be gathered over a full release cycle before we commit to a final API in 3.4 (assuming the PEP is accepted at all, of course, although I'm fairly confident I can persuade people it's a good idea). Here's another potentially useful litmus test question for Guido: if it was spelled "from __experimental__ import ipaddr", would you be more inclined to approve PEP 3144 (IP address library) for 3.3? Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Sun, Aug 28, 2011 at 7:53 PM, Nick Coghlan <ncoghlan@gmail.com> wrote: [Lots of stuff I agree with, and then]
IIRC the issue with that PEP is that there is a stand-off between two authors of competing implementations, and I don't have enough understanding of the issues to be able to tell who's right. (Or maybe they're both right and they're just aiming at different audiences -- but I can't even tell that.) I think we need to have someone who cares more (but is not either of those authors) to review the PEP and its criticism and decide on a way forward. (Or am I being too soft? If nobody cares I'd be happy to toss a coin.) -- --Guido van Rossum (python.org/~guido)

On Mon, 29 Aug 2011 17:00:15 -0700 Guido van Rossum <guido@python.org> wrote:
That's my understanding as well. Also, I seem to remember at least one of the two implementations was criticized on some of its design decisions (was it the fact that networks and addresses used a common class? sorry if I'm making things up here), but the author didn't want to change the API. Regards Antoine.

On Sat, Aug 27, 2011 at 3:57 PM, Dj Gilcrease <digitalxero@gmail.com> wrote:
If something's still experimental, why ship it as stdlib? Why not just keep it third party until integration? No reason to tempt people to do anything that needs a warning. If they want some software, they can install it. Mike

On Sat, Aug 27, 2011 at 7:50 PM, Mike Graham <mikegraham@gmail.com> wrote:
Putting it in the stdlib labeled as experimental would send a signal that it's slated for stdlib inclusion (in this case to replace the re module) and makes it available to everyone with the right Python version. Keeping it third-party means many people will be reluctant to add it as a dependency to any code they put out. That said, I'm not all that keen on my __experimental__ idea. The point of __future__ was that it would be recognized by the parser as it was parsing the file, so it could then modify its parsing tables on the fly. That's not needed for an experimental module. Telling people to use "import regex as re" is probably good enough, as we go that route. But personally, I'd much rather see either the existing re module upgraded, or the regex module replace the re module as of Python 3.3. Either of those sounds like a better solution for users of Python 3.3. But I realize it's more work for the core developers. -- --Guido van Rossum (python.org/~guido)

On Sun, Aug 28, 2011 at 2:35 PM, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
Indeed. And using a DVCS means it is easier these days for people to get hold of experimental code (e.g. anyone that wants to play with the yield from expression can grab it from bitbucket and build it locally: https://bitbucket.org/ncoghlan/cpython_sandbox#pep380) Documenting that packages are up for standard lib inclusion is good (specifically regex right now, but likely a couple of others before 3.3), but I don't think that means actually *shipping* them in the stdlib is a good idea. A meta-package on PyPI for "stdlib-experimental" might be a way to let people grab candidates all at the same time if it's considered worthwhile, but the stdlib itself should only get stuff we plan to keep around. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Sun, Aug 28, 2011 at 12:50 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I still see a huge difference between something in PyPI and something marked as experimental in the stdlib. Something in PyPI may or may not be promoted to stdlib status, but realistically that rarely happens (I think the last time it happened it was a JSON thing?). Something marked as experimental in the stdlib has a much higher likelihood to be promoted to regular stdlib status -- in fact that's the most likely outcome. The main difference between experimental and regular status in the stdlib is that for experimental modules we reserve the right to make incompatible changes in subsequent versions (possibly even in bugfix releases, although that's worth a separate discussion). An experimental feature in the stdlib also signals a certain commitment from the core developers. We shouldn't use experimental as a dumping ground for random stuff; that's what PyPI is for. We should use experimental within the context of the stdlib. Another thing to keep in mind is that not everybody has the same attitude towards installing 3rd party dependencies from PyPI. E.g. at Google, that requires someone to import the 3rd party library into our own code repo first, and that in turn requires a (light-weight) review. The net effect is that a Googler who happens to be the first to want to use a certain 3rd party package has to do a lot of work; it's *not* just a matter of downloading and installing the package. I imagine it's the same within many other organizations -- some better, some worse. (I was going to write something about many package developers being reluctant to require dependencies on other packages, but that's a pretty weak argument, since depending on an experimental stdlib package would limit them to a specific Python version, or dynamic presence-checking which isn't much better than having optional PyPI dependencies.) The power of "batteries included" is still with us, and experimental batteries are still batteries! -- --Guido van Rossum (python.org/~guido)

On Sun, 2011-08-28 at 10:10 -0700, Guido van Rossum wrote:
It looks to me, that Importing from __future__ can do things that a regular import can't do, like add or change core syntax. So they tend to be python core features rather than library features. The term __experimental__ may have differing expectations to different people and may be a little too broad. I like __future__ for *early* core changes, and __lib_future__ or __sdtlib_future__, for early stdlib features. I describe them as "early" because they either aren't quite ready for the current release cycle or they are waiting for something else to be depreciated or done before they can included. I think It makes sense to split core and library future features this way because usually, future core features are built into the core, while a future standard library module can be imported from the disk. Cheers, Ron

On Mon, Aug 29, 2011 at 8:05 AM, ron3200 <ron3200@gmail.com> wrote:
If we want to, we can make __experimental__ just as magical as __future__. That's orthogonal to its semantics in the library case, which is the point of interest at the moment. If people fear relying on experimental features, that's OK - if this idea happens at all, allowing people that aren't prepared to cope with the risk of breakage to easily avoid it would be a key benefit. The essential idea here is to be able to add a feature, but flag the API as potentially unstable for an entire release cycle before promising to maintain the API in perpetuity. Early adopters would get *most* of python-dev's usual guarantees (which most PyPI packages don't offer), while more conservative types can use more stable alternatives. The benefit on our side is that we'd get an full 18-24 months of feedback on a feature from a broad audience before finally locking the API down in the subsequent release. The more I think about this idea, the more I think it could help alleviate some of the concerns that can currently hold up the evolution of the standard library. (The costs of making a mistake with standard library API design are rather high due to the backwards compatibility promise, which means iterative design mostly needs to happen *before* the module is included. PyPI and the PEP process are both useful for that, but they don't really compare to the feedback available from being part of a full standard lib release). We'd still need to be careful not to throw any old rubbish in there, but as a phased introduction for things like "from __experimental__ import ipaddr" (PEP 3144) and "from __experimental__ import re" (re replacement with regex, assuming someone steps up to write the PEP and eventual drop-in replacement looks feasible), the idea definitely has potential.
No, that's *not* what __future__ means. __future__ changes are exactly what will become default behaviour in a future version - they aren't going to change, so code that uses them won't need to change. What may need changing in the case of __future__ is *old* code - the only reason we use a __future__ flag is when old code might break. The semantics of the new marker package would be to indicate that stuff is a little undercooked, but we've decided that it won't get enough exposure through PyPI (either through being too small to overcome NIH syndrome or else too closely coupled to a specific version of the interpreter). That's a completely different meaning, so we shouldn't reuse the same word. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Mon, 2011-08-29 at 21:14 +1000, Nick Coghlan wrote:
How often do you think things in __experimental__ will be aborted or canceled? Another way to think about "experimental" items is to use it in the way that science does. ie... An experiment designed to discover information, or test a very specific idea or theory. In that context, depending on the results of the experiment, a particular feature my be changed to take the results of the experiment into account. That's probably a bit too formal and restrictive, and it will be difficult to do in a very controlled way. The python community, and Python itself, tends to thrive in a more relaxed and informal atmosphere. But it may not hurt to have some guide lines on how to do a python "__experimental__" experiment so that it doesn't end up being a series of trial and error attempts at something half baked. Which do you think fits better with what you have in mind? __experimental__ __pre-view__ Cheers, Ron

On Mon, Aug 29, 2011 at 4:14 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
No, that's *not* what __future__ means. __future__ changes are exactly
Unlike __future__ the API is unstable and therefore code that works with today's __experimental__ may fail tomorrow. If the API changes in an incompatible way, we would probably prefer that the import fail rather than succeed with random results. Therefore, I would propose a mechanism to support this, perhaps something like from __experimental__ import regex {2,3,4} as re which means import the regex module version 2, 3 or 4. A later version will be imported only if it is an expansion, i.e., compatible with all previous APIs and differs only in containing new features. If the version set is omitted from the import it's treated as {1} so that dealing with this is only necessary for those modules which do change in incompatible ways. In general, checking for specific capabilities rather than version numbers is more robust but in this case, perhaps multiple versions of experimental APIs will be discouraged making that less necessary. But we could allow strings specifying required features in the version set. Note that the reason for allowing import with more than one version number is to handle the case when a new version is available but the differences are in parts of the API that don't impact the application. --- Bruce Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com

On Mon, 2011-08-29 at 18:22 -0700, Bruce Leban wrote:
Encouraging limited use of __experimental__ may be good. If it looks like there are going to be more than two three different combinations, then it is likely too soon to even be in __experimental__. It probably needs more thought still. One possibility would be to have experimental features be of the form, which is better? (a or b). Then you could try things out and save timings and test results in a fairly straight forward way. results = {} for x in 'ab': from __experimental__[x] import regex as re <do stuff with re and save some data about it> for x in 'ab': print(results[x]) I can think of a few issues with this. Like how long do things live in __experimental__? Can the letters be reused, or should new letters be used each time? Is "from __experimental__[x] import feature" better or worse than "from __experimental__ import feature[x]"? And do subscripts even work in import statements? Then again, maybe version #'s would work better? Having a way to ask __experimental__ what is has and use that to do the importing could make doing some things easier. Cheers, Ron

I'm sorry, the idea was never to add extra syntax to "from __experimental__". It was supposed to be just a plain import -- in fact plainer than "from __future__". There is hardly a need to use the double underscores; "from experimental" should work just as well. Why? It wasn't meant to be a language change. Just a convention to set aside an experimental namespace in the stdlib. If we're going to attempt to solve the problem of versioned dependencies, I recommend doing it outside the language syntax, at the package installer level, sort of the way setuptools / distribute do it. If you want to explore that idea, please read up on how it's done there and if you have ideas for improvement, start a new thread here. -- --Guido van Rossum (python.org/~guido)

On Tue, 2011-08-30 at 09:47 -0700, Guido van Rossum wrote:
Why? It wasn't meant to be a language change. Just a convention to set aside an experimental namespace in the stdlib.
That sounds fine. I think if the release manager has final say on the content of experimental, (along with you of course), the rest will probably take care of itself.
Hopefully it won't ever get so large or complicated that that is needed. Cheers, Ron

On Tue, Aug 30, 2011 at 11:22 AM, Bruce Leban <bruce@leapyear.org> wrote:
I'd advise against trying to overengineer this. If we decide to go down the path of breaking imports, then we'd probably just do it via naming conventions (e.g. give all experimental modules a suffix based on the Python version where they were first included). However, if you're concerned about subtle breakage due to backwards incompatibilities, then that's a good sign that depending on explicitly experimental modules is a *bad idea*. Adding a package specifically for candidate standard library modules with APIs that aren't yet locked in would be easy and low impact. Messing with the compiler or interpreter to provide additional features would be much higher impact, and consequently a much harder sell. However, you've highlighted the major problem with the idea: just because we *say* that the package is experimental and APIs may change without (programmatic) warning, doesn't mean people will pay attention. There are also a whole host of serialisation problems that arise when dealing with relocated modules, which could cause issues with the eventual migration to the "real" module location. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

This feature reminds me of "staging" in the Linux kernel. I often hear complaints that Python's bringing too many features on board, trying to explain __experimental__, {major, minor, patch} does not seem desirable. Then there's the consideration that any and all stuff floating around in __experimental__ will be overly encouraged, because honestly how many modules won't end up making the cut? Why can't a special tag/section be added in PyPi to indicate that a module is being considered for inclusion in future versions of Python, after all, we're all friends here. This might also give the oppurtunity to showcase some of the newer packaging infrastructure, and how easy it makes it to install third party modules. (Disclaimer: I haven't looked at the new one and I detest the old one.) -1 for adding yet more clutter Also -1 for renaming re: Some people call regex, regexp, you'll only be offending their sensibilities and adding to the confusion. Furthermore, a lot of Python's competitors have builtin regex support, such as =~ in Perl and bash. The terser the syntax for regex, the more at home and less cluttered it will appear to those users. I recommend that if a new regex module is created, that the name regex be used then, and the old be left alone. On Tue, Aug 30, 2011 at 1:03 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

On Wed, Aug 31, 2011 at 12:03 PM, alex23 <wuwei23@gmail.com> wrote:
The key benefit lies in ensuring that anything in the mooted experimental namespace is clearly under python-dev's aegis from at least the following perspectives: - licensing (i.e. redistributed by the PSF under a Contributor Licensing Agreement) - testing (i.e. the module test suites are run on the python.org buildbot fleet and results published via http://www.python.org/dev/buildbot) - issue management (i.e. bugs and feature requests are handled on http://bugs.python.org) - source control (i.e. the master repository for the software is published on http://hg.python.org) Those are the things that will allow the experimental modules to be used under existing legal approvals that allow the use of Python itself (e.g. in a corporate or governmental environment). Handling the actual packaging for distribution is a somewhat orthogonal question. Whether we offer 3 installers (CPython core, standard library, experimental), 2 installers (CPython core + stdlib, experimental) or 1 monolithic installer, or a combination of those approaches, doesn't actually change the surrounding legal framework all that much. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Wed, Aug 31, 2011 at 8:05 AM, Barry Warsaw <barry@python.org> wrote:
I'm sorry, I don't follow. The experimental package would only contain code distributed as part of the stdlib, and the code put in the stdlib's experimental package would get the same care from the core developers as the rest of the stdlib. The only difference would be that we'd drop the guarantee that the APIs offered would still be present in the next release (i.e. from 3.3 -> 3.4; the guarantees would hold from 3.3.1 -> 3.3.2). I hope I'm not contradicting myself? -- --Guido van Rossum (python.org/~guido)

On Aug 31, 2011, at 09:29 AM, Guido van Rossum wrote:
As long as the "guarantees" only cover code distributed in the experimental package with Python, that's okay. Maybe I was reading too much into Nick's comment, but when third party code can situate itself under 'experimental', we obviously can't make those same guarantees of the code we don't distribute. -Barry

On Thu, Sep 1, 2011 at 11:26 AM, Barry Warsaw <barry@python.org> wrote:
That certainly wasn't part of *my* plan any more than letting 3rd parties install themselves anywhere else inside stdlib packages. On the other hand it probably wouldn't matter because the review process doesn't look at what you import -- it only looks at dependencies (i.e. stuff that needs to be downloaded). Whether it goes under experimental would be orthogonal -- the litmus test for "is this covered by the PSF" would be "do I have to install it separately" (when using a Python install built from source, so there are no packages pre-installed by the distro). -- --Guido van Rossum (python.org/~guido)

On Thu, Sep 1, 2011 at 1:05 AM, Barry Warsaw <barry@python.org> wrote:
PEP 402 doesn't really change anything regarding corporate checks of code provenance - those relate to how the code is acquired and installed in the first place, not how you use it at runtime. If engineers are doing end runs around corporate policies, that's not a problem that technology can realistically address :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

alex23 writes:
"Although practicality beats purity." As already mentioned several times in this thread, PyPI, Bitbucket, and hg.python.org sandboxes as-is provide a plenty good technical solution for distribution of experimental code which is almost certain to be included in the core distribution at some point, but currently the API bikeshed is being painted (well, it will be if we can only come to consensus on color!) The problem to be solved is on the other side of the network connection, internal to the *using organizations*. Some of them have much stricter rules for adding new "approved" packages than for upgrading existing ones. In that case, developers "inside" get much freer access to "official experimental" modules, and can participate in development (including objecting to API changes they consider gratuitous :-) in a way that is hard for them to justify if *before* dealing with any API changes they have to run an internal obstacle course just to be allowed to use the code. As I understand Guido's position, "experimental" is a non-starter unless it can be expected to significantly increase beta tests of such modules by developers inside organizations that strictly control their internal code libraries. This requires that the "experimental" modules be distributed with the interpreter. Of course, if the stdlib was separated out, and the current stdlib simply bundled with the interpreter at distribution time, the experimental package could be given the same treatment. But the stdlib hasn't been done yet, and I don't think something labelled "experimental" would have the same credibility with IT Security/QA as the rest of core Python. This last might kill the whole idea, as QA might take the position that "yes, you're just upgrading Python 2.7 from 2.7.4 to 2.7.5, but we have no idea what might be in experimental, so you're going to have to make a separate request for that." (I have never worked in such an organization so I don't know if that's a realistic worry or not.)

On Tue, Aug 30, 2011 at 8:23 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
I wasn't actually proposing to add to (or even change the API of modules in) the experimental package during such micro-version updates. TBH the best use case I can think of would actually be the ipaddr package, which is somewhat controversial but not overly so, and seems to lack a push to ever get it accepted. Putting it in experimental for 3.3 would let us distribute it with Python without committing 100% to the solution it offers over its nearest competitor. However the downside is that that's a very long wait, still provides a pretty strong bias (unless we were to include both competing packages), and still doesn't look like it might get enough beta testing, unless the uptake of 3.3 is huge. So maybe we should just count PyPI downloads and decide that way. "50,000,000 Elvis fans can't be wrong." (An interesting meme by itself. :-) --Guido van Rossum (python.org/~guido)

On Aug 30, 2011, at 08:53 PM, Guido van Rossum wrote:
I think an experimental namespace is actually attacking the wrong problem, or maybe the right problem in the wrong way. ;) I'd much prefer to see some brainstorming on multi-version support, which is something that Robert Collins is always bugging me about. Something like pkg_resource's require() function seems like a good place to start. If we had this built-in, then including ipaddr in the stdlib as it currently stands would be a no-brainer, even with a suboptimal API. It would get lots of testing, and a completely different API could be designed for Python 3.4 without break packages that relied on the old API. Python's deprecation policy could be adjusted to include diminishing support for older versions of a module in the stdlib, and we'd avoid ugliness like unittest2 and such. -Barry

On Wed, Aug 31, 2011 at 8:19 AM, Barry Warsaw <barry@python.org> wrote:
It's a great idea to brainstorm about, just not in this thread. I see the two issues completely orthogonal.
You sound like you have a solution for using multiple versions of an API in the same program. Do you? Then out with it! Otherwise, no, it wouldn't be a no-brainer. -- --Guido van Rossum (python.org/~guido)

On Wed, Aug 31, 2011 at 9:56 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Yes, to some extent. My specific proposal is that experimental (by whatever name) would not change APIs in bugfix release, but would (potentially) change APIs in "major" releases (e.g. 3.3 -> 3.4(*)). (*) Can we pick a terminology so we all agree that "3.3.3" is a "minor release", "3.3" is a "major release", and "3" an "earthshattering release"? Or other terms -- but something that is both agreed upon and clear enough without explanation. I'm tired of having to clarify minor and major every time I use them out of fear they'll be mistaken for "3" and "3.3". -- --Guido van Rossum (python.org/~guido)

On 31 August 2011 18:33, Antoine Pitrou <solipsis@pitrou.net> wrote:
How about 3.3.3 -> 3.3.4 is a "minor" release, 3.3 -> 3.4 is a "feature" release and 3 -> 4 is not something we generally talk about (or "compatibility-breaking" or something like that). I suspect that "minor" for changing the last digit is pretty comprehensible, it's using "major" for 3.3 that confuses people, so let's avoid that term... Paul.

On Wed, Aug 31, 2011 at 10:51, Guido van Rossum <guido@python.org> wrote:
sys.version_info has already made the declaration of what each of the digits represent when it became a named tuple: major, minor, micro (http://hg.python.org/cpython/file/4dcbae65df3f/Python/sysmodule.c#l1273). And if you don't like the naming, then blame me; http://bugs.python.org/issue4285 was the bug that led to the names and they are what I have always used for all software. After that blame Eric Smith for committing the patch. =)

Another approach to __experimental__ could be a semi-official "python-experimental" package on pypi which contains all the stuff that's already been talked about. It's just one extra package to depend on and could follow its own release cycle. I don't know if this is better than including it in the stdlib (I like the idea of __experimental__ myself), but maybe its another idea to consider. David On Sep 1, 2011 8:40 PM, "Barry Warsaw" <barry@python.org> wrote:

The issue with that is that the "y" in 'Python x.y.z' is usually called the minor release (see: http://docs.python.org/devguide/devcycle.html ). Switching which digit the minor release is would be confusing when reading older documents. It might be better to instead keep the old name, "bugfix", since that isn't confusing. Then we would be left with "compatibility-breaking.feature.bugfix", which isn't really bad in any sense I can see (aside perhaps from being new and unusual?) Devin On Wed, Aug 31, 2011 at 1:45 PM, Paul Moore <p.f.moore@gmail.com> wrote:

On Wed, Aug 31, 2011 at 10:23 AM, Guido van Rossum <guido@python.org> wrote:
Given that there's no general industry agreement on what those things mean, "clear enough without explanation" is probably unrealistic. On the other hand, a PEP or some similar document that lays out the terminology used for python releases (and encouraged for python libraries) would let you assume that python people had read that explanation, and make a reference to it easy (i.e."see PEP XXXX"). <mike

On Thu, Sep 1, 2011 at 3:36 AM, Mike Meyer <mwm@mired.org> wrote:
That PEP is PEP 101 (at least as far as Python releases go), and it is currently a little inconsistent (using major.minor.micro in some sections, as the devguide and sys.version_info apparently also do, but also referring to feature releases as major releases on at least one occasion). PEP 386 (which aims to standardise version numbering within the wider Python ecosystem) doesn't mandate a naming scheme (only a numbering one), but does refer to the major.minor.micro names in the explanatory text. Personally, I agree with others that "feature release" and "bug fix release" are as close as we're likely to get to unambiguous names for the way Python itself uses x.y and x.y.z release numbering. Creating a new version of the language itself is rare enough that I don't think it really needs a generic name. Although with the accumulated baggage that 3.0 swept away, "spring cleaning release" might be appropriate :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Guido van Rossum writes:
I'm not sure "earthshattering" is Pythonic enough.<duck /> If you're willing to go with something pedestrian, at least the Debian types, and I think Gentoo, use "epoch" to mean that upstream decided to redo its version numbering. In a sense, I think that applies to Python 3 (which has to be judged a success, since it induced Linus to go to Linux 3!) and Linux 3 for that matter. How about the leading digit being the "Pythonic epoch", and "Python 3 was an epochal release"? Or "epoch-making release"?

On Sep 01, 2011, at 05:43 PM, Stephen J. Turnbull wrote:
How about the leading digit being the "Pythonic epoch", and "Python 3 was an epochal release"? Or "epoch-making release"?
Or maybe: snake reference: skin shedding release funny brit reference: spanish inquisition release -Barry

How about irreversible-release.

On Thu, Sep 1, 2011 at 1:16 PM, Christopher King <g.nius.ck@gmail.com> wrote:
How about irreversible-release.
"Killer release". (An oblique reference to Monty Python's killer joke.) -- --Guido van Rossum (python.org/~guido)

On Aug 31, 2011, at 09:31 AM, Guido van Rossum wrote:
It seems related though. It seems like the primary use case for experimental, especially in reference to the ipaddr module, is "we're including this battery, but it's shape might change, so don't count on it fitting into your socket next release". With multi-version libraries, we don't have to be so wishy-washy. We could say "here's version 0.5 of ipaddr" and the next release could safely include both that version and a stable, completely redesigned API of ipaddr 1.0. The two could co-exist (for a while), supporting both the legacy API, and the new whizzy one we actually like. ;)
I wish I did! -Barry

On Thu, Sep 1, 2011 at 11:34 AM, Barry Warsaw <barry@python.org> wrote:
It still sounds like they are (nearly) orthogonal though. Even if we had such a feature we'd still want to put ipaddr 0.5 into experimental, but most other new stdlib modules would not have to go there. And even if we didn't have such a feature, I'd still consider putting things in experimental, and just telling people to deal with it. Especially since we *don't* have the feature. :-)
That's what I'm talking about! :-) -- --Guido van Rossum (python.org/~guido)

On Wed, Aug 31, 2011 at 1:23 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Yep, that's a nice explanation of the motivation here. People that can just grab a module from PyPI and run it already have plenty of opportunities to provide feedback on in-development APIs. The idea of adding a new namespace within the standard library would be to have an 18-24 month window to gather similar feedback from folks in environments where they *can't* just grab packages from PyPI to try out, but *can* try out the latest version of CPython itself. (I suspect we'd also have an easier time getting feedback from folks that *could* retrieve modules PyPI if they wanted to, but in practice *don't*).
It depends on the organization, but the main hurdle I experienced at my previous employer was the initial licensing and "development pedigree" approval process (to make sure that we were on solid legal ground in redistributing the component to our own customers and that we were prepared to shoulder the risk of latent defects - both bars were significantly lower if we added "for internal use only" to the review request). Due to the way the PSF and python-dev operate, CPython and the standard library check a lot of boxes in that kind of review that smaller projects often miss. Once a component was on the approved list, as long as the licensing didn't change, decisions about upgrades were then in the engineers' hands without needing to get the lawyers involved again. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (20)
-
alex23
-
Antoine Pitrou
-
Barry Warsaw
-
Brett Cannon
-
Bruce Leban
-
Christopher King
-
David Townshend
-
Devin Jeanpierre
-
Dj Gilcrease
-
Eric V. Smith
-
Ethan Furman
-
Greg Ewing
-
Guido van Rossum
-
Matt Joiner
-
Mike Graham
-
Mike Meyer
-
Nick Coghlan
-
Paul Moore
-
ron3200
-
Stephen J. Turnbull