Guido van Rossum writes:

On Tue, Nov 3, 2009 at 11:23 AM, M.-A. Lemburg wrote:

...

One question:

There are currently number of patch waiting on the tracker for additional Unicode feature support and it's also likely that we'll want to upgrade to a more recent Unicode version within the next few years.

How would such indirect changes be seen under the moratorium ?

That would fall under the Case-by-Case Exemptions section. "Within the next few years" sounds like it might well wait until the moratorium is ended though. :-)

(1) Unicode "feature support" (eg, implementing access to various databases, implementation of new algorithms) strikes me as worth a hearing but very dubious under case-by-case (the case I have in mind is when necessary to support (2) or (3)). (2) Adjusting existing implementations of Unicode features (eg, universal newline, which currently does not conform to the relevant TR AFAIK, thus dubious IMO) to conform to (new versions of) Unicode seems worthy of consideration on a cost/benefit basis. I would expect that in most cases the costs that lead to a moratorium would outweigh prospective benefits, though. I don't know of an example where I'd support an exception to the moratorium for this, though. It's hard to imagine that things like the algorithm for UTF-16 are going to change, and support for things like bidi are currently outside of the scope of the moratorium as I understand it -- they'd be stdlib facilities. (3) Upgrading the character repertoire and property databases (eg, case and composition) seems like a shoo-in to me. That's why the databases are versioned. Of course there are costs to upgrading (backward compatibility in some corner cases, eg) that might outweigh the benefits at any given time, but I don't think this should fall under the moratorium if someone finds a benefit to upgrading the databases within the moratorium period.

On Nov 3, 2009, at 3:42 PM, Michael Foord wrote:

Barry Warsaw wrote:

...

On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote:

...

I've checked draft (!) PEP 3003, "Python Language Moratorium", into SVN. As authors I've listed Jesse, Brett and myself.

On python-ideas the moratorium idea got fairly positive responses (more positive than I'd expected, in fact) but I'm bracing myself for fierce discussion here on python-dev. It's important to me that if if this is accepted it is a "rough consensus" decision (working code we already have plenty of :-), not something enforced by a vocal minority or an influential individual such as myself. If there's too much opposition I'll withdraw the PEP so as not to waste everybody's time with a fruitless discussion.

The PEP tries to spell out some gray areas but I'm sure there will be others; that's life. Do note that the PEP proposes to be *retroactive* back to the 3.1 release, i.e. the "frozen" version of the language is the state in which it was released as 3.1.

I think this is a great idea. I'd love to see the energy normally put into evolving the language into making the stdlib really kick ass.

+lots

Ditto. Doug

Stefan Behnel wrote:

Guido van Rossum, 03.11.2009 18:35:

...

Python, as a language is more than the core implementation -- CPython -- with a rich, mature and vibrant community of implementations, such as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit not only to the community, but to the language itself.

I noticed that this says "such as", but if it's intended to name the major implementations, I wonder why Cython is missing from that list.

Is Cython an implementation of Python? It's a grey area, but it seems like Cython is a "Python inspired" language (similar to Shedskin) rather than an implementation of Python: The Cython language is very close to the Python language, but Cython additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code. (From http://www.cython.org/ ) There are several partial implementations, including Python inspired languages, but if we are looking at 'major complete implementations' then the current list seems to be: CPython, Jython, IronPython and PyPy. Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather than a separate implementation. All the best, Michael Foord

Stefan

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/

Stefan Behnel wrote:

Michael Foord, 05.11.2009 10:13:

...

Stefan Behnel wrote:

...

if it's intended to name the major implementations, I wonder why Cython is missing from that list.

Is Cython an implementation of Python?

We certainly aim for Python compatibility. That's a proclaimed 1.0 goal. Not sure if that means Py3 or Py2 syntax, but that's just a parser detail, after all.

Right, but I think the main point is that Cython is not an implementation of Python. It uses Python syntax, calling into the Python runtime for many of its features, and *extending* Python syntax for compilation to C. (Right?) Obviously a grey area - but if you take Cython code and try to run it on ClassicPython then you are likely to have syntax errors. The same is not true of the other full implementations (although where they use native platform features you may not have the same libraries available - but the language is unchanged). All the best, Michael

...

It's a grey area, but it seems like Cython is a "Python inspired" language (similar to Shedskin) rather than an implementation of Python:

The Cython language is very close to the Python language, but Cython additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code.

(From http://www.cython.org/ )

Cython is different from Shedskin because Shedskin /requires/ the 'Python' code to be static. Cython does not. It /optimises/ static code, but it will not complain about dynamic code.

It /does/ make some static assumptions in that it considers builtins true builtins. However, it does not prevent you from replacing them in your code, as long as you do it inside the module. Certainly a restriction compared to Python, where you can import a module into a changed dict environment that redefines 'object', but not a major restriction IMO, and certainly not one that impacts much code.

...

There are several partial implementations, including Python inspired languages, but if we are looking at 'major complete implementations' then the current list seems to be: CPython, Jython, IronPython and PyPy.

Cython certainly is not a complete implementation, sure. It still lacks support for local classes, for example, metaclasses aren't handled, and generators are still in design stage. But if one of the goals of the moratorium is to let implementations catch up with the syntax, then Cython is certainly in line with that. I doubt that there will be much left on the TODO list two years from now.

...

Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather than a separate implementation.

Cython is somewhere in the middle. It's not a fork or branch. Rather, it depends on the normal CPython runtime for certain features, such as the standard library, the type/object implementation, garbage collection, and a couple of dynamic features. But that doesn't prevent it from compiling your module into an executable application.

Stefan

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/

To me, the term "Python implementation" just means something that implements the Python language. That shouldn't prevent it from providing additional features. Jython can interface with Java code, IronPython can interface with .NET code, Cython can interface with C code. Not sure what PyPy allows in addition to being a Python implementation, but there surely are language features also in PyPy that no other Python implementation provides.

Stefan

Hi. Features available in PyPy (or Java or IronPython) are not language features. These are features are available through normal python syntax, just some libraries are available extra. Python is something that passes most of CPython's test suite, modulo stuff that is considered implementation detail (like reference counting). In fact, some of extra PyPy features can be achieved via extension modules like greenlets. Cython might as well at some point become one, but so far it does not even import the whole test suite (correct me if I'm wrong), hence it's not yet python. Even if the ultimate goal is to support whole Python, it's not achieved yet, so it should not be considered implementation of Python (as of today, it might as well be one tomorrow). Cheers, fijal

Stefan Behnel wrote:

Michael Foord, 05.11.2009 11:18:

...

Stefan Behnel wrote:

...

We certainly aim for Python compatibility. That's a proclaimed 1.0 goal.

Right, but I think the main point is that Cython is not an implementation of Python. It uses Python syntax, calling into the Python runtime for many of its features

It is a Python implementation in the sense that it implements syntax and semantics of the Python language, be it through generated C code or by calling into the CPython runtime.

Implementing Python by calling into Python does not seem to really be a 'new implementation'... In that sense Cython is a wrapper not an implementation.

You could theoretically drop the entire interpreter from CPython, and Cython would still work with the remaining parts - not that it would be worth the effort on either side...

...

and *extending* Python syntax for compilation to C.

Programmers can decide themselves if they want to statically type variables, and they can do so without breaking with Python syntax. It is certainly not required to use non-Python syntax to compile Python code, not even if you want to get a fast binary module as a result.

But you extend the language. Coders can write Python or 'not-Python' (or perhaps 'Python plus'). Where you extend the language you are no longer implementing Python but a language oriented tool with Python inspired syntax intended for use with Python.

...

Obviously a grey area - but if you take Cython code and try to run it on ClassicPython then you are likely to have syntax errors. The same is not true of the other full implementations (although where they use native platform features you may not have the same libraries available - but the language is unchanged).

To me, the term "Python implementation" just means something that implements the Python language.

Which Cython doesn't - it can parse Python and it calls into Python. It doesn't implement runtime features (from the sounds of it).

That shouldn't prevent it from providing additional features.

Heh. Imagine if Microsoft had taken IronPython and added new syntax, that wasn't valid Python, and still tried to claim that IronPython was an implementation of Python. Few people would agree...

Jython can interface with Java code, IronPython can interface with .NET code, Cython can interface with C code. Not sure what PyPy allows in addition to being a Python implementation, but there surely are language features also in PyPy that no other Python implementation provides.

No. All of these implementation provide their integration through existing language features of Python, not by adding new *language* level features. All the best, Michael

Stefan

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/

On Thu, Nov 5, 2009 at 13:13, Stefan Behnel wrote:

Would you consider Cython a Python implementation if we implemented this? (which I guess we will do anyway at some point, simply because the extensions are actually invalid code in the given context)

Why do you want to be recognized as a Python implementation, anyway? I don't really understand why this seems so important to you. It's not like python-dev doesn't want to listen to you, right? Cheers, Dirkjan

Thomas Heller writes:

...

Why do you want to be recognized as a Python implementation, anyway? I don't really understand why this seems so important to you.

Advertising, I assume. That's what Stefan often does (at least it feels this way, for me).

Well, it's true that their work often goes unnoticed in favour of more pie-in-the-sky things such as PyPy and unladen-swallow.

On Thu, Nov 5, 2009 at 3:05 PM, Anand Balachandran Pillai wrote:

 Perhaps something to do with naming. "PyPy" and "unladen-swallow"  are obviously more eye-catching than "Cython".

Heh, CLPython is not very catchy either. Maybe that's why it's still under the radar here. - Willem

Stefan, I think your attempts to see Cython accepted as one of the major Python implementations is misguided. It is not self-contained, it is an add-on tool for CPython (like its ancestor PyRex). I think Cython is incredibly useful (and I spoke to many very happy users yesterday at UCB) but trying to present it as a separate Python implementation is not useful. Please stop the discussion about this topic, they are distracting. -- --Guido van Rossum (python.org/~guido)

On Thu, Nov 5, 2009 at 4:53 AM, Thomas Heller wrote:

Dirkjan Ochtman schrieb:

...

On Thu, Nov 5, 2009 at 13:13, Stefan Behnel wrote:

...

Would you consider Cython a Python implementation if we implemented this? (which I guess we will do anyway at some point, simply because the extensions are actually invalid code in the given context)

Why do you want to be recognized as a Python implementation, anyway? I don't really understand why this seems so important to you.

Advertising, I assume.  That's what Stefan often does (at least it feels this way, for me).

I'm skeptical about Cython as a Python implementation. AFAIK it requires CPython to work (for the runtime functionality) and while Cython's syntax is very Python-like, it is probably best characterized as an extended subset (:-). I guess when it runs the entire standard test suite without errors it could claim to be a Python implementation; until then, I would rather consider it a useful tool to go with CPython. (The name is actually a bit confusing, but I don't have a desire to change it.) -- --Guido van Rossum (python.org/~guido)

Maciej Fijalkowski, 05.11.2009 19:01:

...

Most Python implementations do not reimplement the stdlib, or at most a minor part of it, so that's right out of the discussion.

Did you actually check?

Well, I know that Jython uses the original stdlib modules through svn:externals in the build, last thing I heard about IronPython was that they are allowed to use other people's code now, so I imagine they do the same thing, and I wouldn't expect PyPy to rewrite the existing Python code that exists in the stdlib. So the only remaining problem are the C extensions in CPython, and that's the minor part I mentioned above. Stefan

On Thu, Nov 5, 2009 at 10:35 AM, Dino Viehland wrote:

Stefan wrote:

...

It /does/ make some static assumptions in that it considers builtins true builtins. However, it does not prevent you from replacing them in your code, as long as you do it inside the module. Certainly a restriction compared to Python, where you can import a module into a changed dict environment that redefines 'object', but not a major restriction IMO, and certainly not one that impacts much code.

To me this is a deal breaker which prevents Cython from being a Python implementation.  From a talk given by Colin Winter at the LLVM dev meeting (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to do something like this as well and Guido said no.  In this case the breaking change is so subtle that I'd personally hate to run into something like this porting code to Cython and having to figure out why it's not working.

To clarify, I was joking when I told that story (or at least I was joking with Guido when I asked him if we could break that). It clearly *would* be easier if we could just ignore this point of Python compatibility, but that's not an option, so we've had to optimize around it. It's not that hard to do, but it's still extra work. Collin Winter

Stefan wrote:

I assume that this is artificially exaggerated to make a point, as this behaviour is obviously not a technical requirement but an optimisation, which could potentially be disabled.

If there's a way to disable this then that's fine and IMO when it was disabled you'd still be Python. But changing behavior in the name of optimization is no longer just an optimization. For what it's worth in IronPython we're going to look at making this faster in the future as well. We already cache built-in values locally in a module and invalidate the cache when the builtins changes. In the future I'd like to even combine loading a global and invoking it into a single call site for even better optimizations. And you could even imagine an extreme scenario where when built-ins changed all the code in the system that depended upon them is re-compiled punishing users who actually do this but I doubt we'll go that far. But the important thing is that the optimizations don't change the language behavior otherwise I don't think you're still Python. But of course that's what makes it so challenging and fun to try and optimize Python :)

On Fri, Nov 6, 2009 at 6:18 AM, Steven D'Aprano wrote:

I don't know how mature or active it is, so it may not count as either major or complete, but there's also CLPython:

http://common-lisp.net/project/clpython/

CLPython is in steady development, quite complete and stable on the language level (somewhere between 2.5 and 2.6), but missing most built-in library functionality. (It reuses the pure-Python parts of the stdlib.) As its developer, I don't care much about being mentioned in discussions or presentations, as CLPython is not as widely used as IronPython and Jython. But in a PEP like this, which is directly about alternative implementations, it deserves to be mentioned, IMHO. - Willem

On Sun, Nov 8, 2009 at 1:16 AM, Steven D'Aprano wrote:

Willem, the rationale for this PEP is to give alternative implementations the chance to catch up with CPython.

Given your statement that CLPython is quite complete on the language level, but missing standard library features, how do you think the moratorium will help CLPython?

It would be great to have a pause in Python-the-language, so that CLPython might at the end of the moratorium finally be up to date with the latest 2.x and 3.x features. Supporting standard libraries is a bigger challenge, as the ones written in C need rewriting. Whether or not there is a pause on that front does not matter too much to that situation. - Willem

[GvR]

I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday.

But it would have been so much fun to have a never ending python-ideas thread on BEGIN/END blocks ;-) C'est la vie, Raymond

On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote:

I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday.

Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? Cheers, Dirkjan

On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller wrote:

On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman wrote:

...

On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote:

...

I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday.

Let me state first, I think the PEP is great, and I have no objection to its current form.

I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP.

Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x).

The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear.

Is this just crazy talk?

Cheers,

Dirkjan

I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal.

So, -1 from me ;)

jesse

Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth. For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings. I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. -- --Guido van Rossum (python.org/~guido)

On Thu, Nov 5, 2009 at 15:26, Guido van Rossum wrote:

On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller wrote:

...

On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman wrote:

...

On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote:

...

I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday.

Let me state first, I think the PEP is great, and I have no objection to its current form.

I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP.

Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x).

The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear.

Is this just crazy talk?

Cheers,

Dirkjan

I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal.

So, -1 from me ;)

jesse

Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth.

For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings.

I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings.

What exactly are those better ways? Document as deprecated only? -Brett

On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward wrote:

A switch to ENABLE those warnings?

I think a few people I know would still be raising strings like exceptions if not for the deprecation warnings. Most people won't turn on the switch with the extra warnings. --yuv

Gregory P. Smith schrieb:

On Thu, Nov 5, 2009 at 4:13 PM, Yuvgoog Greenle wrote:

...

On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward wrote:

...

A switch to ENABLE those warnings?

I think a few people I know would still be raising strings like exceptions if not for the deprecation warnings. Most people won't turn on the switch with the extra warnings. --yuv

Yep and their code will break in the future as a result. Thats their problem (or whoever employed them and neglected to do code reviews and beat them with a stick for using a Python 1.x "feature" in 2009).

SUGGESTION: Our Whats New document currently has a single "new, improved and deprecated" section in it. We should make Deprecations its own section so that they're easy to find all in one concise place when upgrading code between releases.

Good point, I'll make that change if AMK agrees. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

A.M. Kuchling schrieb:

On Sun, Nov 08, 2009 at 10:27:46PM +0100, Georg Brandl wrote:

...

Good point, I'll make that change if AMK agrees.

It's certainly fine with me. Do we want to only make that change to the 2.7 "What's New", or should we also do it for the 2.6 one?

Why not for 2.6 as well, it's an immediate benefit, especially since the list of deprecations in 2.6 is huge. Georg

...

A switch to enable deprecation warnings would give developers a chance to see them when migrating to a new version of python. And it would prevent users from having to deal with them.

PendingDeprecationWarning is silent by default and requires a switch to be enabled.

So what we need is perhaps not a stdlib moratorium, but rather strict adherence to a sequence of PendingDeprecationWarning -> DeprecationWarning -> Removal? (The first one seems to have been underused so far.) That way nothing could get removed without having had at least two warnings in subsequent minor versions, one silent and one loud. People who want to avoid loud warnings can check for silent ones before upgrading, and everyone is guaranteed to have seen a warning before final removal. - Hagen

On Thu, Nov 5, 2009 at 19:23, Guido van Rossum wrote:

On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon wrote:

...

On Thu, Nov 5, 2009 at 15:26, Guido van Rossum wrote:

...

I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings.

What exactly are those better ways? Document as deprecated only?

Sorry, I have an existence proof, but no construction. :-)

Ideas welcome. Silent deprecations, loud documentation, biweekly home visits, whatever, as long as it doesn't log a message by default.

Well, one option is to come up with the equivalent of -3, but for all warnings; the antithesis of -W. And obviously glaring deprecation warnings in the docs (removal has been discussed but always shot down as someone who comes across old code might still need docs for it). The clarification I need is will this in any way influence when modules are removed. If they stay in for the life of a major version then I want it made clear that bug fixes for the code take lower priority over all other code in the standard library. -Brett

On Fri, Nov 6, 2009 at 10:35, Nick Coghlan wrote:

Longer term, a solution may be to extend the standard deprecation period one release and make pending deprecation warnings required rather than optional. That way, on the ball developers would have a full release to quash deprecation warnings before their users encountered them by default.

That is:

 Release X.Y: deprecated in docs, pending deprecation in code  Release X.Y+1: deprecated in code  Release X.Y+2: removed in code and docs

(Or we could just make that the policy now and not do anything specific in relation to the moratorium and the standard library)

This sounds like an improvement for things like Mercurial, at least. We did support 2.3-2.6 until relatively recently, and I think that's hard to get around for software that actually has to work on the user's box. This is a bit different from webapps, I suspect, where you "only" have to support the servers, which you often have more control over. But supporting 4 releases in a row has been a bit of a PITA. Luckily, we finally felt it was time to drop 2.3, so now we can make use of luxuries such as subprocess... Still, supporting 3 releases seems relatively common in the Python world (after all, parts of Zope still require 2.4, I think), and so it would be nice if the stdlib moved a little bit slower. Cheers, Dirkjan

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dirkjan Ochtman wrote:

But supporting 4 releases in a row has been a bit of a PITA. Luckily, we finally felt it was time to drop 2.3, so now we can make use of luxuries such as subprocess... Still, supporting 3 releases seems relatively common in the Python world (after all, parts of Zope still require 2.4, I think), and so it would be nice if the stdlib moved a little bit slower.

The current release of Zope (finally) requires 2.5 or 2.6: we have a "don't intentionally break 2.4" policy in place, however, FBO those users whose system Python is still 2.4, and who can't / won't build their own Python. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkr0ewgACgkQ+gerLs4ltQ7W2ACeOXPwhpg8Hi4/XZom/B59/7GD 4mwAnRkixQm0A3ZkNRQMPW1JIbEOy1iC =gTbW -----END PGP SIGNATURE-----

On Fri, Nov 6, 2009 at 6:02 PM, Guido van Rossum wrote:

I'm fine with silent deprecations or requiring a flag to turn on deprecation warnings (like Java does).

It's easier for java because they have compile time for the deprecation warnings. Maybe we should somehow silence all warnings when running eggs? --yuv

On Fri, Nov 6, 2009 at 8:25 AM, geremy condra wrote:

And speaking of TIOBE, my impression- and apparently yours- was that Python was on its way up, but TIOBE lists us as being down from a little over a year ago. Anybody know anything about their methodology?

There's a detailed explanation at: http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm Andre

Brett Cannon wrote:

On Fri, Nov 6, 2009 at 08:02, Guido van Rossum wrote:

...

...

The clarification I need is will this in any way influence when modules are removed. If they stay in for the life of a major version then I want it made clear that bug fixes for the code take lower priority over all other code in the standard library. I think we should be as cautious as ever with removing modules. We've had our chance for clean-up without abandon with Py3k; now we should stick with our commitment to backwards compatibility. In fact, we should probably be *more* conservative than we already were given that

On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon wrote: the amount of code written in Python is always increasing and hence the cumulative pain caused by incompatible changes will increase too.

I'm fine with silent deprecations or requiring a flag to turn on deprecation warnings (like Java does).

We can discuss this on the stdlib-sig and come back with a proposal on how to update PEP 4 with an explicit policy on how to handle deprecations (either Nick's proposal or some flag, and then how long to let the deprecation last, i.e. three versions or the life of a major version of Python).

Complete removal of modules during the seems to have caused problems especially for people supporting a range of versions. On the other hand, you want to be able to replace and cease support of modules. So rather that complete removal, perhaps move them to a new package called, for instance, 'old'. And move their doc sections to a new chapter call 'Old Modules' which would begin with something like "The following modules have been replaced but are still present in the *old* package for back compatibility. They are no longer maintained and will possibly disappear in the future, such as if there is ever a Python4." This suggestion based on the "3. Non-essential Built-in Functions" of the 2.x doc, whose contents disappeared in 3.0 (even though that seems not to have been specified in the doc). Terry Jan Reedy

On Fri, Nov 6, 2009 at 3:24 PM, Georg Brandl wrote:

Guido van Rossum schrieb:

...

Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth.

For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings.

Trying to import hashlib and importing md5 on ImportError isn't *too* unclean, is it?

That's not all -- you also have to modify the code that uses the module, unless you use "import as", which has problems of its own. Plus, I may not care (yet) about supporting 2.7, and yet I am forced to change my code to cleanly support 2.6. I really don't like it. -- --Guido van Rossum (python.org/~guido)

On Fri, Nov 6, 2009 at 6:42 PM, Nick Coghlan wrote:

Georg Brandl wrote:

...

Guido van Rossum schrieb:

...

Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth.

For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings.

Trying to import hashlib and importing md5 on ImportError isn't *too* unclean, is it?

Having had to do this myself a few times, I was always a little surprised the "import-or" idea [1] didn't get more support the last time it was proposed.

Ah well, don't need to worry about that idea again until 3.4 or so :)

Cheers, Nick

P.S. For anyone unfamiliar with it, "import-or" was a suggestion made around the time xml.etree was added to the standard library to provide a "import x or y as z" shorthand for:  try:    import x as z  except ImportError:    import y as z

(allowing "from a or b import c" was also part of the suggestion, as was allowing multiple "or" elements in a single import request)

Support for the idea was lukewarm at best, hostile at worst (hence the lack of PEP).

I'm still -1 on that -- while it may sound like a good shorthand if you have to deal with this, it's IMO pretty mysterious if you encounter this and don't already happen to know what it means. Also, it makes no sense without the 'as' clause which makes it a bit awkward. (I could probably come up with a few more arguments against it, just trying to hint that I'm personally less than lukewarm. :-) -- --Guido van Rossum (python.org/~guido)

On 12:10 pm, skip@pobox.com wrote:

Guido> ... it's IMO pretty mysterious if you encounter this and don't Guido> already happen to know what it means.

If you require parens maybe it parses better:

import (a or b or c) as mod

Given that the or operator shortcuts I think that (a or b or c) terminates once a module is found isn't too hard to grasp.

Did everyone forget what the subject of this thread is? Jean-Paul

On Thu, Nov 5, 2009 at 14:53, Dirkjan Ochtman wrote:

On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote:

...

I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday.

Let me state first, I think the PEP is great, and I have no objection to its current form.

I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP.

Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x).

The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear.

Is this just crazy talk?

So you are basically asking for a moratorium on stdlib deprecations. Considering how much was removed in Python 3 from 2.6 this is a really minor worry. And the only deprecation that I can see potentially coming down the pipeline is optparse for argparse and cProfile to help finish PEP 3108. And, as Jesse said in another reply, it would be nice to take this time to reflect upon where we want the stdlib to go and to potentially clear other things out. So I am -0 on this as leaving stuff in for an extra release isn't going to kill us, but I would prefer to not put it off. -Brett

On Fri, 6 Nov 2009 09:05:17 am Guido van Rossum wrote:

On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum wrote:

...

I've checked draft (!) PEP 3003, "Python Language Moratorium", into SVN. As authors I've listed Jesse, Brett and myself.

I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday.

I'm not a core developer, but I have made some objections on the python-ideas list. For what it is worth, I don't think a moratorium is needed, and it is not cost free. Whether the cost is worth it is a matter of opinion, but I've read posts suggesting that there's no downside to a moratorium. I think that's naive. Firstly, the core developers are very conservative in the features they add to the language. Nobody suggests that will change -- there are (as far as I know) a grand total of *one* proposed language changes under consideration for 3.2, namely "yield from". It isn't like the moratorium is needed to stem a flood of vast numbers of incompatible language changes, and this is not likely to change. The given reason for the moratorium is to give the developers of other Python implementations a chance to catch up. But as far as I know -- and I welcome correction -- only one such developer has spoken up, and he says that new language features generally aren't a problem for IronPython, but some standard library features are: http://mail.python.org/pipermail/python-ideas/2009-October/006328.html (In fairness, I should point out that Dino did not vote against the moratorium, he gave a conditional "in favour" vote but asked for some wiggle-room. A moratorium with wiggle-room is not a moratorium.) A moratorium isn't cost-free. With the back-end free to change, patches will go stale over 2+ years. People will lose interest or otherwise move on. Those with good ideas but little patience will be discouraged. I fully expect that, human nature being as it is, those proposing a change, good or bad, will be told not to bother wasting their time, there's a moratorium on at least as often as they'll be encouraged to bide their time while the moratorium is on. A moratorium turns Python's conservativeness up to 11. If Python already has a reputation for being conservative in the features it accepts -- and I think it does -- then a moratorium risks giving the impression that Python has become the language of choice for old guys sitting on their porch yelling at the damn kids to get off the lawn. That's a plus for Cobol. I don't think it is a plus for Python. What one person sees as "stable", another may see as "moribund". Perception does matter. Nick Coglan spoke of Jython having "fell by the wayside" when CPython introduced new style classes: http://mail.python.org/pipermail/python-ideas/2009-October/006431.html I think that speaks to the opposite conclusion that Nick intended. If developers craved stability, and were put off by the pace of changes to CPython, why weren't they migrating to Jython, which had one new production release in six years? http://www.jython.org/archive/22/news.html Nick also described C as changing much more slowly over its life that Python has. This is misleading: it is true that the C standard has been stable, but that doesn't mean that C compilers have been. Changes to the standard were driven by changes introduced by the implementations, not visa versa. Likewise for Cobol, where the first implementation to introduce OOP was released in 1997, but the standard wasn't updated until 2002. Python is nothing like that. Python doesn't have an official standard, and Guido has dismissed the need for an ISO standard. The other implementations have historically let CPython lead as far as language evolution goes. If CPython stops, likely Python stops. Who is going to risk adding language features that break compatibility with the most popular implementation? It won't be Python anymore. The PEP says the moratorium will last "at least" two years from the release of 3.1. Given the normal slow pace of new language features, two years isn't bad -- that's effectively just 3.2. But Guido has suggested it could last to 3.3. Could it last beyond that? 3.4? Until some distant future Python-4000? At the very least, I believe, any moratorium should have a clear end date. A clear end date will be a powerful counter to the impression that Python the language is moribund. It says, this is an exceptional pause, not a permanent halt. -- Steven D'Aprano

John Arbash Meinel wrote:

He wanted to introduce a moratorium at least partially because he was tired of endless threads about anonymous code blocks, etc. Which aren't going to be included in the language anyway, so he may as well make a point to say "and neither will anything else for a while".

I have not said anything on this topic either way. But I think a general moratorium is a false solution to that particular problem. Guido should just make a clear pronouncement (again) and let people quote that in response to initial posts, before they *appear* to gain steam. Each time Python doubles in popularity, the number of people wanting such things will approximately double. That is the price of success. Terry Jan Reedy

Greg Ewing writes:

If anonymous code blocks still get discussed even when they have no chance of being accepted, this suggests that a moratorium is *not* going to stop discussion of new features.

Well, if they get discussed, it's probably that some people can't help participating in the discussions, even though they know the idea is doomed to failure ;)

On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano wrote:

On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote:

...

At the very least, I believe, any moratorium should have a clear end date. A clear end date will be a powerful counter to the impression that Python the language is moribund. It says, this is an exceptional pause, not a permanent halt.

Proposal:

No new language features in odd-numbered point releases (3.3, 3.5, ...). Even-numbered point releases (3.4, 3.6, ...) may include new language features provided they meet the usual standards for new features.

3.2 is a special case: as an even-numbered release, it would normally allow new features, but in recognition of the special nature of the 2.x to 3.1/3.2 migration, no new language features will be allowed.

Advantages:

* It slows down changes to the language while still allowing sufficiently high-standard new features.

* Alternate implementations have a stable language version to aim for. Assuming point releases come ever 12-18 months, that stable language version will last 2-3 years.

* It doesn't have the psychological baggage of an unconditional ban on new features for the indefinite future. It gives a fixed, known schedule for when new features will be permitted, without the uncertainty of "at the BDFL's pleasure".

-- Steven D'Aprano

FWIW, I view a definite end point as a definite plus. Geremy Condra

On Sun, Nov 8, 2009 at 8:22 PM, Jesse Noller wrote:

On Nov 8, 2009, at 7:01 PM, geremy condra wrote:

...

On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano wrote:

...

On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote:

...

At the very least, I believe, any moratorium should have a clear end date. A clear end date will be a powerful counter to the impression that Python the language is moribund. It says, this is an exceptional pause, not a permanent halt.

Proposal:

No new language features in odd-numbered point releases (3.3, 3.5, ...). Even-numbered point releases (3.4, 3.6, ...) may include new language features provided they meet the usual standards for new features.

3.2 is a special case: as an even-numbered release, it would normally allow new features, but in recognition of the special nature of the 2.x to 3.1/3.2 migration, no new language features will be allowed.

Advantages:

* It slows down changes to the language while still allowing sufficiently high-standard new features.

* Alternate implementations have a stable language version to aim for. Assuming point releases come ever 12-18 months, that stable language version will last 2-3 years.

* It doesn't have the psychological baggage of an unconditional ban on new features for the indefinite future. It gives a fixed, known schedule for when new features will be permitted, without the uncertainty of "at the BDFL's pleasure".

-- Steven D'Aprano

FWIW, I view a definite end point as a definite plus.

Geremy Condra

There is a time outlined in the pep.

I quote: "This PEP proposes a temporary moratorium (suspension) of all changes to the Python language syntax, semantics, and built-ins for a period of *at least two years* from the release of Python 3.1." Emphasis mine. Like I say, a definite end point would be much preferred to n > 2. If possible, I'd also like to hear some of Steven's other points addressed. Geremy Condra

On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum wrote:

On Sun, Nov 8, 2009 at 5:45 PM, geremy condra wrote:

...

I quote:

"This PEP proposes a temporary moratorium (suspension) of all changes to the Python language syntax, semantics, and built-ins for a period of *at least two years* from the release of Python 3.1."

Emphasis mine.

I since added this:

"""In particular, the moratorium would include Python 3.2 (to be released 18-24 months after 3.1) but (unless explicitly extended) allow Python 3.3 to once again include language changes."""

...

Like I say, a definite end point would be much preferred to n > 2.

My time machine doesn't work very well in the future. So I can't tell what we'll find necessary 2 years from now. But I would be fine with defining the time limit to be max(time(3.1) + 2 years, time(3.2)). I.e. the moratorium (unless explicitly extended) ends as soon as 3.2 has been released *and* at least 2 years have passed since 3.1.

Ok, thanks for the clarification. Could you spell out what you would consider grounds for a future extension?

...

If possible, I'd also like to hear some of Steven's other points addressed.

They haven't changed my mind.

Ok, but the fact that you (or Steven) hold a particular set of beliefs is a singularly unconvincing argument. Could you explain why you don't agree, if only for the record? Geremy Condra

Thanks Brett. I've moved the moratorium PEP to Status: Accepted. I've added the words about inclusion of 3.2 and exclusion of 3.3 (which were eaten by a svn conflict when I previously tried to add them) and added a section to th end stating that an extension will require another PEP. --Guido On Mon, Nov 9, 2009 at 9:12 AM, Brett Cannon wrote:

On Sun, Nov 8, 2009 at 19:50, geremy condra wrote:

...

On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum wrote:

...

On Sun, Nov 8, 2009 at 5:45 PM, geremy condra wrote:

...

I quote:

"This PEP proposes a temporary moratorium (suspension) of all changes to the Python language syntax, semantics, and built-ins for a period of *at least two years* from the release of Python 3.1."

Emphasis mine.

I since added this:

"""In particular, the moratorium would include Python 3.2 (to be released 18-24 months after 3.1) but (unless explicitly extended) allow Python 3.3 to once again include language changes."""

...

Like I say, a definite end point would be much preferred to n > 2.

My time machine doesn't work very well in the future. So I can't tell what we'll find necessary 2 years from now. But I would be fine with defining the time limit to be max(time(3.1) + 2 years, time(3.2)). I.e. the moratorium (unless explicitly extended) ends as soon as 3.2 has been released *and* at least 2 years have passed since 3.1.

Ok, thanks for the clarification. Could you spell out what you would consider grounds for a future extension?

We feel it's necessary as a group or Guido does, simple as that. You can't plan it since it's over two years away. If the time comes and people feel the moratorium has been beneficial and should go longer, we will extend it. It will most likely be for the same reasons we started it.

...

...

...

If possible, I'd also like to hear some of Steven's other points addressed.

They haven't changed my mind.

Ok, but the fact that you (or Steven) hold a particular set of beliefs is a singularly unconvincing argument.

I disagree. Guido is the BDFL so his set of beliefs is enough unless faced with a huge number of people disagreeing. That has not occurred on this topic.

...

Could you explain why you don't agree, if only for the record?

Enough happens on python-dev based on gut feeling that there isn't a need. If we had to spell out objections to every email we received while discussing a PEP, threads would never end. Heck, I think this PEP discussion as gone on long enough and that Guido could pronounce at this point.

-Brett

-- --Guido van Rossum (python.org/~guido)

Guido van Rossum wrote:

On Sun, Nov 8, 2009 at 3:06 PM, Steven D'Aprano wrote:

...

No new language features in odd-numbered point releases (3.3, 3.5, ...). Even-numbered point releases (3.4, 3.6, ...) may include new language features provided they meet the usual standards for new features.

Oh no, not the eve/odd numbering scheme. Nobody will be able to remember which is which.

In this case, does it really matter? Usually the confusion is with stable/unstable being even/odd or odd/even, but in this case the options are stable/stable (it just happens that certain pairwise versions are related). -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

On Thu, Nov 5, 2009 at 14:20, Arfrever Frehtes Taifersar Arahesis wrote:

2009-11-03 18:35:10 Guido van Rossum napisał(a):

...

I've checked draft (!) PEP 3003, "Python Language Moratorium", into SVN. As authors I've listed Jesse, Brett and myself.

On python-ideas the moratorium idea got fairly positive responses (more positive than I'd expected, in fact) but I'm bracing myself for fierce discussion here on python-dev. It's important to me that if if this is accepted it is a "rough consensus" decision (working code we already have plenty of :-), not something enforced by a vocal minority or an influential individual such as myself. If there's too much opposition I'll withdraw the PEP so as not to waste everybody's time with a fruitless discussion.

The PEP tries to spell out some gray areas but I'm sure there will be others; that's life. Do note that the PEP proposes to be *retroactive* back to the 3.1 release, i.e. the "frozen" version of the language is the state in which it was released as 3.1.

Does moratorium allow to add support for e.g. 'from __future__ import yield_from' in Python 3.2?

No. -Brett