[Python-Dev] PEP 408 -- Standard library __preview__ package

[Steven D'Aprano]

Eli Bendersky wrote:
> Hello,
> 
> Following an earlier discussion on python-ideas [1], we would like to
> propose the following PEP for review. Discussion is welcome.


I think you need to emphasize that modules in __preview__ are NOT expected to 
have a forward-compatible, stable, API. This is a feature of __preview__, not 
a bug, and I believe it is the most important feature.

I see responses to this PEP that assume that APIs will be stable, and that 
having a module fail to graduate out of __preview__ should be an extraordinary 
event. But if this is the case, then why bother with __preview__? It just adds 
complexity to the process -- if __preview__.spam and spam are expected to be 
the same, then just spam straight into the std lib and be done with it.

This PEP only makes sense if we assume that __preview__.spam and spam *will* 
be different, even if only in minor ways, and that there might not even be a 
spam. There should be no expectation that every __preview__ module must 
graduate, or that every standard library module must go through __preview__. 
If it is stable and uncontroversial, __preview__ adds nothing to the process.

Even when there are candidates for inclusion with relatively stable APIs, like 
regex, we should *assume* that there will be API differences between 
__preview__.regex and regex, simply because it is less harmful to expect 
changes that don't eventuate than to expect stability and be surprised by changes.

This, I believe, rules out Antoine's suggestion that modules remain importable 
from __preview__ even after graduation to a full member of the standard 
library. We simply can't say have all three of these statements true at the 
same time:

1) regular standard library modules are expected to be backward compatible
2) __preview__ modules are not expected to be forward compatible
3) __preview__.spam is an alias to regular standard library spam


At least one of them has to go. Since both 1) and 2) are powerful features, 
and 3) is only a convenience, the obvious one to drop is 3). I note that the 
PEP, as it is currently written, explicitly states that __preview__.spam will 
be dropped when it graduates to spam. This is a good thing and should not be 
changed.

Keeping __preview__.spam around after graduation is, I believe, actively 
harmful. It adds complexity to the developer's decision-making process 
("Should I import spam from __preview__, or just import spam? What's the 
difference?"). It gives a dangerous impression that code written for 
__preview__.spam will still work for spam.

We should be discouraging simple-minded recipes like

try:
     import spam
except ImportError:
     from __preview__ import spam
spam.foo(a, b, c)

since they undermine the vital feature of __preview__ that the signature and 
even the existence of spam.foo is subject to change.

I would go further and suggest that __preview__ be explicitly called 
__unstable__. If that name is scary, and it frightens some users off, good! 
The last thing we want is when 3.4 comes around to have dozens of bug reports 
along the line of "spam.foo() and __preview__.spam.foo() have different 
function signatures and aren't compatible". Of course they do. That's why 
__preview__.spam existed in the first place, to allow the API to mature 
without the expectation that it was already stable.

Since __preview__.spam (or, as I would prefer, __unstable__.spam) and spam 
cannot be treated as drop-in replacements, what is __preview__.spam good for? 
Without a stable API, __preview__.spam is not suitable for use in production 
applications that expect to run under multiple versions of the standard library.

I think the PEP needs more use-cases on who might use __preview__.spam, and 
why. These come to my mind:


* if you don't care about Python 3.x+1, then there is no reason not to
   treat Python 3.x's __preview__.spam as stable;

* rapid development proof-of-concept software ("build one to throw away")
   can safely use __preview__.spam, since they are expected to be replaced
   anyway;

* one-use scripts;

* use at the interactive interpreter;

* any other time where forward-compatibility is not required.


I am reminded of the long, often acrimonious arguments that took place on 
Python-Dev a few years back about the API for the ipaddr library. A lot of the 
arguments could have been short-circuited if we had said "putting ipaddr into 
__preview__ does not constitute acceptance of its API".

(On the other hand, if __preview__ becomes used in the future for library 
authors to fob-off criticism for 18 months in the hope it will just be 
forgotten, then this will be a bad thing.)




-- 
Steven