[Python-Dev] PEP 405 (pyvenv) and system Python upgrades

[Carl Meyer]

Hi all,

The recent virtualenv breakage in Python 2.6.8 and 2.7.3 reveals an 
issue that deserves to be explicitly addressed in PEP 405: what happens 
when the system Python underlying a venv gets an in-place bugfix 
upgrade. If the bugfix includes a simultaneous change to the interpreter 
and standard library such that the older interpreter will not work with 
the newer standard library, all venvs created from that Python 
installation will be broken until the new interpreter is copied into them.

Choices for how to address this:

1) Document it and provide a tool for easily upgrading a venv in this 
situation. This may be adequate. In practice the situation is quite 
rare: 2.6.8/2.7.3 is the only actual example in the history of 
virtualenv that I'm aware of. The disadvantage is that if the problem 
does occur, the error will probably be quite confusing and seemingly 
unrelated to pyvenv.

2) In addition to the above, introduce a versioning marker in the 
standard library (is there one already?) and have some code somewhere 
(insert hand-waving here) check sys.version_info against the stdlib 
version, and fail fast with an unambiguous error if there is a mismatch. 
This makes the failure more explicit, but at the significant cost of 
making it more common: at every mismatch, not just in the 
apparently-rare case of a breaking change.

3) Symlink the interpreter rather than copying. I include this here for 
the sake of completeness, but it's already been rejected due to 
significant problems on older Windows' and OS X.

4) Adopt a policy of interpreter/stdlib cross-compatibility within a 
given X.Y version of Python. I don't expect this to be a popular choice, 
given the additional testing requirements it imposes, but it would 
certainly be the nicest option from the PEP 405 standpoint (and may also 
be complementary to proposals for splitting out the stdlib). In the 
2.6.8/2.7.3 case, this would have been technically trivial to do, but 
the choice was made not to do it in order to force virtualenv users to 
adopt the security-fixed Python interpreter.

Thoughts?

Carl