[Python-Dev] feature freeze questions

[Chris Jerdonek]

On Sat, Jul 28, 2012 at 6:21 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Jul 28, 2012 at 8:25 PM, Chris Jerdonek
> <chris.jerdonek at gmail.com> wrote:
>> Also, this response seems somewhat at odds to the response I got to a
>> patch that refactors part of regrest, fixes some bugs in it, and adds
>> its first unit tests (in particular, increasing code coverage):
>>
>> http://bugs.python.org/issue15302
>
> Asking that #15302 be postponed is a risk judgement taking into
> account "what do we have time to review before rc1?". Cleaning up the
> argument parsing in regrtest is definitely a good thing to do, but
> it's not an urgent fix.
>
> A similar judgement call will be made for any test suite changes
> between now and rc1: weighing up the risk of causing problems with the
> release process vs having improved tests at the time of the release.

I guess I'm still not clear on the restrictions to making changes to
test-related code -- whether these restrictions apply just to the RC
phase, or also to the bugfix releases in maintenance branches.

In a previous e-mail, you said in response to the question:

> > Martin remarked that this adds new features to test.support and can
> > only be done in 3.4. Question to release managers. Whether this is
> > acceptable, if we do these functions private (with names that begin
> > with an underscore)? Or would we have to copy the supporting code in
> > every test file with __sizeof__ test?
>
> No, Martin is incorrect on this one. test.support is strictly our turf
> and not covered by the normal standard library rules - the only reason
> it is documented at all is because *we* want docs for it so people
> writing tests don't reinvent too many wheels. Since we reserve the
> right to delete stuff without notice, we can certainly add new stuff.

(from http://mail.python.org/pipermail/python-dev/2012-July/121162.html )

This e-mail is also relevant:

> The extra restrictions during RC are more about not doing things that
> might force the release to be postponed while a broken change gets
> unwound. During normal maintenance, we have a bit more time to recover
> based on buildbot failures or post-checkin review.

http://mail.python.org/pipermail/python-dev/2012-July/121142.html

But with regard to the regrtest changes in issue 15302, which do
fix three bugs (it was marked as a superseder to three issues), Georg commented:

> Please leave this for Python 3.4 -- it is not a bugfix.

It seems like the following would be a very common scenario:

We have bug fixes that could be made to the Python 3.3 maintenance
branch (for example), but the bug fixes are written in a way that
change or otherwise enhance existing test code like test.support or
regrtest.  If enhancements to our test support code can only go into
Python 3.4, then it seems like we are creating an incentive to write
lower quality test code (e.g. avoiding refactoring, and encouraging
practices like cut and paste) -- even for our next minor version.

This creates a trade-off that it doesn't seem like we need to have or
be imposing on ourselves: between having bugs fixed in an earlier
version, and writing higher quality tests for our future versions.

There is also a possible cumulative effect: whereby not committing
test-support enhancements to maintenance branches can also prevent
future bug fixes that depend on that support code from being able to
go in those maintenance branches.

--Chris