I have a few questions about feature freeze: (1) Is increasing test coverage acceptable during feature freeze, even if it does not involve fixing a bug? (2) When adding new tests (e.g. in the course of fixing a bug or increasing test coverage), are we allowed to refactor other tests so that supporting test code can be shared? Or should the tests be added in a less DRY fashion and refactored only after the branch goes back to pre-alpha? (3) What types of documentation changes are allowed during feature freeze? For example, are we only allowed to fix incorrect information, or is it acceptable to improve or add to the information about existing functionality? Thanks, --Chris
A follow-up question: during the RC stage, do documentation & test changes require code review by an additional core dev? This also leads me to think about minor (= bugfix) releases. For example, current commits to the 3.2 branch will (presumably) go into 3.2.4 once it's out. Should these commits have the same restrictions as commits to an upcoming release that's in RC stage? Common sense would suggest they should (since any change can break existing functionality and lead to regressions in a previously stable release), but in practice I don't think this is followed. At least not to the letter, with the devguide stating: You cannot skip the peer review during an RC, no matter how small! Even if it is a simple copy-and-paste change, everything requires peer review from a core developer. Eli
On Thu, Jul 26, 2012 at 6:24 PM, Eli Bendersky <eliben@gmail.com> wrote:
The extra restrictions during RC are more about not doing things that might force the release to be postponed while a broken change gets unwound. During normal maintenance, we have a bit more time to recover based on buildbot failures or post-checkin review. Another way of reading that paragraph is "once we hit RC, start postponing everything to the next bugfix release, *unless* at least 2 core devs agree it really needs to be in the current release" Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 26.07.2012 10:24, schrieb Eli Bendersky:
Test changes: yes, doc changes: only if you think it's necessary. The thing about the docs is: we can fix them basically at any time, since most people use the online version anyway, which is rebuilt daily.
But that can also happen for 3.2 -> 3.3.
That is only for RC phase, yes. Reviews are always useful, of course (calling them "peer review" sounds funny to me though), and should be always done for larger patches -- which we are quite used to nowadays. Georg
On Wed, Jul 25, 2012 at 4:29 PM, Benjamin Peterson <benjamin@python.org> wrote:
Also, this response seems somewhat at odds to the response I got to a patch that refactors part of regrest, fixes some bugs in it, and adds its first unit tests (in particular, increasing code coverage): http://bugs.python.org/issue15302 The regrtest module only affects tests (which I would think qualifies it as "tests"), and this is borne out by the following documentation note (which is essentially the same as the note for test.support): "Note: The test package is meant for internal use by Python only. It is documented for the benefit of the core developers of Python. Any use of this package outside of Python’s standard library is discouraged as code mentioned here can change or be removed without notice between releases of Python. "The test package contains all regression tests for Python as well as the modules test.support and test.regrtest. test.support is used to enhance your tests while test.regrtest drives the testing suite." (from http://docs.python.org/dev/library/test.html#module-test ) --Chris
On Sat, Jul 28, 2012 at 8:25 PM, Chris Jerdonek <chris.jerdonek@gmail.com> wrote:
Asking that #15302 be postponed is a risk judgement taking into account "what do we have time to review before rc1?". Cleaning up the argument parsing in regrtest is definitely a good thing to do, but it's not an urgent fix. A similar judgement call will be made for any test suite changes between now and rc1: weighing up the risk of causing problems with the release process vs having improved tests at the time of the release. Anyone with commit rights gets to make that call, and we have three options: yes, no, or ask the Release Manager (Georg for 3.3). Getting to make that final call is one of the main responsibilities of the RM (the default answer is usually "No", given that whichever of us asks the question clearly has misgivings about the idea of including the change). Once we hit rc1, I expect Georg will fork the 3.3 maintenance branch, and any code changes between then and the final release will only happen specifically at his request (the docs will remain open to any committer). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Jul 28, 2012 at 6:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I guess I'm still not clear on the restrictions to making changes to test-related code -- whether these restrictions apply just to the RC phase, or also to the bugfix releases in maintenance branches. In a previous e-mail, you said in response to the question:
(from http://mail.python.org/pipermail/python-dev/2012-July/121162.html ) This e-mail is also relevant:
http://mail.python.org/pipermail/python-dev/2012-July/121142.html But with regard to the regrtest changes in issue 15302, which do fix three bugs (it was marked as a superseder to three issues), Georg commented:
Please leave this for Python 3.4 -- it is not a bugfix.
It seems like the following would be a very common scenario: We have bug fixes that could be made to the Python 3.3 maintenance branch (for example), but the bug fixes are written in a way that change or otherwise enhance existing test code like test.support or regrtest. If enhancements to our test support code can only go into Python 3.4, then it seems like we are creating an incentive to write lower quality test code (e.g. avoiding refactoring, and encouraging practices like cut and paste) -- even for our next minor version. This creates a trade-off that it doesn't seem like we need to have or be imposing on ourselves: between having bugs fixed in an earlier version, and writing higher quality tests for our future versions. There is also a possible cumulative effect: whereby not committing test-support enhancements to maintenance branches can also prevent future bug fixes that depend on that support code from being able to go in those maintenance branches. --Chris
A follow-up question: during the RC stage, do documentation & test changes require code review by an additional core dev? This also leads me to think about minor (= bugfix) releases. For example, current commits to the 3.2 branch will (presumably) go into 3.2.4 once it's out. Should these commits have the same restrictions as commits to an upcoming release that's in RC stage? Common sense would suggest they should (since any change can break existing functionality and lead to regressions in a previously stable release), but in practice I don't think this is followed. At least not to the letter, with the devguide stating: You cannot skip the peer review during an RC, no matter how small! Even if it is a simple copy-and-paste change, everything requires peer review from a core developer. Eli
On Thu, Jul 26, 2012 at 6:24 PM, Eli Bendersky <eliben@gmail.com> wrote:
The extra restrictions during RC are more about not doing things that might force the release to be postponed while a broken change gets unwound. During normal maintenance, we have a bit more time to recover based on buildbot failures or post-checkin review. Another way of reading that paragraph is "once we hit RC, start postponing everything to the next bugfix release, *unless* at least 2 core devs agree it really needs to be in the current release" Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 26.07.2012 10:24, schrieb Eli Bendersky:
Test changes: yes, doc changes: only if you think it's necessary. The thing about the docs is: we can fix them basically at any time, since most people use the online version anyway, which is rebuilt daily.
But that can also happen for 3.2 -> 3.3.
That is only for RC phase, yes. Reviews are always useful, of course (calling them "peer review" sounds funny to me though), and should be always done for larger patches -- which we are quite used to nowadays. Georg
On Wed, Jul 25, 2012 at 4:29 PM, Benjamin Peterson <benjamin@python.org> wrote:
Also, this response seems somewhat at odds to the response I got to a patch that refactors part of regrest, fixes some bugs in it, and adds its first unit tests (in particular, increasing code coverage): http://bugs.python.org/issue15302 The regrtest module only affects tests (which I would think qualifies it as "tests"), and this is borne out by the following documentation note (which is essentially the same as the note for test.support): "Note: The test package is meant for internal use by Python only. It is documented for the benefit of the core developers of Python. Any use of this package outside of Python’s standard library is discouraged as code mentioned here can change or be removed without notice between releases of Python. "The test package contains all regression tests for Python as well as the modules test.support and test.regrtest. test.support is used to enhance your tests while test.regrtest drives the testing suite." (from http://docs.python.org/dev/library/test.html#module-test ) --Chris
On Sat, Jul 28, 2012 at 8:25 PM, Chris Jerdonek <chris.jerdonek@gmail.com> wrote:
Asking that #15302 be postponed is a risk judgement taking into account "what do we have time to review before rc1?". Cleaning up the argument parsing in regrtest is definitely a good thing to do, but it's not an urgent fix. A similar judgement call will be made for any test suite changes between now and rc1: weighing up the risk of causing problems with the release process vs having improved tests at the time of the release. Anyone with commit rights gets to make that call, and we have three options: yes, no, or ask the Release Manager (Georg for 3.3). Getting to make that final call is one of the main responsibilities of the RM (the default answer is usually "No", given that whichever of us asks the question clearly has misgivings about the idea of including the change). Once we hit rc1, I expect Georg will fork the 3.3 maintenance branch, and any code changes between then and the final release will only happen specifically at his request (the docs will remain open to any committer). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Jul 28, 2012 at 6:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I guess I'm still not clear on the restrictions to making changes to test-related code -- whether these restrictions apply just to the RC phase, or also to the bugfix releases in maintenance branches. In a previous e-mail, you said in response to the question:
(from http://mail.python.org/pipermail/python-dev/2012-July/121162.html ) This e-mail is also relevant:
http://mail.python.org/pipermail/python-dev/2012-July/121142.html But with regard to the regrtest changes in issue 15302, which do fix three bugs (it was marked as a superseder to three issues), Georg commented:
Please leave this for Python 3.4 -- it is not a bugfix.
It seems like the following would be a very common scenario: We have bug fixes that could be made to the Python 3.3 maintenance branch (for example), but the bug fixes are written in a way that change or otherwise enhance existing test code like test.support or regrtest. If enhancements to our test support code can only go into Python 3.4, then it seems like we are creating an incentive to write lower quality test code (e.g. avoiding refactoring, and encouraging practices like cut and paste) -- even for our next minor version. This creates a trade-off that it doesn't seem like we need to have or be imposing on ourselves: between having bugs fixed in an earlier version, and writing higher quality tests for our future versions. There is also a possible cumulative effect: whereby not committing test-support enhancements to maintenance branches can also prevent future bug fixes that depend on that support code from being able to go in those maintenance branches. --Chris
participants (5)
-
Benjamin Peterson -
Chris Jerdonek -
Eli Bendersky -
Georg Brandl -
Nick Coghlan