[python-committers] improving our code quality [my summary of the "PQM" thread]

[Brett Cannon]

The thread on PQM has now grown multiple points and it becoming a
little hard to follow since it is pulling in a bunch of ideas not
directly related to a PQM system, so I am going to attempt to
summarize the points brought up here. SUMMARY: it's great we are
talking about this, but I think this would be better to discuss after
2.6/3.0 is out the door.

What are our dev rules, and are we following them?
---------------------------------------------------------------------

Obviously this is all pointing out the fact that Python's popularity
has grown such that the amount of hours available from core developers
is far exceeded by what we really need to keep stuff going the way we
have been doing things. Short of tossing out commit privileges like
candy on Halloween outside of a grade school, something really should
change.

We already have some things in place that do help. Requiring tests has
helped catch issues on the buildbots and made sure we didn't repeat
the same mistakes. We also all know better than to introduce anything
major during the beta cycle, but the thought of having some solution
for multi-core processing was too tempting so we ignored general
practice for the multiprocessing module and now we are paying for it
late in the release cycle. We also want everyone to run the ENTIRE
testing suite before a commit is made, but I am sure we are all guilty
of skipping a full '-u all' run when the change is thought to be minor
(and I know I never run with random test order like the buildbots).


Code reviews
-------------------

One suggestion on how to improve the situation has been code reviews.
Ideas have ranged from code reviews for everything to "who needs code
reviews?" Doing a code review on everything will obviously lower the
bandwidth on committed code and we already don't have that many people
who review pre-existing patches as it is. But some review should
probably be done for anything significant. I am leery of suggesting we
go with guidelines based on line count, but probably anything that
takes more than an afternoon to implement should probably get a code
review. And yes, working in branches would help with this.


Buildbots
-------------

We also have the buildbots which are handy when you are trying to fix
something that is multi-platform, but otherwise they are just email
generators. I think we really need to work on fix the various flaky
tests so that false-positive failures stop occurring and leading to
people ignoring the buildbot emails. And I personally like Christian's
idea of a single summary email at the end of the day of buildbots that
are still failing that goes to everyone potentially involved in the
breakage along with to a mailing list.


PQM
-------

There is also the PQM suggestion to make sure we always at least have
some general sanity in the code base. But a PQM system would probably
only work once the test suite is not flaky anymore as having some typo
in a comment be rejected because test_kqueue failed again is not going
to go over well with most people. And that also goes for OS-specific
failures on a platform I am not running on. Honestly the PQM system
probably wouldn't be necessary if people just ran the entire test
suite before a checkin. And if people never checked in if there was
any failure (related to their code or not) we would probably get the
test suite cleaned up a lot faster as well.


How are we going to solve this?
------------------------------------------

I think a thorough discussion on how we have been handling development
is needed. We all want to continue to enjoy developing for Python and
we want it to continue to be a high-quality project. Because of this
we might have to compromise on what we might do if this were a
business with paid positions, but I don't think it will be anything
major. And I think the 2.6/3.0 development cycle has shown we really
should be changing something to make our lives easier.

But I honestly think this discussion should wait until after we go
final with 2.6/3.0. Once that happens and we are all not running
around trying to get a release out, then we should start from the
ground up and re-evaluate what we need to change. That includes the
workflow in the issue tracker, how the buildbots are used, what our
expected best practices are, cleaning up the test suite, how serious
we are about moving to a distributed VCS, whether we want a PQM, do we
want code reviews, etc. But I really think we should be putting the
time in now to b3 and working towards hitting final before we delve
into this lengthy conversation.

And yes, I am willing to help help with all of this through PEPs,
documenting what are guidelines are for new developers, doing stuff
through the infrastructure committee, etc. (although importlib will be
finished first, although not necessarily committed if our commit
practices change, since I had to miss 3.0 in order to get PEP 3108
done and I refuse to miss another release).

-Brett