[Python-Dev] GSoC: Core Python development tools

Daniel (ajax) Diniz ajaksu at gmail.com
Mon Mar 23 20:14:44 CET 2009

C. Titus Brown wrote:
> Given the relative paucity of core Python GSoC ideas, I think you should
> go for both of these, *especially* if you have a mentor up front.  So,
> write 'em up, add 'em to the GSoC page, and let's see who we get...
> If we get good applications for both, then I think we can "fund" both of
> them.

Great, thanks a lot for the feedback :)

> I do think you should be prepared for pushback from python-dev on any
> such ideas ;).  Don't get too ambitious about writing up *your* way of
> fixing things, but rather make sure you and the students are prepared to
> adapt to what people on python-dev think.  Mind you, ultimately the
> people doing the work should make the decisions, but input from
> python-dev is usually pretty useful even when it's contradictory!

Pushback? Luxury! :)
Thanks for the great advice, but I'd actually love negative input on
this. Better to find out early that there's no way out of the
bikeshedding tar pit than to waste a slot (and the student's time).

I think many people don't speak up for or against GSoC proposals
because they don't have time to mentor / discuss details. This
particular proposal is doomed if nobody voices the itches to be

So I'm kinda taunting python-dev with a huge proposal that goes around
generalities and tries to make the case for joining efforts with PIDA

Skipping to "Suggestions" and the "Food for thought example" might
help in deciding whether the wall of text is worth a look...

Best regards,

Helper Python core development tools.

  There's some amount of repetitive, required steps to work on Python
development. This proposal is about improving the workflow of core
developers, probably with small glue scripts.  As each developer has
his own personal workflow, these helper scripts should be optional,
easy to extend and aimed at the most common tasks.
  Of course, there may be no real demand for optimizing the workflow.
Avoiding the use of a GSoC slot for unnecessary projects is very
important, so if you think it's a wast of resources, please speak up

  Sometimes, non-obvious bits like the right sequence of svnmerge
commands, the right way to link a Rietveld code review to a given
issue or checking for correct autoconf version might get in the way of
real development.
  Other obvious steps, like handling issue properties (e.g.
Resolution), posting a message that tells in which revs the issue was
fixed of or even checking for changes in tabs versus spaces, also
require a bit of time.
  Regardless of obviousness, forgetting one item (or getting it wrong)
in the development checklist happens from time to time.
  If there are ways to factor these repetitive, required tasks out
from a core dev's concerns, it can only help the development process.
Non-code developers could also benefit from this kind of tool, and
could contribute in a more efficient way to Python development, with
higher code/ticket quality. Quoting the tracker cleanup proposal:
    Optimizing the application of main/core developers' time to Python issues
    is a no-brainer. Being able to add volunteers to the productive time pool
    is also very desirable.

  Identifying which tasks and steps should be optimized cannot be left
to the student nor to the mentor, it must be a collective effort.
Clear goals must be set before someone try to implement them. Sure,
there are many descriptions of workflows in past python-dev (and
python-list) threads, but the GSoC is about code.

  It's up to the mentors/student team. I suggest offering early
releases for the scripts and maybe test repositories, trackers,
Rietveld instance, etc., as ways of making sure the resulting code
will be useful and used by the target public.

  I believe most tools should be developed in a generic way, so that
they could fit in other projects and workflows. IMHO, this would make
the resulting scripts much more valuable.

  Besides some experience with Python and tools of the trade (VCSs,
bug trackers, etc.), the most important requisite is being able to
listen to the community and taking criticism well.

  Ali Afshar from PIDA is willing to mentor if the 'generic tools' way
is accepted. Since it's  about reusing development tools inside an
IDE, any of these helper scripts could be integrated into PIDA, the
only pre-requisite being that they'd not be too deeply linked to the
Python infrastructure. With a small bit of concern about this, it'd be
easy to extend/fork the resulting tools to use with other trackers,
VCSs, test runners and code review tools.
  I am also available to help with the Python workflow part, and
there's an early effort to integrate Rietveld and our bug tracker
under way.
  At least one core Python developer should mentor, preferably one
that knows how to interact well with python-dev. Any project focused
on how people work is prone to flamefests, so diverting most heat away
from the student might be a necessary skill.
  Bikeshedding is also very likely to occur, but the proposal's goals
is to provide a couple of brushes and some paint cans of different
colo(u)rs :)

  Should include a couple of helper scripts, maybe some patches to
currently used tools. Other ideas?

  Wrappers for working with tracker issues, managing patches, running
tests, code reviews, committing (including code checking hooks),
merging, etc. E.g., I'm working on a script that allows one to submit
a Roundup issue and a Rietveld issue in a way that links them

Food for thought example (sorry, not quite how a core dev works):
ajaksu at Belkar:~/py3k$ pyfix 5434 --export ../month_delta --no-assign
Issue 5434: datetime.MonthDelta
Type:  feature request   Stage: not selected
Versions: 3.1            Nosy list: jess.austin,lemburg,tim_one,+ajaksu2
Assigned to: nobody (--no-assign)

SVN exporting current working copy to: ../month_delta
Export complete

Downloading files:
    monthdelta.py -> ../month_delta/patch/monthdelta.py OK
    monthdelta.diff -> ../month_delta/patch/monthdelta.diff OK
Downloaded 2 files:
monthdelta.py jess.austin, 2009-03-11 18:39 prototype implementation in python
monthdelta.diff jess.austin, 2009-03-12 17:54 updated patch, unified
diff against py3k

Downloaded 4 messages:
    ../month_delta/msgs/msg83272.txt Jess Austin: datetime is a wonderful (...)
    ../month_delta/msgs/msg83273.txt Jess Austin: This is my first try at (...)
    ../month_delta/msgs/msg83275.txt Jess Austin: Rietveld link: (...)
    ../month_delta/msgs/msg83480.txt Jess Austin: This prototype python (...)

Rietveld link found: http://codereview.appspot.com/25079

ajaksu at Belkar:~/py3k$ cd ../month_delta
ajaksu at Belkar:~/month_delta$ pyfix patch
PyFixing issue 5434...
One patch found: patch/monthdelta.diff
    jess.austin, 2009-03-12 17:54 updated patch, unified diff against py3k
Applying... OK
ajaksu at Belkar:~/month_delta$ pyfix set stage "patch review"
ajaksu at Belkar:~/month_delta$ pyfix status
PyFixing issue 5434...
Stage: not selected -> "patch review"
D    M Doc/c-api/datetime.rst                 2 chunks     34 lines
D    M Doc/library/datetime.rst              10 chunks    354 lines
B    M Include/datetime.h                     7 chunks     82 lines
T    M Lib/test/test_datetime.py              2 chunks    231 lines
D    M Misc/NEWS                              1 chunk      14 lines
B    M Modules/datetimemodule.c              20 chunks    781 lines
B: build required - T: tests - D: docs - G: backtrace

ajaksu at Belkar:~/month_delta$ pyfix test
Error: build required, try to test anyway? [n]
Test aborted
ajaksu at Belkar:~/month_delta$ pyfix build
./configure --quiet
make -s
Python build finished, but the necessary bits to build these modules
were not found:
bsddb185           sunaudiodev
ajaksu at Belkar:~/month_delta$ pyfix test
./python -E -tt ./Lib/test/regrtest.py -l  -uall -rw
344 tests OK.
25 tests skipped.
ajaksu at Belkar:~/month_delta$ pyfix up -m"LGTM, needs testing on other
PyFixing issue 5434...
No differences found.
Setting stage: not selected -> patch review
Adding message:
    Author: Daniel Diniz
    LGTM, needs testing on other platforms.
ajaksu at Belkar:~/month_delta$

More information about the Python-Dev mailing list