[Twisted-Python] Twisted Sprint Report (2013-09-23)
Twisted Sprint Report (2013-09-23) * Venue PyconUK 2013, Coventry, England * Attendees * Lasse Bromose * Stephen Evans * Holger Kraus * Lukasz Prasol * Christophe Reche * Richard Wall * Summary Five Twisted enthusiasts were introduced to the Twisted development process. We closed 2 tickets, made some progress on 2 others and improved some of the development process documentation. * Details ** http://tm.tl/6406 Christophe, Lukasz and Holger helped improve the Trial Howto Documentation. rwall reviewed and merged. ** http://tm.tl/6486 Lasse and Stephen helped with the removal of "has_key" from twisted.mail and documented various undocumented twisted.mail tests. rwall reviewed and merged. ** https://twistedmatrix.com/trac/wiki/ReviewingDocumentation Holger reviewed and fixed broken links to Nevow and Pydoctor. ** http://tm.tl/6162 Lasse and Stephen reviewed the twisted.web Agent Howto and updated the examples to use task.react. ** http://tm.tl/3324 Lukasz and Holger reviewed Shira's new SMTP server tutorial. (Lukasz will attach their patch / notes when he gets home.) Thanks to all of you for your help. I hope you found it interesting and look forward to more contributions from you in the future. -RichardW.
On 12:07 pm, m-lists@the-moon.net wrote:
Twisted Sprint Report (2013-09-23)
* Venue PyconUK 2013, Coventry, England
* Attendees * Lasse Bromose * Stephen Evans * Holger Kraus * Lukasz Prasol * Christophe Reche * Richard Wall
Woo. Hope you guys had fun. Thanks for the report, Richard. Jean-Paul
On 24 September 2013 14:07, <exarkun@twistedmatrix.com> wrote: <snip>
Woo. Hope you guys had fun. Thanks for the report, Richard.
Yep, it was a good day. There were a few things that could have gone better. Here are a few thoughts. * Easy Tickets In hindsight, I should have prepared a selection of suitable tickets. The "easy" tickets search is useful, but many of them have been worked on already and it's difficult to choose which of these are suitable for sprint contributors without reading through the history. One of the tickets I chose had already been mostly completed and the remaining work (filling in missing docstrings for mail avatar tests) was impossible for me to explain, let alone for the new contributors. The "documentation" ticket report is also useful, but with our current version of trac, there's no way of dynamically creating a report which looks for "easy" AND "documentation" keywords. New versions of trac allow this I think. Meanwhile we could write a custom SQL report for this. And again it requires a little preparation to select tickets which haven't got complications. * Windows Development Two of the guys were running windows and I struggled to help them set up a working development environment. This evening I found this page by cyli which might have helped, but probably needs updating. * https://twistedmatrix.com/trac/wiki/TwistedDevelopment#Win32development * Working in pairs Anyway, we soon gave up on Windows and they paired up with the others who were running Linux, and that worked well. * Dev Requirements It would be nice to have a dev-requirements.txt file so that they could easily install the necessary development tools. pydoctor, coverage, nevow, zope.interface, twistedchecker, etc See: * https://github.com/hynek/structlog/blob/master/dev-requirements.txt * Nevow One of the guys was trying to run the documentation tools and we all got thoroughly confused by the many broken links to Nevow. The official link is https://launchpad.net/divmod.org ...but it doesn't include a link to documentation. There are these unofficial links and yet these are linked to from the Twisted docs. https://launchpad.net/divmod https://launchpad.net/nevow The pypi package is well out of date and has a broken project link https://pypi.python.org/pypi/Nevow/0.10.0 The documentation has a broken download link. http://divmod.readthedocs.org/en/latest/ It's such a shame that Nevow got into this state and that it isn't easier to install. It's fantastic software - especially Athena. And I was always partial to Formless too. There was even a lightening talk from a guy touting his *new* pythonic system for generating HTML ie STAN! * Combinator (or alternatives) I've always used combinator to switch between Twisted branches, but it would nice to have some documentation of non-combinator development methods...what tools do people use to update their paths when they switch between branches? I suppose with git you don't have to. Is it worth considering putting combinator on Pypi? * Git Some of them struggled to get a complete subversion checkout (due to dodgy conference wifi I think) and tried to use git instead. It would have been helpful if I'd re-read the latest Twisted git documentation in preparation. It explains how to create a diff containing just the local changes to a checked out branch. * https://twistedmatrix.com/trac/wiki/GitMirror * The Django Sprint By contrast, the neighbouring Django sprint was much better prepared. They'd run a series of tutorial sessions earlier in the conference, on how to contribute to Django. So their sprinters hit the ground running. * https://groups.google.com/forum/#!topic/django-developers/7JFKNoYl2EU * http://dont-be-afraid-to-commit.readthedocs.org/en/latest/ It's a good idea but a lot more work I guess. Anyway, next time I'll know and maybe it'll be a little easier. -RichardW.
On Sep 24, 2013, at 1:39 PM, Richard Wall <m-lists@the-moon.net> wrote:
On 24 September 2013 14:07, <exarkun@twistedmatrix.com> wrote: <snip>
Woo. Hope you guys had fun. Thanks for the report, Richard.
Yep, it was a good day.
There were a few things that could have gone better. Here are a few thoughts.
Thanks for writing this up. It's always good to get better at sprints, and someone taking the time to take note of what went right and what went wrong.
* Easy Tickets In hindsight, I should have prepared a selection of suitable tickets.
It seems like we learn this lesson every time :-). However, I think that one reason that this always comes up is that preparing that selection of suitable tickets is a challenging task in its own right, that someone (or a group of people) needs to set aside time for.
The "easy" tickets search is useful, but many of them have been worked on already and it's difficult to choose which of these are suitable for sprint contributors without reading through the history.
The "easy" ticket search is always like this because every 'easy' ticket needs to have its keyword status re-evaluated after every single bit of work that is done on it; every review, every commit, every patch; maybe even every bug comment. Does anyone want to start an "easy team" that just keeps tabs on this list and tries to keep it updated? :)
One of the tickets I chose had already been mostly completed and the remaining work (filling in missing docstrings for mail avatar tests) was impossible for me to explain, let alone for the new contributors.
This is one reason why we should be encouraging people to write docs first, along with writing tests first :). Once there's an implementation that needs to be explained, bugs start to bottleneck on the original author. But if there's just an explanation that needs to be implemented, it's generally easier for more people with different levels of skill to jump in.
The "documentation" ticket report is also useful, but with our current version of trac, there's no way of dynamically creating a report which looks for "easy" AND "documentation" keywords.
You can sort of half-ass it by searching for 'easy documentation' or 'documentation easy' :-). That'll get you some results, although obviously not every possible combination.
New versions of trac allow this I think.
I've filed a ticket asking for someone to upgrade Trac, but at this point we have pretty much no concrete reasons to do so. If you have specific features in new versions that would be helpful, it would be good to collect them here: <https://github.com/twisted-infra/trac-config/issues/4>
Meanwhile we could write a custom SQL report for this. And again it requires a little preparation to select tickets which haven't got complications.
Presumably the "tickets which haven't got complications" work is covered by curating the "easy" keyword?
* Windows Development Two of the guys were running windows and I struggled to help them set up a working development environment. This evening I found this page by cyli which might have helped, but probably needs updating. * https://twistedmatrix.com/trac/wiki/TwistedDevelopment#Win32development
This already has been updated; a newer version is here: http://blog.ying.li/2012/03/twisted-development-on-windows-v2.html But even that still needs updating :-\.
* Working in pairs Anyway, we soon gave up on Windows and they paired up with the others who were running Linux, and that worked well.
Sad that the Windows people didn't get a good environment that they could take home, but hey, if they take the lesson that Windows is a mistake that causes them pain, so much the better ;-). But, all kidding aside, we could really use more contributors who use Windows and can give a different perspective on it. If people are going to write Windows software - and it seems like we can't stop them - it would be nice if they used Twisted for it. (And they are not going to make the decision that Windows causes them pain; they are going to decide that either Twisted or Python does.)
* Dev Requirements It would be nice to have a dev-requirements.txt file so that they could easily install the necessary development tools. pydoctor, coverage, nevow, zope.interface, twistedchecker, etc See: * https://github.com/hynek/structlog/blob/master/dev-requirements.txt
What tools actually make use of this file? Why is it desirable to keep this information there, instead of the 'requirements' key in setup.py? (Is there a 'develop_requires' key?)
* Nevow One of the guys was trying to run the documentation tools and we all got thoroughly confused by the many broken links to Nevow.
The official link is https://launchpad.net/divmod.org ...but it doesn't include a link to documentation.
There are these unofficial links and yet these are linked to from the Twisted docs. https://launchpad.net/divmod https://launchpad.net/nevow
The pypi package is well out of date and has a broken project link https://pypi.python.org/pypi/Nevow/0.10.0
The documentation has a broken download link. http://divmod.readthedocs.org/en/latest/
It's such a shame that Nevow got into this state and that it isn't easier to install. It's fantastic software - especially Athena. And I was always partial to Formless too. There was even a lightening talk from a guy touting his *new* pythonic system for generating HTML ie STAN!
I'm happy to give you administrative access to anything you need in order to update Nevow's online presence. Also, a couple of Divmod projects have been successfully split out of that repository recently. If you want to increase Nevow's visibility, you might want to follow the same model that PyFlakes and Vertex followed.
* Combinator (or alternatives) I've always used combinator to switch between Twisted branches, but it would nice to have some documentation of non-combinator development methods...what tools do people use to update their paths when they switch between branches? I suppose with git you don't have to. Is it worth considering putting combinator on Pypi?
Right now I use a (frankly unholy) combination of Combinator and git :-) but I have it on a to-do list to change my setup to use virtualenv and git, since that is where the bulk of the Python community lives now and there is just no good reason to keep fighting the current. Combinator's branch-management stuff is made irrelevant if you're using git for development since you can just switch between branches in one working copy, and its merge-management stuff is made irrelevant if you're using even a recent version of Subversion, let alone Git. I'm currently using squash merges to land stuff on trunk, and I guess I need to write up how to do that. That's the only real place where we still "need" Combinator, because it's hard to explain and error-prone how to land stuff with Subversion and not Combinator. However, you can also do this with Git, and it's even explained here: <http://twistedmatrix.com/trac/wiki/GitMirror#gitsvnbranchmerging>.
* Git Some of them struggled to get a complete subversion checkout (due to dodgy conference wifi I think) and tried to use git instead. It would have been helpful if I'd re-read the latest Twisted git documentation in preparation. It explains how to create a diff containing just the local changes to a checked out branch. * https://twistedmatrix.com/trac/wiki/GitMirror
This documentation is actually in pretty good shape now. I've managed to follow the instructions several times without errors. (The trick is, don't try to use git-svn by yourself without help.)
* The Django Sprint By contrast, the neighbouring Django sprint was much better prepared. They'd run a series of tutorial sessions earlier in the conference, on how to contribute to Django. So their sprinters hit the ground running. * https://groups.google.com/forum/#!topic/django-developers/7JFKNoYl2EU * http://dont-be-afraid-to-commit.readthedocs.org/en/latest/ It's a good idea but a lot more work I guess.
On the one hand, the Django project really does a great job onboarding contributors, and they have an amazing culture as regards things like comprehensible documentation and clear workflow. So we should pay attention to their (in many cases superior) example. On the other hand, they have a little less code (~140ksloc for Django, ~170ksloc for Twisted) and a substantially larger community. By Ohloh's measure, <http://www.ohloh.net/p/twisted> vs. <http://www.ohloh.net/p/django> is 148 to 992; roughly 7 times the user-base. So it's not quite reasonable to judge Twisted by the same standard, or to expect that we can achieve Django's results by copying Django's methods, or even that we could faithfully muster enough effort to copy their methods exactly. So we should definitely try to learn stuff from their methods, but also keep realistic expectations and not get disappointed when we still don't have a Django-size or Django-energy-level sprint :-).
Anyway, next time I'll know and maybe it'll be a little easier.
Good luck! Even though you'll know next time, you will still probably need help. Feel free to ask for help next time; reminding everyone of these difficulties in advance can often help get people interested in doing stuff like grooming the easy queue :-). -glyph
* Dev Requirements It would be nice to have a dev-requirements.txt file so that they could easily install the necessary development tools. pydoctor, coverage, nevow, zope.interface, twistedchecker, etc See: * https://github.com/hynek/structlog/blob/master/dev-requirements.txt
What tools actually make use of this file?
pip install -r dev-requirements.txt
Why is it desirable to keep this information there, instead of the 'requirements' key in setup.py?
There is no ‘requirements’ key – and ‘requires’ are runtime dependencies. You don’t want to put your development tools there.
(Is there a 'develop_requires' key?)
No: http://pythonhosted.org/distribute/setuptools.html#new-and-changed-setup-key... There is tests_require which is also something different from development tools and get used when you run setup.py test which we AFAIK don’t.
Am 24.09.2013 um 22:39 schrieb Richard Wall <m-lists@the-moon.net>:
* Dev Requirements It would be nice to have a dev-requirements.txt file so that they could easily install the necessary development tools. pydoctor, coverage, nevow, zope.interface, twistedchecker, etc
JFTR, we need to fix nevow first to be installable w/o Twisted being already present: *** Downloading/unpacking nevow (from -r dev-requirements.txt (line 4)) Using download cache from /Users/hynek/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2FN%2FNevow%2FNevow-0.10.0.tar.gz Running setup.py egg_info for package nevow Traceback (most recent call last): File "<string>", line 16, in <module> File "/Users/hynek/.virtualenvs/tt/build/nevow/setup.py", line 3, in <module> from nevow import __version__ as version File "nevow/__init__.py", line 5, in <module> from nevow._version import version File "nevow/_version.py", line 2, in <module> from twisted.python import versions ImportError: No module named twisted.python Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 16, in <module> File "/Users/hynek/.virtualenvs/tt/build/nevow/setup.py", line 3, in <module> from nevow import __version__ as version File "nevow/__init__.py", line 5, in <module> from nevow._version import version File "nevow/_version.py", line 2, in <module> from twisted.python import versions ImportError: No module named twisted.python *** I have that on my todo-list since the beginning of time but never got around to it.
Am 30.09.2013 um 20:42 schrieb Glyph <glyph@twistedmatrix.com>:
I have that on my todo-list since the beginning of time but never got around to it.
Do you have a link to a Nevow bug in some tracker (launchpad, perhaps?) that is tracking this?
Yes, I believe it’s https://bugs.launchpad.net/nevow/+bug/1091055
On Sep 30, 2013, at 11:51 AM, Hynek Schlawack <hs@ox.cx> wrote:
Am 30.09.2013 um 20:42 schrieb Glyph <glyph@twistedmatrix.com>:
I have that on my todo-list since the beginning of time but never got around to it.
Do you have a link to a Nevow bug in some tracker (launchpad, perhaps?) that is tracking this?
Yes, I believe it’s https://bugs.launchpad.net/nevow/+bug/1091055
Thanks for the reference. -glyph
On 30 September 2013 19:51, Hynek Schlawack <hs@ox.cx> wrote:
Am 30.09.2013 um 20:42 schrieb Glyph <glyph@twistedmatrix.com>: I have that on my todo-list since the beginning of time but never got around to it. Do you have a link to a Nevow bug in some tracker (launchpad, perhaps?) that is tracking this? Yes, I believe it’s https://bugs.launchpad.net/nevow/+bug/1091055
I started looking at this last week, but after discussing it in #divmod, Tom pointed out that pydoctor (trunk) now uses twisted.web.template instead of Nevow, so it would be a better use of time to help mwh release a new version of Pydoctor on pypi. That doesn't help people who actually want to use Nevow, but it should make it a little simpler to set up a Twisted Development environment. -RichardW.
On Tue, Sep 24, 2013 at 7:07 AM, Richard Wall <m-lists@the-moon.net> wrote:
Twisted Sprint Report (2013-09-23)
* Venue PyconUK 2013, Coventry, England
* Attendees * Lasse Bromose * Stephen Evans * Holger Kraus * Lukasz Prasol * Christophe Reche * Richard Wall
I am really excited about the fact that I don't know all of the people on this list -- we need more developers! :-) Thanks for organizing this sprint, Richard! -- Christopher Armstrong http://radix.twistedmatrix.com/ http://planet-if.com/
On Tue, Sep 24, 2013 at 6:58 PM, Christopher Armstrong < radix@twistedmatrix.com> wrote:
On Tue, Sep 24, 2013 at 7:07 AM, Richard Wall <m-lists@the-moon.net>wrote:
Twisted Sprint Report (2013-09-23)
* Venue PyconUK 2013, Coventry, England
* Attendees * Lasse Bromose * Stephen Evans * Holger Kraus * Lukasz Prasol * Christophe Reche * Richard Wall
I am really excited about the fact that I don't know all of the people on this list -- we need more developers! :-) Thanks for organizing this sprint, Richard!
Reports are hard to write, but fun to read. Keep it up. Very nice. =) -- anatoly t.
On Sep 24, 2013, at 8:58 AM, Christopher Armstrong <radix@twistedmatrix.com> wrote:
On Tue, Sep 24, 2013 at 7:07 AM, Richard Wall <m-lists@the-moon.net> wrote: Twisted Sprint Report (2013-09-23)
* Venue PyconUK 2013, Coventry, England
* Attendees * Lasse Bromose * Stephen Evans * Holger Kraus * Lukasz Prasol * Christophe Reche * Richard Wall
I am really excited about the fact that I don't know all of the people on this list -- we need more developers! :-) Thanks for organizing this sprint, Richard!
Me too. This is great, thanks a ton! -glyph
participants (6)
-
anatoly techtonik -
Christopher Armstrong -
exarkun＠twistedmatrix.com -
Glyph -
Hynek Schlawack -
Richard Wall