Mailman 3 July 2006 - Twisted-web

Formal validation with custom locateChild
by Franz Zieher 18 Jul '06

18 Jul '06

Being a beginner in using nevow/formal I can't see that my form gets validated nor the callbacks are executed when I use my own locateChild method. Do I have to take special care in the locateChild method if the form is posted? Thanks, Franz

2 1

[Twisted-web] Thoughts on documentation, wading into Twisted, Nevow, etc.
by Terry Jones 17 Jul '06

17 Jul '06

Here's a summary of thoughts I've had over the last month or so. Please try to take the following as positive / constructive. I'm happy to be involved in trying to alleviate what some people may agree are problems. Or people may not agree that they're problems, which is fine too - more on this below. A few months ago I started looking at various (mainly Python) frameworks with an eye to using one for a project I'm working on. When I found Twisted (and friends) I was really happy. I've been writing code for what seems like a long time (30 years) and I think I'm pretty critical and demanding when it comes to good and clean design. I certainly had not expected to find something like Twisted and its various additional components. Without having written a line of Twisted code, I made the decision to throw away a full year of my own low-level, efficient, debugged, C code (networking, custom RPC, argument marshaling, tons of sweat). That was a pretty serious step, not taken lightly, and, I still think, a good one. That's why I'm still here, spending hours writing this email, wanting to press ahead, and willing to put in time and effort to help. In one corner then, we have a great foundation - elegant, dynamic, well designed, full of potential, nice programming language, written by talented and responsive developers, etc. But... in the other corner, documentation that very often seems to be a complete shambles. There, I've said it. I don't claim that this is actually a problem. That depends. It may well be that I've happened along at a moment which is not optimal for my needs and abilities; that there is a well-organized design, development, and documentation trajectory; and that I just happened to arrive at a bad point on that curve. I fully sympathize with things being less well documented than they could otherwise be. I love writing code. Following the rise of highlighting editors I've even become quite fond of comments. But I'm not even slightly fond of writing external docs. I think an important part of getting things right in design is to implement something that feels right and then rework it until it really feels right and clean, and is also useful. There's little point in documenting things before they have settled. Writing code, debugging, re-designing, building apps on top of libraries - all those things, to me, should take precedence over documentation. I don't believe in writing fully fleshed-out specs, docs, and tests ahead of implementation. I even have little time or sympathy for people (like myself in this case) who don't have what it takes to use emerging tools before they're fully documented for use. I've spent plenty of my own free time in the last 20 years writing, giving away, and supporting, code I felt like producing. So I think I understand the equation pretty well. But, Twisted and its related components are released. Valuable (to the project) people are trying to use them. This can be frustrating. I think I have a reasonably high frustration threshold (but maybe I'm wrong - maybe I'm just old & cranky). It's quite general to talk about "Twisted and its related components". There are many moving parts (some conveniently accompanied by moving names). So let me just pick on Nevow - it's perhaps in the middle in terms of maturity, documentation, frustration level, etc. I actually don't mean to single out Nevow in particular, but it has qualities that make it a good example target for discussion. Apologies in advance to anyone this seems to pick on - that's not my intention, I have no idea who wrote (or didn't write) some of this stuff. I know Matt is involved as he's been kind enough to several times send me long explanations and code in email. Again, I'm not claiming there's a problem here. I find Nevow to be extremely promising and yet quite cryptic. Sadly, I've built dozens of web sites, many of them commercial, done under contracts with real deadlines, budgets, staff, etc. I hate editing ugly templates in awkward ugly non-code file formats. There's just so much disgusting stuff out there. So to see Nevow and Stan is a real pleasure. But to try to understand it... argh. Hop onto google and try "nevow documentation" and read through some other people's experiences (e.g. http://www.third-bit.com/pywebblog/archives/cat_twisted.html) The recent "nevow documentation" thread (kicked off by Manlio, at http://twistedmatrix.com/pipermail/twisted-web/2006-May/002652.html) is mainly illustrative for its lack of conclusion. I don't find the suggested page at http://divmod.org/trac/wiki/TitleIndex to have much (if anything) to do with documentation - it seems to be links to code. It's not clear if the wiki is more a help or a hindrance (this is probably too condemning, but after spending more than enough time looking at things that one later discovers are out of date, replaced, deprecated, etc., you start to wonder about everything). Nevow comes with many examples, and that's great. But they're still cryptic. E.g., a grep url\\. */*.py in Nevow's examples directory gets me over 30 hits, but I don't see that class documented anywhere and when I try to use it in my code it doesn't work. E.g., formless is used in many examples, but is apparently a dead end. If you disagree with that, you'll be reinforcing my sentiment that things are a bit of a mess :-). I can give more examples. There's quite a bit that seems very cryptic, and I mean beyond the things described in the already very terse and piecemeal Nevow documentation. Of course it takes time to spell things out, but if people want them I will. I already decided to take the evening off, have a couple of beers, and write this email. Before trying to finish on a politically correct, upbeat, & constructive note, here's a question: How was it that Twisted (or just Nevow?) managed to lose the attentions of Kieran Holland? He (she?) was clearly into Nevow, put a lot of effort into understanding things, and yet headed off to Django, which, reading between the lines, he/she didn't really think was even nearly up to the Twisted/Nevow standard. That's a bad loss. People who can read and write code and who are willing to spend serious amounts of time helping ease others into projects are so valuable. The Addendum of Kieran's Stan page (http://www.kieranholland.com/code/documentation/nevow-stan/) seems to spell things out pretty clearly. It feels like the Nevow (and broader?) community has done itself a serious disservice in losing Kieran's attention (but, see above comments on whether this is perceived as a problem, trajectories, etc). To me, totally on the outside, I do not get the impression that there is a clear design, code, and release trajectory in place. It feels like there is a bunch of very talented designers and coders who are banging out an increasing number of cool projects at high speed, leaving a frustratingly incomplete, misleading, and atrophying breadcrumb trail of documentation behind them. On to another question: do people think there a problem here, or not? The answer may be no. I think that's fine, I'm happy to be told to read the code and to shut up. At least that would be definitive, and not at all unwelcome. But, if you think there is a problem, what can we do? Here are some possible concrete steps that wouldn't take huge amounts of time. 1) Go through all Twisted & friends web sites and wiki and _ruthlessly delete everything_ that mentions obsolete, deprecated, or renamed projects / modules, broken links, example code using such, etc. There are many of these things, and they are highly misleading and frustrating and give a bad overall impression. 2) State, where appropriate, that the source code is currently the best documentation. There's no shame in this, and it's much better than having people run across and waste time on obsolete docs, or spend time looking for things that do not exist. E.g., "At this point the nevow.url class is totally undocumented. See examples/{x,y,z} for example usage and the code and comments in nevow/url.py for the best current documentation." 3) Have someone, preferably an outsider (could be me), write up and maintain a no-punches-pulled overview document about using the various Twisted components. Stick a link to this in a prominent place. http://twistedmatrix.com/trac/wiki/WebDevelopmentWithTwisted is a good start. 4) Have someone go through example code to stick in more comments, get rid of old stuff (no need to rewrite if that takes too long, just throw it away). E.g., the widespread use of formless in Nevow examples. Is it in, is it out? If the latter, get rid of it before more people die. 5) Have someone do some web searches for stuff known to be fixed or out of date, and send mail to the authors asking if they'd please consider updating or otherwise clarifying what they've written, could they add a link or a postscript, to help out future people looking to use Twisted & friends? No need to try to hide the fact that things were a bit of a mess - just aim not to have people get an out-of-date impression or follow links to incorrect / obsolete pages (e.g., links to nevow.com). I could probably come up with more suggestions. Given the inclinations of (I assume) many people still reading this message, no-one is too much inclined towards producing actual documentation (and it may not be the right moment, as above). So it makes sense to focus on easy hits: purging old crud, minimizing wasting the time of people looking to get involved, making it clear what the barriers are (e.g., point people at the source, tell them that at this point the only way to understand is to ask questions (but time is precious) and to read the code). OK, enough words for now. Thanks again for all the help, all the code, etc. Terry

9 16

Re: [Twisted-web] Re: Thoughts on documentation, wading into Twisted, Nevow, etc.
by Terry Jones 16 Jul '06

16 Jul '06

Thanks for all the very thoughtful replies to my long mail. I didn't mean to whine about things and then just disappear back into the woodwork, I just have to deal with various other things right now. I'm thinking about what I wrote, the various replies, what might be effectively done, etc. I hope to be able to follow up again soon. Terry

11 36

Fwd: HTTP Bar BoF Summary
by Wilfredo Sánchez Vega 13 Jul '06

13 Jul '06

Just FYI. -wsv Begin forwarded message: > Resent-From: ietf-http-wg(a)w3.org > From: Mark Nottingham <mnot(a)mnot.net> > Date: July 13, 2006 10:18:08 AM PDT > To: HTTP Working Group <ietf-http-wg(a)w3.org> > Cc: discuss(a)apps.ietf.org > Subject: HTTP Bar BoF Summary > > > We talked about potential work on 2616. > > Everyone seemed to agree that cleaning up the document (e.g., > folding in errata, documenting the extensibility model) is a clear > win. There was also considerable interest in going a bit further; > e.g., exploring implementation divergence, potentially deprecating > features that have proven to be interoperability problem, and > generating operational documentation. When asked, no one thought > that this level of activity was a bad idea, several stated keen > interest, and some indicated that they would be willing to > contribute actively. > > No one indicated interest in working on "HTTP/1.2," but rather just > more clearly documenting 1.1. There was also interest in defining > extensions to HTTP, but those who spoke supported focusing on the > 2616 cleanup first. > > 2617 was also discussed, in particular Digest authentication; while > there are some errata that could be incorporated in a backwards- > compatible fashion, several people raised issues that they thought > could only be addressed by a backwards-incompatible revision to > Digest (i.e., a new Digest-like HTTP authentication scheme within > the HTTP authentication framework). Again, there was broad support > for such work, and no detractors spoke up. There was some > discussion of whether this would be relevant to the activities in > the WAE BoF, but those who spoke felt that this effort had a > different scope, and would be most appropriate in an HTTP-focused WG. > > It was agreed that the next steps to explore these items is to: > * Engage implementers, ideally holding an interop (either virtual > or real) to discover and explore various problem areas > * Write a problem statement exploring the issues to be addressed > more deeply > * Prepare a straw-man charter for an umbrella HTTP WG > * Request a BoF in San Diego > > We then recessed to drink beer and eat French fries covered in > gravy and cheese. > > Attendees: > - Phillip Hallam-Baker > - Scott Lawrence > - Lisa Dusseault > - Mark Baker > - Dave Raggett > - Carsten Bormann > - Cyrus Daboo > - Thomas Roessler > - Yngve Pettersen > - Alexei Melkinov > - Mark Nottingham > > -- > Mark Nottingham http://www.mnot.net/

1 0

web2: reverse dns lookup in twisted reactor thread
by Div Shekhar 12 Jul '06

12 Jul '06

Hi, I've been using web2 for WSGI apps for a few months, and it's been working fine. However, I did run into one bug this week: web2 does a reverse dns lookup of the client ip address, and in the twisted reactor thread, no less! When the dns server is unreachable, this delays that http request and blocks twisted as well. I'm planning to solve this by commenting out line 490 in twisted/web2/channel/http.py: host.host = _cachedGetHostByAddr(host.host) I'd recommend that this lookup be moved to wsgi thread and/or be turned off by default. Just wanted to check - am I missing something? Cheers, Div.

4 6

Re: [Twisted-web] Re: Thoughts on documentation, wading into Twisted, Nevow, etc.
by Terry Jones 08 Jul '06

08 Jul '06

I'm not sure which of the many messages to followup on, so I've picked LLoyd's summary. Below are a couple of thoughts on the recent posts. First of all, I wasn't meaning to single out the core of Twisted when talking about documentation. Like some others, I also find the core to be relatively well documented. But there are a variety of additional components (as well as parts of the core, I assume) that seem much less well documented, and some parts that are now deprecated or discontinued but, confusingly, for which apparently current documentation is still online. So I don't think we should simply focus on the core part of Twisted. I thought Cory's comment was good: Cory> Just a final thought: there's still lots of areas that *aren't* Cory> adequately documented by anyone's standards. There are areas not Cory> covered by examples, there are areas not covered by HOWTO's. The Cory> core is covered, but any new application worth writing probably uses Cory> stuff from the fringes. So our hypothetical new Twisted user starts Cory> out writing her new application, gets a handle on the core stuff, and Cory> then discovers those gaps in the fringes. That can be frustrating. Cory> Everyone uses a different part of the fringes, so the fringes need to Cory> be covered almost as well as the core. My feeling is that the documentation falls off pretty steeply. E.g., guard and sessions in Nevow. There's a lot going on in quard.py. I don't think this is a particularly fringe issue - in fact it seems likely to be quite central to many people using twisted.web. I like the idea of a cookbook with recipes written by various people. That sounds like it might be the most useful thing to me anyway. I like the O'Reilly Cookbook series. E.g., the Python and Perl cookbooks. They are immediately useful, even to a complete newbie. A cookbook distributes responsibility and allows indefinite expansion. It would be good to have a cookbook coordinator to set standards on appearance, maintain the cookbook pages online etc. If someone were really keen, they could approach O'Reilly. I think it's important that we make an effort to ensure that whatever we produce does not go out of date. If I (for example) decide to write something up but I don't commit to maintaining it, then when it inevitably becomes outdated it begins the slide towards increasing harm and decreasing good. Along these lines, and given the power and beauty of Twisted :-), I would think it should be possible to have a documentation framework in which example code from a cookbook could be verified as still working. Of course this requires more time and programmer effort. But it's already clear that better documentation will have to be a joint effort between people who feel like helping by writing and people who feel like helping by explaining things. One danger along these lines (this is something I mailed Matt Goodall about wrt Nevow) is that you don't want the framework to obscure clarity. I.e., you don't want an all-powerful framework that hides many things, cutting the examples to the barest elegant minimum, because the poor reader then has to (potentially) try to figure out how to work backwards and construct working standalone code given just a fragment from an example. I'm not saying this is easy - it's not - just that it's a desirable goal. One way I've done something like this is to have a collection of templates, and you use make (or other) to cat these together (and test) entire examples while just including relevant fragments in other documentation. Again, this needs programmer help to set up. We should differentiate between trying to help people who want to use Twisted and people who are shopping around. Both are valid targets for people who want to write, but we may as well keep the two clearly separated. In terms of the latter (people trying to understand what Twisted is, why they might want it), I think something like the finger example (translated out of the 1980s) is very compelling. It's one of the things that sold me. That particular example would benefit from having the new code in each step highlighted. Look at how much the finger example covers. As I wrote to some friends a couple of months ago: | The following has a great example of the kind of thing you can do | with Twisted. | | http://twistedmatrix.com/projects/core/documentation/howto/tutorial/index.h… | | You don't need to read it all of course. Below is a bullet-point | summary of what they do on those pages. | | Amazingly, the final code is just 225 non-blank lines. If you remove | the one-line {class,def,import} statements, and comments, you're left | with a mere 118 lines of code. In case it's not clear, the end point | is a server doing everything mentioned below, all at the same | time. So they wind up handling about 6 different protocols on 6 | different ports, each one independently pluggable. You ain't going to be doing anything like that with Django, TurboGears, or RoR anytime soon - unless I'm greatly mistaken. So, threading vs async aside, I don't think it's hard to differentiate Twisted from what I think of as just "web" frameworks - which are all focused on building web sites. Glyph - thanks for the vote of confidence, but I can't step up to be the Nevow documentation coordinator. I just have too much going on, and I think it would be bad to undertake something like that without feeling like I was going to commit to continuing for some time, to maintaining it etc. I hope I can do other things, like write a cookbook chapter, or write code to help set up a documentation framework, or, my personal choice, use Twisted to make millions and then just send you cash... Terry

1 0

Bay Area Twisted Sprint - July 15th and 16th
by David Reid 08 Jul '06

08 Jul '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This is the official announcement of the Bay Area Twisted Sprint. It is to be held on July 15th and 16th at the offices of Three Rings Design Inc. (the people who brought you that wonderful tool of procrastination "Puzzle Pirates") which are located at: 612 Howard St, Suite 500 San Francisco, CA 94150 Google Map: http://tinyurl.com/pb588 The sprint will last from 10am-8pm on Saturday the 15th and Sunday the 16th, and will focus on closing outstanding bugs (perhaps followed by some sort of maintenance releases.) If you're going to be attending the sprint reply to this email so that our gracious hosts have some idea of what to expect. Thanks - -David - -- "Usually the protocol is this: I appoint someone for a task, which they are not qualified to do. Then, they have to fight a bear if they don't want to do it." -- Glyph Lefkowitz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEr0AirsrO6aeULcgRAvgVAKCJkgO7r/4Bstck16GS3VaYArRUnACfXv8s sE45pM4dGEdPAQ5hok4IyCs= =gfYe -----END PGP SIGNATURE-----

1 0

Re: Twisted-web Digest, Vol 28, Issue 8
by Yunier Rojas García 07 Jul '06

07 Jul '06

I want to make it from scratch, importing modules, but when I receive a message I can't find the way to send it back with the correct answer, and in the dns client (nslookup) says "no answer". On 7/6/06, twisted-web-request(a)twistedmatrix.com < twisted-web-request(a)twistedmatrix.com> wrote: > > Send Twisted-web mailing list submissions to > twisted-web(a)twistedmatrix.com > > To subscribe or unsubscribe via the World Wide Web, visit > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web > or, via email, send a message with subject or body 'help' to > twisted-web-request(a)twistedmatrix.com > > You can reach the person managing the list at > twisted-web-owner(a)twistedmatrix.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Twisted-web digest..." > > > Today's Topics: > > 1. twisted names server ( Yunier Rojas Garc?a ) > 2. Re: twisted names server (Stephen Thorne) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 5 Jul 2006 19:11:30 -0700 > From: " Yunier Rojas Garc?a " <onemoverx(a)gmail.com> > Subject: [Twisted-web] twisted names server > To: twisted-web(a)twistedmatrix.com > Message-ID: > <9f45496e0607051911p7c675637k489126fa07f77aaa(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > can anyone please tell me how can I make a simple dns server using > twisted.names? >

1 0

[Announce] Nevow-based weblog on the works
by Adrian Perez 06 Jul '06

06 Jul '06

Hello everybody! As I mentioned earlier in the Twisted-Web mailing list, I'm coding a OSS weblog which uses Nevow in my spare time. I started it two weeks ago but haven't made a big progress because of exams. Now I will be on holidays (about a week) and I would like you to know about it, because I would like feedback from the community and some people asked for the code. The weblog system is named Bic (like the ball-pens). Currently, it boots up, shows up the main page, per-article pages, has a form for adding articles, and articles may be edited. Talking with the people at #divmod (FreeNode) they recommended me to avoid formless. I plan to move to use formal instead. Bic needs the Pyramid package in order to work. It's an utility library I coded myself with some Python utilities I tend to use regularly. Some notes not written elsewhere: - Accessing /article/<id> (or /<id>) will take you to an article given its identifier. - Accessing /article/<id>/edit (or /<id>/edit) allows for editing articles. - Accessing /new allows for writing an article (this will change). - Accessing /style yields a small page for the installes styles. - Adding "?lang=es" in the request URL changes the language to Spanish. (the thing has localization support, of course). The thing is MIT-licensed and you can get a copy of the code if you want: - Use bzr to get a working copy out of the repository: $ bzr get http://code.connectical.net/pyramid $ bzr get http://code.connectical.net/bic (You can also browse the latest revision with your favourite web browser.) - Get a tarred snapshot of my working copy, the URL is: http://code.connectical.net/nn-releases/pyramid+bic-20060706.tar.gz (The tarball includes both Bic and Pyramid.) Any feedback will be greatly appreciated. I'm quite new to Nevow and Twisted so I am willing to learn how to do the Right Thing(tm). Best regards, P.S: One final reminder: be sure to have Pyramid in your PYTHONPATH before running the Bic tacfile with twistd ;-) -- Adrian Perez "Experience is what you get when you don't get what you want" -- (Dan Stanford)

1 0

twisted names server
by Yunier Rojas García 05 Jul '06

05 Jul '06

can anyone please tell me how can I make a simple dns server using twisted.names?

2 1