OK, I'm already making a lot of these changes, I should have a new version Up Soon (TM).
Placeholders.
Placeholders for what, though?
Placeholders for that magic lib, that I think I'm going to stop looking for and say, "demonstration purposes only" How do they use it? Why is it relevant? I guess I'm stupid or slow, but it took a while for me to realize that Deferreds were basically a standardized callback mechanism. It's not really written anywhere on the tin: Deferred was to me a bit of an unobvious name for what it does, and before recently I've always associated it tightly to scheduling and the reactor. No, this is a common problem. I think it would be great to address the definition a bit more comprehensively. Through example? It does seem to be a bit disconnected from the flow of the regular text.
Perhaps it would be better if it were laid out as a paragraph, and then each sentence, or clause, were examined more closely, perhaps with an accompanying code snippet to clarify it.
As I said, I don't want to explain that snippet. I just want to put it there, ignore it, and hopefully the user will understand more of it as the tutorial goes on.
Right now, it's a placeholder for that magic library that I haven't found yet.
I can see why you might want to do that. In the narrative flow between urllib and Deferreds, there is a missing step; you may need to fake that out. I'm just saying that you should be very clear and say "this is a fake example, merely for the purpose of illustration". I think that Tornado has some ugly callback-based stuff, but it would be better to leave this example fake than to try to teach users how to use that.
Will do.
Are there specific changes you find that could make it harder to read for newcomers?
I thought my previous message was a list of those :-(
Well, it was a draft, and I felt that most of my changes were already improvements over what was there. I didn't expect it to be "harder to read for newcomers".
The concept of Deferreds isn't hard at all, once you understand what they are. The subtle nuances and bits of glue code that Twisted are the things that can trip someone up, and what I'm still learning. A small amount of very specific use cases for Deferreds happen in real-world code and I'd like to show the support that Twisted has for them built-in.
My goals for this document are:
1) A list of guaranteed rules about Deferreds for reference at any time. 2) An introduction to those rules in a format that doesn't require knowledge of others. 3) Showing techniques or tricks that you can play by "exploiting" parts of those rules in the context of a contrived problem. 4) Showing the built-in support for it.
My first goal for this document would be a clear, concise explanation of what a Deferred is and why you need it.
As I said, when you understand what a Deferred is, it's not hard. The hard part is *how* to use it. The urllib, fake library, Deferred example is all I really want to go over about what a deferred is, because it's not hard. I want to: untie a lot of the code samples from the reactor, explain in terms of the two other examples that it's fundamentally the same thing, then and go on to the techniques like errbacks, chaining callbacks, and how separating the callback from the request allows things like DeferredList/gatherResults.
This should help clear up my writing style a bit. I think in terms of separating abstraction layers; I always try separate a fact or rule from logic or a technique that can follow when you can exploit that fact (feel free to ask the people about 'evolution' in #python-offtopic). I also try to think of the code being very linear when it evolves: a new rule is added, you have a problem, you can exploit that rule with a specific technique, the technique is standardized.
Example A: PROBLEM: You need to create a Deferred with a known result RULE: Callbacks will continue running after you've called "callback" or "errback" TECHNIQUE: You can create a Deferred, call 'callback' on it and return it, without any tricky business STANDARDIZED: twisted.internet.defer.success
Example B: PROBLEM: You need to get the results from multiple Deferreds without blocking or too much linearity RULE: More than one Deferred can be created and 'run' at the same time TECHNIQUE: You can add a callback to a Deferred, take the result you get and save it in a list or dictionary STANDARDIZED: DeferredList, gatherResults
These seem more like recipes to me than introductory documentation. Maybe they should be really close together, to try to drive the concept home, but it would be good to really get it clear in the reader's mind why they * fundamentally* need Deferreds, then to cover all the subtle *different* ways you might need them and how you could use them.
But you don't *fundamenally* *need *Deferreds at all. The new document isn't going to convince them that they're the right thing or wrong thing, just to show how to get along in their new home in Deferred-ville, USA. For the recipes, they're going to be reading real-world Twisted code, they better know what these recipes do. And if you can think of some subtly different ways to use them other than the generic "something is happening in the future, I better be prepared for it", I'll put it in the document and attempt to eat my hat.
*snip*
( Also, this is a real recipe, ableit simplified and it makes really easy, delicious chocolate mousse: http://articles.latimes.com/2008/feb/13/food/la-fo-watch13recafeb13 )
This looks like a fantastic example. It's comprehensible, concrete, not too long, and involves a strict metaphor for a real world situation, without mixing in any obscure technology. I would be happy if the entire Deferred tutorial were to be structured around it.
Except it really breaks that "runnable examples" thing. As I said before, recipes are a very convenient match for asynchronous code, but it's not realistic to try to map 1:1 to the other.
You could also tweak it to introduce additional concepts. For example, errbacks: "If the butter burns...".
If the butter burns.... what do I do? Turn off the heat, clean the pan, and restart some more butter? What if I run out of butter? Do I need a "goToStoreToFetchButter" function? Is that another tutorial I should write?
Once I *knew* what a Deferred was, the other pieces started snapping into place.
So it sounds like we're in agreement here: the existing document isn't clear enough about exactly what a Deferred is, it's described too formally and its uses aren't clear enough before we start diving into the technical specifics. Any modification should strive to make it super clear what it is and why you use it.
When I started with Twisted, I used Deferreds because "everything else in Twisted uses it". I didn't really *try* to understand it too much, either. Most of the time I used Twisted, I tried to hack it all up with inlineCallbacks and pretend it was urllib, and then complain in #twisted with a code snippet when it wasn't working. Use 'localhost' URLs and have the user run a 'twistd web' command line for
their server; that should be simple enough :).
For static pages, is there anything fundamentally wrong with " http://twistedmatrix.com/documents/11.1.0/static/pi.txt", if I can do it?