On Sun, Jun 22, 2003 at 05:14:24PM +0200, Matthias Urlichs wrote:
Since "suck" doesn't work for me for a variety of reasons, I decided to replace it with a solution based on twisted.protocol.nntp. After a few hours of hacking, I now have a nice program which nicely saturates my downlink bandwidth. ;-)
Cool!
To make a long story short, the attached patch implements the changes and fixes I needed to actually get there.
The "Allow the article text to be a callable or deferred" change implements the common situation where I ask server B whether it would like to be fed article X before actually pulling that article from server A, and/or where the pull is still in progress.
Hmm, this is the only part of the patch I am unsure about. The API seems a little too tuned to your use-case. I think the way to go for this would be to have a Producer passed in and make the NNTP protocol a Consumer for that (in turn acting as a Producer for its transport object). Would you be willing to make this change? (If you need an example of how this might work, check out smtp.py)
There is one somewhat- incompatible change here, in that I return the GROUP results (article count, high and low numbers) as integers, not as text. In practice they're going to be int()ized anyway, so this should not be a problem.
I think this is fine.
# twisted/protocols/nntp.py # Fixes for news gateways / 'suck'-style operation / INN as server: # - The client uses \n and does NOT esacpe start-of-line dots. # The server uses \r\n and escapes dots ONCE, not twice (ouch). # - POST temporarily blocks streaming. Make sure this is observed, # pass a Deferred out for clients to restart themselves with # - Add a command to allow MODE READER # - Allow bare reply numbers without text # - Allow the article text to be a callable or a deferred # - use CHECK/TAKETHIS if there's a message ID # - return group article numbers (GROUP reply) as numbers # - Clean up article linefeed handling # twisted/test/test_nntp.py # Make sure that dot escapes are passed cleanly. # Make sure that no empty lines are added at the end. # Use the unittest object for checking. # Use client-side line endings for the client, # assume that the server side is transparent. # Make sure that the test doesn't just peter out halfway through. # Remove the commented-out iterate() calls. # loopback() already does the work for us.
Thanks for these test fixes/cleanups, too. Jp -- In the days when Sussman was a novice Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?" asked Minsky. "I am training a randomly wired neural net to play Tic-Tac-Toe." "Why is the net wired randomly?" asked Minsky. "I do not want it to have any preconceptions of how to play." Minsky shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So the room will be empty." At that moment, Sussman was enlightened.