Mailman 3 July 2005 - Twisted

[Twisted-Python] Re: O'Reilly book on Twisted coming in October
by Ed Suominen 04 Jul '05

04 Jul '05

Abe, this is great to hear! Congratulations and very best of success to you and the book. I for one will be looking to buy it ASAP. P.S. - Apparently your blogging software is based on PHP rather than Twisted. :-/ From your link, I just get an error "Error establishing a database connection. This either means that the username and password information in your wp-config.php file is incorrect or we can't contact the database server at localhost." Best regards, Ed Suominen Abe Fettig wrote: > Hi all, > > I'm happy to announce that O'Reilly is going to publish my book on > Twisted. It's scheduled for release in October. The working title is > Twisted Network Programming Essentials. > > I've been working on this book since the end of last year, but I've > been > waiting to announce it until I knew that it was definitely going to be > published in the reasonably near future. At this point it's in the > editing stage, and O'Reilly has put it on their production schedule > for > an October release. So spread the word - there's a Twisted book > coming! > > For a description of what the book covers (and what it doesn't), see > this post on my weblog: > > http://fettig.net/weblog/2005/06/30/my-book-on-twisted/ > > Thanks to all the Twisted developers for creating such an amazing > framework. The more I learn about Twisted (and I learned a lot while > writing this book), the more I'm impressed. I hope this book helps > many > more people to get started with Twisted, and that it's the first of > many > more books to come. > > Abe

1 0

[Twisted-Python] Scheduling of modules
by Moof 04 Jul '05

04 Jul '05

Hiya, I've more or less had twisted sold to me as the be-all and end-all in terms of avoiding having to deal with threads, amongst other things, so I thought I'd ask if twisted is the right thing to use for my problem. I have a number of modules which use John J Lee's mechanize module to do some web scraping to different web sites. Right now mechanize uses standard urllib calls to do its thing, though it may be possible after much refactoring to get it use something more asyncronous. So for the short term rapid development thing, I'm more or less stuck with threads if I want to get any vaguely concurrent running out of these modules. Specifically, I'm going to need to poll a database every minute or so (or write some form of SQL Server 2000 trigger which will call me when things change) and if there are any changes in the database, potentially fire off thirty of these web scraping modules at once. The modules are self-contained, and don't communicate with anything other than the thing that calls them by returning values, so I'm vaguely certain they're more or less thread-safe, inasmuch as I ever can be. My beef is that I can't have more than one scraping module modifying the same site concurrently. This introduces race conditions on the site, rather than in my code. Each module touches only one site, so I need to basically have a module-level lock either in the module or in the thread scheduler to ensure that I'm not running the same module more than once. This makes me think of some sort of queue structure. I either need to have one queue that just works through its requests ignoring any that are currently running, or one queue per module with some sort of central dispatcher that will place a request in the appropriate queue. In real terms, these modules may take up to three minutes to complete the web scraping they are required to do, though most take 20 seconds or so. I'd rather not just have them called one after the other in a blocking manner, as I'd sort of like to have a five or six minute response time whenever a request is placed in the database to fire off a bunch of updates, rather than the close to 20 minute response time I'm currently getting when I fire a complete unittest suite off. These requests may come in several times a day, most commonly hours apart, but I need to be able to react if I get two or three different requests within a five minute period, which would mean firing off the next request to the module as soon as it has completed the current request. Is this something Twisted can help me with? If so, what are my options within Twisted, and what should I be reading up on how to use? I have a vague idea that Twisted has a thread pool, but I'm not sure if it has an event queue that would be suitable for this sort of control, or how I'd go about modifying whatever's there to be useful for this sort of thing. If not, any pointers to patterns that might help me code such a thing up? I'm running on Windows. I need no GUI integration as such, though it will need to run as an NT service. Any input other than though the database could potentially be triggered off by a client programme going through something like perspective broker, or the Windows NT service controller telling me to start up or shut down. Thanks for your help, Moof -- Giles Antonio Radford, alias Moof "Too old to be a chicken and too young to be a dirty old man" Serving up my ego over at <http://metamoof.net/>

1 0

[Twisted-Python] How to implement bandwidth control for a UDP proxy server?
by Eric Hsu 03 Jul '05

03 Jul '05

Hello! Dose anyone have any experience in implementing bandwidth control for UDP packets transferring with twisted? Please enlighten me if you could, thank in advance! : ) - Eric

1 0

[Twisted-Python] how to tap a DelayedCall?
by Andrea Arcangeli 03 Jul '05

03 Jul '05

Hello, I'm still using my setup where I use mktap for the production system so I tweak it with mktap -u/-g/Options() parameters (twisted.python.usage.Options) and the tac -noy model for development with default Option values, but I'm now having a problem with a delayed call in the tap mode. Somewhere inside the makeService hook I execute a reactor.callLater, and that gives the following error: WARNING: unserializing ephemeral twisted.internet.base.DelayedCall (and of course it hangs too since that new callLater is now fundamental ;) Now I think I can trivially move the callLater into the startFactory() hook, and then it should work even with the tap file, but I wondered if there was a more transparent way to make callLater work with tap files like it does with the tac ones just fine. I'm considering going away from the tap model, but first I'd like to see the equivalent of mktap -u/-g and the other Option() parameters available through tac files too (which isn't possible yet AFIK, twistd doesn't take those params, and I don't want to reinvent locally yet another getopt/Options mechanism, the Options configurable via mktap were all I needed, except I don't need serialization just to pass options, I only need the mktap option passing). Hints welcome. Thanks and keep up the great work!

2 3

[Twisted-Python] Minions of Mirth - Independent Persistent World RPG using Twisted
by Josh Ritter 03 Jul '05

03 Jul '05

Hello all... I thought this might be of interest to a few people... I know there are some game developers on this list... and we have a pretty good example of Twisted being used in a game :) We just opened up the alpha on our RPG that uses Twisted for the bulk of it's "RPG Networking"... Twisted has been absolutely great to work with... it's used in our Master Server, World Server, Zone Servers, and Client... We also have an autopatching system it will be used in, though for now that uses *gulp* urllib :| The alpha can be downloaded on both the Windows and OSX platforms.... some further information is below.... Oh, and here's an embedded web server in the world server running Nevow, hooray for Nevow! http://www.prairiegames.com/phpBB2/viewtopic.php?t=116 Vibes and THANKS!!!, -Josh Ritter Prairie Games http://www.prairiegames.com ----------------------------------------------------------- Minions of Mirth(tm) - "Truly Open Alpha" Minions of Mirth(tm) is a massively single player, multiplayer, 3d MUD, Persistent World RPG, by Prairie Games. We are currently in alpha testing on OSX and Windows and are extending an invitation to be a part of the game's creation. You can DOWNLOAD the alpha test simply by registering on our forums: http://www.prairiegames.com/phpBB2/profile.php?mode=register Once you've registered, the download information can be obtained in this thread: http://www.prairiegames.com/phpBB2/viewtopic.php?t=93 The Minions of Mirth "Truly Open Alpha" is a work in progress that features: + Windows and OSX support! + Easy access, only a forum registration is necessary to get in on the fun! http://www.prairiegames.com/phpBB2/profile.php?mode=register + Massively Single Player Gameplay(tm)!!! + Cooperative and Player vs Player Multiplayer + No monthly service fees for Multiplayer! + Autopatcher delivers immediate content and feature upgrades!!! + Host your own persistent worlds with your own original content! + Immortal access to your own world where you can control time, weather, spawns, items, etc + 10 playable races with distinct look, feel, and SIZE!!! Huge Drakken and tiny little Halflings!!! + 9 beautiful zones with huge playable area and vistas! Dark dungeons and sprawling wilderness! + 2 hours of original music! + Masterfully animated 3d monster models with stunningly realistic textures + Spine-chilling sound effects! + Day/Night cycles with gameplay tied to time of day + Immersive weather effects based on zone climate + Multiclass your characters all the way to level 100 in up to 3 classes! + Material based clothing system + Over 50 3d weapon models + 100's of in game items with high resolution graphics and magical effects! + Political Factions + Quest Journal + Online Help + Vendors sell items and restock at regular intervals + Trade items and currency with other players! + 3d Automap + Versatile Spell System (invisibility, wind walking, Teleportation, Healing, Nukes, DoTs, etc) + Summon pets, equip them, and order them to destroy your foes! + Powerful Skills (Feign Death, Bloodletting, Rage, Volitation, Archery, Grecious Wound, Precise Shot, etc) + Form your own party of up to 6 characters + Form multiplayer alliances with up to 36 characters + Add, remove, or even create new characters at Inns, without ever leaving the game! + Powerful modding system including RPG Database (with live recompiles) and Realtime Zone Editing + Advanced users can SSH into their world and edit the game while it's running from an interactive Python shell. + Global, World, Zone, Area, Alliance, and Private chat channels + Global Chat system is hooked up to IRC, so you can chat with people in and out of the game, even spanning worlds! Thanks!, The Entire Team at Prairie Games

2 1

[Twisted-Python] jabber sasl and TLS
by Nathaniel Haggard 02 Jul '05

02 Jul '05

What is the status of the sasl TLS jabber authenticator? There is this code from about a month ago: http://bfs.itlab.musc.edu/pubcvs/viewcvs.cgi/tofu/saslclient.py#rev1.6, but it does not include TLS. Nate

3 4

[Twisted-Python] Event queue in the Python std lib
by Matt Goodall 02 Jul '05

02 Jul '05

Hi, I know some of you guys have already thinking about this but ... A couple of weeks ago we had a bit of a chat (flame? ;-)) in #twisted about the idea of moving parts of Twisted, specifically the event loop, into the std lib. So, at Europython I had a quick chat with GvR about it too. GvR would really, really like to get a simple, flexible event loop into the std lib and thinks that Twisted is likely the best starting point since Twisted probably integrates with more event driven toolkits than anything else. Obviously, having the event loop part in the std lib has a number of benefits but it should also put pressure on all event driven toolkits to conform to the common API. Everything would then work together and the world would be a happier, shinier place ;-). GvR suggested that someone should start a PEP to talk around, which seems like a sensible thing to do, although I don't know if anyone with enough knowledge of the reactor and working with other event loops has time right now. Cheers, Matt -- __ / \__ Matt Goodall, Pollenation Internet Ltd \__/ \ w: http://www.pollenation.net __/ \__/ e: matt(a)pollenation.net / \__/ \ t: +44 (0)113 2252500 \__/ \__/ / \ Any views expressed are my own and do not necessarily \__/ reflect the views of my employer.

9 14

RE: [Twisted-Python] Event queue in the Python std lib
by Henning.Ramm＠mediapro-gmbh.de 01 Jul '05

01 Jul '05

>I would argue that Python needs less in the stdlib, not more; specific >external software packages will probably grow into de-facto standard >packages, but there's no reason for them to be in stdlib. I agree, there are some rather, ehm, suboptimal modules in the stdlib (e.g. ConfigParser, logging), and if one needs a good solution for something, (s)he would always install some addition (or code it her|himself). So perhaps a better way to handle "standard modules" would be a standard package manager (like ActivePerl's ppm). Otherwise it would be nice to declare Twisted "Python standard" this way. (But perhaps it would frighten off Python newbies who are only looking for a simple network library...) Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

[Twisted-Python] Passing arguments to application
by Stephan Popp 30 Jun '05

30 Jun '05

Hi, maybe I haven't searched enough, but I don't find anything that halps me to solve my problem. I've got a serverapplication. The problem is in the following lines: application = service.Application("TwistedServerApp") internet.TCPServer(8800, pb.ServerFactory...) I don't want to start this with a predetermined port. More like: twistd -y TwistedServerApp.tac 8800 Is this somehow possible? sys.argv seems impossible to use. -Stephan

3 3