Mailman 3 March 2005 - Twisted

[Twisted-Python] Design question: xmlrpc server - deferreds - adbapi
by Remy C Cool 12 Mar '05

12 Mar '05

Hello Twisters, As a Twisted newbie, I'm not sure if, what i've coded is the Twisted way to go. In this test, I've created a simple XMLRPC server def main(): # initialize factory factory = server.Site(XMLRPC()) factory.resource.base = factory # initialize server reactor.listenTCP(8000, factory) try: try: reactor.run() except KeyboardInterrupt: print 'CTRL-C received, shutting down server.' except Exception, err: print 'Problem with running server: %s' % err finally: reactor.stop() # The XMLRPC class: class XMLRPC(xmlrpc.XMLRPC): def xmlrpc_test(self, text): d = defer.maybeDeferred(test, text) return d # The test function def test(text): # do something with the text return dbPool.pg.runInteraction(getTestData, text) # The getTestData function def getTestData(cursor, text): # do queries using cursor return result dbPool is a class which sets up dbpools for different databases according to a config file. This setup seems to work, but I'm not sure about the way I'm using the deferreds. Comments welcome :) Grtz, Remi Cool

1 0

[Twisted-Python] Twisted 2.0 prerelease (close!!)
by Christopher Armstrong 12 Mar '05

12 Mar '05

So I did a Twisted 2.0 prerelease. http://twistedmatrix.com/users/radix/2.0/ There is now a TwistedSumo tarball, which contains Twisted and all of its subprojects as well as Zope Interface. Thanks to James Knight for providing the setup.py for this. Despite the crappy web presence, this is a real 2.0 prerelease. I will try to get this published to the web site by this weekend. Please download anything relevant to you and give it a try, and be sure to post your experience to this thread even if it went smoothly. Differences between this prerelease and the pre-prerelease: * It no longer includes *.pyc or *.bak files * It includes loreified documentation * All version numbers should be correct (according to distutils, package.__version__, and README files) * It includes the LICENSE file * Probably other stuff Have fun, -- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// | -- http://twistedmatrix.com |o O| | Founding Member, Hobart Hacking Society w----v----w-+ -- http://hackingsociety.org/chapters/hash

9 15

[Twisted-Python] twisted installation mac
by Jason Van Anden 12 Mar '05

12 Mar '05

Hi, I am attempting to install twisted on os x.3. I did not find the package in the package manager as the installation instructions suggest. I found this thread here: http://twistedmatrix.com/pipermail/twisted-python/2005-February/009613.html - but it does not state what installation to use. Can someone please point me in the right direction? Thank You, J

3 4

[Twisted-Python] NEWS file
by James Y Knight 12 Mar '05

12 Mar '05

Devs, please read the NEWS file, and if there's anything missing, add it. http://svn.twistedmatrix.com/cvs/trunk/twisted/topfiles/NEWS? view=auto&rev=13133&root=Twisted James

2 1

[Twisted-Python] ChangeLog
by James Y Knight 11 Mar '05

11 Mar '05

So, it looks like we've generally included two change listings in our releases: ChangeLog and NEWS. It's pretty obvious what NEWS is supposed to be: a human edited list of the most important changes in the release. There is some question, though, about what a ChangeLog should be. Right now, it is useless, as nobody updates it. Who wants to write two commit messages rather than one, afterall? If we actually want a change log separate from an "important changes" NEWS file, I think it'd be fine to simply autogenerate it from our SVN log messages, because if you want a more concise explanation of what changed, that's what NEWS is for. The advantage of doing this is that people who don't follow SVN get to see all the things we changed, the disadvantage being that it's 6500 lines long and contains some irrelevant parts. E.g. merging trial, reverting merge, merging trial, reverting merge, repeat like 200 more times. Also, things lilke "Use ThreadedResolver instead of BlockingResolver by default" in one commit, and "Tra la la la la, like extra flowers" in the next when you realize you screwed up the first and need to do a 4-character fix. :) On the other hand, perhaps all we really want is a single human-edited changes file, describing in a bit more detail than the 1.3 NEWS file what's changed, but more concise than a full changelog. What do people think? James

4 3

[Twisted-Python] multicast multi-listen not working or just not working on OSX?
by jarrod roberson 11 Mar '05

11 Mar '05

I am having a hard time getting multicast to work in Twisted. Code snippets/examples I have found are either too old or just incorrect. what is the correct way to write a protocol handler that will multilisten on a multicast socket? I am trying to add mDNS responder functionality to my application and twisted complains that port 5353 is already in use. I know I can run multiple copies of mDNSResponder from Apple, is this just a problem with twisted on OSX or is it worse than that. regardless is there a fix to get twisted to work, this application will eventually be deployed in an linux or solaris environment so if it is an OSX thing I can just find another box to test it on.

3 7

[Twisted-Python] [OT] Looking for Twisted/Python developers
by Thomas Vander Stichele 11 Mar '05

11 Mar '05

Hi, Fluendo (http://www.fluendo.com) is looking for someone interested in developing on some of the products we're working on. We are looking for someone well-versed or willing to get up to speed with Twisted and Python. The initial focus would be on working on our on-demand streaming platform integrated with Flumotion, our live streaming server. Willingness to relocate to Spain is a plus but not absolutely necessary. Questions, queries, sample projects, CV: send to me. Thanks, Thomas Dave/Dina : future TV today ! - http://www.davedina.org/ <-*- thomas (dot) apestaart (dot) org -*-> I should dress you up in pearl Finest silk to touch your skin Don't know how to write a love song But don't leave <-*- thomas (at) apestaart (dot) org -*-> URGent, best radio on the net - 24/7 ! - http://urgent.fm/

1 0

RE: [Twisted-Python] App design / Twisted Logging?
by Henning.Ramm＠mediapro-gmbh.de 11 Mar '05

11 Mar '05

Paul, thank you very much for taking the time to answer my confusion. >> -> Do I need a thread for every copy/move operation (large >files via network)? > >Nope. That's the beauty of twisted code. I've written this >sort of handling >multiple packet trains many times. Write a protocol. Hm, sounds like using FTP and not shutil.move via system's SMB? Or is there already something better in twisted? If I'd write my own protocol to wrap shutil.move, the operation would be blocking because I don't work on file block level and the protocol doesn't make sense. I'm not feeling like writing file transfer code on file block level. Or do you think it would be worth the hassle? >> Another part reads some XML files (using minidom) and writes ... >I ran into long processing problems before. There's a simple >solution. Pick >something convenient such as simply chunking everything up >into processing one line at a time. So not using DOM but per-line XML handling. Not that convenient, because I use only small bits of all the XML data that are spread all over the files, but I guess it would become better twisted code. >your constructs using deferreds. And the Flow module allows >you to go even >further than the hard coded version I'm suggesting here. Try >reading through >the Flow module documentation. Just don't get hung up about >using it. For my >purposes, it was usually better just to have the basic concept >in mind than to actually use Flow. That's perhaps a general problem with twisted: There are great solutions for everything, but you need to know them in detail to know which fits your problem. Or you must know how you should reshape your problem to fit in some twisted solution... I know twisted "does it all"(TM), but was is "it"? ;-) >Yep. Frequently, this is where I'd run a deferred() just after >sending the request to the known-to-be-slow process. >Read that section carefully too. Then >your code gets called back immediately when the process >completes. You will >need to understand deferred's to fully utilize twisted's >threading interface, >and then you'll realize that you probably don't need threads. I'm just trying to write a simple directory watcher (I need this at every corner of my app), Patrick Lauber wrote an answer to my initial question on that, but that wasn't really what I needed. It works so far that it calls a deferred callback if it gets a notify on a new/changed file, but only once; next time I get an "AlreadyCalledError" - looks like I don't yet understand deferreds. At the moment it's inherited from pb.Root, because I'll need it to run remotely sometimes, but perhaps it would be better to use a service or something else -- it should run "all the time" if not stopped and call a callback for every file. I attached the file, perhaps someone can point out my biggest mistakes? >first but once you get used to it, deferred's seem just well...obvious. I hope to get into that higher state of mind soon. ;-) >As to logging itself...it might be easier to just write your >own. Usually I >haven't been overly thrilled with the mechanical-looking output of >off-the-shelf loggers and logging is little more than just >printing to a file >anyways, sometimes with some extra (chop it up and reclaim >space) features added. I enjoyed being able to switch the logging output, e.g. from file to database or email per config file without the need to go into the code. I don't feel like re-inventing the wheel, but as Glyph pointed out, the config syntax of standard logging is just ugly and messy; the config syntax of log4[j|perl|net] ist much more logical. Perhaps someone should write a log4twisted module... Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

Re: [Twisted-Python] POSTing http requests
by Lenny G Arbage 11 Mar '05

11 Mar '05

I just realized that this is a bit more involved than I at first expected. It would be nice to be able to send the upload request without holding all of postdata in memory, and getPage(), even when I get past the below problem (which can be done by calling something like encode_multipart_formdata() as defined at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306), doesn't seem to allow for that. Fabian Seisen's urllib2_file at http://fabien.seisen.org/python/urllib2_multipart.html, does work without requiring the upload to sit in memory all at once, but it doesn't appear to be asynchronous and it doesn't appear to work with Python 2.4 right now, either. Is there some way I can make this work with HTTPClientFactory? If not, what should I be looking at? In the case of large file uploads, wouldn't it be advisable to send the headers/seperators first, then do a chunk by chunk read of the file and send these chunks one at a time (but in quick succession). To do that, it seems like a write() method is necessary -- is that in the IConsumer interface? Thanks for any guidance you can give me -- as you can tell I'm new to twisted. Thanks Lenny G. --- Lenny G Arbage <alengarbage(a)yahoo.com> wrote: > How do I correctly set up the payload for a POST > file > upload? I see that I can add a "method='POST'" and > "postdata=something" to getPage(), but it's that > 'something' that I'm stuck on. I'm wondering if > there > isn't a convenience method for putting the right > stuff > in the HTTP header (Content-Type: multipart...") and > in the body (-----seperator, etc). > > I haven't been able to find an answer in the mailing > list for the above (and google doesn't seem to know, > either), but I'm sure there's a really simple way to > do it. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

[Twisted-Python] POSTing http requests
by Lenny G Arbage 11 Mar '05

11 Mar '05

How do I correctly set up the payload for a POST file upload? I see that I can add a "method='POST'" and "postdata=something" to getPage(), but it's that 'something' that I'm stuck on. I'm wondering if there isn't a convenience method for putting the right stuff in the HTTP header (Content-Type: multipart...") and in the body (-----seperator, etc). I haven't been able to find an answer in the mailing list for the above (and google doesn't seem to know, either), but I'm sure there's a really simple way to do it. Thanks in advance Lenny G. __________________________________ Do you Yahoo!? Yahoo! Mail - 250MB free storage. Do more. Manage less. http://info.mail.yahoo.com/mail_250

1 0

Twisted March 2005