Mailman 3 July 2005 - Twisted

[Twisted-Python] XMPP/Jabber server using Twisted
by Jeremy Redburn July 18, 2005

July 18, 2005

I am looking into coding up a simple XMPP server in Python to accept simple XML-RPC calls. I was wondering if there was any sort of "Hello World" client and server examples, or if there is active work being done on XMPP support in Twisted? Thanks much, Jeremy

2 1

[Twisted-Python] IOCP and CANCELLED asynchronous I/O operation
by Justin Johnson July 18, 2005

July 18, 2005

I am wondering if someone would be willing to assist me in debugging a problem I'm experiencing with my current attempt at an implementation of spawnProcess for the IOCP proactor. A patch with my implementation can be found at http://twistedmatrix.com/bugs/issue1008 along with comments and a log file describing my problem. The problem occurs sporadically when running twisted.test.test_process.ProcessTestCase.testProcesstwisted.test.test_process. First I create 3 pipes that will be used … [View More]

1 0

[Twisted-Python] threadedselectreactor - shutting down the foreign event loop
by Toby Dickenson July 18, 2005

July 18, 2005

Im looking at moving my PyQt app to threadedselectreactor. Previously I was keeping twisted and Qt in seperate threads. Avoiding the need to cross that thread boundary has been a great simplification, but Im having some problems relating to the shutdown of my foreign event loop. The examples such as http://svn.twistedmatrix.com/cvs/trunk/doc/core/examples/threadedselect/pyg… handle this by: a. Calling reactor.stop() when there is a GUI shutdown request, b. Using reactor.… [View More]

2 2

[Twisted-Python] Any tutorial/sample-code for khashmir or DHT?
by Eric Hsu July 17, 2005

July 17, 2005

Hi all, I'm trying to learn the DHT (distributed hash table) which could be used to build up a decentralize network. I knew khashmir (http://khashmir.sourceforge.net) which has implemented DHT with twisted. However, the project page of khashmir presents no release, no docs, no sample code, no tutorial... but, it has be included into the Bittorrent sources to implement the "trackless" feature. I'm wondering if someone could provide me or point me to some sample-code or tutorial helping me to … [View More]

1 0

[Twisted-Python] Is it possible to build a .dll for other win32 program/language to invoke?
by Eric Hsu July 17, 2005

July 17, 2005

Hi all! This question may not be suitable to ask here, but please help me if you can, thanks! I have written a server with twisted. The server contains some classes which are responsible for handling protocol related stuffs (e.g. packing/unpacking messages). We've already has an app which was written in delphi, we want to extend it to be able to communicate with this server. Here comes the question: is it possible to build the protocol related classes from the server to be some kind of .dll,… [View More]

5 5

[Twisted-Python] Re: [Twisted-commits] r14171 - Simplify and fix horribly broken deferToThread tests
by Itamar Shtull-Trauring July 16, 2005

July 16, 2005

On Sat, 2005-07-16 at 09:04 -0600, Jp Calderone wrote: > Author: exarkun > Date: Sat Jul 16 09:04:11 2005 > New Revision: 14171 > > Modified: > trunk/twisted/test/test_threads.py > Log: > Simplify and fix horribly broken deferToThread tests This won't catch bugs where the Deferred is never callback()ed.

2 1

[Twisted-Python] Reminder: Australian Twisted Sprint #2: 19th - 21st August 2005, Sydney
by Mary Gardiner July 14, 2005

July 14, 2005

Hey folks, Still no final word on exactly where in Sydney this sprint will be held, but it *will* be held and will be on these dates, so I'm posting again to give people time to book flights before they get expensive. I've made a grant application to Linux Australia to fund some of the sprint, but this is not guarenteed to go through. Watch this thread for details: http://lists.linux.org.au/archives/linux-aus/2005-July/msg00216.html We would very much appreciate people letting us know via … [View More]

2 2

[Twisted-Python] Twisted client to access proxy web server
by Richard July 13, 2005

July 13, 2005

I've used twisted.web.client.getPage() and downloadPage() to get pages from a web server, but was wondering how to use Twisted to get pages from a proxy web server. Can anyone point me at some example Twisted code for a client to access a proxy-server?

3 2

[Twisted-Python] conch example of sftp
by Gustavo Rahal July 13, 2005

July 13, 2005

Hi Does anyone has or know where I can find sftp code examples? AFAIK conch is able of doing files transfers over a ssh connection. I know it may appear that I'm laisy but I don't have much experience with twisted nor conch and the documentation is not that good. Thanks Gustavo

2 1

[Twisted-Python] O'Reilly book on Twisted coming in October
by Abe Fettig July 12, 2005

July 12, 2005

Hi all, I'm happy to announce that O'Reilly is going to publish my book on Twisted. It's scheduled for release in October. The working title is Twisted Network Programming Essentials. I've been working on this book since the end of last year, but I've been waiting to announce it until I knew that it was definitely going to be published in the reasonably near future. At this point it's in the editing stage, and O'Reilly has put it on their production schedule for an October release. So spread … [View More]

5 6

Re: [Twisted-Python] twisted.names and multicast DNS
by Brian Granger July 12, 2005

July 12, 2005

Brian Granger, Ph.D. Assistant Professor of Physics Santa Clara University bgranger(a)scu.edu Phone: 408-551-1891 Fax: 408-554-6965 >>> jarrod.roberson(a)gmail.com 07/09/05 9:38 PM >>> On 7/9/05, Tim Allen <screwtape(a)froup.com> wrote you forgot option number 4, which is what we ended up doing. Put a python binding in over the Apple code and use it integrated into Twisted. Plus you get to use it outside Twisted as well :-) I like this option as Apple's mDNS code is … [View More]

5 5

[Twisted-Python] Weekly Bug Summary
by James Y Knight July 12, 2005

July 12, 2005

Bug summary ______________________ Since 2005-07-06 04:28:36.250000: Bugs opened: 3 Bugs closed: 0 Total open bugs: 401 (+3) Urgent: 9 (+0) Bug: 171 (+1) Feature: 144 (+1) Wish: 77 (+1) New / Reopened Bugs ______________________ ==== Bug ==== [MAIL] IMAP Client does not seem to handle unsolicited server responses (4 days ago) http://twistedmatrix.com/bugs/issue1105 opened by cablehead ==== Feature ==== [PATCH] make example … [View More]

1 0

RE: [Twisted-Python] exceptions in adbapi
by Henning.Ramm＠mediapro-gmbh.de July 12, 2005

July 12, 2005

>errbacks catch exceptions attached to deferreds. There are no >exceptions that will skip over errbacks. If your errbacks are not >being triggered, you're not putting them in the correct location in >the deferred callback chain. My mistake was to set log.error as errback in some locations, and that lead to the quoted messages without proper origin. After some hours of analyzing and refactoring I got rid of those stuff. Sorry for bothering you. Best regards, Henning Hraban … [View More]

1 0

RE: [Twisted-Python] exceptions in adbapi
by Henning.Ramm＠mediapro-gmbh.de July 12, 2005

July 12, 2005

>> Any hints? > >These exceptions are happening in your Deferred callbacks. You need to >attach an error handler (the asynchronous counterpart to an except: >clause) to the deferred you get back as a result of your adbapi call. Sure, that's what I do everywhere where I see a Deferred. But they don't catch these exceptions anyway. Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

2 1

RE: [Twisted-Python] need help getting started with Twisted
by Henning.Ramm＠mediapro-gmbh.de July 12, 2005

July 12, 2005

>Can anyone recommend other resources or tutorials for newbies? The >documentation on the Twisted website was confusing, in part because >it's divided into many different projects and I don't know how those >projects relate or if some of them have been superseded by newer ones. You should work yourself through http://twistedmatrix.com/projects/core/documentation/howto/index.html And then perhaps tell us in detail what you want to achive so that the wizards are able to point you … [View More]

1 0

[Twisted-Python] exceptions in adbapi
by Henning.Ramm＠mediapro-gmbh.de July 11, 2005

July 11, 2005

Hi there! I've a twisted app as database proxy server that handles all database queries for my other apps (via a pb connection). Now I get thousands of exceptions that I can't trace back, two kind: 2005-07-11 15:57:36,707 defer.328 [ERROR] [Failure instance: Traceback: _mysql_exceptions.Warning: Datensätze gefunden: 1 Geändert: 0 Warnungen: 1 D:\Python\lib\threading.py:422:run D:\Python\lib\site-packages\twisted\python\threadpool.py:145:_worker D:\Python\lib\site-packages\twisted\python\… [View More]context.py:52:callWithContext D:\Python\lib\site-packages\twisted\python\context.py:31:callWithContext --- <exception caught here> --- D:\Python\lib\site-packages\twisted\internet\threads.py:25:_putResultInDeferred D:\Python\lib\site-packages\twisted\enterprise\adbapi.py:302:_runInteraction D:\Python\lib\site-packages\twisted\enterprise\adbapi.py:311:_runQuery D:\Python\lib\site-packages\MySQLdb\cursors.py:95:execute D:\Python\lib\site-packages\MySQLdb\cursors.py:114:_execute D:\Python\lib\site-packages\MySQLdb\connections.py:33:defaulterrorhandler ] 2005-07-11 15:57:36,707 defer.328 [ERROR] [Failure instance: Traceback: _mysql_exceptions.IntegrityError: (1062, "Doppelter Eintrag 'TA_V_23_130705' f\xfcr Schl\xfcssel 1.") D:\Python\lib\threading.py:422:run D:\Python\lib\site-packages\twisted\python\threadpool.py:145:_worker D:\Python\lib\site-packages\twisted\python\context.py:52:callWithContext D:\Python\lib\site-packages\twisted\python\context.py:31:callWithContext --- <exception caught here> --- D:\Python\lib\site-packages\twisted\internet\threads.py:25:_putResultInDeferred D:\Python\lib\site-packages\twisted\enterprise\adbapi.py:302:_runInteraction D:\Python\lib\site-packages\twisted\enterprise\adbapi.py:311:_runQuery D:\Python\lib\site-packages\MySQLdb\cursors.py:95:execute D:\Python\lib\site-packages\MySQLdb\cursors.py:114:_execute D:\Python\lib\site-packages\MySQLdb\connections.py:33:defaulterrorhandler ] The first one is absolutely useless - MySQL tells me only that it could process the query ("datasets found: 1, changed 0, warning: 1"); I'd like to block that completely. The second one means that I tried an insert with an existing key ("Double entry '...' for key 1"); also no problem, if I could handle it and do an update query instead. But as you can see, I can't trace the messages back to one of my methods (and I have already every function call in a try..except clause). Any hints? Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept. [View Less]

2 1

[Twisted-Python] Understanding the IOCP reactor and adding spawnProcess
by Justin Johnson July 11, 2005

July 11, 2005

I am attempting to add spawnProcess to iocpreactor. In order to begin this task I've had to do a lot of reading on Windows network programming, specifically the various Windows I/O methods, to attempt to understand what win32eventreactor and iocpreactor are doing, and also just increase my understanding of how reactors work in general. To understand the various Winsock 2 methods that both of these reactors rely upon, I read chapters 1-5 of Network Programming for Microsoft Windows[1]. … [View More]Before actually attempting to add spawnProcess, I would like to present how I think iocpreactor works and how I think I should add spawnProcess, and hopefully be corrected or confirmed in my understanding. If I'm too vague there's a good chance it's because I don't understand it very well. Please feel free to point out things that you might think are obvious but aren't sure I understand. How iocpreactor works --------------------------------- 1. Create an IO Completion Port. 2. Create a socket and associate it with the IOCP. This is the socket we will call AcceptEx (a non-blocking accept) on. The association with the IOCP is made via CreateIoCompletionPort. 3. Setup any scheduled tasks on the reactor. 4. Call AcceptEx (which doesn't block) on the socket. AcceptEx takes an overlapped structure as a parameter. Before making the call, we set two attributes of the struct: the callback and callback_args which will be called when an accept event completes on the socket. The Winsock 2 methods don't actually call the callback. The Winsock 2 methods handle copying data related to the network event that occurred on the socket into the overlapped structure and making that overlapped structure available to GetQueuedCompletionStatus. So when we handle events on sockets via GetQueuedCompletionStatus from within doIteration, we have access to the data related to the event as well as the callback and callback_args we call to handle that event. The callbacks are setup in the xxxOp classes in ops.py and always result in some transport method getting called (such as readDone, connectionDone, etc). 5. From within doIteration, call GetQueuedCompletionStatus (which does block) with a timeout of the time until the next scheduled task needs to be run. If any event occurs on the sockets currently associated with the IOCP before that time expires, GetQueuedCompletionStatus will return (stop blocking). Now we have access to the overlapped structure containing data associated with the event which was copied into the overlapped structure's buffer, such as data received from WSARecv calls, as well as the callback and callback_args. From within doIteration we call the callbacks passing in the data related to the event. Depending on the events we are handling, we may create new sockets (e.g. end point sockets in TCP connections) and associate them with the IOCP as well. All Winsock 2 API calls made are non-blocking accept for GetQueuedCompletionStatus. 6. Step 5 continues until the reactor stops. How to add spawnProcess --------------------------------------- 1. Create the processes via Windows APIs and associate their stdout/err with with the IOCP via CreateIoCompletionPort calls. 2. Close stdin. 3. Notify the ProcessProtocol via protocol.makeConnection (not sure why, looking at win32eventreactor) 4. Receive data from stdout/err via the completion port by calling GetQueuedCompletionStatus from within doIteration. Is this really possible? ProcessProtocol's methods won't get called appropriately by letting the existing callbacks in ops.py make calls to the transport (e.g. connectionDone, readDone)? Thanks for your help. Justin [1] http://www.amazon.com/exec/obidos/ASIN/0735615799 [View Less]

3 7

[Twisted-Python] Receiving large packets with TCP
by Stefan Näwe July 11, 2005

July 11, 2005

Hi. I'm using Twisted (2.0.1) for a TCP-based receiver. That receiver receives different telegrams sent by another process. Those telegrams are sent on different ports (i.e. one telegram type is sent on one port). The sizes of the telegram vary from ~16 bytes to ~8 kB. Everything works perfectly. I've really never been *that* productive. BUT: I'm having trouble receiving the whole contents of large telegrams. For example one telegram is exactly 7452 bytes long but I only get 2920 bytes. My … [View More]

4 5

[Twisted-Python] twisted.names and multicast DNS
by Tim Allen July 11, 2005

July 11, 2005

So after two years of leaving it alone, I decided to have another look at the multicast DNS module I was writing for Twisted, and lo and behold I discover a serious design flaw in my response cache. Here's the basic situation I'm dealing with: Suppose I have two hosts on my network, called ford and arthur. I'm sitting at arthur, and I'm looking for SSH servers advertised via DNS-based Service Discovery and Multicast DNS. At the present moment, ford is switched off. I turn on ford, and … [View More]when the SSH server starts up, ford broadcasts an announcement DNS datagram that says, "In the zone '_ssh._tcp.local', there is a PTR record with the data 'Ford._ssh._tcp.local'. This information expires in 7200 seconds." (it mentions the class too, of course, but the class doesn't affect the discussion below so we'll ignore it from here on). arthur receives the announcement, records the transmission time and the TTL, and caches the PTR record. Thereafter, any program on arthur that requests PTR records in '_ssh._tcp.local' will be given the PTR record about 'Ford._ssh._tcp.local'. Now I turn off Ford, and when the SSH server shuts down, ford broadcasts a goodbye datagram that says, "In the zone '_ssh._tcp.local', there is a PTR record with the data 'Ford._ssh._tcp.local'. This information expires in 0 seconds." - that is to say, exactly the same as the previous announcement but with a TTL of 0. The cache on arthur needs to compare this new record to its cache, see that it matches the definition of an existing record, and update the TTL of the existing record so that it will expire immediately. I'm trying to implement the record cache on arthur at the moment. It needs to be queried by (name, type, class) tuples, so I use such a tuple as the key of a dictionary. Since there can be multiple records available for a particular (name, type, class), the value associated with this key is a list of resource records. Because the t.names.dns.DNSDatagramProtocol gives me instances of the various t.names.dns.Record_* classes, I figured that those instances would be a good representation of the resource records I need to store. In psuedo-python, then, the ideal data structure would look like this: cache = { ('_ssh._tcp.local', PTR, IN): [ (<Record_PTR name=Ford._ssh._tcp.local>, 7200), ] } ...then, when I get a new record from the network, I can look up its (name, type, class) tuple and get the list of records. Then I compare the new record to each of the records in the list: if it matches a record I update the TTL, otherwise I add the record to the list. The problem is this: the Record_PTR class (just like all the other Record_* classes) in t.names.dns stores the TTL as an instance variable! Therefore, when I have an existing record and an updated record, they never match, and I wind up with two identical records in my data structure that differ only by TTL. As far as I can tell, I have three options from here: 1. Write my own versions of all the Record_* classes that don't maintain a TTL value, and translate backwards and forwards whenever I need to talk to Twisted's DNS layer. I'd need to update this code whenever a new DNS record type was added. 2. Write a comparison function for comparing resource records that compares instance variables that aren't named 'ttl'. This would probably need to be updated less frequently, but is horribly hacky. 3. Complain on the twisted-list and hope that somebody has a better suggestion, or changes the code to be more mDNS-friendly. As you can see, I'm trying Option 3 at the moment, but I'm open to suggestions. :) [View Less]

3 3

[Twisted-Python] Questions regarding serviceCollection and Interfaces.
by kgi July 11, 2005

July 11, 2005

Hi all. I'm in the process of moving a skeleton asyncore application to Twisted, and need some help. I'm hoping that someone with much Twisted Enlightenment can point me gently in the direction of understanding. (Amusingly enough, another poster recently summarised my exact experiences: start with asyncore, find its limitations, do a bit of shoehorning and end up trying with Twisted). Because this is new code I want to do The Right Thing vis-a-vis the latest Twisted coding techniques. … [View More]Everything was peachy until I reached a certain step in the "Evolution of Finger" Tutorial. Until a certain point I had completely grokked the approach that looks like this: class Foo ( Protocol ): .... factory = ServerFactory() factory.protocol = Foo application = service.Application ( "fooserver" ) internet.TCPServer ( port, factory ).setServiceParent ( application ) However, then I reached: http://twistedmatrix.com/projects/core/documentation/howto/tutorial/compone… and the approach morphed into: application = service.Application ( "fooserver" ) f = FooService() serviceCollection = service.IServiceCollection ( application ) internet.TCPServer ( port, IFooFactory ( f ) ).setServiceParent ( serviceCollection ) Here are my questions: 1. I simply don't understand the third line of the new approach! :-) In twisted.application.service.py, I see: class IServiceCollection(components.Interface): .... So, "IServiceCollection" is an Interface; my (possibly outdated) understanding is that an interface is simply a specification that you can check against when something claims to implement it, hence all the "implements(Yadda)" calls everywhere (presumably, an exception is raised when something tells porkies about what it implements). But in this case, it looks like we're actually instantiating an interface class. I can't grok what this "means", especially in light of the fact that there is "MultiService" below it that implements IServiceCollection. 2. Similarly, in the last line, "IFooFactory(f)" is taking a Service (which isn't a factory) and passing it as a construction argument to IFooFactory (which also isn't a factory). The result is used as an argument to internet.TCPServer(), which expects a factory! 3. What do I gain by using a serviceCollection over an application as the serviceParent of a TCPServer? Come to think of it, what did I gain in the first place with setServiceParent()? 4. Lastly, there are lots of examples floating around that describe how each Protocol example can reference its own Factory in order for connections to be able to talk to other connections (for example, in the chat server example). However, what approach should I take when I have a single server with multiple Protocols? For example, consider a simple cache: I have a server which accepts TCP connections from lots of clients, and handles queries from these clients. The server actually gets information from an upstream server, but caches the results that it has already seen. In this case in the server I need a Protocol/TCPServer server to handle the multiple clients, and a single Protocol/connectTCP client for the upstream connection. However, it's not clear to me how I would transfer the data from the lineReceived() method of the server Protocol to the sendLine() method of the client Protocol, since each Protocol can see its own Factory, but not the Factory of the other Protocol. Many thanks for your time, and any assistance afforded! Ricky [View Less]

4 5

[Twisted-Python] XML RPC
by Md Moin July 9, 2005

July 9, 2005

I would like to setup a twisted XML RPC server with authentication functionality based on username and password.Is there any simple example available for XML RPC server/client that uses username/password authentication? I am a newbie thus if any of you guys can post a simple example of xml rpc server and client that uses authentication that will help me a lot. Thanx ____________________________________________________ Sell on Yahoo! Auctions no fees. Bid on great items. http://auctions.yahoo.com/

1 0

Re: [Twisted-Python] Could not create and run a Windows Service with Twisted 2.0.1 & Python 2.4.1
by Cory Dodt July 9, 2005

July 9, 2005

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fixed in trunk. I recommend you delete ntsvc.cfg and your dist directory, then rerun setup.py twistedservice with svn trunk code. Jp was correct. C > It's not being imported by user code, it's being imported by t.service.app. > Py2exe doesn't know that, so it loses it. > > Luc, probably ntsvc should deal with this automatically. Let me think > about it > before you put it in the HOWTO. > > C > > > Jp … [View More]

1 0

[Twisted-Python] Race condition in callInThread()
by Jp Calderone July 8, 2005

July 8, 2005

James kindly added a failing test for the nasty behavior of twisted.python.threadpool.ThreadPool._startSomeWorkers(). It seems this will be difficult to fix completely without making API-incompatible changes all the way up to reactor.callInThread, though. Fixing it for *most* cases is straightforward enough: Index: twisted/python/threadpool.py =================================================================== --- twisted/python/threadpool.py (revision 14112) +++ twisted/python/… [View More]

1 0

[Twisted-Python] Could not create and run a Windows Service with Twisted 2.0.1 & Python 2.4.1
by Luc Stepniewski July 8, 2005

July 8, 2005

I've written a Twisted Windows Service which works perfectly with Twisted 1.3 and Python 2.3. I've used the sandboxed ntsvc from moonfallen's svn. I upgraded everything to Twisted 2.0.1 and Python 2.4.1 (even tried ActiveState's), and followed the information provided in moonfallen's README-service (that is, patch pywin32's boot_service.py). It compiles correctly to a .exe, without any supplementary error and I can install it correctly as a service (using 'xxxxxx.exe -install'). But When … [View More]

3 4

[Twisted-Python] File Transfer/BitTorrent
by Thomas HERVE July 8, 2005

July 8, 2005

Hi all, I've seen a lot of threads talking around my problem but haven't found what I was looking for. My need : I want to have a file transfer application, with the following functionnalities (for now) : - Transfer from 1 to N - Integrity control - Failover/Resume - Handle big files (up to 5Gb) - Strong authentication/encryption (with client certificates) - Automatic actions when upload/download finished FTP is not really a possibility (mainly because of the nasty network configuration it … [View More]

3 3