Mailman 3 September 2008 - Twisted

Re: [Twisted-Python] distributing a commercial project based on twisted
by Reza Lotun Sept. 20, 2008

Sept. 20, 2008

Hi, We've successfully used bbfreeze (http://pypi.python.org/pypi/bbfreeze/0.95.4) to build binaries with twisted on linux. To make a deb from that binary is pretty easy, read up on the 'dpkg-deb' command. You can also convert that .deb to an .rpm using a tool called 'alien'. Cheers, Reza

2 1

[Twisted-Python] Looping
by David Wilson Sept. 20, 2008

Sept. 20, 2008

Hi All, I'm trying to pick up Twisted and I'm having trouble with looping. I expected that a Deferred returned from a looping call would be available after each call. But it's only available after the loop is stopped. Am I looking for a different class? Thanks, Dave p.s. Ultimately I'd like to watch a file and post any new lines up to an HTTP server. If there is a direct way to get there, please let me know. Thanks! from twisted.internet import task, defer from twisted.internet import reactor import sys import time def check_file_p(fp): time.sleep(2) #print fp sys.stdout.write('starting check...\n') fp.seek(fp.tell()) list = [] for line in fp.readlines(): #sys.stdout.write(line) list.append(line) str_list = str(list) d = defer.Deferred() #print str_list d.callback(str_list) print d return d def print_list(string): print 'calling string list...' sys.stdout.write('string: ' + string) fn = r"C:\var\twisted.txt" f = open(fn, 'r') #l = task.LoopingCall(runEverySecond) """ didn't work l = task.LoopingCall(check_file_p, f) d = l.start(5.0, False) # call every x seconds """ d = task.deferLater(reactor, 3.5, check_file_p, f) print d d.addCallback(print_list) print d print 'starting reactor' #l.stop() #will stop the looping calls reactor.run()

2 1

[Twisted-Python] Calling deferToThread from a thread
by Reza Lotun Sept. 18, 2008

Sept. 18, 2008

Hi All, I have a data store api that mandates the return of deferreds for each call. I have a number of implementations for the api - an in-memory one for testing, and a number of other network + disk based apis. My issue is this - I'm using py.test for unit testing (I'm aware of trial, but for compatibility reasons with the rest of the project I haven't looked into using it). To test deferred-based code I have set up a separate thread which the reactor runs in (the setup and teardown code is shown below for anyone who is interested). def net(): reactor.run(installSignalHandlers=0) def setup_module(mod): global net_thread net_thread = threading.Thread(target=net) net_thread.start() time.sleep(1) def teardown_module(module): reactor.callFromThread(reactor.stop) This seems to work generally fine. The problem (or at least I *think* it's the problem) is that in one of my implementations uses deferToThread internally to create a deferred which it then returns to the caller. Is it thread safe to call a deferToThread in a separate thread? Normally I'd do a reactor.callFromThread, but I'm not sure how to get a deferred by this construct. I see two options: 1. In my unit test I write a function f which makes the api call and gets the deferred, and sets it to a local variable within the test instance. Then I schedule f to run in the reactor thread via reactor.callFromThread 2. I look into trial which has support for this kind of stuff?? Thanks for any help. Cheers, Reza

3 2

[Twisted-Python] distributing a commercial project based on twisted
by Vasil Vangelovski Sept. 18, 2008

Sept. 18, 2008

Hi Can anyone tell me which is the best way to distribute a commercial project based on Twisted for Linux. It would be great if I can make .rpm and .deb packages which will include twisted and some compiled modules of the project. But I'm not sure how to do this. Should i turn to distutils or tap2rpm?

1 0

[Twisted-Python] Using the Oscar protocol to send SMS text msgs
by David Drouin Sept. 16, 2008

Sept. 16, 2008

I'm wondering if anyone happens to have an example of how to send an SMS message through AIM using twisted.words.protocols.oscar. Sending instant messages based on the code in oscardemo.py was simple enough, but when I tried to send a message to a phone number through an oscar.BOSConnection.sendMessage method such as: self.sendMessage("+19995551212", [(msg,)], wantAck=0, autoResponse=False) an SMS text message does not appear to be sent to the specified phone number, however, the number I tried to send the message to did end up appearing under "Recent Buddies" in AIM after trying this. If I then attempt to send an SMS manually through version 6.9.1.2 of the AIM client it works as expected. I'm guessing that perhaps a connection or ICBM parameter needs to be set to enable SMS messaging for the client connection made through oscar. I looked at the code for the oscar protocol in Pidgin to see if that would shed any light on what I could be doing incorrectly and didn't see anything special being done for SMS text messages there aside from converting them to plain text before sending the message along in the same way as IM's are sent except that the user name is a phone number in the +<area code><number> format I've been trying with. thanks, Dave

1 0

Re: [Twisted-Python] twisted.web.client POST data
by Andrew Francis Sept. 16, 2008

Sept. 16, 2008

Hi Vasil: >I'm trying to send a post request like so >getPage('http://localhost/test/test/', method='POST' postdata='test=asdf') >But the server script doesnt get any post arguments. >I was looking at the code for web.client and cant see where im doing >wrong here. >Any ideas what might be the problem? I'm using an svn checkout of Twisted. I will assume you did things like check your web server's log to see if a connection was made and if there was an error..... Since you are using the POST method, I would expect you would have to set the headers argument (a dictionary). I notice the header argument is missing. I would expect entries such as Content-Type and Content-Length. Cheers, Andrew

2 1

[Twisted-Python] Use twisted.python.logfile.LogFile wirte log，Memory can not be released, does anybody familiar with or encounter such a situation?
by echo Sept. 15, 2008

Sept. 15, 2008

Use twisted.python.logfile.LogFile wirte log，Memory can not be released, does anybody familiar with or encounter such a situation? Environment: FREEBSD 6.3 RELEASE twisted svn trunk today Description of the problem Use "TOP" command observe the used memory, memory first use the BUF, if buf>111M, then don't use buf and use WIRED, Wired to around 132 M， and then use Inact If during the service manually delete the generated file. Memory will be released. twisted.python.logfile.LogFile default log file will be closed at 1 M size, and re-open a file, how the file is closed， but the memory still not released？ sorry for my poor english.. thank you **test code： #!/usr/local/bin/python import datetime import os from twisted.python import logfile class LogFile2(logfile.LogFile): ''' ''' def rotate(self): """ Rotate the file and create a new one. If it's not possible to open new logfile, this will fail silently, and continue logging to old logfile. """ if not (os.access(self.directory, os.W_OK) and os.access(self.path, os.W_OK)): return logs = self.listLogs() logs.reverse() for i in logs: if self.maxRotatedFiles is not None and i >= self.maxRotatedFiles: os.remove("%s.%d" % (self.path, i)) else: os.rename("%s.%d" % (self.path, i), "%s.%d" % (self.path, i + 1)) self._file.close() print self._file, 'closed' os.rename(self.path, "%s.1" % self.path) self._openFile() i=10000000 log = LogFile2('test_logfile.log', '/home/echo/tmp/test/') while i: msg = '%s: test_test_test_test\r\n' % str(i) log.write(msg) i=i-1 log.close() **result。 <closed file '/home/echo/tmp/test/test_logfile.log', mode 'r+' at 0x827ccc8> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827c458> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827ccc8> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827c458> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827ccc8> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827c458> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827ccc8> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827c458> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827ccc8> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827c458> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827ccc8> closeed <closed file '/home/echo/tmp/test/test_logfile.log', mode 'w+' at 0x827c458> closeed .......

2 2

[Twisted-Python] intermittent problem: not accepting new connections
by Alec Matusis Sept. 15, 2008

Sept. 15, 2008

I have had a twisted epoll server that was heavily used, such that it saturated CPU (100% shown by "top", about 5000 connections, intense message relaying). I am using twisted 2.5.0 that I patched for epoll bug. It was run on python 2.4.4 , 2.6.11 kernel on a single core xeon 3.0 GHz CPU. This server has been on for many months, and it has been rock-stable. A couple of days ago I migrated that server to a newer machine: same patched twisted 2.5.0, same python 2.4.4, newer 2.6.24 kernel and a quad core xeon L5420 CPU. CPU usage dropped from 100% to 30%, as expected, with the same rate of client connections. However the server now has the following intermittent problem: about twice a day, it stops accepting new connections for a short period of 5-10 minutes. telnet times out, I get this: root@serv2:/proc/net/netfilter# telnet localhost 5229 Trying 127.0.0.1... Existing connections are not cut, they server receives/delivers messages to/from them just fine. These short periods of not accepting connections do not correlate with increased CPU load or with the overall number of connections to the server. I have had a problem with the same symptoms before, when a server process run out of its quota of file descriptors. However, there were clear messages in the twisted log at that time, and upping the ulimits solved the problem. This time, there are no errors in ANY logs (twisted log. /var/log/messages, etc) I am out of ideas on what this could be, because my setup is exactly the same as I have been using in the last year, except for a faster CPU and a newer kernel? I suspect that there are some new uncaught accept() exceptions in internet/tcp.py in the part where it's looking for EMFILE, ENOBUFS, ENFILE, ENOMEM, ECONNABORTED errors.

6 15

[Twisted-Python] twisted.web.client POST data
by Vasil Vangelovski Sept. 15, 2008

Sept. 15, 2008

I'm trying to send a post request like so getPage('http://localhost/test/test/', method='POST' postdata='test=asdf') But the server script doesnt get any post arguments. I was looking at the code for web.client and cant see where im doing wrong here. Any ideas what might be the problem? I'm using an svn checkout of Twisted.

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Sept. 13, 2008

Sept. 13, 2008

1 0