Mailman 3 March 2011 - Twisted

[Twisted-Python] twisted.web.error.Error & BaseException.message deprecation
by Jason J. W. Williams 11 Mar '11

11 Mar '11

Hi Guys, Been seeing this error for a long time and finally getting off my bum to see about fixing it: twisted/web/error.py:53: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 It looks like the issue is the "self.message = message" assignment in __init__: https://gist.github.com/865097 Before I go about getting rid of the warning, does anyone have a reason Error.message should stick around? -J

2 6

[Twisted-Python] Authentication & Access Control system for web services
by Allen Bierbaum 10 Mar '11

10 Mar '11

I have a REST service I have implemented using twisted.web. Based upon a new requirement I need to put role-based access control security on the service and am trying to find the most twisted way to do it. I would like to have: - Username / password login that is checked against a backend database - Roles and associated privileges associated with each user - Administration interface to edit users, roles, and privileges - "Simple" way to configure the access control requirements on the services. (ex: which services need which roles) Before I role my own code I wanted to check and see if there are any addons for this or if anyone else had attacked this problem with twisted and had some open source code I could look at. I have found a couple of projects for WSGI that I may try to pull ideas from, but I haven't yet found anything that uses the twisted resource model. (http://authkit.org/, http://docs.repoze.org/who/2.0/) Any pointers to twisted projects I could leverage? -Allen

3 5

[Twisted-Python] Agent and "Cannot assign requested address"
by Jason J. W. Williams 10 Mar '11

10 Mar '11

Hi All, I've got a situation where I'm using t.w.c.Agent to make 100,000 POST request against a server. Each time a new Agent instance is built and the request is sent using it. After about 20,000 requests, I get this error: Failure: twisted.internet.error.ConnectError: An error occurred while connecting: 99: Cannot assign requested address. Would building the Agent once and reusing the same instance avoid this? I assume I've run out of client ports. -J

5 9

Re: [Twisted-Python] [Twisted-web] Question regarding callLater() and service creation
by Andrew Bennetts 10 Mar '11

10 Mar '11

This thread belongs on the twisted-python@ list, rather than twisted-web@, as it has nothing to do with twisted.web. Eric Chamberlain wrote: > I'm creating a service using Twisted. It has nothing to do with > networking (I know that Twisted is an event-driven networking > engine... stay with me here :). It's simply polling a database every > 30 seconds. I'm using callLater() once the work has been completed. > My question is does the reactor spawn a new thread once the > callLater() timeout has been reached? If so, this would mean that the > work being done may be effected by the GIL, correct? Incorrect. callLater spawns no threads. There's nothing wrong or even particularly strange with using Twisted for projects unrelated to networking. An event loop with good facilities for spawning and interacting with subprocesses and calling functions at certain times has plenty of uses, even if you don't also need the capacity to handle network IO. That doesn't mean that Twisted is necessarily the best tool for your task, but I wouldn't rule it out automatically either. -Andrew.

2 1

[Twisted-Python] Fwd: Slow data transfer with Twisted + socat + Windows
by Michael Thompson 09 Mar '11

09 Mar '11

---------- Forwarded message ---------- From: Michael Thompson <michaelnt(a)gmail.com> Date: 9 March 2011 07:02 Subject: Re: [Twisted-Python] Slow data transfer with Twisted + socat + Windows To: Jason Heeris <jason.heeris(a)gmail.com> On 9 March 2011 01:47, Jason Heeris <jason.heeris(a)gmail.com> wrote: > On 9 March 2011 03:49, Michael Thompson <michaelnt(a)gmail.com> wrote: >> Does you program just write via TCP to this serial device, it doesn't >> do anything else that could block? > > Both the "bare bones" script and the real app writes and reads from > the device, so I wouldn't expect full speed communications. But I > can't see anything else in my script that might actually block. > >> Have you tried running this under different reactors? > > Yep — no difference under Linux, but the GTK reactor is slower than > the select reactor under Windows (see original email for times). > >> Running the windows program against the Linux socat device might help >> to reduce the number of variables. > > Aha! Didn't think of that, but interesting results... > > socat on Linux, socat_test.py on Windows VM: > - select reactor: 4s > - win32reactor: 4s > - gtk reactor: 220s > > socat on Windows VM, socat_test.py on Linux: > - select reactor: 30s > - gtk reactor: 30s > > Interesting, but not exactly clarifying... And from your first email socat on Windows, socat_test.py on Windows VM select 30s gtk reactor 220s My read of these results is that socat on windows limits the performance to 30s and in addition the gtkreactor on windows has performance problems. The gtkreactor on windows has a different implementation which seems to be missing any way for the GTK loop to notify the reactor of IO. You could try changing the timeout from 0.1 to something smaller, but that will probably hammer the CPU when idle. pyserial includes an example that makes a serial port accessible of TCP you could see if that is any faster than socat. Perhaps you can write a serial to TCP adaptor that encapsulates the serial logic so reduce the IO between this application and the GTK app to a message like start_sync, stop_sync. Michael

3 3

[Twisted-Python] txRiakIdx: Twisted Riak library implementing transparent indexing
by Jason J. W. Williams 09 Mar '11

09 Mar '11

Hi Guys, I've put together a library for my company's needs to implement secondary indexes in Riak transparently. It's built on top of Ray Cote's great txRIak library, and should drop in transparently anywhere txRiak is used. We've open sourced it in case it's useful for anyone else: https://github.com/williamsjj/txriakidx Comments/thoughts are greatly appreciated. -J

1 0

[Twisted-Python] Slow data transfer with Twisted + socat + Windows
by Jason Heeris 09 Mar '11

09 Mar '11

I'm writing an application with Twisted/GTK to manage a serial device, for use on both Windows and Linux. Unfortunately I can't use serial comms directly under Windows[1], so my workaround is to launch socat[2] in a subprocess to relay data between a local TCP port and the serial port. The socat command I use is: socat TCP4-LISTEN:31415 /dev/com4,raw,echo=0,b57600 I can then send and receive data over localhost:31415 same as I would for a serial port. The trouble is, a transaction that takes a minute under Linux (using *either* direct serial comms or the same socat subprocess approach) now takes more than 50 times as long under Windows. I've attached the script I'm using to test, mainly so it's clear what I'm doing — although it's useless unless you happen to have a serial device using a protocol you can shoehorn into that script. I've also attached the output where you can see the difference between different reactors on different platforms. (In my app I use reactor.spawnProcess(...), but I'm not doing that here.) Basically, under Linux either approach takes 4 seconds; under Windows the SelectReactor takes 30s, and the gtk2reactor takes 220s! Profiling the "reactor.run()" call on the different platforms (both for the select reactor) showed that (a) most time was spent in the select.select call, and (b) the time spent in select was 8 times more under Windows than Linux. I have no idea if this is useful information or not. I'm pretty sure it's not a socat-specific issue, because I get the same problem if I use com2tcp[3]. It could be Cygwin related, but I don't know how determine that for sure. Also, the fact that it runs slower using the gtk2reactor (vs. selectreactor) suggests to me that it could be my code, not theirs. I found an old Cygwin thread[4] that claimed setting NODELAY on the listening socket helped with a similar problem, but using > socat TCP4-LISTEN:31415,nodelay [etc] ...made no difference. So I'm a bit stuck. Does anyone know how I can narrow it down further? Has someone else noticed slow TCP behaviour under Windows? Or Cygwin utilities? Have I made some glaringly obvious mistake in my script there? Any help appreciated. :) Cheers, Jason [1] http://twistedmatrix.com/trac/ticket/4862 [2] http://www.dest-unreach.org/socat/ [3] http://com0com.sourceforge.net/ [4] http://www.mail-archive.com/cygwin@cygwin.com/msg66791.html

2 2

[Twisted-Python] Paisley 0.3 Released
by Jason J. W. Williams 08 Mar '11

08 Mar '11

Hello, We wanted to let everyone know that we've released version 0.3 of Paisley (the CouchDB client for Twisted Python). Among the new features: Enhancements * Added CouchDB authentication support (supply username and password args when instantiating) * Re-factored underlying HTTP transport to use Twisted Web Agent (will allow connection pooling in the future). * Re-factored code into formal Python package. API compatible with old package layout. Bug Fixes * Fixed attachment handling so it returns the attachment instead of the owning document (dwb) You can get it from Github or pypi: * http://github.com/smcq/paisley * http://pypi.python.org/pypi/paisley/0.3 Hopefully the new changes will be helpful to folks. Please let us know if you have any questions. -J

1 0

[Twisted-Python] Debug logging
by Juan Antonio Ibañez Santorum 06 Mar '11

06 Mar '11

Hello! I am running a smaill app using twistd. All works OK except I cannot get debugging messages into the log. I think I may to set logging level but I don't know if possible nor where it can be done. I am using the python logging module. Any help wourd be appreciated. Regards

3 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 06 Mar '11

06 Mar '11

1 0