Mailman 3 November 2011 - Twisted

[Twisted-Python] 64 bit windows machine
by Phil Mayers 18 Nov '11

18 Nov '11

I see from the Trac page that you guys are (or were) hunting a 64-bit windows machine. I also see there are no 64-bit Twisted packages on pypi. Are the two related? Do you still want a windows 64-bit machine? When you ask for a Visual Studio license, is not VC Express sufficient? This is possibly something we can provide for you. Cheers, Phil

4 4

[Twisted-Python] Twisted Project Jobs Volunteer
by Adi Roiban 17 Nov '11

17 Nov '11

Hi, I have been using Twisted for about 6 months and looking for ways in which I could help the project. I just read the announcement and I would like to volunteer for one of the Twisted jobs. I am familiar with bzr and git and for the beginning I would like to start with maintaining the version control mirrors (bzr and git) and if this will not consume all my free time, I would also like to take care of buildbot master and slaves (or some other job that you consider is more important). Beside the job description, I was thinking that creating repositories mirrors on Github/Gitorious could be useful. The Launchpad BZR mirror seems to be functional. Hoping that I can be useful, please let know if my application is accepted. Cheers, -- Adi Roiban

20 57

[Twisted-Python] gnupg protocol implementation, does one exist?
by Sam Roberts 16 Nov '11

16 Nov '11

There is a bit of an example in http://twistedmatrix.com/documents/current/core/howto/process.html but I'm sure somebody out there must have fleshed this out into a more complete implementation. Can anybody point me to such a thing, before we write out own? We are currently using GnuPGInterface.py, but its non-twisted child process handling is causing flakyness, especially around the waitpid() call. Thanks, Sam

1 0

[Twisted-Python] reactor.callLater question
by Tobias Oberstein 15 Nov '11

15 Nov '11

2 short questions .. please consider: a) reactor.callLater(0, fun1) reactor.callLater(0, fun2) Is that allowed .. delay == 0? It seems to work .. but I want to be sure. b) reactor.callLater(T, fun1) reactor.callLater(T, fun2) Is the call _order_ first fun1, then fun2 _guaranteed_? For every T, and in particular T == 0?

4 3

[Twisted-Python] data dispatch on massive connection counts
by Tobias Oberstein 15 Nov '11

15 Nov '11

I am testing a WebSocket based pubsub system .. I have 2 questions .. any hints welcome! Environment: + FreeBSD 8.2 i386 + Python 2.7.2 32-bit + Twisted Trunk + new kqueue() reactor The FreeBSD is tuned for massive connection numbers. I can connect 50k WS clients. Fine. 1) Massive dispatching Essentially, what I'm currently doing is: recvs = set([100k instances of protocol.Protocol]) data = "..." for recv in recvs: recv.transport.write(data) Now, writing mass-data to _one_ transport should be done using producer/consumer. But in my case, the data itself is tiny (<100 octets) and the same for all clients, but the number of transports to dispatch that data to is massive. The problem is: while above loop is running, other stuff is being delayed. What would be the right approach to solve that? One idea is to split the loop into 1k chunks and use reactor.callLater to have the sending called again until all recvs are served. Reentry of the reactor via reactor.callLater should make other stuff run in-between, right? Can I use callLater(0, ..) .. that is no delay at all? And is this approach recommended anyway? 2) Too many files. As said, the FreeBSD is tuned for massive connections and I can connect 50k clients. However, the Twisted application not only contains the WebSockets stuff, but also a Twisted Web based web server. Somewhere above 30k connections, I'm beginning to see: twisted/web/server.py, line 132 in process ... twisted/python/filepath.py, line 643 in open <type 'exceptions.IOError'>: [Errno 24] Too many open files However: [autobahn@autobahnhub ~/AutobahnHub/service]$ sysctl kern.maxfiles kern.maxfiles: 204800 [autobahn@autobahnhub ~/AutobahnHub/service]$ sysctl kern.maxfilesperproc kern.maxfilesperproc: 200000 [autobahn@autobahnhub ~/AutobahnHub/service]$ sysctl kern.ipc.maxsockets kern.ipc.maxsockets: 204800 [autobahn@autobahnhub ~/AutobahnHub/service]$ ulimit unlimited It should go well beyond 50k. Doing an lsof on the app PID, I can see the 50k connected TCPs and <100 open files. Why is it denying opening more files? Is there another limit specifically for files, and/or something tunable in Python/Twisted?

5 10

Re: [Twisted-Python] data dispatch on massive connection counts
by Itamar Turner-Trauring 14 Nov '11

14 Nov '11

> Well, this is all absolutely sad. Now we (nearly) have new kqueue, > it does fly, but I can't break above 32k anyway .. Epoll on Linux?

2 1

[Twisted-Python] Using Trial with Eclipse and PyDev
by Tom Sheffler 14 Nov '11

14 Nov '11

I've seen a few answers regarding this topic around the net, but nothing recent. I've enjoyed using the Twisted code-base in Eclipse, and have also liked running unittests in the PyDev framework. However, I haven't been successful integrating Trial tests with the PyDev runner. I did get Trial to run, but the output isn't really understood and parsed by PyDev the way I did it. If anyone has any pointers, I'd appreciate suggestions. - I'd like the PASSED/FAILED/ERROR/SKIPPED states to be understood by PyDev. - I'd also like the _trial_temp/test.log file to be displayed in an Eclipse window Overall, I'm very happy with Trial as it is. Using it in Eclipse almost seems greedy. ;-} Thanks, Tom

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 13 Nov '11

13 Nov '11

1 0

[Twisted-Python] Creating a PKCS#11 Enabled SSL Proxy
by Andrew Thornton 10 Nov '11

10 Nov '11

I am in need of an ssl web proxy that can read a certificate off of a PKCS#11 device and then do client authentication using that certificate. I thought that twisted would be a great way to do this. I was wondering if anyone on this list has put anything like that together before. I have seen the tutorials and walkthroughs the outline the steps to create an http proxy using twisted, but not much about https and nothing about using a wrapper like pkcs11 or PyKCS11. I appreciate any direction that anyone can give.

2 1

[Twisted-Python] Second pre-release of Twisted 11.1.0
by Thomas Hervé 09 Nov '11

09 Nov '11

From the sunny land of Orlando, I'm pleased to announce the second pre-release of Twisted 11.1. You can find the tarballs here: http://people.canonical.com/~therve/Twisted/11.1.0pre2/ Among the 17 bugs fixed since the last pre-release, a fix in the new TLS code in connection lost management, and an error fixed with IOCP under 64bits Windows. Please give it all the testing you can! -- Thomas

3 2