Mailman 3 March 2012 - Twisted

[Twisted-Python] cfreactor and global state
by Itamar Turner-Trauring March 18, 2012

March 18, 2012

twisted.internet.test.reactormixins says: CFReactor uses APIs which manipulate global state, so it's not safe to run its own reactor-builder tests under itself. We also don't have a buildslave running under cfreactor. Which means it's currently completely untested. I filed a ticket to fix that - https://bugs.launchpad.net/twisted-buildbot-configuration/+bug/954680 That still leaves the problem with global state though. Since we want to move towards having tests create new reactors, that's problematic. What global state does it manipulate, and what can we do about it?

2 1

[Twisted-Python] "twistd web --port=9000" no longer works
by Itamar Turner-Trauring March 18, 2012

March 18, 2012

Apparently this was removed as part of deprecating the old strports syntax, but: 1) "twistd web --port=9000" never gave a deprecation warnings. itamar@blake:~$ twistd --version twistd (the Twisted daemon) 11.0.0 Copyright (c) 2001-2011 Twisted Matrix Laboratories. See LICENSE for details. itamar@blake:~$ twistd -n web --port=9000 --path=/ 2012-03-17 20:50:21-0400 [-] Log opened. 2012-03-17 20:50:21-0400 [-] twistd 11.0.0 (/usr/bin/python 2.7.2) starting up. 2012-03-17 20:50:21-0400 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2012-03-17 20:50:21-0400 [-] twisted.web.server.Site starting on 9000 2012-03-17 20:50:21-0400 [-] Starting factory <twisted.web.server.Site instance at 0x180bd40> 2) The current --help just says " -p, --port= strports description of the port to start the server on." It'll be completely unclear to users what that means. I am tempted to revert the ticket that removed this functionality - can anyone tell me why we shouldn't? In particular, before the removal should be re-merged again, the original syntax should be restored, along with decent --help. We should always support just giving the port number and defaulting to TCP. As it is, we've gone from a straightforward command line option that also had some sophisticated features to something that will break in a mysterious, hard to debug manner.

2 1

[Twisted-Python] a weird problem about IOCP reactor and perspective broker
by gelin yan March 17, 2012

March 17, 2012

Hi all I attach two files for this problem. When i used iocp reactor with pb, I noticed once the remote method returned a deferred object, the client app would raise many exceptions..kinda weird. It is ok to return the value directly (no defer) The most interested part is when I used select reactor, everything was fine..I am not sure whether it is a bug on IOCP. OS: windows 7 64 bit py: 2.6.6 32bit twisted version: 12.0.0 Regards gelin yan

2 3

Re: [Twisted-Python] libusb-1 file descriptors
by Vincent Pelletier March 16, 2012

March 16, 2012

Hi. First, sorry for non-threaded reply. I found the thread on gmane while googling a bit. I'll try to explain the design and intended use of python-libusb1 polling integration. I lack twisted knowledge to tell how it could be integrated, though. As for the "poller" interface, it's python's select.poll interface. Basically, libusb needs to be called into explicitly[1] whenever events needing some handling occur. It signals those events through a set of poll- able file descriptors[2]. Alternatively (for OSes which cannot poll arbitrary file descriptors), LibUSBContext.getNextTimeout can be used to know for how long app can sleep (or do its own business) before next libusb event handling time... But that only works when libusb calls (transfer submission, synchronous or not) are all done in a single thread, otherwise they might last too long. When several threads can simultaneously wait for events to occur (ex: a poller waiting on an asynchronous transfer with long timeout, then in a second thread a synchronous call expected to return quickly) a more complex scheme needs to be used. See libusb doc for the whole story. In short, I decided to just write a stand-alone polling thread to support multi-thread access, as I couldn't figure a pythonic way of supporting several concurrent libusb-oriented pollers: C API relies on a mutable boolean-ish set by libusb code, and read by application code. As I try to never expose ctype instances to application code, and thought it should be enough to provide module user with a "start once, then forget" thread for handling this (much easier in python than in C), I went that way. Besides the functions working with that boolean, I exposed all needed methods on LibUSBContext class, so different implementations can be done without hacking/working around, and hopefully without using libusb1 module directly (except for symbolic constants). I am opened to suggestions on how to expose the few missing functions in a pythonic way. [1] LibUSBContext.handleEvents & co. [2] LibUSBContext.getPollFDList, LibUSBContext.setPollFDNotifiers Regards, -- Vincent Pelletier

1 0

[Twisted-Python] Twisted Log Efficiency is poor?
by Peng Xiao -X (penxiao - Digital China at Cisco) March 15, 2012

March 15, 2012

Hi experts, I have a question about twisted log. I used “from twisted.python import log” and “log.msg(msg_str)” to store log for my twisted program (about 40MB log message) But it’s efficiency is poor, it will take more time to finish my program when the log is open, if I close the log, the program will be more faster than before. What is the problem? What should I do? Best Regards, Xiao Peng(肖鹏) Cisco Systems GSP-Beijing (China) Office: +86 10 8507 7181 Mobile: +86 18600560077(preferred)

3 2

[Twisted-Python] Anyone still using Python 2.5?
by Itamar Turner-Trauring March 14, 2012

March 14, 2012

In particular, are you using recent versions of Twisted with Python 2.5? Dropping support for 2.5 would make it easier for us to begin the work of supporting Python 3, so I'm wondering how widespread the use of Python 2.5 is. (If you're using the version of Twisted included with your distro that doesn't really count, since lack of support in newer releases is less relevant.) If you are using Python 2.5, what is the constraint keeping you there? There's Ubuntu LTS 8.04 - any other relevant platforms? On Windows it looks like 10-12% of PyPI downloads are for Python 2.5: http://pypi.python.org/pypi/Twisted -Itamar

3 3

[Twisted-Python] accurate periodic call
by Zoran Bošnjak March 14, 2012

March 14, 2012

Hello all, I was astonished to find out that looping call period depends on the system time by default. The periodic tick can even stall for a long time, if the system time jumps backwards during program execution. It turned out that this is in fact a python problem (not providing a monotonic time, at least not for posix). I urgently need accurate periodic call in my program and I've found the solution below that seems to be working. I kindly ask you for your comments: - Is this monotonic_time implementation OK from python perspective? - Is monkey patch to the reactor OK or is there any other solution more appropriate in this case (I do not want to patch each looping call, but once in the application)? - Does this patch have any negative influence to the rest of the reactor? - How would you implement a periodic function call in twisted application (as accurate as possible)? - Any chance to see something implemented inside twisted and/or python, so that applications don't need this kind of tricks? Thanks a lot for your comments. Zoran #! /usr/bin/env python from twisted.internet import task from twisted.internet import reactor import os import time import ctypes # python MONOTONIC time, borrowed here # http://stackoverflow.com/questions/1205722/how-do-i-get-monotonic-time-dura… if os.name == 'posix': CLOCK_MONOTONIC = 1 # see <linux/time.h> class timespec(ctypes.Structure): _fields_ = [ ('tv_sec', ctypes.c_long), ('tv_nsec', ctypes.c_long) ] librt = ctypes.CDLL('librt.so.1', use_errno=True) clock_gettime = librt.clock_gettime clock_gettime.argtypes = [ctypes.c_int, ctypes.POINTER(timespec)] def monotonic_time(): t = timespec() if clock_gettime(CLOCK_MONOTONIC, ctypes.pointer(t)) != 0: errno_ = ctypes.get_errno() raise OSError(errno_, os.strerror(errno_)) return t.tv_sec + t.tv_nsec * 1e-9 # monkey patch the reactor reactor.seconds = monotonic_time # TODO: check for other platforms!! else: monotonic_time = time.time def tick(): """This function is suppose to execute once a second, regardless of the system time.""" print 'tick', monotonic_time() loop = task.LoopingCall(tick) loop.start(1.0) reactor.run()

8 13

[Twisted-Python] Basic Resource Tree question
by Lusana Ali March 14, 2012

March 14, 2012

Hi, Im trying to setup just a very simple site using a Resource Tree (some code below). The problem I'm having is getting the root / to render. so when i go to domain.com/ I get "No Such Resource No such child resource" The child nodes {terms,profile} are rendering fine. So looking at the resource tree and the main.py which holds the Main() Resource... how can I fix this issue. I have already tried setting isLeaf = True in Main() but that breaks terms and profile Cheers server.py ======================================= # resource tree root = Main() root.putChild('terms', Terms()) root.putChild('profile', Profile()) site = server.Site(root) reactor.listenTCP(6346, site) reactor.run() main.py ======================================= class IndexPage(Resource): def __init__(self): print "created new index page" Resource.__init__(self) def render_GET(self, request): print "rendering get" return "<html>INDEX PAGE</html>" class Main(Resource): def get_child(self, name, request): print "trying to get main" return IndexPage()

3 3

[Twisted-Python] PyCon and Sprint Photos
by Duncan McGreggor March 14, 2012

March 14, 2012

Hey folks, I put up the tiny amount of photos I took this year here, many of which are of Twisted hackers: http://www.flickr.com/photos/oubiwann/sets/72157629579346369/ Enjoy! /me awaits more and better photos from others... d

1 0

[Twisted-Python] HTTPS proxy with twisted?
by Steve Chapel March 13, 2012

March 13, 2012

Hi, I am doing a research project for a security class that involves writing an HTTP and HTTPS proxy. The intent of this proxy is to ensure that communications that should be secure are secure, without bombarding the user with false warnings about potential man-in-the-middle attacks, for example, when a website legitimately uses a self-signed certificate. I will need to write an HTTPS proxy, which will examine the certificates sent from the web server and determine whether the certificate is valid or invalid. If the proxy determines if the certificate is valid, I will need to resign the document. I suppose this will require that the proxy be a certificate authority and will generate certificates for websites, which the proxy will then use to sign the documents. Will this be something that twisted can do easily? If so, where can I find documentation for how to do this? Thanks, Steve

3 4