On Mon, 29 May 2006 15:58:04 -0600, Wilfredo Sanchez <wsanchez(a)wolfwood.twistedmatrix.com> wrote:
>Date: Mon May 29 15:58:03 2006
>New Revision: 16956
> - copied from r16955, trunk/
>Branch for 2518bis updates.
We've stopped using per-user directories in the branches hierarchy. Also, all branch names should be of the form <short descriptive name>-<ticket number>.
See http://divmod.org/trac/wiki/UltimateQualityDevelopmentSystem for documentation on the procedure that should be used for all Twisted development.
I understand that twisted.flow is no longer maintained, and is not
widely considered to represent a good way of writing twisted code.
However, I haven't been able to find any explanation of why this
approach (using generators to simulate co-operative multitasking)
seems to have been abandoned.
Is it simply the case that most people writing twisted code didn't
find it very useful? Or are there more specific arguments against
doing things that way?
I'm trying to make the case at work for using twisted for networking
things (in spite of my preference for lightweight threads plus
sensible concurrency primitives, if Python is the target platform then
twisted is probably the best way to go).
If I can show some full-threaded code next to some co-operative
multitasking code that a) has much the same sort of control flow, but
b) scales much better, and doesn't have to worry about subtle
concurrency issues, then I think it should go fairly well. If I have
to explain about how the event-driven programming model works at the
same time, it might not go *so* well...
Hi, I'm writing a distributed application using XML-RPC over SSL in
Twisted. I need mutual authentication (server and client) but the
XML-RPC Proxy in Twisted doesn't take any SSL context so I rewrote a
Proxy by myself passing my ClientContextFactory subclass to
Everything works fine but now I need to handle some possible error
conditions that could prevent the connection, e.g wrong certificates,
incompatible ciphersuite etc. In other words, I would like to catch
these "exceptions" and handle them but I don't know how to do it in the
Twisted framework (using Deferred I guess..). Any hint?
I have a program where I think that what I want to use is
deferToThread and setTimeout. However, the setTimeout method is marked
as deprecated - but I can't find a good supported alternative. Can
The situation I have is:
I need to do a large-ish number (50-100) of blocking calls in
parallel, and collect the results or any errors. In itself, this seems
like a suitable use for deferToThread (defer each call, and collect
the results in the deferred callback/errback).
The problem is that in rare cases, the blocking call can block
indefinitely. In this case, I need to make the call time out. However,
the underlying API offers no way to time out the call, so I have to do
A quick prototype seems to work OK, using setTimeout to force a
timeout on the deferred, but it generates deprecation warnings for the
What should I be using to achieve this effect?
I remember being impressed with Ada because you could write an infinite
loop without a faked up condition. The idea being that in Ada the
typical infinite loop would normally be terminated by detonation. --
I am quite new to Twisted and I'm having some trouble
getting my XML-RPC server to do exactly what I want.
We have a relatively simple goal: create an XML-RPC
server that will accept a request for a score, call a
function that computes that score, and then return the
score. The two catches are that 1. the client is
actually a .NET web server rather than a Twisted
client, and 2. we could have thousands of requests
come in relatively simultaneously (though more likely
not greater than a few hundred), likely with lots of
downtime between these periods of high activity. We
are planning to create a look-up table to deal with
some of the more common calculations, but I still need
to be able to run the other calculations
non-sequentially. I know that using a Deferred object
is the normal solution, but I'm not entirely sure if
it could work here without threads. I read somewhere
that when the function call is not from another
Twisted application, it's not possible to use the
Deferred object. Is this actually the case? Or would
it work in my situation since all of the "deferring"
would be occurring within the server itself? My
current solution involves using threads and a Deferred
object. Is this the best way to do this? Can it be
done without threads? If so, how and is that a better
solution than the one with threads?
Here is my sample code, which does a simple
calculation rather than the real one I will be using.
Both my Twisted testing client and the .NET web
server are able to talk to this server and get the
correct results back.
from twisted.web import resource, server, xmlrpc
from twisted.internet import defer, threads
from twisted.python import threadable
t = params
d = threads.deferToThread(self.calc,t)
val = d.addCallback(self.handleSuccess)
print t, d
result = [t,t*10]
print 'succeeded', t
print 'failed miserably'
if __name__ == "__main__":
from twisted.internet import reactor
root = resource.Resource()
I have run into an odd problem. I am not sure if it is my issue or Twisted: any help would be appreciated. Under at least some circumstances, twisted.web.client seems to 1) not be able to follow a 301, and 2) throw an unhandled exception, when trying to follow a 301. A specific example and resulting error is given below:
from twisted.internet import defer
from twisted.web import client
from twisted.internet import reactor
protocol = client.HTTPPageGetter
def __init__(self,client_factory = HTTPGetter):
self.factory = client_factory
f = self.factory(url)
k = reactor.connectTCP(host, port, f, timeout=10)
def downloadFailed(self, v):
r = Fetcher()
w = r.download("www.shopzilla.com",80,"/aaaa")
This results in:
Unhandled error in Deferred:
Traceback (most recent call last):
File "/usr/local/lib/python2.4/site-packages/twisted/internet/posixbase.py", line 226, in mainLoop
File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py", line 541, in runUntilCurrent
File "/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line 494, in resolveAddress
File "/usr/local/lib/python2.4/site-packages/twisted/internet/defer.py", line 182, in addCallbacks
--- <exception caught here> ---
File "/usr/local/lib/python2.4/site-packages/twisted/internet/defer.py", line 307, in _runCallbacks
self.result = callback(self.result, *args, **kw)
File "/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line 498, in _setRealAddress
File "/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line 520, in doConnect
connectResult = self.socket.connect_ex(self.realAddress)
File "<string>", line 1, in connect_ex
exceptions.TypeError: an integer is required
Which is apparently due to the fact that doConnect assumes a good address and so does not trap for TypeError.
The bad address that doConnect blows up on ('',None) for (host,port) slips in due to twisted.web.client.handleStatus_301. The example site (Shopzilla.com) posts a URL for the 301 Location that is not fully qualified. handleStatus_301, in the face of such a URL, appears to fail because it relies on getting the host/port from the location URL, but these are not present in it. Thus it passes in the ('',None) to its reactor.connectTCP attempt to follow the redirect, leading to the error above. My kludge fix to handleStatus_301 is given below, where if the host or port are missing I steal them from the transport, which should be correct since it was just used to get the page. I am running with this now, with no errors.
Is this a Twisted issue? If so, is my fix reasonable? If it is not a Twisted issue, what am I doing wrong?
l = self.headers.get('location')
if not l:
url = l
scheme, host, port, path = \
#following 4 lines added kad to fix apparent issue with 301 to a url that is not fully qualified
if self.factory.host == '':
self.factory.host = self.transport.addr
if self.factory.port == None:
self.factory.port = self.transport.addr