Mailman 3 July 2007 - Twisted

[Twisted-Python] Re: [Twisted-commits] r20678 - I hate having software with users
by Jonathan Lange 08 Jul '07

08 Jul '07

On 7/4/07, Paul Swartz <z3p(a)wolfwood.twistedmatrix.com> wrote: > Author: z3p > Date: Tue Jul 3 11:06:39 2007 > New Revision: 20678 > > Removed: > branches/session-2710-2/twisted/conch/test/test_userauth.py > > Log: > I hate having software with users > Why is this being removed? jml

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 07 Jul '07

07 Jul '07

Bug summary ______________________ Summary for 2007-07-01 through 2007-07-08 Bugs opened: 2 Bugs closed: 3 Total open bugs: 984 (-1) |== Type Changes |== Priority Changes |== Component Changes |Defect: -1 |Highest: -1 |Conch: -1 |Task: +0 |Normal: +0 |Core: +0 |Foolscap: -1 |Mail: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2728] svn.twistedmatrix.com not resolving (opened by jwilger) (CLOSED, worksforme) defect core http://twistedmatrix.com/trac/ticket/2728 ===== Normal ===== [#2729] remove Deferred.setTimeout usage from twisted.mail (opened by exarkun) task mail http://twistedmatrix.com/trac/ticket/2729 Closed Bugs ______________________ ===== Highest ===== [#2728] svn.twistedmatrix.com not resolving (opened by jwilger, closed by exarkun, worksforme) defect core http://twistedmatrix.com/trac/ticket/2728 [#2707] twisted.conch.ssh.connection has poor tests (opened by z3p, closed by z3p, fixed) defect conch http://twistedmatrix.com/trac/ticket/2707 ===== Normal ===== [#2647] foolscap needs to move out of the sandbox (opened by warner, closed by warner, fixed) task foolscap http://twistedmatrix.com/trac/ticket/2647

1 0

[Twisted-Python] New to twisted, need a bit of high-level design help.
by Brian L. Troutwine 05 Jul '07

05 Jul '07

Hello all, I'm new to using twisted and am attempting to implement a client and server for use a distributed computing environment with a centralized task server. Clients query the task server for a work list, apply a bunch of transforms to this work list and then return the transformed work list back to the server. The server, for its part, has to send out these work lists, keep track of which client is processing what (for timeout handling) and collate the transformed work lists so that it can writing the data out to disk in a sorted manner. I've spent the better part of a week banging out prototypes, reading the docs and asking questions on IRC (sorry, dear IRC folks, for all the timeouts), but the things I've built feel clunky. I have primarily focused my efforts on using t.s.pb and t.p.amp as I control both ends of the wire. t.p.amp is probably too limited, though, as the values in packet hashes must be <= 65535 bytes, too small for my purposes unless I break my loads into peices, which is not an obvious or easy thing to do given the structure of my computed data. t.s.pb's documentation seems to concern itself with examples involiving client requests, server responses and clients working on the response with no communication back to the server and, though it'd be nicer to work with probably, I can't seem to figure out how to cleanly implement communication back to the server. My prototype server is relatively clean, supporting a remote_checkin which clients call with a valid, unique ID to receive data and a remote_checkout, which clients call with their computed data to register it with the server. The clients, however, are a mess, mostly because the client transforms are not simple and might fail in a multitude of ways. (I really rather need to keep a reference around to the server so I can arbitrarily push data toward it). As my attempts to use twisted for this purpose have been frustrated, due to my inexperience, I'm sure, would it be possible for someone to explain how I could use twisted in the construction of a distributed system with a central data collection and collation server? Any comments are greatly appreciated, from critiques on the bits of twisted I've attempted to use to suggestions for new areas to explore to code samples. (I must apologize that I cannot give any of my prototype code as I do not own the copyright on it.) I've worked myself into rather a rut and need help climbing out. Thanks, Brian

2 3

[Twisted-Python] More general process.Process class
by Markus Schiltknecht 04 Jul '07

04 Jul '07

Hi, I'm currently fiddling with spawnProcess() and such, trying to stream it's output to somewhere else. In a first attempt, I've tried to cache the data in the ProcessProtocol's outReceived() call. Then, I've realized, that twisted already has quite nice abstractions of Producers and Consumers. I would like to use them, as they seem to fit nicely. And in my case, it would even make much sense to *not* read all the data from the process, but leave it there, waiting on the file descriptor, until I'm ready to process the data. Unfortunately, AFAICT the producers attached to the process - namely the ProcessReader() instances created by process.Process - are hard wired to the ProcessProtocol's dataReceived() method. While I would like to use these producers directly, instead of providing such a ProcessProtocol. I'm thinking about ripping out the protocol handling from the process.Process class. Instead let process.Process return a list of producers and consumers in self.pipes, as requested. Then let another wrapper class bridge between those and the ProcessProtocol. Comments? Regards Markus

2 3

[Twisted-Python] profiling twisted
by Markus Schiltknecht 04 Jul '07

04 Jul '07

Hi, first of all, thank you very much for this great peace of software, I'm using twisted for many different projects and really enjoy it's clean and simple design. Now, I'm wondering if there's a profiling tool, which can quickly tell me what events are blocking my server. I have even deferred the file I/O to separate threads, as I've suspected them to make the server behave poorly when under load. But there still seems to be something blocking... What do you use to find the callbacks which block (for too long)? Regards Markus

5 10

[Twisted-Python] Develop application by Twisted.
by Cong Hoang Huu 03 Jul '07

03 Jul '07

Hi. If our company develops an application by Twisted and distributes (or sell) only binary packages to customer (without source code), is that ok? Do we have to transfer source code to customer also? Thanks, Cong Hoang

2 1

[Twisted-Python] Forwarding Help
by Delgado, Edgardo CIV NAVAIR 4.1.4.3 03 Jul '07

03 Jul '07

Greetings, Sorry to bug with this again!. I have already posted this issue, but I haven't received any ideas on how to solve this. So, I'm going to try again. I need help(ideas) on how to write a "server" that listens on one port and forward the info to one or more ports. I.e. "server" will listen on port XXXX and forward that information to ports xxAA, xxBB and xxCC. Can't use UDP, because it need data integrity. Thanks for all of your help!, Edgar

2 1

[Twisted-Python] twisted.web2 server timeout?
by Paul Perez 02 Jul '07

02 Jul '07

I have a newbie question. By the way, even though it is a brain-f**k, I love programming in this software. I just wish there was an easier/faster way to understand it. I got and read the twisted book, but unfortunately it does not have twisted web2 examples... Anyway, I have written a web server using web2 that takes an HTTP request and sends it off to another server to process it. If the other server were to crash, how can I make the http request timeout after 30 seconds and redirect to a timeout page? Following is my resource code. When the response is available, I take the response stream, write to it, and call finish(). class HTTPServer(resource.PostableResource): def __init__(self, cache): resource.Resource.__init__(self) self.cache = cache def _finished(self, result, httprequest, postdata): convq = self.cache['conversations'] s=stream.ProducerStream() self.cache['requests'][id(httprequest)] = (httprequest,postdata,s) if httprequest.args.has_key(CONVID): conversationid = httprequest.args[CONVID][0] try: convq[conversationid].put(httprequest) except: convq[conversationid] = Queue.Queue() convq[conversationid].put(httprequest) else: convq[ANY].put(httprequest) if log: method = httprequest.method raw_uri = urllib.unquote(httprequest.uri) print "%-4s" % method, print "%-6s" % 'CLIENT', print "%-4s" % raw_uri return http.Response(stream=s) def _failed(self, reason, httprequest, postdata): result = str(reason) result += str(httprequest.uri) + '\n' result += '\n'.join(postdata) + '\n' return http.Response(500, stream="An internal server error has occurred.") def http_POST(self, httprequest): postdata = [] d = stream.readStream(httprequest.stream, postdata.append) d.addCallbacks( self._finished, self._failed, callbackArgs=(httprequest, postdata,), errbackArgs=(httprequest, postdata,) ) # Catch any errors that occur in self._finished. d.addErrback(self._failed, httprequest, postdata) return d http_PUT = http_POST http_GET = http_POST ~Paul

1 0

[Twisted-Python] Avoid circullair dependencies.
by Ivanov Maxim 02 Jul '07

02 Jul '07

Some of my logic need protocol to be imoported to be able to send data back. So when I start app protocol imports my logic (where all data is processed) and my logic imports protocol. Right now I simply pass protocol object as argument to constructors, but it seems to be not the best way from my point of view. How to solve this problem?

2 1

[Twisted-Python] Open Source Twisted Email Server
by Ram Peters 01 Jul '07

01 Jul '07

Is there a Open Source Twisted Email Server that have been implemented? thnx.

2 2