Mailman 3 July 2008 - Twisted

[Twisted-Python] synchronous problem
by Wu Zhe July 9, 2008

July 9, 2008

Hi I have a series of files to transfer, each followed by a line sent to specify the file size and crc number. As we can see here I need synchronous behavior, only after the previous file transfer complete, can the for loop step into the next iteration to send the next file via sender.beginFileTransfer. How can i achieve this? Here is the problematic code: def import_query(self, path_list): number_of_files = len(path_list) line = '3#%s' % number_of_files self.client_instance.sendLine(line) sender = FileSender() sender.CHUNK_SIZE = common.chunk_size for path in path_list: f = open(path, 'rb') s = f.read() filesize = len(s) crc = crc32(s) line = '3#%s:%s' % (filesize, crc) self.client_instance.sendLine(line) def transfer_completed(lastsent): f.close() self.client_instance.setLineMode() # send the file self.setRawMode() d = sender.beginFileTransfer(f, self.client_instance.transport) d.addCallback(transfer_completed) -- Wu Zhe Best Regards

2 3

[Twisted-Python] Re: [Twisted-bugs] pydoctor should put the module name at the beginning of the document title
by Stephen Waterbury July 9, 2008

July 9, 2008

trac(a)twistedmatrix.com wrote: > New submission from TimAllen <tim(a)commsecure.com.au>: > [description] pydoctor is not part of twisted -- it's a separate project, with its own bug tracker. See: http://codespeak.net/~mwh/pydoctor/ Steve

2 1

[Twisted-Python] Gc of deferreds
by Rutt, Benjamin July 8, 2008

July 8, 2008

In the following program, what prevents the deferred "d" from being gc'd? (nothing saves the return of foo() in any variable). Do I need to keep a reference to it around to ensure the callbacks will run? #!/usr/bin/env python from twisted.internet import defer,reactor,utils def foo(): d = utils.getProcessOutput('sleep', args=['2']) def cb(data): print "cb fired" return data d.addCallback(cb) return d foo() reactor.callLater(5, reactor.stop) reactor.run() Yet when I run it, it seems everything works fine: $ ./spawner.py cb fired $ I am just questioning the need to keep deferreds in memory somewhere, if they haven't fired their callbacks yet. Thanks.

3 3

[Twisted-Python] uninstall Twisted-8.1.0
by raja pavankumar July 8, 2008

July 8, 2008

hello, I need to uninstall Twisted-8.1.0 and use Twisted-2.2.0. Can some one help me as to how to uninstall Twisted-8.1.0. Thanks in advance rajainnov

4 4

[Twisted-Python] cred realm with __init__ methods under twistd
by Ross Vandegrift July 8, 2008

July 8, 2008

Hello everyone, I'm working on a twisted application that uses cred and pb. I'm having a bit of trouble getting my realm working as I want. In my .tac file, I have this: p = portal.Portal(CNDRealm()) c = checkers.InMemoryUsernamePasswordDatabaseDontUse(user="passwd") p.registerChecker(c) application = service.Application("receiver") service = service.IServiceCollection(application) factory = pb.PBServerFactory(p) internet.TCPServer(8800, factory).setServiceParent(service) My realm, CNDRealm, looks like this: from globals import devs, queues, conditions, devices class CNDRealm(object): implements(portal.IRealm) def __init__(self): for i in devs: queues[i] = list() conditions[i] = threading.Condition() devices[i] = CNDDevice(i[0], socket.gethostbyname(i[0]), i[1], queues[i], conditions[i]) log.msg("Initialized devices: " % devices) def requestAvatar(self, aid, min, *interfaces): log.msg("Avatar requested: %s" % aid) if pb.IPerspective not in interfaces: raise NotImplementedError return pb.IPerspective, CNDSubmitter(aid), lambda: None However, I never get the log message from the __init__ method. I know that it's not being run because those dictionaries are never populated with the data my app needs to be in there. 1) What's going on? I'm pretty surprised to find that an object's __init__ doesn't seem to be getting called! 2) Is there some better place in a twistd app to put general start-up initialization kinds of things? -- Ross Vandegrift ross(a)kallisti.us "The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell." --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37

2 2

[Twisted-Python] [SIP]
by Rémi BUISSON July 7, 2008

July 7, 2008

Hi everyone, I didn't succeed in using twisted.protocols.sip ? Do you have a simple example for me ? It will be very nice ... Thanks in advance !

2 1

[Twisted-Python] Confusion on Twisted versions
by happybrowndog July 6, 2008

July 6, 2008

I'm finding the Twisted version numbers confusing. I have it installed on Debian, where aptitude is telling me that 2.4.4 is installed (with python 2.4). However, the TwistedMatrix site states that there is an 8.1 version. Googling shows that people talk of 2.4x and 8.x in an intermingled fashion. Can someone please clarify?

2 1

[Twisted-Python] The right way to change a global variable in a Twisted server?
by happybrowndog July 6, 2008

July 6, 2008

Is there a right way to change a global variable in a Twisted server (this one is using PerspectiveBroker)? What I am trying to do is allow perspectivebroker clients to update a global variable on the twisted server without colliding, ie: a counter that increases by 1 each time a particular call is made to the Twisted server. Each client has a pb.Avatar attempting to update that global variable counter through a method call on the class extended on pb.Avatar. As the server is select-based, am I wrong to assume that only one client is executing pb.Avatar calls at a time?

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com July 6, 2008

July 6, 2008

1 0

[Twisted-Python] Peloton Distributed Service Application Engine
by Matthew Pontefract July 5, 2008

July 5, 2008

Hello Twisted Users I'd like to announce a new project called Peloton that may be of interest to some of you. Peloton is a distributed service application platform built in Python on top of Twisted. It's goals are: * To make trivial the process of writing, publishing a method * To provide all network interfaces on any relevant protocols and insulate the service writer from all of this * To provide simple, intuitive, modular mechanisms to transform the results of a method into a form appropriate to the end user * To take care of managing services to ensure they stay alive, are distributed across a grid of computers and have requests distributed between them * To manage failure by re-issuing failed requests and managing the grid of nodes in such a way that, as far as is reasonably possible, the client is never troubled to know what lengths were gone to to return a result * To provide scalability * To provide inherent parallelisation for trivial problems * To provide an event bus accessible to server-side components and clients * To provide for easy (hot) upgrade of components and near zero downtime Why? I have worked in banks, small firms, big firms, supercomputing firms. I have worked in Java, in .NET, in Python. In almost all cases I was writing client-server systems where the application was event driven, needed to be resilient and would benefit from distribution for horizontal scalability. In all cases, making the results of server side operations available to as many different third party components as possible (via a multitude of protocols) would have been useful. And nowadays having data easily transformed into snippets of HTML or other formats, perhaps for AJAX apps etc is increasingly useful. Peloton is built to leverage the clarity, simplicity, power and rapid development times of Python and, using the reliable and solid Twisted core, provide the higher level features that give me all the above in a small and easy package. I'm not aiming to re-invent the wheel: I needed an event bus and, having used many, I've elected to use AMQP providers, RabbitMQ being a favourite. But this is all switchable because such functionality is managed by Peloton kernel plugins. Similar projects to Peloton exist: Pythomnic and Divmod-Mantissa to name two. They provide some, not all of the functionality or do things in ways that didn't quite suit me etc. Peloton is new, but I've implemented the ideas previously with the same technology at a supercomputing facility, and I presented the work at PyCon UK 2007. It works, people like it and its use is growing there. That's good enough for me to know that, at least for a section of the community, this is probably of interest. The future Peloton is very early at the moment and much is to come: a scheduler (cron for peloton) properly implemented distributed shared-state engine, different ways of deploying service code, better management tools, and more. Tight integration with Java could extend it's use into banking environments. A suite of services built on Peloton for job scheduling and management will provide another avenue to explore. It's all fun, and all interesting. It builds on the truly fantastic work of others and hopefully makes something novel and interesting. But that's for you to decide and I invite you to scout around the documentation, download the source code and give it a whirl. I'd love to hear what you think, know how you're using it and work with you to make it better. Where from? http://www.peloton-grid.net is the home of Peloton and hosts documentation and details of how to obtain Peloton. I hope it appeals and look forward to discussing with some of you about it. We can always meet on freenode.net, channel #peloton. Regards Matthew

2 1