Mailman 3 April 2007 - Twisted

[Twisted-Python] 64bit Installation Problems
by Tim Flink 05 Apr '07

05 Apr '07

I am trying to install Twisted2.5.0 and have been having some problems. When I run the main setup.py, TwistedCore is installed into /usr/lib64/python23/ while everything else (conch, lore, mail, names, news, runner, web and words) is installed into /ur/lib/python23. So now when I try to import anything outside of core (like Web) I get: exceptions.ImportError: No module named web Is there a way to force TwistedCore to build and install as 32bit or force everything else to install as 64bit? Thanks, Tim System Info: CentOS 4.4 x86_64 Python 2.3.4 Zope Interface 3.3.0

4 3

[Twisted-Python] SSL Negotiation / connectionMade() callback
by dkarbott＠comcast.net 05 Apr '07

05 Apr '07

Hi: I've been playing around with twisted a little. I have written an internet.protocol.Protocol subclass that I want to run over SSL. It seems that the Protocol.connectionMade() callback gets invoked before SSL negotiation is complete. Ideally, I don't want my Protocol to send any data until the SSL negotiation is completley finished. Is there an easy way to get callback when the SSL negotiation is done? Info: Twisted Version: 2.0.1 (emerge'd from gentoo portage) Also, I have the SSL context factories set up to do client authentication. Any help would be much appreciated. -- Darrell

1 0

[Twisted-Python] Using Twisted for distributed computation / experiment running?
by robomancer 05 Apr '07

05 Apr '07

Hello all, I'm looking to use Twisted for distributing computation over a small number (~10) of PCs. I'm wondering if anyone else has some experience with this -- particularly if there is already a solution out there that I can use, so that I'm not reinventing the wheel. Here's a rough outline of what I'd like: Setup phase: given a config file containing a list of machines and the # of CPUs on each machine, update the source code on each machine and start an appropriate number of experiment runners. Run phase: a "master" process assigns an experiment to each runner. When we get a result back, log the result to a file and send a new experiment to that runner. Repeat until all experiments are done. Here are my constraints: 1) The high-level code (at least) is all in Python, so the experiment runners can collect their results by just calling Python functions. 2) I can set up ssh keys on each machine such that logging in remotely can happen without a password. 3) I don't really have to worry about authentication: I can assume that all machines are either on a non-internet-connected LAN or that firewall rules are set up so that the ports aren't accessible except from the "master" machine. 4) I need to be able to add and remove compute nodes at runtime, so I need some sort of admin shell. However, I can wait for currently-processing experiments to finish, so I don't have to worry about the complexity of restarting experiments or migrating them to other machines. 5) It'd be nice (but not required) if the experiment runners could all log some critical messages to the master process. This seems like it would only take a few hours to implement in Twisted (probably with PB), but I wanted to make sure I'm not reinventing the wheel, because it seems likely that someone has done this before.

6 8

[Twisted-Python] Question about plugins tutorial
by Brian Costlow 04 Apr '07

04 Apr '07

I've been reading through the tutorial section on Twisted Plugins at: http://twistedmatrix.com/projects/core/documentation/howto/plugin.html I have a question about this code (the actual plugin from the tutorial) from twisted.plugin import IPlugin from matsim import imatsim class SimpleMaterial(object): implements(IPlugin, imatsim.IMaterial) def __init__(self, yieldStressFactor, dielectricConstant): self._yieldStressFactor = yieldStressFactor self.dielectricConstant = dielectricConstant def yieldStress(self, temperature): return self._yieldStressFactor * temperature steelPlate = SimpleMaterial(2.06842719e11, 2.7 + 0.2j) brassPlate = SimpleMaterial(1.03421359e11, 1.4 + 0.5j) The tutorial implies, but does not say directly, that plug-ins should not just define a class that implements IPlugin and your interface, but instantiate an object. This code (also from the example) iterates over returned plugins (actually according to the example, the objects steelPlate and brassPlate). def displayAllKnownMaterials(): for material in getPlugins(imatsim.IMaterial): displayMaterial(material) If my understanding is correct, then # plugin class Foo(object): implements(IPlugin, imyplugins.IMyPlugIn) def fooStuff(self): # do foo stuff # end plugin #call plugin from app for n in getPlugins(imyplugins.IMyPlugIn): n.fooStuff() Won't work. I'd need to do this # plugin class Foo(object): implements(IPlugin, imyplugins.IMyPlugIn) def fooStuff(self): # do foo stuff f = Foo() # end plugin #call plugin from app for n in getPlugins(imyplugins.IMyPlugIn): n.fooStuff() But I'm looking at the Plugins as a way to abstract multiple back-end actions taken in response to data recieved by a number of asynchronous events (off of LineReciever and XML-RPC). The plug-in is going to need to keep some intermediate state data in some cases, which won't work if they all get the same object. So I was thinking of something along these lines: # plugin class Foo(object): implements(IPlugin, imyplugins.IMyPlugIn) def fooStuff(self): # do foo stuff def fooFactory(self): return Foo() f = Foo() # end plugin #call plugin from app for n in getPlugins(imyplugins.IMyPlugIn): o = n.fooFactory() o.fooStuff() Does this make sense? Is there a better way in the twisted framework to handle this?

4 3

[Twisted-Python] 3 questions about async, threading and dynamic protocol selection
by 甜瓜 04 Apr '07

04 Apr '07

Q1: How to determine a function is 'blocking action' or not? Any function needs CPU times. Indeed, a computational-intensive function is blocking action. But how small/fast function can be classified as non-blocking? Twist requires all user functions to be non-blocked. If reactor calls a blocking function, what will happen? In my mind, reactor maintains a command queue internally (just like windows message queue). The blocking function only postpones the execution of other queued functions, but it does not break the logic of the program. Is that right? Q2: Today when I go through the twist document, I am confused about the threading problem in reactor. What is the different between 'reactor.callFromThread' and a plain call in reactor loop? def callFromThread(f): self.threadCallQueue.append((f, args, kw)) self.wakeUp() It seems equivalent to reactor.callLater(0, f...). What is the real circumstance for calling callFromThread? Further more, how to determine a function is thread-safe in python? Why the twist doc says: "writing data to a transport from a protocol is not thread-safe."? Q3: In my application, I need a facility to dynamically select a protocol to communicate with the server. Eg: When connected, the server sent a string to client to indicate the version of protocol it used. Then, the client can load the proper protocol. But I don't know how to implement this. A 'Factory' can only create one kind of 'Protocol', and it seems the instances of 'Protocol' cannot share the connection (Transport object) to a server. Could you give me some clues? Thank you. -- ShenLei

4 3

[Twisted-Python] Re: Using Twisted for distributed computation / experiment running
by Ed Suominen 03 Apr '07

03 Apr '07

> robomancer <robomancer(a)gmail.com> writes: > >> I'm looking to use Twisted for distributing computation over a small >> number (~10) of PCs. I'm wondering if anyone else has some experience >> with this -- particularly if there is already a solution out there >> that I can use, so that I'm not reinventing the wheel. Take a look at the asynCluster sub-package within my Twisted-Goodies project, http://foss.eepatents.com/Twisted-Goodies. Source (SVN): http://foss.eepatents.com/Twisted-Goodies/browser/trunk/twisted_goodies/asy… API: http://foss.eepatents.com/api/twisted-goodies/twisted_goodies.asyncluster.h… Best regards, Ed Suominen

1 0

[Twisted-Python] listenTCP(), tcp.Port()
by Matthew Glubb 02 Apr '07

02 Apr '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, I want to create a simple server that listens for data but I do not want to specify the port, I want the port to be automatically selected based on the currently available ports on the system. Is there any way to do this using Twisted? Using PHP it is a simple matter of: // Create a socket $sock = @socket_create(AF_INET, SOCK_STREAM, SOL_TCP); // Get the port number used @socket_getsockname($sock, $ip, $port); Where $ip and $port are passed by reference and result in the values for the socket's ip address and port. Looking at tcp.Port(), it doesn't seem like there is any provision for dynamically selecting an available port. Or am I missing something? Regards, Matt m a t t h e w g l u b b ________________________________________________________________________ Z Group PLC Tel: +44 (0) 8700 111 173 Fax: +44 (0) 8707 051 393 Txt: +44 (0) 7800 140 877 Web: <http://www.zgroupplc.com/> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. The opinions expressed in this mail are those of the author and do not necessarily represent the views of the company. If you have received this email in error please notify <service(a)zgroupplc.com> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFGESWCyI6MkdKPngkRAjXyAJ9Yd9o/vWeYG9PhB1NbphbGgN2quwCdHRDh DqF0IRAxqhcKLoKMeH+Ebwg= =8qwC -----END PGP SIGNATURE-----

2 2

[Twisted-Python] 3 basic questions about connectionLost, reactor stop and design
by 甜瓜 02 Apr '07

02 Apr '07

Hi all, Expecting your kind answer regardless how primary my questions are. ^_^ Q1: Protocols has a method: connectionLost(self, reason). What is the type of the second parameter? I see it sometimes is twisted.internet.error.XXX. But who pass this param? Can I break a connection by passing my own 'reason'? The method transport.lostConnection is disappointment to me because I cannot pass any parameter to it to indicate the reason. Q2: How to cleanly exit a twist program? Eg: When a twisted client called transport.lostConnection or cut off net wire physically, only the connection is broken. The reactor is still running in the event loop. -_-!! What is the proper place to put reactor.stop()? Is there any other way to normally exit a twist program? Q3: Let's continue my previous question about 'class attribute'. Phil Mayers gave the key to me: 'if you then modify an instance, the class and other instances are not, and those modifications OVERRIDE ANY CONFLICTING ones on the class.' class A: i = 0 p = A() q = A() p.i = 1 p.__dict__ # member varible added >>> {'i': 1} p.__class__.i = 2 # explicit access to A.i q.i >>> 2 # shared A.i Therefore, in my opinion, the class attribute A::i is always shared by instances. Different from C++, python 'p.i = 1' will add {'i': 1} to p.__dict__, which makes 'i' as the 'member varible' to instance 'p'. It seems 'getattr' prefers 'member variable' than 'class attribute'. So later access to 'p.i' will refer to 'member variable', and the 'class attribute' is masked. But if I use p.__class__.i explicitly, the masked 'class attribute' appears. Till now, Have I misunderstood anything? ^_^ Finally, my question is why twisted use class attribute so widely? Is there any benefit? In my opinion, __init__ function is the only good place to define 'attributes'. But twist distributes 'attributes' in two forms: class and __init__. I want to know the idiom to decide which attribute should be put into which part. Thank you for your patience! -- ShenLei

5 5

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 01 Apr '07

01 Apr '07

Bug summary ______________________ Summary for 2007-03-25 through 2007-04-01 Bugs opened: 12 Bugs closed: 4 Total open bugs: 889 (+8) |== Type Changes |== Priority Changes |== Component Changes |Defect: +7 |Highest: +0 |Core: +6 |Enhancement: +4 |High: +1 |Ftp: +2 |Task: -1 |Normal: +8 |Names: -1 |Low: +1 |Web: +1 |Web2: +1 |Website: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2539] Add epoll reactor to the choosing-reactor doc (opened by therve) enhancement core http://twistedmatrix.com/trac/ticket/2539 [#1160] FTPClient failed commands return a FirstError (opened by orbitz) defect ftp http://twistedmatrix.com/trac/ticket/1160 ===== High ===== [#1107] [PATCH] FTPClient does not fire current commands errback on connectionLost (opened by orbitz) defect ftp http://twistedmatrix.com/trac/ticket/1107 ===== Normal ===== [#2540] twistd daemonizing before calling setup code (opened by hagna) enhancement core http://twistedmatrix.com/trac/ticket/2540 [#2541] Decide on and document Python version support policy (opened by jml) enhancement core http://twistedmatrix.com/trac/ticket/2541 [#2542] twisted.web.microdom.Node.isEqualToNode does no actual comparison (opened by dreid) defect web http://twistedmatrix.com/trac/ticket/2542 [#2543] "Noone" isn't a word (opened by indigo) defect core http://twistedmatrix.com/trac/ticket/2543 [#2544] twisted.web2.channel.[s]cgi contain multiple uses of a function that doesn't exist. (opened by dreid) defect web2 http://twistedmatrix.com/trac/ticket/2544 [#2545] Multiple select reactors to support use of Twisted by non-Twisted libraries (opened by j1m) enhancement core http://twistedmatrix.com/trac/ticket/2545 [#2546] Are stream/push producers assumed to start paused or unpaused? Need more docs. :) (opened by j1m) defect core http://twistedmatrix.com/trac/ticket/2546 [#2547] API Docs has no info about attributes (opened by redbaron) enhancement website http://twistedmatrix.com/trac/ticket/2547 ===== Low ===== [#2548] Need IClientFactory? (opened by j1m) defect core http://twistedmatrix.com/trac/ticket/2548 Closed Bugs ______________________ ===== Highest ===== [#2414] Refactor DNSProtocol and DNSDatagramProtocol (opened by therve, closed by therve, fixed) enhancement names http://twistedmatrix.com/trac/ticket/2414 [#2426] Delete the s*** at the end of failure.py (opened by exarkun, closed by exarkun, fixed) task core http://twistedmatrix.com/trac/ticket/2426 [#1160] FTPClient failed commands return a FirstError (opened by orbitz, closed by therve, ) defect ftp http://twistedmatrix.com/trac/ticket/1160 ===== High ===== [#1107] [PATCH] FTPClient does not fire current commands errback on connectionLost (opened by orbitz, closed by therve, ) defect ftp http://twistedmatrix.com/trac/ticket/1107

1 0