Mailman 3 November 2006 - Twisted

[Twisted-Python] Re: Callback blocked in event loop
by Ed Suominen 11 Nov '06

11 Nov '06

> On Fri, 10 Nov 2006 15:24:51 +0000 (GMT), Matt P wrote: >>> Events may be raised in the server, as a twisted.pb call to >>> fireEvent on the server and also come in on a separate thread (from >>> a JMS message queue) and get posted onto the DefferedQueue for >>> events. >>> >>> If events are raised in the server, remote listeners get their >>> pb.Referenceable callback immediately. It works as I would like. >>> >>> [snip] >>> >>> So, only in the case where teh DeferredQueue.put(...) is called >>> from another thread do I have the problem. I can't see why the >>> blocking. Does this behaviour make sense to anyone!? > > DeferredQueue isn't threadsafe. Instead of calling put() directly from > a non-reactor thread, call it with reactor.callFromThread: > > reactor.callFromThread(q.put, value) > > Jean-Paul Matt, this is exactly the sort of thing for which I wrote the taskqueue module that is patiently awaiting review in ticket #1768 (http://twistedmatrix.com/trac/ticket/1768). It is a critical part of my sAsync project ("SQLAlchemy done asynchronously"). You can check out the latest version as used in that project at: http://foss.eepatents.com/sAsync/browser/trunk/sasync/taskqueue.py. The part that would appear to be of particular interested to you is the ThreadQueue subclass of TaskQueue. Best regards, Ed Suominen

1 0

[Twisted-Python] Newbie to Twisted. Converting Asyncore App to Twisted App
by Sean Roark 10 Nov '06

10 Nov '06

I am new to this Twisted Architecture so if my plan makes no sense, please guide me down a smoother path. I have a program using Asyncore as the communications interface but I want to switch to Twisted. Server1 Server 2 <-------TCP------| | <---TCP------- | My App | ----TCP--------->| |----TCP-----> Basically MyApp takes stuff from Server 1, modifies it in some way, then sends it to server 2 and vice versa for Server 2 to Server 1. It is a translator between two ICD's languages spoken by two different systems. All data in and out is raw. I have modules written that can parse the raw data into objects, translate the objects into the other ICD, then repack the objects into raw data. Currently, My App uses asyncore to manage the socket between the servers as pictured above. Asyncore sticks the raw data on a Queue for the next object to retrieve. Each object is a thread that blocks on the Queue until data arrives. When a object receives data on the Queue, it performs its operation and sticks the modified data on another queue for the next thread to handle. Eventually, data ends up on a Queue being monitored by an Asyncore dispatcher that grabs the data and sends it to a server. Server1 ----TCP----->Asyncore.recv() --put--> Queue --get-->Parser--put--> Queue --get--> Translator --put--> Queue --get--> Packer --put---> Queue --get-->Asyncore.send() I would like to use Twisted for the following reasons: 1. Add a gui, stand alone or web based, allowing users to view statistics being generated by modules and to inject messages to one of the servers. Other services may be necessary in the future 2. We have a problem with asyncore behaving differently on Windows servers then Linux servers with regards to reconnecting after network drops. It is greatly beneficial if we can run on either platform. 3. Twistd is a great way to run this program as a daemon on Linux but still works on Windows 4. My own personal education 5. Twisted seems so cool Questions that have arisen as i try to code this in Twisted. 1. What is the proper method for gluing my processing modules to Twisted? I have followed the finger tutorial and it seems I should call them from a Service class. Finger tutorial part 3<http://twistedmatrix.com/projects/core/documentation/howto/tutorial/style.h…> This leads to question 2 I have toyed with using DeferredQueue's to pass data between the modules. I would pretty much follow the pattern above, but the DeferredQueue's should allow my modules to NOT run as threads. Instead they just fire events when data arrives on their Queue. 2. How do I have My App send data to the servers via events? In the finger tutorial Part 3 there are getUser and getUsers methods that are called by the FingerProtocol class via the FingerClientFactory. This all makes sense for processing data from the network and responding to it, but suppose I want some other object, say my Packer object, to fire an event that will make FingerProtocol send some data from Packer to the server? How do I expose a method to do this if Packer only knows about the Service, but the Service has no access to methods in FingerProtocol? I have come up with two solutions to this problem, neither of which is impressive to me. First solution, use DeferredQueue's to pass all the data around. This does work as a prototype, but I have no Service class wrapping it. class QueueProtocol(Protocol): "Handles the network receive and send work for a send and a receive queue" def dataReceived(self, data): print "Received Data:",data self.factory.putData(data) def connectionMade(self): print "Conn Made" self.sendData() def sendData(self): print "sendData" self.factory.getData().addCallback(self.sendHelp) def sendHelp(self,data): print "SendHelp",data self.transport.write(data+"\r\n") self.sendData() class QueueClientFactory(ClientFactory): "Handle the Connection" protocol = QueueProtocol def __init__(self,recQueue, sendQueue): self.recQueue = recQueue #both deferredQueue's self.sendQueue = sendQueue def putData(self,data): self.recQueue.put(data) def getData(self): "Returns a defered because sendQueue is a DeferredQueue" return self.sendQueue.get() Second solution, that I have not managed to code. I have the feeling I'm way of the mark here. Each component (yes I have been reading the Zope docs) in the application knows about interfaces. All components will implement a receive(data) method and a process(data) method. Each of these objects can pass data to other components by adding another components receive(data) method as a callback to its own process(data) method. Parse def receive(data): self.process(data).addCallback( Translate.receive(data) ) The Parse object receives data from somewhere. The process method takes this raw data and parses it into an object. When process has completed the callback method, defined in the Translate object, is called with the new object as a parameter. Parse's Factory class will take an instance of Translate in __init__. Some guidance would make my brain hurt less. Thanks, -- Sean Roark

1 0

[Twisted-Python] I want to know whether Twisted can run in Python2.5 .
by 张沈鹏 10 Nov '06

10 Nov '06

I want to know whether Twisted can run in Python2.5 .

3 3

[Twisted-Python] Callback blocked in event loop
by Matt P 10 Nov '06

10 Nov '06

Hello I've got a bit of an issue that someone may recognise and bea ble to set me straight! I have a twisted.spread server which, amongst other things, provides an event pub/sub engine. The subscription manager and event publisher runs in the twisted event loop: an event is posted onto a DeferredQueue and the callback on that Queue iterates through listeners calling eventFired(...). Some event listeners run in-server, but others are seperate processes which register by calling the register function on the server and providing a pb.Referenceable that the event publisher can call eventFired(...) on. Events may be raised in the server, as a twisted.pb call to fireEvent on the server and also come in on a separate thread (from a JMS message queue) and get posted onto the DefferedQueue for events. If events are raised in the server, remote listeners get their pb.Referenceable callback immediately. It works as I would like. However, if a message comes in off the JMS bus, the DeferredQueue callback gets called, it in turns calls the referenceable callback and nothing happens. Only when the server handles *any* other Twisted event do the event callbacks execute and my remote listeners get the message. So, only in the case where teh DeferredQueue.put(...) is called from another thread do I have the problem. I can't see why the blocking. Does this behaviour make sense to anyone!? Many thanks in advance for any help!! Matthew

3 2

[Twisted-Python] SMTP server
by Jan Bakuwel 10 Nov '06

10 Nov '06

Hoi all, This is my first message to the list so I think its appropriate to express my appreciation for all the excellent work done by the people who created & contributed to the twisted libraries. I have a small but interesting problem that I'd like to ask your advise on, please see the description below. If you notice that I haven't understood something properly, I hope you'll correct me :-) I've implemented a SMTP server (I've called it "xsmtp") using the twisted & python libraries. A MTA (Postfix) delivers email messages to the xsmtp server: the MTA opens a single SMTP session to the xsmpt server, then multiple "RCPT TO"s follow, one for each recipient. As a result The "validateFrom" procedure is called by the twisted library once, followed by multiple calls to the "validateTo" procedure. This is creating a small problem for me. Since I must process the email in the validateTo procedure, I will now receive multiple copies of the same email (albeit for different recipients). One of the tasks of the xsmtp server is to transmit these emails to another xsmtp server (SMTP cannot be used due to the inherently high latency connection between the two). Ideally I do not transfer the same email twice but would like to leave the delivery to multiple recipients up to the MTA on the other end. In a schema: -> MTA -> xsmtp -- high latency link --> xsmtp -> MTA -> local delivery To implement smtp.IMessageDelivery I have to provide for the following procedures: validateFrom: called once shortly after SMTP session is initiated receivedHeader: called multiple times for each recipient validateTo: called multiple tiems for each recipient One could say that "validateFrom" is the start of the process (of receiving an email) "receivedHeader/validateTo" is the core of the process. What I am missing is something that indicates the end of the process. It would help me tremendously if smtp.IMessageDelivery would also provide an abstract method that is called once the message delivery is completed. Any suggestions, thoughts are very welcome; perhaps this is already possible with the twisted libraries in a different way as I'm suggesting here? best regards, Jan -- "It is an illusion to think that we are free to apply our increasing knowledge in any way we choose. In reality we are struggling to deal with the consequences of it." -- John Gray ---- Electronic Technology Coordinator Information/Communication/Technology Greenpeace International Ottho Heldringstraat 5 1066 AZ AMSTERDAM Netherlands (MET) direct +31 (0)20 7182084 fax +31 (0)20 5148151 reception +31 (0)20 5148150 email jan.bakuwel&int.greenpeace.org private jan.bakuwel&hccnet.nl (replace & by @ in the emailaddress)

2 3

[Twisted-Python] article
by Katarzyna Winiarz 09 Nov '06

09 Nov '06

Hello, We need of an article about project Twisted- Python for the magazine for programers and developers of Linux (www.linuxdevjournal.com). This magazine is published bimonthly in french. The article must be strictly technic and pratically oriented. Longeur it's 10-15 pages in text editor(20 000 - 30 000 caracters). We will pay 25 euros for the page of the texte past the print, without listings and figures. Kasia -- Katarzyna Winiarz katarzyna.winiarz(a)linuxdevjournal.com www.linuxdevjournal.com tél. 0048 22 887 13 44 fax. 0048 22 887 10 11

1 0

[Twisted-Python] Installation of Twisted (Sumo) failing in web, words (I think)
by James Kanze 09 Nov '06

09 Nov '06

I think I'm having a problem with the installation of the twisted Sumo package (TwistedSumo-2006-02-12, but Twisted-2.4.0 doesn't seem to work either). Basically, I'm trying to get PyAIM-t working, under Linux (on a machine with a very stripped down configuration, and on which I don't have root access). PyAIM-t requires Twisted (core, web and words, at least), and recommends the 2.2.0 version, which is in the TwistedSumo package above. So I installed and built the package, to the best of my ability (this is my first time dealing with Python). But PyAIM-t still doesn't seem to find what it is looking for. I added a few print statements where the problem was occuring, so the code looks something like this: try: print sys.path print "trying domish" from twisted.words.xish.domish import SuxElementStream, \ Element, unescapedFromXml print "got domish" # ... except ImportError: print sys.exc_info()[1] # ... The output here is: [... , '/home/team02/jakan/gnu/lib/python2.4/site-packages', ...] trying domish No module named words.xish.domish What makes me thing that the problem is in my installation of twisted: when I invoke "python setup.py" from the root, I get a lot of messages along the lines of: package init filie 'twisted/plugins/__init__.py' not found (or not a regular file) In addition, to get anything in the library directory from the path above, I have to add the --force flag, so that my install command looks like: python setup.py --prefix=/home/team02/jakan/gnu \ --exec-prefix=/home/team02/jakan/gnu/i686 --force I'm sure that I'm doing something stupid and obvious that I'm missing, but I don't quite no where to turn. (This is really my first contact with Python, starting yesterday. I want to learn it, but management wants a proof of concept that what we want to do will work before approving the project.) -- James Kanze (GABI Software) mailto:james.kanze@gmail.com Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

1 0

[Twisted-Python] Twisted article
by Katarzyna Winiarz 09 Nov '06

09 Nov '06

Hello, I am the editor of the magazines Linux Developer's Journal and MSCoder. Those magazines are designed for programers and developer's of plateforms Linux (ldj) and of Microsoft (MSCoder).They are published bimonthly in french, spanish and german. I would like to ask you if someone of you could write for us the article about Twisted in french linguistic version. I would be very content. Best regards Kasia -- Katarzyna Winiarz katarzyna.winiarz(a)linuxdevjournal.com www.linuxdevjournal.com www.mscoder.org tél. 0048 22 887 13 44 fax. 0048 22 887 10 11

2 1

[Twisted-Python] Understanding Deferreds/callback further
by Yi Qiang 09 Nov '06

09 Nov '06

Hi guys, I am still having trouble writing programs that will make the most use of deferreds. I will describe the problem I am trying to solve and the solution I've tried. The problem: I am trying to write a client that gets a job from a PB server, processes it, submits the result and gets another job. I need to be able to do this for multiple jobs (i.e. I need to be able to get 5 jobs, and start 5 similar chains: get job --> work on job --> submit result --> get another job). This looks like a perfectly solvable problem using deferreds since working on the job might take a while and using callbacks seems to be the most appropriate. So I have something like this in my main method: monitor = Monitor(server, port) monitor.connect() def connect(self): factory = pb.PBClientFactory() reactor.connectTCP(self.hostname, self.port, factory) return factory.login(self.credentials).addCallback( self._connected).addErrback( self._catch_failure) Then in self._connected I would like to start the chain of getting jobs/processing them/submitting jobs: def _connected(self, remoteobj): self.remoteobj = remoteobj for worker in self.workers: self.get_jobs(worker).addCallback( self._got_job, worker).addCallback( self.get_jobs, worker).addErrback( self._catch_failure) _got_job takes both a job and a worker and starts to process the job. Which for now returns a defer.succeed(). This is the part where I stumble on though, the exact implementation of how to make it so that one calls back the other. Any help would be appreciated. Please let me know also if you need more specific information. Thanks, Yi

2 6

Re: [Twisted-Python] Understanding Deferreds/callback further
by Samuel Reynolds 09 Nov '06

09 Nov '06

>Ok. The thing I am trying to design/code is a job distribution >system, for a distributed computation model. There is a server >which interacts with the job database (which is just a bsddb). The >server accepts connections from clients and distributes jobs to >them. The server also accepts connections from a 2nd type of >client, which submits jobs to the server. Sounds like a distributed task queue. Users/user processes submit tasks to the master queue manager (the "server") via a defined interface (HTTP post, SOAP, XML-RPC, PB, or ???). The master queue manager adds the requested tasks to the master job queue. A secondary queue manager on each "client" machine (the "monitor") requests tasks from the master queue when its local queue drops below a defined threshold number of jobs (or drops to zero). When the local queue is empty, the secondary queue manager periodically requests tasks from the master. It adds any tasks received to its local queue. For each received task, the secondary queue manager starts a worker task, immediately receiving a deferred. It registers success and error callbacks on the deferred. When it gets the success or error callback from a worker, the secondary queue manager records the result and reports it to the master queue controller (the "server"). When it has successfully reported to the master, it deletes the task from the local queue. Additional wrinkles: (1) Intertask dependencies may require that some tasks be performed sequentially, on the same slave machine, or both. (2) Some tasks may be more important than others, and so must be processed (or at least started) first. (3) Task cancellation at the master may require that the master be able to send asynchronous task-cancel instructions to the slaves. >The workers of course have to submit their results back to the >server, but in the system I envisioned the monitor takes care of >that by continuously polling the status of the workers. If the worker returns a deferred, the monitor doesn't need to poll; it'll get a callback when the worker completes (or fails). HTH. - Sam

1 0

[Twisted-Python] OpenID ?
by Jonathan Vanasco 09 Nov '06

09 Nov '06

Has anyone implemented an OpenID server in twisted? I need to roll out OpenID support in my application. I'm finding a some things I don't like in the Python OpenID implementation thats out there, so I'm hoping that there might be a twisted version already? // Jonathan Vanasco | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | FindMeOn.com - The cure for Multiple Web Personality Disorder | Web Identity Management and 3D Social Networking | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | RoadSound.com - Tools For Bands, Stuff For Fans | Collaborative Online Management And Syndication Tools | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

1 0

[Twisted-Python] PyQt4 Object acting like None.
by Nabil 07 Nov '06

07 Nov '06

Hello everyone. I'm having some trouble with a program I'm writing. The program uses PyQt4 and qt4reactor. When the program is started it requests a file from the server that it downloads and saves onto the client machine. The file is a python module that gets dynamically loaded and used in the program. All of this works fine on a Max OS X 10.4.6 using Python 2.4.4 and Qt-4.1.0. When I try to run the program on window I get this error message: File "C:\path\to\downloaded\module", line X, in init self.panel = PhoneBookPanel(self, parent.getWiget()) exceptions.TypeError: 'NoneType' object is not callable parent: instance of QMainWindow parent.getWidget: return a QWidget that will be the parent widget of self.panel self.panel is a QWidget instance that will be added to parent.getWidget() The thing thats really confusing me is that although it works perfectly on a mac it crashes on windows using the same version of Qt and Python!!

1 0

[Twisted-Python] DeferredQueue Unstable
by Sean Roark 07 Nov '06

07 Nov '06

The API says the DeferredQueue class from the defer module is unstable? Any description about why it is unstable? I would like to use it else I will have to write my own version that is sure to 'unstable'. -- Sean Roark

2 1

[Twisted-Python] Twisted and Torrent Support
by Chas. 06 Nov '06

06 Nov '06

I am looking for both client and server support for bittorrent that is integrated with Twisted (as are the ftp, etc). I was wondering if any one knows of just such support. Thanks, Chaz.

2 1

[Twisted-Python] Problem with qt4reactor.py
by IloChab 06 Nov '06

06 Nov '06

I'm trying to run a simple application with Twisted and Qt4, but I get a really strange behavior from my main script. I can't understand if it's a python or a twisted or a Qt4 problem I'm using this reactor: http://twistedmatrix.com/trac/attachment/ticket/1770/qt4reactor.py Now, if I run this: # >>>>>>>>> import qt4reactor import sys from PyQt4 import QtGui from winIum import Window def main(): app = QtGui.QApplication(sys.argv) qt4reactor.install(app) MainWindow = QtGui.QMainWindow() win = Window(MainWindow) MainWindow.show() from twisted.internet import reactor reactor.run() # <<<<<<<< my window shows and run correctly. If I run this: # >>>>>>>>>>>>> import qt4reactor import sys from PyQt4 import QtGui from winIum import Window def creApp(): app = QtGui.QApplication(sys.argv) qt4reactor.install(app) return app def creWin(): MainWindow = QtGui.QMainWindow() win = Window(MainWindow) MainWindow.show() def main(): app = creApp() creWin() from twisted.internet import reactor reactor.run() # <<<<<<<<<<<<< my window doesn't show and the script doesn't stop but remains trapped in some gui loop. What's the problem I can't see?? Thank you in advance for any help. Licia

1 0

[Twisted-Python] If lineReceived returns deferred, then connection is dropped and this deferred given as error message
by Marcin Kasperski 06 Nov '06

06 Nov '06

I am observing the twisted behaviour which seems rather strange to me. In short: if lineReceived returns some deferred, then the whole connection is aborted, moreover, this deferred is returned as error message. Longer story (the code pieces extracted from quite large app, I have not yet tried writing small example) ### Crucial code fragments class SomeProtocol(basic.LineReceiver): (...) def lineReceived(self, line): (...) return defer.succeed(997) # magic value I can recognize easily class SomeFactory(protocol.ReconnectingClientFactory): (...) def clientConnectionLost(self, connector, reason): logger.warn("Connection lost. Reason: %s\nTraceback: %s" % ( reason.getErrorMessage(), reason.getTraceback())) protocol.ReconnectingClientFactory.clientConnectionLost(self, connector, reason) reactor.connectTCP(HOST, PORT, SomeFactory(...)) reactor.run() ### Things seen in log file myapp : WARNING Connection lost. Reason: <Deferred at 0xB551E1ECL current result: 997> Traceback: Traceback (most recent call last): Failure: <type 'instance'>: <Deferred at 0xB551E1ECL current result: 997> (you recognized this 997, didn't you?) ### Extra info 1. Removing return defer.succeed() from lineReceived helps. 2. I wanted to return deferreds from lineReceived because I wanted to write unit tests testing my factory object (and in some cases to check whether all worked correctly required waiting for some spawned deferred calls).

2 5

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 05 Nov '06

05 Nov '06

Bug summary ______________________ Summary for 2006-10-29 through 2006-11-05 Bugs opened: 17 Bugs closed: 1 Total open bugs: 777 (+16) |== Type Changes |== Priority Changes |== Component Changes |Defect: +6 |Highest: +1 |Core: +8 |Enhancement: +8 |Normal: +15 |Lore: +1 |Task: +2 |Pydoctor: +1 |Web: +1 |Web2: +1 |Website: +1 |Words: +3 New / Reopened Bugs ______________________ ===== Highest ===== [#2203] More complete IRC Server implementation (opened by colin) enhancement words http://twistedmatrix.com/trac/ticket/2203 ===== Normal ===== [#2202] The News on the front page is no longer news. (opened by jml) defect website http://twistedmatrix.com/trac/ticket/2202 [#2204] IRCUser shouldn't send topic if there isn't one. (opened by colin) defect words http://twistedmatrix.com/trac/ticket/2204 [#2205] Simplification of Application creation (opened by jerub) enhancement core http://twistedmatrix.com/trac/ticket/2205 [#2206] Update standard for unit tests (opened by jml) task core http://twistedmatrix.com/trac/ticket/2206 [#2207] lore tarball has files with wrong end of line encoding (opened by thomasvs) enhancement lore http://twistedmatrix.com/trac/ticket/2207 [#2208] a lot of examples have different shebang lines (opened by thomasvs) enhancement core http://twistedmatrix.com/trac/ticket/2208 [#2209] pydoctor has problems with moresystems' using an absolute url prefix (opened by mwh) defect pydoctor http://twistedmatrix.com/trac/ticket/2209 [#2210] pydoctor system pickles that use moresystems can't be loaded (opened by mwh) defect pydoctor http://twistedmatrix.com/trac/ticket/2210 [#2211] unit tests should not have shebang lines (opened by thomasvs) enhancement web http://twistedmatrix.com/trac/ticket/2211 [#2212] Remove mention of threadable.init from the threading chapter (opened by radix) enhancement core http://twistedmatrix.com/trac/ticket/2212 [#2213] Several deprecated twisted.protocols.* modules point to old twistedmatrix.com/projects/ webpages (opened by dreid) task core http://twistedmatrix.com/trac/ticket/2213 [#2214] twisted.web2.xmlrpc.XMLRPC._cbRender uses a bare except and doesn't log the failure. (opened by dreid) enhancement web2 http://twistedmatrix.com/trac/ticket/2214 [#2215] If lineReceived returns true value, the connection is shut down and this value used as an error message (opened by Mekk) defect core http://twistedmatrix.com/trac/ticket/2215 [#2216] Coding Standard update: Lines should not have trailing whitespace (opened by jml) enhancement core http://twistedmatrix.com/trac/ticket/2216 [#2217] Coding Standard is fairly crusty (opened by jml) defect core http://twistedmatrix.com/trac/ticket/2217 [#2218] Add timeout to XMPP IQ requests (opened by ralphm) enhancement words http://twistedmatrix.com/trac/ticket/2218 Closed Bugs ______________________ ===== Normal ===== [#2069] Make nevowwriter the default (opened by jml, closed by mwh, fixed) enhancement pydoctor http://twistedmatrix.com/trac/ticket/2069

1 0

[Twisted-Python] SMTP with authentication
by Alvin Wang 05 Nov '06

05 Nov '06

Hi, I am new to twisted but google and the Oreilly book have failed me. I am trying to create a client to send to a smtp server. All the examples that I have found only work if your SMTP server does not require authentication. I am guessing that I need to use http://twistedmatrix.com/documents/current/api/twisted.mail.smtp.ESMTPSende… Unfortunately, I can find no examples that use it. I did find something from 2004 http://twistedmatrix.com/pipermail/twisted-python/2004-April/007433.html That is prior to the commit for ESMTPSenderFactory http://twistedmatrix.com/pipermail/twisted-commits/2004-July/011608.html Thanks -- Alvin

4 5

Re : Re : [Twisted-Python] Problem with ftp upload
by Stéphane Brault 03 Nov '06

03 Nov '06

Thanks Thomas, indeed I mentioned the wrong ticket number, my mistake, I'm already getting flogged for that ;-) I'd be glad to get your FTP access class. Stéphane ----- Message d'origine ---- De : Thomas Jacob <jacob(a)internet24.de> À : Twisted general discussion <twisted-python(a)twistedmatrix.com> Envoyé le : Vendredi, 3 Novembre 2006, 11h52mn 08s Objet : Re: Re : [Twisted-Python] Problem with ftp upload On Fri, 2006-11-03 at 09:54 +0000, Stéphane Brault wrote: > Thanks Thomas, > > my problem is that I already applied the patch and it still doesn't work. That's what I don't understand. You were mentioning a different ticket number... Anyway, you seem to be waiting for the first storeFile deferred (Consumer) to indicate successful transmission, I'm using the second (Control-Connection), maybe you simply quit to early. If you want, I can mail you my FTP access class for uploading and downloading.... > Stéphane > > > ----- Message d'origine ---- > De : Thomas Jacob <jacob(a)internet24.de> > À : Twisted general discussion <twisted-python(a)twistedmatrix.com> > Envoyé le : Vendredi, 3 Novembre 2006, 10h32mn 52s > Objet : Re: [Twisted-Python] Problem with ftp upload > > There is bug in storeFile's Consumer interface > (Cf. report + patch http://twistedmatrix.com/trac/ticket/1794 ) > > As for disconnecting, why not call ftpClient->quit() after > your upload has finished? > > On Thu, 2006-11-02 at 20:57 +0000, Stéphane Brault wrote: > > Hi, > > > > I'm using twisted ftp api to upload files, I also added the ticket 1914 patch. > > Here is my code : > > class myFTP(object): > > > > def __init__(self, login, password): > > self.creator = ClientCreator(reactor, FTPClient, login, password, passive=0) > > > > > > def connectionFailed(self, f): > > print 'connection error' > > > > > > def success(self, response): > > print 'Success' > > > > > > def fail(self, error): > > print 'Erreur:' + error > > > > > > def doOperation(self, operation, data): > > d = self.creator.connectTCP(FTP_ADDRESS, 21) > > if operation == 'addItems': > > return d.addCallback(self.inventoryUpload, data).addErrback(self.connectionFailed) > > elif .... > > > > > > > > def _upload(self, consumer, file): > > return FileSender().beginFileTransfer(file, consumer).addCallback(lambda _: consumer.finish()).addCallback(lambda _: file.close()) > > > > > > def inventoryUpload(self, ftpClient, file): > > name = '/stock/entrees/export_items_' + encodeDate(datetime.datetime.now()) + '.txt' > > return ftpClient.storeFile(name)[0].addCallback(self._upload, file).addCallbacks(self.success, self.fail) > > > > .... > > > > The operation is called this way: > > call = myFTP(login, password) > > call.doOperation('addItems', StringIO(data)) > > > > The file is created but empty, it seems that _upload is not called, I can't understand why, any hint ? > > I was also wondering how to disconnect once my work is done, I don't think keeping open connections is a good thing. > > > > Thanks, > > > > Stéphane > > > > > > > > > > > > > > > > > > ___________________________________________________________________________ > > Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! > > Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses > > http://fr.answers.yahoo.com > > > > _______________________________________________ > > Twisted-Python mailing list > > Twisted-Python(a)twistedmatrix.com > > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > > > > > > > > ___________________________________________________________________________ > Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! > Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses > http://fr.answers.yahoo.com > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___________________________________________________________________________ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com

1 0

Re : [Twisted-Python] Problem with ftp upload
by Stéphane Brault 03 Nov '06

03 Nov '06

Thanks Thomas, my problem is that I already applied the patch and it still doesn't work. That's what I don't understand. Stéphane ----- Message d'origine ---- De : Thomas Jacob <jacob(a)internet24.de> À : Twisted general discussion <twisted-python(a)twistedmatrix.com> Envoyé le : Vendredi, 3 Novembre 2006, 10h32mn 52s Objet : Re: [Twisted-Python] Problem with ftp upload There is bug in storeFile's Consumer interface (Cf. report + patch http://twistedmatrix.com/trac/ticket/1794 ) As for disconnecting, why not call ftpClient->quit() after your upload has finished? On Thu, 2006-11-02 at 20:57 +0000, Stéphane Brault wrote: > Hi, > > I'm using twisted ftp api to upload files, I also added the ticket 1914 patch. > Here is my code : > class myFTP(object): > > def __init__(self, login, password): > self.creator = ClientCreator(reactor, FTPClient, login, password, passive=0) > > > def connectionFailed(self, f): > print 'connection error' > > > def success(self, response): > print 'Success' > > > def fail(self, error): > print 'Erreur:' + error > > > def doOperation(self, operation, data): > d = self.creator.connectTCP(FTP_ADDRESS, 21) > if operation == 'addItems': > return d.addCallback(self.inventoryUpload, data).addErrback(self.connectionFailed) > elif .... > > > > def _upload(self, consumer, file): > return FileSender().beginFileTransfer(file, consumer).addCallback(lambda _: consumer.finish()).addCallback(lambda _: file.close()) > > > def inventoryUpload(self, ftpClient, file): > name = '/stock/entrees/export_items_' + encodeDate(datetime.datetime.now()) + '.txt' > return ftpClient.storeFile(name)[0].addCallback(self._upload, file).addCallbacks(self.success, self.fail) > > .... > > The operation is called this way: > call = myFTP(login, password) > call.doOperation('addItems', StringIO(data)) > > The file is created but empty, it seems that _upload is not called, I can't understand why, any hint ? > I was also wondering how to disconnect once my work is done, I don't think keeping open connections is a good thing. > > Thanks, > > Stéphane > > > > > > > > > ___________________________________________________________________________ > Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! > Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses > http://fr.answers.yahoo.com > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___________________________________________________________________________ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com

2 1

[Twisted-Python] Problem with ftp upload
by Stéphane Brault 03 Nov '06

03 Nov '06

Hi, I'm using twisted ftp api to upload files, I also added the ticket 1914 patch. Here is my code : class myFTP(object): def __init__(self, login, password): self.creator = ClientCreator(reactor, FTPClient, login, password, passive=0) def connectionFailed(self, f): print 'connection error' def success(self, response): print 'Success' def fail(self, error): print 'Erreur:' + error def doOperation(self, operation, data): d = self.creator.connectTCP(FTP_ADDRESS, 21) if operation == 'addItems': return d.addCallback(self.inventoryUpload, data).addErrback(self.connectionFailed) elif .... def _upload(self, consumer, file): return FileSender().beginFileTransfer(file, consumer).addCallback(lambda _: consumer.finish()).addCallback(lambda _: file.close()) def inventoryUpload(self, ftpClient, file): name = '/stock/entrees/export_items_' + encodeDate(datetime.datetime.now()) + '.txt' return ftpClient.storeFile(name)[0].addCallback(self._upload, file).addCallbacks(self.success, self.fail) .... The operation is called this way: call = myFTP(login, password) call.doOperation('addItems', StringIO(data)) The file is created but empty, it seems that _upload is not called, I can't understand why, any hint ? I was also wondering how to disconnect once my work is done, I don't think keeping open connections is a good thing. Thanks, Stéphane ___________________________________________________________________________ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com

2 1

[Twisted-Python] manhole questions
by Alec Matusis 03 Nov '06

03 Nov '06

Hello I am using manhole with Twisted 2.2 like so: manhole_factory = twisted.manhole.telnet.ShellFactory() manhole_factory.username = config.manhole_user manhole_factory.password = config.manhole_pwd manhole_factory.service = t_service manhole_serv=twisted.application.internet.TCPServer(common.config.t_manhole_ port, manhole_factory).setServiceParent(application) I have two problems: 1) Backspace and Arrow keys do not work with telnet. How can I fix that? 2) When I make a mistake and cause an error in the interpreter, the Traceback gets logged into the application's main log file, thus polluting it. How can I prevent these errors in manhole's telnet from being logged into the application's main log? Additional question: 3) When I maintain a telnet session into manhole in a production server, how does it affect its performance/stability ? Thanks Alec.

1 0

[Twisted-Python] Strange error when using strptime within a nevow application
by Franz Zieher 01 Nov '06

01 Nov '06

I've experiencing a very strange error when using strptime for populating a 'formal' form. If I run the code segment with strptime in ipython it can succesfully transform '12-10-06' with '%d-%m-%Y' into a time tuple! With nevow I get the following exception: ---------------------------------------------------------- 2006/11/01 01:18 W. Europe Standard Time [-] Log opened. 2006/11/01 01:18 W. Europe Standard Time [-] twistd 2.4.0(D:\Python24\python.exe 2.4.3) starting up 2006/11/01 01:18 W. Europe Standard Time [-] reactor class: <class ' twisted.internet.selectreactor.SelectReactor'> 2006/11/01 01:18 W. Europe Standard Time [-] nevow.appserver.NevowSitestarting on 8000 2006/11/01 01:18 W. Europe Standard Time [-] Starting factory < nevow.appserver.NevowSite instance at 0x02797A30> 2006/11/01 01:18 W. Europe Standard Time [HTTPChannel,0,127.0.0.1] 127.0.0.1- - [01/Nov/2006:00:18:24 +0000] "GET / HT P/1.1" 200 7872 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1) Gecko/20061010 Firefox/2.0" 2006/11/01 01:18 W. Europe Standard Time [HTTPChannel,0,127.0.0.1] (NAME,DIR): ('newChild', None) 2006/11/01 01:18 W. Europe Standard Time [HTTPChannel,0,127.0.0.1] IRemainingSegments: ('newChild',) 2006/11/01 01:18 W. Europe Standard Time [HTTPChannel,0,127.0.0.1] Unhandled error in Deferred: 2006/11/01 01:18 W. Europe Standard Time [HTTPChannel,0,127.0.0.1] Traceback (most recent call last): File "d:\python24\lib\site-packages\twisted\internet\defer.py", line 307, in _runCallbacks self.result = callback(self.result, *args, **kw) File "build\bdist.win32\egg\formal\form.py", line 617, in locateForm File "d:\python24\lib\site-packages\twisted\internet\defer.py", line 191, in addCallback callbackKeywords=kw) File "d:\python24\lib\site-packages\twisted\internet\defer.py", line 182, in addCallbacks self._runCallbacks() --- <exception caught here> --- File "d:\python24\lib\site-packages\twisted\internet\defer.py", line 307, in _runCallbacks self.result = callback(self.result, *args, **kw) File "build\bdist.win32\egg\formal\form.py", line 617, in <lambda> File "build\bdist.win32\egg\formal\form.py", line 567, in formFactory File "e:\data\iCAE\python\icaeForms.py", line 92, in form_instance eval('self.%s(form)' % tag) File "<string>", line 0, in ? File "e:\data\iCAE\python\icaeForms.py", line 128, in project print strptime('10-12-06','%d-%m-%Y')[0] File "D:\Python24\lib\_strptime.py", line 269, in ? _TimeRE_cache = TimeRE() File "D:\Python24\lib\_strptime.py", line 188, in __init__ self.locale_time = LocaleTime() File "D:\Python24\lib\_strptime.py", line 74, in __init__ self.__calc_weekday() File "D:\Python24\lib\_strptime.py", line 94, in __calc_weekday a_weekday = [calendar.day_abbr[i].lower() for i in range(7)] exceptions.AttributeError: 'module' object has no attribute 'day_abbr' Any idea what that could be? Thanks Franz

1 0

[Twisted-Python] Wrapping my mind around Interfaces, adapters, factories, protocols
by Ron Arts 01 Nov '06

01 Nov '06

Hi, I am trying to accomplish the following: - have a client connecting to a remote service and collecting information from it - have a service that can be queried for the collected info But I am getting totally confused with all the interface classes, adapters, factories, protocols I should write according to the finger tutorial. I skimmed the mailinglist, but to no avail. My app is laid out almost like the finger tutorial (I left out the web, and PB parts for now), the biggest difference being that the client data should be available to the service that is being queried. So I thought that instead of first create the finger service in the makeService() method, I would first create a TCP client, and pass that to a service to be created later. something like: def makeService(config): s = service.MultiService() m = MyClient() m.username = 'username' m.password = 'password' theserver = '192.168.1.2' b = internet.TCPClient(theserver, 6666, IMyClientFactory(m)) b.setServiceParent(s) return s But for this to work I have to create interfacethis, protocol that, and use components.registerAdapter(), to do something I totally don't understand what it does and why it needs to be done, I have read the finger example many times, but I can't seem to wrap my mind around these concepts, in the little time I have available. Can anybody whip me up a supershort example in pseudo code of what I need to do to make the above makeService work? And explain it a bit? Many thanks, Ron

1 0