Mailman 3 March 2003 - Twisted

[Twisted-Python] Waitable Timers and Events
by X March 6, 2003

March 6, 2003

I have what I think is a unique issue and hoping someone might be able to offer a suggestion. PROBLEM: Can't get Do/Loop and Timer/Loop to Exit I have a static counter in the PAINT event to set up a trigger to call a Do/Loop. This is done to allow the screen to be painted to a certain point and then the loop is designed to finish painting the screen in slow motion. Once the Do/Loop is entered it is controlled (slow motion) by calling the Timer/Loop routine which uses SetWaitableTimer and … [View More]

1 0

[Twisted-Python] Sample Twisted Web NT Service
by Thad Jacobs March 5, 2003

March 5, 2003

#This sample code demonstrates how to mount Twisted Web as a #Windows XP / 2000 / NT Service using win32all's win32service import win32serviceutil import win32service import win32event class TwistedWebNTService(win32serviceutil.ServiceFramework): _svc_name_ = "TwistedWebNTService" _svc_display_name_ = "Twisted Web NT Service" def __init__(self, args): win32serviceutil.ServiceFramework.__init__(self, args) self.hWaitStop = win32event.CreateEvent(None, 0, 0, None) def … [View More]

1 0

RE: [Twisted-Python] NT Service
by Thad Jacobs March 5, 2003

March 5, 2003

Works like a charm!! Thanks, Donovan -----Original Message----- From: Donovan Preston [mailto:dp@twistedmatrix.com] Sent: Wednesday, March 05, 2003 1:19 PM To: twisted-python(a)twistedmatrix.com Subject: Re: [Twisted-Python] NT Service On Wednesday, March 5, 2003, at 01:01 PM, Thad Jacobs wrote: [ snip snip ] > def startTwisted(): > from twisted.internet.app import Application > > #How do I specify a port or load a tap file? > app = Application("web") # This … [View More]

1 0

RE: [Twisted-Python] NT Service
by Thad Jacobs March 5, 2003

March 5, 2003

Glyph, I've got the service up and running, but it doesn't do anything. All I want to do is make a twisted web application that listens on port 8080 and serves files out of c:\web -- or better yet, it would be nice if I could simply instantiate the web.tap file just as I would using "twistd -f web.tap". I'm willing to get under the hood and make this happen, but unfortunately I'm under a bit of a time constraint. I've been RTFM'ing all morning, but the documentation is scattered, and I am … [View More]a bit green in the realms of asynchronous server programming, and have been coding in python for less than a week. Could you advise me as to what direction to take in instantiating a twisted web application from within the service? I was thinking of simply importing twisted.py, but I can't figure out how to make it run with options I dictate programatically, than the system command line switches. Once I've figured this out, I'll be sure to post the full source, so others may benefit. Here's the basic service, almost Identical to the one Bob posted, though I had to import twisted.internet.reactor to get it to work. import win32serviceutil import win32service import win32event class KAQueryService(win32serviceutil.ServiceFramework): _svc_name_ = "KAQueryService" _svc_display_name_ = "Kinematic Query Service" def __init__(self, args): win32serviceutil.ServiceFramework.__init__(self, args) self.hWaitStop = win32event.CreateEvent(None, 0, 0, None) def SvcStop(self): self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING) win32event.SetEvent(self.hWaitStop) def SvcDoRun(self): from twisted.python.log import startLogging # we don't have the console, so we log somewhere # the NT service logging facilities probably isn't the right place startLogging(open('c:/twistedexample.log','a')) self.CheckForQuit() startTwisted() def CheckForQuit(self): from twisted.internet import reactor #print "Check For Quit" rv = win32event.WaitForSingleObject(self.hWaitStop, 10.0) if not rv == win32event.WAIT_TIMEOUT: print "Received Quit from Win32" reactor.stop() reactor.callLater(1.0, self.CheckForQuit) def startTwisted(): from twisted.internet.app import Application #How do I specify a port or load a tap file? app = Application("web") app.run(save=0) print "Finished" if __name__=='__main__': # you could call startTwisted to run as not-a-service # or use it for cross-platform reasons win32serviceutil.HandleCommandLine(KAQueryService) -----Original Message----- From: Glyph Lefkowitz [mailto:glyph@twistedmatrix.com] Sent: Wednesday, March 05, 2003 11:15 AM To: twisted-python(a)twistedmatrix.com Subject: Re: [Twisted-Python] NT Service -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, March 5, 2003, at 12:41 PM, Bob Ippolito wrote: > I'm assuming you've already RTFM'ed about win32all's win32service and > Twisted. Here's a stripped example of what has worked very well for > me for the few times I was forced to deploy under Windows: It would be really great if someone could genericize this and create a version that had a configuration dialog that allowed you to select a particular .tap file to load, rather than instantiating the Application inline. That way deploying on Win32 would be much more directly analagous to deploying on unix. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (Darwin) iD8DBQE+Zky+vVGR4uSOE2wRAqdLAJ9+KogR9IgvsEwlyhVJT4FQhBQVlQCfdkOL c7FtjeYrzf8H0fZuNz6mGi4= =oWYP -----END PGP SIGNATURE----- _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python [View Less]

2 2

[Twisted-Python] Re: [Twisted-commits] mktap now accepts --uid=0 and --gid=0 to really mean root, has command
by Andrew Bennetts March 5, 2003

March 5, 2003

On Tue, Mar 04, 2003 at 12:32:28PM -0600, tv CVS wrote: > Modified files: > Twisted/twisted/scripts/mktap.py 1.26 1.27 > Twisted/twisted/internet/app.py 1.79 1.80 > Twisted/ChangeLog 1.279 1.280 > > Log message: > mktap now accepts --uid=0 and --gid=0 to really mean root, has command > line help for --uid=/--gid=, and understands user and group names in > addition to numbers. [..snip..] > """ Implementation module for the `mktap` command. > """ > @@ -25,… [View More]

5 4

[Twisted-Python] NT Service
by Thad Jacobs March 5, 2003

March 5, 2003

Hello All, I've created a module that makes COM calls to our non-sql database using win32com, and converts the results to XML format (through simple string concatenations). I've created a rpy script that calls this module, and returns the XML to the browser. I am currently running the HTTP server per the introduction using the command line. My question: How should I best deploy this application into a production environment? It will be running on an Windows 2000 server, and needs to … [View More]

3 2

[Twisted-Python] A fun site using Twisted
by Itamar Shtull-Trauring March 5, 2003

March 5, 2003

http://mysteryrobot.com/ Pointed out by someone on the IRC channel, I forget who.

1 0

[Twisted-Python] patch implementing "fetchmany" from enterprise.adbapi
by Clark C. Evans March 4, 2003

March 4, 2003

Please find following a patch to add the ability for fetchmany() to be used from within the enterprise adbapi.py interface. This uses the patches previously posted (and included following) which enable an iterator to be deferred via a thread. Basically, one can use runQueryChunked instead of runQuery and it will call your callback once for each "fetchmany" return, which, depending on the database driver can be about 10-50 rows at a time. This is essential for incremental handling of large … [View More]query results. The patch also tests for "threadsaftey", for some reason mxODBC has an apilevel of "2.0" yet they are missing this attribute (they call it threadlevel). I'm not fond of this part of the patch, but it is included since this is what I tested with (I don't have permissions to change mxODBC). The remainder of the message is hereby public domain. Best, Clark --- adbapi.py.orig Fri Feb 28 16:51:08 2003 +++ adbapi.py Tue Mar 4 02:01:36 2003 @@ -52,7 +52,9 @@ log.msg("Connecting to database: %s %s %s" % (dbapiName, connargs, connkw)) self.dbapi = reflect.namedModule(dbapiName) assert self.dbapi.apilevel == '2.0', 'DB API module not DB API 2.0 compliant.' - assert self.dbapi.threadsafety > 0, 'DB API module not sufficiently thread-safe.' + if hasattr(self.dbapi,"threadsaftey"): + test = self.dbapi.threadsaftey > 0 + assert test, 'DB API module not sufficiently thread-safe.' self.connargs = connargs self.connkw = connkw import thread @@ -99,6 +101,21 @@ curs.close() return result + def _runQueryChunked(self, args, kw): + conn = self.connect() + curs = conn.cursor() + apply(curs.execute, args, kw) + class fetchChunk: + def __init__(self,curs): + self.curs = curs + def next(self): + ret = self.curs.fetchmany() + if not ret: + self.curs.close() + raise StopIteration + return ret + return fetchChunk(curs) + def _runOperation(self, args, kw): """This is used for non-query operations that don't want "fetch*" to be called """ @@ -121,6 +138,10 @@ threads.deferToThread(self._runQuery, args, kw).addCallbacks( callback, errback) + def queryChunked(self, callback, errback, *args, **kw): + threads.deferIterationToThread(self._runQueryChunked, args, kw + ).addCallbacks(callback, errback) + def operation(self, callback, errback, *args, **kw): threads.deferToThread(self._runOperation, args, kw).addCallbacks( callback, errback) @@ -229,6 +250,11 @@ apply(self.dbpool.query, (d.callback, d.errback)+args, kw) return d + def runQueryChunked(self, *args, **kw): + d = defer.MultiCallDeferred() + apply(self.dbpool.queryChunked, (d.callback, d.errback)+args, kw) + return d + def runOperation(self, *args, **kw): d = defer.Deferred() apply(self.dbpool.operation, (d.callback,d.errback)+args, kw) ----- Forwarded message from "Clark C. Evans" <cce(a)clarkevans.com> ----- From: "Clark C. Evans" <cce(a)clarkevans.com> To: twisted-python(a)twistedmatrix.com Subject: [Twisted-Python] patch implementing deferIterationToThread Date: Tue, 4 Mar 2003 05:46:48 +0000 Please find a patch to internet/defer.py and iternet/threads.py which enables an "iterator" to be deferred to a thread. This is useful since often you have a long running process (a database query for example) which you'd like to return results every once and a while rather than delivering all of the output at the end of the process. This patch adds to the given files (it doesn't change any code). The remainder of the message is hereby public domain. # # code to append to defer.py # class MultiCallDeferred(Deferred): """This is a verision of Deferred which can be invoked more than once. This is accomplished by cloning the current deferred object and carrying out the callbacks on the clone. """ def _startRunCallbacks(self, result, isError): clone = Deferred() clone.default = self.default for x in self.callbacks: clone.callbacks.append(x) clone._startRunCallbacks(result, isError) # # code to be added to threads.py # def _putIterationInDeferred(deferred, f, args, kwargs): """Send the results of an iteration to a deferred. The function called should return an object with a next() operator. This is the ideal mechanism to defer a generator. """ from twisted.internet import reactor try: itr = apply(f, args, kwargs) while 1: reactor.callFromThread(deferred.callback, itr.next()) except StopIteration: pass except: f = failure.Failure() reactor.callFromThread(deferred.errback, f) def deferIterationToThread(f, *args, **kwargs): """Run the results of an iterator in a thread.""" d = defer.MultiCallDeferred() reactor.callInThread(_putIterationInDeferred, d, f, args, kwargs) return d # # usage using iterators (or generators for simpler syntax) # from twisted.internet.threads import deferIterationToThread from twisted.internet import reactor class producer: def __init__(self): self.val = 9 def next(self): val = self.val if val < 1: raise StopIteration self.val -= 1 return val def bldr(): return producer() def printResult(x): print x d = deferIterationToThread(bldr) d.addCallback(printResult) try: # if you have generators from __future__ import generators def gene(start=99): while(start > 90): yield start start -= 1 d = deferIterationToThread(gene) d.addCallback(printResult) except: pass reactor.run() _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ----- End forwarded message ----- [View Less]

4 6

[Twisted-Python] fetchmany and deferreds which call back more than once
by Clark C. Evans March 4, 2003

March 4, 2003

I have a result set with potentially 100's (1000's or more) of rows from a database that I need to stream back to the client after HTML formatting. Anyway, I've looked at the enterprise adapter, and it only returns "once" for a given query with the return value of fetchall() of the cursor. While this is interesting, what I'd really like is a way to pass "chunks" of the result back via fetchmany(). I'm kinda stumped as to where to begin, I've gone through much of the database adapter and … [View More]

3 4

[Twisted-Python] Clearer information about virtual hosts
by Matthew J. Probst March 4, 2003

March 4, 2003

I'm quite familiar with Python, but I'm having a hard time wrapping my mind round Twisted documentation. I suspect that the given instructions aren't as useful until one understands a critical mass of the object models used, and that I haven't reached that point yet. I've been trying to set up independent named virtual hosts serving static HTML from different directories. I can't figure out how this can be made to work. As a baseline I'm using the howto located at http://www.… [View More]

1 0