Mailman 3 April 2003 - Twisted

[Twisted-Python] Re: [Twisted-commits] better password handling, also catch SystemExit and don't print it
by Kevin Turner April 18, 2003

April 18, 2003

On Thu, 2003-04-17 at 08:58, z3p CVS wrote: > Twisted/twisted/python/util.py 1.23 1.24 [...] > better password handling, also catch SystemExit and don't print it Hey, you broke my code! Fortunately, you did it after the release candidate, so I don't have to kill you. Sorry I didn't have a unit test in for it before, I do now (test_util.GetPasswordTest). The behaviour I was depending on *was* fully specified by the docstring though, which you blatantly changed. I can understand why … [View More]

1 0

Re: [Twisted-Python] Re: feedback requested -- flow.py goes beta
by Clark C. Evans April 18, 2003

April 18, 2003

Ok. Thanks to extrepum's suggestions, flow can now use the for loop. Also, it handles exceptions in a cleaner way. For more details, see http://clarkevans.com/flow.html flow.py, test_flow.py, and flow.html are currently checked into the Twisted CVS. Best, Clark On Fri, Apr 18, 2003 at 04:55:47PM +0200, Philippe Lafoucri�re wrote: | > producer = flow.wrap([1,2,3]) | > yield producer | > for result in producer: | > // handle success | > yield … [View More]

1 0

[Twisted-Python] blocking question
by Konrad Rokicki April 18, 2003

April 18, 2003

Hey everyone, newbie question: Are there any Python/Twisted idioms for doing blocking? More specifically, in a client I have a method to send a query to the server and another method that gets called when the query is answered. Is there an easy way to say "send the query then wait x seconds for it to return in this function"? It seems like blocking is taboo in twisted and if anyone has an alternative I'd be glad to hear it. I guess I could do this asynchronically but it would be a pain I think. Thanks, Konrad Rokicki

4 3

[Twisted-Python] fwd: Accepted twisted 1.0.4-1 (i386 source all)
by Moshe Zadka April 18, 2003

April 18, 2003

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Format: 1.7 Date: Fri, 18 Apr 2003 03:46:22 +0000 Source: twisted Binary: python2.3-twisted-conch python2.2-twisted-conch python2.1-twisted-conch python2.1-twisted python-twisted-conch twisted-quotes python2.1-twisted-bin twisted twisted-doc-api python2.3-twisted-bin twisted-doc python2.2-twisted python2.3-twisted python2.2-twisted-bin python-twisted Architecture: source all i386 Version: 1.0.4-1 Distribution: unstable Urgency: low Maintainer: … [View More]Moshe Zadka <moshez(a)debian.org> Changed-By: Moshe Zadka <moshez(a)debian.org> Description: python-twisted - Event-based framework for internet applications. (dummy package) python-twisted-conch - The Twisted SSH Implementation (default python version) python2.1-twisted - Event-based framework for internet applications. (Python 2.1) python2.1-twisted-bin - Event-based framework for internet applications python2.1-twisted-conch - The Twisted SSH Implementation (for Python 2.1) python2.2-twisted - Event-based framework for internet applications. python2.2-twisted-bin - Event-based framework for internet applications python2.2-twisted-conch - The Twisted SSH Implementation (for Python 2.2) python2.3-twisted - Event-based framework for internet applications. python2.3-twisted-bin - Event-based framework for internet applications python2.3-twisted-conch - The Twisted SSH Implementation (for Python 2.3) twisted - Event-based framework for internet applications. (transitional) twisted-doc - The official documentation of Twisted. twisted-doc-api - The auto-generated API docs of Twisted twisted-quotes - Quotes from the Twisted development team Changes: twisted (1.0.4-1) unstable; urgency=low . * New upstream version Files: e4ecff7777718f3151b87b626567fa9c 888 net extra twisted_1.0.4-1.dsc d47fc0f316d56af8dd8d513aac4422c0 6023353 net extra twisted_1.0.4.orig.tar.gz 79dc06a577cb2964ce7391a25d7adb88 214 net extra twisted_1.0.4-1.diff.gz 5e349bdf0c7924aa6989adaba389276b 4130 net extra twisted_1.0.4-1_all.deb 04fbfd9babd2d4d993074f9a26d30753 4262 net extra python-twisted_1.0.4-1_all.deb 3bb92f9a4b76e77182f0070526e2eb15 4116 net extra python-twisted-conch_1.0.4-1_all.deb 19193ebf29efa1af348690dc10de0a6b 1440174 net extra twisted-doc_1.0.4-1_all.deb 4b8c8be42548c9923fa1e0b58e064a1c 71724 net extra twisted-quotes_1.0.4-1_all.deb 7b5493effac4d05a86b22b522c193413 3123752 net extra twisted-doc-api_1.0.4-1_all.deb 78e97d259d3e914d63b0850480fc0649 791150 net extra python2.1-twisted_1.0.4-1_all.deb 2b8d9506910a4be2e7a0920fdf62618d 791662 net extra python2.2-twisted_1.0.4-1_all.deb 4735448dec72c541b6a579a43c00df0d 791146 net extra python2.3-twisted_1.0.4-1_all.deb a7d81d65fec0be8181a54432047f8eb7 56048 net extra python2.1-twisted-conch_1.0.4-1_all.deb d52fedad6cde1170d95c9a156c1e5c8f 56208 net extra python2.2-twisted-conch_1.0.4-1_all.deb 5d6419958693ec2252d384bce8a4e76a 56048 net extra python2.3-twisted-conch_1.0.4-1_all.deb 6a31706821867ef2edf438371a33f0fe 27504 net extra python2.1-twisted-bin_1.0.4-1_i386.deb a045902b3e7af2ff7d608ef116c59937 28590 net extra python2.2-twisted-bin_1.0.4-1_i386.deb 054f8bbdb7025d86057e6d6c5800f8d3 28580 net extra python2.3-twisted-bin_1.0.4-1_i386.deb -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE+n06Xx/ymNkbQG9YRAgCKAJ40YNAlUFd7ozHtcHi0rwDROhPG8gCcDeDb kImjwmnMAjq1BWm40p+qw6I= =eqFP -----END PGP SIGNATURE----- Accepted: python-twisted-conch_1.0.4-1_all.deb to pool/main/t/twisted/python-twisted-conch_1.0.4-1_all.deb python-twisted_1.0.4-1_all.deb to pool/main/t/twisted/python-twisted_1.0.4-1_all.deb python2.1-twisted-bin_1.0.4-1_i386.deb to pool/main/t/twisted/python2.1-twisted-bin_1.0.4-1_i386.deb python2.1-twisted-conch_1.0.4-1_all.deb to pool/main/t/twisted/python2.1-twisted-conch_1.0.4-1_all.deb python2.1-twisted_1.0.4-1_all.deb to pool/main/t/twisted/python2.1-twisted_1.0.4-1_all.deb python2.2-twisted-bin_1.0.4-1_i386.deb to pool/main/t/twisted/python2.2-twisted-bin_1.0.4-1_i386.deb python2.2-twisted-conch_1.0.4-1_all.deb to pool/main/t/twisted/python2.2-twisted-conch_1.0.4-1_all.deb python2.2-twisted_1.0.4-1_all.deb to pool/main/t/twisted/python2.2-twisted_1.0.4-1_all.deb python2.3-twisted-bin_1.0.4-1_i386.deb to pool/main/t/twisted/python2.3-twisted-bin_1.0.4-1_i386.deb python2.3-twisted-conch_1.0.4-1_all.deb to pool/main/t/twisted/python2.3-twisted-conch_1.0.4-1_all.deb python2.3-twisted_1.0.4-1_all.deb to pool/main/t/twisted/python2.3-twisted_1.0.4-1_all.deb twisted-doc-api_1.0.4-1_all.deb to pool/main/t/twisted/twisted-doc-api_1.0.4-1_all.deb twisted-doc_1.0.4-1_all.deb to pool/main/t/twisted/twisted-doc_1.0.4-1_all.deb twisted-quotes_1.0.4-1_all.deb to pool/main/t/twisted/twisted-quotes_1.0.4-1_all.deb twisted_1.0.4-1.diff.gz to pool/main/t/twisted/twisted_1.0.4-1.diff.gz twisted_1.0.4-1.dsc to pool/main/t/twisted/twisted_1.0.4-1.dsc twisted_1.0.4-1_all.deb to pool/main/t/twisted/twisted_1.0.4-1_all.deb twisted_1.0.4.orig.tar.gz to pool/main/t/twisted/twisted_1.0.4.orig.tar.gz -- Moshe Zadka -- http://moshez.org/ Buffy: I don't like you hanging out with someone that... short. Riley: Yeah, a lot of young people nowadays are experimenting with shortness. Agile Programming Language -- http://www.python.org/ [View Less]

1 0

[Twisted-Python] c++ style comments in c extension modules
by Jon Dyte April 17, 2003

April 17, 2003

When building twisted 1.0.3 on solaris using the sun 'cc' compiler, several of the c extension modules fail because of the prescence of '//' for a comment in the code. grep -n "\/\/" `find . -name "*.c" -print` ./twisted/internet/cReactor/cDelayedCall.c:126: // self.time += secondsLater ./twisted/internet/cReactor/cDelayedCall.c:170: // self.time = time() + secondsFromNow ./twisted/internet/cReactor/cReactorTCP.c:159: char buf[3*20+3+1+100]; // for good measure ./twisted/internet/… [View More]

3 2

[Twisted-Python] Bloat (was Re: [Twisted-commits] DependentMultiService - chained start/stop of services in a sensible order)
by Glyph Lefkowitz April 17, 2003

April 17, 2003

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This patch strikes me as unnecessary: On Sunday, April 13, 2003, at 03:48 PM, etrepum CVS wrote: > Modified files: > Twisted/twisted/internet/app.py 1.89 1.90 > > Log message: > DependentMultiService - chained start/stop of services in a sensible > order but I would like to discuss it and other patches like it. Not to pick on bob here; there isn't really a policy on avoiding bloat in Twisted, so it can't be said that this … [View More]

6 13

[Twisted-Python] Webware/Twisted experiment
by Ian Bicking April 17, 2003

April 17, 2003

In the webware-sandbox CVS (webware-sandbox.sf.net), in Sandbox.ianbicking.TwistedWebware, I've put some code to run Webware under Twisted. This mostly subsumes the Webware AppServer, with Twisted handling the threads and sending requests directly to Application.dispatchRawRequest. There's certainly some cludges. There's a minor change to MiscUtils.Funcs.hostName to accomodate Twisted, so the CVS version of that file is required. It's an experiment, because it doesn't actually offer … [View More]

2 1

[Twisted-Python] Interfaces
by Jp Calderone April 17, 2003

April 17, 2003

I went to remove the interface inheritance done in twisted.internet.interfaces tonight, thinking it was all my fault. To my surprise, IReadDescriptor and IWriteDescriptor both use it as well though. So what's the story on this? Is it kosher? Next: There is no IPerspective, and I think I want one. What I actually *really* want is a way to specify that any class that implements a particular interface of mine also be a Perspective. If interface inheritance is okay, I can just make my … [View More]

5 5

[Twisted-Python] Woven b0rkage
by Thomas Raschbacher April 17, 2003

April 17, 2003

hi! woven b0rks here when i got an index.rpy which uses woven .. it does endless redirects ... below is just a patch to comment it (whcih works perfectly here..) regards --- /usr/portage/distfiles/cvs-src/Twisted/twisted/web/woven/controller.py 2003-04-14 20:36:49.000000000 +0200 +++ /usr/lib/python2.2/site-packages/twisted/web/woven/controller.py 2003-04-17 14:47:58.000000000 +0200 @@ -183,8 +183,8 @@ DOM. gatheredControllers will render the page to the browser … [View More]

1 0

[Twisted-Python] rewrite of flow.py completed
by Clark C. Evans April 17, 2003

April 17, 2003

Ok. The complete re-write of flow.py based on the work of extrepum ("Bob Ippolito") is finished, with a test case checked in. This differs from extrepum's work [1] in some ways, but is almost identical in others. 1. The implementation and test suite does not use generators, thus it is safe for 2.1 usage... although I can't imagine anyone using it without generators. But the advantage is that it could go into the current Twisted code base. The test suite uses iterators (… [View More]with the corresponding generator commented out). 2. This implementation has 4 distinct components: a) the core 'Flow' class and its supporting items, could be put into something like twisted.python.flow as they only depend upon other twisted.python stuff. b) There is a DeferredFlow which uses Flow as a mix-in with Deferred, with behavior similar to DeferredList. This could go into twisted.internet.defer c) There is a ThreadedIterator which depends upon DeferredFlow, and can be used to merge blocking behavior into the framework. This could go into twisted.internet.threads d) There is a very small class which builds upon ThreadedIterator that uses a pool from twisted.enterprise.adbapi; however, in this context the async features of adbapi more or less get in the way. So, perhaps this is best left as an example. In particular, extrepum's work build on top of Deferred, which I think is unnecessary; thus quite a bit of simplification occurred when stripping out the Deferred stuff, and instead treating DeferredFlow as a cross product or mix-in of Flow and Deferred. 3. Extrepum's work was more 'flat' and didn't take recursive use of the technique into consideration; or perhaps I just didn't understand how it would handle recursive usage. In any case, this implementation allows generators to call other generators and provides a nice stack unwinding, etc. 4. I'd like to eventually look more at extrepum's work to see if he had additional aspects that I've missed. If those aspects are generally useable, then I'll include them. Anyway, that's about it. I'm now rebuilding some of my internal code to use this module and ripping the old flow.py out. This flow overall has been *greatly* simpler usage -- and I can't thank extrepum enough for pointing me to his usage of generators to create async flows within Twisted. I'd also like to thank Itamar, Moshez, Radix, and Exarkun for helping me along and pointing me in the right directions. Best, Clark [1] twistedmatrix.com/pipermail/twisted-python/2003-February/002808.html # Twisted, the Framework of Your Internet # Copyright (C) 2003 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General # Public License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 # USA # # """ Flow -- async data flow This module provides a mechanism for using async data flows through the use of generators. While this module does not use generators in its implementation, it isn't very useable without them. A data flow is constructed with a top level generator, which can have three types of yield statements: flow.Cooperate, flow.Generator, or any other return value with exceptions wrapped using failure.Failure An example program... from __future__ import generators import flow def producer(): lst = flow.Generator([1,2,3]) nam = flow.Generator(['one','two','three']) while 1: yield lst; yield nam if lst.stop or nam.stop: return yield (lst.result, nam.result) def consumer(): title = flow.Generator(['Title']) yield title print title.getResult() lst = flow.Generator(producer()) try: while 1: yield lst print lst.getResult() except flow.StopIteration: pass flow.Flow(consumer()).execute() """ from twisted.python import failure from twisted.python.compat import StopIteration, iter class FlowCommand: """ Objects given special meaning when returned from yield """ pass class Cooperate(FlowCommand): """ Represents a request to delay and let other events process Objects of this type are returned within a flow when the flow would block, or needs to sleep. This object is then used as a signal to the flow mechanism to pause and perhaps let other delayed operations to proceed. """ def __init__(self, timeout = 0): self.timeout = timeout class Generator(FlowCommand): """ Wraps a generator or other iterator for use in a flow Creates a nested generation stage (a producer) which can provide zero or more values to the current stage (the consumer). After a yield of this object when control has returned to the caller, this object will have two attributes: stop This is true if the underlying generator has not been started (a yield is needed) or if the underlying generator has raised StopIteration result This is the result of the generator if it is active, the result may be a fail.Failure object if an exception was thrown in the nested generator. """ def __init__(self, iterable): self._next = iter(iterable).next self.result = None self.stop = 1 def isFailure(self): """ return a boolean value if the result is a Failure """ if self.stop: raise StopIteration() return isinstance(self.result, failure.Failure) def getResult(self): """ return the result, or re-throw an exception on Failure """ if self.isFailure(): raise (self.result.value or self.result.type) return self.result def _generate(self): """ update the active and result member variables """ try: self.result = self._next() self.stop = 0 except StopIteration: self.stop = 1 self.result = None except Cooperate, coop: self.stop = 0 self.result = coop except failure.Failure, fail: self.stop = 1 self.result = failure except: self.stop = 1 self.result = failure.Failure() class Flow: """ A flow contruct, created with a top-level generator/iterator The iterable provided to this flow is the top-level consumer object. From within the consumer, multiple 'yield' calls can be made returning either Cooperate or Generate. If a Generate object is returned, then it becomes the current context and the process is continued. Communication from the producer back to the consumer is done by yield of a non FlowItem """ def __init__(self, iterable): self.results = [] self._stack = [Generator(iterable)] def _addResult(self, result): """ private called as top-level results are added""" self.results.append(result) def _execute(self): """ private execute, execute flow till a Cooperate is found """ while self._stack: head = self._stack[-1] head._generate() if head.stop: self._stack.pop() else: result = head.result if isinstance(result, FlowCommand): if isinstance(result, Cooperate): return result.timeout assert(isinstance(result, Generator)) self._stack.append(result) else: if len(self._stack) > 1: self._stack.pop() else: if self._addResult(result): return def execute(self): """ continually execute, using sleep for Cooperate """ from time import sleep while 1: timeout = self._execute() if timeout is None: break sleep(timeout) from twisted.internet import defer class DeferredFlow(Flow, defer.Deferred): """ a version of Flow using Twisted's reactor and Deferreds In this version, a call to execute isn't required. Instead, the iterable is scheduled right away using the reactor. And, the Cooperate is implemented through the reactor's callLater. Since more than one (possibly failing) result could be returned, this uses the same semantics as DeferredList """ def __init__(self, iterable, delay = 0, fireOnOneCallback=0, fireOnOneErrback=0): """initialize a DeferredFlow @param iterable: top level iterator / generator @param delay: delay when scheduling reactor.callLater @param fireOnOneCallback: a flag indicating that the first good yielded result should be sent via Callback @param fireOnOneErrback: a flag indicating that the first failing yield result should be sent via Errback """ from twisted.internet import reactor defer.Deferred.__init__(self) Flow.__init__(self,iterable) self.fireOnOneCallback = fireOnOneCallback self.fireOnOneErrback = fireOnOneErrback reactor.callLater(delay, self._execute) def execute(self): raise TypeError("Deferred Flow is auto-executing") def _addResult(self, result): """ emulate DeferredList behavior, short circut if event is fired """ if not self.called: if self.fireOnOneCallback: if not isinstance(result, failure.Failure): self.callback((result,len(self.results))) return 1 if self.fireOnOneErrback: if isinstance(result, failure.Failure): self.errback(fail.Failure((result,len(self.results)))) return 1 self.results.append(result) def _execute(self): timeout = Flow._execute(self) if timeout is None: if not self.called: self.callback(self.results) else: from twisted.internet import reactor reactor.callLater(timeout, self._execute) # # The following is a thread package which really is othogonal to # Flow. Flow does not depend on it, and it does not depend on Flow. # Although, if you are trying to bring the output of a thread into # a Flow, it is exactly what you want. The QueryIterator is # just an obvious application of the ThreadedIterator. # class ThreadedIterator: """ This is an iterator base class which can be used to build iterators which are constructed and run within a Flow """ def __init__(self): tunnel = _TunnelIterator(self) self._tunnel = tunnel def __iter__(self): from twisted.internet.reactor import callInThread callInThread(self._tunnel.process) return self._tunnel def next(self): """ The method used to fetch the next value, make sure to return a list of rows, not just a row """ raise StopIteration class _TunnelIterator: """ This is an iterator which tunnels output from an iterator executed in a thread to the main thread. Note, unlike regular iterators, this one throws a PauseFlow exception which must be handled by calling reactor.callLater so that the producer threads can have a chance to send events to the main thread. """ def __init__(self, source): """ This is the setup, the source argument is the iterator being wrapped, which exists in another thread. """ self.source = source self.isFinished = 0 self.failure = None self.buff = [] def process(self): """ This is called in the 'source' thread, and just basically sucks the iterator, appending items back to the main thread. """ from twisted.internet.reactor import callFromThread try: while 1: val = self.source.next() self.buff.extend(val) # lists are thread safe except StopIteration: callFromThread(self.stop) self.source = None def setFailure(self, failure): self.failure = failure def stop(self): self.isFinished = 1 def next(self): if self.buff: return self.buff.pop(0) if self.isFinished: raise StopIteration if self.failure: raise self.failure raise Cooperate() class QueryIterator(ThreadedIterator): def __init__(self, pool, sql, fetchall=0): ThreadedIterator.__init__(self) self.curs = None self.sql = sql self.pool = pool self.data = None self.fetchall = fetchall def __call__(self,data): self.data = data return self def next(self): if not self.curs: conn = self.pool.connect() self.curs = conn.cursor() if self.data: self.curs.execute(self.sql % self.data) else: self.curs.execute(self.sql) if self.fetchall: res = self.curs.fetchall() else: res = self.curs.fetchmany() if not(res): raise StopIteration return res ----- End forwarded message ----- [View Less]

2 15