Mailman 3 September 2001 - Twisted

[Twisted-Python] Resolver patch: updated
by Moshe Zadka Sept. 23, 2001

Sept. 23, 2001

Last version had a slight bug, here it is without it. Index: twisted/internet/main.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/main.py,v retrieving revision 1.27 diff -u -r1.27 main.py --- twisted/internet/main.py 2001/09/19 12:18:09 1.27 +++ twisted/internet/main.py 2001/09/23 13:18:10 @@ -50,9 +50,13 @@ if os.name == "posix": self.uid = uid or os.getuid() self.gid = gid or os.getgid() + … [View More] self.resolver = DummyResolver() - persistentVersion = 1 + persistentVersion = 2 + def upgradeToVersion2(self): + self.resolver = DummyResolver() + def upgradeToVersion1(self): """Version 1 Persistence Upgrade """ @@ -161,6 +165,7 @@ def run(self, save=1): """Run this application, running the main loop if necessary. """ + global resolver if not self.running: threadable.dispatcher.own(self) delayeds.extend(self.delayeds) @@ -174,6 +179,7 @@ return for service in self.services.values(): service.startService() + resolver = self.resolver self.running = 1 threadable.dispatcher.disown(self) if not running: @@ -192,6 +198,11 @@ """ theTimeouts.later(method, seconds) +class DummyResolver: + + def resolve(self, address, success, fail): + fail() + reads = {} writes = {} running = None @@ -199,6 +210,7 @@ if threadable.threaded: delayeds.append(threadtask.theScheduler) shutdowns = [theTimeouts.runEverything] +resolver = DummyResolver() def shutDown(a=None, b=None): """Run all shutdown callbacks (save all running Applications) and exit. @@ -523,6 +535,7 @@ # currentPlugins = os.path.abspath("TwistedPlugins") allPlugins = [systemPlugins, userPlugins, confPlugins] #, currentPlugins] sys.path.extend(allPlugins) + # Sibling Import import process Index: twisted/internet/tcp.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/tcp.py,v retrieving revision 1.22 diff -u -r1.22 tcp.py --- twisted/internet/tcp.py 2001/09/11 04:08:28 1.22 +++ twisted/internet/tcp.py 2001/09/23 13:18:11 @@ -28,6 +28,7 @@ import socket import sys import traceback +import string if os.name == 'nt': EWOULDBLOCK = 10035 @@ -49,7 +50,7 @@ # Sibling Imports import abstract -from main import CONNECTION_LOST, CONNECTION_DONE +import main class Connection(abstract.FileDescriptor, protocol.Transport, @@ -79,9 +80,9 @@ if se.args[0] == EWOULDBLOCK: return else: - return CONNECTION_LOST + return main.CONNECTION_LOST if not data: - return CONNECTION_LOST + return main.CONNECTION_LOST return self.protocol.dataReceived(data) def writeSomeData(self, data): @@ -95,7 +96,7 @@ except socket.error, se: if se.args[0] == EWOULDBLOCK: return 0 - return CONNECTION_LOST + return main.CONNECTION_LOST def connectionLost(self): """See abstract.FileDescriptor.connectionLost(). @@ -145,7 +146,7 @@ self.port = port Connection.__init__(self, skt, protocol) self.doWrite = self.doConnect - self.doConnect() + self.resolveAddress() self.logstr = self.protocol.__class__.__name__+",client" if timeout is not None: main.addTimeout(self.failIfNotConnected, timeout) @@ -161,13 +162,32 @@ # factored out so as to minimise the code necessary for SecureClient return socket.socket(socket.AF_INET,socket.SOCK_STREAM) + def resolveAddress(self): + parts = string.split(self.addr[0], '.') + if len(parts) == 4: + try: + for part in map(int, parts): + if not (0<=part<256): + break + else: + self.setRealdAddress(self.addr[0]) + return + except ValueError: + pass + main.resolver.resolve(self.addr[0], self.setRealAddress, + self.failIfNotConnected) + + def setRealAddress(self, address): + self.realAddress = address + self.doConnect() + def doConnect(self): """I connect the socket. Then, call the protocol's makeConnection, and start waiting for data. """ try: - self.socket.connect(self.addr) + self.socket.connect((self.realAddress, self.addr[1])) except socket.error, se: if se.args[0] == EMYSTERY: self.startWriting() -- The Official Moshe Zadka FAQ: http://moshez.geek The Official Moshe Zadka FAQ For Dummies: http://moshez.org Read the FAQ [View Less]

1 0

[Twisted-Python] Non-blocking Resolution
by Moshe Zadka Sept. 23, 2001

Sept. 23, 2001

Here is a patch for stopping to use the native (blocking!) resolve, and using an internal resolver. Later "mktap" could grow arguments to set the resolver, and other things. I have not yet done UDP, but I plan to do it soon enough, if this patch is deemed acceptable. Note that this means a resolver must be persistable. The DNS resolver is not peristable, but I already have a patch to make it so. Index: twisted/internet/main.py ===================================================================… [View More] RCS file: /cvs/Twisted/twisted/internet/main.py,v retrieving revision 1.27 diff -u -r1.27 main.py --- twisted/internet/main.py 2001/09/19 12:18:09 1.27 +++ twisted/internet/main.py 2001/09/23 13:07:25 @@ -50,9 +50,13 @@ if os.name == "posix": self.uid = uid or os.getuid() self.gid = gid or os.getgid() + self.resolver = DummyResolver() - persistentVersion = 1 + persistentVersion = 2 + def upgradeToVersion2(self): + self.resolver = DummyResolver() + def upgradeToVersion1(self): """Version 1 Persistence Upgrade """ @@ -161,6 +165,7 @@ def run(self, save=1): """Run this application, running the main loop if necessary. """ + global resolver if not self.running: threadable.dispatcher.own(self) delayeds.extend(self.delayeds) @@ -174,6 +179,7 @@ return for service in self.services.values(): service.startService() + resolver = self.resolver self.running = 1 threadable.dispatcher.disown(self) if not running: @@ -192,6 +198,11 @@ """ theTimeouts.later(method, seconds) +class DummyResolver: + + def resolve(self, address, success, fail): + fail() + reads = {} writes = {} running = None @@ -199,6 +210,7 @@ if threadable.threaded: delayeds.append(threadtask.theScheduler) shutdowns = [theTimeouts.runEverything] +resolver = DummyResolver() def shutDown(a=None, b=None): """Run all shutdown callbacks (save all running Applications) and exit. @@ -523,6 +535,7 @@ # currentPlugins = os.path.abspath("TwistedPlugins") allPlugins = [systemPlugins, userPlugins, confPlugins] #, currentPlugins] sys.path.extend(allPlugins) + # Sibling Import import process Index: twisted/internet/tcp.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/tcp.py,v retrieving revision 1.22 diff -u -r1.22 tcp.py --- twisted/internet/tcp.py 2001/09/11 04:08:28 1.22 +++ twisted/internet/tcp.py 2001/09/23 13:07:26 @@ -28,6 +28,7 @@ import socket import sys import traceback +import string if os.name == 'nt': EWOULDBLOCK = 10035 @@ -49,7 +50,7 @@ # Sibling Imports import abstract -from main import CONNECTION_LOST, CONNECTION_DONE +import main class Connection(abstract.FileDescriptor, protocol.Transport, @@ -79,9 +80,9 @@ if se.args[0] == EWOULDBLOCK: return else: - return CONNECTION_LOST + return main.CONNECTION_LOST if not data: - return CONNECTION_LOST + return main.CONNECTION_LOST return self.protocol.dataReceived(data) def writeSomeData(self, data): @@ -95,7 +96,7 @@ except socket.error, se: if se.args[0] == EWOULDBLOCK: return 0 - return CONNECTION_LOST + return main.CONNECTION_LOST def connectionLost(self): """See abstract.FileDescriptor.connectionLost(). @@ -145,6 +146,7 @@ self.port = port Connection.__init__(self, skt, protocol) self.doWrite = self.doConnect + self.resolveAddress() self.doConnect() self.logstr = self.protocol.__class__.__name__+",client" if timeout is not None: @@ -161,13 +163,32 @@ # factored out so as to minimise the code necessary for SecureClient return socket.socket(socket.AF_INET,socket.SOCK_STREAM) + def resolveAddress(self): + parts = string.split(self.addr[0], '.') + if len(parts) == 4: + try: + for part in map(int, parts): + if not (0<=part<256): + break + else: + self.realAddress = self.addr[0] + return + except ValueError: + pass + main.resolver.resolve(self.addr[0], self.setRealAddress, + self.failIfNotConnected) + + def setRealAddress(self, address): + self.realAddress = address + self.doConnect() + def doConnect(self): """I connect the socket. Then, call the protocol's makeConnection, and start waiting for data. """ try: - self.socket.connect(self.addr) + self.socket.connect((self.realAddress, self.addr[1])) except socket.error, se: if se.args[0] == EMYSTERY: self.startWriting() -- The Official Moshe Zadka FAQ: http://moshez.geek The Official Moshe Zadka FAQ For Dummies: http://moshez.org Read the FAQ [View Less]

1 0

[Twisted-Python] fcntl
by Moshe Zadka Sept. 23, 2001

Sept. 23, 2001

FCNTL exists in Python 2.2, but gives a DeprecationWarning. We should all be greatful for this warning in advance of a backwards incompatibility looming in the future. Here is a patch to future-proof the code I've checked it with 1.5.2 and 2.2a3 Please let me know of any objections Index: twisted/internet/process.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/process.py,v retrieving revision 1.6 diff -u -r1.6 process.py --- … [View More]

1 0

[Twisted-Python] Registering functions to be run at shutdown, delayeds
by Itamar Shtull-Trauring Sept. 23, 2001

Sept. 23, 2001

Right now, functions to be run at shutdown are added by appending to twisted.internet.main.shutdowns, and delayeds are added by appending to twisted.internet.main.delayeds. I suggest adding functions for both: Index: main.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/main.py,v retrieving revision 1.27 diff -c -r1.27 main.py *** main.py 2001/09/19 12:18:09 1.27 --- main.py 2001/09/23 08:46:55 *************** *** 373,378 **** ---… [View More]

1 0

[Twisted-Python] (no subject)
by Moshe Zadka Sept. 22, 2001

Sept. 22, 2001

>------------------ control_panel.py -----------< # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or … [View More]FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from twisted.spread import pb from twisted.internet import passport, main class ControlPanelPerspective(pb.Perspective): def perspective_shutdown(self): main.shutDown() def perspective_save(self, name=''): import __main__ __main__.application.save(name) class ControlPanelService(pb.Service): def getPerspectiveNamed(self, name): return ControlPanelPerspective("any", self) if __name__ == '__main__': import control_panel from twisted.internet.main import Application app = Application("control_panel") i = passport.Identity("guest", app) i.setPassword("guest") app.authorizer.addIdentity(i) bf = pb.BrokerFactory(app) svc = control_panel.ControlPanelService("control_panel", app) i.addKeyForPerspective(svc.getPerspectiveNamed('any')) app.listenOn(pb.portno, bf) app.save("start") >------------------------------------------------------------< >----------------- pbshutdown.py -----------------------------< # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from twisted.spread import pb from twisted.internet import tcp, main def failure(error): print "Failure...",error main.shutDown() def disconnected(): print "disconnected." main.shutDown() def connected(perspective): perspective.shutdown(pberrback=main.shutDown, pbcallback=main.shutDown) print "connected." def preConnected(identity): identity.attach("control_panel", "any", None, pbcallback=connected, pberrback=failure) # run a client b = pb.Broker() b.requestIdentity("guest", # username "guest", # password callback = preConnected, errback = failure) tcp.Client("localhost",pb.portno,b) main.run() >------------------------------------------------------------< >------------------- pbsave.py ----------------------------< # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from twisted.spread import pb from twisted.internet import tcp, main def failure(error): print "Failure...",error main.shutDown() def disconnected(): print "disconnected." main.shutDown() def connected(perspective): perspective.save('pbservice', pberrback=main.shutDown, pbcallback=main.shutDown) print "connected." def preConnected(identity): identity.attach("control_panel", "any", None, pbcallback=connected, pberrback=failure) # run a client b = pb.Broker() b.requestIdentity("guest", # username "guest", # password callback = preConnected, errback = failure) tcp.Client("localhost",pb.portno,b) main.run() >------------------------------------------------------------< -- The Official Moshe Zadka FAQ: http://moshez.geek The Official Moshe Zadka FAQ For Dummies: http://moshez.org Read the FAQ [View Less]

2 2

[Twisted-Python] twistd possible hole
by Moshe Zadka Sept. 22, 2001

Sept. 22, 2001

twistd adds the running-directory to the system include path. I'm not sure I like it, because it must have write access to the running directory to dump pickles, and having applications have write permissions there. What's more, in the case where the running directory is shared between several twistd instances, it means one twistd instance can corrupt others via messing with their path. Why was it done? If there is no good reason, I suggest we drop it. -- The Official Moshe Zadka FAQ: http://… [View More]

2 2

[Twisted-Python] accepttests mail test fails with python 1.5.2 due to binascii.hexlify
by Tommi Virtanen Sept. 22, 2001

Sept. 22, 2001

[tv@ki ~/work/twistedmatrix/Twisted]$ WEBBROWSER=links USER=tv IRCCLIENT=run-xchat PYTHONPATH=. ./bin/accepttests /------ |#### |#### Starting test. |#### \------ /------ |#### |#### Starting mail test |#### \------ Running System Command: './bin/mktap mail --domain foo.bar=dump --user postmaster=postmaster' Saving mail application to mail.tap... Saved. Running System Command: './bin/mktap --append mail.tap mail --relay 127.0.0.1,8025=dump2 --smtp 8026 --pop 8111' Saving mail … [View More]

1 0

[Twisted-Python] --python and --plugin
by Moshe Zadka Sept. 21, 2001

Sept. 21, 2001

I'd like to add a feature, that if there is a 'getPorts' in the namespace then twistd will create an application itself, and then call getPorts() on it which should return a list of [(port num), factory], and then will call application.listenOn on those tuples. What is the big win? we can give mktap --plugin and --pyton options too, which means it will be possible (and easy!) to configure a .tap given a plugin, which is currently painful. -- The Official Moshe Zadka FAQ: http://moshez.geek … [View More]

1 0

[Twisted-Python] Mail Support
by Moshe Zadka Sept. 20, 2001

Sept. 20, 2001

Attention everyone: mail support (twisted.mail) is going to be rewritten completely, breaking backwards compatibility with domains written by 3rd parties. If you happen to have written any domain, please let me know. Problems which I do not want to fix in a backwards compatible way: * exists() and saveMessage() are assumed to be synchronous. They should not be * The order of parameters to exists and saveMessage is completely wrong How I am going to fix it: * exists() and saveMessage() … [View More]

1 1

[Twisted-Python] Event/Notifier Across Pb Boundaries
by Moshe Zadka Sept. 19, 2001

Sept. 19, 2001

I've written a small module to do event/notifications. It's a fairly small and nice example of how to write pb protocols as well as being something useful >----------- twisted/internet/event.py ---------------< # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation.… [View More] # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from twisted.spread import pb from twisted.internet import passport from twisted.python import reflect class EventPublishPerspective(pb.Perspective): def perspective_subscribe(self, event, subscriber): self.service.subscribe(event, subscriber) def perspective_notifyEvent(self, event, *args, **kw): self.service.notifyEvent(event, args, kw) class EventPublishService(pb.Service): def __init__(self, *args, **kw): apply(pb.Service.__init__, (self,)+args, kw) self.eventSubscribers = {} def subscribe(self, event, subscriber): if not self.eventSubscribers.has_key(event): self.eventSubscribers[event] = [] self.eventSubscribers[event].append(subscriber) def notifyEvent(self, event, args, kw): for subscriber in self.eventSubscribers.get(event, ()): try: apply(subscriber.notifyEvent, (event,)+args, kw) except pb.ProtocolError: pass def getPerspectiveNamed(self, name): return EventPublishPerspective("any", self) class EventNotifier(pb.Referenceable): def registerAll(self, perspective): dct = {} reflect.addMethodNamesToDict(self.__class__, dct, "event_") for name in dct.keys(): perspective.subscribe(name, self) def remote_notifyEvent(self, event, *args, **kw): method = getattr(self, 'event_'+event, None) if method is None: return apply(method, args, kw) if __name__ == '__main__': import event from twisted.internet.main import Application app = Application("event") i = passport.Identity("guest", app) i.setPassword("guest") app.authorizer.addIdentity(i) bf = pb.BrokerFactory(app) svc = event.EventPublishService("event", app) i.addKeyForPerspective(svc.getPerspectiveNamed('any')) app.listenOn(pb.portno, bf) app.save("start") >-----------------------------------------------------------< >----------- run this to get "event-start.tap" -------------< from twisted.internet import event, passport, main from twisted.spread import pb app = main.Application("event") i = passport.Identity("guest", app) i.setPassword("guest") app.authorizer.addIdentity(i) bf = pb.BrokerFactory(app) svc = event.EventPublishService("event", app) i.addKeyForPerspective(svc.getPerspectiveNamed('any')) app.listenOn(pb.portno, bf) app.save("start") >--------------------------------------------< >------------ this connects and gets a notification -----------< from twisted.spread import pb from twisted.internet import tcp, main, event class HelloEvent(event.EventNotifier): def event_hello(self, who): print "hello", who main.shutDown() def failure(error): print "Failure...",error main.shutDown() def connected(perspective): hello = HelloEvent() hello.registerAll(perspective) print "registered" perspective.notifyEvent("hello", "world", pberrback=failure) print "connected." def preConnected(identity): identity.attach("event", "any", None, pbcallback=connected, pberrback=failure) b = pb.Broker() b.requestIdentity("guest", "guest", callback = preConnected, errback = failure) tcp.Client("localhost",pb.portno,b) main.run() >-------------------------------------------< -- The Official Moshe Zadka FAQ: http://moshez.geek The Official Moshe Zadka FAQ For Dummies: http://moshez.org Read the FAQ [View Less]

1 0

[Twisted-Python] Adaptive Port
by Moshe Zadka Sept. 19, 2001

Sept. 19, 2001

Index: twisted/internet/tcp.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/tcp.py,v retrieving revision 1.22 diff -u -r1.22 tcp.py --- twisted/internet/tcp.py 2001/09/11 04:08:28 1.22 +++ twisted/internet/tcp.py 2001/09/19 06:49:49 @@ -277,6 +277,7 @@ """ s = socket.socket(socket.AF_INET,socket.SOCK_STREAM) s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) + s.setblocking(0) return s … [View More]

1 0

[Twisted-Python] Usenix paper on event dispatch mechanisms
by Itamar Shtull-Trauring Sept. 18, 2001

Sept. 18, 2001

From Usenix 2001: * Scalability of Linux Event-Dispatch Mechanisms, http://lass.cs.umass.edu/~abhishek/papers/usenix01/paper.ps Relevance: Doing multiple accept() calls instead of one (after a select() call tells you a new connection is available) can speed up your application considerably under high loads.

3 4

[Twisted-Python] Patterns in Twisted
by Itamar Shtull-Trauring Sept. 16, 2001

Sept. 16, 2001

I've been reading POSA2, and I see Twisted uses a lot of the patterns there. You can find more info on each pattern by searching for it in google - most of the content in the book is also in publically accessable papers that can be found on the web. I'm including here the one sentence description from http://www.cs.wustl.edu/~schmidt/POSA/. = Reactor = "* The Reactor architectural pattern allows event-driven applications to demultiplex and dispatch service requests that are delivered to … [View More]

1 0

[Twisted-Python] Suggested API addition to threadtask
by Itamar Shtull-Trauring Sept. 16, 2001

Sept. 16, 2001

Hi all, Background: internet.threadtask lets threads schedule tasks that are then executed by the main event loop (Active Object pattern). Problem: The results of these tasks are not returned to the threads, nor do the threads have any way of seeing if any exceptions were raised when doing the task. In most cases this is not an issue (e.g. writing data to a connection) but it is a problem with things like opening a new tcp.Client connection. Suggested Solution: Optionally allow returning … [View More]a DeferredResult object when scheduling tasks, which can then be used to get the result (or exception) of the task when it is eventually executed by the event loop thread. Example: Notice how we can catch the exceptions raised by the scheduled task. ================================================== def toInt(x): return int(x) dresult = threadtask.scheduleWithResult(toInt, args=("23",)) if dresult.haveResult(): try: result = dresult.get() except ValueError: print "Not an integer" else: print "It's an integer: %d" % result =================================================== Feedback I'm looking for: 1) Initially I had the operation that gets the result (DeferredResult.get) block, but this is a bad idea since it can deadlock threads waiting for a result that will never be returned, e.g. when shutting down the server. Instead now I raise a NotReady exception if no result is available. Would callbacks be better? 2) Better method names, "scheduleWithResult" is ugly. 3) Is this idea (DeferredResults) applicable to internet.task as well? Attached is my updated code. # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA """Allow threads to schedule tasks to be run by the main event loop. This lets threads execute non-thread safe code by adding it to the scheduler. This module should not be used by non-threaded modules, instead they should use twisted.internet.task. Tasks added to this scheduler will *not* be stored persistently. This is an implementation of the Active Object pattern, and can be used as part of the queueing layer for the Async/Half-Async pattern. For more details: 1) POSA2 book - http://www.cs.wustl.edu/~schmidt/POSA/ 2) Active Object - http://www.cs.wustl.edu/~schmidt/PDF/Act-Obj.pdf 3) Async/Half-Async - http://www.cs.wustl.edu/~schmidt/PDF/PLoP-95.pdf """ # twisted import from twisted.python import threadable, reflect class Scheduler: """I am a thread-aware delayed scheduler of for synchronous event loops. Each thread has a set of tasks, which have interleaved execution phases. """ def __init__(self): self.threadTasks = {} def __getstate__(self): dict = copy.copy(self.__dict__) dict['threadTasks'] = {} return dict def addTask(self, function, args=[], kwargs={}): """Schedule a function to be called by the main event-loop thread. The result of the function will not be returned. """ threadTasks = self.threadTasks hadNoTasks = (threadTasks == {}) thread = reflect.currentThread() if not threadTasks.has_key(thread): threadTasks[thread] = [] threadTasks[thread].append((function, args, kwargs)) if hadNoTasks: main.wakeUp() def addTaskWithResult(self, function, args=[], kwargs={}): """Schedule a function, returning a DeferredResult object. The result of the function can be gotten by calling the get() method of this DeferredResult object. """ args = [function,] + list(args) dresult = DeferredResult() self.addTask(dresult._doAction, args, kwargs) return dresult def timeout(self): """Either I have work to do immediately, or no work to do at all. """ if self.threadTasks: return 0. else: return None def runUntilCurrent(self): threadTasks = self.threadTasks for thread, tasks in threadTasks.items(): func, args, kwargs = tasks.pop(0) apply(func, args, kwargs) if len(tasks) == 0: del threadTasks[thread] synchronized = ["addTask", "runUntilCurrent"] threadable.synchronize(Scheduler) class NotReady(RuntimeError): """The DeferredResult does not yet have a result.""" class DeferredResult: """Result of a scheduled operation that is deferred. We can get the result of the operation by calling the get() method, which will raise a NotReady exception if the result is not available. Otherwise it returns the result of the operation, or if it raised an exception, reraise that exception. The exception catching will only work for class-based exceptions, but hopefully by now string exceptions are rare. """ def __init__(self): self._finished = 0 def haveResult(self): """Did we get the result yet?""" return self._finished def get(self): """Return the result or if it's not available raise NotReady""" if not self._finished: raise NotReady, "we don't have the result yet." if hasattr(self, '_exception'): raise self._exception else: return self._result def _doAction(self, function, *args, **kwargs): """Run the action whose result we are. Should only be called by the scheduler.""" try: result = apply(function, args, kwargs) except Exception, e: self._exception = exception else: self._result = result self._finished = 1 theScheduler = Scheduler() def schedule(function, args=[], kwargs={}): """Schedule a function to be called by the main event loop thread""" global theScheduler theScheduler.addTask(function, args, kwargs) def scheduleWithResult(function, args=[], kwargs={}): """Schedule a function to be called by the main event loop thread. Returns a DeferredResult object, allowing us to get the result of the function. """ global theScheduler return theScheduler.addTaskWithResult(function, args, kwargs) # sibling import - done here to prevent circular import import main [View Less]

1 0

[Twisted-Python] What you thought would always last has passed you by...
by Glyph Lefkowitz Sept. 14, 2001

Sept. 14, 2001

Apologies for the dearth of information on the mailing list and releases lately. I've been working on a new job, visiting my family, and some other things. However, I have been pouring quite a bit of effort into Twisted Python as well. The most exciting new development is Perspective Broker, the erstwhile replacement for Gloop; I'm currently starting to remove the old gloop-protocols in .web and .reality and replace them with the newer, more robust perspective protocol. Perspective Broker … [View More]

3 3

[Twisted-Python] Traceback with CVS HEAD
by j.her＠t-online.de Sept. 13, 2001

Sept. 13, 2001

jh@zaibach:~$ mkwebserver --personal Traceback (innermost last): File "/home/jh/bin/mkwebserver", line 128, in ? service = distrib.ResourcePublisher(site) TypeError: not enough arguments; expected 3, got 2

1 1

Re: [Twisted-Python] Reality Servers
by Chris.Hadgis＠mincom.com Sept. 13, 2001

Sept. 13, 2001

On Mon, Sep 03, 2001 at 09:41:31AM +1000, Chris.Hadgis(a)mincom.com wrote: > > ... broadcastToPair() sends the message to all containers > > associated with the figure's current location. [snip] > > My proposed solution is to not include the location's containers > > in the container list. > I'd say that this is the wrong solution. Indeed it is. Last night I sorted out the problem. broadcastToPair() calls pairHears() for each container. Each call to pairHears() … [View More]

1 0

[Twisted-Python] UDP patch
by Itai Ben-Yaacov Sept. 12, 2001

Sept. 12, 2001

Hi Following a suggestion from Moshez, I rewrote his UDP classes in order to get them somewhat more usable (eventually aiming at DNS protocol implementation). the example in doc/examples/udp-echoserv.py is now obsolete and will not work. Instead, UDP integrates neatly into doc/examples/echoserv.py Pezz. -- Itaï Ben-Yaacov Équipe de Logique Mathématique UFR de Mathématiques (case 7012) Université Paris 7 2 place Jussieu 75251 Paris Cedex 05 Email: pezz(a)logique.jussieu.fr

1 0

[Twisted-Python] WebDAV in twisted.web.server
by Benjamin Bruheim Sept. 10, 2001

Sept. 10, 2001

Hello, I have been taking a look on WebDAV, especially RFC 2518, "HTTP Extension for Distributed Authoring -- WEBDAV". The extension concerns four areas, queries and manipulation of properties, handling of collections, locking and namespace operations. The two last areas are not crucial. Each of these areas has its set of methods, which is dispatched the same way as 'GET' and 'PUT', but usually with a XML-file appended to the request. The twisted.web architecture of resources is well suited … [View More]for adding WebDAV. resource.Resource is fitting the term Resource as it is used in the RFC. And twisted.web.server.Request's way of handling requests/responses seems open-ended enough too add a xml-reader/writer that concates the reply with XML (mimed). I will here try to examine the changes that need to be done to twisted.web.server (and protocols.http). Barely nothing else is examined, or needed. I will blend some explanations into my proposal in case my understanding of twisted.* is wrong :). The methods 'GET', 'PUT', 'HEAD', etc, is not touched by WebDAV. These methods by definitions only concern 'files' (eg. GET retrieves the entity described by the URI). The WebDAV-Resource '/foo/' is a Resource Collection, and '/foo/file' is a member of the collection. If '/foo/faa/file' exists, then the 'faa'-collection needs to exist because the resource-tree has to be consistent. The trailing slash tells if the files is a collection or not; this is not a rule so the server may correct the client with a MOVED_PERMANENTLY in the same way web.static does. The tree may however contain non-DAV-compliant resources, so there won't be a need to force DAV onto everything. If a putChild should be put into a collection is a discussion on its own :) (I will however mean that a DAVResource should be a good starting point for making everything DAV, by letting the stuff to go DAV inherit DAVResource) Since resource is the dispatcher of methods, then WebDAV can be a simple twisted.web.resource. The request has arguments in the Header and this gets conviently passed to the assigned resource. The request may contain an XML with additional arguments as well. Request.process seems to handle the task well, except that its 'POST'-handler may be rewritten to collect the XML for most methods. 'GET' is done in the regular way, no XML, just send the entity the URI point at (for collections, send eg. the index.html. I don't know the case here, really). This is resource-stuff. All the action will be in resource.render (it will render the response XML, I guess). Minor thing, yet: If a request take long time to accomplish (eg. MOVE 100M of data over wire) there is a reply 102 PROCESSING which tells the client to keep the line open and wait. I don't know how the details, but I guess a NOT_DONE_YET may stop the block :) The problems with this is that it will send a response later on, I guess this can be hacked in. The changes that need to be done to protocols.http is simply adding the new error-codes, since that is all that is in it :) If we want to add namespace operations (MOVE, COPY) the http-client needs to be made DAV-compliant. Note that COPY/MOVE also can be done by FTP too ;) The changes: Request.resource: retrieve the XML in the request. Handle the response 102 PROCESSING protocols.http: Errorcodes The conclusion: Everything will happen in the DAV-Resource. :) [View Less]

3 5

[Twisted-Python] Server not shutting down in Windows
by Itamar Sept. 10, 2001

Sept. 10, 2001

Hey, In windows 2000, threaded server, my server does nut shutdown when main.shutDown() is called. It seens that main.waker is the problem, since it is not removed from the list of readers registered with the event loop. Does the following patch seem reasonable (someone better test on unix) or does someone have a better suggestion? It does seem to solve the problem: Index: main.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/… [View More]

1 0

[Twisted-Python] twisted.wed.widgets and defer.Deferred() giving a list.
by Tommi Virtanen Sept. 10, 2001

Sept. 10, 2001

Hi. I didn't commit this, as it's a bit of a conceptual change. Please give feedback. Patch is at the end. This change allows web.widgets to call defer.Deferred's callback() with a list instead of a string, making it possible to stream a web page to the browser by returning a string and then a Deferred object. Here's a snippet of a web UI I'm doing that streams LDAP search results to the browser as soon as the server sends them --… [View More] see handle_entry(): class LDAPSearchAndPrint(ldap.LDAPSearch): def __init__(self, ldapclient, callback, filter=pureldap.LDAPFilterMatchAll): ldap.LDAPSearch.__init__(self, ldapclient, baseObject='dc=example, dc=com', filter=filter, ) self.result="" self.callback=callback self.count=0 def handle_success(self): self.callback(["<p>%d entries matched."%self.count]) def handle_entry(self, objectName, attributes): result="<p>%s\n<ul>\n"%objectName for a,l in attributes: assert len(l)>0 if len(l)==1: result=result+" <li>%s: %s\n"%(a, l[0]) else: result=result+" <li>%s:\n <ul>\n"%a for i in l: result=result+" <li>%s\n"%i result=result+" </ul>\n" result=result+"</ul>\n" c=self.callback d=defer.Deferred() self.callback=d.callback c([result, d]) self.count=self.count+1 def handle_fail(self, resultCode, errorMessage): self.callback("fail: %d: %s"%(resultCode, errorMessage or "Unknown error")) --- twisted/web/widgets.py 2001/09/10 17:13:24 1.8 +++ twisted/web/widgets.py 2001/09/10 17:18:37 @@ -283,12 +283,17 @@ def callback(self, result, position, decNeedsHeaders): if result != FORGET_IT: self.needsHeaders = self.needsHeaders - decNeedsHeaders - if isinstance(result, defer.Deferred): - self._addDeferred(result, position) - self.lst[position] = result + if not isinstance(result, types.ListType): + result=[result] + for i in xrange(len(result)): + if isinstance(result[i], defer.Deferred): + self._addDeferred(result[i], position+i) + self.lst[position:position+1] = result + assert self.position <= position self.keepRendering() - if isinstance(result, defer.Deferred): - result.arm() + for r in result: + if isinstance(r, defer.Deferred): + r.arm() def keepRendering(self): -- tv(a){{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);} [View Less]

1 0

Re: [Twisted-Python] WebDAV in twisted.web.server
by Benjamin Bruheim Sept. 9, 2001

Sept. 9, 2001

I'm resending this because I didn't notice it coming up on the ml; sorry if it is a duplicate > I was wondering if it would be possible to implement WebDAV fully in > Resources, and not touching anything else. Resources can munge whatever > headers they want, etc, and we could just have a base DAVResource do all > the XML/Mime encoding. Yes, my last post pointed out how that could be done. Though, the stuff I pointed out (eg to make process able to forward the request content even … [View More]

1 0

[Twisted-Python] Time out on client connections
by Moshe Zadka Sept. 9, 2001

Sept. 9, 2001

Here's a patch, for your review: (against 0.10.1, but should apply cleanly) diff -urN Twisted/twisted/internet/main.py changed/twisted/internet/main.py --- Twisted/twisted/internet/main.py Mon Aug 20 14:10:14 2001 +++ changed/twisted/internet/main.py Sun Sep 9 14:27:01 2001 @@ -179,10 +179,12 @@ run() threadable.dispatcher.disown(self) +tempDelayed = delay.Delayed() # A delay for non-peristent delayed actions +tempDelayed.ticks = 1 reads = {} writes = {} running … [View More]

1 0

[Twisted-Python] tcp.GetHost in twisted.web.server
by Maj-Britt Bruheim Sept. 8, 2001

Sept. 8, 2001

Just checking before I commit :) Why do this: _host = socket.gethostbyaddr(socket.gethostname())[0] ... def getHost(self): return self._host when this works? def getHost(self): return self.transport.GetHost() This at least doesn't make the crash go bad in win98 // phed

3 2

[Twisted-Python] Re: [Twisted-commits] CVS: Twisted/twisted/web widgets.py,1.6,1.7
by Moshe Zadka Sept. 8, 2001

Sept. 8, 2001

On Sat, 08 Sep 2001, Glyph Lefkowitz <glyph(a)twistedmatrix.com> wrote: > -import string, time, types, traceback, copy, pprint, sys, os > +import string, time, types, traceback, copy, pprint, sys, os, string string is imported twice. No biggie, but it ain't nice -- The Official Moshe Zadka FAQ: http://moshez.geek The Official Moshe Zadka FAQ For Dummies: http://moshez.org Read the FAQ

1 0