Mailman 3 March 2002 - Twisted

[Twisted-Python] Commits list problems
by Itamar Shtull-Trauring 06 Mar '02

06 Mar '02

Commits that change stuff in the root aren't sent to the commits list. So if I made a change to the ChangeLog along with my change, like the good programmer I am, nothing goes out to the commit list. So, things that were lost - refactor of tap making APIs, and the fact that I checked in the win32 event loop. This is BAD. This is ANNOYING. Someone fix it.

1 0

[Twisted-Python] Web Widgets Examples
by cpaul＠telemetrybox.org 06 Mar '02

06 Mar '02

I was wondering if anybody on this list has an example of using Web Widgets to render an HTML page? Aloha, Charles

2 2

[Twisted-Python] Mime Types ins static.File
by cpaul＠telemetrybox.org 05 Mar '02

05 Mar '02

Aloha, Today I shut down Apache on my workstation and started using twisted.web on port 80 - everything is sailing smooth, I even have a startup script in rc.d. The only issue I am running into so far has to do with mime types. In static, mime types are defined by the dictionary File.contentTypes - Is there an elegant way to dynamically set the mime types at mktap time? Or would it be better to hack static.py to use Python's mimetypes module along with a common mime.types file? I am currently using: contentTypes = mimetypes.read_mime_types('/usr/pkg/etc/httpd/mime.types') for my default mime types dictionary (this mime.types is from Apache). This is obviously not a cross-platform solution. By default, a NetBSD machine does not come with a mime.types database, instead it has to be installed with Apache or Netscape. Would it make sense for Twisted to come with it's own mime.types file? Thanks, Charles

1 0

[Twisted-Python] Win32 events main loop
by Andrew Bennetts 05 Mar '02

05 Mar '02

Hi all, Here is the first *rough* cut of the Twisted main loop using Win32 events, based upon internet.poll. It seems to be okay for simpleserv.py, but I haven't tested it for anything else. I'll leave that to a day when I'm not feeling so ill... I pretty much guarantee it is mostly broken -- think of it as a proof of concept. I think if I experiment with MsgWaitForMultipleObjects instead of WaitForMultipleObjects, I can make it respond to Win32 window messages properly; e.g. Ctrl-C and stopping when running as a service might work nicely -- COM stuff would probably benefit too. Enjoy! -Andrew. --- win32.py --- # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA """A win32event based implementation of the twisted main loop. To install the event loop (and you should do this before any connections, listeners or connectors are added): from twisted.internet import win32 win32.install() """ # Win32 imports from win32file import WSAEventSelect, FD_READ, FD_WRITE, FD_CLOSE, \ FD_ACCEPT, FD_CONNECT from win32event import CreateEvent, WaitForMultipleObjects, \ WAIT_OBJECT_0, WAIT_TIMEOUT, INFINITE # Twisted imports from twisted.python import log, threadable import main # globals reads = {} writes = {} events = {} def _addEvent(fd, why, events=events): print '_addEvent:', fd, why event = CreateEvent(None, 0, 0, None) WSAEventSelect(fd, event, why) events[event] = (fd, why) return event def addReader(reader, reads=reads): """Add a FileDescriptor for notification of data available to read. """ print 'addReader:', reader if not reads.has_key(reader): reads[reader] = _addEvent(reader, FD_READ|FD_ACCEPT|FD_CONNECT) def addWriter(writer, writes=writes): """Add a FileDescriptor for notification of data available to write. """ print 'addWriter:', writer if not writes.has_key(writer): writes[writer] =_addEvent(writer, FD_WRITE) def removeReader(reader): """Remove a Selectable for notification of data available to read. """ if reads.has_key(reader): del events[reads[reader]] del reads[reader] def removeWriter(writer, writes=writes): """Remove a Selectable for notification of data available to write. """ if writes.has_key(writer): del events[writes[writer]] del writes[writer] def removeAll(): """Remove all selectables, and return a list of them.""" result = reads.keys() + writes.keys() reads.clear() writes.clear() events.clear() return result def doWaitForMultipleEvents(timeout, reads=reads, writes=writes): if timeout is None: #timeout = INFINITE timeout = 5000 else: timeout = timeout * 1000 print 'timeout is:', timeout handles = events.keys() val = WaitForMultipleObjects(handles, 0, timeout) print 'WaitForMultipleObjects returned:', val if val == WAIT_TIMEOUT: return elif val >= WAIT_OBJECT_0 and val < WAIT_OBJECT_0 + len(handles): print 'Here' fd, why = events[handles[val - WAIT_OBJECT_0]] print fd, why log.logOwner.own(fd) try: if why & FD_READ|FD_ACCEPT|FD_CONNECT: fd.doRead() elif why == FD_WRITE: fd.doWrite() except: log.deferr() why = FD_CLOSE if why == FD_CLOSE: removeReader(fd) removeWriter(fd) try: fd.connectionLost() except: log.deferr() log.logOwner.disown(fd) def install(): """Install the poll()-based event loop.""" main.addReader = addReader main.addWriter = addWriter main.removeReader = removeReader main.removeWriter = removeWriter #main.doSelect = doPoll main.doSelect = doWaitForMultipleEvents main.removeAll = removeAll def initThreads(): """Do initialization for threads.""" if main.wakerInstalled: # make sure waker is registered with us removeReader(main.waker) addReader(main.waker) threadable.whenThreaded(initThreads) __all__ = ["install"]

2 2

[Twisted-Python] twistedmatrix client
by Kelly Roestel 04 Mar '02

04 Mar '02

I am new to python and twistedmatrix, so any help would be great. I have been able to write a server and connect to it via python sockets. But I would like to figure out how to create a twisted client. But, I have found no docs on creating clients. Is there some place that I can look for docs, or can some one help me with client setup. What I am passing to the client from the server is a simple list with four items. Thanks for the help -- Kelly Roestel Network Admin DolphinSearch, Inc

2 1

[Twisted-Python] Aloha, New Member
by cpaul＠telemetrybox.org 03 Mar '02

03 Mar '02

Aloha, My name is Charles Paul, and I am interested in learning more about how to use Twisted. Aloha, Charles

3 3

[Twisted-Python] A new testing strategy for protocols
by Itamar Shtull-Trauring 03 Mar '02

03 Mar '02

A long time issue we've had is how to test protocols under "real" circumstances. For DNS testing, I decided to do just that - see the code in twisted/test/test_names.py. Basically, I start a server listening using UDP, and have a client connect to it, all within the testing code and automatically. It works pretty well, after a bit of debugging. Data is pushed by calling main.iterate(). Now, in a lot of cases twisted.protocols.loopback may be good enough, but this approach is really necessary for thorough testing in certain cases. FTP leaps to mind, but also if we want automated testing of TOC or OSCAR clients against real servers. One drawback at least is that these tests will fail miserably on Jython (main.iterate on Jython doesn't push data), so I'm probably going to add code that makes sure they don't run at all on Jython. Another drawback is the related issue that I'm also testing DNS queries against real servers - this assumes a working net connection.

1 0

[Twisted-Python] Psyco is our friend
by Itamar Shtull-Trauring 03 Mar '02

03 Mar '02

Psyco == http://sf.net/projects/psyco/ So, I ran a Banana benchmark, attached. The results: Psyco on 1.8585100174 seconds (5380.65434483 messages a second) Now I edit the file and turn off psyco: Psyco off 4.81016397476 seconds (2078.93120743 messages a second) Nice, eh? import time, psyco from twisted.spread import banana PSYCO = 1 if PSYCO: psyco.bind(banana.encode) psyco.bind(banana.int2b128) psyco.bind(banana.b1282int) psyco.bind(banana.decode) psyco.bind(banana.Banana.dataReceived) psyco.bind(banana.Banana._encode) s = banana.encode([1, 2, "hello", "there"]) banana.decode(s) if PSYCO: print "Psyco on" else: print "Psyco off" # run the actual benchmark start = time.time() for i in xrange(10000): s = banana.encode([i, i*2, "hello", "there"]) banana.decode(s) elapsed = time.time() - start print elapsed, "seconds (%s messages a second)" % (10000 / elapsed)

1 0

[Twisted-Python] Unused variables in doSelect
by Andrew Bennetts 01 Mar '02

01 Mar '02

This is a minor thing, but... doSelect has the following comment (in 0.15.2): # Since this loop should really be as fast as possible, # I'm caching these global attributes so the interpreter # will hit them in the local namespace. reads=reads, writes=writes, rhk=reads.has_key, whk=writes.has_key): But rhk and whk are unused. Further on in the function, it does: for selectables, method, dict in ((r, "doRead", reads), (w,"doWrite", writes)): hkm = dict.has_key I suggest that either the rhk and whk variables are removed for clarity, or that that for loop becomes: for selectables, method, dict, hkm in ((r, "doRead", reads, rhk), (w,"doWrite", writes, whk)): Incidentally, the reason why I'm looking at doSelect in the first place is that something seems to have changed in 0.15.2 (since 0.15.0) that (on Windows at least) prevents Ctrl-C or Ctrl-Break from interrupting the process immediately -- it can take up to a minute for it to respond. I haven't yet figured out why, there's nothing obvious wrong. This also applies to stopping it when running as a Win NT service. On the other hand, it no longer constantly uses 1% of the cpu even when idle. I can reproduce this behaviour with simpleserv.py, so it's not my code causing it :) I wouldn't mind looking over the cvs history to look for relevant changes, but the CVSWeb link is missing from the new website. Regards, -Andrew.

3 2

[Twisted-Python] Twisted DNS issues
by Itamar Shtull-Trauring 01 Mar '02

01 Mar '02

<itamar> next, if you fail to connect to a DNS server, you get a traceback <itamar> something about passing extra args to connectionLost <moshez> itamar: yeah, I know. <moshez> itamar: ugh. <itamar> now, moving onwards <itamar> consider this code: <moshez> itamar: I didn't know about that <itamar> r = dns.ResolveConfResolver() <itamar> d = defer.Deferred().addCallback(printer).addErrback(printer) <itamar> d.arm() <itamar> r.resolve(d, "www.zoteca.com") <itamar> the result should be that printer() is based the IP, right? <itamar> but it doesn't <moshez> itamar: whatis it? <itamar> because in SentQuery.getAnswer, we do: <itamar> process = getattr(self, 'processAnswer_%d' % message.answers[0].type, None) <itamar> if process is None: <itamar> self.errback( <itamar> etc. <itamar> now, the first of the answers always seems to have type 5 <itamar> answers[1], 2, etc. seem to have the content <itamar> so it always fails <itamar> because you don't have processAnswer_5 <moshez> itamar: oh, I see. <itamar> conclusion: DNS needs a good accepttests suite

1 0