Mailman 3 June 2006 - Twisted

[Twisted-Python] How to get at applications in a reactor?
by David E. Konerding 21 Jun '06

21 Jun '06

Hi folks,. I've set up a convenient telnet manhole into my application. The application runs a web site serving up Static files and some Resources. What I want to do is telnet into the manhole and get access to the application object. Unfortunately, the useful information is not bound to the local scope of the telnet manhole, so I have to "stash" the application object when it is instantiated in some global place (such as sys._myapplication). I'd much rather just be able to import a reactor and get at the Sites which have been added to it. Browsing around the reactor, however, hasn't been that illuminating, and none of the reference docs seem to point at some global registry of added Sites/Applications, etc. Dave

3 3

[Twisted-Python] Twisters at EuroPython ?
by Thomas HERVE 21 Jun '06

21 Jun '06

Hi, My company should pay me the trip to assist EuroPython at Geneva (which is a great thing!). I'd probably be there from the 1st to the 5th of July, and was wondering if some folks were motivated by discuss/code/whatever, particulary in the pre-conference time. No sprint are registered for now in the wiki page : http://wiki.python.org/moin/EuroPython2006. I'm interested about nearly any topics (code, bug squashing, doc, tutorial, etc). Among Twisted guys I identify Moshe (I'll definitely go to wuuuuu at his talk :)), who didn't say when he'll be there. I suppose there should be other mailling-list readers, so please make yourself known! -- Thomas ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

1 0

[Twisted-Python] pb over windows named pipes?
by Micky Latowicki 21 Jun '06

21 Jun '06

Hi. We are interested in running pb using windows named pipes as transport, to enjoy the OS-level access control they offer. We seek advice regarding: 1. what must be written to make this happen 2. how hard would it be 3. what existing code can we learn from Thanks in advance Micky Latowicki, Retell Development Laboratories

5 7

Re: [Twisted-Python] twisted threads: synchronous function calls
by Ed Suominen 21 Jun '06

21 Jun '06

> From: Andrew Bennetts <andrew-twisted(a)puzzling.org> On Tue, Jun 20, > 2006 at 03:52:28PM +0200, Markus Schiltknecht wrote: [...] >> Now, I might have missed something, but that's sure functionality >> which Twisted should provide. > > See http://twistedmatrix.com/trac/ticket/1042 > > It has code for a function that does the same basic thing that yours > does, but with a bit more polish, e.g. if the result is a Deferred, > it won't wake the calling thread until the Deferred has a result, and > it handles exceptions. > > It just needs someone to write some tests and docs, and then it can > be merged. The function in that issue has actually been used by a few > people, so it seems to do the job. I liked Andrew's idea and incorporated the code of his ticket into my syncbridge module, which I'm using as part of my sAsync package, http://foss.eepatents.com/sAsync/ In addition to the topic here, the syncbridge module expands considerably on the theme of queuing up synchronous tasks. It took a *lot* of work to get the synchronous/asynchronous interface debugged and free (apparently) of race conditions. And it has a pretty complete suite of unit tests, although there's just one test for this particular method right now. The sAsync package is GPL, but I just re-licensed the syncbridge module itself to MIT in hopes of getting Glyph to accept it into Twisted. See ticket http://twistedmatrix.com/trac/ticket/1768. Best regards, Ed

1 0

[Twisted-Python] twisted threads: synchronous function calls
by Markus Schiltknecht 20 Jun '06

20 Jun '06

Hi, I had to use a thread... that thread must also call a function in the twisted reactor thread. No problem, I thought, looked up in the fine manual and found the reactor.callFromThread(somefunc, someargs...). Wonderful, so in my separate thread I wrote: x = reactor.callFromThread(myfunc, myargs...) ..but... callFromThread returns before myfunc finishes. It seems to be useful for async function calls, but not for my case. I ended up writing something like: def syncCall(): waitCond = threading.Condition() waitCond.result = 0 def _process(cond): cond.acquire() cond.result = ..some function call.. cond.notify() cond.release() waitCond.acquire() reactor.callFromThread(_process, waitCond) waitCond.wait() waitCond.release() return waitCond.result Now, I might have missed something, but that's sure functionality which Twisted should provide. In my first attempt I let the function return a Defer. I expected the reactor to 'send back' the result to the calling thread only after the Defer.callback() was called. IMHO that would be the most elegant and intuitively way: def myfunc(myarg): Defer d reactor.callLater(10, d.callback) return d # from the separate thread one would write: reactor.callFromThreadSync(myfunc, args) Am I facing problems I'm not aware of? Did I miss something? Regards Markus

3 2

Re: [Twisted-Python] General question on components, interfaces, etc.
by Terry Jones 20 Jun '06

20 Jun '06

>>>>> "glyph" == glyph <glyph(a)divmod.com> writes: glyph> It allows you to use the same language to talk about plugins as you glyph> would about any other kind of adapter. Effectively, the plugin glyph> system is a way to adapt your "system environment" to an object glyph> which provides a given interface. If you use the twisted plugin glyph> system (or some other system which uses Interface objects) then it glyph> is pretty easy to factor your code such that implementations of the glyph> desired interface can be obtained implicitly through the environment glyph> or explicitly by user code passing in an implementation -- or by glyph> passing in an object that is adaptable to that interface, and then glyph> doing adaptation. Hi, thanks for the explanation. I have another basic question... If were not using a component-based approach and instead I simply implemented the interface I wanted (in, say, myImp.py), then I could "import myImp" in multiple source files and everything would be fairly straightforward (if inflexible when it came to using different implementations). If I allow a user or sysadmin to specify the implementation to use on the command line, then my main() in main.py can use some kind of plugin approach to find and load the right source file. That's presumably going to use some form of __import__, such as myImp = __import__(....) so I then have a module-level global variable in main.py called myImp. But my other source files do not yet know what to import. It seems like an obvious solution is to have other source files that want to use the implementation refer to main.myImp instead of importing anything themselves. Or, equivalently, one could have a plugin.py module with an init() function that loads all required plugins and makes them accessible as module-level global variables for others to use to get to classes and methods in the loaded implementation. Is that the recommended way to deal with accessing dynamically imported components? Regards, Terry

1 0

[Twisted-Python] Re: [Twisted-commits] r17259 - Revert 17255 - win32 test suit failures
by James Y Knight 19 Jun '06

19 Jun '06

On Jun 17, 2006, at 3:00 PM, Jp Calderone wrote: > Revert 17255 - win32 test suit failures FYI, the error was: ======================================================================== ======= [ERROR]: twisted.web2.dav.test.test_options.OPTIONS.test_DAV1 Traceback (most recent call last): File "c:\buildslave\win32-select\W32-full2.4-select\Twisted\twisted \web2\dav\test\util.py", line 164, in do_serialize r = f(*args) File "c:\buildslave\win32-select\W32-full2.4-select\Twisted\twisted \web2\dav\test\util.py", line 121, in send d = request.locateResource(request.uri) File "c:\buildslave\win32-select\W32-full2.4-select\Twisted\twisted \web2\server.py", line 408, in locateResource assert segments[0] == "", "URL path didn't begin with '/': %s" % (path,) exceptions.AssertionError: URL path didn't begin with '/': %5Cdir2% 5Ctest_delete.py James

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 18 Jun '06

18 Jun '06

Bug summary ______________________ Summary for 2006-06-11 through 2006-06-18 Bugs opened: 17 Bugs closed: 20 Total open bugs: 618 (-3) |== Type Changes |== Priority Changes |== Component Changes |Defect: -2 |Highest: -2 |Conch: -5 |Enhancement: +0 |High: -3 |Core: -2 |Task: +1 |Normal: +3 |Mail: -1 |Low: +1 |Runner: +1 |Lowest: +0 |Web: -1 |Web2: +4 |Website: +2 |Words: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#1815] Update website documentation for web2 0.2.0 (opened by dreid) (CLOSED, fixed) enhancement website http://twistedmatrix.com/trac/ticket/1815 [#1817] Several methods in web2.dav that should return deferreds do not (opened by wsanchez) enhancement web2 http://twistedmatrix.com/trac/ticket/1817 [#1818] connectionLost in examples is omitting reason arg (opened by jerub) (CLOSED, fixed) defect mail http://twistedmatrix.com/trac/ticket/1818 [#1829] Various simple lore errors exist in twisted documentation (opened by exarkun) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/1829 ===== High ===== [#1830] Sell the Twisted reference PDF, bound as a physical book (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/1830 ===== Normal ===== [#1814] twisted.web2.auth should provide an UnauthorizedResponse (opened by dreid) enhancement web2 http://twistedmatrix.com/trac/ticket/1814 [#1816] More informative docstring for chainDeferred (opened by edsuom) enhancement core http://twistedmatrix.com/trac/ticket/1816 [#1819] give permission to update the website to the people who occasionally need to do it (opened by dreid) enhancement website http://twistedmatrix.com/trac/ticket/1819 [#1820] TwistedRunner 0.2.0 install on Cygwin in build_ext step (-lrpc link statement is missing) (opened by jenisys) defect runner http://twistedmatrix.com/trac/ticket/1820 [#1821] Turn deferredgenerator wiki page into howto (opened by itamarst) enhancement website http://twistedmatrix.com/trac/ticket/1821 [#1823] twisted.words.im.tap / proxyui is untested and riddled with bugs. (opened by jerub) enhancement words http://twistedmatrix.com/trac/ticket/1823 [#1824] typo in plugin howto (opened by exarkun) defect core http://twistedmatrix.com/trac/ticket/1824 [#1825] web2.dav COPY and MOVE methods fail if root directory of DAV resource is unwritable (opened by troy(a)osc.edu) defect web2 http://twistedmatrix.com/trac/ticket/1825 [#1827] web2.auth.wrapper.UnauthenticatedResource does not reply with allowed methods for wrapped resource (opened by troy(a)osc.edu) defect web2 http://twistedmatrix.com/trac/ticket/1827 [#1828] DAV server using web2.dav fails several properties tests in Litmus 0.10.5 (opened by troy(a)osc.edu) defect web2 http://twistedmatrix.com/trac/ticket/1828 ===== Low ===== [#1822] connectTCP should re-resolve domain names on each connection attempt (opened by itamarst) enhancement core http://twistedmatrix.com/trac/ticket/1822 ===== Lowest ===== [#1826] Typo in twisted.plugin example (opened by terrycojones) (CLOSED, duplicate) defect core http://twistedmatrix.com/trac/ticket/1826 Closed Bugs ______________________ ===== Highest ===== [#1818] connectionLost in examples is omitting reason arg (opened by jerub, closed by stephen, fixed) defect mail http://twistedmatrix.com/trac/ticket/1818 [#1565] VT102Writer.__str__ method in manhole.py does not check for special case (opened by htj, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/1565 [#1629] Unjellyable.unjellyFor does not return self (opened by ml.flex, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/1629 [#1817] Several methods in web2.dav that should return deferreds do not (opened by wsanchez, closed by exarkun, ) enhancement web2 http://twistedmatrix.com/trac/ticket/1817 [#1815] Update website documentation for web2 0.2.0 (opened by dreid, closed by exarkun, fixed) enhancement website http://twistedmatrix.com/trac/ticket/1815 [#1829] Various simple lore errors exist in twisted documentation (opened by exarkun, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/1829 [#664 ] use reactor pool instead of internal pool (opened by davep, closed by davep, wontfix) enhancement core http://twistedmatrix.com/trac/ticket/664 ===== High ===== [#417 ] Twisted.web deferred handling bug (opened by slyphon, closed by exarkun, invalid) defect web http://twistedmatrix.com/trac/ticket/417 [#466 ] OSCAR protocol bug (opened by exarkun, closed by exarkun, fixed) defect conch http://twistedmatrix.com/trac/ticket/466 [#427 ] cred backwards compatibility fails for updateApplication (opened by acapnotic, closed by exarkun, wontfix) defect conch http://twistedmatrix.com/trac/ticket/427 [#1009] Document defgen (opened by hypatia, closed by edsuom, fixed) defect core http://twistedmatrix.com/trac/ticket/1009 ===== Normal ===== [#405 ] uid/gid related permissions checking methods for t.w.filepath (opened by slyphon, closed by exarkun, invalid) enhancement conch http://twistedmatrix.com/trac/ticket/405 [#464 ] simple enterprise example (opened by davep, closed by exarkun, invalid) enhancement conch http://twistedmatrix.com/trac/ticket/464 [#396 ] [PATCH] Maildir folder and courier Maildir++ support (quota) (opened by sirius, closed by exarkun, wontfix) enhancement mail http://twistedmatrix.com/trac/ticket/396 [#450 ] [PATCH] add --pidfile '' for not creating pidfiles (opened by rgl, closed by exarkun, wontfix) enhancement core http://twistedmatrix.com/trac/ticket/450 [#497 ] client support for SFTP (opened by z3p, closed by exarkun, fixed) enhancement conch http://twistedmatrix.com/trac/ticket/497 [#1819] give permission to update the website to the people who occasionally need to do it (opened by dreid, closed by tekNico, ) enhancement website http://twistedmatrix.com/trac/ticket/1819 [#1782] web2's WSGIHandler incorrectly advertises twisted as a multiprocess environment (opened by pjenvey, closed by foom, fixed) defect web2 http://twistedmatrix.com/trac/ticket/1782 [#665 ] extend reactor pool api to handle suggested additional threads (opened by davep, closed by davep, wontfix) enhancement core http://twistedmatrix.com/trac/ticket/665 ===== Lowest ===== [#1826] Typo in twisted.plugin example (opened by terrycojones, closed by mithrandi, duplicate) defect core http://twistedmatrix.com/trac/ticket/1826

1 0

[Twisted-Python] PB and other distributed object protocols
by Stephen Waterbury 18 Jun '06

18 Jun '06

Hi Gang, Preface: I'm not a protocol guru, or even a "distributed object" (DO) guru, so this message is from the point of view of a DO novice looking for insights from some of the twisted minions (/gurus) and other cybergods who hang out here. I've been monitoring Brian's work on newpb (PBng?), and I love everything about it. Except how long it's taking. :( Brian: I know you have a real job, too, so I completely understand! (Incidentally, you're the only one whose commit messages I save, because they're always so well documented.) I've read the TODO list, and pretty much everything is beyond my abilities (and time) to contribute to -- sorry! -- but is it possible to put any kind of approximate timeline on some of it? PB is what I really want to use for my project, but I need something that's close to ready-for-prime-time, and I've been waiting literally years for PB, so in the meantime I'm looking at other candidates. I would definitely prefer a "twistable" technology -- i.e., one for which some kind of async adapter can be written from twisted -- and also one that has bindings for other languages (mainly Java, since my organization is Java-heavy). I've dismissed the idea of CORBA -- although one or more of the python orbs is probably twistable, CORBA has such a negative image that I'd probably have to lie about using it, and it does have lots of weird baggage. [Aside: I always liked Bill Janssen's ILU (how could you not like a distributed object system that had bindings for both python and lisp??) and I was sad to see it go. Interesting that Bill is now in the Python community.] Over the weekend I read the docs for ICE (http://www.zeroc.com/), sort of a streamlined, WAN-optimized CORBA. It has nice features, and even support for (threaded) async operations which seems (to my novice understanding) twistable. But its type system and python binding seem very unpythonic to me, based on lots of C++-isms, etc. Ugh. I just today discovered "json-rpc", which looks pretty cool. I currently use xml-rpc in my twisted app, and I've thought that json would be a neat replacement for xml (more concise + very pythonic syntax so probably *way* more efficient python marshalling/unmarshalling). And someone finally did it -- great! Has anyone here experimented with it? Anyone thought about twistifying it? If not, I might take a stab (I'd start by looking at what Itamar [I think] did with xml-rpc). Cheers, Steve

5 8

[Twisted-Python] @raise and deferreds
by Wilfredo Sánchez Vega 17 Jun '06

17 Jun '06

I'm curious about what conventions we have for documenting exceptions that are raised in deferreds, which turn into errbacks. Say I have a function which returns a deferred, and several flavors of Failure might have been raised. I'd like to document that thusly: def foo(): """ Do something fooey. @raise FooError: if it's too fooey. @raise BarError: if there is any bar; this is foo, not bar. @raise HolyCrapError: if your pants are on fire. @return: a deferred containing a Foo. """ ... This is a little weird, since the exceptions are actually being raised, but are being delivered in errbacks. The alternative is to describe all of the possible exceptions in the @return clause, but that is really clunky. So I'd like to have a convention whereby everyone knows that if you have a function that returns a deferred, that the exceptions described in the docstring refer to exceptions delivered via errbacks rather than via raising. Is that making any sense? Alternatively, we could make up a new keyword (@errback?), but I don't know how troublesome doing so is. If we do that, maybe another keyword (@callback) makes sense as well. -wsv

3 2