Mailman 3 February 2002 - Twisted

[Twisted-Python] Object Oriented Database Library For Persistent Storage in Python
by Itamar Shtull-Trauring 19 Feb '02

19 Feb '02

Comments? <http://pyobjfs.sourceforge.net/index.html>

1 0

[Twisted-Python] Potential PB Security Problem (And Solution)
by Glyph Lefkowitz 19 Feb '02

19 Feb '02

While auditing some code that I'd written to use PB, I noticed a class of bug that I had previously not considered. When writing a PB remote interface, it's easy to write some code that looks like this: class MyObject(Referenceable): def remote_IOnlyExpectCopies(self, someCopy): someCopy.someOperation(self.somePrivilegedData) This is, of course, a potential security hole. Assuming somePrivilegedData is a basic type (and therefore serializeable), a hostile client could send a Referenceable of their own to IOnlyExpectCopies. A sufficiently vigilant application author would, of course, be able to protect against this by distrusting any arguments sent to a remote method and performing type checks on them, but then, application authors (including myself) are rarely "sufficiently" vigilant :-). My proposed solution is to change the way remote methods are invoked: instead of emulating regular Python methods, they would be accessed through a 'callRemote' method: fairly simply, calling the remote method 'foo' on 'bar' would look like this: bar.callRemote("foo", baz, boz=qux) This way, the above example of a local method call would blow up a remote object were passed to it. Pros of this approach: * it fixes this potentially common security problem * it makes it possible to grep for all places where a method is invoked remotely * it removes some overhead (creation of the RemoteMethod instance) Cons of this approach: * it's no longer possible to treat remote objects and objects that have methods which return Deferreds identically * slightly more typing * massive refactoring required, lots of user code might break I think the pros clearly outweigh the cons, so I'm going to start changing things over, potentially with a backwards-compatibility release in the interim. If anyone has a different idea, let me know. -- "Cannot stand to be one of many -- I'm not what they are." -Guster, "Rocketship" glyph lefkowitz; ninjaneer, freelance demiurge glyph @ [ninjaneering|twistedmatrix].com

9 20

[Twisted-Python] Windows issues...
by Benjamin Schollnick 19 Feb '02

19 Feb '02

Folks, I've got some questions, with windows usage of Twisted Matrix... I'm considering using TM as the framework for a revival of a another program that I use to write.... But, I ran into *ALOT* of issues on the PC side... (I wasn't even able to get it to work on the Mac due to Line-ending issues, I haven't figured a simple way to convert the entire tree yet). 1) setup.py install died....It stopped when it couldn't find "CL.EXE", while building something in the twisted.spread.cbanana section... 2) A lot of the files, don't have a .py extension. (i.e. mktap, twistd, etc) This is a major hindrance, because either I have to rename the file, or execute it explicitly "python mktap ....etc....". I have windows set to run the .py files via python automatically. 3) web.distrib has some issues. Mainly attempting to use the pwd unit under windows. I was able to add: try: ..... ..... except: pass to the import statement and the actual PWD / user area, without any obvious harmful problems.... 4) Error checking is extremely light.... mktap web failed, until I fixed #3. The program didn't report any useful error messages, it just repeated the help text indicating what types (web, ftp, etc) could be used for the command line. 5) Even after fixing #3, mktap web --static static, isn't working. 6) Under Windows, the default should be -n for twistd, and the PID file shouldn't be created. At least, as it is now. twistd just "stops", after it starts up and drops us back to the command line, and examination shows that the program isn't running. So at least as a workaround, the -n mode works.... I guest a better question is, has twistedmatrix been tested under Windows? It really seems to be unix centric in it's programming... - Benjamin

7 11

[Twisted-Python] twisted.enterprise docs suck
by andrew-twisted＠puzzling.org 19 Feb '02

19 Feb '02

Because the twisted.enterprise docs currently suck (i.e. I couldn't figure it out without help :), I decided to write a small tutorial that should hopefully explain things a bit better. I've put a copy up at http://poetry.puzzling.org/twisted_db.html. I'd appreciate any feedback anyone has, in particular: * Have I stuffed up anything major? * Is my terminology ok (e.g. "...dispatch queries to your ConnectionPool...") * Am I demonstrating "best practice" use of twisted.enterprise, or should I be structuring my code differently? If someone wants to place this (or an edited version of it) on twistedmatrix.com, or distribute it with Twisted, or both, they are welcome. Regards, -Andrew.

5 4

[Twisted-Python] Building Clients with Twisted ... and a gopher protocol.
by drt＠un.bewaff.net 18 Feb '02

18 Feb '02

I'm trying to build a non interactive client app with twisted. Let's assume it is a webcrawler. I'm not sure how to do that. Might be like this: Create a Loop Object which checks if the desired number of connections are running and if not create new ones an add them as connectors. Or should I rip out application.run() and write my own look to better handle the starting of new crawlers? Any suggestion would be welcome. To get a first experience with twisted I hacked a gopher protocol. Nothing fancy, nothing finished but any comments will be welcome. http://c0re.jp/c0de/misc/twisted-0.15.0-gopher.py drt -- teenage mutant ninja hero coders from da c0re - http://c0re.jp/ me - http://koeln.ccc.de/~drt/

3 2

[Twisted-Python] spread question
by Levi 17 Feb '02

17 Feb '02

I was playing with twisted.spread.pb and was wondering if it is possible for the server to pass an object to the client, and then call methods on that object and have the methods run on the client. I know you can (it's quit easy actually) have the client pass the object to the server which can then call it's methods, but it'd be easier to impliment what I've got in mind if the object originates at the server. -Levi

2 1

[Twisted-Python] Integrating web.guard and web.twcgi
by Moshe Zadka 14 Feb '02

14 Feb '02

OK, here's the deal. There's a Whole Lotta (TM) CGIs that use REMOTE_USER which the web server is supposed to set if the user sent the apppropriate HTTP authentication headers and the .htaccess (or whatever - in our case the .cred database) matches. Specifically, that's how roundup works. No, I'm not in the mood to integrate Roundup with Twisted, thanks for asking. So, any idea how to get web.twcgi to properly set REMOTE_USER based on a parent's .guard? I'm drawing a blank here... (perhaps a .guard should stamp the Request object as it is passing it through...)

1 0

[Twisted-Python] [Fwd: [Zope3-dev] Plans for z3.py]
by Itamar Shtull-Trauring 13 Feb '02

13 Feb '02

An interesting idea.

1 0

[Twisted-Python] HotShot vs. Old Profiler
by Itamar Shtull-Trauring 12 Feb '02

12 Feb '02

So, I tried comparing old profiler with HotShot (2.2's new C based profiler). I ran a simple twisted web server that serves from memory. Running with Python 2.2 I got 350 hits/second. time command says it takes 11 CPU seconds, and with time the hits/sec is a bit lower. Running with old profiler: ~70 hits/second, profiler said it took 2-3 CPU seconds to run (don't ask me why) Running with HotShot: 290 hits/second, it said it took 15 CPU seconds (11 of which were in doSelect on its own). Conclusions: HotShot is much much much much much much much better than old profiler We need to optimize doSelect something wicked HotShot still has bugs (it would fail occasionally) I have a command-line interface to hotshot at http://itamarst.org/software/

1 0

[Twisted-Python] [Fwd: Re: [Zope3-dev] Twisted as server]
by Itamar Shtull-Trauring 12 Feb '02

12 Feb '02

I'm going to read this in the morning when I'm more awake. Meanwhile someone else may be interested. Shane is discussing a mixed async/sync mode where threads can write directly to the socket.

2 2