Mailman 3 November 2003 - Twisted

[Twisted-Python] Scaling problem
by Uwe C. Schroeder Nov. 11, 2003

Nov. 11, 2003

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi guys, is there a way to have one service start several processes ? When using twisted everyone connecting to the server ends in the same process. This leads to "slowing down" when requesting information whenever a certain number of people go on. Each request takes some time to process and meanwhile the server is less responsive. I have one particularly expensive function that takes a bit. So if several people happen to use this function at a time the server slows down, up to where it takes a minute to complete. It's not a problem of system ressources (it's a quad processor xeon server with plenty of RAM and ultra 320 scsi drives running only this application and the postgres backend). The machine has a load of 0.2 while slowing down. Basically most of the machine is idle, since it only runs 2 processes and has 4 processors. Is there a way to have twisted do something like apache or postgres and split the incoming connection to several processes ? It's not a web application, I'm solely using pb. BTW: You might suggest to break down the expensive function into smaller pieces - no way. It has to run in one database transaction otherwise I might end up with a inconsistent database - no good in finance business. Thanks UC - -- Open Source Solutions 4U, LLC 2570 Fleetwood Drive Phone: +1 650 872 2425 San Bruno, CA 94066 Cell: +1 650 302 2405 United States Fax: +1 650 872 2417 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE/p+2zjqGXBvRToM4RAiLKAKDKjRWiezqXkSLQA0P/TCCcgYwckACePksV 3qxzLR3w0x3jKgKdWf1oNTA= =LzUM -----END PGP SIGNATURE-----

11 21

[Twisted-Python] A real Newbie question - how to get started
by Graham Hansell Nov. 11, 2003

Nov. 11, 2003

Dear All Twisters out there. I am a complete Twisted newbie (with a little bit of Python experience) and having just installed this (what looks like) a great package, I would like advice on my next step. I am looking to start doing simple (spider) based web clients but really would like a guide or tutorial before jsut spending hours pulling my hair out. Where do you advise me to go. I have had a look at the Tutorial on Finger but this seems like a major development in an area I don't even understand. Is there any specific newbie guides being built or assembled possibly by those on this list. Thanks for your help with something that could just change my life (well a bit of it anyway). Graham

2 1

[Twisted-Python] Re: request getUser()/getPassword() in Woven and resources...
by Stuart Hungerford Nov. 11, 2003

Nov. 11, 2003

Hi all, Was this question about getUser() etc ever resolved? > My question goes along the previous one in this thread. > How do you get the browser to prompt for a username and password? > My code is not woking and goes something like this: On every page request > a certain function is called, say genpage()... > > def genpage(request) > request.getUser() > request.getPassword() > > return "hello world" > > > -> Browser does not prompt though. (And this does not even involve woven) > > _stephan Cheers, Stu -- :: Stuart Hungerford (stuart.hungerford(a)anu.edu.au) :: ANU Internet Futures Group

1 0

[Twisted-Python] setUp and rendering
by vicky lupien Nov. 11, 2003

Nov. 11, 2003

I'm working on a web based application. I found that the function setUp is done before the rendering (wmfactory, etc...) of the page. But I need to have a function that is done after the rendering. Is there a function that I can override that do what I need. Vicky

1 0

[Twisted-Python] HTTP Basic Auth using cred, an example
by Itamar Shtull-Trauring Nov. 10, 2003

Nov. 10, 2003

I've put up some code showing how to use HTTP Basic authentication together with cred to guard a resource at http://www.twistedmatrix.com/users/roundup.twistd/twisted/issue393 You need to implement a realm that returns resources, and then wrap it in a portal with an appropriate username/password credential checker. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting

2 1

[Twisted-Python] Twisted Java
by Phil Christensen Nov. 10, 2003

Nov. 10, 2003

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 so, what's the chance of this ever supporting the new auth framework? i don't want to get into a "why use java, anyway?" debate, but i was just curious if there were any plans to continue supporting Java with the twisted framework. thanks, - -phil -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (Darwin) iD8DBQE/r/8cu8x7iJQ4TqURAugoAKCFBpft+N+IhkoSVk+b7+/+AqxJLQCfXrM/ wDDO+TcyH8kpsDANd58xhEE= =w56/ -----END PGP SIGNATURE-----

2 1

[Twisted-Python] pb help needed.
by Syver Enstad Nov. 10, 2003

Nov. 10, 2003

I have a bug in my woven application which manifests itself as win32select failing after doing many http requests. The error is the constant WSAENOBUFS and I it seems pretty likely that this is caused by some socket related resource leaking for each request I make. This hypothesis is further supported by observing that a host of pb.ClientFactory instances are stopped when shutting down the application after it crashes. This seems strange to me as I am thorough in shutting down the connector after every request is finished. Turning to that trusty friend printf style debugging, shows that the PBClientFactory is stopped correctly when performing requests to pages that use the referenceable and wait for it's deferred's to complete. There are two cases when it doesn't stop. The first is when I press reload quickly a number of times. Then I just see start messages because my request finish method is not called (it is a callback to the deferred returned by request.notifyFinish()). The second case is when I call disconnect on the connector before it's deferred has been realised. I see in the log the following pattern: Starting pb.PBClientFactory, disconnect and then the attached event, stopping factory is not called at all. After stepping I see that disconnect doesn't do anything when it is called before the connection has been realized. So I have a bug, and it's in my code. My question is how does one manage pb resources correctly and easily (I don't worry about performance just yet) when each user needs to have it's own connection?

2 4

[Twisted-Python] 1.1 Will Not Build Under Win32
by John Abel Nov. 10, 2003

Nov. 10, 2003

Hi! I noticed a couple of posts about 1.1rc's failing to build on Win32, due to the the installer not being able to find certain files (rpc.h, and portmap.h), but I've not seen any updates. This problem still exists with 1.1. Is there a way round it? Or, do I need to create a bug report for it? Regards John

6 7

[Twisted-Python] Quotient mktap problem
by rddda Nov. 8, 2003

Nov. 8, 2003

anyone here using Quotient? it builds, but on 'mktap quotient' i get an error where it tries to build 'atop'. it tries to import '_bsddb'. I am using bsddb3 - which i believe is the more recent version of bsddb. i can fix the import, but then i have problem with procs within. stops at DBEnv() which is supposed to be in bsddb3, but doesn't seem to be available. any thoughts/clues? twisted-1.1.0 quotient-0.7.0 on OpenBSD __________________________________ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree

2 1

[Twisted-Python] Re: [Twisted-commits] make log.err return the thing you pass to it
by Andrew Bennetts Nov. 8, 2003

Nov. 8, 2003

On Sat, Nov 08, 2003 at 04:51:25AM -0700, radix CVS wrote: > Modified files: > Twisted/twisted/python/log.py 1.52 1.53 > > Log message: > make log.err return the thing you pass to it Isn't this a fairly significant change to the API? Now doing deferred.addErrback(log.err) will log the error, and then log it again when the deferred is garbage-collected... I guess it won't actually break anything, just add some confusion to log files, but what's the new idiom for d.addErrback(log.err) # log and swallow any errors that happened going to be? -Andrew.

3 4