Mailman 3 November 2005 - Twisted

[Twisted-Python] integrating threadedselectreactor w/ pygtk
by David E. Konerding Nov. 22, 2005

Nov. 22, 2005

Hi, I am attempting to use the threadedselectreactor with pygtk. I didn't see a formula for using pygtk w/ tsr, so this I came up with the code below. From what I can tell, pygtk lets you call GUI functions from any thread so long as the gtk event loop lock is held, and there is no equivalent to "callFromThread". The alternative seems to be to generate a signal in the CallAfter callback and emit it, but I can't see why that would be necessary. Can anybody comment? from twisted.internet … [View More]

2 2

Re: [Twisted-Python] Unjelly - recursion limit reached
by Brian Warner Nov. 22, 2005

Nov. 22, 2005

>>> This is a pretty scary limitation. Is anybody working on a non- >>> recursion based version of jelly? Am I going to have to move to >>> Stackless Python? Sorry to respond to this so late.. my Twisted time has been awfully bursty this month. FWIW, newpb does not suffer from this sort of stack-frame limitation. Rather than using the C/Python runtime stack to keep track of its current place in the object graph, it maintains a "Slicer Stack" (and corresponding … [View More]Unslicer Stack on the receiving end), which is really just a list of Slicer/Unslicer instances. Each edge of the object graph path (from the top-most one-per-connection Root object down to the thing currently being serialized or unserialized) gets a Slicer/Unslicer on this list. Ridiculously deep object graphs (like the linked-list structure you describe) will consume heap memory for this list, but not stack frames. The maximum number of stack frames consumed is a constant, not affected by the shape or complexity of the data being serialized. (I haven't measured it but I'd guess it's less than a dozen frames). The runtime memory footprint is roughly linear with the depth of the object graph. Each Slicer contains a generator.. I don't know how much memory is consumed to maintain the state of a typical Slicer, but I'm not sure I'd want a million of them. > My object graph is a natural and clean data structure. I don't want > to lose that just because of limitations in my tools. PB is all about transferring your existing object graph cleanly and accurately. newpb just does it better than oldpb. Some of the limitations of oldpb (well, really just that 640k string limit) are there to minimize the damage if something goes really wrong (like you connect an HTTP client to the PB socket, and it tries to interpret "GET HTTP/1.1 index.html" as a bajillion-byte STRING token). newpb provides better ways to specify these restrictions (the whole 'schema' approach), so there ought to be fewer non-user-specified arbitrary limits. That said, some graphs are more efficient to transfer than others. A linked list of small integers will take about three times as many bytes to serialize as a flat list, and will consume a far greater amount of memory while doing so. The newpb Slicers are not clever enough to do tail-recursion. > Is this currently being addressed or in a road map? Am I unknowingly > volunteering for it? :-o Well, it sounds like you might be a great candidate for beta-testing newpb :). Keep tuned to this channel for an alpha release announcement sometime soon. cheers, -Brian [View Less]

2 1

[Twisted-Python] Newbie - how to modify my db-intensive app to use Twisted
by Frank Millman Nov. 22, 2005

Nov. 22, 2005

Hi all I sent a post recently which explains my situation. I will recap briefly. I am developing a multi-user accounting/business app, which is database intensive. My current version runs on each client, and makes a direct connection to the database. I want to change it to a server-based solution, where the server connects to the database, and each client connects to the server. Twisted lends itself to handling communication between the server and client, and to queuing all the database … [View More]commands. I have spent some time trying to figure out how to change my program to work in a Twisted fashion. I have made some progress, but now I have hit a stumbling block. I am sure that the problem is more to do with my approach than with Twisted, but it feels serious enough that I am leaning towards abandoning this approach and reverting to a multi-threaded server. I don't really want to do this, so I will explain my problem in the hope that someone can suggest an easy solution, which does not involve a total rethink of what I have come up with so far. As mentioned above, my program is database intensive. I have built in the ability to handle multiple table relationships, such as 'a customer record must have a valid branch code, currency code, salesperson code, etc'. If the user attempts to add or alter any of these fields, the program automatically checks that the new value exists as a foreign key on the relevant table. For this reason, and for a variety of other reasons, the program executes many SQL commands 'behind the scenes', without the application layer being aware of it. Each of these 'blocks' the program, but as the program runs on each client, it does not matter, as the user has to wait for a response before they can continue anyway. Moving this concept onto Twisted does not seem to work, as it can be running sessions for multiple users, and each 'block' will block the main loop, resulting in a major performance hit. The correct solution, as I understand it, is to change each SQL command to a deferred with a callback. I can do this on the inner layers, where the actual SQL command is executed. But I would also have to do it on each outer layer that calls a function which 'may' trigger a SQL command. There are many of these, and it goes to the heart of my approach of hiding the underlying database complexity from the application layer. Changing this feels like a daunting task, and frankly I would not know where to start. The alternative is to use a multi-threaded approach. In this case, I can keep my existing approach largely unchanged, as it does not matter if one thread blocks, the others will continue without being affected. I don't know if I have explained myself very clearly, but if anyone has any words of advice, I will be very grateful. Thanks Frank Millman [View Less]

4 6

[Twisted-Python] Get an object that provides an interface
by Gustavo Dagostin da Silva Nov. 21, 2005

Nov. 21, 2005

Hello, Is it possible to get an object that provides (from a class that implements) a interface without mentioning the class? In other words: class ISomething(Interface): def method(self): """ Do something """ class SomethingImplementor: implements(ISomething) def method(self): print "I exist!!" obj = ISomething() # and obj gets an instance of "SomethingImplementor", without # mentioning it. The documentation of component usage in twisted only mentions interfaces in the … [View More]

2 1

[Twisted-Python] unix signals and twistd
by Phil Christensen Nov. 21, 2005

Nov. 21, 2005

Hey all, I've got a Twistd application that maintains a datafile separate from the serialized app stored in the .tap file. I've set it up to automatically load this file at startup, and save it at shutdown, but I'd like to implement a way to do incremental saves while the server is running. The ideal setup would be to listen for a signal; I could then either run a cron script, or use some other external app. I've listened for signals in other python apps, but I thought I'd check … [View More]

1 0

RE: [Twisted-Python] Newbie - how to modify my db-intensive app touseTwisted
by Henning.Ramm＠mediapro-gmbh.de Nov. 21, 2005

Nov. 21, 2005

My actual project is similar: Several different apps need a database connection, but instead I let them connect via Twisted PB to a Proxy that keeps the only db conn. I fixed most of my troubles using Interaction and 'lock tables' in them (MySQL or at least the Python driver doesn't support real transactions). example: # self.dbpool = adbapi.ConnectionPool(self.dbApiName, **Conf) def remote_deleteOld(self, days=21): try: d = self.dbpool.runInteraction(self._deleteOld, … [View More]

1 0

[Twisted-Python] Getting started with twisted
by David Pratt Nov. 21, 2005

Nov. 21, 2005

Hi. I am brand new to twisted and hoping to get my feet wet but need some help identifying the right products to use. I am also hoping someone can help point me to some code or an app that might have similar functionality to look at to see how things work. I am currently building a pythonwx app and want twisted to communicate with a web server over ssl in REST style (HTTP 1.1) with PUT, GET etc. I am interested in sending and receiving requests in xml to synchronize web data with … [View More]

1 0

RE: [Twisted-Python] UnicodeDecodeError with twisted.conch.ssh client
by Henning.Ramm＠mediapro-gmbh.de Nov. 18, 2005

Nov. 18, 2005

>> I'm glad I don't need any chars that aren't 7-bit ASCII. >> Perhaps conch could be teached to handle Unicode strings, too? > >I think you are a bit confused about the way Unicode works. A string >object is NOT "7-bit ASCII". Sorry, I really should know, using a lot of encodings all over the place... But anyway, it would be nice if conch would handle unicode objects, too. Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

3 2

[Twisted-Python] Status of Deferred cancelation
by Maciek Kaminski Nov. 17, 2005

Nov. 17, 2005

What is the status of issue 990 (http://twistedmatrix.com/bugs/issue990)? Is it going to be merged into trunk? Maciej Kaminski

1 0

RE: [Twisted-Python] UnicodeDecodeError with twisted.conch.ssh client
by Henning.Ramm＠mediapro-gmbh.de Nov. 17, 2005

Nov. 17, 2005

>> I use password auth, and both user and password use only >7-bit ASCII chars. > >Did you pass a string or a unicode string? I suspect this code path >would only be happy with regular strings (i.e. isinstance(s, str) == >True). Thank you for the hint - 'str' at every place where data is sent helped. I'm glad I don't need any chars that aren't 7-bit ASCII. Perhaps conch could be teached to handle Unicode strings, too? Best regards, Henning Hraban Ramm Südkurier … [View More]

2 1