Mailman 3 January 2006 - Twisted

[Twisted-Python] IMAP4 plain authentication
by David Watson Jan. 23, 2006

Jan. 23, 2006

Hi, My first post to this list, so apologies in advance if this has already been discussed. I`ve been experimenting with some of the sample code from Abe Fettings (very good) Twisted book and I`m a little stuck with one IMAP authentication method. The code is: from twisted.protocols import imap4 from twisted.internet import protocol, defer class IMAPFolderListProtocol(imap4.IMAP4Client): def serverGreeting(self, capabilities): … [View More]self.registerAuthenticator( imap4.CramMD5ClientAuthenticator(self.factory.username)) self.registerAuthenticator( imap4.LOGINAuthenticator(self.factory.username)) authenticating = self.authenticate(self.factory.password) authenticating.addCallback(self.__loggedIn) authenticating.chainDeferred(self.factory.deferred) def __loggedIn(self, results): return self.list("", "*").addCallback(self.__gotMailboxList) def __gotMailboxList(self, list): return [boxInfo[2] for boxInfo in list] def connectionLost(self, reason): if not self.factory.deferred.called: # connection was lost unexpectedly! self.factory.deferred.errback(reason) class IMAPFolderListFactory(protocol.ClientFactory): protocol = IMAPFolderListProtocol def __init__(self, username, password): self.username = username self.password = password self.deferred = defer.Deferred() def clientConnectionFailed(self, connection, reason): self.deferred.errback(reason) if __name__ == "__main__": from twisted.internet import reactor import sys, getpass def printMailboxList(list): list.sort() for box in list: print box reactor.stop() def handleError(error): print >> sys.stderr, "Error:", error.getErrorMessage() reactor.stop() if not len(sys.argv) == 3: print "Usage: %s server login" % sys.argv[0] sys.exit(1) server = sys.argv[1] user = sys.argv[2] password = getpass.getpass("Password: ") factory = IMAPFolderListFactory(user, password) factory.deferred.addCallback( printMailboxList).addErrback( handleError) reactor.connectTCP(server, 143, factory) reactor.run() However, when I try to connect to my IMAP4 server using this code, I get: Error: No supported authentication schemes available: Server supports ['PLAIN'], client supports ['CRAM-MD5', 'LOGIN'] I can`t seem to find a suitable PLAIN method in IMAP4client, so before I re-invent the wheel, is there a standard Twisted way to handle this situation (or am I simply missing something obvious)? Thanks, David [View Less]

3 2

[Twisted-Python] CPUShare-Twisted
by Andrea Arcangeli Jan. 22, 2006

Jan. 22, 2006

Hello everyone, if you've pending patches that have not been applied or you've no time for unittests until your project works, you can consider switching to CPUShare-Twisted. Mercurial allows distributed development so we can merge our branches against each other with very little effort. If you've suggestion for betters names you're welcome. I could have done it over sourceforge but they lack mercurial AFIK and since I've the setup on cpushare.com I was quick to add Twisted to it. We can move … [View More]it elsewhere later if needed (especially if it consumes too much network bandwidth ;). I have no interest to maintain twisted except for the fact I'm using it in semi-production and I plan to use it in production, and so I need it to be stable, and with all possible bugfixes and features applied without formalities. I assume other business usages of twisted may have the same needs, so perhaps we can join our efforts with this project. My choice was to either dump twisted or to maintain it locally, and so I clearly have an interest to make the CPUShare-Twisted branch public and to include other people fixes and features (I was already maintaining it over the last year except it was just a patch with fixes that I failed to push in SVN, and not a real fork). You can find the quickstart, basic objectives (including the not trying to reinvent the wheel) and mailing list details on the project page: http://www.cpushare.com/twisted I'll keep tracking SVN. I'll also keep submitting patches upstream since one can always hope for the best. Also note, over the last year I've fixed at least one bug in core twisted basic protocols that would have never been found with any unittest out there. Careful auditing and reading code and filtering of the patches, and thinking deeply about the design before writing code (to write the code in a way that won't break easily over time), is much more important than spending time on unittests. Unittests still makes perfect sense after stuff is included and works in basic testing, but they should be separated from the logic of committing valid patches to the tree. NOTE: before clicking on that page you may want to also answer these questions: 1) is python much better than ruby and all other language on earth? 2) is twisted much better than any other framework to write network apps? 3) are twisted and python fast and scalable enough for all applications? 4) is the single threaded model scalable enough for all applications in smp? 5) is async programming using deferreds simpler to code for a webserver that is stateless and that only does sql queries over the network? 6) would you rather prefer to go broke than to use code "not-invented-here" or not written with python and twisted? (of course the python interpreter the c compiler and the underlying operative system are magically excluded from the not-invented-here clause for whatever unknown reason) I'm welcome anybody who wants to join cpushare-twisted, but if the answer of _any_ of the above questions is "yes", I think you may be better off ignoring this email. [View Less]

5 8

[Twisted-Python] Re: [Twisted-commits] r15665 - Rewritten endpoints (TCP and UNIX) with unittests
by Jean-Paul Calderone Jan. 22, 2006

Jan. 22, 2006

On Sun, 22 Jan 2006 02:47:24 -0700, David Reid <dreid(a)wolfwood.twistedmatrix.com> wrote: >Author: dreid >Date: Sun Jan 22 02:47:23 2006 >New Revision: 15665 > >Added: > branches/dreid/endpoints/twisted/test/test_endpoints.py >Modified: > branches/dreid/endpoints/twisted/internet/endpoints.py >Log: >Rewritten endpoints (TCP and UNIX) with unittests This branch seems to have no ticket number, so I'm commenting here. twisted/test/test_endpoints.py: … [View More]Several test methods of TestEndpoints define nested functions which discard the Deferred returned by a Port's stopListening method. These need to be waiting on that Deferred. Why do some tests bind to 127.0.0.1 and some to 127.0.0.2? None of the tests assert that the protocol/port is hooked up to the right address, either on the local or remote side. None of the tests exercise client-side binding. The TCP and UNIX tests are basically identical, except for the endpoint class being used. This is a lot of unnecessary code duplication. Implement the tests with the endpoint class parameterized and then call them all twice, once with TCPEndpoint, once with UNIXEndpoint. Lots of trailing whitespace throughout the file. twisted/internet/endpoint.py: Interface definitions have docstrings - they don't need "pass" (pass is only required to satisfy grammar requirements for /some/ statement, and the docstrings do just fine for that). The docstrings of neither IClientEndpoint.connect nor IServerEndpoint.listen define the error type which the Deferred may fail with. Is it really true that IServerEndpoint.listen will never callback its Deferred? The tests seem to think otherwise. Typo in IServerEndpoint.listen's docstring - "incomfing". Also, these interfaces belong in twisted.internet.interfaces. _EndpointClientFactory.buildProtocol should not be using callLater(0). There is no guarantee that will be the correct time to deliver notification, and it is unnecessarily inefficient. Instead, you probably want something like a ProtocolWrapper from twisted.protocols.policies. self.callable(addr) in the same buildProtocol is user-code. It would probably be useful for exceptions it raises to cause the Deferred associated with the factory to errback. Currently, errors it causes will be logged and the Deferred passed back to the application will never fire or errback. Similar comment for _EndpointServerFactory.buildProtocol's invocation of self.callable(). TCPEndpoint.listen - try/bareword except? Why? This looks like a perfect place for twisted.internet.defer.execute(). AddressToEndpoint looks like it should just be a function that returns either a TCPEndpoint or a UNIXEndpoint. Trailing whitespace *and tabs* throughout. Jean-Paul [View Less]

2 1

[Twisted-Python] Deferred Groups?
by Duncan McGreggor Jan. 22, 2006

Jan. 22, 2006

I have a question about an approach I used... I'm worried that I've over-worked it and have over-looked a more elegant and standard solution. I have the need to fire off network connections in groups. Deferreds added to a DeferredList don't fit the bill (because there's no control over all the deferreds in the list). As an example, if you wanted to make a whole batch of concurrent connections, but didn't want to incur the overhead of firing off more than 20 simultaneous connections, you'… [View More]d split your destination hosts up into groups of 20. As a group was completed, a callback could fire off the next group, etc. What's more, I didn't want to put this kind of control in a factory or a protocol. In my mind, that didn't seem the proper place for it... I have some test code that generates the following output. I created a class that instantiates a ClientFactory instance and then manages a set of deferreds and deferred lists. Here's some sample output: Entered main run() loop. deferred #1 is connecting to adytum.us... deferred #2 is connecting to del.icio.us... deferred #3 is connecting to google.com... Left main run() loop. deferred #1 connected! deferred #2 connected! deferred #3 connected! Finished with Group 1 deferred #4 is connecting to last.fm... deferred #5 is connecting to washingtonpost.com... deferred #6 is connecting to yahoo.com... deferred #5 connected! deferred #6 connected! deferred #4 connected! Finished with Group 2 deferred #7 is connecting to microsoft.com... deferred #8 is connecting to amazon.com... deferred #7 connected! deferred #8 connected! Finished with Group 3 Finished all groups. Here are some pertinent parts of the code (full pasting: http://pastebin.adytum.us/39/1 ): class DeferredsByGroup(object): [...] def initiate(self): self.groups = self.splitLoad(self.hosts, self.hosts_per_group) # get the first group and let the callback handle the next one host_group = self.groups.next() dl = self.setupGroupDeferred(host_group) dl.addCallback(self.getNextGroup) dl.addErrback(self.noMoreGroups) def splitLoad(self, host_list, per_group): group_count, remainder = divmod(len(host_list), per_group) if remainder: group_count += 1 for i in xrange(group_count): yield host_list[i*per_group:i*per_group+per_group] [...] def getNextGroup(self, null): group = self.groups.next() dl = self.setupGroupDeferred(group) dl.addCallback(self.getNextGroup) dl.addErrback(self.noMoreGroups) def setupHostDeferred(self, host): self.host_counter += 1 name = "deferred #%s" % self.host_counter f = Factory(host, name) d = f.deferred d.addCallback(self.updateData) return d def setupGroupDeferred(self, group): self.group_counter += 1 # iterate through each host in the group deferreds = [ self.setupHostDeferred(host) for host in group ] # set up this group as a deferred list dl = defer.DeferredList(deferreds) dl.addCallback(self.handleGroup) dl.addErrback(self.handleFailure) return dl [...] And this is instantiated in the following manner: hosts = [ 'adytum.us', 'del.icio.us', 'google.com', 'last.fm', 'washingtonpost.com', 'yahoo.com', 'microsoft.com', 'amazon.com', ] hosts_per_group = 3 runner = DeferredsByGroup(hosts, hosts_per_group) runner.initiate() reactor.run() [View Less]

3 5

Re: [Twisted-Python] Re: [Twisted-web] CPUShare-Twisted
by glyph＠divmod.com Jan. 22, 2006

Jan. 22, 2006

On Sun, 22 Jan 2006 07:52:18 +0100, Andrea Arcangeli <andrea(a)cpushare.com> wrote: >On Sat, Jan 21, 2006 at 11:55:44PM -0500, glyph(a)divmod.com wrote: >> If you've pending patches that have not been applied, would you please >> consider instead to agitate for those patches on the mailing lists, and add > >These are the very old ones (ignore the web2 part that is recent). > >http://www.cpushare.com/hg/Twisted/?cs=400da64bd5a6 >IIRC you said that (some … [View More]stuff) I'm sorry I was unclear, and you typed all that stuff to no effect. Discussions that draw attention to unapplied patches should really refer to bug URLs in the tracker. If they don't, nobody can tell how long the patches have been languishing, who was supposed to apply them, or why they weren't applied, unless there are links to dozens of previous mailing list messages in each post. Also, summaries of these discussions should be attached to the ticket by the reporter or the maintainer, if they advance the issue at all. Overall, without some support from the tracker, we just don't know whether the issues are really stuck on a serious problem, or whether someone has just become confused about what is required to make progress on the bug. Even the absence of information on a bug can be useful. "Why hasn't anybody replied to this for 6 months? Was there some discussion on IRC?" can lead someone to post a helpful summary of current status if they have a recollection of where it did end up... sometimes, the bugs have even been fixed, and nobody has noted that fact. >Sure I understand (twisted devs will work from bugtracker) Thank you. >My developmnt is generally test-driven. Maybe, but it sure doesn't sound like it. If your development is test-driven that means you are used to writing unit-tests *first*, not hacking in a fix and testing *later*, which is what you have repeatedly suggested. "TDD" is not the same thing as "unit testing". FWIW Twisted does not require TDD. I do not personally do TDD much of the time. I think tests need to be added before a feature is added, but I don't always have a clear enough picture of what the code will look like to write tests, before I've tried to write the code. >I'm only opposed to unit-test >mandatory development to fix bugs or add new features. Writing unit-test >isn't the only way to test code. It's nice to have a unit test, even a >simple one, but it shouldn't be mandatory. Hmm. "Unit Test Mandatory Development" - UTMD. That sounds like a good acronym. I think I will use it in the future. One productive use of this set of threads is that I've repeated our testing policy - UTMD - in a few different ways. I'm not sure that I can explain it to you (you seem to have some resistance to understanding) but maybe this will be useful to others. Whether unit tests *should* be required is a discussion that requires some kind of value system. What's good, what's bad, etc. Right now unit testing *is* required. There is a reason. Unit testing is not the objective here, rather, requiring unit tests provides a mechanism to satisfy a greater requirement. If you can suggest a better way to achieve that requirement then perhaps we can discuss other strategies. Twisted is used by lots of different people in lots of different ways. Before test requirements were adopted, it was quite common for a developer who fixed a bug in one system on one platform broke another system on another platform. We are trying to improve Twisted, and such changes are not improvements. Such changes simply muddle around the set of places and times where Twisted works correctly, they don't enlarge it. Conceptually this makes sense. Software is extremely complex, etc. If you want to fix a bug, you need a way to verify that it doesn't introduce a new bug, or at least a way to verify that *previously*-verified behavior is still working as expected, in previously-verified environments. That is the goal that unit tests serve. Without unit tests, *we do not know* whether a particular change will continue to work in the face of future changes, or if it broke past changes. We can reason about the breakage on past changes, at least, but to think that we can actually understand the impact of the patch on a system the size of even something modest like Twisted is hubris. Every software project has embarrassing releases that break obvious frequently-used functionality - even projects with *better* testing track-records than Twisted has. I believe Linus coined the term "brown paper release", for the brown paper bag you have to wear on your head to prevent being recognized after such an event. However, we can't even attempt to reason about future changes, because we can't possibly consider them when reviewing a current change. Is it likely that other things might break this later? How could we possibly know, without a way to accurately predict the future? Consider that different people review different patches to Twisted at different times, and they have different skill levels. I have written and read a LOT of Twisted code, and I doubt that even I understand 90% of it. This partial ignorance makes reviewing past changes a lot like reviewing future changes - a change might break something in a Twisted subsystem the reviewer didn't even know about. Of course, unit tests are imperfect too; we don't have 100% coverage, and even if we did, we wouldn't have 100% coverage in combination. Still, they are the best option that we know about. Can you suggest an alternative to unit tests that would accomplish this goal of providing some level of knowledge as to whether Twisted is probably improving or just changing randomly between releases? Here are some objections which don't really address the question, just so I can head these off before they are asked. Andrea - some of these are quite silly and I don't mean to imply that you are necessarily going to ask all of these questions, but I am now writing this for a general audience, and these *are* questions others have asked me. "but, my changes are so simple, what could they break" There is a story about a butterfly and a hurricaine that you need to read. Simple changes can have complex effects that break things horribly. "not EVERYTHING in Twisted has to be tested. some easy stuff could break, it's not likely since it doesn't change too often, and you could just do another release" This leads to a game of whack-a-mole. One bug pops up, you smash it down. That makes another bug pop up. You smash it down. The whole time, you feel like you are being very productive, because you are fixing all these bugs! Really though, you're just making the same two motions over and over again between different releases. 'back and forth' is an oversimplification, of course. In reality the cycle probably takes hundreds of releases and goes through dozens of features in various combinations. Nevertheless, things get fixed, and other things break. "you can just test it manually" No, you can't. There is a HUGE combinatorial explosion of work involved - did you test it with every revision? Did you test it on Windows? Did you test it on a slow machine? Did you test it with Python2.3? Did you test it with Python 2.3 - on Windows? Did you test the OTHER thing with Python 2.3 on Windows? Did you test the other thing with Python 2.4 on Windows? What about FreeBSD? What about QNX? What about AIX?!??!? What about Linux 2.4? Right now this matrix has over 20,000 units of work in it, just based on the current buildbots and the tests they're supposed to be running (and as you can see on the buildbot page, we are still trying to get the EXISTING features into shape, it is no wonder we don't want to rush to accept new ones quickly). Every unit test that is added does the work of manual testing on 9 configurations on 4 platforms every time someone does a commit, which is several times a day. Do the math. Even replicating our *current* automated testing with a manual replacement would take something like a million dollars a week in tester salaries, if we were to pay them. Open source does produce some really good free labor, but not NEARLY that much, and Twisted is a small project besides. "other projects release untested code, why can't you" Didn't your mother ever tell you "If Billy jumped off a bridge, would you jump off a bridge too?" (Wait, am I being "Mommist" now? ;-)) "Some projects separate testing from development, such as the LTP" Forgetting any flames about Linux's stability for a moment (let's be fair: for such a large piece of software that changes so fast, it's amazingly robust), this is the brute force approach. LTP is sponsored by IBM and SGI. It is a HUGE project - at one point, I visited the LTP booth at LinuxWorld. Twisted does not have a booth at any conference. Just the test project on linux has ten times as many people as Twisted. With fewer resources, we have to have a better strategy, otherwise we will not find any bugs. And by the way, even with all that testing that the LTP does, sometimes Twisted finds even regressions in Linux, remember? :) >> [..] I am sure that it will be full of bugs. > >Time will tell. Code that has passing tests almost by definition has fewer bugs than code which does not have passing tests. At the very least, it has more bugs in an unresolved quantum state, because you haven't observed them - so the probability of actual bugs is higher. They whole point of this fork is that you want to put more bugs in and don't want to take the effort to verify that they won't be introduced. I am not making a prediction about your skill, I am making an observation on the nature of the project. >Since you made your prediction I'll make mine. I'm sure >axiom is wasted time in its current API (at least as far as twisted is >concerned). Axiom was developed for a specific application. It is not appropriate for everyone. Some people like it, some people don't. The ones who like it can go ahead and use it. >I don't see how you can advertize axiom saying "We do plan to add some >later, and perhaps also support other databases in the future.". >Sure you can add it, but if you do it, the whole axiom api will fall apart >unless you want to make synchronous queries over the network. The only >two deferreds you have are during startup and in the testsuite, just >grep for the word Deferred. I think you mean we are going to add more Deferreds later? There will be a different operational mode for 'transact' which returns a Deferred; the exact spelling hasn't been determined yet, but surely the semantics for that mode will be different and it will not work with all existing axiom code. (Of course, existing axiom code will not invoke that mode, so it will continue to work side-by-side code which does use it happily enough.) >Making synchronous sql queries in the >twisted async model is unacceptable for anything serious. Everyone is welcome to think that Axiom is not very serious. I am not a serious person. Twisted, in fact, is not serious, as you yourself pointed out - Twisted.Quotes proves it. >Infact even >sqllite queries are obviously unacceptable once the db grows beyond the >size of the cache Only if you're not using an index. An implication of the current axiom model is that you had better be damn sure that you've got indexes in the right place. >(and for sure you can't scale the queries over >different servers to have more ram-cache when using sqllite). Who says your application has to scale by doing multi-machine queries within a single database? Google's search team doesn't (at least according to the papers they've published), and I think they know something about scale. You could apply their same general technique, or the one Netezza uses, to Axiom: either (google style) segment your application data into logical groups, and have high-level queries only talk to appropriate nodes, or (netezza) make null queries really fast (netezza has crazy stuff for this, I think, but Axiom would just use indexes), then run every query on every node in parallel, return results to an aggregating node. It currently requires extra work, but in our application at least, you rarely want to query the whole universe. At some point I imagine we will add support for that. Anyway, I hope that indicates that I have considered the issue of scale a little bit. Right now scale is not my biggest concern but I am confident we can handle it. If Divmod were to have a potential customer approach us and say, "we want Axiom to scale to a hundred million node cluster, and we have a very complex application, and we want to get to that scale within a month. can we do that?" I'd say no. That doesn't mean it will never scale. If a potential client were instead to say, "we have a six month timeframe, and such-and-such budget, can we launch something and eventually scale to a billion users with axiom?" I'd say yes, probably. Depends on our allocation in the budget, of course ;-). The application would have to be aware of scaling issues in its own code. It turns out that you always do anyway. The "sufficiently parallel cluster" of RDBMS machines is like the "sufficiently smart compiler" that LISP people talk about. The existence of projects like memcached indicates that there is a general problem with the idea that you can just use one giant database and scale it up and up and up. >Ironically axiom current api would have a chance to work well with >threads, with twisted single threaded async model not. Database-managed concurrency is not the same thing as shared-state threading. You might superficially implement database-managed concurrency with shared-state threads for convenience, but the whole programming model is different - most importantly, you don't touch locking from application code, ever. I need to write a blog post about that or something, but I doubt I will do it justice. There are easily 3 CS Ph.D dissertations in that topic and I am not the person to write them. In fact, you can use the current Axiom API with threads, mostly, and it works about as well as most other Python ORMs. There are some concurrency issues (also present in several other systems) which I'd like to fix before that is a suggested use though. >There are good python storage packages to use with twisted and threads ^ That word right there is debatable. I've used, and even even written, a few of those and I'm not happy with them. Again - for a particular application. Divmod's application is very ambitious and it is not clear that Axiom is the best possible approach for it. But it seems to be working out OK. >I can't imagine why you insist on making your inferior >solution with a design that can't work well with twisted. Your point: Axiom does not work well with Twisted. Your evidence: you do not think Axiom works well with Twisted. This is a rhetorical fallacy. It is called a "circular" argument. I have a favorite rhetorical fallacy too, but it's not this one. My point: Axiom works great with Twisted! My evidence: There are about 30 people in #divmod who think Axiom works great with Twisted. This rhetorical fallacy is called an "ad populum" argument, and it's still wrong, but it has a bit more heft to it. >I'm feeling guilty for risking hitting the harddisk for a few msec when Hard disk? You mean "filesystem", surely. Linux decides to put things which are in RAM onto disk and which are on disk into RAM all the time. >people clicks on the mailing list archives, and infact I keep two >webservers exactly to avoid hurting the scalability of the ssl one. Aah. And how do you do that? Inter-process communication. Divmod does have problems that require extremely low-latency response and concurrency, but it turns out that these are the exception, not the rule. Allen Short is currently putting Voice-over-IP audio playback into a subprocess so that performance does not suffer from delays which are perfectly acceptable for the interactive web app (everybody has to hit the database to display these web pages anyway, and there is only one disk, so the performance is not going to change if it's in parallel) but are excruciatingly long for delays between sound samples. We also have plans to scale our service up amongst large groups of commodity machines, with separate, small axiom databases running on each one. Axiom databases do scale up in size better than you have suggested (I have tested very responsive query and insert performance up to ~5G databases so far, and there is no indication it would slow down significantly anywhere up to a terabyte) but you are definitely not going to be able to run a million-subscriber service out of a single Axiom database. You make spawning a second webserver sound like a really serious problem. It's not. When your application need parallelism, to maximize utilization, spawn a process. Sometimes it's OK to block. At the beginning of the project, I thought very much like you are suggesting, absolutely terrified of blocking for any reason, reasoning about what the kenel would do, about what my program would do, but without any solid performance numbers. I got over it and wrote some simple code that stored and retrieved objects with SQLite, then did some basic measurements and discovered that it was actually adequately fast. >> Perhaps instead you could change the version >> from SVN-Trunk to 'HG-CPUShare', so that the CPUShare-ness of the code is > >Ok, I'll make this change right away. I already did that for the web2 side. and thanks again for that. >This is a very fair requirement (changing the version is trivial). >However I don't see much point in changing the commands if the module >name is the same. Either I change both, or none. Definitely the version is the most important thing. I suggested the command-names because that way pasted shell output without tracebacks would also be visibly identifiable without having to say 'please run xxx --version'. If you don't think that would be appropriate, I don't mind. [View Less]

1 0

[Twisted-Python] TimeoutError in BaseConnector
by Duncan McGreggor Jan. 21, 2006

Jan. 21, 2006

Hey all, I have an issue with the following in t.i.base.BaseConnector: if self.timeout is not None: self.timeoutID = self.reactor.callLater(self.timeout, transport.failIfNotConnected, error.TimeoutError()) Might it be more useful and less misleading to describe this a little more? I was quite perplexed at an unexpected timeout error, until I realized it was being generated by the connector. I kept getting this message: User timeout caused connection failure. While … [View More]

2 2

[Twisted-Python] Problems with TwistedSumo-2005-11-06 on WinXP
by Raúl Alexis Betancort Santana Jan. 21, 2006

Jan. 21, 2006

Hi all, I'm trying to run an app developed on Linux Debian 3.1r1, with python 2.4 and twisted 2.1.0 on windows XP with python 2.4 and TwistedSumo-2005-11-06 (Twisted 2.1.0), and I get the followin error ... c:\python24\lib\site-packages\twisted\python\context.py:74: DeprecationWarning: threadable.whenThreaded is deprecated. Use application-level logic instead. threadable.whenThreaded(initThreads) Traceback (most recent call last): File "prueba.py", line 11, in ? from twisted.internet … [View More]

2 2

[Twisted-Python] All night Twisted sprint @SuperHappyDevHouse.
by David Reid Jan. 20, 2006

Jan. 20, 2006

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 After attending several (read: all of them) SuperHappyDevHouse[1] events, with several other Twisted developers, I've decided to organize an official Twisted sprint for the January 28th event. In case you're not aware SuperHappyDevHouse is an all night Hack-a-thon organized by Jeff Lindsay[2] and David E. Weekly[3] (of the Community Colo[4]) in the Bay Area, CA. It's typically filled with a lot of Web2.0 startups, (and the companies that buy them … [View More]

1 0

[Twisted-Python] Twisted and SMTP Footer
by Alfred Chiesa Jan. 19, 2006

Jan. 19, 2006

Hello All! Quick question: Can I use Twisted with an existing sendmail server to add a footer to every Email going through said SMTP server? Should regular old python be the best route? Thanks!

1 0

[Twisted-Python] twisted.protocols.dns NAPTR extension question
by Andrzej Leszczynski Jan. 19, 2006

Jan. 19, 2006

Hi, I have a question regarding extension twisted.protocols.dns by NAPTR DNS Resource Record (RFC 2915). Adding the new class Record_NAPTR and the NAPTR (35) constant is trivial however when it comes to decoding the names "flags", "service", "regexp" and "replacement" I realized that "class Name" attempts to read all the names in the byte stream and concatenate them together with "." used as a separator. I work on my copy of dns.py so I hacked it up a little and the code looks like here: … [View More]

1 0

Re: making sqlalchemy work with twisted (was Re:[Twisted-Python]SQLAbstraction Layer_
by Paul G Jan. 19, 2006

Jan. 19, 2006

----- Original Message ----- From: "Michael Sparks" <Michaels(a)rd.bbc.co.uk> To: <twisted-python(a)twistedmatrix.com> Cc: "Paul G" <paul-lists(a)perforge.com> Sent: Thursday, January 19, 2006 8:01 AM Subject: Re: making sqlalchemy work with twisted (was Re:[Twisted-Python]SQLAbstraction Layer_ > On Thursday 19 Jan 2006 03:55, Paul G wrote: >> ----- Original Message ----- >> From: "Jean-Paul Calderone" <exarkun(a)divmod.com> >> > On Wed, 18 … [View More]Jan 2006 21:15:28 -0500, Paul G <paul-lists(a)perforge.com> >> >><exarkun(a)divmod.com> >> >>>>new: attribute access -> sqlalchemy accessor -> sqlalchemy >> >>>>sqlengine -> >> >>>>async call into dbapi with deferToThread-> control returned to >> >>>>reactor -> another coop thread gets control >> >>>>... async dbapi result handler -> return to sqlengine -> return to >> >>>>accessor -> return attribute > ... >> >>>Unfortunately, this cannot be implemented in CPython without going >> >>>to extreme lengths. > ... >> > Extreme lengths are extension modules that move >> > pieces of the C call stack around. See Stackless Python and the > >> > greenlets module. >> >> yeah, i've looked at those in the past. are you saying they are the only >> way to do this and generators won't work? > > Generators are really only single layer, unless you start manually > chaining > them which requires changing all the other bits of code. i'm familiar with what generators are/do ;) my question was really meant to ask: have you already tried/thought in detail about something similar with generators and found that it can be done, can't be done or can't be done cleanly enough to make any sense. > You want to suspend > execution deeper in the call stack which means you'd need Greenlets or > Stackless (or *really* sod about nastily with threads...). well, the problem with generators is that there's no way to yield several frames up/down (depending on your arch ;) the stack. so you're talking trampolines and possibly nasty stack growth if you have to recurse too much before you can unwind etc. i've started hacking this up, but it's giving me a headache. > At that point > you're into whether you think that's a good idea or not. > > I don't know other people's feelings on the matter, but to me it just > feels > wrong in a system that has clear handoff generally speaking to have some > very hidden handoff as well. At minimum it implies an understanding burden > on the maintainer. At worst you now have the potential to *really* screw > things up. i'm not arguing for this to be done in twisted, merely seeing if it makes sense to do locally. -p [View Less]

1 0

[Twisted-Python] SQL Abstraction Layer
by David Blewett Jan. 19, 2006

Jan. 19, 2006

I was reading through the enterprise module in twisted, and then I came across this website: http://www.penguin.cz/~ondrap/sqlabstr.php It makes using SQL in Python much more readable. I was thinking that it would be nice to incorporate some of the functionality of this module. It has more than would probably be necessary (creating classes dynamically from the schema in the underlying database for example), but I think it would make a nice addition to the enterprise module. Any thoughts?… [View More]

7 17

[Twisted-Python] Weekly Bug Summary
by James Y Knight Jan. 18, 2006

Jan. 18, 2006

Bug summary ______________________ Since 2006-01-11 06:00:01.820000: Bugs opened: 12 Bugs closed: 3 Total open bugs: 533 (+9) Urgent: 9 (+2) Bug: 254 (+5) Feature: 182 (+1) Wish: 88 (+1) New / Reopened Bugs ______________________ ==== Urgent ==== [TRIAL] trial misreports pyunit test names (6 hours ago) http://twistedmatrix.com/bugs/issue1425 opened by itamarst [CORE] Running 'setup.py all install' inside TwistedSumo-2005-11-06 fails to install … [View More]

1 0

[Twisted-Python] Documentation virtual sprint, TODAY, Sat 14th January
by Mary Gardiner Jan. 17, 2006

Jan. 17, 2006

Hi all, Today (well, soon for non-Australians) is of course the day of the documentation virtual sprint. What are we doing? We're reviewing the existing Twisted documentation from both user and expert points of view to highlight obvious areas of improvement. How can you help? Read Twisted documentation and make suggestions for improvement. We need reviews from both Twisted users (the intended audience of the documentation) and Twisted experts (who are the people who, if the docs aren't good, … [View More]

5 8

[Twisted-Python] simple dns server with twisted.names
by Yves Glodt Jan. 17, 2006

Jan. 17, 2006

Hi, I have read about twisted.names and I guess it can help me in what I have to achieve.. I have a database which contains a list of ip-addresses and corresponding hostnames. Now, how could I do to, e.g., write a daemon which would, on dns queries, query the database to resolve the name? best regards, Yves

4 4

[Twisted-Python] Re: [Twisted-commits] r15582 - I meant change it like this, not like that.
by Itamar Shtull-Trauring Jan. 16, 2006

Jan. 16, 2006

On Sat, 2006-01-14 at 13:49 -0700, James Knight wrote: > +from twisted.web2 import That module name violates the coding standard, btw.

1 0

[Twisted-Python] Simple OSCAR Bot
by pokerface Jan. 16, 2006

Jan. 16, 2006

Greetings, list. I'm attempting to wrap my brain around Twisted for the purposes of creating a simple OSCAR bot. The purpose of this bot will be to send a broadcast announcement to a mutable list of screennames to help with admining gaming servers. Every example I've seen of OSCAR bots has the bot log in and respond to all sorts of messages that it gets sent, staying logeed in the entire time. My needs are simpler; the bot need only log in, send its messages, and log right back out. From … [View More]

1 0

[Twisted-Python] hobart meatspace sprint - the final hurrah
by Tim Stebbing Jan. 15, 2006

Jan. 15, 2006

We're looking at having a final Hobart AU sprint this Feb, before we Hobartians go our separate ways. This would be on the weekend of the 11th-12th and perhaps a day either side if people are interested. Sprint topics suggested are currently: * Fix VFS (deferred support, move the protocol specific bits into the various parts of twisted, integrate dav) * Trial GUI ? Current meatbag attendees: tjs, radix, cablehead, teratorn, jml? Current virtual attendees: lifeless? Currently thinking about … [View More]

1 0

[Twisted-Python] Re: simple dns server with twisted.names
by Yves Glodt Jan. 15, 2006

Jan. 15, 2006

On Fri, 13 Jan 2006 13:49:17 +0100, Yves Glodt <y.glodt at sitasoftware.lu> wrote: > On Fri, 13 Jan 2006 13:49:17 +0100, Yves Glodt <y.glodt at > sitasoftware.lu> wrote: >> Hi, >> >> I have read about twisted.names and I guess it can help me in what >> I have to achieve.. >> >> I have a database which contains a list of ip-addresses and >> corresponding hostnames. >> >> Now, how could I do to, e.g., write a daemon which … [View More]

1 0

[Twisted-Python] Repeatedly calling Deferred.callback()
by Brian Granger Jan. 13, 2006

Jan. 13, 2006

Hi, Are Deferred objects designed to have their callback method called multiple times it there are a multiple (discrete) results that arrive asynchronously? If not, what is the proper construct in Twisted for returning multiple asynchronous results from a single method? Thanks Brian

4 4

[Twisted-Python] Documentation virtual sprint: 14th January
by Mary Gardiner Jan. 13, 2006

Jan. 13, 2006

Hi all, A reminder to interested parties that I set the date for this to this coming Saturday: 14th January. What follows is a reminder about what I want to do. I'll set up something more specific for this in a day or two. I envisage I'll have a worklist of some kind on a wiki, and that participants will also be chatting on #twisted as usual to resolve anything else. -Mary ----- Forwarded message from Mary Gardiner <mary-twisted(a)puzzling.org> ----- I'd like to run a one day Twisted … [View More]

2 2

[Twisted-Python] Weekly Bug Summary
by James Y Knight Jan. 11, 2006

Jan. 11, 2006

Bug summary ______________________ Since 2006-01-04 06:00:02.200000: Bugs opened: 13 Bugs closed: 7 Total open bugs: 524 (+6) Urgent: 7 (-1) Bug: 249 (+5) Feature: 181 (+0) Wish: 87 (+2) New / Reopened Bugs ______________________ ==== Urgent ==== [TRIAL] Trial temp directory behavior is a little off (4 hours ago) http://twistedmatrix.com/bugs/issue1413 opened by jml [CORE] Does connectionLost belong in IFileDescriptor? (2 days ago) CLOSED http://… [View More]

1 0

[Twisted-Python] Defect in twisted/persisted/sob.py
by Paul Reznicek Jan. 10, 2006

Jan. 10, 2006

After update to trunk, twistd do not start as a daemon with following error: Starting my-app: twistd Failed to load application: global name 'mainMod' is not defined Following patch helped, please consider to apply to trunk: Index: sob.py =================================================================== --- sob.py (Revision 15568) +++ sob.py (Arbeitskopie) @@ -157,6 +157,7 @@ @param style: string (one of 'source', 'xml' or 'pickle') @param passphrase: string … [View More]

2 1

[Twisted-Python] mixing a Cacheable and a Viewable
by Forest Darling Jan. 10, 2006

Jan. 10, 2006

I want to make a server side object that the client can access via a RemoteCache, but also access view_ functions from the Cacheable via .callRemote() This way a player can see an object based on its perpective (like fog of war) and a player can also call server-side methods for game actions (like to move a hero.) from the server and remote classes trying to inherit both doesn't work, so how do I do this? This kind of problem must have come up before... Heres the link to my non-working … [View More]

1 0

[Twisted-Python] slightly silly hack
by zooko＠zooko.com Jan. 10, 2006

Jan. 10, 2006

Ever wanted to execute your trial tests as though they were commands? You can use tab completion in bash, for example: $ ./amdlib/storage/test/test_storefile.py -s It's easy! Give your file executable permission and at the top of the file type this line: "#!/usr/bin/env python". At the bottom type: """ if __name__ == "__main__": import os, sys mods = [] fullname = os.path.realpath(os.path.abspath(__file__)) for pathel in sys.path: fullnameofpathel = os.path.realpath(… [View More]

1 0