Mailman 3 July 2008 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 26 Jul '08

26 Jul '08

1 0

[Twisted-Python] trial and adbapi.ConnectionPool
by Tim Allen 25 Jul '08

25 Jul '08

In the projects I'm working on at the moment, we're using t.e.adbapi.ConnectionPool wrapping psycopg2 to handle connections to a PostgreSQL database. The setUp method of our unit-tests drops and recreates the database each time, and we've discovered that intermittently tests will fail because a previous test is still connected to the database when the subsequent test runs. Annoyingly, it happens on some machines and not others. I've attached a unit-test that reliably reproduces the problem for me on the machines where I've observed it at all. Looking at the PostgreSQL logs, a normal test-run does something much like this: - connection 1 incoming - connection 2 incoming - connection 1 executes a statement - connection 2 executes a statement - connection 1 disconnects - connection 2 disconnects - the next test tries to drop the database and succeeds. When things go wrong, it often looks more like this: - connection 1 incoming - connection 2 incoming - connection 1 executes a statement - connection 2 executes a statement - connection 1 disconnects - the next test tries to drop the database and fails because the database is in use. - connection 2 disconnects It turns out that the easiest work-around for this problem is to force a garbage-collection run after each test (with trial --force-gc or calling gc.collect()), so I have a work-around in the cause of these errors is outside my control, but I'm worried that perhaps I'm abusing the ConnectionPool in some way I haven't noticed. Are there known issues with shutting down adbapi.ConnectionPool in tests? Should adbapi.ConnectionPool.close() return a deferred that fires when the all the connections in the pool are guaranteed to have shut down? Can anyone see anything wrong in my sample code?

3 2

[Twisted-Python] From prototype to production
by nadine 25 Jul '08

25 Jul '08

Hi, Many thanks to the creators and developers of Twisted. I have been using Twisted to develop a prototype (using an Dell desktop) for an inline DPI server. Needless to say, Twisted made the development time much shorter, and I am very grateful for that. I now want to migrate from prototype to production. I was fully expecting to use an intel IXP server, and having to learn its OS and low-level programming languages. But intel itself told me that they are discontinuing that family, and are recommending the use of multi-core general-purpose servers. If that is the case, then it is tempting to think that I can use the exact same python software I developed for the prototype in the production server. Mind you, for the prototype I was testing three simultaneous users. For production, I am likely to have 5000 simultaneous users per server. Is this thinking correct? Are there any obvious scalability issues that I would encounter if I used python in production? Is it automatically obvious that I need to reprogram my methods in C, or even a lower-level language? Thanks. N. _________________________________________________________________ Stay in touch when you're away with Windows Live Messenger. http://www.windowslive.com/messenger/overview.html?ocid=TXT_TAGLM_WL_messen…

2 1

[Twisted-Python] Twisted + Pyglet = txpyglet?
by Nathan 24 Jul '08

24 Jul '08

When I got asked the third (fourth?) time to show how I "integrated" pyglet and twisted together, I decided to package it up and put it out there. I called it txpyglet because of all the tx-prefix-discussion-stuff, but it's really more of a sample app than a twisted library. A short screencast demo and a link to a zipfile of the goods can be found here: http://nathanstocks.blogspot.com/2008/07/txpyglet.html For the actual "integration" method, just look at the last 50-ish lines of the txpyglet.py file. P.S. Yes, this method of calling pyglet's loop is blocking. Yes, I know that's bad. And yes, I'm open to suggestions for a better way for those so inclined. In fact, I think some people have already put out better ways--I just haven't had time to look into them... ~ Nathan

1 0

[Twisted-python] help about create a tcp connection pool using twisted
by 陳毅泰 24 Jul '08

24 Jul '08

Hi, if I want to make a tcp connection pool by twisted, how can I do that?? I am confuse about what should I keep?? The procotol instance or something else, thanks a lot...

2 1

[Twisted-Python] ANNOUNCING Allmydata.org "Tahoe", the Least-Authority Filesystem, v1.2
by zooko 23 Jul '08

23 Jul '08

Dear twisted-python(a)twistedmatrix.com: This open source storage system is built on top of Twisted. Twisted is great! Regards, Zooko We are pleased to announce the release of version 1.2.0 of the "Tahoe" Least Authority Filesystem. The "Tahoe" Least Authority Filesystem is a secure, decentralized, fault-tolerant filesystem. All of the source code is available under a Free Software, Open Source licence (or two). This filesystem is encrypted and distributed over multiple peers in such a way it continues to function even when some of the peers are unavailable, malfunctioning, or malicious. A one-page explanation of the security and fault-tolerance properties that it offers is visible at: http://allmydata.org/source/tahoe/trunk/docs/about.html This is the successor to Allmydata.org "Tahoe" Least Authority Filesystem v1.1, which was released June 11, 2008 [1]. This release fixes a security issue in Tahoe v1.1, fixes a few small issues in the web interface, adds a "check health" operation for mutable files, and adds logging/operations/deployment improvements. See the known_issues.txt file [2] and the NEWS file [3] for details. COMPATIBILITY The version 1 branch of Tahoe is used as the basis of the consumer backup product from Allmydata, Inc. -- http://allmydata.com . Tahoe v1.2 is fully compatible with Tahoe v1.0. v1.2 clients produce files which can be read by v1.0 clients. v1.2 clients can read files produced by clients of all versions >= v0.8. v1.2 servers can serve v1.0 clients and v1.2 clients can use v1.0 servers. This is the third release in the version 1 series. We believe that this version of Tahoe is stable enough to rely on as a permanent store of valuable data. The version 1 branch of Tahoe will be actively supported and maintained for the forseeable future, and future versions of Tahoe will retain the ability to read files and directories produced by Tahoe v1 for the forseeable future. WHAT IS IT GOOD FOR? With Tahoe, you can distribute your filesystem across a set of computers, such that if some of the computers fail or turn out to be malicious, the filesystem continues to work from the remaining computers. You can also share your files with other users, using a cryptographic capability-based access control scheme. Because this software is the product of less than two years of active development, we do not categorically recommend it for the storage of data which is extremely confidential or precious. However, we believe that the combination of erasure coding, strong encryption, and careful engineering make Tahoe safer than common alternatives, such as RAID, or traditional backup onto a remote server, removable drive, or tape. This software comes with extensive unit tests [4], and there are no known security flaws which would compromise confidentiality or data integrity. (For all currently known issues please see the known_issues.txt file [2].) This release of Tahoe is suitable for the "friendnet" use case [5] -- it is easy to create a filesystem spread over the computers of you and your friends so that you can share disk space and share files. LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" [6] for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1.0. The Transitive Grace Period Public Licence says that you may distribute proprietary derived works of Tahoe without releasing the source code of that derived work for up to twelve months, after which time you are obligated to release the source code of the derived work under the Transitive Grace Period Public Licence. See the file "COPYING.TGPPL.html" [7] for the terms of the Transitive Grace Period Public Licence, version 1.0. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe works on Linux, Mac OS X, Windows, Cygwin, and Solaris. For installation instructions please see "docs/install.html" [8]. HACKING AND COMMUNITY Please join us on the mailing list [9] to discuss uses of Tahoe. Patches that extend and improve Tahoe are gratefully accepted -- the RoadMap page [10] shows the next improvements that we plan to make and CREDITS [11] lists the names of people who've contributed to the project. The wiki Dev page [12] contains resources for hackers. SPONSORSHIP Tahoe is sponsored by Allmydata, Inc. [13], a provider of commercial backup services. Allmydata, Inc. contributes hardware, software, ideas, bug reports, suggestions, demands, and money (employing several allmydata.org Tahoe hackers and instructing them to spend part of their work time on this free-software project). Also they distribute customized t-shirts just for some of our favorite contributors. We are eternally grateful! Zooko O'Whielacronx on behalf of the allmydata.org team July 21, 2008 Boulder, Colorado, USA [1] http://allmydata.org/trac/tahoe/browser/relnotes.txt?rev=2716 [2] http://allmydata.org/trac/tahoe/browser/docs/known_issues.txt [3] http://allmydata.org/trac/tahoe/browser/NEWS [4] http://allmydata.org/trac/tahoe/wiki/Dev [5] http://allmydata.org/trac/tahoe/wiki/UseCases [6] http://allmydata.org/trac/tahoe/browser/COPYING.GPL [7] http://allmydata.org/source/tahoe/trunk/COPYING.TGPPL.html [8] http://allmydata.org/source/tahoe/trunk/docs/install.html [9] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev [10] http://allmydata.org/trac/tahoe/roadmap [11] http://allmydata.org/trac/tahoe/browser/CREDITS?rev=2677 [12] http://allmydata.org/trac/tahoe/wiki/Dev [13] http://allmydata.com

2 1

[Twisted-Python] Waiting on a Deferred
by Saketh 23 Jul '08

23 Jul '08

Hello everyone, Is it possible to call asynchronous methods synchronously? I'm asking because I have an xmlrpc.XMLRPC server which distributes a calculation across several remote servers, collects the results in a DeferredList, and wraps it all inside an xmlrpc_something method. Thus, it needs to return the result to a web browser synchronously. [Interface] -- [XMLRPC server] -- [Calculators] This seems like it would be a common thing, getting the result of a distributed calculation for a browser client, but I haven't been able to figure it out. I appreciate any assistance! Thanks, Saketh

4 5

[Twisted-Python] Converting async to sync code (blocking until deferreds fire)
by Jack Whitham 22 Jul '08

22 Jul '08

Hi, What is the best way to convert asynchronous code (non-blocking, returning deferreds) to synchronous code (blocking)? I have a client that is written using Twisted. Most of the methods within the client object return Deferreds because they send messages to a server and replies may take some time. This is ideal for the wxwidgets GUI that was originally intended to use the client. But now I want to write test scripts for the server that are synchronous. For example, the client provides the method "GetBitInfo" which returns a deferred. The deferred is called back with a list as its parameter after the server responds to a request. I would like to be able to do something like this: > def GotBitInfo(l): pass > > if __name__ == "__main__": > c = Client() > > d = c.GetBitInfo() > d.addCallback(GotBitInfo) > waitForDeferred(d) > > d = c.DoSomethingElse() In other words, how do I waitForDeferred(), i.e. block until a Deferred fires? Of course I could rewrite my script as asynchronous code; but what I really want is a simple Python-like interface for the client code in order to make things as simple as possible for other users. I don't want them to *have* to learn how to write asynchronous code - I want it to be optional. If the snippet of code listed above is possible, then I can wrap the client methods with code that waits for deferreds to fire, then returns the values they were called with. Then the users will have two ways to use the code: asynchronous (for GUIs, proxy servers, etc.) and synchronous (for test scripts). I noticed that there is a _wait method in your trial code that seems to do what I want, but it looks complicated and I'm not sure it is the right answer. Its predecessor was also deprecated, which is a bad sign (looks like "do not do this!!"). Hence I am looking for expert advice. Thanks. -- Jack Whitham jack(a)cs.york.ac.uk

6 6

[Twisted-Python] txAMQP released
by Esteve Fernandez 21 Jul '08

21 Jul '08

Hi all, I'm pleased to announce that Fluidinfo [1] is releasing a Twisted compatible AMQP stack built on the Python Qpid library. Download from http://foss.fluidinfo.com We've been using it for a while at Fluidinfo and have converted all the Qpid tests to use it, so it should be considered fairly stable. Although we'll try to keep it in sync with Qpid's stable releases, we can't promise to track the trunk due to lack of resources. Feedback and contributions will be greatly appreciated! :-) Cheers. PS: I don't know which license is the most appropiate to make it compatible with Qpid and Twisted, so I guess the ASL should suffice. [1] - Fluidinfo - http://www.fluidinfo.com

1 0

[Twisted-Python] James Brake has invited you to join Friendster
by James Brake 21 Jul '08

21 Jul '08

You're invited to join James Brake's network of friends. By joining Friendster, you can reconnect with old friends, meet new friends, start a blog, build a custom profile, keep track of birthdays, and so much more! You can even stay in touch if you move away, switch email addresses, or lose your mobile phone. Click below to join James's network. http://www.friendster.com/join.php?inviteref=72570452&invite=-8C6ksHVjn… ***************************************************************** If you do not wish to receive notification emails from Friendster, please click below: http://www.friendster.com/blockemails.php?invite=dHdpc3RlZC1weXRob25AdHdpc3…

1 0

Twisted July 2008