Mailman 3 January 2003 - Twisted

[Twisted-Python] Adding DeprecationWarnings to twisted.internet.main
by Andrew Bennetts Jan. 13, 2003

Jan. 13, 2003

Hi, Is there any reason why we shouldn't add: warnings.warn( 'twisted.internet.main is deprecated. ' 'Use twisted.internet.reactor instead.', DeprecationWarning, stacklevel=2 ) to every function in twisted.internet.main (except perhaps installReactor)? At the very least, I think a PendingDeprecationWarning would be appropriate (they are invisible by default). There are still parts of Twisted that use this old module (e.g. t.e.adbapi), and I expect this will be the case until we have some nice big warnings jump up at us telling us to fix them :) -Andrew.

2 2

[Twisted-Python] developer sprint at PyCon
by Christopher Armstrong Jan. 12, 2003

Jan. 12, 2003

So here are three suggestions for what we can hack on during the devel sprint. * Adapterize/Componentize PB (this came up when Sylane was trying to make an object both Referenceable and Cacheable today) * Twisted Reality * Woven I'm more interested in the first two, personally. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://twistedmatrix.com/users/radix.twistd/

1 0

[Twisted-Python] twisted.im has no tests
by Kevin Turner Jan. 12, 2003

Jan. 12, 2003

WARNING: The next person to work with twisted.im, be it radix, myself, or someone else, is most likely going to break shit, because this package needs some reconstructive surgery. Currently, there are NO TESTS for this package, nothing to tell us how much we've broken. And if we don't know how much we've broken, there's no reason to feel guilty about it, right? The upshot of this is that if you're using this package and there's something that you need not to break, or maybe broken a little but at least not be rewritten out of existence, WRITE TESTS NOW. Cheers, - Kevin -- The moon is first quarter, 62.1% illuminated, 8.5 days old.

2 2

[Twisted-Python] Another twisted-im patch
by screwtape＠froup.com Jan. 12, 2003

Jan. 12, 2003

Notwithstanding the last message about twisted.im being nuked and replaced with something almost but not quite entirely different :) here's a diff that I feel makes twisted-im a little more useful. -- ___________ ____________________________ | Screwtape | Reply-To: munged on Usenet |________ ______ ____ __ _ _ _ | | "The cruel angel's thesis bleeds" |

1 0

[Twisted-Python] Documentation Lacks
by Moshe Zadka Jan. 11, 2003

Jan. 11, 2003

Hi. The following things are undocumented -- any volunteers would be welcome: - conch HOWTO - Twisted chat HOWTO (im, words server, writing bots) - Add tkmktap/tktwistd to basics.html - Twisted news HOWTO - Twisted names HOWTO

4 8

[Twisted-Python] twisted.lore: `class="API" base="twisted.foo"' feature broken
by Christopher Armstrong Jan. 7, 2003

Jan. 7, 2003

Or seems to be, anyway. e.g., http://twistedmatrix.com/documents/howto/woven Notice that all links that don't have the entire `twisted.foo...' prefix are broken; the `base' attribute isn't being honored (you won't see the attributes in the source of that page, but look at doc/howto/woven.html in CVS). I looked at twisted.lore.tree and didn't see anything obviously wrong. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://twistedmatrix.com/users/radix.twistd/

2 1

[Twisted-Python] Writing Unittests for Twisted Applications
by Bernhard Herzog Jan. 7, 2003

Jan. 7, 2003

Hi all, I've been working on a project using twisted for the last few weeks. In the process I've written quite a few unittests most of which need to run a twisted reactor either directly as reactor.run or via an Application instance's run() method. It's amazing how difficult that is, given that twisted itself has an extensive set of unittests. Running only one test with a main loop is simple but running several tests that need to listen and unlisten the same ports or that have timeouts, etc. seems much more difficult than it should be. How difficult would it be to implement at least one of the following features? 1. a way to reset a reactor so that e.g. a new unit test could start with a clean state. It's OK for me if the unittest itself has to take care to do some things automatically such as disconnecting connections or unlistenting. 2. A base class (derived from unittest.TestCase or as a mix-in class) for tests that need to run the main loop. There should perhaps be two of them, one for plain reactor usage and for test with Application instances. Obviously this would require having 1. to a certain extent. However, it's OK if the class knew how to achieve a reset in as far as that is necessary for test cases. For instance, the base class could require that derived classes don't call the reactor directly to listen or to add delayed calls and go through some method instead to e.g. keep track of the ids returned by reactor.callLater. AFAICT neither is currently implemented in twisted. There even seems to be one unit test (test_sister) that's not implemented because of this. I have effectively implemented no. 2 as part of my unit-tests but only as much as I needed it so far. It didn't take much code but there are some tricky things: 1. After and unlisten when the loop has already been left, call reactor.iterate() to make sure the port is properly freed again. This was relatively easy to figure out because some twisted test cases do this. 2. Application.run() will only run the reactor mainloop if main.running is not true. Unfortunately, main.running is not unset when main.shutdown or main.crash is called, so when trying to enter application main loops more than once, even for different application instances, once has to explicitly unset main.running. My code will eventually be available under GPL as part of the 'GREAT-ER'[1] software we're developing, but it's not yet in CVS. If there's interest in including my test case base class in twisted I could make it available sooner and under LGPL. Bernhard [1] http://great-er.intevation.de/ -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/

3 5

[Twisted-Python] How do I upload files using FTPClient?
by Daniel Burr Jan. 6, 2003

Jan. 6, 2003

On Tue, 7 Jan 2003 10:37:22, Andrew Bennetts wrote: >Yes, that's my fault. I haven't gotten around to >that yet -- I might have a >stab at it tonight, it shouldn't actually be that >hard. I'll try to make sure >I get this done in time for the 1.0.2 release. That'd be great; I'm interested to see how you do it, as I don't yet fully understand the Twisted way. I have always used Python's ftplib, so doing this in a fully async manner is still a little mind-bending for me. >Let me know if there are any other FTP commands >you want implemented -- I've >also got MDTM on my todo list, which should be >trivial (in fact, the >apt-proxy guys have already implemented it! :). Well, since you asked, I could really use MKD and RMD too, since I am copying a directory hierarchy from one server to another, and the target server may not already have the correct directory layout. >The server is desperately in need of a >rewrite/refactoring to >use twisted.web (so that you can have dynamic >resources, and the like). >This is on my longer-term todo list... > >-Andrew. That sounds pretty cool, although I'll be perfectly happy if FTPClient can reproduce the current functionality of ftplib. On a semi-related note, I'm thinking about trying to write a Protocol/Factory to handle the SCC protocol used by PVCS, SourceSafe, and a few other commercial source control tools. I'm thinking in terms of a Twisted.vcs plugin. Is this a good idea, or is this really not what Twisted is best used for? Thanks for the help, L. Daniel Burr Thanks for your help, L. Daniel Burr

2 1

[Twisted-Python] [OT] PyCon accomodation?
by Andrew Bennetts Jan. 6, 2003

Jan. 6, 2003

Hi all... I'm intending on going to PyCon (http://www.python.org/pycon/), but I'm not sure where I should stay for the week -- the places at http://www.python.org/cgi-bin/moinmoin/PyConAccommodation sound very expensive to my Australian dollars :) Basically I'm looking to do this on the cheap; the airfare is expensive enough... ideally, I'd probably share a room in a budget hotel with someone. This has the bonus that they could help a clueless Aussie traveller find his way around ;) So, who's going to PyCon, and do they have any ideas on where I could stay? I'd probably be looking to stay from Sunday night 23rd (so I could attend the code sprint) through to Saturday the 29th, although I haven't thought this out in detail. -Andrew.

7 7

[Twisted-Python] cReactor fixes
by Brian Warner Jan. 6, 2003

Jan. 6, 2003

Hi all.. For some reason not yet entirely clear to me, I started to put some time into bringing cReactor up to date. My hope was to make it functional enough to pass the full test suite. At the moment, it fails 120 of the 413 tests in the suite (that doesn't include test_threads, which I had to disable to prevent the test suite from hanging forever). In tracking down the reasons for these failures, I've come across a number of places where the reactor interface (as defined by IReactorCore and friends in twisted.internet.interfaces) was ambiguous about something, and the default reactor has implemented different functionality than the cReactor. Several tests have come to rely upon default.py's implementation, and fail when run under cReactor even though it appears to implement the same interface. To resolve these, we need to nail down the ambiguities in the Interface, so that all reactors can implement the same thing. The two issues I've encountered so far: IReactorCore.addSystemEventTrigger() [1 test failure] The interface mentions three (startup, shutdown, persist) as examples of "system events", and says they will be fired internally by the Reactor. It also says: An implementor of this interface must only implement those events described here. The cReactor implements this literally, and refuses to accept calls to addSystemEventTrigger or fireSystemEvent with event names outside of ['startup', 'shutdown', 'persist']. The default reactor (and the ones that inherit system event code from it, which is all of them except cReactor) use a dict for the events, and therefore accept any string as a system event type. test_internet.InterfaceTestCase.testTriggerSystemEvent creates several new event types ("test", "defer", "remove") and runs tests with them. This test thus fails under cReactor. RESOLUTION: I think we decided that cReactor should accept arbitrary event names, and I'm working on code to implement that. That restrictive sentence should be removed from the docs. I'd also like to see a better explanation of what event types will be fired by whom: if the API allows an arbitrary string, but the Reactor will only fire "startup" and "shutdown" for you, then you have to know that it is your own responsibility to fire the new triggers that you've added. IReactorCore.stop() [38 test failures] The interface description for .stop and .crash make the claim that once the Reactor is shut down with .stop(), it may not be restarted with .run(). Reactors which have been stopped with .crash() *may* be restarted. cReactor implements this literally. Once reactor.stop() has been called, the reactor is moved into a state from which .run() and .iterate() raise a RuntimeError "the reactor is shut down!" exception. The default reactor does not enforce this: .run and .stop may be called as many times as you like. All tests in a given invocation of bin/trial or admin/runtests are executed with the same Reactor instance. Indeed everything done by a single python process is performed with the same Reactor instance. Normal apps don't care: they start the reactor in main() or under twistd, and it runs until the app shuts down. The tests, however, start and stop the reactor all the time, generally once per test case. Under cReactor, the first test to do reactor.stop() will terminate the reactor permanently, causing failures of all later tests that attempt reactor.run() or reactor.iterate(). This also tends to kill tests which re-bind to the same port multiple times, as the reactor.iterate() needed to release the port fails, leaving the port bound, causing a "Address already in use" error on the second test. POTENTIAL RESOLUTION: We need to decide whether the default reactor should obey the restriction defined by IReactorCore, or whether the interface (and cReactor) should be changed to match the behavior of the default reactor. Another possibility which might be easier to implement would be to enforce the no-restart restriction, but also create IReactorCore.reset(), and declare that it will return the reactor to the state it had at boot time, allowing .run() to be called on it once again. This method would also remove all DelayedCall events, remove any TCP/UDP sockets or listening ports, delete any systemEventTriggers that were added, and generally get rid of any other state which might have accumulated since startup. We could have the test suite use this to reset the reactor just after each test. The other test failures are the result of functionality that is missing from cReactor. In general, cReactor has not been maintained as new features were added to the other reactors. Here is a summary of all the failures. (note that some tests fail for multiple reasons). IReactorTCP.connectTCP [8 failures] client support (outbound sockets) is missing IListeningPort.getHost [22 failures] simple missing function, mostly used to determine the port number of a system-selected listening port IReactorProcess [9 failures] launching child processes is unimplemented rebuild attempts: missing __dict__ [5 failures] twisted.python.rebuild attempts to rebuild a C module with no .__dict__ IReactorMulticast [5 failures] multicast support is not implemented IReactorUDP [23 failures] UDP support is unimplemented IReactorCore.addSystemEventTrigger() [1 test failure] IReactorCore.stop() [38 test failures] Some of these aren't hard to fix: IListeningPort.getHost should only take a few lines, connectTCP is pretty straightforward, and IReactorUDP should be a fairly easy cut-and-paste job from the TCP code. But others are tricky: IReactorProcess is fairly high-level, and IReactorMulticast might be relying upon some portability code in the python socket module that wouldn't be available at the C layer. I think it is now pretty clear what work needs to be done. Everyone is welcome to take a crack at it :). I will probably finish the getDelayedCall and addSystemEventTrigger changes I'm working on, commit them, and then let others pursue the pieces that are useful to them. (the default reactor is currently good enough for my needs; I only put this much time into cReactor because I wanted to understand the test failures well enough to know that my changes weren't the cause). cheers, -Brian

5 8