Mailman 3 September 2005 - Twisted

[Twisted-Python] Python 2.2 Support
by Jonathan Lange Sept. 10, 2005

Sept. 10, 2005

So, how long are we going to support it?

4 3

[Twisted-Python] Broker leak in spread server on login failure = denial of service?
by David K. Hess Sept. 10, 2005

Sept. 10, 2005

I believe I've discovered a broker leak when a checker denies a login and that it can be used to launch a denial of service attack on a spread server. When a checker throws a Failure exception, since requestAvatar in the realm doesn't get called, the realm (or anything else) doesn't end up with a reference it can use to disconnect the network connection that was denied login. If a malicious (or just buggy as was my case) client tries to login over and over again and doesn't … [View More]

3 3

[Twisted-Python] Epydoc + interfaces
by Maciej Szumocki Sept. 9, 2005

Sept. 9, 2005

Hello all, When i tried to generate api docs for my application (that uses Twisted 2.0 and Zope interfaces) i noticed that interface documentation is missing. However, in Twisted API docs, interface info generated by epydoc seems to be fine. I assume there is some kind of workaround that makes epydoc able to generate interface info that you guys used when generating Twisted API docs. Please share the secret :) Maciej Szumocki

4 3

Re: [Twisted-Python] Re: Disabling PB (de)serialization
by Brian Warner Sept. 9, 2005

Sept. 9, 2005

> Mostly, *very* long lists of small objects, each containing a few numbers > and short strings. What order of magnitude are we talking about? Would something like [(1,12,"foo","bar") for i in range(10000)] be close? If so, I'll use this as one of the benchmark cases. > In my case, the data is going between instances of a VNC server and a > VNC viewer. The viewer, of course, isn't sending much, but using PB rref > method calls to transfer the data from the VNC server was enough … [View More]

3 2

[Twisted-Python] Multiplexing multiple transports over one
by Stefano Masini Sept. 9, 2005

Sept. 9, 2005

It shouldn't be too difficult to implement, or maybe it's already there... I'm looking for some mechanism that allows me to multiplex multiple protocols over the same connection. Say I open a socket on port 3000, and I run both http and smtp over it. Of course this is sort of a port redirection thing, like ssh does, so I need a way to specify which host and port should be contacted by the other end of the tunnel. It looks like consh.ssh.forwarding should do exactly this, but since I never … [View More]

3 3

[Twisted-Python] Weekly Bug Summary
by James Y Knight Sept. 8, 2005

Sept. 8, 2005

Bug summary ______________________ Since 2005-08-31 21:44:42.620000: Bugs opened: 6 Bugs closed: 1 Total open bugs: 423 (+5) Urgent: 10 (+0) Bug: 188 (+2) Feature: 153 (+2) Wish: 72 (+1) New / Reopened Bugs ______________________ ==== Bug ==== [DOCUMENTATION, MAIL] mail documentation links to non-existant example (4 minutes ago) http://twistedmatrix.com/bugs/issue1162 opened by hypatia [FTP] FTPClient failed commands return a FirstError (… [View More]

1 0

[Twisted-Python] Disabling PB (de)serialization
by Nicola Larosa Sept. 7, 2005

Sept. 7, 2005

I'd like to make a PB proxy "transparent". It should be able to get serialized data via PB without deserializing it, and forward it via PB to another proxy, without having to serialize it again. The second proxy will receive the data, and serialize it before sending it to the final destination. Currently the data are serialized and deserialized three plus threes times, and that's rather inefficient when dealing with large data. How do I selectively disable (de)serialization in PB? -- Nicola … [View More]

4 13

[Twisted-Python] Re: Disabling PB (de)serialization
by Brian Warner Sept. 7, 2005

Sept. 7, 2005

>> In TP1, the business data should be serialized, and wrapped in an outer >> layer, containing the destination coordinates. PX1 and PX2 should be able >> to only unwrap and deserialize the outer layer only, while trasparently >> forwarding the serialized business data. > > At a guess you'd need to write your own Unjellier class or something. I > suspect this will be much easier in newpb, but I'm not *sure*. Brian? Hm, tricky. In oldpb, yeah, you need to get … [View More]your hands dirty with Jelly. When you invoke rref.callRemote(methname, *args, **kwargs), what really happens is: the RemoteReference is turned into a per-connection objid a pending-call slot is allocated, assigned a unique number 'callid' the following tuple is serialized and sent over the wire: ("call", objid, callid, methname, args, kwargs) at the far end, the tuple is fully unserialized, then examined. The "call" token triggers a lookup of 'objid' to find the target Referenceable, which then gets its remoteMessageReceived method invoked. This method receives the (methname, args, kwargs) values and is responsible for calling the final remote_foo method. The (args,kwargs) received by remoteMessageReceived have been unbananaed but not yet unjellied (see below), so most of the deserialization work has already been spent. There is no clean place in this sequence to tell the underlying Protocol instance to switch from the banana-unserializing dataReceived() mode to some just-copy-the-data mode: in general, everything gets unserialized before you even find out what the target object is. So the way I'd do what you're trying to do is to serialize the arguments myself, then pass the resulting blob to the method that knows about the dispatch/forwarding rules. Terminology: "Jelly" is the layer that turns arbitrary object graphs into s-expressions (nested lists of primitive types like strings), while "Banana" is the layer that serializes these s-expressions into a series of bytes. There is a Protocol subclass named Banana which is attached to the wire and emits sexps as they arrive, then the Broker is a subclass of Banana which interprets these sexps as commands. When these commands require sexps to be turned into objects, it uses Jelly. To jelly arbitrary data, you just do this: from twisted.spread import banana, jelly def encode(o): sexp = jelly.jelly(o, taster=jelly.globalSecurity) return banana.encode(sexp) def decode(s): sexp = banana.decode(s) return jelly.unjelly(sexp, taster=jelly.globalSecurity) The sending side would then look like: def callThroughDispatcher(methname, *args, **kwargs): allargs = encode((methname, args, kwargs)) d = target.callRemote("dispatch", methname, allargs) return d the dispatch side could look like: def remote_dispatch(self, methname, allargs): return self.targets[methname].callRemote("doit", allargs) and the final target could look like: def remote_doit(self, allargs): methname, args, kwargs = decode(allargs) m = getattr(self, "remote_%s" % methname) return m(*args, **kwargs) The downsides of this approach: the "serialization domain" is rather small, and does not include the established connection, so this will fail if your arguments include Referenceables or RemoteReferences. (normally you would be able to pass new references through to the other side, but our encode() method does not know about the connection and therefore cannot manipulate the reference tables that would enable this) there is some overhead to encode()/decode(): it must create a new Banana instance, attach it to a dummy (StringIO) transport, then iterate it until all the serialized data is accumulated. This overhead is probably small in comparison to the time it takes to serialize a large/complicated object graph, but there will definitely be a break-even point somewhere, below which is makes more sense to let the existing Brokers do their own serialization. In newpb, you'll have more options, but this use-case won't necessarily be all that that much easier. The most useful new feature would be the pluggable Slicers/Unslicers, which give you more control over serialization and unserialization. It might be possible to write a faster serialization layer (in C).. I suppose that might help something. The fundamental problem is layer-mixing: you want to change the behavior of the very lowest-layer code (serialization/unserialization) based upon decisions made at much higher layers (target object or method name). This is sort of what dual-mode protocols like SMTP and HTTP do, where they switch between LineReceiver and RawDataReceiver depending upon protocol state, but in SMTP the transition is simple enough to implement at a very low level (just wait for a blank line). In both oldpb and newpb, the transition is indicated at a much much higher protocol level. A related problem is that serialization is closely tied to a context, in this case some state in the per-connection Broker object. Most of the advanced serialization features (being able to handle arbitrary object graphs, shared objects, pointing at remote Referenceables, etc) depend upon this context. The "Dispatcher" thing you want to do drags some of this deserialization into a second context, one which may not share enough state with the first one, so it may be infeasible to serialize anything but simple self-contained datatypes this way. I'm curious, what does your takes-a-long-time-to-serialize data look like? I need some performance-test-cases to benchmark newpb serialization code with, and if my test data looks more like your actual data, then newpb will be that much faster for your application. Are there a lot of large strings? Long lists? densely-connected graphs? hope that's useful.. -Brian [View Less]

3 2

[Twisted-Python] Telnet & sending commands to pine application
by Michał Tyde Sept. 7, 2005

Sept. 7, 2005

I have got a telnet client, and I send to him a commands via protocols.telnet.transport.write(command="pine"). I have run a pine application. Now I want to send a Q (Quit) message and y (yes) to close the pine. I send this messages using the same protocols.telnet.transport.write(command). Pine doesn't respond for my commands. I thought that when I send a single char I should close a pine. Maybe I should send this command to pine application not to a server (how to do it?). Thanks in advance for all answers

1 0

[Twisted-Python] Twisted Virtual Sprint
by Mary Gardiner Sept. 7, 2005

Sept. 7, 2005

Hi everyone, We've had some modest success now with two in-person sprints with Twisted hackers travelling up and down the east coast of Australia to work on crazy stuff (vfs, ftp... trial). At the last sprint, we discussed having a "virtual sprint", essentially a weekend during which interested people get together and work on Twisted. Unlike for the Australian sprints, this would be Saturday and Sunday and wouldn't include Friday. Communication would be over IRC most likely, and maybe VoIP. (… [View More]

5 7