Mailman 3 May 2001 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] ANN: The week ends, the week begins.
by Glyph Lefkowitz 14 May '01

14 May '01

OUTER SPACE (AP) -- Broadcasting from his self-styled "satellite of fiery doom" in a superlunar orbit on Monday, Evil Overlord Glyph Lefkowitz declared that a new version of Twisted Python, 0.8.8, was ready for download. In an interview, a bleary-eyed, haggard looking Glyph dismissed rumours that he had escaped to the orbital space-station in order to escape the rampaging mob of Twisted Python users angered over the uncharacteristically unstable state of the 0.8.5 and 6 releases, as well as those who had lost family members to Chief Reality Engineer Allen Short's so-called "bugfixing" spree. "Complete nonsense!", he quipped, "My fans are as loyal as ever." Although, the wary Overlord was heard to inquire briefly after the interview about the status of the aforementioned rampaging mob. Mr. Short was not available for comment. ---- In this release, you'll find better API documentation, an FTP server, and the new PB&J protocol/serialization suite. (this the intended replacement for gloop, previously entitled `gloop2'). Go hit your favorite your url (that's http://twistedmatrix.com/, in case you forgot), and download some firey doom today! :) -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph

1 0

[Twisted-Python] userinfo, etc, for twisted.ftp
by Benjamin Bruheim 14 May '01

14 May '01

Hepp, twisted.ftp is now in an nearly usable state. There are some problems with active ftp'ing (net.TCPClient does not exactly have the same interface as net.TCPServer), but this is really an minor issue. And for those who want to try it, use the bsd-ftp and remember to turn on binary transfers! The client seems to modify the files to conform to ascii :) ATM there is no auth'ing in twisted.ftp; and this is really the biggest issue with it. So, I wonder, how should it perform auth? Where can it find the homedir? And setting the access to the user? Well, much of this information could be found in the sources already, but I would like to know how much should be done unix-style or twisted.style :) phed =================================================================== EASY and FREE access to your email anywhere: http://Mailreader.com/ ===================================================================

1 0

[Twisted-Python] configuration usecases
by Moshe Zadka 14 May '01

14 May '01

1. Selector -- a selector will have exactly one configurable parameter: an array of things which conform to the server interface, and an array of things which conform to the delayed interface 2. POP3 Server -- (optionally) a name, a port and something that conforms to the "dictionary" interface (.has_key, [], .get which are consistent) where the keys are strings, and the values are things that conform to the domain interface, and to the POP3 domain extensions (authenticateAPOPUser) 3. DomainDictionary -- contents, a mapping of strings to things conforming to the domain interface 4. MaildirDbmDomain -- add a user/password, remove a user, change a password 5. HTTPServer -- something conforming to the resource interface 6. FolderResource -- a mapping of strings to resources, and all those of DataResource 7. DataResource -- code (an int), headers (a mapping of strings to strings) and the data (a long-string) 8. TelnetServer -- username, password -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez(a)debian.org |http://www.{python,debian,gnu}.org

2 1

[Twisted-Python] Configuration, Final Spec
by Moshe Zadka 13 May '01

13 May '01

Rewritten, standalone, spec OK, let me first state some axioms: 1. Everything should be configurable the same way, from within twisted. Glyph has mentioned that he's not familiar with what's in bin/ anymore: he just mktelnetserver, and configures from within the telnet server everything. This is what should happen, except not limited to the telnet server. 2. The configuration interface must not be ui specific. This is obvious, right? Command line and web based aren't the only option. No reason why we can't implement a special purpose client/server for configurin g twisted. 3. It should be easy to optimize the configuration for a ui This is like the last one -- since we might deal with very good UIs, we need to give them enough information for using all their abilities to help us. 4. It should be very easy to make an object configurable. This is very important -- the harder it is, the less it will be easy to add *good* code to twisted. This is what may be the single most idiotic thing in Zope (and DC is aware of it! and thinking of how to fix it!). Let's learn from their mistakes: the less methods, the better. The more the learning curve is gradual (not needing to learn a class/method/interface before you need the functionality), the better. OK, so what do we need to do about it? Here's a rough proposition: the configurable *interface*, which will be a class, but not a class people should inherit from, will contain the following methods: .getParameters() --> return a dictionary: name of parameter, Parameter object .setParameter(name, answer) --> notify the application that an answer has been given to a particular question It can throw an InvalidParameter exception with a string for a reason. This is for application-level verification. Note that application-level verification is discouraged. .endParameters() --> the "application" promises that no methods of the object will be called between a series of .setParameter()s and .endParameters(). So, this means that if the UI got a bunch of answers, it will call .setParameter() several times, and then .endParameters(). The UI will *check* for this method's existance, and will not call it if it doesn't exist. .setParent(parent) --> if the object is created inside an InterfaceProperty of some other object, module nested ArrayParameters and DictParameters, .setParent will be called with that object, *if it exists*. Parameter objects are meant to be open ended. They can contain a default. Here is the general interface of the Question, that all objects conform too: .hasValue() --> boolean, whether the Parameter already contains an value .getValue() --> will only work if .hasValue() is true, returns the answer .setValue(val) --> make .hasValue() true .title --> A string, the human-readable description of the property. Objects which can be created by the UI should have an initParameters attribute which should be a sequence of Parameters, which correspond to the mandatory arguments of the __init__ method. Well, my proposal would not be complete if I didn't say what parameters are available. Keep in mind, though, that the set of parameters is *open ended*. That does not violate the light-and-lean guidelines, since a specific Parameter class will only be used if the functionality is needed. Without further ado: class BooleanParameter: class IntParameter: (can have .min and .max) class FloatParameter: class StringParameter: (can have .maxlength) class LongStringParameter: (same as above -- it's a hint to the ui) class InterfaceParameter: (specify interface, valid answers are objects) class ArrayParameter: (an array of the same kind of question) class DictParameter: (a dictionary mapping strings -> same kind of question) The constructor will always take a title as a first argument, and possibly some more parameters if appropriate. I'm thinking of adding some generic methods, the most interesting one is traverse, which either returns a Parameter or throws a ValueError("node parameter"). A class will be said to implement an interface if the interface class is a member of the __implements__ attribute. We can refine it to saying that the interface is a superclass of some member of __implements__: def implements(klass, interface): for member in klass.__implements__: if issubclass(member, interface): return 1 return 0 This leaves open the question of how classes register with the UI at all. A possibly radical solution will be to have the UI scan through sys.modules, and each class will be checked for the __implements__ attribute. A less radical option is to have a register(klass) function which registers it with the UI. register() can be used from within the module, if it is an active participant, or from within an importing module. example: class Server: # note -- not inheriting from anything initParameters = (StringParameter("Server Name"), IntParameter("Port")) def __init__(self, name, port): self.name, self.port = name, port def getParameters(self): name = StringQuestion("Server Name") name.setValue(self.name) port = IntQuestion("Port") port.setValue(self.port) return {'port': port, 'name': name} def setParameter(self, name, answer): # note: no need to use int(answer) # for port: an IntQuestion has an integer as a .getValue() setattr(self, name, answer) -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez(a)debian.org |http://www.{python,debian,gnu}.org -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez(a)debian.org |http://www.{python,debian,gnu}.org

2 3

[Twisted-Python] Configuration: The Never Ending Story.
by Moshe Zadka 12 May '01

12 May '01

OK, let me first state some axioms: 1. Everything should be configurable the same way, from within twisted. Glyph has mentioned that he's not familiar with what's in bin/ anymore: he just mktelnetserver, and configures from within the telnet server everything. This is what should happen, except not limited to the telnet server. 2. The configuration interface must not be ui specific. This is obvious, right? Command line and web based aren't the only option. No reason why we can't implement a special purpose client/server for configuring twisted. 3. It should be easy to optimize the configuration for a ui This is like the last one -- since we might deal with very good UIs, we need to give them enough information for using all their abilities to help us. 4. It should be very easy to make an object configurable. This is very important -- the harder it is, the less it will be easy to add *good* code to twisted. This is what may be the single most idiotic thing in Zope (and DC is aware of it! and thinking of how to fix it!). Let's learn from their mistakes: the less methods, the better. The more the learning curve is gradual (not needing to learn a class/method/interface before you need the functionality), the better. OK, so what do we need to do about it? Here's a rough proposition: the configurable *interface*, which will be a class, but not a class people should inherit from, will contain the following methods: .getQuestions() --> return a dictionary: name of question, Question object .getAnswer(name, answer) --> notify the application that an answer has been given to a particular question It can throw an InvalidAnswer exception with a string for a reason. This is for application-level verification, and is discouraged. .endAnswer() --> the "application" promises that no methods of the object will be called between a series of .getAnswer()s and .endAnswer(). So, this means that if the UI got a bunch of answers, it will call .getAnswer() several times, and then .endAnswer(). The UI will *check* for this method's existance, and will not call it if it doesn't exist. Question objects are meant to be open ended. They can contain a default. Here is the general interface of the Question, that all objects conform too: .hasAnswer() --> boolean, whether the Question already contains an answer/default .getValue() --> will only work if .hasAnswer() is true, returns the answer .setValue(val) --> make .hasAnswer() true Objects which can be created by the UI should have an __init__ which can be called without arguments. If there is any initialization which requires arguments, it should be done in endAnswer(). The UI also promises not to call .endAnswer() for an object if there any questions which have not been answered and do not have a default (.hasAnswer is false.) Well, my proposal would not be complete if I didn't say what questions are available. Keep in mind, though, that the set of questions is *open ended*. That does not violate the light-and-lean guidelines, since a specific Question class will only be used if the functionality is needed. Without further ado: class BooleanQuestion: class IntQuestion: (can have .min and .max) class FloatQuestion: class StringQuestion: (can have .maxlength) class LongStringQuestion: (same as above -- it's a hint to the ui) class InterfaceQuestion: (specify interface, valid answers are objects) class ArrayQuestion: (an array of the same kind of question) class DictQuestion: (a dictionary mapping strings -> same kind of question) example: class Server: # note -- not inheriting from anything name = port = None def getQuestions(self): name = StringQuestion() if self.name is not None: name.setValue(self.name) port = IntQuestion() if self.port is not None: port.setValue(self.port) return {'port': port, 'name': name} def getAnswer(self, name, answer): # note: no need to use int(answer) # for port: an IntQuestion has an integer as a .getValue() setattr(self, name, answer) Open questions: * How do we connect classes to interfaces? Suggestion: each class has an attribute __implements__ containig a list of interfaces. Alternatively, an interface is really a list of (callable, Questions) tuples, which are answered and passed to the callable. Modules with relevant classes register with the correct interface. * How do we let objects created inside another object who their parent are? Suggestion: if an object from on Interface question has a method .setParent, call it with the parent as argument. This should be done by the Question object, so it knows how to call it for each object in a ListQuestion * How do we let objects "title" a question? Suggestion: a question has a string argument on __init__ titling it. * How do we allow the object to signify groupings of questions? Suggestion: this means the design is bad -- break down the object into smaller objects -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez(a)debian.org |http://www.{python,debian,gnu}.org

2 5

[Twisted-Python] the magical port 0
by grolgh 12 May '01

12 May '01

Hello, As you might know, binding to port 0, actually binds an unallocated port; but despite how useful this might be, twisted.net returns port 0 anyway. The fix is easy, and that is to set port = skt.getsockname()[1] right after it has been binded. I don't if this applies to win32, or any other platform, but I assume that it does. I have not done this in CVS, but I hope someone with more insight into twisted.net could do it :) phed www.informative.f2s.com (be gone evil .zip-hating provider)

2 1

[Twisted-Python] neverending connection
by grolgh 10 May '01

10 May '01

Hello twisted.minions :) This is my first mail on tp-ml; hello! I'm currently working on twisted.ftp and it's about 30% into acceptable working state. To hack it you need to press enter twice :) But I've run into a known problem. I just noticed that TCPServer.loseConnection never trigs TCPServer.connectionLost. I think this is the famous never-ending-connection-bug, and I think the bug has its origin in selectable.doWrite. I'll here present my current findings with it. Note that doSelect will run its selectable.connectionLost() if selectable's doRead or doWrite returns anything non-zero. I'll take it step by step. A call to loseConnection, will set __reap Afterwards, __reap appears in doWrite which will write out its buffer and return CONNECTION_DONE doSelect terminates the connection Well, if we take a closer look at doWrite we find that it has several layers of if's before a CONNECTION_DONE is returned. Especially "if not self.unsent"! :/ Well, hope anyone finds this useful; and squash that horrible thing out. :)

1 0

[Twisted-Python] Handling of spurious \r\n
by j.her＠t-online.de 10 May '01

10 May '01

glyph, I changed the fix to ignore an empty line at the start of a http request, as you suggested. At most ONE such line is ignored. And I extended the test case with such a request. Ciao, Jürgen

1 0

[Twisted-Python] Another brown paper bug release.
by Glyph Lefkowitz 09 May '01

09 May '01

I just put out 0.8.7, a bugfix release for .6, which was leaking resources until the webserver crashed, for a second time (while the release was on freshmeat's front page :-\). There are still a number of known problems, which continue to baffle me: twisted.protocols.http is buggy, especially _parse_command. I don't know why, yet: it *looks* right... There regular tracebacks in /var/www/twistd.log, and I'm pretty sure it's not because of a proliferation of buggy HTTP clients. These didn't appear before. POST doesn't work if you're in germany. Nobody can see this problem except for Jürgen, and then only to twistedmatrix.com; however, I think that it's a real bug. I am vaguely suspicious that this has something to do with protocols.basic.LineReceiver prematurely closing the process's connection, but I have absolutely *no* idea why. CGIs work flawlessly for me, both GET and POST, and I've constructed some pretty pathological tests. This whole incident indicates the need for a better testing procedure; now that people actually rely on twistedmatrix.com to do stuff, I think that the release process should take a few days in order to ensure that *all* aspects of the code are tested. Proposals for something more formal will be gladly accepted :) -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph

3 3