Mailman 3 - Twisted - python.org

[Twisted-Python] twisted.web split
by Moshe Zadka May 1, 2001

May 1, 2001

Hello, all. I talked a bit with glyph and radix about splitting twisted.web. It's over 1500 lines now, and I think it's becoming unmanageable. Here is how I propose it will be split: - twisted/web/html.py -- HTML generation helpers - twisted/web/protocol.py -- Web protocols (HTTP and Gloop, currently) - twisted/web/resource.py -- Resources. Specifically: - twisted/web/resource.py class Data(Resource): class Interface(Resource): class FileTransfer: class DirectoryListing(Interface): … [View More]

4 3

[Twisted-Python] GLOOP2 -- No more Mr. Passive Resistance
by Moshe Zadka April 30, 2001

April 30, 2001

Glyph and I talked a bit about revising Gloop. I don't have yet a proposal for the high level protocol, but I did put together a simple protocol for transferring objects. There's a super-pre-alpha implementation already, though. It's at http://moshez.org/gloop2 -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For … [View More]

1 0

[Twisted-Python] twisted.web uses twisted.protocols
by Moshe Zadka April 30, 2001

April 30, 2001

Well, not the official version, but here's a patch: Index: twisted/web.py =================================================================== RCS file: /cvs/TwistedPython/twisted/web.py,v retrieving revision 1.82 diff -c -r1.82 web.py *** twisted/web.py 2001/04/27 13:27:37 1.82 --- twisted/web.py 2001/04/30 04:56:59 *************** *** 29,34 **** --- 29,35 ---- from twisted import reflect from twisted import gloop from twisted import copyright + from twisted.protocols import http # … [View More]Useful constants *************** *** 206,219 **** ! class Request: ! """web.Request(blob) - This defines HTTP request. 'blob' is a list of lines that - represent the request to parse. - """ - - content = None code = HTTP.OK # HTML generation helpers --- 207,214 ---- ! class Request(http.HTTPRequestHandler): code = HTTP.OK # HTML generation helpers *************** *** 238,270 **** else: return name ! def __init__(self, blob): ! # parse the blob ! # ... ! split = string.split ! find = string.find ! lower = string.lower ! ! blobs = split(blob, '\r\n') ! reqstring = blobs.pop(0) ! received = self.received = {} ! args = self.args = {} self.stack = [] self.headers = {} - - reqfields = split(reqstring, ' ') ! assert 2 <= len(reqfields) <=3, "Invalid request format." - self.method, self.uri = reqfields[0], reqfields[1] - - if len(reqfields) == 2: - # we are dealing with HTTP/0.9 - self.clientproto = "HTTP/0.9" - else: - self.clientproto = reqfields[2] - - ###self.uri = urllib.unquote(self.uri) x = split(self.uri,'?') if len(x) == 1: --- 233,257 ---- else: return name ! def _parse_argstring(self, argstring, split=string.split): ! for kvp in split(argstring, '&'): ! keyval = map(urllib.unquote, split(kvp, '=')) ! if len(keyval) != 2: ! continue ! arg = self.args[key] = self.args.get(key, []) ! arg.append(value) ! ! def handleRequest(self, command, selector, version, headers, content): ! from string import split ! self.received = headers ! self.args = {} self.stack = [] self.headers = {} ! self.method, self.uri = command, selector ! self.clientproto = version ! self.content = content x = split(self.uri,'?') if len(x) == 1: *************** *** 273,311 **** if len(x) != 2: print "May ignore parts of this invalid URI:",repr(self.uri) self.path, argstring = urllib.unquote(x[0]), x[1] ! for kvp in split(argstring,'&'): ! keyval = map(urllib.unquote, split(kvp, '=')) ! if len(keyval)==2: ! key, value = keyval ! if args.has_key(key): ! args[key].append(value) ! else: ! args[key] = [value] ! for header in blobs: ! x = find(header,":") ! if x != -1: ! received[lower(header[:x])] = header[x+2:] ! else: ! print 'Invalid HTTP/1.1 protocol message?' ! print header ! try: ! self.requiredContent = int(received['content-length']) ! except: ! self.requiredContent = 0 def __repr__(self): return '<%s %s %s>'% (self.method, self.uri, self.clientproto) - - _host = socket.gethostbyaddr(socket.gethostname())[0] ! def process(self, server): "Process a request." # Log the request to a file. print self ! self.setHeader('server', Server.version) self.setHeader('date', date_time_string()) self.setHeader('content-type', "text/html") self.setHeader('connection', 'close') --- 260,279 ---- if len(x) != 2: print "May ignore parts of this invalid URI:",repr(self.uri) self.path, argstring = urllib.unquote(x[0]), x[1] ! self._parse_args(argstring) ! self.process() def __repr__(self): return '<%s %s %s>'% (self.method, self.uri, self.clientproto) _host = socket.gethostbyaddr(socket.gethostname())[0] ! def process(self): "Process a request." # Log the request to a file. print self ! self.setHeader('server', self.server.version) self.setHeader('date', date_time_string()) self.setHeader('content-type', "text/html") self.setHeader('connection', 'close') *************** *** 327,336 **** raise 'bad content-type' # Resource Identification ! self.server = server ! self.selector = server.selector ! self.server_port = server.port ! resource = server.resource(self) body = resource.render(self) if body == NOT_DONE_YET: return --- 295,303 ---- raise 'bad content-type' # Resource Identification ! self.selector = self.handler.server.selector ! self.server_port = self.handler.server.port ! resource = self.handler.server.resource(self) body = resource.render(self) if body == NOT_DONE_YET: return *************** *** 371,392 **** Write some data as a result of an HTTP request. The first time this is called, it writes out response data. """ ! if self.startedWriting: ! self.handler.write(data) ! else: self.startedWriting = 1 if self.clientproto != "HTTP/0.9": message = HTTP.responses.get(self.code, "Unknown Status") ! self.write("%s %s %s\r\n" % (HTTP.protocol_version, ! str(self.code), message) ) ! for header in self.headers.items(): ! self.write("%s: %s\r\n" % header) ! self.write('\r\n') ! self.write(data) def finish(self): ! self.handler.stopConsuming() ! del self.handler def setHeader(self, k, v): self.headers[string.lower(k)] = v --- 338,355 ---- Write some data as a result of an HTTP request. The first time this is called, it writes out response data. """ ! if not self.startedWriting: self.startedWriting = 1 if self.clientproto != "HTTP/0.9": message = HTTP.responses.get(self.code, "Unknown Status") ! self.sendStatus(self.code, message) ! for name, value in self.headers.items(): ! self.sendHeader(name, value) ! self.endHeaders() ! self.handler.write(data) def finish(self): ! self.handler.close() def setHeader(self, k, v): self.headers[string.lower(k)] = v *************** *** 1393,1447 **** class HTTPHandler(net.GenericHandler): ! contentLength = 0 request = None - recvd = "" def connectionLost(self, reason): self.request = None net.GenericHandler.connectionLost(self, reason) - def _process(self, request): - """ (private) """ - request.handler = self - request.process(self.server) - def handleData(self, data): ! recvd = self.recvd = self.recvd + data ! if not self.request: ! # HTTP/0.9 special handling. ! firstend = string.find(recvd, '\r\n') ! reqend = -1 ! if firstend != -1: ! first = recvd[:firstend] ! spaces = string.count(first, ' ') ! if spaces == 1: ! # we are HTTP/0.9 ! reqend = firstend ! ! if reqend < 0: ! # we didn't encounter 0.9, so proceed normally. ! reqend = string.find(recvd, '\r\n\r\n') ! ! if reqend != -1: ! reqdata = recvd[:reqend] ! self.recvd = recvd = recvd[reqend+4:] ! ! request = Request(reqdata) ! request.client = self.client ! request.handler = self ! cl = request.requiredContent ! if cl: ! self.contentLength = cl ! self.request = request ! else: ! self._process(request) ! return ! if self.contentLength: ! if len(self.recvd) >= self.contentLength: ! self.request.content = recvd[:self.contentLength] ! self._process(self.request) class Server(net.GenericServer, authenticator.SessionManager, config.Configurable): handler = HTTPHandler --- 1356,1376 ---- class HTTPHandler(net.GenericHandler): ! request = None def connectionLost(self, reason): self.request = None net.GenericHandler.connectionLost(self, reason) def handleData(self, data): ! if self.request is None: ! self.request = Request(self) ! self.request.server = self.server ! self.request.handleData(data) ! def close(self): ! self.loseConnection() class Server(net.GenericServer, authenticator.SessionManager, config.Configurable): handler = HTTPHandler [View Less]

1 0

[Twisted-Python] I was looking for offhanded ways to improve us.
by Glyph Lefkowitz April 29, 2001

April 29, 2001

Looks like 0.9 has been quite a while in coming. The Rt. Hon. Rev. Short and I have been discussing lots of interesting ideas for improvement of twisted reality; I quite some time looking at Squeak and considering strategies for adopting Smalltalk to improve the reliability of the core, but eventually gave up. It looks like the strategy for insulating the game engine from unstable code is going to be server distribution; much like twisted web allows multiple users to run untrusted (… [View More]except for the fact that gloop sends pickles, but this is an implementation deficiency not a design flaw) web servers on the same machine, I think that the central "reality reservoir" server on Zaibach will allow users to install doors to their areas. I also think that there's going to be a small paradigm shift in TR's approach to game code; I think that twisted.reality was attempting to be unrealistically game-agnostic. The core will start to make a few more assumptions about things like inventory, damage, and containment. The supporting "bulky" code (containing big string and tuple constants et. al.) will continue to live in twisted/library/, but I don't think it's possible to elegantly encapsulate anything meaningful in such a bare-bones simulation that's currently twisted/reality.py. I've written a draft "official twisted python coding standard", currently at http://www.twistedmatrix.com/users/glyph/standard.html Comments, corrections, and pointers to inconsistencies in the existing code will be welcomed, but if anybody says anything about NerdyCaps, I'm going to break some teeth. And finally, I'd like to see some discussion of a release strategy, as charming as "whenever we get around to it" has been :). I recommend something like this (note that I don't put dates next to *anything*... 0.8.5: the fixes which are currently in (CGI, refactorings to Twisted Reality, swing faucet, etc), preliminary demo code, segregation of Inheritance into a separate package, "dead" protocols directory. 0.9.0: "live" protocols implementation, used by at least .web (hopefully .gloop too); full, publicly running TR demo (assuming also some more improvements to the TR core, some more library functionality). 0.9.1: Simple distributed TR server. 0.9.2: Distributed TR server + SSL certificates. 0.9.3: Postgres protocol client, asynchronous DB API spec. 0.9.4: Some sort of relational storage for TR objects (?) 0.99.*: Writing tests and hammering the heck out of what we've got, adding minor features and polish to TR and the TR demo. 1.0.0: unit tests for everything. Media firestorm, we take radix's leash and dash and I run the mop-up operation in-character in the TR demo :) Here, there's some speculation as to whether we should do a fork to "stable" and "unstable" versions of TPy. Any thoughts? Further #s here assume such a fork doesn't happen. These are more of an overarching plan than specific functionality. 1.1.0: Rudimentary relational storage of Things, Radix's config interface. Finished version of Inheritance, packaged separately. Batteries not included. 1.2.0: "dict" protocol client and server for TR. More and better of the config interface. Meatier back-end for mail handling, real mail server (perhaps integrated with PMS?) 1.3.0: better Faucet, perhaps with PyGame. Start of work on _Inheritance_II:Acquisition_, which has a "massively" multiplayer "mode" as well as the multi- and single-player modes. (This will, in truth, be a different game, set in a similiar but much larger world.) The game won't be a strict sequel, but more in the way that Beyond Zork was a Zork sequel. 1.4.0: Fairly complete inet.d replacement: pop3, imap, telnet, talk, ident, IRC client-compatible server; all implemented with Moshe's protocols framework. 1.5.0: 100% for-real relational storage of TR objects, finished version of the schema and extensibility mechanisms for that schema. 1.6.0: Peer-to-peer TR additions; each client is a server too. Finish _Acquisition_, beginnings of serious work on an even larger game. 1.7.0: _Inheritance_III:Collection_, single-player only, but a testbed for features used in a larger game, mostly those which involve automatically generating large areas. If we do spatiality, here's where it gets exercised. 1.8.0: generalizations of peer-to-peer framework and trust web, to do something like mojo nation. 1.9.0: Robustness improvements. netcat from /dev/urandom to a gloop or twisted.web socket fails gracefully. I go to jail for a year because of features released with 1.8.0. 2.0.0: Full implementation of CORBA, NFS, XML-RPC, SMB, SSH, and NTP, all of which are TR-aware. Javascript interpreter. Enterprise. Highly concurrent resource control for multi-user Python. Moxie. Green eggs. Appleshare server and client. Ham. X font server. Kerberos. Zephyr. CODA. built-in remote debugging. Completely transparent python-relational interface with syntax additions built using MobiusPython. Kitchen sink. Bathroom sink. Implement "corridors" before Lilo gets around to it, move OPN to twistedmatrix.com. _Inheritance_IV:Subtype_Polymorphism_. 2.0.1: Internet. 2.0.2: Divunal. -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph [View Less]

3 5

[Twisted-Python] Twisted.Web memory usage, and other things
by Thyrr April 29, 2001

April 29, 2001

I'm planning on setting up a web server and was wondering how much memory TwistedPython uses up compared to Apache, especially since my RAM is limited. I would also like to know is TwistedPython supports virtual hosts at this time, and whether there's any way to have a persistant interpreter like mod_perl and mod_php. Oh, and while I'm asking questions, how does one create a multi-user interactive fiction? Strikes me as an odd concept as I've never seen one, but an interesting possibility. … [View More]

2 1

[Twisted-Python] twisted.protocols.http: RFC
by Moshe Zadka April 18, 2001

April 18, 2001

Glyph and I talked and decided the HTTP handling should be more flexible. Here is a preliminary patch for the flexibility: on finishing headers, the handler calls handleHeaders() Presumably, that call causes .setCallback(some_callable) to be called on the handler. Then, content chunks are passed back via some_callable(chunk) when there is no more, some_callable('') is called. I haven't checked it in yet. Please loot at HTTPCallBackHandler and let me know what you think. Of course, I can still … [View More]read everything into memory if I'm using HTTPRequestHandler. Please note that the handler is smart about content-length... -- Moshe Zadka <moshez(a)lerner.co.il> Web Developer, Python Developer import string from LineReceiver import LineReceiver class HTTPClient(LineReceiver): __length = None __buffer = '' def sendCommand(self, command, selector): self.write('%s %s HTTP/1.0\r\n' % (command, selector)) def sendHeader(self, name, value): self.write('%s: %s\r\n' % (name, value)) def endHeaders(self): self.write('\r\n') def handleLine(self, line): if line: self.handleHeader(line) else: self.handleEndHeaders() self.setRawMode() def handleEndHeaders(self): self.__buffer = self.__buffer + '\n' def handleRawData(self, data): if not data: self.handleResponse(self.__buffer) self.__buffer = '' return if self.length is not None: data, rest = data[:self.length], data[self.length:] self.length = self.length - len(data) else: rest = '' self.__buffer = self.__buffer + data if self.length == 0: self.handleResponse(self.__buffer) self.__buffer = '' def handleHeader(self, line): __buffer = __buffer + line + '\n' if string.find(line, 'content-length: ') == 0: self.length = int(string.strip(string.split(line, ':', 1)[0])) class HTTPHandler(LineReceiver): __length = 0 __header = '' __first_line = 1 def _parse_command(self, command): parts = string.split(command) if len(parts)<3: parts.append('HTTP/0.9') # isn't backwards compat great! if len(parts) != 3: self.sendError(405, 'Bad command') raise ValueError(str(parts)) return parts def sendStatus(self, code, resp=''): self.write('HTTP/1.0 %s %s\r\n' % (code, resp)) def sendHeader(self, name, value): self.write('%s: %s\r\n' % (name, value)) def endHeaders(self): self.write('\r\n') def sendError(self, code, resp=''): self.sendStatus(code, resp) self.endHeaders() def handleLine(self, line): if self.__first_line: self.__first_line = 0 command, request, version = self._parse_command(line) self.handleCommand(command, request, version) if version == 'HTTP/0.9': self.handleEndHeaders() self.callHandleEndContent() elif line == '': if self.__header: self.callHandleHeader(self.__header) self.__header = '' self.handleEndHeaders() if self.__length == 0: self.callHandleEndContent() else: self.setRawMode() elif line[0] in ' \t': self.__header = self.__header+'\n'+line else: if self.__header: self.callHandleHeader(self.__header) self.__header = line def callHandleHeader(self, line): assert line if string.find(string.lower(line), 'content-length: ') == 0: self.__length = int(string.strip(string.split(line, ':', 1)[1])) self.handleHeader(line) def callHandleEndContent(self): self.__first_line = 1 self.handleEndContent() def handleRawData(self, data): if not data: self.callHandleEndContent() if len(data) < self.__length: self.handleContentChunk(data) self.__length = self.__length - len(data) else: self.handleContentChunk(data[:self.__length]) self.callHandleEndContent() self.setLineMode(data[self.__length:]) from cStringIO import StringIO import rfc822 class HTTPHeadersHandler(HTTPHandler): def handleCommand(self, command, selector, version): self.__command = command self.__selector = selector self.__version = version self.__headers = StringIO() def handleHeader(self, line): self.__headers.write(line+'\n') def handleEndHeaders(self): self.__headers.write('\n') self.__headers.seek(0) headers = rfc822.Message(self.__headers) self.handleHeaders(self.__command, self.__selector, self.__version, headers) del self.__command, self.__selector, self.__version, self.__headers class HTTPCallbackOnContentHandler(HTTPHeadersHandler): def setCallBack(self, call): self.__call = call def handleContentChunk(self, data): self.__call(data) def handleEndContent(self): self.__call('') class HTTPRequestHandler(HTTPHeadersHandler): def handleHeaders(self, command, selector, version, headers): self.__command, self.__selector, self.__version, self.__headers = \ command, selector, version, headers self.__content = StringIO() def handleContentChunk(self, data): self.__content.write(data) def handleEndContent(self): data = self.__content.getvalue() del self.__content self.handleRequest(self.__command, self.__selector, self.__version, self.__headers, data) def _test(): class DummyHTTPHandler(HTTPRequestHandler): def __init__(self): import StringIO self.output = StringIO.StringIO() self.write = self.output.write def handleRequest(self, command, selector, version, headers, data): request = "'''\n"+str(headers)+"\n"+data+"'''\n" self.sendStatus(200, "OK") self.sendHeader("Request", selector) self.sendHeader("Command", command) self.sendHeader("Version", version) self.sendHeader("Content-Length", len(request)) self.endHeaders() self.write(request) requests = '''\ GET / HTTP/1.0 GET / HTTP/1.1 Accept: text/html POST / HTTP/1.1 Content-Length: 10 0123456789HEAD / ''' requests = string.replace(requests, '\n', '\r\n') expected_response = "HTTP/1.0 200 OK\015\012Request: /\015\012Command: GET\015\012Version: HTTP/1.0\015\012Content-Length: 9\015\012\015\012'''\012\012'''\012HTTP/1.0 200 OK\015\012Request: /\015\012Command: GET\015\012Version: HTTP/1.1\015\012Content-Length: 27\015\012\015\012'''\012Accept: text/html\012\012'''\012HTTP/1.0 200 OK\015\012Request: /\015\012Command: POST\015\012Version: HTTP/1.1\015\012Content-Length: 38\015\012\015\012'''\012Content-Length: 10\012\0120123456789'''\012HTTP/1.0 200 OK\015\012Request: /\015\012Command: HEAD\015\012Version: HTTP/0.9\015\012Content-Length: 9\015\012\015\012'''\012\012'''\012" a = DummyHTTPHandler() a.handleData(requests) value = a.output.getvalue() if value != expected_response: for i in range(len(value)): if value[i] != expected_response[i]: print `value[i-5:i+10]`, `expected_response[i-5:i+10]` break raise AssertionError if __name__ == '__main__': _test() [View Less]

1 0

[Twisted-Python] PIRP for Twisted Python
by Moshe Zadka April 18, 2001

April 18, 2001

Hello, dear Twisted developers! Attached is a twisted.protocol for PIRP, DJB's publishing protocol. I think it would be interesting to write an interface for twisted.web so Resources would be publishable both by the web and by PIRP (and possibly, later, other protocols?) The big point would probably to think up an interface to the request which is portable enough. I think something like: - request.components(): return the requested path as a list of components - request.received_headers(… [View More]

1 0

[Twisted-Python] First Port to twisted.protocols
by Moshe Zadka April 17, 2001

April 17, 2001

I'll check in the protocols stuff later today. In the mean time, for you early adopters, http://moshez.org/protocls.tgz has the latest and snazziest. And....as a special surprise, I've ported twisted.web over to stop knowing so much HTTP. HTTP knowledge used to be smeared all over Request and HTTPHandler, and now neither of them knows much HTTP.

1 3

[Twisted-Python] twisted.net refactoring
by Moshe Zadka April 17, 2001

April 17, 2001

Hi! I've started to work on refactoring twisted.net for minimal dependancies. Currently, in the vein of "research project", what interests me is how layering of protocols in an event-oriented framework can be accomplished. Please have a look at http://moshez.org/protocols.tgz I've written tests for all the protocol handlers I wrote, and I'd be happy to work with anyone who wants to write tests for the irc client and telnet server. Basically, the idea is to turn low-level events (basically just … [View More]

2 2

[Twisted-Python] weird problem with subclassing library.door.Door
by Chris Armstrong April 14, 2001

April 14, 2001

Ok, I made a subclass of Door called SecretDoor. The point of this door is that it's opened with a remote button and is not visible unless it is open. I tried implementing this by overriding _close() in my class. class SecretDoor(door.Door): def verb_open(self, sentence): sentence.subject.hears("This door is opened elsewhere.") def _close(self, actor): self._undescribe() Unfortunately, this did not work. The _undescribe call was apparently being cancelled out by … [View More]

1 0