[Twisted-Python] clean up app.getApplication
i was doing something else, couldn't quite understand the code at first (where does the listcomp begin, where does it end?), so here's a more explicit version: Index: app.py =================================================================== RCS file: /cvs/Twisted/twisted/application/app.py,v retrieving revision 1.3 diff -u -r1.3 app.py --- app.py 21 Sep 2003 01:07:23 -0000 1.3 +++ app.py 27 Sep 2003 22:16:16 -0000 @@ -81,9 +81,14 @@ return None def getApplication(config, passphrase): - s = [(config[t], t) - for t in ['python', 'xml', 'source', 'file'] if config[t]][0] - filename, style = s[0], {'file':'pickle'}.get(s[1],s[1]) + for t in ['python', 'xml', 'source', 'file', 'tapdir', None]: + style = t + filename = config[style] + if filename: + break + if style == 'file': + style = 'pickle' + try: log.msg("Loading %s..." % filename) application = service.loadApplication(filename, style, passphrase)
On Sun, Sep 28, 2003 at 12:17:00AM +0200, Paul Boehm wrote:
That original version was pretty icky, yea. Thangs for posting this here (though someone may tell you it belongs on the bug tracker at http://www.twistedmatrix.com/bugs/, which it probably does), because it reminded me of something we should be able to get rid of now. "python" style is indicated by the string "python". "xml" style is indicated by the string "xml". "source" style is indicated by the string "source". "pickle" style is indicated by the string "file". Now, let's play a game that everyone who's watched Seasame Street should know -- Which of these things is not like the others? Is there any reason *not* to fix this now? Jp
-- Examinations are formidable even to the best prepared, for even the greatest fool may ask more than the wisest man can answer. -- C.C. Colton
On Sat, Sep 27, 2003 at 06:37:23PM -0400, Jp Calderone wrote:
ugh.. using webinterfaces without a mouse is horror.. so i'll do it again and submit another patch here: Index: strports.py =================================================================== RCS file: /cvs/Twisted/twisted/application/strports.py,v retrieving revision 1.1 diff -u -r1.1 strports.py --- strports.py 26 Sep 2003 22:44:48 -0000 1.1 +++ strports.py 27 Sep 2003 22:56:19 -0000 @@ -46,12 +46,21 @@ """ from __future__ import generators +_types = {} + +def registerType(name, helper): + global _types + _types[name] = helper + print _types + def _parseTCP(factory, port, interface="", backlog=5): return (int(port), factory), {'interface': interface, 'backlog': int(backlog)} +registerType("tcp", _parseTCP) def _parseUNIX(factory, address, mode='666', backlog=5): return (address, factory), {'mode': int(mode, 8), 'backlog': int(backlog)} +registerType("unix", _parseUNIX) def _parseSSL(factory, port, privateKey="server.pem", certKey=None, sslmethod=None, interface='', backlog=5): @@ -64,10 +73,7 @@ cf = ssl.DefaultOpenSSLContextFactory(privateKey, certKey, **kw) return ((int(port), factory, cf), {'interface': interface, 'backlog': int(backlog)}) - -_funcs = {"tcp": _parseTCP, - "unix": _parseUNIX, - "ssl": _parseSSL} +registerType("ssl", _parseSSL) _OP, _STRING = range(2) def _tokenize(description): @@ -149,7 +155,7 @@ args, kw = _parse(description) if not args or (len(args)==1 and not kw): args[0:0] = [default or 'tcp'] - return (args[0].upper(),)+_funcs[args[0]](factory, *args[1:], **kw) + return (args[0].upper(),)+_types[args[0]](factory, *args[1:], **kw) def service(description, factory, default=None): """Return the service corresponding to a description
On Sun, Sep 28, 2003 at 12:58:04AM +0200, Paul Boehm wrote:
ugh.. using webinterfaces without a mouse is horror.. so i'll do it again and submit another patch here:
i've updated the patch (see end of mail) all trial tests pass sample code why i need the patch included also, i want to ask the list if everyone is okay with me getting cvs access to create and maintain twisted.proxy, a generic proxy, currently capable of transparent proxying using netfilter, pseudotransparent proxying using a shared lib (seperate c code required), and web.proxy, which i'll update or rewrite soon. and a rfc: in the sample code below, i find the way i handle listening in a service instead of the application, a lot more natural. if i stopService() at runtime, i expect the ports of that service to go down at the same time. anything speaking against implementing the neccesary code in application.Service? my expectation is that after the app refactor, noone will want to listen directly in application anymore, so we could even start deprecating the old interface. sample code / usecase: ----------------------------------- # twisted imports from twisted.application import strports from twisted.application.service import Service # sibling imports from proxy.interfaces import * from proxy import adapters flavors = { 'any': IProxyFactory, # a proxy, any proxy! 'transparent': ITransProxyFactory, # a transparent proxy 'http': IHTTPProxyFactory, # a proxy speaking HTTP } def parsePROXY(type, service, port, subproto="any", transport="tcp"): adapter = flavors[subproto] return(transport.upper(), (int(port), adapter(service)), {}) strports.registerTransport("proxy", parsePROXY) class ProxyService(Service): """I am a service that starts proxies, and stores shared policy for them.""" __implements__ = IProxyService def __init__(self, name, parent, *portspecs): self.portstrs = [] self.listeners = [] self.setName(name) self.setServiceParent(parent) for pspec in portspecs: self.listen(pspec) def listen(self, description): self.portstrs.append(description) def doListen(self, description): transport = strports.listen(description, self) self.listeners.append(transport) def startService(self): Service.startService(self) for description in self.portstrs: self.doListen(description) def stopService(self): Service.stopService(self) for port in self.listeners: port.loseConnection() self.listeners.remove(port) ----------------------------------- patch: Index: strports.py =================================================================== RCS file: /cvs/Twisted/twisted/application/strports.py,v retrieving revision 1.1 diff -u -r1.1 strports.py --- strports.py 26 Sep 2003 22:44:48 -0000 1.1 +++ strports.py 28 Sep 2003 02:50:51 -0000 @@ -46,14 +46,22 @@ """ from __future__ import generators -def _parseTCP(factory, port, interface="", backlog=5): - return (int(port), factory), {'interface': interface, +_transportHelpers = {} + +def registerTransport(name, helper): + global _transportHelpers + _transportHelpers[name] = helper + +def _parseTCP(type, factory, port, interface="", backlog=5): + return type, (int(port), factory), {'interface': interface, 'backlog': int(backlog)} +registerTransport("tcp", _parseTCP) -def _parseUNIX(factory, address, mode='666', backlog=5): - return (address, factory), {'mode': int(mode, 8), 'backlog': int(backlog)} +def _parseUNIX(type, factory, address, mode='666', backlog=5): + return type, (address, factory), {'mode': int(mode, 8), 'backlog': int(backlog)} +registerTransport("unix", _parseUNIX) -def _parseSSL(factory, port, privateKey="server.pem", certKey=None, +def _parseSSL(type, factory, port, privateKey="server.pem", certKey=None, sslmethod=None, interface='', backlog=5): from twisted.internet import ssl if certKey is None: @@ -62,12 +70,9 @@ if sslmethod is not None: kw['sslmethod'] = getattr(ssl.SSL, sslmethod) cf = ssl.DefaultOpenSSLContextFactory(privateKey, certKey, **kw) - return ((int(port), factory, cf), + return (type, (int(port), factory, cf), {'interface': interface, 'backlog': int(backlog)}) - -_funcs = {"tcp": _parseTCP, - "unix": _parseUNIX, - "ssl": _parseSSL} +registerTransport("ssl", _parseSSL) _OP, _STRING = range(2) def _tokenize(description): @@ -104,7 +109,7 @@ add(sofar) return args, kw -def parse(description, factory, default=None): +def parse(description, factory, default='tcp'): """Parse a description of a reliable virtual circuit server @type description: C{str} @@ -148,8 +153,10 @@ """ args, kw = _parse(description) if not args or (len(args)==1 and not kw): - args[0:0] = [default or 'tcp'] - return (args[0].upper(),)+_funcs[args[0]](factory, *args[1:], **kw) + type = default + else: + type = args.pop(0) + return _transportHelpers[type](type.upper(), factory, *args, **kw) def service(description, factory, default=None): """Return the service corresponding to a description @@ -185,4 +192,4 @@ name, args, kw = parse(description, factory, default) return getattr(reactor, 'listen'+name)(*args, **kw) -__all__ = ['parse', 'service', 'listen'] +__all__ = ['registerTransport', 'parse', 'service', 'listen']
On Sun, 28 Sep 2003 04:58:00 +0200 Paul Boehm <paul@soniq.net> wrote:
Personally I'd prefer not having this in Twisted itself. Not sure why though :P -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting
Itamar Shtull-Trauring wrote:
"separate C code required"? That seems reason enough to exclude it for me. What about forwarding is computationally intense enough to *require* C code? Especially separate C code?
On Sun, Sep 28, 2003 at 02:30:01AM -0400, Glyph Lefkowitz wrote:
huh? you really didn't bother to read what i wrote in irc. there's not a single line of c code in proxy, no mention of c code, and no requirement for c code. yes, i've written a shared library interface to one protocol.. and i'm not even going to publish that.. annoyed, paul
Paul Boehm wrote:
huh? you really didn't bother to read what i wrote in irc.
Nope. If you want responses on the mailing list, write to the mailing list :)
there's not a single line of c code in proxy, no mention of c code, and no requirement for c code.
Ahem -
That seems to be a mention of both c code and a requirement for c code. I haven't seen anything either on IRC or here to indicate how this seperate, required C code is neither required nor C :)
yes, i've written a shared library interface to one protocol.. and i'm not even going to publish that..
I understand that this comment may not apply in this situation, but I'd like to be clear: the reason I responded as I did is that I don't like the idea of adding plugin interfaces to Twisted specifically to introduce a dependency on so that unpublished, proprietary, or even separate modules are required. Sometimes that's unavoidable (support for proprietary platforms requires proprietary code) but unless there is an overriding concern it's undesireable.
On Sun, Sep 28, 2003 at 12:17:00AM +0200, Paul Boehm wrote:
That original version was pretty icky, yea. Thangs for posting this here (though someone may tell you it belongs on the bug tracker at http://www.twistedmatrix.com/bugs/, which it probably does), because it reminded me of something we should be able to get rid of now. "python" style is indicated by the string "python". "xml" style is indicated by the string "xml". "source" style is indicated by the string "source". "pickle" style is indicated by the string "file". Now, let's play a game that everyone who's watched Seasame Street should know -- Which of these things is not like the others? Is there any reason *not* to fix this now? Jp
-- Examinations are formidable even to the best prepared, for even the greatest fool may ask more than the wisest man can answer. -- C.C. Colton
On Sat, Sep 27, 2003 at 06:37:23PM -0400, Jp Calderone wrote:
ugh.. using webinterfaces without a mouse is horror.. so i'll do it again and submit another patch here: Index: strports.py =================================================================== RCS file: /cvs/Twisted/twisted/application/strports.py,v retrieving revision 1.1 diff -u -r1.1 strports.py --- strports.py 26 Sep 2003 22:44:48 -0000 1.1 +++ strports.py 27 Sep 2003 22:56:19 -0000 @@ -46,12 +46,21 @@ """ from __future__ import generators +_types = {} + +def registerType(name, helper): + global _types + _types[name] = helper + print _types + def _parseTCP(factory, port, interface="", backlog=5): return (int(port), factory), {'interface': interface, 'backlog': int(backlog)} +registerType("tcp", _parseTCP) def _parseUNIX(factory, address, mode='666', backlog=5): return (address, factory), {'mode': int(mode, 8), 'backlog': int(backlog)} +registerType("unix", _parseUNIX) def _parseSSL(factory, port, privateKey="server.pem", certKey=None, sslmethod=None, interface='', backlog=5): @@ -64,10 +73,7 @@ cf = ssl.DefaultOpenSSLContextFactory(privateKey, certKey, **kw) return ((int(port), factory, cf), {'interface': interface, 'backlog': int(backlog)}) - -_funcs = {"tcp": _parseTCP, - "unix": _parseUNIX, - "ssl": _parseSSL} +registerType("ssl", _parseSSL) _OP, _STRING = range(2) def _tokenize(description): @@ -149,7 +155,7 @@ args, kw = _parse(description) if not args or (len(args)==1 and not kw): args[0:0] = [default or 'tcp'] - return (args[0].upper(),)+_funcs[args[0]](factory, *args[1:], **kw) + return (args[0].upper(),)+_types[args[0]](factory, *args[1:], **kw) def service(description, factory, default=None): """Return the service corresponding to a description
On Sun, Sep 28, 2003 at 12:58:04AM +0200, Paul Boehm wrote:
ugh.. using webinterfaces without a mouse is horror.. so i'll do it again and submit another patch here:
i've updated the patch (see end of mail) all trial tests pass sample code why i need the patch included also, i want to ask the list if everyone is okay with me getting cvs access to create and maintain twisted.proxy, a generic proxy, currently capable of transparent proxying using netfilter, pseudotransparent proxying using a shared lib (seperate c code required), and web.proxy, which i'll update or rewrite soon. and a rfc: in the sample code below, i find the way i handle listening in a service instead of the application, a lot more natural. if i stopService() at runtime, i expect the ports of that service to go down at the same time. anything speaking against implementing the neccesary code in application.Service? my expectation is that after the app refactor, noone will want to listen directly in application anymore, so we could even start deprecating the old interface. sample code / usecase: ----------------------------------- # twisted imports from twisted.application import strports from twisted.application.service import Service # sibling imports from proxy.interfaces import * from proxy import adapters flavors = { 'any': IProxyFactory, # a proxy, any proxy! 'transparent': ITransProxyFactory, # a transparent proxy 'http': IHTTPProxyFactory, # a proxy speaking HTTP } def parsePROXY(type, service, port, subproto="any", transport="tcp"): adapter = flavors[subproto] return(transport.upper(), (int(port), adapter(service)), {}) strports.registerTransport("proxy", parsePROXY) class ProxyService(Service): """I am a service that starts proxies, and stores shared policy for them.""" __implements__ = IProxyService def __init__(self, name, parent, *portspecs): self.portstrs = [] self.listeners = [] self.setName(name) self.setServiceParent(parent) for pspec in portspecs: self.listen(pspec) def listen(self, description): self.portstrs.append(description) def doListen(self, description): transport = strports.listen(description, self) self.listeners.append(transport) def startService(self): Service.startService(self) for description in self.portstrs: self.doListen(description) def stopService(self): Service.stopService(self) for port in self.listeners: port.loseConnection() self.listeners.remove(port) ----------------------------------- patch: Index: strports.py =================================================================== RCS file: /cvs/Twisted/twisted/application/strports.py,v retrieving revision 1.1 diff -u -r1.1 strports.py --- strports.py 26 Sep 2003 22:44:48 -0000 1.1 +++ strports.py 28 Sep 2003 02:50:51 -0000 @@ -46,14 +46,22 @@ """ from __future__ import generators -def _parseTCP(factory, port, interface="", backlog=5): - return (int(port), factory), {'interface': interface, +_transportHelpers = {} + +def registerTransport(name, helper): + global _transportHelpers + _transportHelpers[name] = helper + +def _parseTCP(type, factory, port, interface="", backlog=5): + return type, (int(port), factory), {'interface': interface, 'backlog': int(backlog)} +registerTransport("tcp", _parseTCP) -def _parseUNIX(factory, address, mode='666', backlog=5): - return (address, factory), {'mode': int(mode, 8), 'backlog': int(backlog)} +def _parseUNIX(type, factory, address, mode='666', backlog=5): + return type, (address, factory), {'mode': int(mode, 8), 'backlog': int(backlog)} +registerTransport("unix", _parseUNIX) -def _parseSSL(factory, port, privateKey="server.pem", certKey=None, +def _parseSSL(type, factory, port, privateKey="server.pem", certKey=None, sslmethod=None, interface='', backlog=5): from twisted.internet import ssl if certKey is None: @@ -62,12 +70,9 @@ if sslmethod is not None: kw['sslmethod'] = getattr(ssl.SSL, sslmethod) cf = ssl.DefaultOpenSSLContextFactory(privateKey, certKey, **kw) - return ((int(port), factory, cf), + return (type, (int(port), factory, cf), {'interface': interface, 'backlog': int(backlog)}) - -_funcs = {"tcp": _parseTCP, - "unix": _parseUNIX, - "ssl": _parseSSL} +registerTransport("ssl", _parseSSL) _OP, _STRING = range(2) def _tokenize(description): @@ -104,7 +109,7 @@ add(sofar) return args, kw -def parse(description, factory, default=None): +def parse(description, factory, default='tcp'): """Parse a description of a reliable virtual circuit server @type description: C{str} @@ -148,8 +153,10 @@ """ args, kw = _parse(description) if not args or (len(args)==1 and not kw): - args[0:0] = [default or 'tcp'] - return (args[0].upper(),)+_funcs[args[0]](factory, *args[1:], **kw) + type = default + else: + type = args.pop(0) + return _transportHelpers[type](type.upper(), factory, *args, **kw) def service(description, factory, default=None): """Return the service corresponding to a description @@ -185,4 +192,4 @@ name, args, kw = parse(description, factory, default) return getattr(reactor, 'listen'+name)(*args, **kw) -__all__ = ['parse', 'service', 'listen'] +__all__ = ['registerTransport', 'parse', 'service', 'listen']
On Sun, 28 Sep 2003 04:58:00 +0200 Paul Boehm <paul@soniq.net> wrote:
Personally I'd prefer not having this in Twisted itself. Not sure why though :P -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting
Itamar Shtull-Trauring wrote:
"separate C code required"? That seems reason enough to exclude it for me. What about forwarding is computationally intense enough to *require* C code? Especially separate C code?
On Sun, Sep 28, 2003 at 02:30:01AM -0400, Glyph Lefkowitz wrote:
huh? you really didn't bother to read what i wrote in irc. there's not a single line of c code in proxy, no mention of c code, and no requirement for c code. yes, i've written a shared library interface to one protocol.. and i'm not even going to publish that.. annoyed, paul
Paul Boehm wrote:
huh? you really didn't bother to read what i wrote in irc.
Nope. If you want responses on the mailing list, write to the mailing list :)
there's not a single line of c code in proxy, no mention of c code, and no requirement for c code.
Ahem -
That seems to be a mention of both c code and a requirement for c code. I haven't seen anything either on IRC or here to indicate how this seperate, required C code is neither required nor C :)
yes, i've written a shared library interface to one protocol.. and i'm not even going to publish that..
I understand that this comment may not apply in this situation, but I'd like to be clear: the reason I responded as I did is that I don't like the idea of adding plugin interfaces to Twisted specifically to introduce a dependency on so that unpublished, proprietary, or even separate modules are required. Sometimes that's unavoidable (support for proprietary platforms requires proprietary code) but unless there is an overriding concern it's undesireable.
participants (4)
-
Glyph Lefkowitz -
Itamar Shtull-Trauring -
Jp Calderone -
Paul Boehm