Mailman 3 January 2005 - Twisted-web

Test
by Philippe Lafoucrière 05 Aug '21

05 Aug '21

Hi all just a test

3 2

various formless types
by andrea＠cpushare.com 16 Feb '05

16 Feb '05

These are some formless types I'm using in my app. I think it's a good idea to share it with you, since they can be useful not just to me. The reason of the webform.freeformDefaultContentPattern change is to use div instead of stan, I'm doing some strange stuff, so without div it becomes not compliant html. I need the context as last parameter of the processor handler, not in the below types, but in others. Note also the RequiredChoice, I believe the check "x not in self.choices" is required for security reasons, most people will expect the client does the right thing, but it's not forced to without such check. I only use requiredchoice and never choice. I also wonder if it really worth the new API that slows down it so much with the normal rendering, but let's say that's a more general problem with compy and getinterfaces. Plus the new API is way confusing, integers gets converted to strings and if you conver them back to integers with valueToKey the processor will forget the value, the processor remembers the value only if the coerce returns a string. That might actually be a bug. Anyway the below workaround in my requiredchoice works (i.e. always try to convert to int in the coerce for the security check, but keep returning a string). I only use lists of strings or integers as choices for requiredchoice of course, so I'm fine. class IntegerRange(annotate.Integer): def __init__(self, min = None, max = None, *args, **kw): super(IntegerRange, self).__init__(*args, **kw) self.min = min self.max = max self.requiredFailMessage = 'Please enter an integer between %d and %d.' \ % (self.min, self.max) def coerce(self, val, configurable): v = super(IntegerRange, self).coerce(val, configurable) if v > self.max or v < self.min: raise annotate.InputError("Integer must be between %d and %d." % (self.min, self.max)) return v class RealRange(annotate.Real): def __init__(self, min = None, max = None, *args, **kw): super(RealRange, self).__init__(*args, **kw) self.min = min self.max = max self.requiredFailMessage = 'Please enter a real number between %d and %d.' \ % (self.min, self.max) def coerce(self, val, configurable): v = super(RealRange, self).coerce(val, configurable) if v > self.max or v < self.min: raise annotate.InputError("Real must be between %d and %d." % (self.min, self.max)) return v class MaxlengthString(annotate.String): def __init__(self, *args, **kwargs): if 'maxlength' in kwargs: self.maxlength = kwargs['maxlength'] del kwargs['maxlength'] else: self.maxlength = 10 super(MaxlengthString, self).__init__(*args, **kwargs) def coerce(self, val, configurable): r = super(MaxlengthString, self).coerce(val, configurable) if search_for_invalid_chars.search(r): raise annotate.InputError("%r contains invalid characters." % val) if len(r) > self.maxlength: raise annotate.InputError("%r is too long, maximum is %d." % (val, self.maxlength)) return r def invalid_email(email): return not re.match(r'^[\w\-~@.+]+\Z', email) or \ re.search(r'@[^@]*@', email) or \ not re.match(r'^[\w\-~.+]+@.*[\w\-~+]+\.[\w]{2,}\Z', email) class Email(MaxlengthString): def coerce(self, val, configurable): r = super(Email, self).coerce(val, configurable) if invalid_email(val): raise annotate.InputError("%r is not a valid email address." % val) return r class MaxlengthStringRenderer(webform.StringRenderer): def input(self, context, slot, data, name, value): if data.typedValue.getAttribute('hidden'): T='hidden' else: T='text' return slot[ tags.input(id=formutils.keyToXMLID(context.key), type=T, name=name, value=value, maxlength=data.typedValue.maxlength, _class="freeform-input-%s" % T)] class ProcessTypedContext(compy.Adapter): __implements__ = iformless.IInputProcessor, def process(self, context, boundTo, data): """data is a list of strings at this point """ typed = self.original val = data[0] if typed.unicode: try: val = val.decode(getPOSTCharset(context), 'replace') except LookupError: val = val.decode('utf-8', 'replace') if typed.strip: val = val.strip() if val == '' or val is None: if typed.required: raise annotate.InputError(typed.requiredFailMessage) else: return typed.null return typed.coerce(val, boundTo, context) class MaxlengthPassword(MaxlengthString): requiredFailMessage = 'Please enter a password.' class MaxlengthPasswordRenderer(webform.PasswordRenderer): def input(self, context, slot, data, name, value): return [ tags.input(id=formutils.keyToXMLID(context.key), name=name, type="password", _class="freeform-input-password", maxlength=data.typedValue.maxlength), " Again ", tags.input(name="%s____2" % name, type="password", _class="freeform-input-password", maxlength=data.typedValue.maxlength), ] class Decimal(annotate.Typed): requiredFailMessage = 'Please enter a decimal number.' def coerce(self, val, configurable): # TODO: This shouldn't be required; check. # val should never be None, but always a string. if val is None: return None try: return decimal.Decimal(val) except decimal.InvalidOperation: raise annotate.InputError("%r is not a decimal number." % val) class ForcedBoolean(annotate.Boolean): pass class ForcedBooleanRenderer(webform.BooleanRenderer): def input(self, context, slot, data, name, value): ## The only difference here is the "checked" attribute; the value is still the same because ## we want true to be passed to the server when the checkbox is checked and the form ## is posted. node = tags.input(id=formutils.keyToXMLID(context.key), type="checkbox", name=name, value='True', _class="freeform-input-checkbox") if value: node(checked="checked") # HTML forms are so weak. If the checkbox is not checked, no value at all will be # in request.args with the name data.name. So let's force the value False to always # be in request.args[data.name]. If the checkbox is checked, the value True will # be first, and we will find that. return slot[node, tags.input(type="hidden", name=name, value="")] class RequiredChoice(annotate.Choice): requiredFailMessage = 'Please choose an option.' def coerce(self, val, configurable): r = super(RequiredChoice, self).coerce(val, configurable) if self.required and r == self.default: raise annotate.InputError(self.requiredFailMessage) try: x = int(val) except ValueError: x = val if x not in self.choices: raise annotate.InputError("Your web browser is malfunctioning") return r compy.registerAdapter(MaxlengthStringRenderer, MaxlengthString, iformless.ITypedRenderer) compy.registerAdapter(MaxlengthStringRenderer, Email, iformless.ITypedRenderer) compy.registerAdapter(MaxlengthPasswordRenderer, MaxlengthPassword, iformless.ITypedRenderer) compy.registerAdapter(ReadonlyTextRenderer, ReadonlyText, iformless.ITypedRenderer) compy.registerAdapter(webform.ChoiceRenderer, RequiredChoice, iformless.ITypedRenderer) compy.registerAdapter(ForcedBooleanRenderer, ForcedBoolean, iformless.ITypedRenderer) compy.registerAdapter(processors.ProcessPassword, MaxlengthPassword, iformless.IInputProcessor) webform.freeformDefaultContentPattern = tags.invisible[ tags.label(_class="freeform-label", _for=stan.slot('id'))[ stan.slot('label') ], tags.div(_class="freeform-input")[ stan.slot('input') ], tags.div(_class="freeform-error")[ stan.slot('error') ], tags.div(_class="freeform-description")[tags.label(_for=stan.slot('id'))[ stan.slot('description') ]]].freeze()

2 1

load balancing and performance
by Andrea Arcangeli 03 Feb '05

03 Feb '05

I successfully load balanced the static server with pythondirector. However it has a few problems: 1) it cannot handle ssl 2) even if it can handle ssl I still need to share the session cookies and I'm thinking that using sql or atop to do that isn't necessary: the session doesn't need to survive a reboot, a reboot would be a visible disruption of the services anyway, so I'm thinking to write a little session daemon that only stores the cookies using pb to do that. In theory shared memory would be much more efficient in smp, but at least this socket model has the advantage of scaling in the cluster too 3) the load balancer misses an API with the real webserver to pass up the client IP address, that's annoying, especially the logs gets screwed Other than that the speed of the load balancer is excellent, and I get exactly the double number of pages rendered per second (after fixing a small bug in pythondirector that confused ab2). Perhaps I'm going with wrong priorities though, the major offender is compy, compy must be dropped from Nevow ASAP :). Leaving it as a compatibility API may be ok, but internally compy can't invoke getComponents anymore if we do care about writing remotely optimal code. We should try with the new zope.interfaces API first and see if it underperforms so horribly as getInterfaces does. Secondly I'm looking into caching the html and to render some fragment only once every 10 seconds in the background (so the downloads will never have to wait for a rendering of some mostly static fragment anymore). So overall there's an huge room for improvements. What do other people think?

7 32

Source of twisted websites
by Sridhar Ratna 30 Jan '05

30 Jan '05

Hi, I hope both divmod.org and twistedmatrix.com runs from twisted.web. Can you point me to the source code (or svn repos) for those websites? Also is there any site, that makes use of twisted.web.distrib? -- Sridhar Ratna - http://srid.bsdnerds.org

2 2

Page object design: wrapping a dictionary containing data
by Mary Gardiner 30 Jan '05

30 Jan '05

I've got a Page object that derives all its data from a dictionary, eg: { 'project_name': 'project', 'project_description': 'hello', # ... you get the idea } What's a good way for the Page object to wrap that dictionary that doesn't involve doing this: def data_name(self, ctx, data): return self.d['project_name'] def data_description(self, ctx, data): return self.d['project_description'] # and so on for every key in the dictionary I vaguely recall that the equivalent of this used to be possible in Woven: def data_project(self, ctx, data): return self.d and then that the data items were accessible by (say) stan.directive("project/project_name"). This doesn't look like its supported in Nevow. Is this correct? What alternative designs are there for something like this? -Mary

7 12

default virtual server page fails
by Andrea Arcangeli 29 Jan '05

29 Jan '05

It seems self is null. Using the class name rather than the null instance worked around it. (I don't need this fix since I pass my own default page, but I found it while writing my code) Index: nevow/vhost.py =================================================================== --- nevow/vhost.py (revision 1131) +++ nevow/vhost.py (working copy) @@ -19,7 +19,7 @@ """ def getStyleSheet(self): - return self.stylesheet + return VirtualHostList.stylesheet def data_hostlist(self, context, data): return self.nvh.hosts.keys()

1 0

Nevow error with unicode localization in appserver.py
by Paul Reznicek 29 Jan '05

29 Jan '05

Below suggested patches to omit broken output, if unicode is returned instead of string. Problem occurred during localization with gettext, see this sample code: ------------------------------------------------------------------- import gettext from twisted.internet import reactor from twisted.protocols import http from nevow import inevow, rend, loaders, tags, appserver, static, guard class Index(rend.Page): addSlash = True child_css = static.File('static/css') child_img = static.File('static/img') docFactory = loaders.xmlfile('static/index.html') def renderHTTP(self, ctx): request = inevow.IRequest(ctx) # Dynamic LOCALIZATION =============================================== # inspired by http://www.ospace.net/index.en.html language = request.getHeader('accept-language').split(',')[0][:2] try: tran = gettext.translation('PROJECT', 'static/locale', languages = [language]) except IOError: print 'Cannot find catalog for "', language, '", installing null translations' tran = gettext.NullTranslations() tran.install(unicode = 1) # ==================================================================== # got from: nevow/examples/http_auth.py username, password = request.getUser(), request.getPassword() if (username, password) == ('', ''): request.setHeader('WWW-Authenticate', 'Basic realm="Whatever"') request.setResponseCode(http.UNAUTHORIZED) return _("Authentication required.") # THIS RETURN UNICODE ## They provided a username and password, so let's let them in! hurray self.data_username, self.data_password = username, password return rend.Page.renderHTTP(self, ctx) site = appserver.NevowSite(Index()) reactor.listenTCP(8080, site) reactor.run() ------------------------------------------------------------------- Paul Reznicek Necessary patch: =================================================================== --- nevow/appserver.py (Revision 1126) +++ nevow/appserver.py (local) @@ -167,8 +167,8 @@ self.deferred.callback("") def _cbFinishRender(self, html, ctx): - if isinstance(html, str): - self.write(html) + if isinstance(html, basestring): + self.write(html.encode('utf8')) server.Request.finish(self) elif html is errorMarker: ## Error webpage has already been rendered and finish called I believe, the wsgi.py can have similar problem: =================================================================== --- nevow/wsgi.py (Revision 1126) +++ nevow/wsgi.py (local) @@ -84,8 +84,8 @@ p = rend.FourOhFour() result = p.renderHTTP(pctx) - if isinstance(result, str): - request.write(result) + if isinstance(result, basestring): + request.write(result.encode('utf8')) else: ## Exhaust the generator list(result)

2 2

How to develop a module to download a file from a path
by Mosas 29 Jan '05

29 Jan '05

Dear All, I want know how to develop a module to download a file from a path using twisted.web.static.File Class . I think using this class we can create easily Actually I am new to twisted concepts. So If anybody know regarding this mail me. regards prabahar ________________________________________________________________________ Yahoo! India Matrimony: Find your life partner online Go to: http://yahoo.shaadi.com/india-matrimony

1 1

Nevow error with unicode localization in appserver.py REVISED
by Paul Reznicek 28 Jan '05

28 Jan '05

REVISED: After some other tests - there must be encoding ONLY on unicode strings, otherwise binary file transfer fails (try nevow/examples/image_uploader) Paul Reznicek Necessary patch: =================================================================== --- appserver.py (Revision 1126) +++ appserver.py (local) @@ -170,6 +170,9 @@ if isinstance(html, str): self.write(html) server.Request.finish(self) + elif isinstance(html, unicode): + self.write(html.encode('utf8')) + server.Request.finish(self) elif html is errorMarker: ## Error webpage has already been rendered and finish called pass I believe, the wsgi.py can have similar problem: =================================================================== --- wsgi.py (Revision 1126) +++ wsgi.py (local) @@ -86,6 +86,8 @@ result = p.renderHTTP(pctx) if isinstance(result, str): request.write(result) + elif isinstance(result, unicode): + request.write(result.encode('utf8')) else: ## Exhaust the generator list(result)

1 0

Re: [Twisted-web] livepage breaks after reload
by noema 28 Jan '05

28 Jan '05

Ahh this is killing me! Part of the problem that I described earlier in this thread is that simply adding a locateChild stops javascript events from triggering the connected python function/methode. I am trying to render the same page no matter what url (past the domain) is given. This single page should render differently depending on the url, though. So in locateChild I store the url segments and return self and an empty segment tuple. This seems to work but, as I said, breaks the livepage functionality. WHY is locateChild DOING thaaAAt???? What follows is some code that illustrates exactly that. Javascript events come perfectly through until I uncommend the localChild methode. #noema #################### START OF .TAC FILE ######################## import random from twisted.application import service, internet from nevow import loaders, rend, tags, livepage, url class MyPage(rend.Page): addSlash = True #def locateChild(self, context, segments): # #store the rest of the url for any kind of manual processing # self.postpath = segments # return self, () def onLinkAction(self, client): client.set('atag', random.choice(('one', 'two', 'three', 'four', 'five'))) def render_link(self, context, data): return tags.a(onclick=livepage.handler(self.onLinkAction), href=url.here, id="atag")['click!'] def render_glue(self, context, data): return livepage.glue docFactory = loaders.stan( tags.html[ tags.head[ tags.title["mypage"], ], tags.body[ tags.div[ render_link ], tags.span(render=render_glue) ] ]) application = service.Application('mypage') internet.TCPServer(8080, livepage.LiveSite(MyPage()))\ .setServiceParent(application) #################### END OF .TAC FILE ##########################

2 4