Mailman 3 September 2013 - Twisted

[Twisted-Python] Twisted Names EDNS(0) support [Was Twisted at PyConUK 2013]
by Richard Wall Sept. 17, 2013

Sept. 17, 2013

On 17 September 2013 16:34, Phil Mayers <p.mayers(a)imperial.ac.uk> wrote: > On 17/09/13 15:29, Richard Wall wrote: > >> PS. I'd love to merge https://tm.tl/5675 before the conference, so if >> you're familiar with the DNS RFCs, I'd really appreciate a code >> review. I'm rwall on #twisted-dev and #twisted if you want to discuss >> it > > > Deep breath... I'll give this a go. Thanks Phil, I'd seen your previous work in this area and was hoping you'd reply. :) You might be interested to read the reviews of https://twistedmatrix.com/trac/ticket/5675 where we introduced support for parsing of OPT pseudo records. Meanwhile, I'll see if I can help exarkun with the code review checklist. Thanks again and I look forward to your feedback. -RichardW.

3 2

Re: [Twisted-Python] Twisted Names EDNS(0) support [Was Twisted at PyConUK 2013]
by Richard Wall Sept. 17, 2013

Sept. 17, 2013

On 17 September 2013 17:18, Richard Wall <m-lists(a)the-moon.net> wrote: > You might be interested to read the reviews of > https://twistedmatrix.com/trac/ticket/5675 where we introduced support > for parsing of OPT pseudo records. Sorry, wrong link. This is the OPT record ticket - https://twistedmatrix.com/trac/ticket/5668 And this is the EDNS plan: * https://twistedmatrix.com/trac/wiki/EDNS0 -RichardW.

1 0

[Twisted-Python] [ANN] structlog – bring context & structure to your logs without boilerplate
by Hynek Schlawack Sept. 17, 2013

Sept. 17, 2013

Hi folks, today, I’ve release the 0.2.0 of my structlog project whose ambition is no less than changing the way we log in Python. :) And since it sports dedicated support for Twisted <http://www.structlog.org/en/latest/twisted.html> (but is not limited to it, it wraps *any* logger), I’d like to introduce you to it. Basically, its premise is that events happen in a context and you want to log out both effortlessly. Nowadays logs are usually consumed by parsers anyway, so regular, easily parsable logs are a good thing™. This is a nice example of a Twisted application showing off a bit what structlog is capable of: import sys import uuid import structlog import twisted from twisted.internet import protocol, reactor logger = structlog.getLogger() class Counter(object): i = 0 def inc(self): self.i += 1 def __repr__(self): return str(self.i) class Echo(protocol.Protocol): def connectionMade(self): self._counter = Counter() self._log = logger.new( connection_id=str(uuid.uuid4()), peer=self.transport.getPeer().host, count=self._counter, ) def dataReceived(self, data): self._counter.inc() log = self._log.bind(data=data) self.transport.write(data) log.msg('echoed data!') if __name__ == "__main__": structlog.configure( processors=[structlog.twisted.EventAdapter()], logger_factory=structlog.twisted.LoggerFactory(), ) twisted.python.log.startLogging(sys.stderr) reactor.listenTCP(1234, protocol.Factory.forProtocol(Echo)) reactor.run() It will give you an output like: 2013-09-17 17:40:23+0200 [-] Log opened. 2013-09-17 17:40:23+0200 [-] Factory starting on 1234 2013-09-17 17:40:23+0200 [-] Starting factory <twisted.internet.protocol.Factory instance at 0x108301488> 2013-09-17 17:40:28+0200 [Echo,0,127.0.0.1] peer='127.0.0.1' count=1 connection_id='4e2ee31c-b3ff-478e-ae06-7b1a492fce45' data='foo\n' event='echoed data!' 2013-09-17 17:40:33+0200 [Echo,0,127.0.0.1] peer='127.0.0.1' count=2 connection_id='4e2ee31c-b3ff-478e-ae06-7b1a492fce45' data='bar\n' event='echoed data!' 2013-09-17 17:40:44+0200 [Echo,1,127.0.0.1] peer='127.0.0.1' count=1 connection_id='3118f673-482b-471f-a206-e8f97f9a9c2c' data='qux\n' event='echoed data!' 2013-09-17 17:40:53+0200 [Echo,0,127.0.0.1] peer='127.0.0.1' count=3 connection_id='4e2ee31c-b3ff-478e-ae06-7b1a492fce45' data='twisted prevails\n' event='echoed data!' (you can find this and more examples at <http://www.structlog.org/en/latest/examples.html>) In short: - It allows you to build context by binding values to loggers. This context is just a dictionary. Once you log an event out (again, arbitrary number of key/value pairs), it gets merged with the context, processed by configurable processor chain and passed to your original logger. - Your loggers are immutable (by default infidels can use thread local storage) and you’ll get a new one on each binding. Immutable local data is awesome. - You can define processors that can mangle, filter, and format your log entries. - Configure once, then just call getLogger (which is a Twisted-friendly alias for get_logger) in regular code. - Of course there is a JSON renderer built right in – just tell log stash about it and be merry. *Please* have a look at <http://www.structlog.org/en/0.2.0-0/getting-started.html>, I don’t want to reproduce it here. :) Credit where credit is due: I picked up the idea of bound loggers from David Reid’s work on otter <https://github.com/rackerlabs/otter/tree/master/otter/log> which in turn was based on ideas by JP (I’ve been told). You can find everything you need to know at http://www.structlog.org/ – I’d be happy about any constructive feedback and even more so about contributions! Cheers, Hynek

1 0

[Twisted-Python] Python 3: bytes vs. str in twisted.python.filepath
by Harry Bock Sept. 15, 2013

Sept. 15, 2013

Hi all, My name is Harry Bock. I'm interested in helping out porting Twisted to Python 3, and I've popped in IRC a few times to introduce myself and ask a few questions. A few developers agreed that working on trial dependencies would be a big help. In doing some porting work on trial, I stumbled upon a previous porting effort (possibly by Itamar?) for twisted.python.filepath and related modules. It seemed like the porting effort included forcing all pathname inputs to be byte strings instead of native strings. After some investigation, I believe this is the wrong approach, but I wanted to start a discussion here first. Some thoughts: (a) As of Python 3.3, use of the ANSI API in Windows is deprecated[1], so many functions in os and os.path raise DeprecationWarning when given byte strings as input. Although win32 is not an initial target of the porting effort, we'll have to support it eventually and the API should be supported before then. (b) Misunderstandings at the application level about the underlying filesystem's path encoding is not the problem of the Twisted API. Correct me if I'm wrong, but that's the responsibility of the system administrator or individual user (at least on UNIX) to set the LANG environment variable, or for the application to call setlocale(3) to explicitly override it. (c) If we do not allow unicode strings, we will be forcing the application developer to decide how to encode paths when using the FilePath API. Per (b) above, the user will have to call sys.getfilesystemencoding()[2] to divine what encoding to use before using the API at all, which to me is terribly annoying and would just add str.encode calls everywhere. Thus, my vote is that on Python 2.x, Twisted should accept either the native str or unicode types for path names, and on Python 3.x, only accept the str type to prevent deprecation issues with system calls. I have a patch set that will make this happen including unittest modifications; if there's a consensus I'm happy to open a ticket and submit the patches. Thanks! [1] http://bugs.python.org/issue13374 [2] http://docs.python.org/3/library/sys.html#sys.getfilesystemencoding

5 7

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com Sept. 15, 2013

Sept. 15, 2013

1 0

[Twisted-Python] Deferreds / promises as first-class objects for function calls
by Terry Jones Sept. 12, 2013

Sept. 12, 2013

I just wrote a blog post that people who like thinking about deferreds might find interesting. Somehow it took me 7 years to one day look at deferreds in this way: http://blogs.fluidinfo.com/terry/2013/09/12/promises-are-first-class-object… (With apologies for mainly describing things in terms of promises instead of deferreds.) Terry

1 0

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com Sept. 8, 2013

Sept. 8, 2013

1 0

[Twisted-Python] Passing additional arguments to errback
by Maciej Wasilak Sept. 6, 2013

Sept. 6, 2013

Dear list, I've found such a code example on "Stack Overflow" regarding errbacks: ___________________________ class YourExample(object): def your_example(self): self.agent = Agent(reactor, pool=pool) self.deferred = self.agent.request( 'GET', self.url, Headers({'User-Agent': ['Mozilla/5.0']}) ) self.deferred.addCallback(self.gotResponse).addErrback(self.gotBadResponse) def gotBadResponse(self,raised): """you might have cleanup code here, or mark the url as bad in the database, or something similar""" pass __________________________ Normally only Failure object is passed to gotBadResponse() . I would like to pass full response body to it - is it possible? Do I have to encapsulate the response body inside Failure object? Best regards Maciek

3 7

[Twisted-Python] Can't check elements without uri in twisted.words.xish.domish.Element
by Goffi Sept. 6, 2013

Sept. 6, 2013

G'day, in the method "elements" of twisted.words.xish.domish.Element, the function "generateElementsQNamed" is called event if the uri if None, so the uri is checked against None instead of not checked at all. I think the function "generateElementsNamed" should be called instead, it looks like a bug for me. Can you confirm ? cheers Goffi

2 2

[Twisted-Python] ftpclient connection lost callback
by Peter Ryan Sept. 5, 2013

Sept. 5, 2013

I am writing a simple ftp client which I am trying to dispose of. Right now I send a quit command: (the ftp_client reference is a FTPClient built with a ClientCreator) deferred = ftp_client.quit() And I register a callback: def quitSent(response): print "quit acknowledged...connected?", ftp_client.transport.connected deferred.addCallback(quitSent) If I invoke the above the connection is printed as connected. However if I schedule a callback to check the status later: def status(): print "quit acknowledged...connected?", ftp_client.transport.connected reactor.callLater(1, self.status) If I schedule a callback for a second later the transport is not connected. I've searched quite a bit but I cannot determine a way to register a callback for when the TCP transport is actually disconnected. Can anyone offer some insight? I'd like to know explicitly when the client is closed. Thanks, Peter

2 3