[Twisted-Python] Announcing Eliot 0.7: Logging for complex and distributed systems (with nice Twisted support)
Imagine you have a client that sends a request to a server and gets back a 500 error. What caused it? If you're using Eliot you can trace the logs across processes: $ cat server.log client.log | python eliottree.py e076ca50-9abc-44b2-95d8-85cf6956bc33 +-- main@1/started |-- process: client `-- timestamp: 2015-04-28 16:50:00.379033 +-- eliot:remote_task@2,2,1/started |-- process: server `-- timestamp: 2015-04-28 16:50:00.389533 +-- http_request@2,1/started |-- process: client |-- timestamp: 2015-04-28 16:50:00.379096 |-- x: 8 `-- y: 0 +-- divide@2,2,2,1/started |-- process: server |-- timestamp: 2015-04-28 16:50:00.389674 |-- x: 8 `-- y: 0 +-- divide@2,2,2,2/failed |-- exception: exceptions.ZeroDivisionError |-- process: server |-- reason: integer division or modulo by zero `-- timestamp: 2015-04-28 16:50:00.389724 +-- eliot:remote_task@2,2,3/failed |-- exception: exceptions.ZeroDivisionError |-- process: server |-- reason: integer division or modulo by zero `-- timestamp: 2015-04-28 16:50:00.389750 +-- http_request@2,3/failed |-- exception: requests.exceptions.HTTPError |-- process: client |-- reason: 500 Server Error: INTERNAL SERVER ERROR `-- timestamp: 2015-04-28 16:50:00.396112 +-- main@3/failed |-- exception: requests.exceptions.HTTPError |-- process: client |-- reason: 500 Server Error: INTERNAL SERVER ERROR `-- timestamp: 2015-04-28 16:50:00.396181 Intrigued? Learn more at https://eliot.readthedocs.org [1]. eliottree.py is an add-on utility for formatting Eliot logs, and can be found at https://github.com/jonathanj/eliottree [2] Links: ------ [1] https://eliot.readthedocs.org [2] https://github.com/jonathanj/eliottree
On Apr 28, 2015, at 1:57 PM, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
Imagine you have a client that sends a request to a server and gets back a 500 error. What caused it? If you're using Eliot you can trace the logs across processes:
$ cat server.log client.log | python eliottree.py e076ca50-9abc-44b2-95d8-85cf6956bc33 +-- main@1/started |-- process: client `-- timestamp: 2015-04-28 16:50:00.379033
(etc) This looks cooler all the time. You could probably make it look even cooler by using unicode line-drawing characters :-). Also, is there a web frontend which can render these? Will there be? Will any of the additional structuring or filtering of log messages from the new logging system be helpful to Elliot? If so, do you have any pointers for Eeshan where he might improve existing messages? -glyph
On 2015-04-28 18:15, Glyph Lefkowitz wrote:
You could probably make it look even cooler by using unicode line-drawing characters :-).
Jonathan Jacobs is responsible for the neat UI.
Also, is there a web frontend which can render these? Will there be?
Since the underlying structure is a sequence of independent messages you can e.g. feed Eliot messages into ElasticSearch and then use Kibana, which is quite helpful. This does involve a little bit of "manual" tracing of the tree structure (sorting by the relevant column, basically.) I haven't spent much time using Eliot in anger due to last year's pivot at work so haven't thought that much about building UIs yet.
Will any of the additional structuring or filtering of log messages from the new logging system be helpful to Elliot? If so, do you have any pointers for Eeshan where he might improve existing messages?
At work we have a thing that routes Twisted logs into Eliot; being able to route structured messages will correspondingly improve the quality of the resulting logs. Insofar as Eliot's tree structure is callstack based (with additional support for Deferreds) most Twisted (or stdlib, or whatever) log messages will typically end up in the correct place in the action tree if you route them to Eliot.
participants (2)
-
Glyph Lefkowitz
-
Itamar Turner-Trauring