Is there existing code to log-with-bells-on for runtime algorithm diagnostics?
Hi, I'm in need of a system for logging the step-wise results and diagnostic metadata about a python function implementation of an algorithm that I'm developing. The specific algorithm is not of great consequence except that it's for scientific computing and may produce large (e.g., '00s or maybe '000s, but not "big data" scale) amounts of intermediate numerical data that can be complex to understand when debugging its progress. In fact, I'm trying to build a general purpose tool for exploring the inner workings of numerical algorithms for teaching and learning purposes, e.g. for graduate student training or for figuring out parameter choices in difficult applications. I want to be able to insert commands inside of the loops that log certain variable states, completed stages of the algorithm (assume it's hierarchical), text warnings, and possibly even 'pointers' to graphical output objects of some of the intermediate data (e.g. matplotlib object handles for lines, points). Then I can trace the work done afterwards or step through the process in an IDE debugger and make interactive calls to access recent steps, plot certain relationships in the current state, etc. The basic logger's "levels" of output don't really apply here. I at least want categories, if not hierarchical sub-categories. I don't think the built-in logger is sophisticated enough for this, being a flat record of freeform text AFAIU, but the API looks appealing. I'm considering an in-memory sqlite DB to store structured records at any logged step, and an accompanying dictionary to store references to any python object metadata, keyed by a unique ID in the DB log. Then I'd write an API for it that resembles the logger's. It's not super hard for me to write my own thing here, but I'm wondering if anyone has come across any existing solutions in this vein, or has any advice before I go further in designing a solution? I can't really believe that no-one has attempted this before, but it's been really hard to find any existing work through online search. Thanks, Rob
On 21 April 2015 at 12:02, Rob Clewley <rob.clewley@gmail.com> wrote:
The basic logger's "levels" of output don't really apply here. I at least want categories, if not hierarchical sub-categories.
Python's built-in logging has a hierarchy of loggers according to the name - you can set up log handling for the logger named 'foo', and things from the logger 'foo.bar' will go to that. The convention is to instantiate each logger with a module name, so you can configure logging by package, but I think you could use it with any hierarchy. It is text-only, as far as I know, but it might save you some work. Thomas
On Tue, Apr 21, 2015 at 8:02 PM, Rob Clewley <rob.clewley@gmail.com> wrote:
Hi,
I'm in need of a system for logging the step-wise results and diagnostic metadata about a python function implementation of an algorithm that I'm developing. The specific algorithm is not of great consequence except that it's for scientific computing and may produce large (e.g., '00s or maybe '000s, but not "big data" scale) amounts of intermediate numerical data that can be complex to understand when debugging its progress.
In fact, I'm trying to build a general purpose tool for exploring the inner workings of numerical algorithms for teaching and learning purposes, e.g. for graduate student training or for figuring out parameter choices in difficult applications.
The term you want to search for is "structured logging". http://www.structlog.org/en/stable/ http://eliot.readthedocs.org/en/stable/ https://twiggy.readthedocs.org/en/latest/logging.html#structured-logging http://netlogger.lbl.gov/ -- Robert Kern
All of these ideas and links are very helpful, thank you! -Rob
Just to follow up on this thread, for interested readers' future reference... On Tue, Apr 21, 2015 at 4:22 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Tue, Apr 21, 2015 at 8:02 PM, Rob Clewley <rob.clewley@gmail.com> wrote:
In fact, I'm trying to build a general purpose tool for exploring the inner workings of numerical algorithms for teaching and learning purposes, e.g. for graduate student training or for figuring out parameter choices in difficult applications.
The term you want to search for is "structured logging".
http://www.structlog.org/en/stable/ http://eliot.readthedocs.org/en/stable/ https://twiggy.readthedocs.org/en/latest/logging.html#structured-logging http://netlogger.lbl.gov/
I posted a new blog entry about my prototypical diagnosis and visualization tools for python numerical algorithms, built over matplotlib: http://robclewley.github.io/logging-and-diagnostic-tools-for-numeric-python-... They utilize structlog, and I hooked up a noSQL DB (tinydb) to the logging to enable search capabilities of the log post-mortem. Comments and PRs most welcome. Thanks to everyone for the pointers. -Rob
participants (3)
-
Rob Clewley -
Robert Kern -
Thomas Kluyver