[IPython-dev] parallel ouptut before exception

[David Froger]

> There is a way to do this, but it's not provided by IPython by default. The
> reason being that right now, IPython collates output according to which engine
> it comes from, so it can't provide incremental output. You can stream outputs
> as they arrive by watching the metadata, you just want to be sure to identify
> which engine everything is coming from as they arrive.

Hello MinRK,

Do mean like using AsyncResult, as you show in this gist?
    http://nbviewer.ipython.org/gist/minrk/6171348

Actually, I'm not trying get incremental output, just to get the final output,
after running a cell in blocking mode.

Sorry if I missunderstood something...

For now, the solution I've found is using the logging module with a StreamHandler:

file foo.py:
    import logging
    def foo():
        logger = logging.getLogger('foo')
        logger.debug("message")
        raise ValueError, "This is a value error"

Notebook:
[0] from IPython.parallel import Client
    client = Client(profile="mpi") # cluster was started with 4 procs

[1] %%px --block
    import logging
    import StringIO
    def initialize_logger(name, level):
        output = StringIO.StringIO()
        logger = logging.getLogger(name)
        logger.setLevel(level)
        stream_handler = logging.StreamHandler(output)
        stream_handler.setLevel(level)
        logger.addHandler(stream_handler)
        return output
    output = initialize_logger('foo', logging.DEBUG)

[2] %%px --block
    import foo
    foo.foo()

[3] %%px
    print output.getvalue()