[IPython-dev] parallel ouptut before exception
David Froger
david.froger.ml at mailoo.org
Fri Jun 19 05:09:40 EDT 2015
> There is a way to do this, but it's not provided by IPython by default. The
> reason being that right now, IPython collates output according to which engine
> it comes from, so it can't provide incremental output. You can stream outputs
> as they arrive by watching the metadata, you just want to be sure to identify
> which engine everything is coming from as they arrive.
Hello MinRK,
Do mean like using AsyncResult, as you show in this gist?
http://nbviewer.ipython.org/gist/minrk/6171348
Actually, I'm not trying get incremental output, just to get the final output,
after running a cell in blocking mode.
Sorry if I missunderstood something...
For now, the solution I've found is using the logging module with a StreamHandler:
file foo.py:
import logging
def foo():
logger = logging.getLogger('foo')
logger.debug("message")
raise ValueError, "This is a value error"
Notebook:
[0] from IPython.parallel import Client
client = Client(profile="mpi") # cluster was started with 4 procs
[1] %%px --block
import logging
import StringIO
def initialize_logger(name, level):
output = StringIO.StringIO()
logger = logging.getLogger(name)
logger.setLevel(level)
stream_handler = logging.StreamHandler(output)
stream_handler.setLevel(level)
logger.addHandler(stream_handler)
return output
output = initialize_logger('foo', logging.DEBUG)
[2] %%px --block
import foo
foo.foo()
[3] %%px
print output.getvalue()
More information about the IPython-dev
mailing list