[IPython-dev] [IPython-user] Development plans update
Ville M. Vainio
vivainio at gmail.com
Sun Feb 3 17:03:49 EST 2008
Very busy now, but few points.
> sense. Anytime you make a call to the core the call _may_ be put into
> a queue if the core is busy. This is a standard FIFO queue. The
> reason this happens is to enable a user to continue working on new
> input lines while the old ones compute. If you are familiar with
I'm not sure this should be the default behaviour. I like my stuff
synchronous, most of the time. I think there should be the synchronous
mode (where there are no problems with tab completion), and async mode
(where the delays with tab completion can occur on problematic
> Mathematica, you can edit and submit new input cells while old ones
> work. But, when the queue is active, everything goes through the
> queue - tab completion, prompt stuff. etc. This is done to make sure
> the entire thing is deterministic.
Tab completion does not need to be deterministic IMO. It's a very
"interactive" thing, and you can press tab a couple of times while you
wait for the completion to be "ready". Or the frontend can retry the
completion when it knows the queue is empty. But I still think the
completions should be served immediately in a thread.
> There is also another complication with prompts. The architecture
> already supports multiple users connecting to the same ipython
> engine/core process. All of the users frontends will have access to
> the current prompt number, but, you can imagine that the frontend has
> to in this case make decisions about what the prompts look like - the
> core simply can't as is knows nothing about how many user are
The frontend can just send the Itpl string to the core which expands
it. Of course a "generate_prompt" like hook would happen in the core,
but if that is what the user wants that's what he will get.
> But, there is another more important reason that tab completion can't
> always happen. I can give you an example - remember that every
> command that is sent to the core _may_ be put into a queue and
> executed later.
> In : run_some_long_calculation() # the core starts
> working on this and it will take 5 minutes
> In : a = Thing() # The user gets this prompt
> immediately and enters this command.
> # because the first
> command is still executing, this command is put on the
> # queue. It will be
> executed when the first one is done - in 5 minutes.
> In : a. <TAB> # Again, the user gets this
> prompt immediately. They want to tab complete
> # on a, but it is truly
> impossible as the command to create a has not been
> # run yet.
> As you can see, the other really fundamental change in ipython1 is
> that everything _may_ be completely asynchronous. This requires a
> huge paradigm shift in terms of thinking.
(see the solution above - retry until the queue is empty).
> > I think the display hooks should still be in core, but return strings
> > instead of direct printing.
> Yes, this is more of what it looks like. Currently when you call the
> method to run code in the core, it gives you back a python dict of
> various things (stdout, stderr, etc.). The display hooks could
> possible manipulate those dictionaries. But, it is important to
> remember that multple users (with different display desires) could be
> connected to the same core instance. Thus, I think that much of the
> display logic will need to be the frontends.
We can't pass "live" complicated objects to the frontend (pickling
wont work), so I think the issue is twofold:
- The display hooks on server output those dicts
(generics.result_display can return those).
- Frontend does whatever it wants with that dict.
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
More information about the IPython-dev