[IPython-dev] [IPython-user] Development plans update

[Ville M. Vainio]

Very busy now, but few points.

> sense.  Anytime you make a call to the core the call _may_ be put into
> a queue if the core is busy.  This is a standard FIFO queue.  The
> reason this happens is to enable a user to continue working on new
> input lines while the old ones compute.  If you are familiar with

I'm not sure this should be the default behaviour. I like my stuff
synchronous, most of the time. I think there should be the synchronous
mode (where there are no problems with tab completion), and async mode
(where the delays with tab completion can occur on problematic
non-GIL-releasing code).

> Mathematica, you can edit and submit new input cells while old ones
> work.  But, when the queue is active, everything goes through the
> queue - tab completion, prompt stuff. etc.  This is done to make sure
> the entire thing is deterministic.

Tab completion does not need to be deterministic IMO. It's a very
"interactive" thing, and you can press tab a couple of times while you
wait for the completion to be "ready". Or the frontend can retry the
completion when it knows the queue is empty. But I still think the
completions should be served immediately in a thread.

> There is also another complication with prompts.  The architecture
> already supports multiple users connecting to the same ipython
> engine/core process.  All of the users frontends will have access to
> the current prompt number, but, you can imagine that the frontend has
> to in this case make decisions about what the prompts look like - the
> core simply can't as is knows nothing about how many user are
> connected.

The frontend can just send the Itpl string to the core which expands
it. Of course a "generate_prompt" like hook would happen in the core,
but if that is what the user wants that's what he will get.

> But, there is another more important reason that tab completion can't
> always happen.  I can give you an example - remember that every
> command that is sent to the core _may_ be put into a queue and
> executed later.
>
> In [1]: run_some_long_calculation()             # the core starts
> working on this and it will take 5 minutes
>
> In [2]: a = Thing()                      # The user gets this prompt
> immediately and enters this command.
>                                                # because the first
> command is still executing, this command is put on the
>                                                # queue.  It will be
> executed when the first one is done - in 5 minutes.
>
> In [3]: a.     <TAB>                   # Again, the user gets this
> prompt immediately.  They want to tab complete
>                                                # on a, but it is truly
> impossible as the command to create a has not been
>                                                # run yet.
>
> As you can see, the other really fundamental change in ipython1 is
> that everything _may_ be completely asynchronous.  This requires a
> huge paradigm shift in terms of thinking.

(see the solution above - retry until the queue is empty).
> > I think the display hooks should still be in core, but return strings
> > instead of direct printing.
>
> Yes, this is more of what it looks like.  Currently when you call the
> method to run code in the core, it gives you back a python dict of
> various things (stdout, stderr, etc.).  The display hooks could
> possible manipulate those dictionaries.  But, it is important to
> remember that multple users (with different display desires) could be
> connected to the same core instance.  Thus, I think that much of the
> display logic will need to be the frontends.

We can't pass "live" complicated objects to the frontend (pickling
wont work), so I think the issue is twofold:

- The display hooks on server output those dicts
(generics.result_display can return those).
- Frontend does whatever it wants with that dict.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'