[IPython-dev] On sqlite history performance improvements...

[Brian Granger]

On Sat, Apr 9, 2011 at 1:51 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey Thomas,
>
> the history fixes of swapping the context manager and write loop do
> make a huge difference, so I've pushed your fixes already (it's the
> right algorithmic order, so no question on this change).  I rebased it
> to avoid a merge handle on a single commit.  Thanks a lot for this
> fix.
>
> But I think we still should look into further improving the
> performance/usability of the sqlite history support.  A few notes:
>
> - it's worth trying what happens with the writeout method in a thread.
>  If the python sqlite module releases the gil for the disk i/o
> operation (which I dearly hope it does), then it will be a net win and
> will completely solve the slight but annoying pause the system
> currently has on every prompt when using a laptop with a slow disk on
> battery.
>
> - we also need to expose the cache parameter as a configurable, though
> this may need to wait for the config work to be completed, not sure
> right now if the history object is already accessible to the config
> file mechanisms.
>
> - and we certainly want to have a magic to set it at runtime without
> the get_ipython()....whatever=10 dance.  Name? %history_buffer,
> %history_cache, %history_delay, ...?

I think we probably want to discuss a good model for this type of
thing.  Simple adding more magics to access IPython's runtime will
only lead to a bloated main namespace.  If we tied this type of thing
into the config sytem we might be able to get all of this done with a
single magic:

%config Foo.bar=10
%config InteractiveShell.autocall=False

I agree that the get_ipython() .... dance is not a very pleasant
model, but I think we can find something that will scale better in
covering IPython's runtime.

Cheers,

Brian

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com