[IPython-dev] On sqlite history performance improvements...

Fernando Perez fperez.net at gmail.com
Sat Apr 9 16:51:03 EDT 2011


Hey Thomas,

the history fixes of swapping the context manager and write loop do
make a huge difference, so I've pushed your fixes already (it's the
right algorithmic order, so no question on this change).  I rebased it
to avoid a merge handle on a single commit.  Thanks a lot for this
fix.

But I think we still should look into further improving the
performance/usability of the sqlite history support.  A few notes:

- it's worth trying what happens with the writeout method in a thread.
 If the python sqlite module releases the gil for the disk i/o
operation (which I dearly hope it does), then it will be a net win and
will completely solve the slight but annoying pause the system
currently has on every prompt when using a laptop with a slow disk on
battery.

- we also need to expose the cache parameter as a configurable, though
this may need to wait for the config work to be completed, not sure
right now if the history object is already accessible to the config
file mechanisms.

- and we certainly want to have a magic to set it at runtime without
the get_ipython()....whatever=10 dance.  Name? %history_buffer,
%history_cache, %history_delay, ...?

Cheers,

f



More information about the IPython-dev mailing list