[IPython-dev] History
Brian Granger
ellisonbg at gmail.com
Fri Feb 18 13:06:11 EST 2011
On Fri, Feb 18, 2011 at 9:11 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2/18/11 9:48 AM, Fernando Perez wrote:
>
>> Finally, but importantly, I'm somewhat reluctant to go to sqlite until
>> we've fully shown that accomplishing our design goals is
>> hard/impossible with a simple json persisted data structure. While
>> sqlite is indeed lightweight and available to us, it's also a more
>> complex api to program against than simply storing a list/dict on
>> disk. I'm OK accepting that complexity price *if we need it*, but I'm
>> not convinced we do yet. Specifically, we need to answer: what is
>> precisely the functionality that we want, that is hard/impossible to
>> implement on json and easy/possible with sqlite?
>
> Well, if the computation resources devoted to translating raw input to their
> executable form is important, then I suspect the serialization to JSON of the
> whole history every time it gets saved would often dominate it (translation
> happens rarely; loading/saving happens often).
>
> SQLite is safer.
> SQLite allows concurrency.
> SQLite allows us to store absolutely everything we have ever typed, but only
> load what we want to use in a session.
Absolutely, I think these points are *very* important and make sqlite
worth thinking about.
> SQLite is more easily extended to also record outputs. You get a basic notebook
> format practically for free! :-)
Some thoughts about this:
* The history is a chronological picture of the command that happen on
a particular engine session.
* The notebook is a "spatial" picture of a subset of commands that is
completely independent of a particular session.
* The history can contain things that are not in a notebook (from a
terminal IPython) and a notebook can contain things not in a history.
* In the past, we have envisioned the notebook server as being a
completely separate process/server than the kernel. For examples, you
could imagine a centralized notebook server for a group that is used
by frontends/kernels running all over the world.
* Notebooks are inherently multiuser (notebooks in a notebook server
are associated with users, they can be shared, etc.).
But...it is an interesting idea to combine the kernel and notebook
server. There could be some significant benefits for this, but it is
definitely not the model we have been thinking. Any ideas about how
this would work?
One other thought: possibly, something like the history storage
format should be abstracted through an API that we could implement
using sqlite/json/mongodb/etc.
Cheers,
Brian
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> that is made terrible by our own mad attempt to interpret it as though it had
> an underlying truth."
> -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com
More information about the IPython-dev
mailing list