[IPython-dev] Storing data …

Brian Granger ellisonbg at gmail.com
Wed Jul 10 12:45:00 EDT 2013


On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:
> Hi
>
> I'd like to associate data generated within the notebook with the actual notebook for that session.
>
> Right now I'm writing files to the file system, but that places the additional burden of having the manage the data storage.
>
> [1] How are people dealing with the task of storing data (which may be specific to that ipython notebook, or even better, specific to the notebook's session).

For the most part we are encouraging users to store notebook+data in
git repos.  Trying to bundle data inside a notebook would pull the
notebook format in a direction we don't want to go - plus git is
perfect for this as long as the data is not too big (if it is big see
git-annex).  With that said, MIn had a nice hack that stored data in
the notebook document.  Min, do you remember where this is?

But part of our philosophy is that pretty much no code, data or
notebooks should ever live outside a version controlled directory.  We
are pretty fanatical about this.  When we think about the notebook
promoting reproducible and sharable science, version control is
critically important.  Once you accept this perspective, it becomes
unthinkable for a notebook to ever be found outside its repo where its
data lives.  Part of the reason we are fanatical about this is that
the second you start to imagine other ways of bundling other assets
with notebooks, you are heading down a path of 1) reinventing version
control systemsor 2) you are doing silly things like zipping up all of
the files into a impenetrable binary blob.

> [2] Is there something (existing or planned) in the API that allows me to store session level data in the ipynb?

I don't think we will go in this direction.

Cheers,

Brian

> Thanks,
>
> F
>
> Francis Reyes
> Advanced Light Source
> 1 Cyclotron Road
> Berkeley, CA 94720
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com



More information about the IPython-dev mailing list