[IPython-dev] Storing data …

MinRK benjaminrk at gmail.com
Wed Jul 10 14:16:09 EDT 2013


On Wed, Jul 10, 2013 at 9:45 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:
> > Hi
> >
> > I'd like to associate data generated within the notebook with the actual
> notebook for that session.
> >
> > Right now I'm writing files to the file system, but that places the
> additional burden of having the manage the data storage.
> >
> > [1] How are people dealing with the task of storing data (which may be
> specific to that ipython notebook, or even better, specific to the
> notebook's session).
>
> For the most part we are encouraging users to store notebook+data in
> git repos.  Trying to bundle data inside a notebook would pull the
> notebook format in a direction we don't want to go - plus git is
> perfect for this as long as the data is not too big (if it is big see
> git-annex).  With that said, MIn had a nice hack that stored data in
> the notebook document.  Min, do you remember where this is?
>

My guess is you are referring to this one<http://nbviewer.ipython.org/5303304>,
where I made download links to in-memory data (which technically does embed
the data into the notebook document).



>
> But part of our philosophy is that pretty much no code, data or
> notebooks should ever live outside a version controlled directory.  We
> are pretty fanatical about this.  When we think about the notebook
> promoting reproducible and sharable science, version control is
> critically important.  Once you accept this perspective, it becomes
> unthinkable for a notebook to ever be found outside its repo where its
> data lives.  Part of the reason we are fanatical about this is that
> the second you start to imagine other ways of bundling other assets
> with notebooks, you are heading down a path of 1) reinventing version
> control systemsor 2) you are doing silly things like zipping up all of
> the files into a impenetrable binary blob.
>
> > [2] Is there something (existing or planned) in the API that allows me
> to store session level data in the ipynb?
>
> I don't think we will go in this direction.
>
> Cheers,
>
> Brian
>
> > Thanks,
> >
> > F
> >
> > Francis Reyes
> > Advanced Light Source
> > 1 Cyclotron Road
> > Berkeley, CA 94720
> >
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130710/0fa7907b/attachment.html>


More information about the IPython-dev mailing list