[IPython-dev] Storing data …

[Brian Granger]

Ahh, yes thanks Min!

On Wed, Jul 10, 2013 at 11:16 AM, MinRK <benjaminrk at gmail.com> wrote:
>
>
>
> On Wed, Jul 10, 2013 at 9:45 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> On Wed, Jul 10, 2013 at 9:24 AM, Francis Reyes <fereyes at lbl.gov> wrote:
>> > Hi
>> >
>> > I'd like to associate data generated within the notebook with the actual
>> > notebook for that session.
>> >
>> > Right now I'm writing files to the file system, but that places the
>> > additional burden of having the manage the data storage.
>> >
>> > [1] How are people dealing with the task of storing data (which may be
>> > specific to that ipython notebook, or even better, specific to the
>> > notebook's session).
>>
>> For the most part we are encouraging users to store notebook+data in
>> git repos.  Trying to bundle data inside a notebook would pull the
>> notebook format in a direction we don't want to go - plus git is
>> perfect for this as long as the data is not too big (if it is big see
>> git-annex).  With that said, MIn had a nice hack that stored data in
>> the notebook document.  Min, do you remember where this is?
>
>
> My guess is you are referring to this one, where I made download links to
> in-memory data (which technically does embed the data into the notebook
> document).
>
>
>>
>>
>> But part of our philosophy is that pretty much no code, data or
>> notebooks should ever live outside a version controlled directory.  We
>> are pretty fanatical about this.  When we think about the notebook
>> promoting reproducible and sharable science, version control is
>> critically important.  Once you accept this perspective, it becomes
>> unthinkable for a notebook to ever be found outside its repo where its
>> data lives.  Part of the reason we are fanatical about this is that
>> the second you start to imagine other ways of bundling other assets
>> with notebooks, you are heading down a path of 1) reinventing version
>> control systemsor 2) you are doing silly things like zipping up all of
>> the files into a impenetrable binary blob.
>>
>> > [2] Is there something (existing or planned) in the API that allows me
>> > to store session level data in the ipynb?
>>
>> I don't think we will go in this direction.
>>
>> Cheers,
>>
>> Brian
>>
>> > Thanks,
>> >
>> > F
>> >
>> > Francis Reyes
>> > Advanced Light Source
>> > 1 Cyclotron Road
>> > Berkeley, CA 94720
>> >
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com