[IPython-dev] Project idea: Automatic lab notebook for iPython

[Satrajit Ghosh]

hi max,

also do look at emerging provenance standard:

http://www.w3.org/TR/prov-dm/

cheers,

satra

On Fri, Apr 19, 2013 at 5:28 AM, Maximilian Albert <
maximilian.albert at gmail.com> wrote:

> Hi Peter,
>
> I wasn't aware of the Core Provenance Library but it looks like a very
> interesting project. I'll definitely check it out.
>
> On a related note, in case people haven't seen it I just wanted to throw
> out a link to the Sumatra  [1] project, which has a similar goal. However,
> as far as I could gather from its website, the Core Provenance Library
> seems to take a very different approach (by trying to be as unobtrusive for
> the workflow as possible, IIUC?). So I guess the link is not that useful
> for you, Peter, but I thought it might be of general interest. If anybody
> knows any other provenance tracking tools, I'd be interested to hear about
> them.
>
> Best regards,
> Max
>
> [1] http://pythonhosted.org/Sumatra/
>
>
> 2013/4/17 Peter Macko <pmacko at eecs.harvard.edu>
>
>> Hi iPython developers,
>>
>> Here is a new project idea: automatic lab notebook for iPython and
>> iPython Notebook, which would keep track of how each of your output
>> files was produced, linking this "history" (or a "lineage") of an object
>> across different iPython sessions and different iPython notebooks, and
>> storing it persistently. This is frequently referred to in the Computer
>> Science literature as "provenance."
>>
>> It will enable you to ask questions like "what did I do to produce this
>> plot?" - and for example, it will tell you that you downloaded the input
>> data set on Monday from such and such website, you ran all these
>> commands to process the data on Tuesday, and then produced this plot on
>> Thursday from a different iPython session. Note that this goes beyond
>> (and is complementary in purpose to) iPython Notebook, since the history
>> of a file is tracked across different sessions and Notebooks, and when
>> you ask a question, you will get only the relevant information,
>> suppressing any additional things that you did that are unrelated to the
>> file in which you are interested.
>>
>> We are in touch with computational scientists all the way from
>> bioinformatics to physics that are very interested in this feature! We
>> met their needs partially by developing a cross-platform, multi-lingual
>> library (https://code.google.com/p/core-provenance-library/) that they
>> can use to annotate their Python (and non-Python) scripts in order to
>> track the lineage of their objects.
>>
>> Our vision is that this will be all done fully automatically, without
>> requiring the users to manually annotate their scripts. But
>> unfortunately neither of us who are involved in this project has the
>> resources or the knowledge of the iPython code-base to tackle this
>> challenge. We need your help to make this happen! We have some ideas
>> about how we might go about this, but we need someone who knows more
>> about iPython to talk them over and to spearhead the actual development.
>> Please let us know if you can help!
>>
>> Thank you,
>>
>> Peter Macko
>>
>> Harvard School of Engineering and Applied Sciences
>> 33 Oxford St.
>> Cambridge, MA 02138
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130419/6b8fd782/attachment.html>