[IPython-dev] Notebook format "incompatible" changes
Wes Turner
wes.turner at gmail.com
Mon Nov 3 23:00:12 EST 2014
> I'll do some more poking around, but think this is worth having!
Thank you so much!
On Mon, Nov 3, 2014 at 4:51 PM, Nicholas Bollweg <nick.bollweg at gmail.com>
wrote:
> Re: JSON-LD:
>
> One of the goals of JSON-LD is to be able to add context to a well-defined
> corpus of data without changing its native format: in this case, the
> developers mean what they mean, and its important that it be meant this
> way... so the challenge is how to extract a useful semantic model of the
> document out of the existing format.
>
> The simplest approach is to use the context of:
>
>> {
>> "@context": {
>> "@vocab": "http://ipython.org/nbformat/v4/"
>> }
>> }
>>
>
> This will map all object keys, and values which are told to be @ids, as
> URIs in that namespace: example
> <http://json-ld.org/playground/index.html#startTab=tab-expanded&json-ld=https%3A%2F%2Fgist.githubusercontent.com%2Fbollwyvl%2F09f0241b45fc3a98df8b%2Fraw%2F8e8928d9ae471f23af2eaba9365d0ac1d770a4f6%2Ftest4.base.vocab.ipynb&context=%7B%22%40context%22%3A%7B%22%40base%22%3A%22http%3A%2F%2Fipython.org%2Fnbformat%2Fv4%2F%22%2C%22%40vocab%22%3A%22http%3A%2F%2Fipython.org%2Fnbformat%2Fv4%2F%22%7D%7D>
> .
>
> (the playground doesn't have a way to do out-of-band expansion contexts,
> but it wouldn't have to be embedded like that)
>
>
> A more thorough-going approach might yield some more interesting things,
> but this is a good starting point, and just a few additions to the above
> context would be get really close to reflecting what is being said in a
> given notebook.
>
> IPython notebook nbformat v4 JSONSchema:
>> https://github.com/minrk/ipython/blob/nbformat4/IPython/nbformat/v4/nbformat.v4.schema.json
>>
>
> The schema is a great start: I haven't tried it, but there are some tools
> to automatically generate context from schema
> <https://www.npmjs.org/package/schema-jsonld-context>:
>
> some things that might be *interesting* to map to JSON-LD:
>
> - patternProperties in mimebundle.
> - In this case, though, it's referring to large, but agreed-upon
> enumeration of values (and not, like package.json's dependencies,
> an infinite number of package names).
> - with the above context, these would be lumped into the root of
> the namespace: http://ipython.org/nbformat/v4/text/html
> - this isn't so bad, probably
> - all the enums, i.e. cell_type: markdown
> - with the naive context above, it will map to a string
> - by setting the @type of cell_type to be @id in the context, markdown
> would expand to the URI
> http://ipython.org/nbformat/v4/markdown
> - this isn't so bad
> - Another option is to treat enums as more xml-like literals of
> a specific type, by setting @type of cell_type to be something like
> CellType
> - also not so bad
> - The advantage to having URIs vs. literals (they are both
> queryable) is that URIs can be the subject of something, and not just the
> object... not sure what we'd want to say in this case.
> - the wild west of metadata
> - JSON-LD can't tell the difference between cell metadata and
> notebook metadata
> - this is not so bad, as it is always "isolated" within the
> context of the <thing>s metadata, and wouldn't "pollute" the parent
> - with the naive context, everything will just fall into the
> main namespace.
> - this is bad. i don't see anything that can be done about it
> - all the lists
> - in JSON-LD, one can specify @container: @list
> - these are pretty bad in RDF, as it uses a bizarre lisp-like first
> and rest to represent them
> - nothing in the root that can map to an @id or @type
> - @id: not going there today
> - @type: nbformat is close, but there's nothing but duck typing to
> say, "I am a notebook"
> - loading these up into a graph would be *interesting*, as they
> would all just be blank nodes knocking around
>
> I'll do some more poking around, but think this is worth having!
>
> On Mon, Nov 3, 2014 at 4:21 AM, Wes Turner <wes.turner at gmail.com> wrote:
>
>> Tardy to the party!
>>
>> IPython notebook nbformat v4 JSONSchema:
>> https://github.com/minrk/ipython/blob/nbformat4/IPython/nbformat/v4/nbformat.v4.schema.json
>>
>>
>> Metadata Documentation:
>> https://github.com/minrk/ipython/blob/nbformat4/docs/source/notebook/nbformat.rst#metadata
>>
>>
>> JSON-LD JSONSchema:
>> https://github.com/json-ld/json-ld.org/blob/master/schemas/jsonld-schema.json
>>
>> On Mon, Nov 3, 2014 at 3:05 AM, Matthias Bussonnier <
>> bussonniermatthias at gmail.com> wrote:
>>
>>> Hi,
>>> Le 3 nov. 2014 à 09:32, Wes Turner <wes.turner at gmail.com> a écrit :
>>>
>>> Thanks!
>>>
>>> 1. Is there a link to the jsonschema? In the documentation?
>>>
>>>
>>> Please see the PR. https://github.com/ipython/ipython/pull/6045
>>>
>>> 2. How feasible would it be to write a JSON-LD context?
>>>
>>>
>>> This should have been discussed when we were in the refactoring of the
>>> notebook format,
>>> not once it's ready to merge. Metadata are free format you can add
>>> things like that if you like though.
>>>
>>> 3. Where/how can/could Dublin Core metadata be added? It would be great
>>> to be able to index these documents with a title and authors.
>>>
>>>
>>> Same as above.
>>>
>>> The two last issues of extra metadata have extensively discussed, the
>>> new format do not change
>>> the discussions/problems that have been made. You can also refer to
>>> theses.
>>>
>>> Cheers,
>>> --
>>> M
>>>
>>>
>>>
>>>
>>>
>>> https://en.wikipedia.org/wiki/Dublin_Core#DCMI_Metadata_Terms
>>>
>>> On Sun, Nov 2, 2014 at 5:23 PM, Fernando Perez <fperez.net at gmail.com>
>>> wrote:
>>>
>>>>
>>>> On Sat, Nov 1, 2014 at 11:01 AM, Brian Granger <ellisonbg at gmail.com>
>>>> wrote:
>>>>
>>>>> And thanks to the whole team (esp Min) for working on all of this!
>>>>>
>>>>
>>>> +lots. This has been a huge amount of slow, careful, not-very-sexy
>>>> work. I am really thankful for the patience the whole team has had working
>>>> on this, to help us set the notebook format as a solid foundation for
>>>> long-term sharing and archival of computational work.
>>>>
>>>> This kind of effort provides few immediate rewards, but can have very
>>>> significant long-term value. Thanks a lot for everyone who pitched in on
>>>> that PR...
>>>>
>>>> Cheers,
>>>>
>>>> f
>>>>
>>>>
>>>> --
>>>> Fernando Perez (@fperez_org; http://fperez.org)
>>>> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
>>>> fernando.perez-at-berkeley: contact me here for any direct mail
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Wes Turner
>>> https://westurner.github.io/
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>>
>> --
>> Wes Turner
>> https://westurner.github.io/
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
--
Wes Turner
https://westurner.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141103/b1f0dfd6/attachment.html>
More information about the IPython-dev
mailing list