[IPython-dev] Notebook format "incompatible" changes

[Matthias Bussonnier]

Hi Carlos, 

TL:DR; at the end. 

With the notebook format v4, the way notebooks are written on disk will change.
After now some time with v3, we know that a few things could have been done better
or in a more general way, thus we improved the notebook format. 

Once 3.0 and 2.4 are released this shouldn't change much for the end user.
I'll go over some of the changes in the notebook format and their reasons, but you 
should read the IPep for all details. 


Get rid of python specific names. 
	pyin, pyout, pyerr are renamed input input, output, and error (roughly, read IPep for exact names). 

Uniformise jpg/png/text keys in output to be a mimetype. 
	You can now store application/x-pdf or doc/microsoft-word in a notebook. 
	If the frontend now how to read it, it will do something with it i suppose.
	It remove a lot of special casing. 

Get rid of worksheets, almost nobody use it, and there are other smarter way to 
store them if we wanted to.

Text cells had their text under the 'text' key, and code cell under 'source'. 
Which is a bit silly, as it forces you to do some special casing where not necessary.

Heading cell are now gone (from the file format), Technology evolve fast, and we can 
now detect the #*n in markdown cell and convert properly to LaTeX, and add anchor 
in nbviewer.

More importantly, we know have a jsonschema that describe the format. 
So we can validate the notebook, and know that for example, a prompt number
is either null, or a number. v3 was implicitly allowing '*' (star), which would happened 
if you were saving while cell are running, which took by surprise both us and external
library where the conversion of some notebook made crashes. This include also security risk.
Which I won't develop, cause I won't develop, but those who know know I love javascipt
injection, and you can do nasty things on known websites. By insuring the type of each field
of the notebook you lower the attack area, and protect FooBar corp user from attacks
even if FooBar corp does not really respond to you when you do responsible disclosure and 
finally give up. 

I won't describe "All the things", but you see the big picture. 
It's better, faster, stronger.

TL;DR:

Wat ? You don't know ?! I've heard that IPython notebook format v3 might have been responsible
for the death of a huge number of kittens due to developers banging their head on their desk. 
It is also probable that the need of extra computing power to run more test because of it's complexity
is in part responsible for global warming. Also the new format prevent the use of %pylab that kills 
the endangered species of newbies. The naming convention was also really strange for php  
coders that don't understand the py prefixes.

As we like kittens, newbies, developers, php coders (but not php itself [1]), and dislike
global warming we decided to fix that.

It also comes as ipynb v4+  that have a 5.5 inches screen, and we plan to jump from v8 to v10 
directly. We also removed touch-id from metadata so that police cannot force youth unlock your
ipynb.

Hope that shine a light on a few of the reason and what will change. 
Tell us if you have any more questions. 


Cheers, 
-- 
M

[1]: But MinRK Love Javascript. 
 




Le 2 nov. 2014 à 18:49, Carlos Córdoba <ccordoba12 at gmail.com> a écrit :

> Hi,
> 
> Could you explain a bit more what this change is about?
> 
> Cheers,
> Carlos
> 
> El 01/11/14 a las 13:01, Brian Granger escribió:
>> Matthias,
>> 
>> Thanks for posting here about these changes!
>> 
>> And thanks to the whole team (esp Min) for working on all of this!
>> 
>> Cheers,
>> 
>> Brian
>> 
>> On Sat, Nov 1, 2014 at 3:12 AM, Matthias Bussonnier
>> <bussonniermatthias at gmail.com> wrote:
>>> Hello Jovyans and other beings from Jupyter galilean moons system,
>>> 
>>> 
>>> I'm writing to you to warn that on Monday (probably morning) a relatively
>>> important change
>>> in IPython will land on master. In particular it does change the notebook
>>> structure quite a bit.
>>> 
>>> Cf https://github.com/ipython/ipython/pull/6045 for more information.
>>> 
>>> As nothing is ever perfect, please do back-up your notebooks before
>>> upgrading. With this new update,
>>> saved notebook of this new of IPython version won't be compatible with
>>> older IPython anymore, nor will nbviewer be able to render them.
>>> 
>>> The new notebook format will be back-ported on older (2.x) version of
>>> IPython and the new notebook format
>>> will be supported on nbviewer, we will just need a few days to port all the
>>> changes.
>>> 
>>> There is of course be a way to manually downgrade the notebook from v4 to v3
>>> using nbconvert.
>>> 
>>> Eventually the more adventurous can test the branch this week-end.
>>> 
>>> As usual, be prepared for data loss and bugs, so update your git branches,
>>> refresh submodules, clear your browser caches, roll up your sleeves, and
>>> send bug reports.
>>> 
>>> Cheers,
>>> --
>>> Matthias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141102/7d54e2a9/attachment.html>