[IPython-dev] Extending the Notebook File Format

Carl Smith carl.input at gmail.com
Mon Nov 19 19:03:59 EST 2012

Firstly, I'd like to say sorry to everyone on this list. I've been
pretty vocal lately, mostly suggesting features that I'm only loosely
committed to working on, or criticising features other people worked
hard on. I really don't mean to be difficult. That said, I did want to
bring up one more issue :)

The current file format, .ipynb, works well for actual notebooks, but
doesn't include the ancillary files, so it's hard to share your work.
Obviously, users can put the notebook in the root directory of a
project tree, ensure everything they want to share is in there
somewhere, then zip it, but doing this by hand is not exactly
convenient, and doesn't deal with stuff like config files well.

I'd like to suggest creating a second file format, for packaged
notebooks, that is just the current file format, but with the file and
whatever else you'd like to include inside a zipped directory. IPython
should recognise these package files and handle them, unzipping them
behind the scenes.

A user could still create a regular notebook, and work loosely, so
they wouldn't need to worry about project directories. They could save
this as a regular notebook, just as they can now. Nothing should
change on that front, but, if the user was interested in sharing or
publishing their notebook along with their data, they would be
encouraged to create a notebook project. Only a project could be
reliably packaged.

It'd be nice to have a projectify tool that could take a notebook and
check any paths that it uses, any extensions, any customisations or
config files and raise warnings with dialogues if they're not where
they need to be, so you could usually turn a sketchy notebook into a
tidy project with a few clicks of the OK button.

The Notebook interface would include New File and New Project options,
so users will normally start with a regular file if they're just
hacking stuff, and would start with a project if they intend to share
it, but being able to turn a notebook into a project would be nice.
The same tool would be able to check a project before it was packaged
to ensure everything was still all present and correct.

When working on a project, hitting Save would just update the notebook
file, but hitting Package would run the validator and go through the
motions, before zipping the project, ready for sharing. This process
would copy any needed files from the user's current notebook profile
too, so all that stuff would also be preserved in the package in a way
that IPython knows how to handle.

Longer term, this format could be extended to include an optional
config file. If it was present, it'd be used for things like creating
packages with many notebooks, that could be read like a book, or could
include information on mounting big datasets or installing
dependencies and so on, which would be especially handy for consistent
environments like NotebookCloud or Wakari, where these types of things
can be done pretty reliably with a script.

Again, sorry to be so noisy just lately, and thanks for being so good
about it. I'm just sharing ideas in the hope that there's enough value
in some of them to justify the bandwidth I've consumed.

Cheers all


More information about the IPython-dev mailing list