[IPython-dev] custom web notebook cells

[Brian Granger]

Nicholas,

Awesome, I can tell the notebook has you thinking about lots of cool
things.  I love that.  I will write more soon, but here is the short
answer:

* Once we merge the Javascript plugins branch, we will be able to do
all of the types of "widgets" you mention (SVG, tangle, spreadsheet,
controls, etc) with our current code cells.  In fact, we even have a
huge grant from the Sloan foundation to work on this *exactly* for the
next two years:

http://ipython.org/sloan-grant.html

* I am almost positive that we will grow some sort of node dependency
over time - probably not for running the notebook, but most likely for
dev, testing etc.

* For testing, our best option so far is phantom.js and casper.js.

Cheers,

Brian

On Fri, Dec 14, 2012 at 9:05 PM, Nicholas Bollweg
<nick.bollweg at gmail.com> wrote:
> sorry, my initial setup of digest makes it hard to get at the thread
> structure directly, so hopefully this gets in the right place.
>
> the feedback has been very good, impressive community (and product, of
> course)
> you have built here!
>
>     > * We need to maintain the ability to install ipython without
> setuptool.
>
> whoa, very "there is no spoon." just went back and read some tickets that
> mention setuptools/distutils. will respect the community's desires, here!
>
> so i guess, if primarily-javascript packages *were* to be installed via
> python,
> the IPython Way could be to have some %load_ext cells at the
> beginning of the notebook that would somehow change the static
> lookup path at runtime. i'll revisit my investigation of the extensions!
> once
> there is one example like this, it should be easy to repeat the pattern.
>
>     > Before we start actually coding on the require.js stuff or
>     > reorganizing that stuff, we need to figure out what we want to do
>     > exactly.  Here is an IPEP for us to work all of that out:
>
> thanks for the pointer to the IPEP5, very good scope! while i am learning to
> love node, i might avoid bringing in the node dependency until
> someone really has that itch to scratch... i could imagine the 0mq layer
> being
> perfect for interacting between python and node... under the correct
> conditions.
>
> ghost.py can handle the testing requirements, though it will crank up the
> travis test runs by a pretty serious amount, due to the installation of qt
> and
> friends... ah! but The Other Notebook already uses qt, so maybe this isn't
> as
> bad of a hit. here's the ghost.py .travis.yml that is the recipe i have used
> for getting it running:
>
>     https://github.com/jeanphix/Ghost.py/blob/master/.travis.yml
>
> having node around to do npm might not actually meet the needs of
> the notebook plugin builder (see below)... webassets looks good, but doesn't
> handle the dependency tree. tough choices ahead.
>
>     > * We have to serve these files on the web.  I don't want to have our
>     >   web server walking all over a users system to pull files as that is
> a
>     >   huge security risk.  We need to isolate the location of these files.
>
> yes, definitely a concern... need to wrap my head around this... what makes
> the
> most sense? options:
>
> - copy stuff into the ipython location
>   - seems bad
> - have multiple static_paths
>   - my current implementation, relies on runtime setuptools
> - collect all static deps into a tmpdir which only contains the files needed
>   - has potential...
>
>     > * Some plugins don't have any python code so I am not sure it makes
>     >   sense to bundle things into a python package.
>
> indeed, i have fought with this demon a few times, most recently with apache
> allura's homerolled asset management system. putting my sysadmin hat on,
> i'd have the fewest dependency management systems possible... apt-get/yum +
> pip.
> it's an immediate turnoff if i have to do much more than that. packaging as
> python does have the whole pypi advantage, though.
>
> of course, what could end up happening is that the plugins would have
> versions
> as well, and it could be that two different .ipynb's you wanted to look at
> would use different versions. this suggests, perhaps, that the plugins
> should
> be installed into the notebook itself, which also chills me to the bone.
>
> i guess we need to understand how a user would want to use these...
> definitely
> have to do more thinking.
>
>     > But, I would like to learn more about your usage cases.  There might
>     > be other ways of supporting what you want to do:
>     > 1) Use an existing cell type with metadata.
>     > 2) Provide a custom UI for an existing cell type but don't change the
>     >    notebook format.
>     > 3) Actually add a new cell type to the notebook format officially if
> it
>     >    is something that a majority of our users would use often.
>
> my long-term vision is to build on what mathematica/jmp/tableau can do:
> support
> freeform exploration by a developer/researcher/student of a data-driven
> problem, which gradually solidifies into a dashboard (for lack of a better
> term) that allows an end user to play in a constrained data space, but
> access
> advanced features.
>
> while getting there, i want to better be able to document what i am thinking
> about it one, context-switching-free space that versions/merges well.
>
> i never thought of doing anything that would do (3): i am indeed
> sensitive to the long-term viability of data formats, and want to do
> whatever i
> can to avoid contributing to "data death"... all the stuff i want to do is
> very
> web-focused (for both UI and output), and is therefore inherently
> text-based...
> it might just be Really Boring text if you are at the command line (hi,
> SVG!).
> i was thinking about it primarily from a UI point of view, the current
> implementation of which centers around the IPython.Cell subclasses... so
> basically what i want is what you describe in (2).
>
> my use cases for custom cells are sort of summed up (mostly as
> implementation ideas) in this ticket on my "write some d3 visualizations
> with
> blockly" project:
>
>     https://github.com/bollwyvl/blockd3/issues/10
>
> but here's some riffing on what i've learned in the last 24 hours :)
>
> use cases:
>
> - VisualCodeCell http://code.google.com/p/blockly/
>     - motivation
>         - i've helped students reach the the "a ha" moment with
>           blockly's spiritual ancestor, scratch, but feel like the problems
>           that python is able to tackle are much greater. i see the ipython
>           notebook as a great home for the blockly python functionality, as
> it
>           can be learned by the absolute novice... when i was a python
> novice,
>           i think i had the "a ha" moment about five years ago when i
> learned
>           about tab completion and ? and ?? in the ipython console!
>
>           i have been experimenting with blockly (which can generate python)
> a
>           good deal, but exclusively for javascript. presently blockly is
> kind
>           of lame for a python workflow.
>     - implementation
>         - CodeCell. blockly xml would fit nicely inside cell metadata, which
> i have yet to explore, but need to!
>     - challenges
>         - the downside here is that any editing of the python outside of the
>           web environment (or if you didn't have the ipython-blockly plugin
> i
>           envision), the blockly environment can't reverse-engineer its xml
>           from python. however, since blockly is more aimed at the novice,
> this
>           might not cause that much of an issue.
> - DiagramCell http://code.google.com/p/svg-edit/
>     - motivation
>         - i love me some svg... it is deeply ingrained in my day-to-day
> design
>           and communication workflow with inkscape, and has become
> increasingly
>           important from my web application development tasks with the
> meteoric
>           rise of d3. svgedit is Almost As Good as inkscape, and for a lot
> of
>           things i want to do, it would be sufficient for a lot of quick
>           vector-based documentation things that i'd want to attach.
>     - implementation
>         - MarkdownCell
>     - challenges
>         - none, really
> - SpreadsheetCell https://github.com/bollwyvl/blockd3/issues/SlickGrid
>     - motivation
>         - some data really is tabular. i would love to be able to whip up a
>           quick table (not using markdown!) and then pop that into a numpy
>           calculation chain.
>
>           also, being able to quickly walk around higher-order numpy arrays
>           would be great, suggesting an excel-like sheet navigation for a
> given
>           array.
>     - implementation
>         - CodeCell. perhaps some metadata for formatting
>     - challenges
>         - not trying to implement Resolver One here, but a little bit of
>           functions would be cool... i like some of the things that the
> google
>           spreadsheet has done to blow out a calculation into a bunch of
> cell
>           from input into one cell
> - TangleCell http://worrydream.com/Tangle/
>     - motivation
>         - the tangle reactive document paradigm is awesome. i have done some
> work with integrating this with markdown, but it never felt right,
>           and took a lot of `exec` to work properly, which always gave me
> the
>           willies. writing constraints directly on the backend and then just
>           tying them to frontend in markdown seems like the high road.
>     - implementation
>         - MarkdownCell
> - ControlCell
>     - motivation
>         - sometimes, i just want a button or a slider for changing a value.
>     - implementation
>         - CodeCell. metadata.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com