[IPython-dev] custom web notebook cells
Jason Grout
jason-sage at creativetrax.com
Sat Dec 15 16:20:34 EST 2012
On 12/15/12 2:00 PM, Brian Granger wrote:
> I think there is a big design pattern that is emerging in the
> notebook. Currently CodeCells have an input area and output area.
> Obviously, these were originally created for code and the output of
> the code. But, with the Javascript plugins, the output area has
> really become more than merely output. In fact here is how I view it
> now...
>
> * The input area is used for python code that initializes a "widget".
> That Python code is used to create the widget, configure it, provide
> input to it, etc.
> * The output area is used for the widget UI. Widgets can be arbitrary
> JS/HTML/CSS. Widgets can make calls back to the kernel and receive
> further JSON data back. Widgets can even be nested.
That's a nice way to think about it. Can we extend it one level
further? One thing I've long dreamed for Sage is to have javascript
widgets in the *input* area as well, and a way to flip back and forth
between a text representation and other representations for the widgets.
So, for example, you could have:
a=<<numpy array>>
and the <<numpy array>> would be a small javascript spreadsheet control.
Hit a button or key, and the spreadsheet control converts to an
equivalent text representation (which is what is actually sent over the
wire to the kernel). Or I'd like to have:
n=<<slider>>
where <<slider>> is a jquery slider, letting me pick the value for n.
Or have:
f=<<math expression>>
where <<math expression>> is a mathjax-enabled input widget that lets me
type in normal mathematics. Hit a button and it gets converted to the
text equation representation.
Or:
Process(<<image>>)
and <<image>> is an actual picture (which can be flipped to a text
representation, which might be ascii encoded, or might be a statement
downloading an image, or something else).
>
> Here is an example that I have started to work on. The new Web Audio
> API allows Javascript to access the raw audio data from a microphone.
> Using this, it is possible to create a widget that records audio and
> processes and visualizes the audio in JS+Python+matplotlib+d3+etc.
Wouldn't it be cool to have, in the *input*:
audio=<<widget>>
p=Process(audio)
p.save('newaudio.wav')
where widget has a record, playback, etc. buttons. Hit the record
button, and it records a data file and ships it to the server and sets
the result to a. Hit another button, and you could actually see a text
representation of what `audio` is going to be set to, or even better, a
waveform representing what `audio` will be set to.
These ideas also encapsulate a lot of the other cell types mentioned.
Why have different types of cells, when you can just have different
widgets inside the same cell? What if I want to have a spreadsheet
widget representing a numpy array and an audio widget representing an
audio file in the same input cell?
Thanks,
Jason
More information about the IPython-dev
mailing list