[SciPy-User] Pylab - standard packages

[Fernando Perez]

On Fri, Sep 21, 2012 at 8:59 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:
> This conversation moves fast!  Replies inline below.

Busy day on the intertubes, indeed :)

> On 9/21/12 7:43 PM, Fernando Perez wrote:
>
>> And right now the web server is necessary but that's an implementation
>> detail, one could write something that speaks zmq directly to the
>> kernels, without any http traffic at all.  That would be trickier to
>> use remotely, but it would be a perfectly sensible local
>> implementation of an IDE client.
>
> Which also brings up that zmq is also a dependency of pylab which many
> distributions are not likely to have currently...

EPD and Anaconda both ship it, and given the brutal complexity of
Sage's deployment, even if you guys aren't shipping it right now, I
can't imagine it would be a particular burden to do so.  Pythonxy
doesn't have it, but they do carry a ton of other complex things and
all they need is a binary windows installer, which Min has done a
great job providing, so I don't see that as a big deal.  On Linux all
the major distros ship it.

> How stable is the current format?  For that matter, what *is* the
> current IPython "Computable Document Format" :).  Is it documented
> somewhere?  The best I could find was at
> http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html#the-notebook-format

Ah, the attentive reader will have noticed how I said an 'open'
format, I never said 'documented' :)  Indeed, we've been derelict on
that front, and have enjoyed the luxury of being the only
implementation, which lets us get away with it.  There is code and
tests, but we should document more explicitly the structure of the
format.

> It seems like there were discussions a little while ago on the IPython
> list about changing the format, or adding something to the format, or
> something.  I don't recall exactly what it was off the top of my head.
> Right now, it seems that the format is defined by the IPython
> implementation.  That's fine, but the legislation in the standard
> probably should have something a little more formalized.

Absolutely.

> The protocol is documented more clearly, I think.  But it also has been
> changing, like the top-level metadata field we added (thanks again!).
> There also have been questions about possibly having custom user message
> types.  I'd like to push that discussion further too.  So maybe it's
> time to introduce version numbers for the protocol, so we can nail down
> a specific something to talk about?

Yes, so far it has evolved based on our needs and the feedback of
people like you (well, mostly you :).  We should indeed formalize this
further so that these two ideas, format and protocol, are really
clearly laid out.

If we also specify cleanly what the ipython 'syntax' is in a single
place, that will round up all of the ways in which ipython has gone
'beyond python'.  Thomas has been doing great work on rationalizing
this on IPEP 2 (https://github.com/ipython/ipython/issues/2293) so I
think we'll get there eventually.

Because ultimately, that's all that 'makes ipython':

1. a "python VM on steroids" with additional syntax, magics, OS
access, etc.  A lot of this is used in tutorials, books and talks so
defining its boundaries more clearly is important.  And by now it's
pretty stable, the only change we've had in years is the %% for cell
magics and I don't foresee much need for new stuff.

2. a protocol to control this VM (and by controlling more than one of
them at a time, you get parallel stuff).

3. a format for storing the interactions with this VM as a file.

We need to formalize all of this more explicitly, so that IPython can
be *one* implementation, the reference one, but not necessarily the
only one.

In a way, it's almost like the process that python itself had, where
CPython went from the only implementation to being the reference one.
In a small way I think we can follow that model, adapted to the needs
of scientific work.

> +1 to having a reference implementation in the spec of some sort of
> computable document format.  And +1 to having something in the pylab
> that can serve as a backend for a frontend (for example, speaking the
> IPython protocol), along with a reference frontend.

> (codemirror apparently has vim keybindings, by the way:
> http://codemirror.net/demo/vim.html)

Yes, we've just never done the work of exposing UI for people to
access them and store their preferences, that's the problem.

> Hooray for getting grants!  I guess you're working on buying more votes :).

You may hear from us soon, believe me :)

Cheers,

f