[IPython-dev] IPython3/Jupyter Magics for Python-based kernels

Doug Blank doug.blank at gmail.com
Sat Aug 23 13:26:01 EDT 2014

On Sat, Aug 23, 2014 at 1:12 AM, MinRK <benjaminrk at gmail.com> wrote:

> I think if you want to add something like magics for Python-wrapped
> kernels based on your magickernel.py code, that would be interesting.
> Standardizing magics across kernel languages is something that is
> attractive on its face, but we deemed outside the scope of the message /
> kernel specification, because it is probably not feasible to do in a
> language agnostic way (For example, IPython magics can contain Python
> string-formatting code with Python expressions that are evaluated in the
> interactive namespace). Clarification of discussions like this are part of
> the reasoning behind the Jupyter/IPython split:
>    - Jupyter is language-agnostic computing tools, such as the message
>    protocol, notebook document format, and notebook UI.
>    - IPython is a better interactive Python environment, with some
>    extensions to the Python language for interactive use.
>    - IPython provides a Jupyter Kernel.
>    - Magics are 100% an IPython feature, and not in any way related to
>    Jupyter.
>    - Other kernels are welcome to implement extensions to their language
>    (including Python!) as they see fit,
>    as similar or different to IPython’s magics system as makes sense.
> Implementing magics for basic Python-wrapped kernels (i.e. not R, Julia,
> Ruby, Haskell, etc.) would be simple, and potentially useful, depending on
> how many of these kernels are developed. I do not expect most first-class
> kernels to go this route, in which case they would not share any magic
> code. Further, I don’t think the magic code developed for this use case
> would even be shared by IPython.
> But it would still be an interesting exercise, and might encourage other
> kernels to take the multi-interpreter approach if it is successful.
I see your point, but from a general Jupyter user's perspective, IF a
kernel supports a magic-like metacommand, THEN it would be nice to be
uniform in syntax, and behavior.

Imagine that 5 different kernels implement a magic like "%%time" with 5
different syntaxes, and 5 slightly different results.

I am not advocating that the IPython team need to do anything. I am just
suggesting that where possible, it would be great from the user's
perspective if there was some standardization across these metacommands.

I am finding that there are many "magics" (I know that I'm defining that
word differently than IPython uses it) that can be implemented in a manner
that doesn't affect (nor interact with) the underlying language. In
MagicKernel, these commands act as pre and post processors, but are not
"in" the language as they are in IPython's Python.

Sounds like a fun experiment!
Well, it is very useful to have the set that are independent of the
language, and we are implementing them. If they are useful for other
Python-based kernels, then great! All of the following kernels in pip now
have these standard magics: eval_kernel (simple Python evaluator),
echo_kernel (from docs), chatbot_kernel, and scheme_kernel.


> -MinRK
> On Wed, Aug 20, 2014 at 4:22 AM, Doug Blank <doug.blank at gmail.com> wrote:
>> IPython developers,
>> One of the nice things that the IPython team did for the next version was
>> to create a base class for creating new kernels for code written in Python,
>> called wrapper kernels [1].
>> Unfortunately, such kernels don't have ! shell commands, nor magics. This
>> is understandable, as those are tightly integrated with IPython's Python.
>> However, one could get some of the basic functionality of these features on
>> top of the wrapper kernel, in a generic way. For example, you could run
>> shell commands, or have some magics that operate independent of the kernel.
>> We have created a prototype of magics for wrapper kernels [2]. The code
>> is in magickernel.py, and the magics are in the folder magics. The current
>> prototype works like this:
>> ```
>> In [1]: %magics
>> Line magics:
>>      %cd PATH - change current directory of session
>>      %connect_info - show connection information
>>      %download URL [FILENAME] - download file from URL
>>      %html CODE - display code as HTML
>>      %install_magic URL - download and install magic from URL
>>      %javascript CODE - send code as JavaScript
>>      %latex TEXT - display text as LaTex
>>      %magics - show installed magics
>>      %reload_magics - reload the magics from the installed files
>>      %shell COMMAND - run the line as a shell command
>>      %time COMMAND - show time to run line
>> Cell magics:
>>     %%file FILENAME - write contents of cell to file
>>     %%html - display contents of cell as HTML
>>     %%javascript - send contents of cell as JavaScript
>>     %%latex - display contents of cell as LaTeX
>>     %%shell - run the contents of the cell as shell commands
>>     %%time - show time to run cell
>> Shell shortcut:
>>     ! COMMAND ... - execute command in shell
>> Any cell magic can be made persistent for rest of session by using %%%
>> prefix.
>> ```
>> The idea is to be able to easily add some standard-like magics to
>> 3rd-party kernels by using MagicKernel. See the sample wrapperkernel
>> EchoKernel with magics [3]. Also, the idea here is not to replicate too
>> much what comes with IPython's Python, but to provide a small, simple core
>> set of magics that can't be easily done in the kernel language. You can
>> install MagicKernel from source, or using:
>> $ pip install calico --upgrade
>> This is a prototype (with some bugs), and we very much hope for others to
>> provide feedback or code, and more work needs to be done. What magics would
>> you need in your 3rd-party kernel, that can be accomplished this way? Will
>> this design be sufficient?
>> We hope that this is useful!
>> -Doug
>> [1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html
>> [2] - https://bitbucket.org/ipre/calico/src/master/src/calico/
>> [3] -
>> https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140823/03ce2834/attachment.html>

More information about the IPython-dev mailing list