[IPython-dev] IScala: a Scala-language backend for IPython

[Mateusz Paprocki]

Hi,

On 10 September 2013 21:59, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi Mateusz,
>
>
> On Sun, Sep 8, 2013 at 12:54 PM, Mateusz Paprocki <mattpap at gmail.com> wrote:
>> Hi,
>>
>> I would like to announce IScala: a Scala-language backend for IPython,
>> see [1]. Scala is general purpose, object-functional, statically typed
>> programming language for JVM. IScala is an early work in progress, but
>> it's already fairly usable. All three types of frontends are
>> supported. Required dependencies are IPython 1.0+ and Java Runtime
>> Environment 1.6+. See README.md for installation and usage details.
>
> This is great! Just last week I was playing a bit with Scala, this
> will make my life much easier :)

Thanks. Let me know if you need any advice.

>> Motivation for this project came when I saw IJulia announcement. The
>> hard part - actually "interpreting" Scala - was done in Scala REPL by
>> Scala developers. Scala is a compiled language, so interpreting it
>> means that source code is compiled (with some fancy wrappers),
>> resulting class files loaded into JVM and code run via Java
>> reflection. Interpreted code behaves exactly the same as compiled code
>> and it runs at full speed. There are a few issues with this, however.
>> Scala is a pure object-oriented language, so at top level you can put
>> only packages, traits, classes and singleton objects. No expressions
>> and other types of statements are allowed. Interpreter removes this
>> restriction, by wrapping source code in singleton objects, so that you
>> can simply write e.g. `val x = 1` to define a value, which is
>> otherwise illegal.
>
> Very interesting: I've been thinking precisely about this problem in
> the context of creating an IPython kernel for Go, and wondering what
> it would take.  Go doesn't have the hardcore OO stance of the JVM, but
> some similar issues regarding persistence of state and compilation
> arise. The lessons you've learned here would likely be very useful.

I don't have any experience with Go, so unfortunately can't help here.

>> During development of IScala I also created a set of notebooks with
>> Scala code, see [2]. Those are based on a subset of Twitter's Scala
>> lessons [3]. I didn't encode notebooks directly in the JSON-based
>> format, but I invented an adhoc format based on Markdown with support
>> for cells. There is a simple converter that allows to generate
>> IPython-compatible notebooks.
>
> I would recommend using the 'real' IPython notebook format as your
> default, so that you can benefit automatically from the entire rest of
> our machinery (nbconvert, nbviewer, etc).

I actually benefit from those tools, because I use nbcovert to
generate html from generated ipynb files and ipynb files are stored
along with html on gh-pages branch, so you can use nbviewer on them.

> Our long term view is for the format to be completely language
> agnostic, so we're willing to make any changes and fixes to the
> underlying format to accomodate Scala cleanly.  We're already looking
> at a bunch of such fixes for Julia, and having another language at
> hand will help us correctly identify the right abstraction points and
> how to leave generality in the base format while allowing each
> language to represent its own information with all the detail and
> precision desired.

I think the IPython's notebook format is already quite language
independent. It's the interpretation of the format that has to be
improved (e.g. adding syntax highlight for other languages). I would
invent my adhoc format (xipynb) even if I was creating notebooks with
Python code, because editing JSON manually is not an option for me
(for quite obvious reasons). Neither is using Notebook app for major
editorial work (although, for running code is OK), at least until
vim-like key bindings are supported on cell and intra-cell level (the
other is equally important, because writing code/text vim style and
switching to emacs style for cell manipulation would be very
inefficient) (vim is too deeply wired in my mind). However, I may not
like JSON, but it's good you chose an encoding of the format that has
a (stable) specification and allows for easy automatic processing. I
think that there should be a format that can be easily processed,
converted etc. and understood by Notebook app, but which doesn't have
to be easily editable without specialized tools, and then, there could
be formats that aren't that easy to process, but allow for easy
editing (etc.). Then, scripts for converting to/from those formats
could be plugins to nbconvert. I don't expect Notebook app or html
generator to understand those extra formats directly.

Speaking about JSON, if you have a moment, you may want to look into
[1]. I implemented message specification there in an object-oriented
and type safe way (there is room for improvements). Originally I just
copied text from [2] and filled in "blanks", so the structure is
similar and comments are in place. It may be advantageous to abstract
the specification over JSON+Python's types. So, if there's a `list`,
then write what kind of list it should be, e.g. `list<int>` (here
Java/C++ style), the same for dictionaries/maps and tuples. If there
is "MIME type" involved, then define it, because I can assume it's a
string, but I could imagine it being a 2-tuple. Also use enumerations
instead of strings. In my opinion this will make the specification
much more formal and understandable and allow for removal of
substantial number of comments, which are otherwise unavoidable.

But those little issues when looking at big picture. In general my
experience have been pleasant and finally I have an environment for
interactive computing with Scala.

[1] https://github.com/mattpap/IScala/blob/master/src/main/scala/Msg.scala
[2] http://ipython.org/ipython-doc/dev/development/messaging.html

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Mateusz