[pypy-dev] [ANN] Python compilers workshop at SciPy this year

John Camara john.m.camara at gmail.com
Wed Mar 23 16:22:30 EDT 2016


Hi Fijal,

I agree that jffi would be both a large project and without someone leading
it, it would likely not get any where.  But I tend to disagree that it
would be a separate goal for the conference.  I realize the goal of the
summit is to talk about native-code compilation for Python and most would
argue that means executing C code, assembly, or at the very least executing
code at the speed of "C code".  But the reality now is,
numerical/scientific programming increasingly needs executing in a
clustered environment.  So I think we need to be careful to not only solve
yesterday's problems but make sure we are covering the current day and
future ones.

Today, big data and analytics, which is driving most numerical/scientific
programming, is becoming almost exclusively run in a clustered environment,
with the Apache Spark ecosystem as the de facto standard.  A few years
back, Python's ace up its sleeve for the scientific community was the
numpy/scipy ecosystem but we have recently lost that edge by falling behind
in clustered computing.  At this point in time our best move forward on the
numerical/scientific fronts is to become best buddies with the Spark
ecosystem and make sure we can bring bridge the numpy/scipy ecosystem to
it.  That is we merge the best of both worlds and suddenly Python becomes
to go to language again for numerical/scientific computing.  Of course we
still need to address what should have been yesterday's problem and deal
with the "native-code compilation" issues.

John

On Wed, Mar 23, 2016 at 2:47 PM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> Hi John
>
> I understand why you're bringing this up, but it's a huge project on
> it's own, worth at least a couple months worth of work. Without  a
> dedicated effort from someone I'm worried it would not go anywhere.
> It's kind of separated from the other goals of the summit
>
> On Wed, Mar 23, 2016 at 8:16 PM, John Camara <john.m.camara at gmail.com>
> wrote:
> > Hi Nathaniel,
> >
> > I would like to suggest one more topic for the workshop. I see a big need
> > for a library (jffi) similar to cffi but that provides a bridge to Java
> > instead of C code. The ability to seamlessly work with native Java
> data/code
> > would offer a huge improvement when python code needs to work with the
> > Spark/Hadoop ecosystem. The current mechanisms which involve serializing
> > data to/from Java can kill performance for some applications and can
> render
> > Python unsuitable for these cases.
> >
> > John
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > https://mail.python.org/mailman/listinfo/pypy-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20160323/5ca38ada/attachment.html>


More information about the pypy-dev mailing list