[pypy-dev] Questions on the pypy+numpy project

[Romain Guillebert]

Hi everyone

I guess people want to know what is the current status of the ctypes
backend for Cython, you can read the last status update there :
http://mail.python.org/pipermail/pypy-dev/2011-September/008260.html

Of course I'm available for any kind of questions :)

Cheers
Romain

On Mon, Oct 17, 2011 at 11:14:08PM +0200, Stefan Behnel wrote:
> Alex Gaynor, 17.10.2011 18:14:
> >On Mon, Oct 17, 2011 at 12:01 PM, Stefan Behnel wrote:
> >>Maciej Fijalkowski, 17.10.2011 17:46:
> >>  - pypy's numpy *will* integrate in some sort of way with existing
> >>>C/fortran libraries, but this way *will* be different than current
> >>>CPython C API. It's really just too hard to get both.
> >>
> >>Why reinvent yet another wheel when you could make Cython a common language
> >>to write extensions and wrapper code for both? Even if that requires a few
> >>feature restrictions for Cython users or adaptations to their code to keep
> >>it portable, it's still better than forcing users into a complete vendor
> >>lock-in on both sides.
> >
> >There's no fundamental objection to Cython, but there are practical ones.
> 
> I'm very well aware of that. There are both technical and practical
> issues. I didn't hide the fact that the Python+ctypes backend for
> Cython is quite far from being ready for use, for example.
> 
> 
> >  a) Most of NumPy isn't Cython, so just having Cython gives us little.
> 
> There has been the move towards a smaller core for NumPy, and we
> perceive substantial interest, both inside and outside of the
> Scientific Python community, in writing new wrapper code in Cython
> and even in rewriting existing code in Cython to make it more
> maintainable. Even generated wrappers were and are being rewritten,
> e.g. to get rid of SWIG. Rewriting several hundred to thousand lines
> of C code in Cython can often be done within a few days, depending
> on test coverage and code complexity, and from what we hear, this is
> actually being done or at least seriously considered in several
> projects. It's helped by the fact that CPython users do not have to
> make the switch right away, but can often migrate or add a module at
> a time.
> 
> I agree that simply supporting Cython is not going to magically
> connect huge amounts of foreign code to PyPy. It just makes it a lot
> easier to get closer to that goal than by inventing yet another way
> of interfacing that is not supported by anything else.
> 
> Also note that there isn't just NumPy. A relatively large part of
> Sage is written in Cython, for example, especially those parts that
> glue the rest together, which consists of huge amounts of C, C++ and
> Fortran code. After all, Cython's predecessor Pyrex has been around
> for almost ten years now.
> 
> 
> > b)
> >Is the NumPy on Cython house in order?  AFAIK part of the MS project
> >involved rewriting parts of NumPy in Cython and modularising Cython for
> >targets besides CPython.  And that this was *not* merged.  For me to be
> >convinced Cython is a good target, I'd need belief that there's an interest
> >in it being a common platform, and when I see that there's work done, by
> >core developers, which sits unmerged (with no timeline) I can't have faith
> >in that.
> 
> I understand that objection. The Cython project is largely driven by
> the interest of core developers and users (now, how unexpected is
> that?), and none of the developers currently uses IronPython or
> PyPy. So, while we'd like to see Cython support other targets (and
> the core developers agree on that goal), there isn't really a strong
> incentive for ourselves to move it into that direction. It's a bit
> of a chicken and egg problem - why support other platforms that
> no-one uses it for, and who'd use it on a platform that's not as
> well supported as CPython?
> 
> I'd personally like to get the ctypes backend merged, but it's not
> exactly in a state that is ready-to-merge soonish. There's a branch,
> and Romain (our GSoC student for the project) is still working on
> it, but obviously with much less time for it, so I'm sure he could
> use another helping hand.
> 
> https://github.com/hardshooter/CythonCTypesBackend
> 
> The IronPython port is a different beast. It ran almost completely
> in cloak mode, outside of the scope of the core developers, and it's
> neither clear what the exact design goals were, nor what was
> eventually achieved or in what status the code branch currently is.
> The project itself died from sudden lack of interest on the side of
> the financial supporters (MS) at some point, and it appears that
> there is currently no-one who can easily take it over. Sad, but
> really nothing to blame the Cython developers for. I'd be happy to
> see it revived, if there is any interest.
> 
> https://bitbucket.org/cwitty/cython-for-ironpython/overview
> 
> Stefan
> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev