[Numpy-discussion] How a transition to C++ could work

[Matthew Brett]

Hi,

On Sun, Feb 19, 2012 at 12:49 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> On Sun, Feb 19, 2012 at 2:32 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> Thanks for this - it's very helpful.
>>
>> On Sat, Feb 18, 2012 at 11:18 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>> > The suggestion of transitioning the NumPy core code from C to C++ has
>> > sparked a vigorous debate, and I thought I'd start a new thread to give
>> > my
>> > perspective on some of the issues raised, and describe how such a
>> > transition
>> > could occur.
>> >
>> > First, I'd like to reiterate the gcc rationale for their choice to
>> > switch:
>> > http://gcc.gnu.org/wiki/gcc-in-cxx#Rationale
>> >
>> > In particular, these points deserve emphasis:
>> >
>> > The C subset of C++ is just as efficient as C.
>> > C++ supports cleaner code in several significant cases.
>> > C++ makes it easier to write cleaner interfaces by making it harder to
>> > break
>> > interface boundaries.
>> > C++ never requires uglier code.
>> >
>> > Some people have pointed out that the Python templating preprocessor
>> > used in
>> > NumPy is suggestive of C++ templates. A nice advantage of using C++
>> > templates instead of this preprocessor is that third party tools to
>> > improve
>> > software quality, like static analysis tools, will be able to run
>> > directly
>> > on the NumPy source code. Additionally, IDEs like XCode and Visual C++
>> > will
>> > be able to provide the full suite of tab-completion/intellisense
>> > features
>> > that programmers working in those environments are accustomed to.
>> >
>> > There are concerns about ABI/API interoperability and interactions with
>> > C++
>> > exceptions. I've dealt with these types of issues on enough platforms to
>> > know that while they're important, they're a lot easier to handle than
>> > the
>> > issues with Fortran, BLAS, and LAPACK in SciPy. My experience has been
>> > that
>> > providing a C API from a C++ library is no harder than providing a C API
>> > from a C library.
>> >
>> > It's worth comparing the possibility of C++ versus the possibility of
>> > other
>> > languages, and the ones that have been suggested for consideration are
>> > D,
>> > Cython, Rust, Fortran 2003, Go, RPython, C# and Java. The target
>> > language
>> > has to interact naturally with the CPython API. It needs to provide
>> > direct
>> > access to all the various sizes of signed int, unsigned int, and float.
>> > It
>> > needs to have mature compiler support wherever we want to deploy NumPy.
>> > Taken together, these requirements eliminate a majority of these
>> > possibilities. From these criteria, the only languages which seem to
>> > have a
>> > clear possibility for the implementation of Numpy are C, C++, and D.
>>
>> On which criteria did you eliminate Cython?
>
>
> The "mature compiler support" one.

I took you to mean that the code would compile on any platform.  As
Cython compiles to C, I think Cython passes, if that is what you
meant.  Maybe you meant you thought that Cython was not mature in some
sense, but if so, I'm not sure which sense you mean.

> As glue between C/C++ and Python, it
> looks great, but Dag's evaluation of Cython's maturity for implementing the
> style of functionality in NumPy seems pretty authoritative. So people don't
> have to dig through the giant email thread, here's the specific message
> content from Dag, and it's context:
>
> On 02/18/2012 12:35 PM, Charles R Harris wrote:
>>
>> No one in their right mind would build a large performance library using
>> Cython, it just isn't the right tool. For what it was designed for -
>> wrapping existing c code or writing small and simple things close to
>> Python - it does very well, but it was never designed for making core
>> C/C++ libraries and in that role it just gets in the way.
>
> +1. Even I who have contributed to Cython realize this; last autumn I
> implemented a library by writing it in C and wrapping it in Cython.

As you probably saw, I think the proposal was indeed to use Cython to
provide the higher-level parts of the core, while refactoring the rest
of the C code underneath it.  Obviously one could also refactor the C
into C++, so the proposal to use Cython for some of the core is to
some extent orthogonal to the choice of C / C++.    I don't know the
core, perhaps there isn't much of it that would benefit from being in
Cython, I'd be interested to know your views.  But, superficially, it
seems like an attractive solution to making (some of) the core easier
to maintain.

Best,

Matthew