[Numpy-discussion] How a transition to C++ could work

[Mark Wiebe]

On Sun, Feb 19, 2012 at 3:45 AM, David Cournapeau <cournape at gmail.com>wrote:

> On Sun, Feb 19, 2012 at 9:19 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> > On Sun, Feb 19, 2012 at 2:56 AM, David Cournapeau <cournape at gmail.com>
> > wrote:
> >>
> >> Hi Mark,
> >>
> >> thank you for joining this discussion.
> >>
> >> On Sun, Feb 19, 2012 at 7:18 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> >> > The suggestion of transitioning the NumPy core code from C to C++ has
> >> > sparked a vigorous debate, and I thought I'd start a new thread to
> give
> >> > my
> >> > perspective on some of the issues raised, and describe how such a
> >> > transition
> >> > could occur.
> >> >
> >> > First, I'd like to reiterate the gcc rationale for their choice to
> >> > switch:
> >> > http://gcc.gnu.org/wiki/gcc-in-cxx#Rationale
> >> >
> >> > In particular, these points deserve emphasis:
> >> >
> >> > The C subset of C++ is just as efficient as C.
> >> > C++ supports cleaner code in several significant cases.
> >> > C++ makes it easier to write cleaner interfaces by making it harder to
> >> > break
> >> > interface boundaries.
> >> > C++ never requires uglier code.
> >>
> >> I think those arguments will not be very useful: they are subjective,
> >> and unlikely to convince people who prefer C to C++.
> >
> >
> > They are arguments from a team which implement both a C and a C++
> compiler.
> > In the spectrum of possible authorities on the matter, they rate about as
> > high as I can imagine.
>
> There are quite a few arguments who are as authoritative and think
> those arguments are not very strong. They are as unlikely to change
> your mind as the gcc's arguments are unlikely to convince me I am
> afraid.


I imagine only points 2 and 3 are controversial for you, 1 and 4 are pretty
straightforward, yes? We could dig into the specifics of these points if
you'd like.


> >
> > This is a necessary part of providing a C API, and is included as a
> > requirement of doing that. All C++ libraries which expose a C API deal
> with
> > this.
>
> The only two given examples given so far for a C library around C++
> code (clang and zeromq) do not use exceptions. Can you provide an
> example of a C++ library that has a C API and does use exception ?
>

I couldn't find a nice example with a short search, unfortunately.


> If not, I would like to know the technical details if you don't mind
> expanding on them.
>

Sure. First, one would standardize on having all exceptions be derived from
std::exception. (std::bad_alloc, which we discussed before, and all other
standard exceptions, do). Then, each function exposed to the C API where
internally C++ exceptions are used would look roughly like:

int api_function(int param1, float param2, PyArrayObject *param3)
{
   try {
      ... implementation ..

      return 0;.
   } catch(std::bad_alloc&) {
      PyErr_NoMemory();
      return -1;
   } catch(numpy::convergence_error& e) {
      PyErr_SetString(NpyExc_ConvergenceError, e.what());
      return -1;
   } catch(std::exception& e) {
      PyErr_SetString(PyExc_RuntimeError, e.what());
      return -1;
   }
}


>
> >
> >>
> >> How can one make sure C++ extensions built
> >> by different compilers can work ?
> >
> >
> > This is no different from the situation in C. Already in C on Windows,
> one
> > can't build NumPy with a different version of Visual C++ than the one
> used
> > to build CPython.
>
> This is a different situation. On windows, the mismatch between VS is
> due to the way win32 has been used by python itself - it could
> actually be fixed eventually by python (there are efforts in that
> regard). It is not a language issue.
>

I've already tried fixing this and building NumPy with Visual C++ 2010, and
the memory allocation/deallocation issues were pretty easy to fix. The
problem was that NumPy C code takes a FILE* object from a file opened from
within Python code. The root of the issue is when CPython uses a different
C runtime library (MSVCR##.dll) than NumPy.


> Except for that case, numpy has a pretty good record of allowing
> people to mix and match compilers. Using mingw on windows and intel
> compilers on linux are the typical cases, but not the only ones.


In these cases the compiler is adopting the name-mangling ABI of the
compiler it's matching. On Windows, the intel compiler uses the Visual C++
ABI, and on Linux, it uses the gcc ABI. But, since the CPython API is a C
API, things would still work fine even if the name-mangling were different.


> >>
> >> I would
> >> expect you would like to use templates for iterators in numpy - you
> >> can you make them available to 3rd party extensions without requiring
> >> C++.
> >
> >
> > Yes, something like the nditer is a good example. From C, it would have
> to
> > retain an API in the current style, but C++ users could gain an
> > easier-to-use variant.
>
> Providing an "official" C++ library on top of the current C API would
> certainly be nice for people who prefer C++ to C. But this is quite
> different from using C++ at the core.
>

That's true.


> The current way iterators work would be very hard (if at all possible
> ?) to rewrite in idiomatic in C++ while keeping even API compatibility
> with the existing C one. For numpy 2.0, we can somehow relax on this.
> If it is not too time consuming, could you show a simplified example
> of how it would work to write the iterator in C++ while providing a C
> API in the spirit of what we have now ?


I think the C iterator API could stay pretty much the same as it is now.
Implementing it on top of a more flexible C++ iterator API should be no
problem. I'm thinking mostly of the nditer API, which does not use macros.
For the other iterators, like the neighborhood iterator, which depends on
macros for inlining functionality (correct me if I'm mischaracterizing it),
they would probably stay essentially the same. Equivalent C++ iterators
which provide a simpler interface for C++ programmers could be done beside
them, but because of the desire to push functionality into header files via
macros, that particular part of the implementation couldn't be shared.

-Mark


> David
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120219/f86f0a6f/attachment.html>