On 2009-07-01, Charles R Harris wrote:

On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau wrote:

...

I would like to add an explicit configuration test to check that our complex type is compatible with the C99 complex type (when available). Is this ok?

Seems OK to me.

...

As currently defined, complex c types (npy_cfloat, etc...) are not defined in a way such as they are binary compatible with the C99 complex type. Strictly speaking, packing the complex type is an ABI break, but we already make the assumption in ufunc, so we would have wrong results/crashes currently if it were not packed, so I believe the check

Is there a reason not to pack our complex number struct? I think if we bump the ABI version, changing this should be OK.

...

is harmless by itself. The rationale is that I would like to add support for complex math in npy_math (cabs, cexp, etc...). As I would like to use the C99 complex type (when available), this requires that numpy complex type is ABI compatible with C99 type.

Useful goal: this would allow using npy_math functions as a compatibility fallback when the C99 ones are not available.

What do you mean by "packing" ?

C99 defines a complex type such that the real and imag parts are packed as in an array: double complex t; real = ((double*)&t)[0]; imag = ((double*)&t)[1]; Numpy defines this type as a struct, which may imply additional padding. -- Pauli Virtanen

On 2009-07-01, Charles R Harris wrote: [clip]

OK. I don't see a problem then. As David says the ufuncs already work that way. <checks c99> Hmm... do we want to implement creal{l,,f} and cimag{l,,f} also?

They might come useful.

They are built in functions in gcc, can we detect that?

Don't know, probably we can?

Anyway, those functions would make rewriting the current complex functions pretty easy. Or do we want to rewrite the current functions to be ABI compatible with c99?

I think that for scipy.special this would be useful. Of course, the operator semantics for complex numbers can't be used there, but still sticking closer to C99 could be useful. A separate question is whether we want to implement Numpy's complex ufuncs using the C99 ones when they are available. I had a branch for this: http://github.com/pv/numpy-work/tree/c99-complex-umath This has some risks: the system-provided complex-valued functions may be broken in different ways, or suffer from some subtle flaws. This is likely more common than having broken real-valued functions that have been around quite a while longer. (To give an example: casinh(1e-20) == 0 with GNU Libc 2.9.) -- Pauli Virtanen

On Thu, Jul 2, 2009 at 9:02 AM, David Cournapeau wrote:

True, but we can deal  with this once we have tests: we can force to use our own, fixed implementations on broken platforms. The glibc complex functions are indeed not great, I have noticed quite a few problems for special value handling (e.g. cabs(inf + I * NAN) is nan, but should be inf, etc....).

Ok, here we are. There are two branches. - the first one, complex_umath_tests, is a branch with thorough covering of special values. I use the C99 standard for reference: http://github.com/cournape/numpy/tree/complex_umath_tests The main file is http://github.com/cournape/numpy/blob/c29e793c220f48e317adafb8c765acd4db13cb... Around 10 tests fail on Linux ATM. - the second branch is the interesting one: http://github.com/cournape/numpy/tree/C99_complex_math It implements clog, cexp, creal, cimag, carg, cabs, ccos, csin and csqrt. The list is dictated by my needs for windows 64, but the list can grow arbitrarily, of course. Most the implementations are taken from msun (the math library of FreeBSD). Unfortunately, they don't implement that many functions compared to the glibc. If I merge the two branches and use the npymath complex functions in umath, there is no unit test failures anymore :) I think I will merge the complex_umath_tests branch soon (platform-specific failures on build bot will be interesting), unless someone sees a problem with it. cheers, David

On Fri, Jul 3, 2009 at 4:44 AM, Pauli Virtanen wrote:

I think we tried this already (my c99-umath-funcs branch had TestC99 special case tests that were in Numpy trunk for a while).

The outcome was that the current implementations of the complex functions don't have essentially any special-case behavior that's consistent across different platforms. And our buildbot coverage is not large enough, so users will report failing tests even if buildbots are OK... After repeating this cycle a couple of times, IIRC only some special cases of log survived :)

I think the situation is better now, because we have the infrastructure to deal with this: we have portable nan, inf, negative zero and corresponding macros (I have just added copysign recently to the trunk, which was the last missing for every case AFAIK). Preliminary tests on both windows (with MS compilers) and linux indicate that many special cases are already dealt with correctly. I don't have tests for branch cuts yet, though, only signed 0, nan and inf handling.

Of course, if you meant to merge the tests first to the new implementations and that to trunk, this sounds better.

The idea was to merge all the tests, but set them as knownfailure, and enable them everytime I merge a new function into the trunk. This way, one can easily check the differences between successive revisions ? But that's not a crucial point. I can keep everything in one branch if you think it is better. David

On Thu, Jul 2, 2009 at 5:25 AM, Charles R Harris wrote:

On Wed, Jul 1, 2009 at 1:57 PM, Pauli Virtanen wrote:

...

On 2009-07-01, Charles R Harris wrote:

...

On Wed, Jul 1, 2009 at 1:59 AM, David Cournapeau wrote:

...

   I would like to add an explicit configuration test to check that our complex type is compatible with the C99 complex type (when available). Is this ok?

Seems OK to me.

...

...

   As currently defined, complex c types (npy_cfloat, etc...) are not defined in a way such as they are binary compatible with the C99 complex type. Strictly speaking, packing the complex type is an ABI break, but we already make the assumption in ufunc, so we would have wrong results/crashes currently if it were not packed, so I believe the check

Is there a reason not to pack our complex number struct? I think if we bump the ABI version, changing this should be OK.

This bothers me a bit. Since the two forms should normally be compatible maybe we can use a union or some other way to preserve the current ABI. It would be nice to have a deprecation warning too, but I can't figure out how to deprecate a struct declaration. A compiler warning perhaps?

I think it is a non issue, because we already rely on the assumption. I was just thinking about checking that sizeof(complex) == sizeof(double) * 2 at configuration check as a safeguard (and deal with it on platforms where it fails, using compiler-dependent pragma). Currently, if it is not packed, the complex ufunc are horribly wrong :) I don't see how using a union would help preserving anything: the only way to force packing would be to define complex as double[2] (as fftw does for fftw_complex), but this is not practical because you can't return an array in C. cheers, David