[Numpy-discussion] Re: sum and mean methods behaviour

[Peter Verveer]

Hi all,

I was thinking a bit more about the changes to reduce() that Todd proposed, 
and have some questions:

The problem that the output may not be able to hold the result of an operation 
is not unique to the reduce() method. For instance adding two arrays of type 
UInt can also give you the wrong answer: 

>>> array(255, UInt8) + array(255, UInt8)
254

So, if this is a general problem, why should only the reduce method be 
enhanced to avoid this? If you implement this, should this capability not be 
supported more broadly than only by reduce(), for instance by universal 
functons such as 'add'? Would it not be unexpected for users that only 
reduce() provides such added functionality?

However, as Paul Dubois pointed out earlier, the original design philosphy of 
Numeric/numarray was to let the user deal with such problems himself and keep 
the package small and fast. This seems actually a sound decision, so would it 
not be better to avoid complicating numarray with these type of changes and 
also leave reduce as it is?

Personally I don't have a need for the proposed changes to the reduce 
function. My original complaint that started the whole discussion was that 
the mean() and sum() array methods did not give the correct result in some 
cases. I still think they should return a correct double precision value, 
even if the universal functions may not. That could be achieved by a separate  
implementation that does not use the universal functions. I would be prepared 
to provide that implementation either to replace the mean and sum methods, or 
as a separate add-on.

Cheers, Peter

> 1. Add a type parameter to sum which defaults to widest type.
>
> 2. Add a type parameter to reductions (and fix output type handling).
> Default is same-type as it is now.  No major changes to C-code.
>
> 3. Add a WidestType(array) function:
>
>         Bool                        --> Bool
>         Int8,Int16,Int32,Int64      --> Int64
>         UInt8, UInt16,UInt32,UInt64 --> UInt64 (Int64 on win32)
>         Float32, Float64            --> Float64
>         Complex32, Complex64        --> Complex64
>
> The only thing this really leaves out, is a higher performance
> implementation of sum/mean which Peter referred to a few times. 
> Peter, if you want to write a specialized module, we'd be happy 
> to put it in the add-ons package.