[Numpy-discussion] Silent Broadcasting considered harmful

[josef.pktd at gmail.com]

On Sun, Feb 8, 2015 at 5:17 PM, Stefan Reiterer <domors at gmx.net> wrote:
> Actually I use numpy for several years now, and I love it.
> The reason that I think silent broadcasting of sums is bad
> comes simply from the fact, that I had more trouble with it, than it helped
> me.
>
> I won't stop using numpy because of that, but I think this behavior may
> backfire,
> and thats the reason I started this discussion. Till now the only way out of
> the misery
> is to make proper unit tests, and to be careful as hell with dimensions and
> shape checks.

I fully agree with the last part. We need a lot more checks and be a
lot more careful in numpy than in matlab. But that's a fundamental
difference between the array versus matrix approach.

For me the main behavior I had to adjust to was loosing a dimension in
any reduce operation, mean, sum, ...

if x is 2d
x - x.mean(1)
we loose a dimension, and it doesn't broadcast in the right direction

x - x.mean(0)
perfect, no `repeat` needed, it just broadcasts the way we need.

Josef

>
> Providing optional warnings just would be an elegant way out of this.
>
> Cheers,
> Stefan
> Gesendet: Sonntag, 08. Februar 2015 um 22:56 Uhr
> Von: "Matthew Brett" <matthew.brett at gmail.com>
>
> An: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
> Betreff: Re: [Numpy-discussion] Silent Broadcasting considered harmful
> Hi,
>
> On Sun, Feb 8, 2015 at 1:39 PM, Simon Wood <sgwoodjr at gmail.com> wrote:
>>
>>
>> On Sun, Feb 8, 2015 at 4:24 PM, Stefan Reiterer <domors at gmx.net> wrote:
>>>
>>> I don't think this is a good comparison, especially since broadcasting is
>>> a feature not a necessity ...
>>> It's more like turning off/on driving assistance.
>>>
>>> And as already mentioned: other matrix languages also allow it, but they
>>> warn about it's usage.
>>> This has indeed it's merits.
>>> Gesendet: Sonntag, 08. Februar 2015 um 22:17 Uhr
>>> Von: "Charles R Harris" <charlesr.harris at gmail.com>
>>> An: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
>>> Betreff: Re: [Numpy-discussion] Silent Broadcasting considered harmful
>>>
>>>
>>> On Sun, Feb 8, 2015 at 2:14 PM, Stefan Reiterer <domors at gmx.net> wrote:
>>>>
>>>> Yeah I'm aware of that, that's the reason why I suggested a warning
>>>> level
>>>> as an alternative.
>>>> Setting no warnings as default would avoid breaking existing code.
>>>> Gesendet: Sonntag, 08. Februar 2015 um 22:08 Uhr
>>>> Von: "Eelco Hoogendoorn" <hoogendoorn.eelco at gmail.com>
>>>> An: "Discussion of Numerical Python" <numpy-discussion at scipy.org>
>>>> Betreff: Re: [Numpy-discussion] Silent Broadcasting considered harmful
>>>> > I personally use Octave and/or Numpy for several years now and never
>>>> > ever needed braodcasting.
>>>> But since it is still there there will be many users who need it, there
>>>> will be some use for it.
>>>>
>>>> Uhm, yeah, there is some use for it. Im all for explicit over implicit,
>>>> but personally current broadcasting rules have never bothered me,
>>>> certainly
>>>> not to the extent of justifying massive backwards compatibility
>>>> violations.
>>>> Take It from someone who relies on broadcasting for every other line of
>>>> code.
>>>>
>>>
>>>
>>> It's how numpy works. It would be like getting into your car and being
>>> warned that it has wheels.
>>>
>>> Chuck
>>> _______________________________________________ NumPy-Discussion mailing
>>> list NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>
>> I agree, I do not think this is a good comparison. All cars have wheels,
>> there are no surprises there. This is more like a car that decides to do
>> something completely different from everything that you learned about in
>> driving school.
>
>> I find the broadcasting aspect of Numpy a turn off. If I go to add a 1x3
>> vector to a 3x1 vector, I want the program to warn me or error out. I
>> don't
>> want it to do something under the covers that has no mathematical basis or
>> definition. Also, Octave may provide a warning, but Matlab errors
>> out..."Matrix dimensions must agree". Which they must, at least in my
>> world.
>
> In a previous life, many of us were very serious users of Matlab,
> myself included.
>
> Matlab / Octave have a model of the array as being a matrix, but numpy
> does not have this model. There is a Matrix class that implements
> this model, but usually experienced numpy users either never use this,
> or stop using it.
>
> I can only say - subjectively I know - that I did not personally
> suffer from this when I switched to numpy from Matlab, partly because
> I was fully aware that I was going to have to change the way I thought
> about arrays, for various reasons. After a short while getting used
> to it, broadcasting seemed like a huge win. I guess the fact that
> other languages have adopted it means that others have had the same
> experience.
>
> So, numpy is not a straight replacement of Matlab, in terms of design.
>
> To pursue the analogy, you have learned to drive an automatic car.
> Numpy is a stick-shift car. There are good reasons to prefer a
> stick-shift, but it does mean that someone trained on an automatic is
> bound to feel that a stick-shift is uncomfortable for a while.
>
> Best,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>