[Numpy-discussion] Ransom Proposals

[Tim Hochberg]

Christopher Barker wrote:

> I want to start by saying a am +1 on everything Tim is advocating, 
> just so you all know it's not just him. I'm not writing any code, 
> however.

Thanks Chris.

>
> A few comments:
>
> Travis Oliphant wrote:
>
>> It would seem that most of Tim's complaints are directly against 
>> polymorphism.
>
>
> I don't think so. I think Tim (and I) just want it to be really clear 
> what the polymorphic functions are and what aren't. Also, for 
> polymorphism to work, there always has to be some definition of what 
> "similar" objects are: in static languages, this means they are 
> derived from the same class. In duck-typed languages, it means they 
> share the same behavior, so all we're doing here is deciding exactly 
> how much behavior they need to share.
>
> >  But, then Python has
>
>> this same problem, because
>>
>> l += a
>>
>> doesn't do 'in-place' for the list, but does do inplace if 'l' were 
>> an array.
>
>
> Yes, it does do it for the list, but differently. However, maybe a 
> tuple is a better example:
>
> >>> a = (1,2,3)
> >>> id(a)
> 331328
> >>> a += (3,4,5)
> >>> a
> (1, 2, 3, 3, 4, 5)
> >>> id(a)
> 236712
>
> Whereas with numpy:
>
> >>> import numpy as N
> >>> a = N.array((1,2,3))
> >>> id(a)
> 6436864
> >>> a += (3,4,5)
> >>> a
> array([4, 6, 8])
> >>> id(a)
> 6436864
>
> So there are two differences: addition is defined differently, and one 
> is in-place, and one isn't. Personally, I've always thought that the 
> "in place" operators should never work with immutable types: in=place 
> should mean in-place,, not "in-place if possible, but not if not 
> possible. and yes, this is completely analogous to the issue at hand.

The problem with this is that then inplace add doesn't work with 
numbers; a huge use case that can't be neglected.  Really though, 
in-place addition for immutable types works fine unless one is overusing 
'is'. Issues can come up with  mixing mutable and imutatble types 
though, or two mutable types if the left handed one doesn't define iadd.

> And then we have this (already pointed out in this thread)
>
> >>> l = [1,2,3]
> >>> l
> [1, 2, 3]
> >>> a
> array([4, 6, 8])
> >>> l += a
> >>> l
> array([ 5,  8, 11])

It is a bug. Travis brought this up on PyDev and they agreed that there 
is a problem with PyNumber_InplaceAdd. Armin's looking at this now in 
the hope of getting a fix in for 2.5. The result should be a list:  
[1,2,3,5,8,11].

> >>> b
> [4, 6, 8]
> >>> l = [1,2,3]
> >>> l += b
> >>> l
> [1, 2, 3, 4, 6, 8]
>
> >>> l = [1,2,3]
> >>> t = (3,4,5)
> >>> l+=t
> >>> l
> [1, 2, 3, 3, 4, 5]
>
> How this is anything but a bug is beyond me! Lists do define the 
> in-place operator, and really do it in-place. That being the case, I 
> would certainly never expect += to change the type of a list! It 
> doesn't with a tuple. Is this a numpy issue or a Python one?

A Python one. However, I'll be writing more on this in a moment; the 
behaviour that causes the issue with list is slated to be fixed, but I 
think that there's some other cases to consider.

>
>> I don't view it as a problem of function behavior as much as problem 
>> with documentation and "mismatch between what a particular user 
>> expects and what is actually done."
>
>
> Quite true. However, what I advocate is that to keep that mismatch as 
> rare as possible, the ONLY functions that should have the "sometimes a 
> view and sometimes a copy" behavior should be functions that 
> explicitly exist to provide that behavior, such as asarray(). All it 
> does is save typing and increase the potential for confusion and 
> errors to built it in to functions that have other uses.
>
> Travis Oliphant wrote:
>
>> on making methods that return views raise an error when impossible 
>> and not changing and/or deprecating functions (i.e. the functions are 
>> simple wrappers around the methods).
>
>
>  +1
>
>> on making methods that return views raise an error when impossible 
>> and changing the function to make a copy
>
>
>  +1 : Only if the functions are moved to a separate "backwards 
> compatible" name space.
>
FWIW, I'm fine if the functions always copy and methods always returns 
views, since this gets rid of the view/copy dilema. I believe that Sasha 
previously raised objections to this on backwards compatibility grounds. 
I believe Sasha preferred leaving the functions alone and just not 
recomending their use. I was fine with that too as long as the functions 
got segregated. Travis didn't like that, which loops us back around to 
the current proposal.

Regards,

-tim