[Numpy-discussion] Numpy integers to integer powers again again

[josef.pktd at gmail.com]

On Wed, Oct 26, 2016 at 3:57 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Wed, Oct 26, 2016 at 1:39 PM, <josef.pktd at gmail.com> wrote:
>
>>
>>
>> On Wed, Oct 26, 2016 at 3:23 PM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>>
>>>
>>> On Tue, Oct 25, 2016 at 10:14 AM, Stephan Hoyer <shoyer at gmail.com>
>>> wrote:
>>>
>>>> I am also concerned about adding more special cases for NumPy scalars
>>>> vs arrays. These cases are already confusing (e.g., making no distinction
>>>> between 0d arrays and scalars) and poorly documented.
>>>>
>>>> On Mon, Oct 24, 2016 at 4:30 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>>>
>>>>> On Mon, Oct 24, 2016 at 3:41 PM, Charles R Harris
>>>>> <charlesr.harris at gmail.com> wrote:
>>>>> > Hi All,
>>>>> >
>>>>> > I've been thinking about this some (a lot) more and have an alternate
>>>>> > proposal for the behavior of the `**` operator
>>>>> >
>>>>> > if both base and power are numpy/python scalar integers, convert to
>>>>> python
>>>>> > integers and call the `**` operator. That would solve both the
>>>>> precision and
>>>>> > compatibility problems and I think is the option of least surprise.
>>>>> For
>>>>> > those who need type preservation and modular arithmetic, the np.power
>>>>> > function remains, although the type conversions can be surpirising
>>>>> as it
>>>>> > seems that the base and power should  play different roles in
>>>>> determining
>>>>> > the type, at least to me.
>>>>> > Array, 0-d or not, are treated differently from scalars and integers
>>>>> raised
>>>>> > to negative integer powers always raise an error.
>>>>> >
>>>>> > I think this solves most problems and would not be difficult to
>>>>> implement.
>>>>> >
>>>>> > Thoughts?
>>>>>
>>>>> My main concern about this is that it adds more special cases to numpy
>>>>> scalars, and a new behavioral deviation between 0d arrays and scalars,
>>>>> when ideally we should be trying to reduce the
>>>>> duplication/discrepancies between these. It's also inconsistent with
>>>>> how other operations on integer scalars work, e.g. regular addition
>>>>> overflows rather than promoting to Python int:
>>>>>
>>>>> In [8]: np.int64(2 ** 63 - 1) + 1
>>>>> /home/njs/.user-python3.5-64bit/bin/ipython:1: RuntimeWarning:
>>>>> overflow encountered in long_scalars
>>>>>   #!/home/njs/.user-python3.5-64bit/bin/python3.5
>>>>> Out[8]: -9223372036854775808
>>>>>
>>>>> So I'm inclined to try and keep it simple, like in your previous
>>>>> proposal... theoretically of course it would be nice to have the
>>>>> perfect solution here, but at this point it feels like we might be
>>>>> overthinking this trying to get that last 1% of improvement. The thing
>>>>> where 2 ** -1 returns 0 is just broken and bites people so we should
>>>>> definitely fix it, but beyond that I'm not sure it really matters
>>>>> *that* much what we do, and "special cases aren't special enough to
>>>>> break the rules" and all that.
>>>>>
>>>>>
>>> What I have been concerned about are the follow combinations that
>>> currently return floats
>>>
>>> num: <type 'numpy.int8'>, exp: <type 'numpy.int8'>, res: <type
>>> 'numpy.float32'>
>>> num: <type 'numpy.int16'>, exp: <type 'numpy.int8'>, res: <type
>>> 'numpy.float32'>
>>> num: <type 'numpy.int16'>, exp: <type 'numpy.int16'>, res: <type
>>> 'numpy.float32'>
>>> num: <type 'numpy.int32'>, exp: <type 'numpy.int8'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int32'>, exp: <type 'numpy.int16'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int32'>, exp: <type 'numpy.int32'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int64'>, exp: <type 'numpy.int8'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int64'>, exp: <type 'numpy.int16'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int64'>, exp: <type 'numpy.int32'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int64'>, exp: <type 'numpy.int64'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.int64'>, exp: <type 'numpy.int64'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.uint64'>, exp: <type 'numpy.int8'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.uint64'>, exp: <type 'numpy.int16'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.uint64'>, exp: <type 'numpy.int32'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.uint64'>, exp: <type 'numpy.int64'>, res: <type
>>> 'numpy.float64'>
>>> num: <type 'numpy.uint64'>, exp: <type 'numpy.int64'>, res: <type
>>> 'numpy.float64'>
>>>
>>> The other combinations of signed and unsigned integers to signed powers
>>> currently raise ValueError due to the change to the power ufunc. The
>>> exceptions that aren't covered by uint64 + signed (which won't change) seem
>>> to occur when the exponent can be safely cast to the base type. I suspect
>>> that people have already come to depend on that, especially as python
>>> integers on 64 bit linux convert to int64. So in those cases we should
>>> perhaps raise a FutureWarning instead of an error.
>>>
>>
>>
>> >>> np.int64(2)**np.array(-1, np.int64)
>> 0.5
>> >>> np.__version__
>> '1.10.4'
>> >>> np.int64(2)**np.array([-1, 2], np.int64)
>> array([0, 4], dtype=int64)
>> >>> np.array(2, np.uint64)**np.array([-1, 2], np.int64)
>> array([0, 4], dtype=int64)
>> >>> np.array([2], np.uint64)**np.array([-1, 2], np.int64)
>> array([ 0.5,  4. ])
>> >>> np.array([2], np.uint64).squeeze()**np.array([-1, 2], np.int64)
>> array([0, 4], dtype=int64)
>>
>>
>> (IMO: If you have to break backwards compatibility, break forwards not
>> backwards.)
>>
>
> Current master is different. I'm not too worried in the array cases as the
> results for negative exponents were zero except then raising -1 to a power.
> Since that result is incorrect raising an error  falls on the fine line
> between bug fix and compatibility break. If the pre-releases cause too much
> trouble.
>


naive question: if cleaning up the inconsistencies already (kind of) breaks
backwards compatibility and didn't result in a big outcry, why can we not
go with a Future warning all the way to float. (i.e. use the power function
with specified dtype instead of ** if you insist on int return)

Josef


>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20161026/67c5c976/attachment.html>