[Numpy-discussion] Moving forward with value based casting

[Sebastian Berg]

On Wed, 2019-06-05 at 17:14 -0700, Tyler Reddy wrote:
> A few thoughts:
> 
> - We're not trying to achieve systematic guards against integer
> overflow / wrapping in ufunc inner loops, right? The performance
> tradeoffs for a "result-based" casting / exception handling addition
> would presumably be controversial? I know there was some discussion
> about having an "overflow detection mode"  (toggle) of some sort that
> could be activated for ufunc loops, but don't think that gained much
> traction/ priority. I think for floats we have an awkward way to
> propagate something back to the user if there's an issue.

No, that is indeed a different issue. It would be nice to provide the
option of integer overflow warnings/errors, but it is different since
it should not affect the dtypes in use (i.e. we would never upcast to
avoid the error).

> - It sounds like the objective is instead primarily to achieve pure
> dtype-based promotion, which is then effectively just a casting
> table, which is what I think you mean by "cache?"

Yes, the cache was a bad word, I used it thinking of user types where a
large table would probably not be created on the fly.

> - Is it a safe assumption that for a cache (dtype-only casting
> table), the main tradeoff is that we'd likely tend towards
> conservative upcasting and using more memory in output types in many
> cases vs. NumPy at the moment? Stephan seems concerned about that,
> presumably because x + 1 suddenly changes output dtype in an
> overwhelming number of current code lines and future simple examples
> for end users.

Yes. That is at least what we currently have. For x + 1 there is a good
point with sudden memory blow up. Maybe an even nicer example is
`float32_arr + 1`, which would have to go to float64 if 1 is
interpreted as `int32(1)`.

> - If np.array + 1 absolutely has to stay the same output dtype moving
> forward, then "Keeping Value based casting only for python types" is
> the one that looks most promising to me initially, with a few further
> concerns:

Well, while it is annoying me. I think we should base that decision of
what we want the user API to be only. And because of that, it seems
like the most likely option.
At least my gut feeling is, if it is typed, we should honor the type
(also for scalars), but code like x + 1 suddenly blowing up memory is
not a good idea.
I just realized that one (anti?)-pattern that is common is the:

arr + 0.  # make sure its "inexact/float"

is exactly an example of where you do not want to upcast unnecessarily.


> 1) Would that give you enough refactoring "wiggle room" to achieve
> the simplifications you need? If value-based promotion still happens
> for a non-NumPy operand, can you abstract that logic cleanly from the
> "pure dtype cache / table" that is planned for NumPy operands?

It is tricky. There is always the slightly strange solution of making
dtypes such as uint7, which "fixes" the type hierarchy as a minimal
dtype for promotion purpose, but would never be exposed to users.
(You probably need more strange dtypes for float and int combinations.)

To give me some wiggle room, what I was now doing is to simply decide
on the correct dtype before lookup. I am pretty sure that works for
all, except possibly one ufunc within numpy. The reason that this works
is that almost all of our ufuncs are typed as "ii->i" (identical
types).
Maybe that is OK to start working, and the strange dtype hierarchy can
be thought of later.


> 2) Is the "out" argument to ufuncs a satisfactory alternative to the
> "power users" who want to "override" default output casting type? We
> suggest that they pre-allocate an output array of the desired type if
> they want to save memory and if they overflow or wrap integers that
> is their problem. Can we reasonably ask people who currently depend
> on the memory-conservation they might get from value-based behavior
> to adjust in this way?

The can also use `dtype=...` (or at least we can fix that part to be
reliable). Or they can cast type the input. Especially if we want to
use it only for python integers/floats, adding the `np.int8(3)` is not
much effort.

> 3) Presumably "out" does / will circumvent the "cache / dtype casting
> table?"

Well, out fixes one of the types, if we look at the general machinery,
it would be possible to have:

ff->d
df->d
dd->d

loops. So if such loops are defined we cannot quite circumvent the
whole lookup. If we know that all loops are of the `ff->f` all same
dtype kind (which is true for almost all functions inside numpy),
lookup could be simplified.
For those loops with all the same dtype, the issue is fairly straight
forward anyway, because I can just decide how to handle the scalar
before hand.

Best,

Sebastian


> 
> Tyler
> 
> On Wed, 5 Jun 2019 at 15:37, Sebastian Berg <
> sebastian at sipsolutions.net> wrote:
> > Hi all,
> > 
> > Maybe to clarify this at least a little, here are some examples for
> > what currently happen and what I could imagine we can go to (all in
> > terms of output dtype).
> > 
> > float32_arr = np.ones(10, dtype=np.float32)
> > int8_arr = np.ones(10, dtype=np.int8)
> > uint8_arr = np.ones(10, dtype=np.uint8)
> > 
> > 
> > Current behaviour:
> > ------------------
> > 
> > float32_arr + 12.  # float32
> > float32_arr + 2**200  # float64 (because np.float32(2**200) ==
> > np.inf)
> > 
> > int8_arr + 127     # int8
> > int8_arr + 128     # int16
> > int8_arr + 2**20   # int32
> > uint8_arr + -1     # uint16
> > 
> > # But only for arrays that are not 0d:
> > int8_arr + np.array(1, dtype=np.int32)  # int8
> > int8_arr + np.array([1], dtype=np.int32)  # int32
> > 
> > # When the actual typing is given, this does not change:
> > 
> > float32_arr + np.float64(12.)                  # float32
> > float32_arr + np.array(12., dtype=np.float64)  # float32
> > 
> > # Except for inexact types, or complex:
> > int8_arr + np.float16(3)  # float16  (same as array behaviour)
> > 
> > # The exact same happens with all ufuncs:
> > np.add(float32_arr, 1)                               # float32
> > np.add(float32_arr, np.array(12., dtype=np.float64)  # float32
> > 
> > 
> > Keeping Value based casting only for python types
> > -------------------------------------------------
> > 
> > In this case, most examples above stay unchanged, because they use
> > plain python integers or floats, such as 2, 127, 12., 3, ...
> > without
> > any type information attached, such as `np.float64(12.)`.
> > 
> > These change for example:
> > 
> > float32_arr + np.float64(12.)                        # float64
> > float32_arr + np.array(12., dtype=np.float64)        # float64
> > np.add(float32_arr, np.array(12., dtype=np.float64)  # float64
> > 
> > # so if you use `np.int32` it will be the same as np.uint64(10000)
> > 
> > int8_arr + np.int32(1)      # int32
> > int8_arr + np.int32(2**20)  # int32
> > 
> > 
> > Remove Value based casting completely
> > -------------------------------------
> > 
> > We could simply abolish it completely, a python `1` would always
> > behave
> > the same as `np.int_(1)`. The downside of this is that:
> > 
> > int8_arr + 1  # int64 (or int32)
> > 
> > uses much more memory suddenly. Or, we remove it from ufuncs, but
> > not
> > from operators:
> > 
> > int8_arr + 1  # int8 dtype
> > 
> > but:
> > 
> > np.add(int8_arr, 1)  # int64
> > # same as:
> > np.add(int8_arr, np.array(1))  # int16
> > 
> > The main reason why I was wondering about that is that for
> > operators
> > the logic seems fairly simple, but for general ufuncs it seems more
> > complex.
> > 
> > Best,
> > 
> > Sebastian
> > 
> > 
> > 
> > On Wed, 2019-06-05 at 15:41 -0500, Sebastian Berg wrote:
> > > Hi all,
> > > 
> > > TL;DR:
> > > 
> > > Value based promotion seems complex both for users and ufunc-
> > > dispatching/promotion logic. Is there any way we can move forward
> > > here,
> > > and if we do, could we just risk some possible (maybe not-
> > existing)
> > > corner cases to break early to get on the way?
> > > 
> > > -----------
> > > 
> > > Currently when you write code such as:
> > > 
> > > arr = np.array([1, 43, 23], dtype=np.uint16)
> > > res = arr + 1
> > > 
> > > Numpy uses fairly sophisticated logic to decide that `1` can be
> > > represented as a uint16, and thus for all unary functions (and
> > most
> > > others as well), the output will have a `res.dtype` of uint16.
> > > 
> > > Similar logic also exists for floating point types, where a lower
> > > precision floating point can be used:
> > > 
> > > arr = np.array([1, 43, 23], dtype=np.float32)
> > > (arr + np.float64(2.)).dtype  # will be float32
> > > 
> > > Currently, this value based logic is enforced by checking whether
> > the
> > > cast is possible: "4" can be cast to int8, uint8. So the first
> > call
> > > above will at some point check if "uint16 + uint16 -> uint16" is
> > a
> > > valid operation, find that it is, and thus stop searching. (There
> > is
> > > the additional logic, that when both/all operands are scalars, it
> > is
> > > not applied).
> > > 
> > > Note that while it is defined in terms of casting "1" to uint8
> > safely
> > > being possible even though 1 may be typed as int64. This logic
> > thus
> > > affects all promotion rules as well (i.e. what should the output
> > > dtype
> > > be).
> > > 
> > > 
> > > There 2 main discussion points/issues about it:
> > > 
> > > 1. Should value based casting/promotion logic exist at all?
> > > 
> > > Arguably an `np.int32(3)` has type information attached to it, so
> > why
> > > should we ignore it. It can also be tricky for users, because a
> > small
> > > change in values can change the result data type.
> > > Because 0-D arrays and scalars are too close inside numpy (you
> > will
> > > often not know which one you get). There is not much option but
> > to
> > > handle them identically. However, it seems pretty odd that:
> > >  * `np.array(3, dtype=np.int32)` + np.arange(10, dtype=int8)
> > >  * `np.array([3], dtype=np.int32)` + np.arange(10, dtype=int8)
> > > 
> > > give a different result.
> > > 
> > > This is a bit different for python scalars, which do not have a
> > type
> > > attached already.
> > > 
> > > 
> > > 2. Promotion and type resolution in Ufuncs:
> > > 
> > > What is currently bothering me is that the decision what the
> > output
> > > dtypes should be currently depends on the values in complicated
> > ways.
> > > It would be nice if we can decide which type signature to use
> > without
> > > actually looking at values (or at least only very early on).
> > > 
> > > One reason here is caching and simplicity. I would like to be
> > able to
> > > cache which loop should be used for what input. Having value
> > based
> > > casting in there bloats up the problem.
> > > Of course it currently works OK, but especially when user dtypes
> > come
> > > into play, caching would seem like a nice optimization option.
> > > 
> > > Because `uint8(127)` can also be a `int8`, but uint8(128) it is
> > not
> > > as
> > > simple as finding the "minimal" dtype once and working with
> > that." 
> > > Of course Eric and I discussed this a bit before, and you could
> > > create
> > > an internal "uint7" dtype which has the only purpose of flagging
> > that
> > > a
> > > cast to int8 is safe.
> > > 
> > > I suppose it is possible I am barking up the wrong tree here, and
> > > this
> > > caching/predictability is not vital (or can be solved with such
> > an
> > > internal dtype easily, although I am not sure it seems elegant).
> > > 
> > > 
> > > Possible options to move forward
> > > --------------------------------
> > > 
> > > I have to still see a bit how trick things are. But there are a
> > few
> > > possible options. I would like to move the scalar logic to the
> > > beginning of ufunc calls:
> > >   * The uint7 idea would be one solution
> > >   * Simply implement something that works for numpy and all
> > except
> > >     strange external ufuncs (I can only think of numba as a
> > plausible
> > >     candidate for creating such).
> > > 
> > > My current plan is to see where the second thing leaves me.
> > > 
> > > We also should see if we cannot move the whole thing forward, in
> > > which
> > > case the main decision would have to be forward to where. My
> > opinion
> > > is
> > > currently that when a type has a dtype associated with it
> > clearly, we
> > > should always use that dtype in the future. This mostly means
> > that
> > > numpy dtypes such as `np.int64` will always be treated like an
> > int64,
> > > and never like a `uint8` because they happen to be castable to
> > that.
> > > 
> > > For values without a dtype attached (read python integers,
> > floats), I
> > > see three options, from more complex to simpler:
> > > 
> > > 1. Keep the current logic in place as much as possible
> > > 2. Only support value based promotion for operators, e.g.:
> > >    `arr + scalar` may do it, but `np.add(arr, scalar)` will not.
> > >    The upside is that it limits the complexity to a much simpler
> > >    problem, the downside is that the ufunc call and operator
> > match
> > >    less clearly.
> > > 3. Just associate python float with float64 and python integers
> > with
> > >    long/int64 and force users to always type them explicitly if
> > they
> > >    need to.
> > > 
> > > The downside of 1. is that it doesn't help with simplifying the
> > > current
> > > situation all that much, because we still have the special
> > casting
> > > around...
> > > 
> > > 
> > > I have realized that this got much too long, so I hope it makes
> > > sense.
> > > I will continue to dabble along on these things a bit, so if
> > nothing
> > > else maybe writing it helps me to get a bit clearer on things...
> > > 
> > > Best,
> > > 
> > > Sebastian
> > > 
> > > 
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at python.org
> > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190606/d7d955df/attachment-0001.sig>