The meaning of threshold in numpy.set_printoptions()?
Hi, The help says. threshold : int, optional Total number of array elements which trigger summarization rather than full repr (default 1000). But the following code shows that 7 triggers summarization instead of 5. Does anybody know how to interpret the meaning of `threshold`? Thanks. In [1]: import numpy as np ...: np.set_printoptions(threshold=5) ...: print(np.arange(6)) ...: print(np.arange(7)) ...: [0 1 2 3 4 5] [0 1 2 ..., 4 5 6]  Regards, Peng
Whether summarization happens is also determined by edgeitems (defaults to 3), which is the number of items to print on each edge when summarizing. It wouldn’t make sense for summarization to cause the output to be longer than if no summarization occured! Feel free to submit an improvement to the documentation there on github. Eric On Mon, 11 Dec 2017 at 19:40 Peng Yu <pengyu.ut@gmail.com> wrote:
Hi,
The help says.
threshold : int, optional Total number of array elements which trigger summarization rather than full repr (default 1000).
But the following code shows that 7 triggers summarization instead of 5. Does anybody know how to interpret the meaning of `threshold`? Thanks.
In [1]: import numpy as np ...: np.set_printoptions(threshold=5) ...: print(np.arange(6)) ...: print(np.arange(7)) ...: [0 1 2 3 4 5] [0 1 2 ..., 4 5 6]
 Regards, Peng _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
Hi, question says it all. I looked through the basic and advanced indexing, but I could not find the rule that is applied to make x[np.newaxis,:] and x[np.newaxis] the same. Kind regards, Joe
On Tue, Dec 12, 2017 at 12:02 AM, Joe <solarjoe@posteo.org> wrote:
Hi,
question says it all. I looked through the basic and advanced indexing, but I could not find the rule that is applied to make x[np.newaxis,:] and x[np.newaxis] the same.
I think it's the general rule that all indexing expressions have an invisible "..." on the right edge. For example, x[i][j][k] is an inefficient and IMO somewhat confusing way to write x[i, j, k], because x[i][j][k] is interpreted as: > x[i, ...][j, ...][k, ...] > x[i, :, :][j, :][k] That this also applies to newaxis is a little surprising, but I guess consistent. n  Nathaniel J. Smith  https://vorpus.org
Ah, ok, now that I knew what to look for I guess I found it: "If the number of objects in the selection tuple is less than N , then : is assumed for any subsequent dimensions." https://docs.scipy.org/doc/numpy1.13.0/reference/arrays.indexing.html This is the one, right? Am 12.12.2017 09:09 schrieb Nathaniel Smith:
On Tue, Dec 12, 2017 at 12:02 AM, Joe <solarjoe@posteo.org> wrote:
Hi,
question says it all. I looked through the basic and advanced indexing, but I could not find the rule that is applied to make x[np.newaxis,:] and x[np.newaxis] the same.
I think it's the general rule that all indexing expressions have an invisible "..." on the right edge. For example, x[i][j][k] is an inefficient and IMO somewhat confusing way to write x[i, j, k], because x[i][j][k] is interpreted as:
> x[i, ...][j, ...][k, ...] > x[i, :, :][j, :][k]
That this also applies to newaxis is a little surprising, but I guess consistent.
n
On Tue, 20171212 at 14:19 +0100, Joe wrote:
Ah, ok, now that I knew what to look for I guess I found it:
"If the number of objects in the selection tuple is less than N , then : is assumed for any subsequent dimensions."
https://docs.scipy.org/doc/numpy1.13.0/reference/arrays.indexing.htm l
This is the one, right?
Yeah, plus if it is not a tuple, it actually behaves the same as a tuple, e.g. `arr[obj]` is identical to `arr[obj,]` (or `arr[(obj,)]` which is the same). There are some weird exception when obj is a list a sequence but not an array. Note also that while everything has an implicit `, ...` at the end of indexing, if you have exactly as many integers to index as dimensions you get a scalar, if you would add the Ellipsis you would get an array back. Anyway, too many weird details for day to day stuff :). And all of that should be covered in the docs?  Sebastian
Am 12.12.2017 09:09 schrieb Nathaniel Smith:
On Tue, Dec 12, 2017 at 12:02 AM, Joe <solarjoe@posteo.org> wrote:
Hi,
question says it all. I looked through the basic and advanced indexing, but I could not find the rule that is applied to make x[np.newaxis,:] and x[np.newaxis] the same.
I think it's the general rule that all indexing expressions have an invisible "..." on the right edge. For example, x[i][j][k] is an inefficient and IMO somewhat confusing way to write x[i, j, k], because x[i][j][k] is interpreted as:
> x[i, ...][j, ...][k, ...] > x[i, :, :][j, :][k]
That this also applies to newaxis is a little surprising, but I guess consistent.
n
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
Hi, the best example I found was this one: https://stackoverflow.com/a/29319864/7919597 def func_for_scalars_or_vectors(x): x = np.asarray(x) scalar_input = False if x.ndim == 0: x = x[None] # Makes x 1D scalar_input = True # The magic happens here if scalar_input: return np.squeeze(ret) return ret Is this as good as it gets or do you have other suggestions? Cheers, Joe
On Wed, Dec 13, 2017 at 4:50 AM, Joe <solarjoe@posteo.org> wrote:
Hi,
the best example I found was this one:
https://stackoverflow.com/a/29319864/7919597
def func_for_scalars_or_vectors(x): x = np.asarray(x) scalar_input = False if x.ndim == 0: x = x[None] # Makes x 1D scalar_input = True
# The magic happens here
if scalar_input: return np.squeeze(ret) return ret
Is this as good as it gets or do you have other suggestions?
In general structure, yeah. I'd probably use `np.isscalar(x)` for the test and `x = np.atleast_1d(x)` for the coercion for readability, but otherwise, that's it.  Robert Kern
Using np.iscalar is a bad idea, as it fails for 0d arrays. x.ndim is the better option there. I’d maybe suggest not specialcasing 0d arrays though, and using: def func_for_scalars_or_vectors(x): x = np.asanyarray(x) # convert scalars to 0d arrays # The magic happens here return ret[()] # convert 0d arrays to scalars Eric On Tue, 12 Dec 2017 at 11:58 Robert Kern <robert.kern@gmail.com> wrote: On Wed, Dec 13, 2017 at 4:50 AM, Joe <solarjoe@posteo.org> wrote:
Hi,
the best example I found was this one:
https://stackoverflow.com/a/29319864/7919597
def func_for_scalars_or_vectors(x): x = np.asarray(x) scalar_input = False if x.ndim == 0: x = x[None] # Makes x 1D scalar_input = True
# The magic happens here
if scalar_input: return np.squeeze(ret) return ret
Is this as good as it gets or do you have other suggestions?
In general structure, yeah. I'd probably use `np.isscalar(x)` for the test and `x = np.atleast_1d(x)` for the coercion for readability, but otherwise, that's it.
 Robert Kern _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Wed, Dec 13, 2017 at 5:52 AM, Eric Wieser <wieser.eric+numpy@gmail.com> wrote:
Using np.iscalar is a bad idea, as it fails for 0d arrays. x.ndim is the better option there.
I’d maybe suggest not specialcasing 0d arrays though, and using:
def func_for_scalars_or_vectors(x): x = np.asanyarray(x) # convert scalars to 0d arrays
# The magic happens here
return ret[()] # convert 0d arrays to scalars
Good call. I didn't think that the empty tuple was valid for anything but indexing into 0d arrays, but of course, following the rules of indexing, it works as required for other arrays too.
 Robert Kern
The different approaches and corresponding "code bloat" here is one of the most annoying things I have found about using numpy+scipy. Furthermore, the flip side to the handling of the inputs, including both type and shape, is getting the output to match the input, including both type and shape. Getting the output correct is exacerbated by different intermediate functions not following any standards whatsoever around returning an output that "matches" the input. This makes me long for strong typing in Python and even tempts me to port my production code to something "safe" like Rust (including proper generic types and concurrencysafety guaranteed by the compiler). I think I saw some other discussion recently about numpy joining forces with Python 3's gradual type system. Is there any draft formal proposal for this yet? If numpy+scipy wants to scale to "bigger" projects, I think it behooves the community to get rid of this messiness. On Tue, Dec 12, 2017 at 2:03 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Wed, Dec 13, 2017 at 5:52 AM, Eric Wieser <wieser.eric+numpy@gmail.com> wrote:
Using np.iscalar is a bad idea, as it fails for 0d arrays. x.ndim is the better option there.
I’d maybe suggest not specialcasing 0d arrays though, and using:
def func_for_scalars_or_vectors(x): x = np.asanyarray(x) # convert scalars to 0d arrays
# The magic happens here
return ret[()] # convert 0d arrays to scalars
Good call. I didn't think that the empty tuple was valid for anything but indexing into 0d arrays, but of course, following the rules of indexing, it works as required for other arrays too.
 Robert Kern
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Tue, Dec 12, 2017 at 5:07 PM Mark Campanelli <mark.campanelli@gmail.com> wrote:
I think I saw some other discussion recently about numpy joining forces with Python 3's gradual type system. Is there any draft formal proposal for this yet? If numpy+scipy wants to scale to "bigger" projects, I think it behooves the community to get rid of this messiness.
We're still figuring this out, but if you're interested please follow along (and pitch in!): https://github.com/numpy/numpy_stubs https://github.com/numpy/numpy/issues/7370 https://github.com/python/typing/issues/516 Cheers, Stephan
The real magic happens when you ducktype, and ensure your function works both for arrays and scalars on its own. This is more often possible than you might think! If you really need, e.g., the shape, you can do `getattr(input, 'shape', ())` and things may well work for scalars, and also for objects that behave somewhat like arrays ("duck arrays"). If you do go for coercion, I agree with Eric' comment about `np.asanyarray`  it may now just work for `MaskedArray` (but not for, e.g., SparseArray; that needs real ducktyping). All the best, Marten
On Tue, Dec 12, 2017 at 6:20 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
The real magic happens when you ducktype, and ensure your function works both for arrays and scalars on its own. This is more often possible than you might think!
Sadly, this still doesn't work in a typestable way. NumPy's ufuncs convert 0dimensional arrays into scalars. The typing rules for functions like np.sin() look like:  scalar or 0d array > scalar  1d or higher dimensional array > array I'm not entirely sure, but I suspect this was a practical rather than principled choice. NumPy scalars are "duck arrays" of sorts (e.g., with shape and dtype attributes) which helps to some extent, but the bugs that slip through are even harder to understand. This wart reminds me of how mixed basic/advanced indexing reorders sliced dimensions to make the result more "intuitive", which only works in some cases. I usually favor coercing all arguments to my functions to numpy arrays with np.asarray(), but to guarantee the return type you would also need to coerce it with np.asarray(), too.
Hi, yet another question. I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False] that might e.g result from import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False] x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False] I understood that they are equivalent to x[(False,)] I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :) In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right? Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int? Cheers, Joe
Increasingly, NumPy does not considers booleans to be integer types, and indexing is one of these cases. So no, it will not be treated as a tuple of integers, but as a 0d mask Eric On Wed, 13 Dec 2017 at 12:44 Joe <solarjoe@posteo.org> wrote:
Hi,
yet another question.
I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False]
that might e.g result from
import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False]
x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False]
I understood that they are equivalent to
x[(False,)]
I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :)
In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right?
Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int?
Cheers, Joe _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
Hello, thanks for you feedback. Sorry, if thie question is stupid and the case below does not make sense. I am just trying to understand the logic. For x = np.random.rand(2,3) x[True] x[(True,)] or x[False] x[(False,)] where True and False are not arrays, it will pick the first or second row. Is this basic indexing then with one the rules  obj is an integer  obj is a tuple of slice objects and integers. ? Am 13.12.2017 21:49 schrieb Eric Wieser:
Increasingly, NumPy does not considers booleans to be integer types, and indexing is one of these cases.
So no, it will not be treated as a tuple of integers, but as a 0d mask
Eric
On Wed, 13 Dec 2017 at 12:44 Joe <solarjoe@posteo.org> wrote:
Hi,
yet another question.
I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False]
that might e.g result from
import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False]
x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False]
I understood that they are equivalent to
x[(False,)]
I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :)
In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right?
Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int?
Cheers, Joe _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion [1]
Links:  [1] https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
It sounds like you're using an old version of numpy, where boolean scalars were interpreted as integers. What version are you using? Eric On Thu, Dec 14, 2017, 04:27 Joe <solarjoe@posteo.org> wrote:
Hello, thanks for you feedback.
Sorry, if thie question is stupid and the case below does not make sense. I am just trying to understand the logic. For
x = np.random.rand(2,3)
x[True] x[(True,)]
or
x[False] x[(False,)]
where True and False are not arrays, it will pick the first or second row.
Is this basic indexing then with one the rules  obj is an integer  obj is a tuple of slice objects and integers. ?
Am 13.12.2017 21:49 schrieb Eric Wieser:
Increasingly, NumPy does not considers booleans to be integer types, and indexing is one of these cases.
So no, it will not be treated as a tuple of integers, but as a 0d mask
Eric
On Wed, 13 Dec 2017 at 12:44 Joe <solarjoe@posteo.org> wrote:
Hi,
yet another question.
I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False]
that might e.g result from
import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False]
x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False]
I understood that they are equivalent to
x[(False,)]
I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :)
In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right?
Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int?
Cheers, Joe _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion [1]
Links:  [1] https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Thu, 20171214 at 16:24 +0000, Eric Wieser wrote:
It sounds like you're using an old version of numpy, where boolean scalars were interpreted as integers. What version are you using? Eric
Indeed, you are maybe using a pre 1.9 version (post 1.9 should at least have a DeprecationWarning or some such, though you might not notice it IIRC). For newer versions you should get boolean indexing, the result of it may be a bit confusing. It is advanced indexing, basically with False giving you an empty array (with an extra dimension of size 0) and True being much like an `np.newaxis`. It all makes perfect sense if you think of it of a 0d array picking.... The same thing is true for example for lists of booleans.  Sebastian
On Thu, Dec 14, 2017, 04:27 Joe <solarjoe@posteo.org> wrote:
Hello, thanks for you feedback.
Sorry, if thie question is stupid and the case below does not make sense. I am just trying to understand the logic. For
x = np.random.rand(2,3)
x[True] x[(True,)]
or
x[False] x[(False,)]
where True and False are not arrays, it will pick the first or second row.
Is this basic indexing then with one the rules  obj is an integer  obj is a tuple of slice objects and integers. ?
Am 13.12.2017 21:49 schrieb Eric Wieser:
Increasingly, NumPy does not considers booleans to be integer types, and indexing is one of these cases.
So no, it will not be treated as a tuple of integers, but as a 0d mask
Eric
On Wed, 13 Dec 2017 at 12:44 Joe <solarjoe@posteo.org> wrote:
Hi,
yet another question.
I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False]
that might e.g result from
import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False]
x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False]
I understood that they are equivalent to
x[(False,)]
I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :)
In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right?
Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int?
Cheers, Joe _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion [1]
Links:  [1] https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
Hi, you are right. I am using two different versions, Numpy 1.10.4 and 1.9.0 Both show this behavior. Numpy 1.11.1 also does, but now raises VisibleDeprecationWarning: using a boolean instead of an integer will result in an error in the future Thanks! Am 14.12.2017 21:37 schrieb Sebastian Berg:
On Thu, 20171214 at 16:24 +0000, Eric Wieser wrote:
It sounds like you're using an old version of numpy, where boolean scalars were interpreted as integers. What version are you using? Eric
Indeed, you are maybe using a pre 1.9 version (post 1.9 should at least have a DeprecationWarning or some such, though you might not notice it IIRC). For newer versions you should get boolean indexing, the result of it may be a bit confusing. It is advanced indexing, basically with False giving you an empty array (with an extra dimension of size 0) and True being much like an `np.newaxis`. It all makes perfect sense if you think of it of a 0d array picking....
The same thing is true for example for lists of booleans.
 Sebastian
On Thu, Dec 14, 2017, 04:27 Joe <solarjoe@posteo.org> wrote:
Hello, thanks for you feedback.
Sorry, if thie question is stupid and the case below does not make sense. I am just trying to understand the logic. For
x = np.random.rand(2,3)
x[True] x[(True,)]
or
x[False] x[(False,)]
where True and False are not arrays, it will pick the first or second row.
Is this basic indexing then with one the rules  obj is an integer  obj is a tuple of slice objects and integers. ?
Am 13.12.2017 21:49 schrieb Eric Wieser:
Increasingly, NumPy does not considers booleans to be integer types, and indexing is one of these cases.
So no, it will not be treated as a tuple of integers, but as a 0d mask
Eric
On Wed, 13 Dec 2017 at 12:44 Joe <solarjoe@posteo.org> wrote:
Hi,
yet another question.
I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False]
that might e.g result from
import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False]
x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False]
I understood that they are equivalent to
x[(False,)]
I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :)
In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right?
Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int?
Cheers, Joe _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion [1]
Links:  [1] https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On minor thing that instead of 'ret' there should be 'x'. With kind regards, gdg On Dec 12, 2017 22:51, "Joe" <solarjoe@posteo.org> wrote: Hi, the best example I found was this one: https://stackoverflow.com/a/29319864/7919597 def func_for_scalars_or_vectors(x): x = np.asarray(x) scalar_input = False if x.ndim == 0: x = x[None] # Makes x 1D scalar_input = True # The magic happens here if scalar_input: return np.squeeze(ret) return ret Is this as good as it gets or do you have other suggestions? Cheers, Joe _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
On Wed, Dec 13, 2017 at 5:00 AM, Kirill Balunov <kirillbalunov@gmail.com> wrote:
On minor thing that instead of 'ret' there should be 'x'.
No, `x` is the input. The code that actually does the computation (elided here by the `# The magic happens here` comment) would have assigned to `ret`.  Robert Kern
Oh, sorry for noise... With kind regards, gdg On Dec 12, 2017 23:05, "Robert Kern" <robert.kern@gmail.com> wrote:
On Wed, Dec 13, 2017 at 5:00 AM, Kirill Balunov <kirillbalunov@gmail.com> wrote:
On minor thing that instead of 'ret' there should be 'x'.
No, `x` is the input. The code that actually does the computation (elided here by the `# The magic happens here` comment) would have assigned to `ret`.
 Robert Kern
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@python.org https://mail.python.org/mailman/listinfo/numpydiscussion
participants (10)

Eric Wieser

Joe

Kirill Balunov

Mark Campanelli

Marten van Kerkwijk

Nathaniel Smith

Peng Yu

Robert Kern

Sebastian Berg

Stephan Hoyer