I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me. Thoughts? Chuck
I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me.
Thoughts?
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Thanks for the correction and as well as the fix. While it worked for integer and floats (not complex ones), I got an error when using complex
import numpy as np np.__version__ # pull request version '2.0.0.dev-88f9276' np.array([1,2], dtype=np.complex) array([ 1.+0.j, 2.+0.j]) np.array([1,2, np.NA], dtype=np.complex) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py",
ca=np.array([1,2], dtype=np.complex, maskna=True) ca[1]=np.NA ca Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py",
On 01/14/2012 04:31 PM, Charles R Harris wrote: dtypes. This error that is also present in array creation of complex dtypes. Is this known or a new bug? If it is new, then we need to identify what functionality should handle np.NA but are not working. Bruce $ python Python 2.7 (r27:82500, Sep 16 2010, 18:02:00) [GCC 4.5.1 20100907 (Red Hat 4.5.1-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later
On Mon, Jan 16, 2012 at 8:37 AM, Bruce Southey <bsouthey@gmail.com> wrote:
** On 01/14/2012 04:31 PM, Charles R Harris wrote:
I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me.
Thoughts?
Chuck
_______________________________________________ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks for the correction and as well as the fix. While it worked for integer and floats (not complex ones), I got an error when using complex dtypes. This error that is also present in array creation of complex dtypes. Is this known or a new bug?
If it is new, then we need to identify what functionality should handle np.NA but are not working.
Bruce
$ python Python 2.7 (r27:82500, Sep 16 2010, 18:02:00) [GCC 4.5.1 20100907 (Red Hat 4.5.1-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy as np np.__version__ # pull request version '2.0.0.dev-88f9276' np.array([1,2], dtype=np.complex) array([ 1.+0.j, 2.+0.j]) np.array([1,2, np.NA], dtype=np.complex) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later ca=np.array([1,2], dtype=np.complex, maskna=True) ca[1]=np.NA ca Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later
Looks like a different bug involving the *.real and *.imag views. I'll take a look. Chuck
On Mon, Jan 16, 2012 at 8:52 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Mon, Jan 16, 2012 at 8:37 AM, Bruce Southey <bsouthey@gmail.com> wrote:
** On 01/14/2012 04:31 PM, Charles R Harris wrote:
I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me.
Thoughts?
Chuck
_______________________________________________ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks for the correction and as well as the fix. While it worked for integer and floats (not complex ones), I got an error when using complex dtypes. This error that is also present in array creation of complex dtypes. Is this known or a new bug?
If it is new, then we need to identify what functionality should handle np.NA but are not working.
Bruce
import numpy as np np.__version__ # pull request version '2.0.0.dev-88f9276' np.array([1,2], dtype=np.complex) array([ 1.+0.j, 2.+0.j]) np.array([1,2, np.NA], dtype=np.complex) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py",
ca=np.array([1,2], dtype=np.complex, maskna=True) ca[1]=np.NA ca Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py",
$ python Python 2.7 (r27:82500, Sep 16 2010, 18:02:00) [GCC 4.5.1 20100907 (Red Hat 4.5.1-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later
Looks like a different bug involving the *.real and *.imag views. I'll take a look.
Looks like views of masked arrays have other problems: In [13]: a = ones(3, int16, maskna=1) In [14]: a.view(int8) Out[14]: array([1, 0, 1, NA, 1, NA], dtype=int8) I'm not sure what the policy should be here. One could construct a new mask adapted to the view, raise an error when the types don't align (I think the real/imag parts should be considered aligned), or just let the view unmask the array. The last seems dangerous. Hmm... Chuck
On Mon, Jan 16, 2012 at 8:14 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Mon, Jan 16, 2012 at 8:52 AM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Mon, Jan 16, 2012 at 8:37 AM, Bruce Southey <bsouthey@gmail.com>wrote:
** On 01/14/2012 04:31 PM, Charles R Harris wrote:
I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me.
Thoughts?
Chuck
_______________________________________________ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks for the correction and as well as the fix. While it worked for integer and floats (not complex ones), I got an error when using complex dtypes. This error that is also present in array creation of complex dtypes. Is this known or a new bug?
If it is new, then we need to identify what functionality should handle np.NA but are not working.
Bruce
import numpy as np np.__version__ # pull request version '2.0.0.dev-88f9276' np.array([1,2], dtype=np.complex) array([ 1.+0.j, 2.+0.j]) np.array([1,2, np.NA], dtype=np.complex) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py",
ca=np.array([1,2], dtype=np.complex, maskna=True) ca[1]=np.NA ca Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py",
$ python Python 2.7 (r27:82500, Sep 16 2010, 18:02:00) [GCC 4.5.1 20100907 (Red Hat 4.5.1-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later
Looks like a different bug involving the *.real and *.imag views. I'll take a look.
Looks like views of masked arrays have other problems:
In [13]: a = ones(3, int16, maskna=1)
In [14]: a.view(int8) Out[14]: array([1, 0, 1, NA, 1, NA], dtype=int8)
This looks like a serious bug to me, to avoid memory corruption issues it should raise an exception. -Mark
I'm not sure what the policy should be here. One could construct a new mask adapted to the view, raise an error when the types don't align (I think the real/imag parts should be considered aligned), or just let the view unmask the array. The last seems dangerous. Hmm...
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Mon, Jan 16, 2012 at 8:37 AM, Bruce Southey <bsouthey@gmail.com> wrote:
** On 01/14/2012 04:31 PM, Charles R Harris wrote:
I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me.
Thoughts?
Chuck
_______________________________________________ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks for the correction and as well as the fix. While it worked for integer and floats (not complex ones), I got an error when using complex dtypes. This error that is also present in array creation of complex dtypes. Is this known or a new bug?
If it is new, then we need to identify what functionality should handle np.NA but are not working.
Bruce
$ python Python 2.7 (r27:82500, Sep 16 2010, 18:02:00) [GCC 4.5.1 20100907 (Red Hat 4.5.1-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy as np np.__version__ # pull request version '2.0.0.dev-88f9276' np.array([1,2], dtype=np.complex) array([ 1.+0.j, 2.+0.j]) np.array([1,2, np.NA], dtype=np.complex) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later ca=np.array([1,2], dtype=np.complex, maskna=True) ca[1]=np.NA ca Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/core/numeric.py", line 1445, in array_repr ', ', "array(") File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 459, in array2string separator, prefix, formatter=formatter) File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 263, in _array2string suppress_small), File "/usr/lib64/python2.7/site-packages/numpy/core/arrayprint.py", line 724, in __init__ self.real_format = FloatFormat(x.real, precision, suppress_small) ValueError: Cannot construct a view of data together with the NPY_ARRAY_MASKNA flag, the NA mask must be added later
The location of this problem is easy to find, but the fix isn't completely trivial as it seems there is no easy way to copy masks between arrays. There may be, but I haven't found it. Also there is the unfortunate fact that real and imag are array methods and work for non-complex arrays In [6]: a = ones(3, 'S') In [7]: a.real Out[7]: array(['1', '1', '1'], dtype='|S1') In [8]: a.imag Out[8]: array(['', '', ''], dtype='|S1') which makes a simple view impractical. Not that views seem to work. Another complication of the NA stuff is that there two types of NA, a potential multivalued NA, and a simple boolean NA. I think we need to pick between the two as supporting both makes a mess. Because of the common complaint about memory usage, I vote for simple boolean which offers the option of bit arrays for the masks. Chuck
participants (3)
-
Bruce Southey -
Charles R Harris -
Mark Wiebe