printing structured arrays
Hi, I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array? Here an example of what I mean: data = [ (1, 2), (3, 4.1) ] dtype = [('x', float), ('y', float)] print '### ndarray' a = numpy.array(data) print a print '### structured array' a = numpy.array(data, dtype=dtype) print a Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] Thanks Bruce
On Fri, Mar 5, 2010 at 8:00 AM, Bruce Schultz <bruce.schultz@gmail.com>wrote:
Hi,
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Here an example of what I mean:
data = [ (1, 2), (3, 4.1) ] dtype = [('x', float), ('y', float)] print '### ndarray' a = numpy.array(data) print a print '### structured array' a = numpy.array(data, dtype=dtype) print a
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)]
Thanks Bruce
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I still couldn't figure out how floating point numbers look nicely on screen in cases like yours (i.e., trying numpy.array2string()) but you can make sure by using numpy.savetxt("file", array, fmt="%.1f") you will always have specified precision in the written file. -- Gökhan
On Sat, Mar 6, 2010 at 8:35 AM, Gökhan Sever <gokhansever@gmail.com> wrote:
On Fri, Mar 5, 2010 at 8:00 AM, Bruce Schultz <bruce.schultz@gmail.com> wrote:
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)]
Thanks Bruce
I still couldn't figure out how floating point numbers look nicely on screen in cases like yours (i.e., trying numpy.array2string()) but you can make sure by using numpy.savetxt("file", array, fmt="%.1f") you will always have specified precision in the written file.
Using numpy.array2string() gives the same format as the output above. Using numpy.savetxt() creates the same nicely formatted file containing the lines below for both structured and unstructured arrays. 1.0 2.0 3.0 4.1 But I was mainly curious about this because I just want to quickly dump data out to the console for debugging, and the unstructured format is obviously much easier to read. It seems like from other discussion in the thread that the quick solution is to convert back to a unstructured array with something like view((float, 2)), but that seems a bit clumsy. Cheers Bruce
On 10/03/10 10:09, Bruce Schultz wrote:
On Sat, Mar 6, 2010 at 8:35 AM, Gökhan Sever <gokhansever@gmail.com> wrote:
On Fri, Mar 5, 2010 at 8:00 AM, Bruce Schultz <bruce.schultz@gmail.com> wrote:
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)]
I still couldn't figure out how floating point numbers look nicely on screen in cases like yours (i.e., trying numpy.array2string()) but you can make sure by using numpy.savetxt("file", array, fmt="%.1f") you will always have specified precision in the written file.
Using numpy.array2string() gives the same format as the output above.
I started looking at how array2string() is implemented, and came up with this patch which formats my structured array nicely, the same as an unstructured array. It was mainly done as a proof of concept, so it only works for floats and I'm probably doing the wrong thing to detect a structured array by comparing the dtype to void. Maybe someone with more numpy experience can tell me if I'm on the right track... === modified file 'numpy/core/arrayprint.py' --- numpy/core/arrayprint.py 2010-02-21 16:16:34 +0000 +++ numpy/core/arrayprint.py 2010-03-10 13:48:22 +0000 @@ -219,6 +219,10 @@ elif issubclass(dtypeobj, _nt.unicode_) or \ issubclass(dtypeobj, _nt.string_): format_function = repr + elif issubclass(dtypeobj, _nt.void): + #XXX this is for structured arrays.... + format_function = StructuredFormatter(a) + separator = '\n ' else: format_function = str @@ -231,6 +235,17 @@ return lst +class StructuredFormatter: + def __init__(self, a): + self.data = a + self.dtype = a.dtype #XXX use the dtype to build column formatters + + def __call__(self, x): + ff = FloatFormat(self.data.view(float), _float_output_precision, + _float_output_suppress_small) + return '[' + ' '.join([ff(n) for n in x]) + ']' + + def _convert_arrays(obj): import numeric as _nc newtup = [] Cheers Bruce
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
Thanks for any additional hint, Timmie
On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
.view(float) should do it, to created a ndarray view of the structured array data Josef
Thanks for any additional hint, Timmie
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Mon, Mar 8, 2010 at 2:01 PM, <josef.pktd@gmail.com> wrote:
On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
.view(float) should do it, to created a ndarray view of the structured array data
Plus a reshape. I usually know how many columns I have, so I put in axis 1 and leave axis 0 as -1. In [21]: a.view(float).reshape(-1,2) Out[21]: array([[ 1. , 2. ], [ 3. , 4.1]]) Skipper
On Mon, Mar 8, 2010 at 2:04 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Mon, Mar 8, 2010 at 2:01 PM, <josef.pktd@gmail.com> wrote:
On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
.view(float) should do it, to created a ndarray view of the structured array data
Plus a reshape. I usually know how many columns I have, so I put in axis 1 and leave axis 0 as -1.
In [21]: a.view(float).reshape(-1,2) Out[21]: array([[ 1. , 2. ], [ 3. , 4.1]])
a.view(float).reshape(len(a),-1) #if you don't want to count columns I obviously haven't done this in a while. And of course, it only works if all elements of the structured array have the same type. Josef
Skipper _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Mon, Mar 8, 2010 at 2:17 PM, <josef.pktd@gmail.com> wrote:
On Mon, Mar 8, 2010 at 2:04 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Mon, Mar 8, 2010 at 2:01 PM, <josef.pktd@gmail.com> wrote:
On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
.view(float) should do it, to created a ndarray view of the structured array data
Plus a reshape. I usually know how many columns I have, so I put in axis 1 and leave axis 0 as -1.
In [21]: a.view(float).reshape(-1,2) Out[21]: array([[ 1. , 2. ], [ 3. , 4.1]])
a.view(float).reshape(len(a),-1) #if you don't want to count columns
I obviously haven't done this in a while. And of course, it only works if all elements of the structured array have the same type.
For the archives with heterogeneous dtype. import numpy as np b = np.array([(1.0, 'string1', 2.0), (3.0, 'string2', 4.1)], dtype=[('x', float),('str_var', 'a7'),('y',float)]) b[['x','y']].view(float).reshape(len(b),-1) # note the list within list syntax #array([[ 1. , 2. ], # [ 3. , 4.1]]) Skipper
On Mon, Mar 8, 2010 at 2:24 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Mon, Mar 8, 2010 at 2:17 PM, <josef.pktd@gmail.com> wrote:
On Mon, Mar 8, 2010 at 2:04 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Mon, Mar 8, 2010 at 2:01 PM, <josef.pktd@gmail.com> wrote:
On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
.view(float) should do it, to created a ndarray view of the structured array data
Plus a reshape. I usually know how many columns I have, so I put in axis 1 and leave axis 0 as -1.
In [21]: a.view(float).reshape(-1,2) Out[21]: array([[ 1. , 2. ], [ 3. , 4.1]])
a.view(float).reshape(len(a),-1) #if you don't want to count columns
I obviously haven't done this in a while. And of course, it only works if all elements of the structured array have the same type.
For the archives with heterogeneous dtype.
import numpy as np
b = np.array([(1.0, 'string1', 2.0), (3.0, 'string2', 4.1)], dtype=[('x', float),('str_var', 'a7'),('y',float)])
b[['x','y']].view(float).reshape(len(b),-1) # note the list within list syntax
#array([[ 1. , 2. ], # [ 3. , 4.1]])
nice, I've never seen selection of multiple columns before. I didn't know it is possible to get a subset of columns this way
b[['x','y']] array([(1.0, 2.0), (3.0, 4.0999999999999996)], dtype=[('x', '<f8'), ('y', '<f8')]) b['x'] array([ 1., 3.]) b[['x']] array([(1.0,), (3.0,)], dtype=[('x', '<f8')])
Josef
Skipper _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hello, thanks to all who responded and have their input here. I added a little code snippet to show the view and reshape: http://www.scipy.org/Cookbook/Recarray What do you think? Is this worth to go into the official docs? The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite sparse... I still wonder why there is not a quick function for such a view / reshape conversion. Best regards, Timmie
On Mon, Mar 8, 2010 at 5:50 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, thanks to all who responded and have their input here.
I added a little code snippet to show the view and reshape:
http://www.scipy.org/Cookbook/Recarray
What do you think? Is this worth to go into the official docs? The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite sparse...
I still wonder why there is not a quick function for such a view / reshape conversion.
Thanks, the docs for working with arrays with structured dtypes are sparse Note that in your example .view(np.ndarray) doesn't do anything
struct_diffdtype[['str_var', 'x', 'y']].view(np.ndarray).reshape(len(struct_diffdtype),-1) array([[(1.0, 'string1', 2.0)], [(3.0, 'string2', 4.0999999999999996)]], dtype=[('x', '<f8'), ('str_var', '|S7'), ('y', '<f8')])
struct_diffdtype[['x', 'y']].view(np.ndarray).reshape(len(struct_diffdtype),-1) array([[(1.0, 2.0)], [(3.0, 4.0999999999999996)]], dtype=[('x', '<f8'), ('y', '<f8')])
view on columns with floating values (is this a copy???)
struct_diffdtype[['x', 'y']].view(float).reshape(len(struct_diffdtype),-1) array([[ 1. , 2. ], [ 3. , 4.1]])
and float view on strings is not possible
struct_diffdtype[['str_var', 'x', 'y']].view(float).reshape(len(struct_diffdtype),-1) Traceback (most recent call last): ValueError: new type not compatible with array.
Josef
Best regards, Timmie
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
josef.pktd@gmail.com schrieb:
On Mon, Mar 8, 2010 at 5:50 PM, Tim Michelsen <timmichelsen@gmx-topmail.de> wrote:
Hello, thanks to all who responded and have their input here.
I added a little code snippet to show the view and reshape:
http://www.scipy.org/Cookbook/Recarray
What do you think? Is this worth to go into the official docs? The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite sparse...
I still wonder why there is not a quick function for such a view / reshape conversion.
Thanks, the docs for working with arrays with structured dtypes are sparse
Note that in your example .view(np.ndarray) doesn't do anything Please note that I wanted to demonstrate many different ways of view & reshape.
I updated the page: http://www.scipy.org/Cookbook/Recarray
Is this worth to go into the official docs? The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite sparse...
I still wonder why there is not a quick function for such a view / reshape conversion.
Thanks, the docs for working with arrays with structured dtypes are sparse I cannot recover my longin for docs.scipy.org.
Would you advice to add the elaborated example?
Tim Michelsen wrote:
I still wonder why there is not a quick function for such a view / reshape conversion.
Because it is difficult (impossible?) to do in the general case. .view() really isn't that bad, in fact, it remarkably powerful and flexible! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
I still wonder why there is not a quick function for such a view / reshape conversion.
Because it is difficult (impossible?) to do in the general case. .view() really isn't that bad, in fact, it remarkably powerful and flexible! I would not drop .view() but rather add a convenience function for struct_1dtype_float_alt = struct_1dtype.view((np.float, len(struct_1dtype.dtype.names)))
On Mar 8, 2010, at 1:55 PM, Tim Michelsen wrote:
Hello, I am also looking into the convertsion from strcutured arrays to ndarray.
I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?
Output is: ### ndarray [[ 1. 2. ] [ 3. 4.1]] ### structured array [(1.0, 2.0) (3.0, 4.0999999999999996)] How could we make this structured array look like the above shown ndarray with shape (2, 2)?
if you're 100% sure all your fields have the same dtype (float), ``a.view((float,2))`` is the simplest. Note the tuple (dtype, len(a.dtype.names)).
participants (7)
-
Bruce Schultz -
Chris Barker -
Gökhan Sever -
josef.pktd＠gmail.com -
Pierre GM -
Skipper Seabold -
Tim Michelsen