column_stack with mixed data types
I have data in a spreadsheet where the first column is an integer. the second is a float, columns 3-5 are strings, and columns 6 and 7 are floats. I have each column as a list, but when I use column_stack, I get back a 2D array of strings. What is the easiest way to get a recarray out of this list of lists? Is recarray my best/only choice? Thanks, Ryan
In writing a function to parse my data, I ran into some unexpected behavior. Is it intentional that a recarray can only be created by a list of tuples and not by a list of lists? Here is what I ran into: ipdb>biglist[0:10] Out[68]: [[7, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''], [8, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''], [9, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''], [10, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''], [11, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''], [12, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''], [13, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, 'With headliner'], [14, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, 'With headliner'], [15, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, ''], [16, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, '']] ipdb>tuplelist=[tuple(row) for row in biglist] ipdb>tuplelist[0:10] Out[68]: [(7, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''), (8, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''), (9, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''), (10, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''), (11, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''), (12, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''), (13, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, 'With headliner'), (14, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, 'With headliner'), (15, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, ''), (16, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, '')] ipdb>dt Out[68]: dtype([('Test Number', '<i4'), ('Foam Thick. (mm)', '<f8'), ('Foam Description', '|S14'), ('Impactor', '|S14'), ('Mass (kg)', '|S14'), ('Drop Height (in.)', '<f8'), ('Flag Height (in.)', '<f8'), ('Notes', '|S14')]) ipdb>array(biglist, dtype=dt) *** TypeError: expected a readable buffer object This does not work for a list of lists, but does work for a list of tuples: ipdb>array(tuplelist, dtype=dt) Out[68]: array([(7, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''), (8, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''), (9, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.75, ''), (10, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''), (11, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''), (12, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, ''), (13, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, 'With headliner'), (14, 20.0, 'HE 4.0 pcf', 'Flat', 'Large', 32.6875, 4.6875, 'With headliner'), (15, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, ''), (16, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, ''), (17, 20.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.25, ''), (18, 15.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.0, ''), (19, 15.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.0, ''), (20, 15.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 32.6875, 5.0, ''), (21, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.5625, ''), (22, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.5625, ''), (23, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.5625, ''), (24, 10.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.2812999999999999, ''), (25, 10.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.2812999999999999, ''), (26, 10.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.2812999999999999, ''), (27, 5.0, 'HE 4.0 pcf', 'Flat', 'Small', 32.6875, 2.0937999999999999, ''), (28, 20.0, 'P420', 'Flat', 'Small', 32.6875, 2.7812999999999999, ''), (29, 20.0, 'P420', 'Flat', 'Small', 32.6875, 2.7812999999999999, ''), (30, 20.0, 'E175', 'Flat', 'Small', 32.6875, 2.7812999999999999, ''), (31, 20.0, 'E175', 'Flat', 'Small', 32.6875, 2.7812999999999999, ''), (32, 20.0, 'E175', 'Flat', 'Small', 32.6875, 2.7812999999999999, ''), (33, 20.0, 'E175', 'Flat', 'Small', 32.6875, 2.7812999999999999, ''), (34, 15.0, 'E175', 'Flat', 'Small', 32.6875, 2.5, ''), (35, 15.0, 'E175', 'Flat', 'Small', 32.6875, 2.5, ''), (36, 15.0, 'E175', 'Flat', 'Small', 32.6875, 2.5, ''), (37, 20.0, 'E175', 'Hemi', '0.2493', 32.6875, 5.1875, ''), (38, 20.0, 'E175', 'Hemi', '0.2493', 32.6875, 5.1875, ''), (39, 20.0, 'E175', 'Hemi', '0.2493', 32.6875, 5.1875, ''), (40, 15.0, 'E175', 'Hemi', '0.2493', 32.6875, 5.0, ''), (41, 15.0, 'E175', 'Hemi', '0.2493', 32.6875, 5.0, ''), (42, 15.0, 'E175', 'Hemi', '0.2493', 32.6875, 5.0, ''), (43, 15.0, 'E175', 'Flat', 'Small', 10.5625, 2.5625, ''), (44, 15.0, 'E175', 'Flat', 'Small', 10.5625, 2.5625, ''), (45, 15.0, 'E175', 'Flat', 'Small', 10.5625, 2.5625, ''), (46, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.5625, 2.5625, ''), (47, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.5625, 2.5625, ''), (48, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.5625, 2.5625, ''), (49, 10.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.25, 2.25, ''), (50, 10.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.25, 2.25, ''), (51, 10.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.25, 2.25, ''), (52, 10.0, 'E175', 'Flat', 'Small', 10.25, 2.25, ''), (53, 10.0, 'E175', 'Flat', 'Small', 10.25, 2.25, ''), (54, 10.0, 'E175', 'Flat', 'Small', 10.25, 2.25, ''), (55, 5.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.125, 2.1875, ''), (56, 5.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.125, 2.1875, ''), (57, 5.0, 'HE 4.0 pcf', 'Flat', 'Small', 10.125, 2.1875, ''), (58, 15.0, 'HE 4.0 pcf', 'Flat', 'Large', 12.5, 4.5, ''), (59, 15.0, 'HE 4.0 pcf', 'Flat', 'Large', 12.5, 4.5, ''), (60, 15.0, 'HE 4.0 pcf', 'Flat', 'Large', 12.5, 4.5, ''), (61, 15.0, 'E175', 'Flat', 'Large', 12.5, 4.5, ''), (62, 15.0, 'E175', 'Flat', 'Large', 12.5, 4.5, ''), (63, 15.0, 'E175', 'Flat', 'Large', 12.5, 4.5, ''), (64, 10.0, 'HE 4.0 pcf', 'Flat', 'Large', 12.125, 4.375, ''), (65, 10.0, 'HE 4.0 pcf', 'Flat', 'Large', 12.125, 4.375, ''), (66, 10.0, 'HE 4.0 pcf', 'Flat', 'Large', 12.125, 4.375, ''), (67, 10.0, 'E175', 'Flat', 'Large', 12.125, 4.375, ''), (68, 10.0, 'E175', 'Flat', 'Large', 12.125, 4.375, ''), (69, 10.0, 'E175', 'Flat', 'Large', 12.125, 4.375, ''), (70, 10.0, 'E175', 'Hemi', '0.2493', 12.875, 4.8125, ''), (71, 10.0, 'E175', 'Hemi', '0.2493', 12.875, 4.8125, ''), (72, 10.0, 'E175', 'Hemi', '0.2493', 12.875, 4.8125, ''), (73, 10.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 12.875, 4.8125, ''), (74, 10.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 12.875, 4.8125, ''), (75, 10.0, 'HE 4.0 pcf', 'Hemi', '0.2493', 12.875, 4.8125, ''), (76, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 18.3125, 2.5, ''), (77, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 18.3125, 2.5, ''), (78, 15.0, 'HE 4.0 pcf', 'Flat', 'Small', 18.3125, 2.5, ''), (79, 15.0, 'E175', 'Flat', 'Small', 18.3125, 2.5, ''), (80, 15.0, 'E175', 'Flat', 'Small', 18.3125, 2.5, ''), (81, 15.0, 'E175', 'Flat', 'Small', 18.3125, 2.5, ''), (82, 20.0, 'E175', 'Flat', 'Small', 45.6875, 2.7187999999999999, ''), (83, 20.0, 'E175', 'Flat', 'Small', 45.6875, 2.7187999999999999, ''), (84, 20.0, 'E175', 'Flat', 'Small', 45.6875, 2.7187999999999999, ''), (85, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 45.6875, 2.7187999999999999, ''), (86, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 45.6875, 2.7187999999999999, ''), (87, 20.0, 'HE 4.0 pcf', 'Flat', 'Small', 45.6875, 2.7187999999999999, '')], dtype=[('Test Number', '<i4'), ('Foam Thick. (mm)', '<f8'), ('Foam Description', '|S14'), ('Impactor', '|S14'), ('Mass (kg)', '|S14'), ('Drop Height (in.)', '<f8'), ('Flag Height (in.)', '<f8'), ('Notes', '|S14')]) ipdb> On 7/30/07, Ryan Krauss <ryanlists@gmail.com> wrote:
I have data in a spreadsheet where the first column is an integer. the second is a float, columns 3-5 are strings, and columns 6 and 7 are floats. I have each column as a list, but when I use column_stack, I get back a 2D array of strings. What is the easiest way to get a recarray out of this list of lists? Is recarray my best/only choice?
Thanks,
Ryan
Ryan Krauss wrote:
In writing a function to parse my data, I ran into some unexpected behavior. Is it intentional that a recarray can only be created by a list of tuples and not by a list of lists?
Yeah. The array() constructor needs some information for it to work its magic. Requiring tuples for the actual records is an arbitrary choice, but a reasonable one given that. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
I just tend to think in terms of lists rather than tuples. Why is a tuple a more reasonable choice than a list? (I'm really asking and not being argumentative, since you can't hear my tone.) Ryan On 7/30/07, Robert Kern <robert.kern@gmail.com> wrote:
Ryan Krauss wrote:
In writing a function to parse my data, I ran into some unexpected behavior. Is it intentional that a recarray can only be created by a list of tuples and not by a list of lists?
Yeah. The array() constructor needs some information for it to work its magic. Requiring tuples for the actual records is an arbitrary choice, but a reasonable one given that.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Ryan Krauss wrote:
I just tend to think in terms of lists rather than tuples. Why is a tuple a more reasonable choice than a list? (I'm really asking and not being argumentative, since you can't hear my tone.)
The key thing is that the type of the container of records is different from the type of the record, so you'd either have to have a list of tuples (or a list of lists of ... tuples) or a tuple of lists. There is a tendency to use lists for homogeneous collections and tuples for inhomogeneous collections; Guido says that this is the main difference in how he uses tuples and lists. When you get an answer to a DB-API2 SQL query, you get a list of (usually augmented) tuples. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On 7/31/07, Robert Kern <robert.kern@gmail.com> wrote:
Ryan Krauss wrote:
I just tend to think in terms of lists rather than tuples. Why is a tuple a more reasonable choice than a list? (I'm really asking and not being argumentative, since you can't hear my tone.)
The key thing is that the type of the container of records is different from the type of the record, so you'd either have to have a list of tuples (or a list of lists of ... tuples) or a tuple of lists. There is a tendency to use lists for homogeneous collections and tuples for inhomogeneous collections; Guido says that this is the main difference in how he uses tuples and lists. When you get an answer to a DB-API2 SQL query, you get a list of (usually augmented) tuples.
This is the best explanation I have heard about this yet. Where does he say this ? Just for reference .... Thanks, Sebastian
Sebastian Haase wrote:
On 7/31/07, Robert Kern <robert.kern@gmail.com> wrote:
I just tend to think in terms of lists rather than tuples. Why is a tuple a more reasonable choice than a list? (I'm really asking and not being argumentative, since you can't hear my tone.) The key thing is that the type of the container of records is different from the type of the record, so you'd either have to have a list of tuples (or a list of
Ryan Krauss wrote: lists of ... tuples) or a tuple of lists. There is a tendency to use lists for homogeneous collections and tuples for inhomogeneous collections; Guido says that this is the main difference in how he uses tuples and lists. When you get an answer to a DB-API2 SQL query, you get a list of (usually augmented) tuples.
This is the best explanation I have heard about this yet. Where does he say this ? Just for reference ....
http://mail.python.org/pipermail/python-dev/2003-March/033964.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On 7/31/07, Robert Kern <robert.kern@gmail.com> wrote:
Sebastian Haase wrote:
On 7/31/07, Robert Kern <robert.kern@gmail.com> wrote:
I just tend to think in terms of lists rather than tuples. Why is a tuple a more reasonable choice than a list? (I'm really asking and not being argumentative, since you can't hear my tone.) The key thing is that the type of the container of records is different from the type of the record, so you'd either have to have a list of tuples (or a list of
Ryan Krauss wrote: lists of ... tuples) or a tuple of lists. There is a tendency to use lists for homogeneous collections and tuples for inhomogeneous collections; Guido says that this is the main difference in how he uses tuples and lists. When you get an answer to a DB-API2 SQL query, you get a list of (usually augmented) tuples.
This is the best explanation I have heard about this yet. Where does he say this ? Just for reference ....
http://mail.python.org/pipermail/python-dev/2003-March/033964.html
Thanks, apparently this still hasn't made it into the Python Style Guid though: http://www.python.org/dev/peps/pep-0008/ Refer also: [Python-Dev] Tuples vs lists Aahz aahz@pythoncraft.com 12 Mar 2003 http://mail.python.org/pipermail/python-dev/2003-March/033981.html [Python-Dev] Christian Tismer tismer@tismer.com 12 Mar 2003 http://mail.python.org/pipermail/python-dev/2003-March/033966.html he says: """I never realized this, and I'm a bit stunned. (but by no means negative about it, just surprized)""" Cheers, Sebastian
participants (3)
-
Robert Kern -
Ryan Krauss -
Sebastian Haase