[Numpy-discussion] using loadtxt to load a text file in to a numpy array

Freddie Witherden freddie at witherden.org
Fri Jan 17 08:18:38 EST 2014


On 17/01/14 13:09, Aldcroft, Thomas wrote:
> I've been playing around with porting a stack of analysis libraries to
> Python 3 and this is a very timely thread and comment.  What I
> discovered right away is that all the string data coming from binary
> HDF5 files show up (as expected) as 'S' type,, but that trying to make
> everything actually work in Python 3 without converting to 'U' is a big
> mess of whack-a-mole.  
> 
> Yes, it's possible to change my libraries to use bytestring literals
> everywhere, but the Python 3 user experience becomes horrible because to
> interact with the data all downstream applications need to use
> bytestring literals everywhere.  E.g. doing a simple filter like
> `string_array == 'foo'` doesn't work, and this will break all existing
> code when trying to run in Python 3.  And every time you try to print
> something it has this horrible "b" in front.  Ugly, and it just won't
> work well in the end.

In terms of HDF5 it is interesting to look at how h5py -- which has to
go between NumPy types and HDF5 conventions -- handles the problem as
described here:

  http://www.h5py.org/docs/topics/strings.html

which IMHO got it about right.

Regards, Freddie.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140117/53ed3e11/attachment.sig>


More information about the NumPy-Discussion mailing list