Saving and loading a structured array from a TEXT file
Is there a way to save a structured array in a text file? My problem is not so much in the saving procedure, but rather in the 'reloading' procedure. See below In [3]: import numpy as np In [4]: r = np.ones(3,dtype=[('name', '|S5'), ('foo', '<i8'), ('bar', '<f8')]) In [5]: r.tofile('toto.txt',sep='\n') bash-4.2$ cat toto.txt ('1', 1, 1.0) ('1', 1, 1.0) ('1', 1, 1.0) In [7]: r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/cls1fs/clseng/10/<ipython-input-7-b07ba265ede7> in <module>() ----> 1 r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype) ValueError: Unable to read character files of that array type -- Emmanuel
On 23 Jan 2012, at 21:15, Emmanuel Mayssat wrote:
Is there a way to save a structured array in a text file? My problem is not so much in the saving procedure, but rather in the 'reloading' procedure. See below
In [3]: import numpy as np
In [4]: r = np.ones(3,dtype=[('name', '|S5'), ('foo', '<i8'), ('bar', '<f8')])
In [5]: r.tofile('toto.txt',sep='\n')
bash-4.2$ cat toto.txt ('1', 1, 1.0) ('1', 1, 1.0) ('1', 1, 1.0)
In [7]: r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/cls1fs/clseng/10/<ipython-input-7-b07ba265ede7> in <module>() ----> 1 r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype)
ValueError: Unable to read character files of that array type
I think most of the np.fromfile functionality works for binary input; for reading text input np.loadtxt and np.genfromtxt are the (currently) recommended functions. It is bit tricky to read the format generated by tofile() in the above example, but the following should work: cnv = {0: lambda s: s.lstrip('('), -1: lambda s: s.rstrip(')')} r2 = np.loadtxt('toto.txt', delimiter=',', converters=cnv, dtype=r.dtype) Generally loadtxt works more smoothly together with savetxt, but the latter unfortunately does not offer an easy way to save structured arrays (note to self and others currently working on npyio: definitely room for improvement!). HTH, Derek
On 23 Jan 2012, at 22:07, Derek Homeier wrote:
In [4]: r = np.ones(3,dtype=[('name', '|S5'), ('foo', '<i8'), ('bar', '<f8')])
In [5]: r.tofile('toto.txt',sep='\n')
bash-4.2$ cat toto.txt ('1', 1, 1.0) ('1', 1, 1.0) ('1', 1, 1.0)
cnv = {0: lambda s: s.lstrip('('), -1: lambda s: s.rstrip(')')} r2 = np.loadtxt('toto.txt', delimiter=',', converters=cnv, dtype=r.dtype)
Generally loadtxt works more smoothly together with savetxt, but the latter unfortunately does not offer an easy way to save structured arrays (note to self and others currently working on npyio: definitely room for improvement!).
For the record, in that example np.savetxt('toto.txt', r, fmt='%s,%d,%f') would work as well, saving you the custom converter for loadtxt - it could just become tedious to work out the format for more complex structures, so an option to construct this automatically from r.dtype could certainly be a nice enhancement. Just wondering, is there something like the inverse operator to np.format_parser, i.e. mapping each dtype to a default print format specifier? Cheers, Derek
participants (2)
-
Derek Homeier -
Emmanuel Mayssat