Read file with comma decimal separator
Hello, I have a file that used comma as a decimal separator. How can I read a file like that using loadtxt or genfromtxt ? Thanks, Florian
Am Mittwoch, 31. Juli 2013, 15:25:56 schrieb Florian Lindner:
Hello,
I have a file that used comma as a decimal separator. How can I read a file like that using loadtxt or genfromtxt ?
Since I'm not sure how many columns the file will have, I tried: conv = {} for i in range(1000): conv[i] = lambda a: a.replace(",", ".") data = np.loadtxt(f, skiprows = 2, converters = conv) but: File "/usr/lib/python2.7/site-packages/numpy/lib/npyio.py", line 817, in loadtxt converters[i] = conv IndexError: list assignment index out of range Regards, Florian
s = StringIO("1,1.3,abcde") data = np.genfromtxt(s, dtype=[('myint','i8'),('myfloat','f8'), ... ('mystring','S5')], delimiter=",") data array((1, 1.3, 'abcde'),
.... Use the 'delimiter' key in the genfromtxt routine. Like in the docu dtype=[('myint', '<i8'), ('myfloat', '<f8'), ('mystring', '|S5')]) Best Marcel On 31.07.2013, at 15:31, Florian Lindner <mailinglists@xgm.de> wrote:
Am Mittwoch, 31. Juli 2013, 15:25:56 schrieb Florian Lindner:
Hello,
I have a file that used comma as a decimal separator. How can I read a file like that using loadtxt or genfromtxt ?
Since I'm not sure how many columns the file will have, I tried:
conv = {} for i in range(1000): conv[i] = lambda a: a.replace(",", ".")
data = np.loadtxt(f, skiprows = 2, converters = conv)
but:
File "/usr/lib/python2.7/site-packages/numpy/lib/npyio.py", line 817, in loadtxt converters[i] = conv IndexError: list assignment index out of range
Regards, Florian _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On Jul 31, 2013, at 15:31 , Florian Lindner <mailinglists@xgm.de> wrote:
Am Mittwoch, 31. Juli 2013, 15:25:56 schrieb Florian Lindner:
Hello,
I have a file that used comma as a decimal separator. How can I read a file like that using loadtxt or genfromtxt ?
A quick and dirty approach would be to create a generator that would parse your initial input and replace the ',' by '.' on each line. You'd just have to feed the generator to `genfromtxt`:
X = StringIO('1,1\t1,2\t1,3\n2,1\t2,2\t,2,3') replaced = (line.replace(",", ".") for line in X) np.genfromtxt(replaced, delimiter="\t")
Of course, that'd work only if you don't intend to use "," as your delimiter, in which case you're out of luck.
You could preread the input using StringIO import StringIO import numpy as np s = open('test.txt').read().replace(',','.') data = np.loadtxt(StringIO.StringIO(s)) print data On 31 July 2013 23:40, Pierre Gerard-Marchant <pgmdevlist@gmail.com> wrote:
On Jul 31, 2013, at 15:31 , Florian Lindner <mailinglists@xgm.de> wrote:
Am Mittwoch, 31. Juli 2013, 15:25:56 schrieb Florian Lindner:
Hello,
I have a file that used comma as a decimal separator. How can I read a file like that using loadtxt or genfromtxt ?
A quick and dirty approach would be to create a generator that would parse your initial input and replace the ',' by '.' on each line. You'd just have to feed the generator to `genfromtxt`:
X = StringIO('1,1\t1,2\t1,3\n2,1\t2,2\t,2,3') replaced = (line.replace(",", ".") for line in X) np.genfromtxt(replaced, delimiter="\t")
Of course, that'd work only if you don't intend to use "," as your delimiter, in which case you're out of luck. _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
Hi, I do not know if this is still relevant but I just saw a function in pandas which has an option to specify decimal and header, this gives you a dataframe object, you can get the values using DataFrame.values property. pandas.io.parsers.read_table(*filepath_or_buffer,...,decimal=',', header=None)* * * See this example import pandas as pd In [31]: X = StringIO('1,1\t1,2\t1,3\n2,1\t2,2\t2,3') In [32]: table = pd.io.parsers.read_table(X, decimal=",", header=None) In [33]: table Out[33]: 0 1 2 0 1.1 1.2 1.3 1 2.1 2.2 2.3 In [34]: table.values Out[34]: array([[ 1.1, 1.2, 1.3], [ 2.1, 2.2, 2.3]]) -- Sasha * * * * 2013/7/31 Florian Lindner <mailinglists@xgm.de>
Hello,
I have a file that used comma as a decimal separator. How can I read a file like that using loadtxt or genfromtxt ?
Thanks, Florian _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
participants (5)
-
Florian Lindner -
gary ruben -
Marcel Blattner -
Oleksandr Huziy -
Pierre Gerard-Marchant