[Numpy-discussion] long integers in genfromtxt

[josef.pktd at gmail.com]

I was trying to find out what the "helpful" message
"TypeError: expected a readable buffer object"  means

and it seems genfromtxt has problems identifying long integers (at
least on Windows 32)

>>> np.array(4160680000,int)
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    np.array(4160680000,int)
OverflowError: long int too large to convert to int


>>> s = '''Date,Open,High,Low,Close,Volume,Adj Close
... 2010-02-12,1075.95,1077.81,1062.97,1075.51,4160680000,1075.51
... 2010-02-11,1067.10,1080.04,1060.59,1078.47,4400870000,1078.47'''


>>> sh = StringIO(s)
>>> data = np.genfromtxt(sh, delimiter=",", dtype=None, names=True)
Traceback (most recent call last):
  File "C:\Programs\Python25\Lib\site-packages\numpy\lib\io.py", line
1367, in genfromtxt
    output = np.array(data, dtype=ddtype)
TypeError: expected a readable buffer object

>>> sh = StringIO(s)
>>> data = ts.tsfromtxt(sh, delimiter=",", dtype=None, datecols=(0,), names=True)
Traceback (most recent call last):
  File "\Programs\Python25\Lib\site-packages\scikits\timeseries\extras.py",
line 504, in tsfromtxt
  File "\Programs\Python25\Lib\site-packages\scikits\timeseries\_preview.py",
line 1312, in genfromtxt
TypeError: expected a readable buffer object

>>> dt= [('','S10'),('',float),('',float),('',float),('',float),('',int),('',float)]
>>> sh = StringIO(s)
>>> data = np.genfromtxt(sh, delimiter=",", dtype=dt, names=True)
Traceback (most recent call last):
  File "C:\Programs\Python25\Lib\site-packages\numpy\lib\io.py", line
1388, in genfromtxt
    rows = np.array(data, dtype=[('', _) for _ in dtype_flat])
TypeError: expected a readable buffer object

>>> sh = StringIO(s)
>>> data = ts.tsfromtxt(sh, delimiter=",", dtype=dt, datecols=(0,), names=True)
Traceback (most recent call last):
  File "\Programs\Python25\Lib\site-packages\scikits\timeseries\extras.py",
line 451, in tsfromtxt
  File "\Programs\Python25\Lib\site-packages\scikits\timeseries\_preview.py",
line 798, in easy_dtype
  File "\Programs\Python25\Lib\site-packages\scikits\timeseries\_preview.py",
line 364, in __call__
  File "\Programs\Python25\Lib\site-packages\scikits\timeseries\_preview.py",
line 329, in validate
TypeError: object of type 'bool' has no len()

using float works, but I needed the debugger to figure out what the
problem with my data is (or whether I just make mistakes)

>>> dt= [('','S10'),('',float),('',float),('',float),('',float),('',float),('',float)]
>>> sh = StringIO(s)
>>> data = np.genfromtxt(sh, delimiter=",", dtype=dt, names=True)
>>> data
array([ ('2010-02-12', 1075.95, 1077.8099999999999, 1062.97, 1075.51,
4160680000.0, 1075.51),
       ('2010-02-11', 1067.0999999999999, 1080.04, 1060.5899999999999,
1078.47, 4400870000.0, 1078.47)],
      dtype=[('Date', '|S10'), ('Open', '<f8'), ('High', '<f8'),
('Low', '<f8'), ('Close', '<f8'), ('Volume', '<f8'), ('Adj_Close',
'<f8')])
>>>

>>> ts.version.version
'0.91.3'

Josef