[Numpy-discussion] Possible roadmap addendum: building better text file readers

[Travis Oliphant]

The architecture of this system should separate the iteration across the I/O from the transformation *on* the data.   It should also allow the ability to plug-in different transformations at a low-level --- some thought should go into the API of the low-level transformation.    Being able to memory-map text files would also be a bonus (but this would require some kind of index to allow seeking through the file).    

I have some ideas in this direction, but don't have the time to write them up just yet. 

-Travis


On Feb 27, 2012, at 2:44 PM, Matthew Brett wrote:

> Hi,
> 
> On Mon, Feb 27, 2012 at 2:58 PM, Pauli Virtanen <pav at iki.fi> wrote:
>> Hi,
>> 
>> 27.02.2012 20:43, Alan G Isaac kirjoitti:
>>> On 2/27/2012 2:28 PM, Pauli Virtanen wrote:
>>>> ISO specifies comma to be used in international standards
>>>> (ISO/IEC Directives, part 2 / 6.6.8.1):
>>>> 
>>>> http://isotc.iso.org/livelink/livelink?func=ll&objId=10562502&objAction=download
>>> 
>>> I do not think you are right.
>>> I think that is a presentational requirement:
>>> rules of presentation for documents that
>>> are intended to become international standards.
>> 
>> Yes, it's an requirement for the standard texts themselves, but not what
>> the standard texts specify. Which is why I didn't think it was so
>> relevant (but the wikipedia link just prompted an immediate [citation
>> needed]). I agree that using something else than '.' does not make much
>> sense.
> 
> I suppose if anyone out there is from a country that uses commas for
> decimals in CSV files and does not want to have to convert them before
> reading them will be keen to volunteer to help with the coding.  I am
> certainly glad it is not my own case,
> 
> Best,
> 
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion