[Numpy-discussion] loadtxt ndmin option

Pierre GM pgmdevlist at gmail.com
Tue May 31 15:28:28 EDT 2011

On May 31, 2011, at 6:37 PM, Derek Homeier wrote:

> On 31 May 2011, at 18:25, Pierre GM wrote:
>> On May 31, 2011, at 5:52 PM, Derek Homeier wrote:
>>> I think stuff like multiple delimiters should have been dealt with
>>> before, as the right place to insert the ndmin code (which includes
>>> the decision to squeeze or not to squeeze as well as to add  
>>> additional
>>> dimensions, if required) would be right at the end before the  
>>> 'unpack'
>>> switch, or  rather replacing the bit:
>>>   if usemask:
>>>       output = output.view(MaskedArray)
>>>       output._mask = outputmask
>>>   if unpack:
>>>       return output.squeeze().T
>>>   return output.squeeze()
>>> But there it's already not clear to me how to deal with the
>>> MaskedArray case...
>> Oh, easy.
>> You need to replace only the last three lines of genfromtxt with the  
>> ones from loadtxt  (808-833). Then, if usemask is True, you need to  
>> use ma.atleast_Xd instead of np.atleast_Xd. Et voilà.
>> Comments:
>> * I would raise an exception if ndmin isn't correct *before* trying  
>> to read the file...
>> * You could define a `collapse_function` that would be  
>> `np.atleast_1d`, `np.atleast_2d`, `ma.atleast_1d`... depending on  
>> the values of `usemask` and `ndmin`...
>> If you have any question about numpy.ma, don't hesitate to contact  
>> me directly.
> Thanks for the directions! I was not sure about the usemask case  
> because it presently does not invoke .squeeze() either...

The idea is that if `usemask` is True, you build a second array (the mask), that you attach to your main array at the very end (in the `output=output.view(MaskedArray), output._mask = mask` combo...). Afterwards, it's a regular MaskedArray that supports the .squeeze() method...

> On a  
> possibly related note, genfromtxt also treats the 'unpack'ing of  
> structured arrays differently from loadtxt (which returns a list of  
> arrays in that case) - do you know if this is on purpose, or also  
> rather missing functionality (I guess it might break recfromtxt()...)?

Keep in mind that I haven't touched genfromtxt since 8-10 months or so. I wouldn't be surprised that it were lagging a bit behind loadtxt in terms of development. Yes, there'll be some tweaking to do for recfromtxt (it's OK for now if `ndmin` and `unpack` are the defaults) and others, but nothing major.

More information about the NumPy-Discussion mailing list