[Tutor] Writing elements of an array to a file using CSV module

[Evert Rol]

> Thanks for the reply. I've tried your suggestion and am still getting
> an error (posted below code). Since my posting, I've also added the
> code to convert the numpy array to a list. My complete script is:
> 
> -----
> # -*- coding: utf-8 -*-
> 
> import pupynere
> import os
> import csv
> import glob
> 
> def get_file_details(filename):
>    sub1='3B42.'
>    sub2='.nc'
>    extract=filename.split(sub1)[-1].split(sub2)[0]
>    elements=extract.split('.')
>    ure={'0':'00:00:00',
>         '3':'03:00:00',
>         '6':'06:00:00',
>         '9':'09:00:00',
>         '12':'12:00:00',
>         '15':'15:00:00',
>         '18':'18:00:00',
>         '21':'21:00:00'}
>    date='20'+elements[0]
>    time=ure[elements[1]]
>    return[date,time]
> 
> def get_rain_data(filename):
>    # trmm blocks covering study area
>    rows=[87,86,85,86]
>    cols=[833,833,833,834]
> 
>    # open file
>    file = pupynere.netcdf_file(filename,'r')
>    # read data
>    precipitation=file.variables['precipitation']
>    # close file    file.close()
> 
>    # read values for blocks of interest
>    precip=precipitation[rows,cols]
>    # convert numpy array to list
>    precip=precip.tolist()
>    return[precip]

You just converted precip to a list (precip.tolist()), then, in the next line, you're turning that into a list.
So you get a list within a list, which results in the errors below. Exactly as you hinted at yourself below.

(In fact, if you now would do
>>> data[0]
[0.0, 0.0, 0.0, 0.0]
because this is the representation (repr). Not the 0.0 you previously showed.)

Once you get rid of the extra list, you can use either float(data[0]) and %f (if you require detailed formatting) or just simply %s, as Alan suggested.

  Evert



> 
> 
> # get list of .nc files in folder
> os.chdir('F:\\Hanlie\\UCT\\M.Sc\\Data\\TRMM\\2000\\02_Februarie')
> filepattern='*.nc'
> nc_files=glob.glob(filepattern)
> 
> # write to CSV-file with spaces as delimiter
> csv.register_dialect('spaces', delimiter=' ')
> outfilename="trmm_c83a_feb2000.txt"
> out_text=open(outfilename, 'wb')
> writer = csv.writer(out_text,dialect='spaces')
> 
> for file in nc_files:
>    file_details=get_file_details(file)
>    data=get_rain_data(file)
>    writer.writerow(('%s' % (file_details[0]),'%s' %
> (file_details[1]),'%f' % float(data[0])))
> 
> out_text.close()
> -----
> 
> I now get the error:
> -----
> Traceback (most recent call last):
>  File "F:\Hanlie\UCT\M.Sc\Data\TRMM\lees_netcdf_per_trmm_blok_skryf_vir_pcraster.py",
> line 76, in <module>
>    writer.writerow(('%s' % (file_details[0]),'%s' %
> (file_details[1]),'%f' % float(data[0])))
> TypeError: float() argument must be a string or a number
> -----
> 
> I tried the other suggestion on the list (posted by Alan Gould),
> namely to replace the %f formatting string with %s in the
> write.writerow statement:
> -----
> writer.writerow( ( file_details[0], file_details[1], str(data[0])  ) )
> -----
> 
> This produces a file with lines such as:
> -----
> 20000201 00:00:00 "[0.0, 0.0, 0.0, 0.0]"
> -----
> 
> I seem to have a list within a list, I'm not sure why. I could write
> the list elements to variables before I write to the file, but it
> doesn't seem like a very elegant solution.
> 
> Can anyone see why I have such a nested data structure? Or is the
> problem something completely different?
> 
> Thanks
> Hanlie
> 
> 
> 2010/10/14, Evert Rol <evert.rol at gmail.com>:
>>> I have a numpy array ('data') that is the result of reading a netCDF
>>> file, and it typically looks like this:
>>> 
>>> array([ 0.,  0.,  0.,  0.], dtype=float32)
>>> 
>>> 
>>> I want to write this, after a date and time, to a CSV file, so the CSV
>>> file would have the entry:
>>> 
>>>   2000-02-01,09:00,0.0,0.0,0.0,0.0
>>> 
>>> 
>>> The code I'm using to test it, is:
>>> 
>>> # file_details[0] is the date and file_details[1] is the time
>>> writer.writerow(('%s' % (file_details[0]),'%s' %
>>> (file_details[1]),'%f' % (data[0])))
>>> 
>>> 
>>> In my opinion, this should give an output line:
>>> 
>>> 2000-02-01,09:00,0.0
>>> 
>>> 
>>> Instead, I get an error
>>> 
>>> TypeError: float argument required, not numpy.ndarray
>>> 
>>> 
>>> 
>>> Can someone please explain to me why data[0] is giving a numpy array
>>> rather than one element? I don't understand it, since I get:
>>> 
>>>>>> data[0]
>>> 0.0
>> 
>> This means very little, since a string '0.0' will show the same output when
>> printed:
>>>>> print '0.0'
>> 0.0
>> 
>> The exception says 'TypeError', so try:
>> 
>>>>> type(data[0])
>> <type 'numpy.float32'>
>> 
>> Which is indeed what you specified at the start using dtype=float32.
>> 
>> The standard string formatting doesn't know what to do with a numpy.float32,
>> so you'll need to convert it to a float:
>> 
>>>>> '%f' % float(data[0])
>> '0.000000'
>> 
>> (use proper float formatting to get at '0.0', which is what you apparently
>> want.)
>> 
>> 
>>  Evert
>> 
>>