On 07.12.2011, at 5:07AM, Olivier Delalleau wrote:
I *think* it may work better if you replace the last 3 lines in your loop by:
a=all_TSFC[0] if len(all_TSFC) > 1: N.maximum(a, TSFC, out=a)
Not 100% sure that would work though, as I'm not entirely confident I understand your code.
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com> Something fancier I think, I am able to compare the result with my previous method so I can easily see I am doing something wrong. see code below:
all_TSFC=[] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() all_TSFC.append(TSFC) a=TSFC[0] for b in TSFC[1:]: N.maximum(a,b,out=a)
I also understood TSFC is already the array you want to work on, so above you'd just take a slice and overwrite the result in the next file iteration anyway. Iterating over the list all_TSFC should be correct, but I understood you don't want to load the entire input into memory in you working code. Then you can simply skip the list, just need to take care of initial conditions - something like the following should do: path=path+'/' a = None for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() if not is instance(a,N.ndarray): a=TSFC else: N.maximum(a, TSFC, out=a) HTH, Derek
big_array=N.ma.concatenate(all_TSFC) Max=big_array.max(axis=0) print "max is", Max,"a is", a