[Tutor] print date and skip any repeats

[questions anon]

Ok, I am continuing to get stuck. I think I was asking the wrong question
so I have posted the entire script (see below).
What I really want to do is find the daily maximum for a dataset (e.g.
Temperature) that is contained in monthly netcdf files where the data are
separated by hour.
The major steps are:
open monthly netcdf files and use the timestamp to extract the hourly data
for the first day.
Append the data for each hour of that day to a list, concatenate, find max
and plot
then loop through and do the following day.

I can do some of the steps separately but I run into trouble in working out
how to loop through each hour and separate the data into each day and then
repeat all the steps for the following day.

Any feedback will be greatly appreciated!



oneday=[]
all_the_dates=[]
onedateperday=[]


#open folders and open relevant netcdf files that are monthly files
containing hourly data across a region
for (path, dirs, files) in os.walk(MainFolder):
        for ncfile in files:
            if ncfile.endswith(ncvariablename+'.nc'):
                print "dealing with ncfiles:", path+ncfile
                ncfile=os.path.join(path,ncfile)
                ncfile=Dataset(ncfile, 'r+', 'NETCDF4')
                variable=ncfile.variables[ncvariablename][:,:,:]
                TIME=ncfile.variables['time'][:]
                ncfile.close()

                #combine variable and time so I can make calculations based
on hours or days
                for v, time in zip((variable[:,:,:]),(TIME[:])):

                    cdftime=utime('seconds since 1970-01-01 00:00:00')
                    ncfiletime=cdftime.num2date(time)
                    timestr=str(ncfiletime)
                    utc_dt = dt.strptime(timestr, '%Y-%m-%d %H:%M:%S')
                    au_tz = pytz.timezone('Australia/Sydney')
                    local_dt =
utc_dt.replace(tzinfo=pytz.utc).astimezone(au_tz)
                    strp_local=local_dt.strftime('%Y-%m-%d_%H') #strips
time down to date and hour
                    local_date=local_dt.strftime('%Y-%m-%d') #strips time
down to just date

                    all_the_dates.append(local_date)

#make a list that strips down the dates to only have one date per day
(rather than one for each hour)
onedateperday = sorted ( list (set (all_the_dates)))

#loop through each day and combine data (v) where the hours occur on the
same day
for days in onedateperday:
    if strp_local.startswith(days):
        oneday.append(v)

big_array=np.ma.dstack(oneday) #concatenate data
v_DailyMax=big_array.max(axis=2) # find max

#then go on to plot v_DailyMax for each day
map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33,
        llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i')
map.drawcoastlines()
map.drawstates()
map.readshapefile(shapefile1, 'REGIONS')
x,y=map(*np.meshgrid(LON,LAT))
plottitle=ncvariablename+'v_DailyMax'+days
cmap=plt.cm.jet
CS = map.contourf(x,y,v_DailyMax, 15, cmap=cmap)
l,b,w,h =0.1,0.1,0.8,0.8
cax = plt.axes([l+w+0.025, b, 0.025, h])
plt.colorbar(CS,cax=cax, drawedges=True)
plt.savefig((os.path.join(OutputFolder, plottitle+'.png')))
plt.show()
plt.close()


On Wed, Sep 24, 2014 at 8:27 AM, questions anon <questions.anon at gmail.com>
wrote:

> brilliant, thank you!
>
> On Tue, Sep 23, 2014 at 5:05 PM, Dave Angel <davea at davea.name> wrote:
>
>>
>> Please post in text mode, not html.
>>
>> questions anon <questions.anon at gmail.com> Wrote in message:
>> >
>> > lastdate=all_the_dates[1]
>> > onedateperday.append(lastdate)
>> > print onedateperday
>> > for date in all_the_dates:
>> >      if date !=lastdate:
>> >           lastdate=date
>> >          onedateperday.append(lastdate)
>>
>> There are a number of things you don't explicitly specify.  But if
>>  you require the dates in the output list to be in the same order
>>  as the original list, you have a bug in using element [1]. You
>>  should be using element zero, or more simply:
>>
>> lastdate=None
>> for date in all_the_dates:
>>     if date !=lastdate:
>>         onedateperday.append(date)
>>         lastdate=date
>>
>> But if you specified a bit more, even simpler answers are possible.
>>
>> For example,  if output order doesn't matter, try:
>>      onedateperday = list (set (all_the_dates))
>>
>> If output order matters, but the desired order is sorted,
>>     onedateperday = sorted ( list (set (all_the_dates)))
>>
>> Other approaches are possible using library functions,  but this
>>  should be enough.
>>
>>
>>
>> --
>> DaveA
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20140924/b6ef8ec9/attachment.html>