[SciPy-user] flagging no data in timeseries
Tim Michelsen
timmichelsen at gmx-topmail.de
Sun Jan 13 18:44:20 EST 2008
Hello again,
>>>>>> myvalues_ts_hourly = masked_where(myvalues_ts_hourly , -999)
>
>> All I got as output was:
>> > --
>>
>> What does that mean?
>
> That it's masked. The '--' is the defulat way to display masked values. Check
> the mask directly w/
> myvalues_ts_hourly.mask
I did as you suggested. Now, all values are flagged/masked.
Please give me a comment on my code below.
I have also created a small sample data set further down this message. I
would highly appreciate if you could give me a code example on how to
read this data in with all -999 values masked as NoData and then create
a timeseries object with it.
Many thanks in advance.
Kind regards,
Timmie
*** START CODE ***
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import numpy as N
import maskedarray as MA
import datetime
import timeseries as TS
from timeseries.lib.moving_funcs import mov_average_expw
from timeseries import plotlib as TPL
import pylab
# project settings contains file names etc.
import project_settings
### some functions
## plot functions
def plot_series(series):
"""
do a simple plot on a time series
input: a time series object
"""
fig = TPL.tsfigure()
fsp = fig.add_tsplot(111)
fsp.tsplot(series, '-') # comment for moving average only
fsp.format_dateaxis()
dates = series.dates
quarter_starts = dates[dates.quarter != (dates-1).quarter]
fsp.set_xticks(quarter_starts.tovalue())
fsp.grid()
fsp.tsplot(series, '-', mov_average_expw(series, 40), 'r--')
# comment for moving average only
# fsp.tsplot(mov_average_expw(series, 40), 'r--') # uncomment for
moving average only
fsp.set_xlim(int(series.start_date), int(series.end_date))
pylab.show()
pylab.savefig('myvalues.png')
# pylab.savefig('myvalues_'+location+'.png')
def csv_report(series, csv_file_name):
"""
write a CSV report for a series
input timeseries object
output file name
"""
csv_file = open(csv_file_name, 'w')
strfmt = lambda x: '"'+str(x)+'"'
#fmtfunc = [None, None, strfmt]
# csvReport = TS.Report(series, fmtfunc=fmtfunc, mask_rep='#N/A',
delim=',', fixed_width=False)
csvReport = TS.Report(series, fmtfunc=None, delim=',',
fixed_width=False)
csvReport() # output to sys.stdout
csvReport(output=csv_file) # output to file
### program code
# load data into array
data = N.loadtxt(project_settings.datafile, comments='#',
delimiter='\t', converters=None, skiprows=2, usecols=None, unpack=False)
# define start date
D_hr_start = TS.Date(freq='HR', year=2001, month=5, day=10, hour=0)
# subscript only desired data column
desired_values = data[:,5]
# create timeseries object
desired_values_ts_hourly = TS.time_series(desired_values,
start_date=D_hr_start)
# mask NoData values (-999)
desired_values_ts_hourly_masked = MA.masked_where( -999,
desired_values_ts_hourly)
# show masked values
print desired_values_ts_hourly_masked.mask
### prepare simple reports
# timeseries to use for report
# report_series = desired_values_ts_hourly
report_series = desired_values_ts_hourly_masked
# output
csv_report(report_series, project_settings.csvfile)
*** END CODE ***
*** START SAMPLE DATA ***
date hour_of_day value
01.02.2004 1 247
01.02.2004 2 889
01.02.2004 3 914
01.02.2004 4 292
01.02.2004 5 183
01.02.2004 6 251
01.02.2004 7 953
01.02.2004 8 156
01.02.2004 9 991
01.02.2004 10 557
01.02.2004 11 581
01.02.2004 12 354
01.02.2004 13 485
01.02.2004 14 655
01.02.2004 15 -999
01.02.2004 16 -999
01.02.2004 17 -999
01.02.2004 18 744
01.02.2004 19 445
01.02.2004 20 374
01.02.2004 21 168
01.02.2004 22 995
01.02.2004 23 943
01.02.2004 24 326
02.02.2004 1 83.98
02.02.2004 2 302.26
02.02.2004 3 310.76
02.02.2004 4 -999
02.02.2004 5 62.22
02.02.2004 6 85.34
02.02.2004 7 324.02
02.02.2004 8 53.04
02.02.2004 9 336.94
02.02.2004 10 189.38
02.02.2004 11 197.54
02.02.2004 12 120.36
02.02.2004 13 164.9
02.02.2004 14 222.7
02.02.2004 15 34.74
02.02.2004 16 85.34
02.02.2004 17 53.04
02.02.2004 18 252.96
02.02.2004 19 151.3
02.02.2004 20 -999
02.02.2004 21 57.12
02.02.2004 22 338.3
02.02.2004 23 320.62
02.02.2004 24 110.84
*** END SAMPLE DATA ***
--
Operating systems
Ubuntu
Linux 2.6.22-14-generic #1 SMP Tue Dec 18 08:02:57 UTC 2007 i686 GNU/Linux
***
more infos
ii grass 6.2.3-1lesejk1
Geographic Resources Analysis Support System
ii grass-doc 6.2.3-1lesejk1
Geographic Resources Analysis Support System
ii libgdal1-1.4.0-grass 1.4.1-1ubuntu1
GRASS extension for the Geospatial Data Abst
rc libgrass 6.0.2-2.1ubuntu1
GRASS GIS development libraries
ii qgis-plugin-grass 0.9.0-1
Plugin for accessing GRASS data from QGIS
ii libqgis0 0.8.1-1
QGIS Geographic Information System - shared
ii libqgis1 0.9.0-1
QGIS Geographic Information System - shared
ii qgis 0.9.0-1
Geographic Information System (GIS)
ii qgis-plugin-grass 0.9.0-1
Plugin for accessing GRASS data from QGIS
ii lyx 1.5.1-2ubuntu1
Document Processor
ii lyx-common 1.5.1-2ubuntu1
Architecture-independent files for LyX
***
and as dualboot
Windows XP Home Edition
More information about the SciPy-User
mailing list