[Tutor] Help for Python Beginner with extracting and manipulating data from thousands of ASCII files
alan.gauld at btinternet.com
Mon Oct 1 01:33:44 CEST 2012
On 30/09/12 23:07, Cecilia Chavana-Bryant wrote:
> Hola again Python Tutor!
> With a friend's help I have the following code to extract reflectance
> data from an ASCII data file, do a bit of data manipulation to calibrate
> the data and then write the calibrated file into an out file.
> I have successfully calibrated one ASCII file at a time with this code.
> However, I have 1,000s of files that I need to calibrate so I would like
> some help to modify this code so it can:
> 1. Use one calibration file (Cal_FileP17.txt) on data files created from
> July to the 18th Sep and a different calibration file (Cal_FileP19.txt)
> for data files created from the 19th of Sep onwards.
> 2. Find all the .txt files in a folder called ASCII_files, which is
> subdivided into 12 different folders and calibrate all these files
Number 2 is easier to solve and the os.walk() and glob.glob()
functions should provide all the tools you need.
Number 1 is more tricky since there is no obvious way to determine the
arbitrary start/stop dates you specify. So I'd suggest you need to
generalise the requirement to take a start/stop date as well as the
calibration file name and the input data file pattern. Use those as
input parameters to a function that generates the list of files to
process and then calls your existing code (wrapped in a new function)
and possibly provide default values for all/some of the parameters.
Another option is to add the start/end dates to the calibration file if
you have control of that, but personally I'd stick with input parameters...
Author of the Learn to Program web site
More information about the Tutor