[Numpy-discussion] Reading footer lines

[Peter Cock]

On Tue, Aug 13, 2013 at 1:20 PM, Resmi <l.resmi at gmail.com> wrote:
> Hi,
>
> I've a list of long files of numerical data ending with footer lines
> (beginning with #). I am using numpy.loadtxt to read the numbers, and
> loadtxt ignores these footer lines. I want the numpy code to read one of the
> footer lines and extract words from it. Is there a way to use loadtxt for
> this? If there weren't many files I could have used the line number (which
> keep varying between the files) of the footer line along with linecache.
> Nevertheless there should be a generic way to do this in numpy?
>
> As a workaround, I've tried using os.system along with grep. And I get the
> following output :
>
>>>> os.system("grep -e 'tx' 'data.dat' ")
>  ## tx =    2023.06
> 0
>
> Why is there a 0 in the output? The file has no blank lines.

The os.system function call returns the integer return value
(error level) of the command invoked, by convention zero
for success but the value is set by the tool (here grep).

> Since I want the value 2023.06 in the numpy code for later use I tried to
> pipe the output to a variable:-
> test = os.system(command)
> But that isn't working as test is getting assigned the value 0.
> Tried subprocess.call(['grep','-e','tx','data.dat']) which is also ending up
> in the same fashion.
>
> It'll be great if I can get to know (i) a way to read the footer lines (ii)
> to sort out the operation of os.system and subprocess.call output
> processing.

Don't use os.system, instead you should use subprocess:
http://docs.python.org/2/library/subprocess.html

The standard library module commands would also work
but is not cross platform, and is also deprecated in favour
of subprocess: http://docs.python.org/2/library/commands.html

Peter