porting shell scripts: system(list), system_pipe(lists)

[Donald 'Paddy' McCarthy]

eichin at metacarta.com wrote:
> One of my recent projects has involved taking an accretion of sh and
> perl scripts and "doing them right" - making them modular, improving
> the error reporting, making it easier to add even more features to
> them.  "Of course," I'm redoing them in python - much of the cut&paste
> reuse has become common functions, which then get made more robust and
> have a common style and are callable from other (python) tools
> directly, instead of having to exec scripts to get at them.  The usual
> "glorious refactoring."
> 
<<SNIP>>
> 
> Implementing pipelines takes rather a bit more work, and one might
> (not unreasonably) throw up one's hands and just use os.system and
> some re.sub's to do the quoting.  However, I had enough cases where
> the goal really was to run a complex shell pipeline (I also had cases
> where the pipeline converted nicely to some inline python code,
> especially with the help of the gzip module) that I sat down and
> cooked up a pipeline class.
> 
> The interface I ended up with is pretty simple:
>         g_pipe = pipeline()
>         g_pipe.stdin(open("blort.gz", "r"))
>         g_pipe.append(["gunzip"])
>         g_pipe.append(["sort", "-u"])
>         g_pipe.append(["wc", "-l"])
>         g_pipe.stdout(open("blort.count", "w"))
>         print g_pipe.run()
> 
> is equivalent to the sh:
>     gunzip < blort.gz | sort -u | wc -l > blort.count
> 
<<SNIP>>
> 
> 			_Mark_ <eichin at metacarta.com>
> 
> [1] in the Invader Zim sense :)

I think that your pipeline code looks nothing like the original sh 
script pipeline which to me counts heavily against it.
Just playing at the cygwin prompt...
   $ ls -l|wc -l > /tmp/lines_in_dir
   $ cat /tmp/lines_in_dir
       465
   $ python
   >>> from os import system
   >>> system(r'''/bin/ls -l|/bin/wc -l > /tmp/lines_in_dir2''')
   0
   >>> system(r'''/bin/cat /tmp/lines_in_dir2''')
       463
   0

I prefer the above because it looks like the original sh command.
Of course, if script security is very important then you may want to 
change the way things are implemented again.

Cheers, Paddy.