creating pipelines in python

[Wolodja Wentland]

On Sun, Nov 22, 2009 at 14:49 -0800, per wrote:
> i am looking for a python package to make it easier to create a
> "pipeline" of scripts (all in python). what i do right now is have a
> set of scripts that produce certain files as output, and i simply have
> a "master" script that checks at each stage whether the output of the
> previous script exists, using functions from the os module. this has
> several flaws and i am sure someone has thought of nice abstractions
> for making these kind of wrappers easier to write.
> does anyone have any recommendations for python packages that can do
> this?

There are various possibilities. I would suggest you have a look at [1]
which details the creation of pipelines with generators that can be used
within *one* program. If you want to chain different programs together
you can use the subprocess package in the stdlib of Python 2.6.

[1] http://www.dabeaz.com/generators/
-- 
  .''`.     Wolodja Wentland    <wentland at cl.uni-heidelberg.de> 
 : :'  :    
 `. `'`     4096R/CAF14EFC 
   `-       081C B7CD FF04 2BA9 94EA  36B2 8B7F 7D30 CAF1 4EFC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20091123/35706a47/attachment-0001.sig>