creating pipelines in python

Paul Rudin paul.nospam at
Mon Nov 23 10:02:01 CET 2009

per <perfreem at> writes:

> hi all,
> i am looking for a python package to make it easier to create a
> "pipeline" of scripts (all in python). what i do right now is have a
> set of scripts that produce certain files as output, and i simply have
> a "master" script that checks at each stage whether the output of the
> previous script exists, using functions from the os module. this has
> several flaws and i am sure someone has thought of nice abstractions
> for making these kind of wrappers easier to write.
> does anyone have any recommendations for python packages that can do
> this?

Not entirely what you're looking for, but the subprocess module is
easier to work with for this sort of thing than os. See e.g. <>

More information about the Python-list mailing list