I've released v0.3 of pipe_controller (*), my experimental tool to simulate pipes in Python:

https://bitbucket.org/vasudevram/pipe_controller

(*) I had earlier been calling it PipeController, but that is the name of the main class in the package. From now on I'm referring to it as pipe_controller (the Python module name).

Changes in v0.3:

- you can now swap components of a pipe_controller pipe at run time, and it can separately save the results of running the pipe, before and after a component swap, to separate output files; the test program test_pipe_controller_04.py in the download package shows how to do this.

That is, you can do something like this (using UNIX syntax, though pipe_controller is in Python and works differently):

foo | bar | baz # with output going to file 1

then swap the positions of foo and baz, then run the pipe again:

baz | bar | foo # with output going to file 2

and so on - any number of times, all in the same program run.

This feature lets you experiment with, and validate, your pipeline logic, to make sure that it does what you intended, e.g. you can check the output both before and after swapping components of the pipe, to decide which order you really need - or which order is optimal - see next paragraph.

The feature can also be used to time the execution of two or more different versions of the pipeline (with vs. without swapping of components), to see which runs faster, for cases where changing the order of those components makes no difference to the output, e.g. if those two components are commutative, in the mathematical sense (like a + b = b + a).

Blog post about swapping pipe components at run time with pipe_controller:

http://jugad2.blogspot.in/2012/10/swapping-pipe-components-at-runtime.html

pipe_controller v0.3 can be downloaded here:

https://bitbucket.org/vasudevram/pipe_controller/downloads

(Click on the "Branches" tab on that page)

- Vasudev Ram www.dancingbison.com jugad2.blogspot.com twitter.com/vasudevram