[ANN] pipefunc 0.32.1: Simplified function composition and pipeline creation for scientific workflows

Dear Python community, I am pleased to announce the release of pipefunc, a new open-source Python library designed to simplify function composition and pipeline creation for scientific computing, data processing, and machine learning workflows. BACKGROUND: In scientific computing and data science, we often deal with complex workflows involving multiple interdependent functions and large parameter spaces. pipefunc addresses these challenges by automatically generating Directed Acyclic Graphs (DAGs) from Python functions and efficiently handling N-dimensional parameter sweeps. KEY FEATURES: 1. Automatic DAG generation from Python functions 2. Efficient handling of N-dimensional parameter sweeps 3. Seamless scaling from local execution to HPC environments (including SLURM clusters) 4. Built-in pipeline visualization 5. Integrated resource usage profiling 6. N-dimensional map-reduce support 7. Automatic type checking through annotations 8. Minimal boilerplate code required VERSION: 0.32.1 PYTHON REQUIRED: 3.10+ LICENSE: MIT INSTALLATION: pip install pipefunc or conda install pipefunc For more information, please visit: Documentation: https://pipefunc.readthedocs.io/ Source Code: https://github.com/pipefunc/pipefunc PyPI: https://pypi.org/project/pipefunc/ We welcome feedback, contributions, and feature requests from the community. Please visit our GitHub repository for more information on how to get involved. Best regards, Bas Nijholt bas@nijho.lt
participants (1)
-
Bas Nijholt