[Distutils] Enterprise Python - Multicomponent Projects
tinchester at gmail.com
Wed Apr 16 18:18:16 CEST 2014
Hello packaging community,
I'm investigating ways of setting up Python projects at my workplace.
We're predominantly a Java shop, but we might be dipping our toes in
Python waters (finally!) due to a fortuitous project and my multi-year
insistence, so I'm contemplating how to set up our Python build system
to minimize workflow differences for other developers (well, and myself).
I've actually written uš a lengthy description of Maven and why we use
it but I'll spare you for now. :) To keep the story short, I'm
interested in options for setting up a multi-module Python project. By
'multi-module' I don't mean a single setuptools-based project with
several .py files inside, but a way of triggering a complex build
process with a single command that would build all sub-modules
(essentially sub-projects) and produce a number of end artifacts - just
like Maven. Imagine a repository containing 30 separate Django apps,
packaged independently, 10 utility libraries, 10 Django projects
combining those app, and 10 RPM building projects for packaging it all
up for deployment.
As far as I know, just using setuptools isn't adequate for a workflow
like this - setuptools deals with the build process (testing, packaging,
etc) of a single project only. Solutions that come to mind are: a
hierarchy of Makefiles, shell scripts, or maybe Twitter's Pants, which
sort of looks like Maven for Python but would probably need
contributions to do what we want, and looks predisposed to building PEX
files which, while very interesting, I'm not looking to do right now.
None of these solutions are really ideal, especially if I want to
support development on Windows (which I absolutely want).
I've even thought about actually using Maven, but that's just a
Pandora's box of problems waiting to happen.
I'd appreciate insight on this from anyone who's thought about (and
maybe solved) problems like this. I'm also willing to engage and
contribute to improving the situation, especially if there's low hanging
fruit to be picked. How do other companies handle large Python
repositories with a lot of subcomponents?
More information about the Distutils-SIG