[Distutils] Enterprise Python - Multicomponent Projects

Jim Fulton jim at zope.com
Wed Apr 16 18:57:30 CEST 2014

On Wed, Apr 16, 2014 at 12:18 PM, Tin Tvrtković <tinchester at gmail.com> wrote:
> Hello packaging community,
> I'm investigating ways of setting up Python projects at my workplace. We're
> predominantly a Java shop, but we might be dipping our toes in Python waters
> (finally!) due to a fortuitous project and my multi-year insistence, so I'm
> contemplating how to set up our Python build system to minimize workflow
> differences for other developers (well, and myself).
> I've actually written uš a lengthy description of Maven and why we use it
> but I'll spare you for now. :) To keep the story short, I'm interested in
> options for setting up a multi-module Python project. By 'multi-module' I
> don't mean a single setuptools-based project with several .py files inside,
> but a way of triggering a complex build process with a single command that
> would build all sub-modules (essentially sub-projects) and produce a number
> of end artifacts - just like Maven. Imagine a repository containing 30
> separate Django apps, packaged independently, 10 utility libraries, 10
> Django projects combining those app, and 10 RPM building projects for
> packaging it all up for deployment.
> As far as I know, just using setuptools isn't adequate for a workflow like
> this - setuptools deals with the build process (testing, packaging, etc) of
> a single project only. Solutions that come to mind are: a hierarchy of
> Makefiles, shell scripts, or maybe Twitter's Pants, which sort of looks like
> Maven for Python but would probably need contributions to do what we want,
> and looks predisposed to building PEX files which, while very interesting,
> I'm not looking to do right now. None of these solutions are really ideal,
> especially if I want to support development on Windows (which I absolutely
> want).
> I've even thought about actually using Maven, but that's just a Pandora's
> box of problems waiting to happen.
> I'd appreciate insight on this from anyone who's thought about (and maybe
> solved) problems like this. I'm also willing to engage and contribute to
> improving the situation, especially if there's low hanging fruit to be
> picked. How do other companies handle large Python repositories with a lot
> of subcomponents?

Checkout Buildout, http://www.buildout.org

It addresses a lot of the same problems.  Of course, for Python, compiling is
(mostly) not all that important.  For us (Zope Corporation) buildout
is more about
assembly/deployment, both for development and production, that building.


Jim Fulton

More information about the Distutils-SIG mailing list