[Distutils] Enterprise Python - Multicomponent Projects

Tres Seaver tseaver at palladion.com
Wed Apr 16 19:41:11 CEST 2014

Hash: SHA1

On 04/16/2014 12:57 PM, Jim Fulton wrote:
> On Wed, Apr 16, 2014 at 12:18 PM, Tin Tvrtković <tinchester at gmail.com>
> wrote:
>> Hello packaging community,
>> I'm investigating ways of setting up Python projects at my
>> workplace. We're predominantly a Java shop, but we might be dipping
>> our toes in Python waters (finally!) due to a fortuitous project and
>> my multi-year insistence, so I'm contemplating how to set up our
>> Python build system to minimize workflow differences for other
>> developers (well, and myself).
>> I've actually written uš a lengthy description of Maven and why we
>> use it but I'll spare you for now. :) To keep the story short, I'm
>> interested in options for setting up a multi-module Python project.
>> By 'multi-module' I don't mean a single setuptools-based project
>> with several .py files inside, but a way of triggering a complex
>> build process with a single command that would build all sub-modules
>> (essentially sub-projects) and produce a number of end artifacts -
>> just like Maven. Imagine a repository containing 30 separate Django
>> apps, packaged independently, 10 utility libraries, 10 Django
>> projects combining those app, and 10 RPM building projects for 
>> packaging it all up for deployment.
>> As far as I know, just using setuptools isn't adequate for a
>> workflow like this - setuptools deals with the build process
>> (testing, packaging, etc) of a single project only. Solutions that
>> come to mind are: a hierarchy of Makefiles, shell scripts, or maybe
>> Twitter's Pants, which sort of looks like Maven for Python but would
>> probably need contributions to do what we want, and looks
>> predisposed to building PEX files which, while very interesting, I'm
>> not looking to do right now. None of these solutions are really
>> ideal, especially if I want to support development on Windows (which
>> I absolutely want).
>> I've even thought about actually using Maven, but that's just a
>> Pandora's box of problems waiting to happen.
>> I'd appreciate insight on this from anyone who's thought about (and
>> maybe solved) problems like this. I'm also willing to engage and
>> contribute to improving the situation, especially if there's low
>> hanging fruit to be picked. How do other companies handle large
>> Python repositories with a lot of subcomponents?
> Checkout Buildout, http://www.buildout.org
> It addresses a lot of the same problems.  Of course, for Python,
> compiling is (mostly) not all that important.  For us (Zope
> Corporation) buildout is more about assembly/deployment, both for
> development and production, that building.

buildout also allow for reproducible builds of things that *do* require
compilation (e.g., via the 'zc.reciple.cmmi' recipe for
'configure-make-make install' software).

- -- 
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/


More information about the Distutils-SIG mailing list