[Distutils] Enterprise Python - Multicomponent Projects

[Brian Wickman]

Pants can definitely do what you want, but you're probably right in that it
requires significant up-front investment.  It's not strictly tied to PEX
files (it can recursively generate setup.py-based projects from a
transitive closure of BUILD dependencies) but the learning curve is
non-trivial and most attention is given to the Java/Scala backends.  Python
support will improve over time but it may be a year or more before this is
really easy to adopt.

~brian



On Wed, Apr 16, 2014 at 12:18 PM, Tin Tvrtković <tinchester at gmail.com>wrote:

> Hello packaging community,
>
> I'm investigating ways of setting up Python projects at my workplace.
> We're predominantly a Java shop, but we might be dipping our toes in Python
> waters (finally!) due to a fortuitous project and my multi-year insistence,
> so I'm contemplating how to set up our Python build system to minimize
> workflow differences for other developers (well, and myself).
>
> I've actually written uš a lengthy description of Maven and why we use it
> but I'll spare you for now. :) To keep the story short, I'm interested in
> options for setting up a multi-module Python project. By 'multi-module' I
> don't mean a single setuptools-based project with several .py files inside,
> but a way of triggering a complex build process with a single command that
> would build all sub-modules (essentially sub-projects) and produce a number
> of end artifacts - just like Maven. Imagine a repository containing 30
> separate Django apps, packaged independently, 10 utility libraries, 10
> Django projects combining those app, and 10 RPM building projects for
> packaging it all up for deployment.
>
> As far as I know, just using setuptools isn't adequate for a workflow like
> this - setuptools deals with the build process (testing, packaging, etc) of
> a single project only. Solutions that come to mind are: a hierarchy of
> Makefiles, shell scripts, or maybe Twitter's Pants, which sort of looks
> like Maven for Python but would probably need contributions to do what we
> want, and looks predisposed to building PEX files which, while very
> interesting, I'm not looking to do right now. None of these solutions are
> really ideal, especially if I want to support development on Windows (which
> I absolutely want).
>
> I've even thought about actually using Maven, but that's just a Pandora's
> box of problems waiting to happen.
>
> I'd appreciate insight on this from anyone who's thought about (and maybe
> solved) problems like this. I'm also willing to engage and contribute to
> improving the situation, especially if there's low hanging fruit to be
> picked. How do other companies handle large Python repositories with a lot
> of subcomponents?
>
> Kind regards,
> Tin
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140416/80095292/attachment-0001.html>