Build bugs in Python 2.2.1?
mwh at python.net
Mon Aug 19 18:32:12 CEST 2002
Hmm, I thought this deserved a proper reply, that I should take some
time writing, so of course I haven't got around to writing it yet.
Jonathan Hogg <jonathan at onegoodidea.com> writes:
> On 12/8/2002 14:14, in article lkvg6gmddn.fsf at pc150.maths.bris.ac.uk,
> "Michael Hudson" <mwh at python.net> wrote:
> > Please don't get discouraged from working on distutils or Python's
> > build process! But please also understand that there is a reason for
> > most of the extant stuff, and as Martin says, vague comments along the
> > lines of "it would be nice if ..." are not really helpful.
> If I have appeared vague before, I'm sorry. I haven't come here with a
> solution because I don't have one. I tried to demonstrate where and why I
> think there is a problem and suggest a possible idea for a solution.
Well, time has passed sufficiently that I can't remember your possible
idea for a solution. Was it to use autoconf for more or less
> In the interests of not seeming like a spoiled child (I was pretty tired
> last night when I gave up), I'll explain one more time. If someone doesn't
> understand or sees that I've missed something, please say so. I'm not trying
> to call-for-the-head-of distutils. It just doesn't work for me and here's
> One of my day jobs is managing multi-platform UNIX sites. I use a system
> called Arusha for doing so. This allows me to manage packages by noting what
> kind of package it is and any important differences or exceptions that
> should be taken into account.
> For most of the autoconf-style packages I install I can just specify
> something along the lines of:
> <prototype team="." name="GNU"/>
> and it works. It gets built from source on each platform, installed,
> deployed onto all of the systems, and revealed into the users' namespace.
> For packages that have dependencies, for instance on OpenSSL, I can usually
> add a simple addition to this specifying the dependency and adding the
> necessary argument to allow configure to enable/find it:
> <dependency type="normal" name="openssl" method="deploy"/>
> <param name="ssldir">!field.dependencyDeployDir('openssl')</param>
> <param name="configure_args">--with-ssl-dir=@param:ssldir@</param>
> This translates into an underlying shell script something along the lines
> CFLAGS=... LDFLAGS=... \
> ./configure --prefix=... --with-ssl-dir=/._ark-deploy/openssl--0.96e
> and that's all I need to do to make it work.
Hmm, it seems that the reason this works in Arusha is that Arusha has
support for these sort of things build in? I mean there doesn't seem
to be much inherently simpler about this, it's just what everyone else
> Unfortunately, life is not so simple with Python, because while having a
> configure front-end, the Python modules build process cannot be controlled
> with configure. I can't pass in the necessary CFLAGS and LDFLAGS to enable
> the SSL module to be built correctly either, because the flags are ignored
> by setup.py.
I do think setup.py should take more hints from the environment, but
I'm not sure how to do this reasonably.
Maybe it would be an idea to have *setup.py* take
--with-foo-dir=... style arguments? Though getting them past
distutils spectactularly baroque options handling stuff might be a bit
> To compile the SSL module I have to edit Modules/Setup. This sounds fine to
> most people, but the only way to do that in a repeatable way is to generate
> a patch. Since the patch file will vary depending on which version of
> OpenSSL I compile against (I have to be able to maintain multiple different
> versions) and possibly which platform I'm compiling for, the patch must be
> generated programmatically. The patch also varies depending on the version
> of Python so I have to maintain multiple patch templates.
Would it be any easier to patch setup.py instead?
> If someone can point out the blindingly obvious thing I've missed that will
> make all this horror go away, then please let me know.
> It's quite funny in a way because Arusha is written in Python, and that's
> because I sold it to the main guy behind Arusha as being a fantastic
> language. And yet, Python is the hardest language to build using it ;-)
Well, this is partly because Python by its nature has a large and
mostly optional set of dependencies, some of which have a tendency to
hide in obscure corners of the file system (yes, I mean *you*, bsddb).
I know for a fact that *I* have no interest in converting setup.py to
the bizarre mix of m4 macros and sh that is configure.in, and *even
less* in maintaining said code.
Would being able to pass module specific arguments to setup.py make
your life easier? You'd still have to fight the build process a bit,
but it might be less irritating than it currently is.
And to wedge a couple of comment in at the end because they didn't
seem to fit anywhere else:
1) you realize that Python's setup.py hasn't really made your life
harder than it was before?
2) I think that setup.py's main intent was to help the "average user",
and whatever that means, I doubt it covers what you're up to.
This is the fixed point problem again; since all some implementors
do is implement the compiler and libraries for compiler writing, the
language becomes good at writing compilers and not much else!
-- Brian Rogoff, comp.lang.functional
More information about the Python-list