[Distutils] People want CPAN

Pauli Virtanen pav at iki.fi
Wed Nov 11 21:22:36 CET 2009

ke, 2009-11-11 kello 16:47 +0100, Tarek Ziadé kirjoitti:
> >> If it's the later, I guess you will be able to use the
> >> upcoming "sysconfig" module,
> >> that gives you the install schemes, depending on sys.prefix/sys.exec_prefix.
> >
> > Where is the sysconfig sources ? I don't see it in bitbucket.
> That's in python's svn, in a tarek_sysconfig branch. It's a revamp of
> distutils/sysconfig.py with the schemes from distutils/command/install.py
> (work in progress)

What if the user passes a different install prefix via

	python setup.py install --prefix=FOO

I believe David would like to know FOO here. Since sysconf is not a part
of distutils, will it know what FOO is?

> > For me, one of the core idea of an improved distutils would be to make
> > this much easier. All compilers options form distutils would be in
> > simple data files with simple API, no objects, no class with countless
> > methods and complex protocol. Distutils v2 would have a default "dumb"
> > build tool, but you could use whatever tool instead if desired.
> The default compiler class exists for that, it's CCompiler, and is
> mostly a placeholder for options.
> But it's C oriented.  In my mind, implementing a new compiler for
> Distutils means overriding it, and implementing, mainly:
> - preprocess()
> - compile()
> - create_static_lib()
> - link()
> Now that's quite complex, and we could probably have a single method
> (compile) that would do the required work of compiling an extension
> the way it wants to.

I think one question here is that how do the different compilers speak
to each other?

Consider the following chain needed to compile a Cython file linking to
some Fortran and C files to a Python module:

- cython foo.pyx -> foo.c
- compile foo.c -> foo.o
- compile bar.c -> bar.o
- compile quux.f90 -> quux.o
- link foo.o bar.o quux.o -> foo.so

This is a completely possible use-case, so it would be good if distutils
could handle it. Also, dependency handling would be nice here. Changing
bar.c or foo.pyx should trigger relinking foo.so etc.

> So, yes, being able to register an arbitrary compiler class, with
> arbitrary options passed through the Extension
> could make it simpler:
> setup(
> ..
> ext_modules=[Extension('Foo', files=['foo.d'], compiler='pyd')],
> ..)

Does this work easily out when there are multiple source files (in
different languages) for a single extension module?

Also, the option of determining the necessary compiler and compiler
options from file extensions does not always work. For example, Fortran
90 files come in two flavors, fixed-form and free-form, which often are
not differentiated by the file extension. However, they may require
different compiler flags to compile. (Hopefully, however, nobody uses
the fixed-form for F90 any more...)

Pauli Virtanen

More information about the Distutils-SIG mailing list