Hello, I've noted that distutils manages depends in a way I cannot understand. Suppose I have a minimal setup.py: from distutils.core import setup, Extension setup( name='foo', version='1.0', ext_modules=[ Extension('foo', sources=['foo.c'], depends=['fop.conf'] # <---- note the typo foo->fop ), ] ) Now setup.py will rebuild all every time, this is because the policy of newer_group in build_extension is to consider 'newer' any missing file. http://bit.ly/build_ext_471 def build_extension(self, ext): ... depends = sources + ext.depends if not (self.force or newer_group(depends, ext_path, 'newer')): logger.debug("skipping '%s' extension (up-to-date)", ext.name) return else: logger.info("building '%s' extension", ext.name) ... Can someone suggest me the reason of this choice instead of missing='error' (at least for ext.depends)? Cheers, Matteo
Hi Matteo,
Now setup.py will rebuild all every time, this is because the policy of newer_group in build_extension is to consider 'newer' any missing file. Here you certainly mean “older”.
[...] Can someone suggest me the reason of this choice distutils’ notion of dependencies directly comes from make. A missing (not existing) target is perfectly normal: it’s usually a generated file that make needs to create (i.e. compile from source files). In this world, you want to (re-)compile when the target is older than the sources, or when the target is missing.
So here your extension module is a target that needs to be created, and when distutils does not find a file with the name you give in depends, it just thinks it’s another thing that will be generated. This model is inherently prone to typos; I’m not sure how we can improve it to let people catch possible typos. Cheers
On Fri, Feb 3, 2012 at 5:52 PM, Éric Araujo
Hi Matteo,
Now setup.py will rebuild all every time, this is because the policy of
newer_group in build_extension is to consider 'newer' any missing file.
Here you certainly mean “older”.
No, and this is the problem: newer_group(depends, ext_path, 'newer')) if (some dep is newer than the target): rebuild
[...] Can someone suggest me the reason of this choice
distutils’ notion of dependencies directly comes from make. A missing (not existing) target is perfectly normal: it’s usually a generated file that make needs to create (i.e. compile from source files). In this world, you want to (re-)compile when the target is older than the sources, or when the target is missing.
Here is a simple Makefile that has the behavior I was expecting from distutils too: $ cat Makefile all: missing.dep echo "Done!" $ make make: *** No rule to make target `missing.dep', needed by `all'. Stop. So here your extension module is a target that needs to be created, and
when distutils does not find a file with the name you give in depends, it just thinks it’s another thing that will be generated.
So, if I understand correctly, starting today a better name could be 'generates' instead of 'depends'? This model is inherently prone to typos; I’m not sure how we can improve
it to let people catch possible typos.
Yes, perhaps the name of the list and the explanation in the docs are both a bit confusing: http://docs.python.org/distutils/apiref.html#distutils.ccompiler.CCompiler.c... *depends*, if given, is a list of filenames that all targets depend on. If
a source file is older than any file in depends, then the source file will be recompiled.
Can this be a better explanation? "If a source file is older than any file in depends {+or if some depend is missing+}" Cheers -- Matteo Bertini http://www.slug.it/naufraghi
participants (2)
-
Matteo Bertini
-
Éric Araujo