[Python-Dev] Bug in build system for cross-platform builds

[Martin Panter]

On 15 March 2016 at 08:04, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 15 March 2016 at 15:15, Martin Panter <vadmium+py at gmail.com> wrote:
>> _freeze_importlib.o: _freeze_importlib.c Makefile
>>
>> _freeze_importlib: _freeze_importlib.o [. . .]
>>         $(LINKCC) [. . .]
>>
>> importlib_external.h: _bootstrap_external.py _freeze_importlib
>>         _freeze_importlib _bootstrap_external.py importlib_external.h
>>
>> importlib.h: _bootstrap.py _freeze_importlib
>>         _freeze_importlib _bootstrap.py importlib.h
>
> Ah, I understand now - the fundamental problem is with a checked in
> file depending on a non-checked-in file, so if you clean out all the
> native build artifacts when cross-compiling, the makefile will attempt
> to create target versions of all the helper utilities (pgen,
> _freeze_importlib, argument clinic, etc).
>
> Would it help to have a "make bootstrap" target that touched all the
> checked in generated files with build dependencies on non-checked-in
> files, but only after first touching the expected locations of the
> built binaries they depend on?

That sounds similar to “make touch”, with a couple differences. One
trouble I forsee is the conflict with shared prerequisites. E.g. “make
bootstrap” would have to create some dummy object files as
prerequisites of the pgen program, but we would first have build
others e.g. Parser/acceler.o properly for the main Python library. It
all feels way too complicated to me. The Python build system is
complicated enough as it is.

Maybe it is simplest to just add something in the spirit of Xavier’s
suggested patch. This would mean that we keep the generated files
checked in (to help with Windows and cross compiled builds), we keep
the current rules that force normal makefile builds to blindly
regenerate the files, but we add some flag or configure.ac check to
disable this regeneration if desired.