[Python-Dev] Bug in build system for cross-platform builds

[Martin Panter]

On 14 March 2016 at 13:26, R. David Murray <rdmurray at bitdance.com> wrote:
> On Mon, 14 Mar 2016 03:04:08 -0000, "Gregory P. Smith" <greg at krypto.org> wrote:
>> On Sun, Mar 13, 2016 at 7:41 PM Martin Panter <vadmium+py at gmail.com> wrote:
>> > Include/graminit.h
>> > Python/graminit.c
>> > Python/importlib.h
>> > Python/importlib_external.h
>> >
>> > A question for other Python developers: Why are these generated files
>> > stored in the repository? [. . .]
>>
>> They should not be regenerated every build, if they are, that seems like a
>> bug in the makefile to me (or else the timestamp checks that make does vs
>> how your code checkout happened).

The reason the current Python 3 build regenerates some files, is
because of the makefile prerequisites. For example, Include/graminit.h
currently depends on Parser/pgen, which needs to be compiled for the
native build host.

>> Having them checked in is convenient for
>> cross builds as it is one less thing that needs a build-host-arch build.
>
> [. . .]
> And yes, checking in these platform-independent artifacts is very
> intentional: less to build, fewer external dependencies in the build
> process...you don't need to *have* python to *build* python, which you
> would have to if they were not checked in.

Okay so it sounds like the generated files (more listed in .hgtouch)
have to stay. Reasons given:

* Some need Python to generate them (bootstrap problem)
* Relied on by Windows build system
* General convenience (less build steps, less prerequisites, less
things to go wrong)

One more idea I am considering is to decouple the makefile rules from
the main build. So to update the generated files you would have to run
a separate command like “make graminit” or “make frozen”. The normal
build would never regenerate them; although perhaps it could still
result in an error or warning if they appear out of date.