[Python-Dev] Bug in build system for cross-platform builds
vadmium+py at gmail.com
Fri Mar 11 19:48:07 EST 2016
On 11 March 2016 at 23:16, Russell Keith-Magee <russell at keith-magee.com> wrote:
> On Sat, Mar 12, 2016 at 6:38 AM, Martin Panter <vadmium+py at gmail.com> wrote:
>> make clean # Work around confusion with existing in-source build
>> mkdir native
>> (cd native/ && ../configure)
>> make -C native/ Parser/pgen
>> mkdir mingw
>> (cd mingw/ && ../configure --host=i486-mingw32 --build=x86)
>> make -C mingw/ PGEN=../native/Parser/pgen
>> Actually it was not as smooth as the above commands, because pgen
>> tends to get overwritten with a cross-compiled version. Perhaps we
>> could add a PGEN_FOR_BUILD override, like HOSTPGEN in the patch used
> That might fix the pgen problem, but _freeze_importlib still remains. I
> suppose the same thing might be possible for _freeze_importlib as well…
Yes. I never got up to it failing in my experiments, but I think I
would propose a FREEZE_IMPORTLIB override variable for that too.
>> Would it be more correct to say instead that in 3.4 you did a separate
>> native build step, precompiling pgen and _freeze_importlib for the
>> native build host? Then you hoped that the child Make was _not_
>> invoked in the cross-compilation stage and your precompiled
>> executables would not be rebuilt?
> Yes - as far as I can make out (with my admittedly hazy understanding), that
> appears to be what is going on. Although it’s not that I “hoped” the build
> wouldn’t happen on the second pass - it was the behavior that was previously
> relied, and on was altered.
Do you have a copy/patch/link/etc to the actual commands that you
relied on? It’s hard to guess exactly what you were doing that broke
without this information.
>> > As best as I can work out, the solution is to:
>> > (1) Include the parser generator and _freeze_importlib as part of the
>> > artefacts of local platform. That way, you could use the version of pgen
>> > and
>> > _freeze_importlib that was compiled as part of the local platform build.
>> > At
>> > present, pgen and _freeze_importlib are used during the build process,
>> > but
>> > aren’t preserved at the end of the build; or
>> I don’t understand. After I run Make, it looks like I get working
>> executables leftover at Programs/_freeze_importlib and Parser/pgen. Do
>> you mean to install these programs with “make install” or something?
> Making them part of the installable artefacts would be one option, but they
> don’t have to be installed, just preserved.
What commands are you running that cause them to not be preserved at
the end of the build?
> For example, as a nasty hack, I’ve been able to use this approach to get the
> build working for 3.5. After the native build, I copy _freeze_importlib to a
> “safe” location. I then copy it back into place prior to the target build.
> It works, but it’s in no way suitable for a final build.
>> > (2) Include some concept of the “local compiler” in the build process,
>> > which
>> > can be used to compile pgen and _freeze_importlib; or
>> On the surface solution (2) sounds like the ideal fix. But I guess the
>> local native compiler might also require a separate set of CPPFLAGS,
>> pyconfig.h settings etc. In other words it is sounding like a whole
>> separate “configure” run. I am thinking it might be simplest to just
>> require this native “configure” run to be done manually.
> That run is going to happen anyway, since you have to compile and build for
> the native platform.
More information about the Python-Dev