[Distutils] CentOS5 is EOL, impact on manylinux1?

[Nick Coghlan]

On 7 April 2017 at 06:32, Nathaniel Smith <njs at pobox.com> wrote:
> On Wed, Apr 5, 2017 at 2:37 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> And as it turns out, not only are we able to get bare metal machines
>> to run our VMs on (rather than messing about with nested virt
>> support), but the CentOS boxes are pre-cached on the local network, so
>> they should be pretty quick to download (we haven't actually got our
>> CI up and running yet, so I won't be able to vouch for that personally
>> until we do).
>
> I think bare metal access only matters for running x86 with hardware
> accelerated virtualization? If we want to emulate a totally different
> architecture like ppc64le then IIUC qemu does that as a regular
> user-space program. You definitely can run qemu in this mode on
> travis-ci, e.g. check out all the virtual architectures that rust
> builds on:
>
>    https://travis-ci.org/rust-lang/rust/
>
> So I think travis-ci and centos-ci are equivalent on this axis – if we
> want to go the qemu route, then either works, and if we don't, then
> neither works?

My understanding of the problem is that Travis can't actually run the
64-bit CentOS VMs fast enough to get build jobs done in a reasonable
time period (due to the resource constraints imposed on the test VMs),
whereas the CentOS systems have more resources available to them
(since they have the whole machine to themselves).

"Try it and see" would be the best way to confirm whether or not
qemu-ppc64le-on-Travis works, though.

> For me the big question is whether emulation is actually a good idea.
> When rust announced their plans I remember seeing some skepticism
> about whether one can really trust emulated machines for this kind of
> use case, though I can't find it again now... but it's definitely true
> that all the major distributions go to great lengths to use real
> hardware in their build farms, despite the obvious drawbacks (not just
> in terms of maintenance, but also the ongoing pain of using tiny
> little arm and mips machines that take dozens of hours to build
> things). They know a whole lot more about this than I do so I assume
> they have *some* reason :-).

It's one thing using emulation for a service being provided for free
in a community project, something else entirely to be doing it for
commercial or commercially-sponsored software.

One particular problem with it is that any use of profile-guided
optimisation would be optimising for the performance characteristics
of the emulator, not those of the real hardware (that's also a problem
for cross-compilation: as far as I am aware, you *can't* readily use
profile guided optimisation in the cross-compilation case).

> It might be useful to get in touch with some of the distro's ppc64le
> wranglers to get their opinion, if anyone knows any of them...

FWIW, I believe some of the CentOS CI wranglers are also the Centos
build system wranglers :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia