On Fri, Apr 7, 2017 at 10:41 PM, Nick Coghlan
On 7 April 2017 at 06:32, Nathaniel Smith
wrote: On Wed, Apr 5, 2017 at 2:37 AM, Nick Coghlan
wrote: And as it turns out, not only are we able to get bare metal machines to run our VMs on (rather than messing about with nested virt support), but the CentOS boxes are pre-cached on the local network, so they should be pretty quick to download (we haven't actually got our CI up and running yet, so I won't be able to vouch for that personally until we do).
I think bare metal access only matters for running x86 with hardware accelerated virtualization? If we want to emulate a totally different architecture like ppc64le then IIUC qemu does that as a regular user-space program. You definitely can run qemu in this mode on travis-ci, e.g. check out all the virtual architectures that rust builds on:
https://travis-ci.org/rust-lang/rust/
So I think travis-ci and centos-ci are equivalent on this axis – if we want to go the qemu route, then either works, and if we don't, then neither works?
My understanding of the problem is that Travis can't actually run the 64-bit CentOS VMs fast enough to get build jobs done in a reasonable time period (due to the resource constraints imposed on the test VMs), whereas the CentOS systems have more resources available to them (since they have the whole machine to themselves).
"Try it and see" would be the best way to confirm whether or not qemu-ppc64le-on-Travis works, though.
Ah, good point – Travis imposes a 1 hour time limit on builds, and for manylinux1 we're at about half that already with all the supported versions of Python... though some of that is compensating for centos 5 brokenness (building our own openssl) and some of it is Python versions we could probably drop at this point (2.6 at least, since there will never be a version of pip that supports both python 2.6 and manylinux2+!).
For me the big question is whether emulation is actually a good idea. When rust announced their plans I remember seeing some skepticism about whether one can really trust emulated machines for this kind of use case, though I can't find it again now... but it's definitely true that all the major distributions go to great lengths to use real hardware in their build farms, despite the obvious drawbacks (not just in terms of maintenance, but also the ongoing pain of using tiny little arm and mips machines that take dozens of hours to build things). They know a whole lot more about this than I do so I assume they have *some* reason :-).
It's one thing using emulation for a service being provided for free in a community project, something else entirely to be doing it for commercial or commercially-sponsored software.
Oh sure. I actually don't know which case this is -- does anyone care about ppc64le who isn't working with some big IBM support contract? We've had at least one @ibm address in one of these threads...
One particular problem with it is that any use of profile-guided optimisation would be optimising for the performance characteristics of the emulator, not those of the real hardware (that's also a problem for cross-compilation: as far as I am aware, you *can't* readily use profile guided optimisation in the cross-compilation case).
I thought PGO uses profiling to observe branches and variable values, which emulation shouldn't effect? Fortunately this part really doesn't matter much to us anyway though -- nothing in the docker image gets shipped to end-users. The important thing would be how the package distributors are running the image to do *their* builds, and that's not our problem. -n -- Nathaniel J. Smith -- https://vorpus.org