I've put combined code here:
https://gist.github.com/lyssdod/f51579ae8d93c8657a5564aefc2ffbca
Just download it, make executable and run.
amd64 Alpine:
# ./guess_pyruntime.py
Interpreter extracted: /lib/ld-musl-x86_64.so.1
Running on musl
amd64 Gentoo glibc:
# ./guess_pyruntime.py
Interpreter extracted: /lib64/ld-linux-x86-64.so.2
Running on glibc version 2.27
On Thu, Feb 21, 2019 at 3:57 AM Alexander Revin <lyssdod@gmail.com> wrote:
>
> Hi Nathaniel,
>
> Thanks for your answer.
>
> Basing on your example of RHEL and Ubuntu, let's take RHEL 6 which
> uses glibc 2.12. If you cross-compile for it (using the same gcc RHEL
> uses), wheel surely will work on Ubuntu 18.10 :)
> I think it's not of an issue, since wheels are built with these
> minimal runtime requirements anyway, unless they're built on a local
> machine – but in this case they will just work™; anyway, having a
> working toolchain is not the scope of Python tooling. Gentoo has an
> awesome crossdev tool for creating cross-toolchains, and there's
> crosstool-ng of course. It looks like there are workarounds for
> putting code built on newer systems to older ones though, but they
> seem to be pretty tedious ([1])
>
> Speaking of runtime detection, I see it this way for example – since
> one of the most reliable ways to check for program's dependencies is
> invoking something like ldd or objdump, it can essentially be done the
> same way:
>
> 1. Pick the minimal required code to extract ELF's ".interp" field [2]
> (I used code from [3]);
> 2. Process sys.executable with it;
>
> Here what it returns (grepping by "/lib" because it starts from a new line):
>
> Gentoo amd64 glibc
> # python3 readelf.py $(which python3) | grep "/lib"
> b'/lib64/ld-linux-x86-64.so.2\x00'
>
> Alpine amd64 docker (official python3 alpine image):
> # python3 readelf.py $(which python3) | grep "/lib"
> b'/lib/ld-musl-x86_64.so.1\x00'
>
> 3. Essentially that's enough in my opinion, but we can go further and
> do what ldd does:
>
> # /lib/ld-musl-x86_64.so.1 --list $(which python3)
> /lib/ld-musl-x86_64.so.1 (0x7fbc36567000)
> libpython3.7m.so.1.0 => /usr/local/lib/libpython3.7m.so.1.0 (0x7fbc3622a000)
> libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7fbc36567000)
>
> Basically it's just string matching here, and the only question now if
> the name of dynamic linker is enough or all libs should be iterated
> until perfect "musl" or "libc" match. Parsing "Dynamic section" turns
> out to be pretty useless – it's empty on Alpine (or parsing code is
> buggy). If ".interp" field is not available, then interpreter is
> statically linked :)
>
> 4. If any glibc-specific functionality is needed at this point, code
> from PEP 513 is really good. Maybe it's also better to put it first
> and use ELF parsing if it failed to open glibc in the first place.
>
>
> Thanks,
> Alex
>
>
>
> [1] https://snorfalorpagus.net/blog/2016/07/17/compiling-python-extensions-for-old-glibc-versions/
> [2] https://www.linuxjournal.com/article/1060
> [3] https://github.com/detailyang/readelf/blob/master/readelf/readelf.py#L545
>
> On Wed, Feb 20, 2019 at 1:50 AM Nathaniel Smith <njs@pobox.com> wrote:
> >
> > On Tue, Feb 19, 2019 at 3:28 PM Alexander Revin <lyssdod@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > I have an idea regarding Python binary wheels on non-glibc platforms,
> > > and it seems that initially I've posted it to the wrong list ([1])
> > >
> > > Long story short, the proposal is to use platform tuples (like
> > > compiler ones) for wheel names, which will allow much broader platform
> > > support, for example:
> > >
> > > package-1.0-cp36-cp36m-amd64_linux_gnu.whl
> > > package-1.0-cp36-cp36m-amd64_linux_musl.whl
> > >
> > > So eventually only {platform tag} part will be modified. Glibc/musl
> > > detection is quite trivial and eventually will be based on existing
> > > one in PEP 513 [2].
> >
> > The challenge here is: the purpose of a target triple is to tell a
> > compiler/linker toolchain which kind of code they should generate,
> > e.g. when cross-compiling. The purpose of a wheel tag is to tell you
> > whether a given wheel will work on a given system. It turns out these
> > are different things :-).
> >
> > For example, Ubuntu 18.10 and RHEL 6 are both 'amd64-linux-gnu',
> > because they use the same instruction set, the same binary format
> > (ELF), etc. But if you build a wheel on Ubuntu 18.10, it definitely
> > will not work on RHEL 6. (The other way around might work, if you do
> > other things right.)
> >
> > In practice Windows and macOS are already fine; the place where this
> > would be useful is Linux wheels for platforms that use non-Intel-based
> > architectures or non-glibc-libcs. We do have an idea for making it
> > easier to support newer glibcs and also extending to all
> > architectures: https://mail.python.org/archives/list/distutils-sig@python.org/thread/6AFS4HKX6PVAS76EQNI7JNTGZZRHQ6SQ/
> >
> > Adding musl is a bit trickier since I'm not sure what the details of
> > their ABI compatibility are, and they intentionally make it difficult
> > to figure out whether you're running on musl. But if someone could
> > convince them to publish more information then we could fix that too.
> >
> > -n
> >
> > --
> > Nathaniel J. Smith -- https://vorpus.org