Hi,
I spent some time trying to cross-compile NumPy for Linux ARM64 on a
macOS host as part of an OpenWRT compilation [1]. It was not so bad but
certainly not trivial and I was wondering if we could use my experience
to help streamline the process.
Thanks, that would be great!
We have quite some issues with cross-compilation, and usually they're hard to resolve (plus it's not something we officially support - but there's a clear need for improvement). Note that conda is also a cross-compiler (to the same system), sometimes worth looking into how it deals with (numpy.)distutils issues.
I encountered two challenges:
1. Python distutils did not allow me to provide a proper cross-ranlib
program path. I opened an issue for this on the Python bug tracker [2].
That seems like a straightforward enough patch. distutils isn't well-maintained but hopefully that gets merged soon. If it doesn't get responses in a couple of weeks, please feel free to ping me and I can review/test (I'm not a core Python dev, but it may help push it forwards).
2. numpy distutils scripts check sys.platform in many places.
sys.platform returns the platform on which compilation occurs (darwin in
my case) which in case of cross-compilation is not the one on which the
code will be run (linux). This of course results in invalid compiler and
linker flags.
I made a quick'n'dirty fix [3] and got numpy to compile and run
correctly (but I have yet to run the full test suite). Of course this is
not a proper way to solve this so I was hoping to get some feedback on
the preferred/best way. I did not find any info about an official way to
do this with distutils.
One idea (from @commodo on GitHub) would be to just change my hardcoded
string ("Linux") to something like this:
target_platform = os.getenviron(“TARGET_PLATFORM”, sys.platform)
That seems reasonable enough. Prefixing that name with NPY_ would be good, to not get accidental clashes.
I'm not sure if there's a more standard/preferred way, I haven't come across it in distutils. There's some issues with reasonably detailed discussion for SciPy - this one may be useful:
https://github.com/scipy/scipy/issues/8571
It could also be further refactored by moving it to a single place but
while certainly more elegant and "proper" it may ultimately make it less
readable and harder to follow. I am also unsure where would be the best
place to put it and how to pass it around to other modules.
Keeping all platform comparisons where they are now seems right. Just need one central place to parse NPY_TARGET_PLATFORM - perhaps in numpy/distutils/misc_util.py
Cheers,
Ralf
[1]: https://github.com/openwrt/packages/pull/9797
[2]: https://bugs.python.org/issue37916
[3]:
https://github.com/openwrt/packages/pull/9797/commits/e701e122fb9c78d5cecffb88c1d750178a239083#diff-1ea8d839590bcdc406854d3998a96704
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion