[Numpy-discussion] Proposal: split numpy.distutils into it's own package

[Dustin Spicuzza]

Hey all,

[copied from https://github.com/numpy/numpy/issues/17620, as requested
by the feature request guidelines]

Cross-compiling scipy and other projects that depend on numpy's
distutils is a huge pain right now, because to do it [in addition to
lots of other details that you have to get right] you have to have both
a native and cross-compiled version of numpy installed. It seems pretty
unreasonable that I need a native version of numpy installed to compile
scipy. One might ask, why is this needed?

Well, scipy's setup.py uses numpy's distutils fork for the fortran
support (I think). Because numpy.distutils is a subpackage of numpy, if
you're working with a cross-compiled version of numpy, it eventually
tries to import some .so that wasn't compiled for your host and
everything dies -- thus you have to have a native numpy installed to
allow numpy.distutils to import correctly.

As far as I can tell, numpy's distutils fork is pure python and doesn't
actually use anything else in numpy itself, and so is completely
separable from numpy. If it were its own top-level package that scipy et
al could use, then cross-compiling would be significantly less tricky.

The mechanics of this would work like so:

* contents of numpy.distutils would be moved to 'numpy_distutils' package
* importing numpy.distutils would raise a deprecation warning and
redirect to numpy_distutils, probably using import hook magic
* scipy and other packages would now utilize numpy_distutils instead of
numpy.distutils at build time

Initially, I considered proposing creating a separate pypi package for
numpy_distutils, but I expect that would be more trouble than it's
worth. One could also propose creating a new PEP 517 build backend, or
move to cmake, or other huge changes, but those also seem more trouble
than they're worth.

Thanks for your consideration.

Dustin