[Numpy-discussion] Splitting MaskedArray into a separate package

[Stefan van der Walt]

On Wed, 23 May 2018 13:30:49 -0700, Ralf Gommers wrote:
> > Good point, which certainly needs to be discussed.  My thought was to
> > move it out into a separate package that could be maintained more in the
> > spirit of a scikit by people who care deeply about its functionality.
> >
> That would be good in principle, but it's only possible that way once the
> specific hacks you refer to above are removed. As long as MaskedArray
> depends on implementation details of ndarray, evolving them in lock-step
> will be necessary. And that is much easier when they're in the same
> package.

Yes, I agree: no special hacks should exist inside of NumPy for
MaskedArrays.  We should, in this instance, become consumers of our
public facing API, and refactor that API as necessary to support it.

> Regarding whether a split-off package will actually be developed, I think
> that depends on having at least one champion for it stepping up. If we just
> move it over into github.com/numpy/maskedarray, I think it will get less
> rather than more attention.

Wouldn't this be a good test of whether MaskedArrays are as valuable as
is being argued?  If so, a community will form around it, and if not it
may fade into obscurity.

Perhaps there is a fear that, in the transition period (i.e., before
potential contributors realize that the NumPy core team is no longer
doing active maintenance) the project may flounder.  But I suspect that
is unlikely to happen, as long as we keep an eye on its test suite from
the NumPy side (perhaps execute its test suite as part of NumPy CI).

Why is the scikit model successful, considering packages could just as
well be part of SciPy?  I would guess: a strong sense of ownership, the
ability to rapidly evolve, better focus, and a lower barrier to entry
(fewer moving pieces) may all play a role.  When you own a small
package, you know no-one else will take care of problems, so you pay
careful attention.

Best regards,
Stéfan