[Numpy-discussion] Cleanup of the `np.lib` namespace (was: ENH: Proposal to add atleast_nd function)

[Ralf Gommers]

On Fri, Feb 19, 2021 at 5:25 PM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> On Thu, 2021-02-18 at 11:10 +0100, Ralf Gommers wrote:
> > On Wed, Feb 17, 2021 at 9:26 PM Oscar Benjamin <
> >
> <snip>
> > >
> >
> > It isn't, but it's relatively straightforward and can be done without
> > thinking about the issues around our other namespaces. Basically:
> > - today `numpy.lib` is a public but fairly useless namespace, because
> > its
> > contents get star-imported to the main namespace; only subsubmodules
> > like
> > `numpy.lib.stride_tricks` are separate
> > - we want to stop this star-importing, which required some tedious
> > work of
> > fixing up how we handle __all__ dicts in addition to making exports
> > explicit
> > - then, we would like to use `numpy.lib` as a namespace for utilities
> > and
> > assorted functionality that people seem to want, but does not meet
> > the bar
>
>
> Initilaly, I did not like the idea of using `np.lib.xyz.function` a lot
> (the `lib` feels a bit unnecessary, compared to fft, etc.), but I am
> starting to warm up to it more. At least it is at least easy to
> remember.
>

The `lib` feels quite necessary to me - it's either cordoning it off
nicely, or moving it to another package completely I'd say. Otherwise we're
again mixing the few high-quality and well-defined `np.xxx` namespaces with
more random stuff.


>
> > for the main namespace and doesn't fit in our other decent namespace
> > (fft,
> > linalg, random, polynomial, f2py, distutils).
> > - TBD if there should be subsubmodules under `numpy.lib` or not
>
>
> I think we should discuss this part :).  I was thinking that we should
> probably aim for _only_ having (or showing) submodules in `lib`?
>
> In particular, I was aiming for hiding all names in `np.lib` that are
> also part of the main namespace (not necessarily deprecating, thanks to
> Python 3.7+ magic).
> The reason is, that I would like to be able to find the "interesting"
> submodules with tab completion and I won't be able to do that, if I
> have a wall of random functions.
>

That sounds like a good idea (the hiding, not deprecating).


> That would leave us with the _current_ "submodules":
>
> * mixins
>
* scimath (same as np.emath though)
>

Mixins seems good to keep, assuming there's a plan to add some more mixins.
A namespace with a single object in it is a bit weird.

scimath is duplicate with emath, so should get an underscore.


> And the ones which are mostly fully exposed to the main namespace:
>
> * type_check
> * index_tricks
> * function_base
> * nanfunctions
> * shape_base
> * stride_tricks
> * twodim_base
> * ufunclike
> * histograms
>

> * polynomial  (not to be confused with numpy.polynomial)
> * utils
> * arraysetops
> * npyio  (includes some additional rarely used funcs)
> * arraypad
> * recfunctions  (needs to be explicitly imported currently)
>

> Looking at those, there are a few I would consider hiding completely
> (since they don't add anything, and the name doesn't feel great):
>
> * function_base
> * ufunlike
> * type_check
> * twodim_base
> * arraypad
> * utils
> * polynomial  (due to the name confusion with `np.polynomial`)
>
> While I am probably OK with keeping the others around, although for
> some the namespace won't be well groomed (e.g. there is another
> `shape_base` in `numpy.core`.
> Others, like `arraysetops` are fully exported to the main namespace,
> but represent complete topical groupings, so I am less sure what we
> should aim for there.
>

For all the ones that get imported into the main namespace, we should give
the subsubmodules an underscore in the name. It makes no sense to have the
same functionality exposed in two different places. Unless we'd want to
clean up the main namespace for some of that stuff in the long term.

Then I think we only have a few left - mixins, recfunctions, npyio,
stride_tricks.

Cheers,
Ralf




>
> Cheers,
>
> Sebastian
>
>
> > - it should be explicitly documented that this is a "lower bar
> > namespace"
> > and that we discourage other array/tensor libraries from copying its
> > API
> >
> > We had a good discussion about this in the community meeting
> > yesterday.
> > Sebastian volunteered to sort out the star-import issue.
> >
> >
> > > I've been thinking about something possibly similar for sympy which
> > > also has a bloated top-level namespace (and has no other place for
> > > public API to go).
> > >
> >
> > A larger plan for cleaning up main namespace bloat, as well as
> > dealing with
> > our unmaintained namespaces (numpy.dual, numpy.emath, etc.) is still
> > needed.
> >
> > Cheers,
> > Ralf
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20210220/634759d7/attachment.html>