<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Courier;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="DE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="color:black">Thanks to all the feedback, we have a new PR of NEP-31.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black">Please find the full-text quoted below:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">============================================================</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">NEP 31 — Context-local and global overrides of the NumPy API</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">============================================================</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">:Author: Hameer Abbasi <<a href="mailto:habbasi@quansight.com"><span style="color:#00006A">habbasi@quansight.com</span></a>></span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">:Author: Ralf Gommers <<a href="mailto:rgommers@quansight.com"><span style="color:#00006A">rgommers@quansight.com</span></a>></span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">:Author: Peter Bell <<a href="mailto:pbell@quansight.com"><span style="color:#00006A">pbell@quansight.com</span></a>></span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">:Status: Draft</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">:Type: Standards Track</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">:Created: 2019-08-22</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Abstract</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">--------</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This NEP proposes to make all of NumPy's public API overridable via an</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">extensible backend mechanism.</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Acceptance of this NEP means NumPy would provide global and context-local</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">overrides, as well as a dispatch mechanism similar to NEP-18 [2]_. First</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">experiences with ``__array_function__`` show that it is necessary to be able</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">to override NumPy functions that *do not take an array-like argument*, and</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">hence aren't overridable via ``__array_function__``. The most pressing need is</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">array creation and coercion functions, such as ``numpy.zeros`` or</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``numpy.asarray``; see e.g. NEP-30 [9]_.</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This NEP proposes to allow, in an opt-in fashion, overriding any part of the</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">NumPy API. It is intended as a comprehensive resolution to NEP-22 [3]_, and</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">obviates the need to add an ever-growing list of new protocols for each new</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">type of function or object that needs to become overridable.</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Motivation and Scope</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">--------------------</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The motivation behind ``uarray`` is manyfold: First, there have been several</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">attempts to allow dispatch of parts of the NumPy API, including (most</span><span lang="EN-US" style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">prominently), the ``__array_ufunc__`` protocol in NEP-13 [4]_, and the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__array_function__`` protocol in NEP-18 [2]_, but this has shown the need</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">for further protocols to be developed, including a protocol for coercion (see</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">[5]_, [9]_). The reasons these overrides are needed have been extensively</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">discussed in the references, and this NEP will not attempt to go into the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">details of why these are needed; but in short: It is necessary for library</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">authors to be able to coerce arbitrary objects into arrays of their own types,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">such as CuPy needing to coerce to a CuPy array, for example, instead of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">a NumPy array.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">These kinds of overrides are useful for both the end-user as well as library</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">authors. End-users may have written or wish to write code that they then later</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">speed up or move to a different implementation, say PyData/Sparse. They can do</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">this simply by setting a backend. Library authors may also wish to write code</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">that is portable across array implementations, for example ``sklearn`` may wish</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">to write code for a machine learning algorithm that is portable across array</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementations while also using array creation functions.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This NEP takes a holistic approach: It assumes that there are parts of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">the API that need to be overridable, and that these will grow over time. It</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">provides a general framework and a mechanism to avoid a design of a new</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">protocol each time this is required. This was the goal of ``uarray``: to</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">allow for overrides in an API without needing the design of a new protocol.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This NEP proposes the following: That ``unumpy`` [8]_ becomes the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">recommended override mechanism for the parts of the NumPy API not yet covered</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">by ``__array_function__`` or ``__array_ufunc__``, and that ``uarray`` is</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">vendored into a new namespace within NumPy to give users and downstream</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">dependencies access to these overrides. This vendoring mechanism is similar</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">to what SciPy decided to do for making ``scipy.fft`` overridable (see [10]_).</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Detailed description</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">--------------------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Using overrides</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The way we propose the overrides will be used by end users is::</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # On the library side</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import numpy.overridable as unp</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def library_function(array):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> array = unp.asarray(array)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Code using unumpy as usual</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return array</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # On the user side:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import numpy.overridable as unp</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import uarray as ua</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import dask.array as da</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> ua.register_backend(da)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> library_function(dask_array) # works and returns dask_array</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> with unp.set_backend(da):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> library_function([1, 2, 3, 4]) # actually returns a Dask array.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Here, ``backend`` can be any compatible object defined either by NumPy or an</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">external library, such as Dask or CuPy. Ideally, it should be the module</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``dask.array`` or ``cupy`` itself.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Composing backends</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">There are some backends which may depend on other backends, for example xarray</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">depending on `numpy.fft`, and transforming a time axis into a frequency axis,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">or Dask/xarray holding an array other than a NumPy array inside it. This would</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">be handled in the following manner inside code::</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> with ua.set_backend(cupy), ua.set_backend(dask.array):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Code that has distributed GPU arrays here</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Proposals</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The only change this NEP proposes at its acceptance, is to make ``unumpy`` the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">officially recommended way to override NumPy, along with making some submodules</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">overridable by default via ``uarray``. ``unumpy`` will remain a separate</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">repository/package (which we propose to vendor to avoid a hard dependency, and</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">use the separate ``unumpy`` package only if it is installed, rather than depend</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">on for the time being). In concrete terms, ``numpy.overridable`` becomes an</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">alias for ``unumpy``, if available with a fallback to the a vendored version if</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">not. ``uarray`` and ``unumpy`` and will be developed primarily with the input</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">of duck-array authors and secondarily, custom dtype authors, via the usual</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">GitHub workflow. There are a few reasons for this:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Faster iteration in the case of bugs or issues.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Faster design changes, in the case of needed functionality.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``unumpy`` will work with older versions of NumPy as well.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* The user and library author opt-in to the override process,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> rather than breakages happening when it is least expected.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> In simple terms, bugs in ``unumpy`` mean that ``numpy`` remains</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> unaffected.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* For ``numpy.fft``, ``numpy.linalg`` and ``numpy.random``, the functions in</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> the main namespace will mirror those in the ``numpy.overridable`` namespace.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> The reason for this is that there may exist functions in the in these</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> submodules that need backends, even for ``numpy.ndarray`` inputs.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Advantanges of ``unumpy`` over other solutions</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``unumpy`` offers a number of advantanges over the approach of defining a new</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">protocol for every problem encountered: Whenever there is something requiring</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">an override, ``unumpy`` will be able to offer a unified API with very minor</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">changes. For example:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``ufunc`` objects can be overridden via their ``__call__``, ``reduce`` and</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> other methods.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Other functions can be overridden in a similar fashion.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``np.asduckarray`` goes away, and becomes ``np.overridable.asarray`` with a</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> backend set.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* The same holds for array creation functions such as ``np.zeros``,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> ``np.empty`` and so on.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This also holds for the future: Making something overridable would require only</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">minor changes to ``unumpy``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Another promise ``unumpy`` holds is one of default implementations. Default</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementations can be provided for any multimethod, in terms of others. This</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">allows one to override a large part of the NumPy API by defining only a small</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">part of it. This is to ease the creation of new duck-arrays, by providing</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">default implementations of many functions that can be easily expressed in</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">terms of others, as well as a repository of utility functions that help in the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementation of duck-arrays that most duck-arrays would require. This would</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">allow us to avoid designing entire protocols, e.g., a protocol for stacking</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">and concatenating would be replaced by simply implementing ``stack`` and/or</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``concatenate`` and then providing default implementations for everything else</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">in that class. The same applies for transposing, and many other functions for</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">which protocols haven't been proposed, such as ``isin`` in terms of ``in1d``,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``setdiff1d`` in terms of ``unique``, and so on.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">It also allows one to override functions in a manner which</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__array_function__`` simply cannot, such as overriding ``np.einsum`` with the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">version from the ``opt_einsum`` package, or Intel MKL overriding FFT, BLAS</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">or ``ufunc`` objects. They would define a backend with the appropriate</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">multimethods, and the user would select them via a ``with`` statement, or</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">registering them as a backend.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The last benefit is a clear way to coerce to a given backend (via the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``coerce`` keyword in ``ua.set_backend``), and a protocol</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">for coercing not only arrays, but also ``dtype`` objects and ``ufunc`` objects</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">with similar ones from other libraries. This is due to the existence of actual,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">third party dtype packages, and their desire to blend into the NumPy ecosystem</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">(see [6]_). This is a separate issue compared to the C-level dtype redesign</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">proposed in [7]_, it's about allowing third-party dtype implementations to</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">work with NumPy, much like third-party array implementations. These can provide</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">features such as, for example, units, jagged arrays or other such features that</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">are outside the scope of NumPy.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Mixing NumPy and ``unumpy`` in the same file</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Normally, one would only want to import only one of ``unumpy`` or ``numpy``,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">you would import it as ``np`` for familiarity. However, there may be situations</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">where one wishes to mix NumPy and the overrides, and there are a few ways to do</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">this, depending on the user's style::</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> from numpy import overridable as unp</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import numpy as np</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">or::</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import numpy as np</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Use unumpy via np.overridable</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Duck-array coercion</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">There are inherent problems about returning objects that are not NumPy arrays</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">from ``numpy.array`` or ``numpy.asarray``, particularly in the context of C/C++</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">or Cython code that may get an object with a different memory layout than the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">one it expects. However, we believe this problem may apply not only to these</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">two functions but all functions that return NumPy arrays. For this reason,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">overrides are opt-in for the user, by using the submodule ``numpy.overridable``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">rather than ``numpy``. NumPy will continue to work unaffected by anything in</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``numpy.overridable``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">If the user wishes to obtain a NumPy array, there are two ways of doing it:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">1. Use ``numpy.asarray`` (the non-overridable version).</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">2. Use ``numpy.overridable.asarray`` with the NumPy backend set and coercion</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> enabled</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Aliases outside of the ``numpy.overridable`` namespace</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">All functionality in ``numpy.random``, ``numpy.linalg`` and ``numpy.fft``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">will be aliased to their respective overridable versions inside</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``numpy.overridable``. The reason for this is that there are alternative</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementations of RNGs (``mkl-random``), linear algebra routines (``eigen``,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``blis``) and FFT routines (``mkl-fft``, ``pyFFTW``) that need to operate on</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``numpy.ndarray`` inputs, but still need the ability to switch behaviour.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This is different from monkeypatching in a few different ways:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* The caller-facing signature of the function is always the same,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> so there is at least the loose sense of an API contract. Monkeypatching</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> does not provide this ability.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* There is the ability of locally switching the backend.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* It has been `suggested <<a href="http://numpy-discussion.10968.n7.nabble.com/NEP-31-Context-local-and-global-overrides-of-the-NumPy-API-tp47452p47472.html%3e%60_"><span style="color:#00006A">http://numpy-discussion.10968.n7.nabble.com/NEP-31-Context-local-and-global-overrides-of-the-NumPy-API-tp47452p47472.html>`_</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> that the reason that 1.17 hasn't landed in the Anaconda defaults channel is</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> due to the incompatibility between monkeypatching and ``__array_function__``,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> as monkeypatching would bypass the protocol completely.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Statements of the form ``from numpy import x; x`` and ``np.x`` would have</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> different results depending on whether the import was made before or</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> after monkeypatching happened.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">All this isn't possible at all with ``__array_function__`` or</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__array_ufunc__``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">It has been formally realised (at least in part) that a backend system is</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">needed for this, in the `NumPy roadmap <<a href="https://numpy.org/neps/roadmap.html#other-functionality>`_"><span style="color:#00006A">https://numpy.org/neps/roadmap.html#other-functionality>`_</span></a>.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">For ``numpy.random``, it's still necessary to make the C-API fit the one</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">proposed in `NEP-19 <<a href="https://numpy.org/neps/nep-0019-rng-policy.html%3e%60_"><span style="color:#00006A">https://numpy.org/neps/nep-0019-rng-policy.html>`_</span></a>.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This is impossible for `mkl-random`, because then it would need to be</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">rewritten to fit that framework. The guarantees on stream</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">compatibility will be the same as before, but if there's a backend that affects</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``numpy.random`` set, we make no guarantees about stream compatibility, and it</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">is up to the backend author to provide their own guarantees.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Providing a way for implicit dispatch</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">It has been suggested that the ability to dispatch methods which do not take</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">a dispatchable is needed, while guessing that backend from another dispatchable.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">As a concrete example, consider the following:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. code:: python</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> with unumpy.determine_backend(array_like, np.ndarray):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> unumpy.arange(len(array_like))</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">While this does not exist yet in ``uarray``, it is trivial to add it. The need for</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">this kind of code exists because one might want to have an alternative for the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">proposed ``*_like`` functions, or the ``like=`` keyword argument. The need for these</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">exists because there are functions in the NumPy API that do not take a dispatchable</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">argument, but there is still the need to select a backend based on a different</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">dispatchable.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The need for an opt-in module</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The need for an opt-in module is realised because of a few reasons:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* There are parts of the API (like `numpy.asarray`) that simply cannot be</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> overridden due to incompatibility concerns with C/Cython extensions, however,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> one may want to coerce to a duck-array using ``asarray`` with a backend set.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* There are possible issues around an implicit option and monkeypatching, such</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> as those mentioned above.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">NEP 18 notes that this may require maintenance of two separate APIs. However,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">this burden may be lessened by, for example, parametrizing all tests over</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``numpy.overridable`` separately via a fixture. This also has the side-effect</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">of thoroughly testing it, unlike ``__array_function__``. We also feel that it</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">provides an oppurtunity to separate the NumPy API contract properly from the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementation.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Benefits to end-users and mixing backends</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Mixing backends is easy in ``uarray``, one only has to do:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. code:: python</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Explicitly say which backends you want to mix</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> ua.register_backend(backend1)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> ua.register_backend(backend2)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> ua.register_backend(backend3)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Freely use code that mixes backends here.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The benefits to end-users extend beyond just writing new code. Old code</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">(usually in the form of scripts) can be easily ported to different backends</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">by a simple import switch and a line adding the preferred backend. This way,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">users may find it easier to port existing code to GPU or distributed computing.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Related Work</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">------------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Other override mechanisms</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* NEP-18, the ``__array_function__`` protocol. [2]_</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* NEP-13, the ``__array_ufunc__`` protocol. [3]_</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* NEP-30, the ``__duck_array__`` protocol. [9]_</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Existing NumPy-like array implementations</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Dask: <a href="https://dask.org/"><span style="color:#00006A">https://dask.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* CuPy: <a href="https://cupy.chainer.org/"><span style="color:#00006A">https://cupy.chainer.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* PyData/Sparse: <a href="https://sparse.pydata.org/"><span style="color:#00006A">https://sparse.pydata.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Xnd: <a href="https://xnd.readthedocs.io/"><span style="color:#00006A">https://xnd.readthedocs.io/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Astropy's Quantity: <a href="https://docs.astropy.org/en/stable/units/"><span style="color:#00006A">https://docs.astropy.org/en/stable/units/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Existing and potential consumers of alternative arrays</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Dask: <a href="https://dask.org/"><span style="color:#00006A">https://dask.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* scikit-learn: <a href="https://scikit-learn.org/"><span style="color:#00006A">https://scikit-learn.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* xarray: <a href="https://xarray.pydata.org/"><span style="color:#00006A">https://xarray.pydata.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* TensorLy: <a href="http://tensorly.org/"><span style="color:#00006A">http://tensorly.org/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Existing alternate dtype implementations</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``ndtypes``: <a href="https://ndtypes.readthedocs.io/en/latest/"><span style="color:#00006A">https://ndtypes.readthedocs.io/en/latest/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Datashape: <a href="https://datashape.readthedocs.io"><span style="color:#00006A">https://datashape.readthedocs.io</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Plum: <a href="https://plum-py.readthedocs.io/"><span style="color:#00006A">https://plum-py.readthedocs.io/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Alternate implementations of parts of the NumPy API</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``mkl_random``: <a href="https://github.com/IntelPython/mkl_random"><span style="color:#00006A">https://github.com/IntelPython/mkl_random</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Courier;color:black">* ``mkl_fft``: <a href="https://github.com/IntelPython/mkl_fft"><span style="color:#00006A">https://github.com/IntelPython/mkl_fft</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``bottleneck``: <a href="https://github.com/pydata/bottleneck"><span style="color:#00006A">https://github.com/pydata/bottleneck</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Courier;color:black">* ``opt_einsum``: <a href="https://github.com/dgasmith/opt_einsum"><span style="color:#00006A">https://github.com/dgasmith/opt_einsum</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Implementation</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">--------------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The implementation of this NEP will require the following steps:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Implementation of ``uarray`` multimethods corresponding to the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> NumPy API, including classes for overriding ``dtype``, ``ufunc``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> and ``array`` objects, in the ``unumpy`` repository.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Moving backends from ``unumpy`` into the respective array libraries.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``uarray`` Primer</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">~~~~~~~~~~~~~~~~~</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">**Note:** *This section will not attempt to go into too much detail about</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">uarray, that is the purpose of the uarray documentation.* [1]_</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">*However, the NumPy community will have input into the design of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">uarray, via the issue tracker.*</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``unumpy`` is the interface that defines a set of overridable functions</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">(multimethods) compatible with the numpy API. To do this, it uses the</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``uarray`` library. ``uarray`` is a general purpose tool for creating</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">multimethods that dispatch to one of multiple different possible backend</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementations. In this sense, it is similar to the ``__array_function__``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">protocol but with the key difference that the backend is explicitly installed</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">by the end-user and not coupled into the array type.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Decoupling the backend from the array type gives much more flexibility to</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">end-users and backend authors. For example, it is possible to:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* override functions not taking arrays as arguments</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* create backends out of source from the array type</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* install multiple backends for the same array type</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This decoupling also means that ``uarray`` is not constrained to dispatching</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">over array-like types. The backend is free to inspect the entire set of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">function arguments to determine if it can implement the function e.g. ``dtype``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">parameter dispatching.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Defining backends</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">^^^^^^^^^^^^^^^^^</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``uarray`` consists of two main protocols: ``__ua_convert__`` and</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__ua_function__``, called in that order, along with ``__ua_domain__``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__ua_convert__`` is for conversion and coercion. It has the signature</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``(dispatchables, coerce)``, where ``dispatchables`` is an iterable of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``ua.Dispatchable`` objects and ``coerce`` is a boolean indicating whether or</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">not to force the conversion. ``ua.Dispatchable`` is a simple class consisting</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">of three simple values: ``type``, ``value``, and ``coercible``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__ua_convert__`` returns an iterable of the converted values, or</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``NotImplemented`` in the case of failure.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">``__ua_function__`` has the signature ``(func, args, kwargs)`` and defines</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">the actual implementation of the function. It recieves the function and its</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">arguments. Returning ``NotImplemented`` will cause a move to the default</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">implementation of the function if one exists, and failing that, the next</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">backend.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Here is what will happen assuming a ``uarray`` multimethod is called:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">1. We canonicalise the arguments so any arguments without a default</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> are placed in ``*args`` and those with one are placed in ``**kwargs``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">2. We check the list of backends.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> a. If it is empty, we try the default implementation.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">3. We check if the backend's ``__ua_convert__`` method exists. If it exists:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> a. We pass it the output of the dispatcher,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> which is an iterable of ``ua.Dispatchable`` objects.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> b. We feed this output, along with the arguments,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> to the argument replacer. ``NotImplemented`` means we move to 3</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> with the next backend.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> c. We store the replaced arguments as the new arguments.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">4. We feed the arguments into ``__ua_function__``, and return the output, and</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> exit if it isn't ``NotImplemented``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">5. If the default implementation exists, we try it with the current backend.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">6. On failure, we move to 3 with the next backend. If there are no more</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> backends, we move to 7.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">7. We raise a ``ua.BackendNotImplementedError``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Defining overridable multimethods</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">To define an overridable function (a multimethod), one needs a few things:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">1. A dispatcher that returns an iterable of ``ua.Dispatchable`` objects.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">2. A reverse dispatcher that replaces dispatchable values with the supplied</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> ones.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">3. A domain.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">4. Optionally, a default implementation, which can be provided in terms of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> other multimethods.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">As an example, consider the following::</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import uarray as ua</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def full_argreplacer(args, kwargs, dispatchables):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def full(shape, fill_value, dtype=None, order='C'):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return (shape, fill_value), dict(</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> dtype=dispatchables[0],</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> order=order</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> )</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return full(*args, **kwargs)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> @ua.create_multimethod(full_argreplacer, domain="numpy")</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def full(shape, fill_value, dtype=None, order='C'):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return (ua.Dispatchable(dtype, np.dtype),)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">A large set of examples can be found in the ``unumpy`` repository, [8]_.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This simple act of overriding callables allows us to override:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Methods</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Properties, via ``fget`` and ``fset``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Entire objects, via ``__get__``.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Examples for NumPy</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">^^^^^^^^^^^^^^^^^^</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">A library that implements a NumPy-like API will use it in the following</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">manner (as an example)::</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> import numpy.overridable as unp</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> _ua_implementations = {}</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> __ua_domain__ = "numpy"</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def __ua_function__(func, args, kwargs):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> fn = _ua_implementations.get(func, None)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return fn(*args, **kwargs) if fn is not None else NotImplemented</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def implements(ua_func):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def inner(func):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> _ua_implementations[ua_func] = func</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return func</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> return inner</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> @implements(unp.asarray)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def asarray(a, dtype=None, order=None):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Code here</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Either this method or __ua_convert__ must</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # return NotImplemented for unsupported types,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Or they shouldn't be marked as dispatchable.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Provides a default implementation for ones and zeros.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> @implements(unp.full)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> def full(shape, fill_value, dtype=None, order='C'):</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> # Code here</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Backward compatibility</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">----------------------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">There are no backward incompatible changes proposed in this NEP.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Alternatives</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">------------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The current alternative to this problem is a combination of NEP-18 [2]_,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">NEP-13 [4]_ and NEP-30 [9]_ plus adding more protocols (not yet specified)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">in addition to it. Even then, some parts of the NumPy API will remain</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">non-overridable, so it's a partial alternative.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The main alternative to vendoring ``unumpy`` is to simply move it into NumPy</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">completely and not distribute it as a separate package. This would also achieve</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">the proposed goals, however we prefer to keep it a separate package for now,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">for reasons already stated above.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">The third alternative is to move ``unumpy`` into the NumPy organisation and</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">develop it as a NumPy project. This will also achieve the said goals, and is</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">also a possibility that can be considered by this NEP. However, the act of</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">doing an extra ``pip install`` or ``conda install`` may discourage some users</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">from adopting this method.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">An alternative to requiring opt-in is mainly to *not* override ``np.asarray``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">and ``np.array``, and making the rest of the NumPy API surface overridable,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">instead providing ``np.duckarray`` and ``np.asduckarray``</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">as duck-array friendly alternatives that used the respective overrides. However,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">this has the downside of adding a minor overhead to NumPy calls.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Discussion</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">----------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* ``uarray`` blogpost: <a href="https://labs.quansight.org/blog/2019/07/uarray-update-api-changes-overhead-and-comparison-to-__array_function__/"><span style="color:#00006A">https://labs.quansight.org/blog/2019/07/uarray-update-api-changes-overhead-and-comparison-to-__array_function__/</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* The discussion section of NEP-18: <a href="https://numpy.org/neps/nep-0018-array-function-protocol.html#discussion"><span style="color:#00006A">https://numpy.org/neps/nep-0018-array-function-protocol.html#discussion</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* NEP-22: <a href="https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html"><span style="color:#00006A">https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Dask issue #4462: <a href="https://github.com/dask/dask/issues/4462"><span style="color:#00006A">https://github.com/dask/dask/issues/4462</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* PR #13046: <a href="https://github.com/numpy/numpy/pull/13046"><span style="color:#00006A">https://github.com/numpy/numpy/pull/13046</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Dask issue #4883: <a href="https://github.com/dask/dask/issues/4883"><span style="color:#00006A">https://github.com/dask/dask/issues/4883</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Issue #13831: <a href="https://github.com/numpy/numpy/issues/13831"><span style="color:#00006A">https://github.com/numpy/numpy/issues/13831</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Discussion PR 1: <a href="https://github.com/hameerabbasi/numpy/pull/3"><span style="color:#00006A">https://github.com/hameerabbasi/numpy/pull/3</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Discussion PR 2: <a href="https://github.com/hameerabbasi/numpy/pull/4"><span style="color:#00006A">https://github.com/hameerabbasi/numpy/pull/4</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">* Discussion PR 3: <a href="https://github.com/numpy/numpy/pull/14389"><span style="color:#00006A">https://github.com/numpy/numpy/pull/14389</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">References and Footnotes</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">------------------------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [1] uarray, A general dispatch mechanism for Python: <a href="https://uarray.readthedocs.io"><span style="color:#00006A">https://uarray.readthedocs.io</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [2] NEP 18 — A dispatch mechanism for NumPy’s high level array functions: <a href="https://numpy.org/neps/nep-0018-array-function-protocol.html"><span style="color:#00006A">https://numpy.org/neps/nep-0018-array-function-protocol.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [3] NEP 22 — Duck typing for NumPy arrays – high level overview: <a href="https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html"><span style="color:#00006A">https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [4] NEP 13 — A Mechanism for Overriding Ufuncs: <a href="https://numpy.org/neps/nep-0013-ufunc-overrides.html"><span style="color:#00006A">https://numpy.org/neps/nep-0013-ufunc-overrides.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [5] Reply to Adding to the non-dispatched implementation of NumPy methods: <a href="http://numpy-discussion.10968.n7.nabble.com/Adding-to-the-non-dispatched-implementation-of-NumPy-methods-tp46816p46874.html"><span style="color:#00006A">http://numpy-discussion.10968.n7.nabble.com/Adding-to-the-non-dispatched-implementation-of-NumPy-methods-tp46816p46874.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [6] Custom Dtype/Units discussion: <a href="http://numpy-discussion.10968.n7.nabble.com/Custom-Dtype-Units-discussion-td43262.html"><span style="color:#00006A">http://numpy-discussion.10968.n7.nabble.com/Custom-Dtype-Units-discussion-td43262.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [7] The epic dtype cleanup plan: <a href="https://github.com/numpy/numpy/issues/2899"><span style="color:#00006A">https://github.com/numpy/numpy/issues/2899</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [8] unumpy: NumPy, but implementation-independent: <a href="https://unumpy.readthedocs.io"><span style="color:#00006A">https://unumpy.readthedocs.io</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [9] NEP 30 — Duck Typing for NumPy Arrays - Implementation: <a href="https://www.numpy.org/neps/nep-0030-duck-array-protocol.html"><span style="color:#00006A">https://www.numpy.org/neps/nep-0030-duck-array-protocol.html</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">.. [10] <a href="http://scipy.github.io/devdocs/fft.html#backend-control"><span style="color:#00006A">http://scipy.github.io/devdocs/fft.html#backend-control</span></a></span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">Copyright</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">---------</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:Courier;color:black">This document has been placed in the public domain.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">NumPy-Discussion <numpy-discussion-bounces+hameerabbasi=yahoo.com@python.org> on behalf of Hameer Abbasi <einstein.edison@gmail.com><br>
<b>Reply to: </b>Discussion of Numerical Python <numpy-discussion@python.org><br>
<b>Date: </b>Thursday, 5. September 2019 at 17:12<br>
<b>To: </b><numpy-discussion@python.org><br>
<b>Subject: </b>Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p>Hello everyone;<o:p></o:p></p>
<p>Thanks to all the feedback from the community, in particular Sebastian Berg, we have a new draft of NEP-31.<o:p></o:p></p>
<p>Please find the full text quoted below for discussion and reference. Any feedback and discussion is welcome.<o:p></o:p></p>
<p><o:p> </o:p></p>
<pre>============================================================<o:p></o:p></pre>
<pre>NEP 31 — Context-local and global overrides of the NumPy API<o:p></o:p></pre>
<pre>============================================================<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>:Author: Hameer Abbasi <a href="mailto:habbasi@quansight.com"><habbasi@quansight.com></a><o:p></o:p></pre>
<pre>:Author: Ralf Gommers <a href="mailto:rgommers@quansight.com"><rgommers@quansight.com></a><o:p></o:p></pre>
<pre>:Author: Peter Bell <a href="mailto:pbell@quansight.com"><pbell@quansight.com></a><o:p></o:p></pre>
<pre>:Status: Draft<o:p></o:p></pre>
<pre>:Type: Standards Track<o:p></o:p></pre>
<pre>:Created: 2019-08-22<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Abstract<o:p></o:p></pre>
<pre>--------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This NEP proposes to make all of NumPy's public API overridable via an<o:p></o:p></pre>
<pre>extensible backend mechanism.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Acceptance of this NEP means NumPy would provide global and context-local<o:p></o:p></pre>
<pre>overrides, as well as a dispatch mechanism similar to NEP-18 [2]_. First<o:p></o:p></pre>
<pre>experiences with ``__array_function__`` show that it is necessary to be able<o:p></o:p></pre>
<pre>to override NumPy functions that *do not take an array-like argument*, and<o:p></o:p></pre>
<pre>hence aren't overridable via ``__array_function__``. The most pressing need is<o:p></o:p></pre>
<pre>array creation and coercion functions, such as ``numpy.zeros`` or<o:p></o:p></pre>
<pre>``numpy.asarray``; see e.g. NEP-30 [9]_.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This NEP proposes to allow, in an opt-in fashion, overriding any part of the<o:p></o:p></pre>
<pre>NumPy API. It is intended as a comprehensive resolution to NEP-22 [3]_, and<o:p></o:p></pre>
<pre>obviates the need to add an ever-growing list of new protocols for each new<o:p></o:p></pre>
<pre>type of function or object that needs to become overridable.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Motivation and Scope<o:p></o:p></pre>
<pre>--------------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The motivation behind ``uarray`` is manyfold: First, there have been several<o:p></o:p></pre>
<pre>attempts to allow dispatch of parts of the NumPy API, including (most<o:p></o:p></pre>
<pre>prominently), the ``__array_ufunc__`` protocol in NEP-13 [4]_, and the<o:p></o:p></pre>
<pre>``__array_function__`` protocol in NEP-18 [2]_, but this has shown the need<o:p></o:p></pre>
<pre>for further protocols to be developed, including a protocol for coercion (see<o:p></o:p></pre>
<pre>[5]_, [9]_). The reasons these overrides are needed have been extensively<o:p></o:p></pre>
<pre>discussed in the references, and this NEP will not attempt to go into the<o:p></o:p></pre>
<pre>details of why these are needed; but in short: It is necessary for library<o:p></o:p></pre>
<pre>authors to be able to coerce arbitrary objects into arrays of their own types,<o:p></o:p></pre>
<pre>such as CuPy needing to coerce to a CuPy array, for example, instead of<o:p></o:p></pre>
<pre>a NumPy array.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>These kinds of overrides are useful for both the end-user as well as library<o:p></o:p></pre>
<pre>authors. End-users may have written or wish to write code that they then later<o:p></o:p></pre>
<pre>speed up or move to a different implementation, say PyData/Sparse. They can do<o:p></o:p></pre>
<pre>this simply by setting a backend. Library authors may also wish to write code<o:p></o:p></pre>
<pre>that is portable across array implementations, for example ``sklearn`` may wish<o:p></o:p></pre>
<pre>to write code for a machine learning algorithm that is portable across array<o:p></o:p></pre>
<pre>implementations while also using array creation functions.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This NEP takes a holistic approach: It assumes that there are parts of<o:p></o:p></pre>
<pre>the API that need to be overridable, and that these will grow over time. It<o:p></o:p></pre>
<pre>provides a general framework and a mechanism to avoid a design of a new<o:p></o:p></pre>
<pre>protocol each time this is required. This was the goal of ``uarray``: to<o:p></o:p></pre>
<pre>allow for overrides in an API without needing the design of a new protocol.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This NEP proposes the following: That ``unumpy`` [8]_ becomes the<o:p></o:p></pre>
<pre>recommended override mechanism for the parts of the NumPy API not yet covered<o:p></o:p></pre>
<pre>by ``__array_function__`` or ``__array_ufunc__``, and that ``uarray`` is<o:p></o:p></pre>
<pre>vendored into a new namespace within NumPy to give users and downstream<o:p></o:p></pre>
<pre>dependencies access to these overrides. This vendoring mechanism is similar<o:p></o:p></pre>
<pre>to what SciPy decided to do for making ``scipy.fft`` overridable (see [10]_).<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Detailed description<o:p></o:p></pre>
<pre>--------------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Using overrides<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The way we propose the overrides will be used by end users is::<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> # On the library side<o:p></o:p></pre>
<pre> import numpy.overridable as unp<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> def library_function(array):<o:p></o:p></pre>
<pre> array = unp.asarray(array)<o:p></o:p></pre>
<pre> # Code using unumpy as usual<o:p></o:p></pre>
<pre> return array<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> # On the user side:<o:p></o:p></pre>
<pre> import numpy.overridable as unp<o:p></o:p></pre>
<pre> import uarray as ua<o:p></o:p></pre>
<pre> import dask.array as da<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> ua.register_backend(da)<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> library_function(dask_array) # works and returns dask_array<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> with unp.set_backend(da):<o:p></o:p></pre>
<pre> library_function([1, 2, 3, 4]) # actually returns a Dask array.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Here, ``backend`` can be any compatible object defined either by NumPy or an<o:p></o:p></pre>
<pre>external library, such as Dask or CuPy. Ideally, it should be the module<o:p></o:p></pre>
<pre>``dask.array`` or ``cupy`` itself.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Composing backends<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>There are some backends which may depend on other backends, for example xarray<o:p></o:p></pre>
<pre>depending on `numpy.fft`, and transforming a time axis into a frequency axis,<o:p></o:p></pre>
<pre>or Dask/xarray holding an array other than a NumPy array inside it. This would<o:p></o:p></pre>
<pre>be handled in the following manner inside code::<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> with ua.set_backend(cupy), ua.set_backend(dask.array):<o:p></o:p></pre>
<pre> # Code that has distributed GPU arrays here<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Proposals<o:p></o:p></pre>
<pre>~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The only change this NEP proposes at its acceptance, is to make ``unumpy`` the<o:p></o:p></pre>
<pre>officially recommended way to override NumPy. ``unumpy`` will remain a separate<o:p></o:p></pre>
<pre>repository/package (which we propose to vendor to avoid a hard dependency, and<o:p></o:p></pre>
<pre>use the separate ``unumpy`` package only if it is installed, rather than depend<o:p></o:p></pre>
<pre>on for the time being). In concrete terms, ``numpy.overridable`` becomes an<o:p></o:p></pre>
<pre>alias for ``unumpy``, if available with a fallback to the a vendored version if<o:p></o:p></pre>
<pre>not. ``uarray`` and ``unumpy`` and will be developed primarily with the input<o:p></o:p></pre>
<pre>of duck-array authors and secondarily, custom dtype authors, via the usual<o:p></o:p></pre>
<pre>GitHub workflow. There are a few reasons for this:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* Faster iteration in the case of bugs or issues.<o:p></o:p></pre>
<pre>* Faster design changes, in the case of needed functionality.<o:p></o:p></pre>
<pre>* ``unumpy`` will work with older versions of NumPy as well.<o:p></o:p></pre>
<pre>* The user and library author opt-in to the override process,<o:p></o:p></pre>
<pre> rather than breakages happening when it is least expected.<o:p></o:p></pre>
<pre> In simple terms, bugs in ``unumpy`` mean that ``numpy`` remains<o:p></o:p></pre>
<pre> unaffected.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Advantanges of ``unumpy`` over other solutions<o:p></o:p></pre>
<pre>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>``unumpy`` offers a number of advantanges over the approach of defining a new<o:p></o:p></pre>
<pre>protocol for every problem encountered: Whenever there is something requiring<o:p></o:p></pre>
<pre>an override, ``unumpy`` will be able to offer a unified API with very minor<o:p></o:p></pre>
<pre>changes. For example:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* ``ufunc`` objects can be overridden via their ``__call__``, ``reduce`` and<o:p></o:p></pre>
<pre> other methods.<o:p></o:p></pre>
<pre>* Other functions can be overridden in a similar fashion.<o:p></o:p></pre>
<pre>* ``np.asduckarray`` goes away, and becomes ``np.overridable.asarray`` with a<o:p></o:p></pre>
<pre> backend set.<o:p></o:p></pre>
<pre>* The same holds for array creation functions such as ``np.zeros``,<o:p></o:p></pre>
<pre> ``np.empty`` and so on.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This also holds for the future: Making something overridable would require only<o:p></o:p></pre>
<pre>minor changes to ``unumpy``.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Another promise ``unumpy`` holds is one of default implementations. Default<o:p></o:p></pre>
<pre>implementations can be provided for any multimethod, in terms of others. This<o:p></o:p></pre>
<pre>allows one to override a large part of the NumPy API by defining only a small<o:p></o:p></pre>
<pre>part of it. This is to ease the creation of new duck-arrays, by providing<o:p></o:p></pre>
<pre>default implementations of many functions that can be easily expressed in<o:p></o:p></pre>
<pre>terms of others, as well as a repository of utility functions that help in the<o:p></o:p></pre>
<pre>implementation of duck-arrays that most duck-arrays would require.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>It also allows one to override functions in a manner which<o:p></o:p></pre>
<pre>``__array_function__`` simply cannot, such as overriding ``np.einsum`` with the<o:p></o:p></pre>
<pre>version from the ``opt_einsum`` package, or Intel MKL overriding FFT, BLAS<o:p></o:p></pre>
<pre>or ``ufunc`` objects. They would define a backend with the appropriate<o:p></o:p></pre>
<pre>multimethods, and the user would select them via a ``with`` statement, or<o:p></o:p></pre>
<pre>registering them as a backend.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The last benefit is a clear way to coerce to a given backend (via the<o:p></o:p></pre>
<pre>``coerce`` keyword in ``ua.set_backend``), and a protocol<o:p></o:p></pre>
<pre>for coercing not only arrays, but also ``dtype`` objects and ``ufunc`` objects<o:p></o:p></pre>
<pre>with similar ones from other libraries. This is due to the existence of actual,<o:p></o:p></pre>
<pre>third party dtype packages, and their desire to blend into the NumPy ecosystem<o:p></o:p></pre>
<pre>(see [6]_). This is a separate issue compared to the C-level dtype redesign<o:p></o:p></pre>
<pre>proposed in [7]_, it's about allowing third-party dtype implementations to<o:p></o:p></pre>
<pre>work with NumPy, much like third-party array implementations. These can provide<o:p></o:p></pre>
<pre>features such as, for example, units, jagged arrays or other such features that<o:p></o:p></pre>
<pre>are outside the scope of NumPy.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Mixing NumPy and ``unumpy`` in the same file<o:p></o:p></pre>
<pre>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Normally, one would only want to import only one of ``unumpy`` or ``numpy``,<o:p></o:p></pre>
<pre>you would import it as ``np`` for familiarity. However, there may be situations<o:p></o:p></pre>
<pre>where one wishes to mix NumPy and the overrides, and there are a few ways to do<o:p></o:p></pre>
<pre>this, depending on the user's style::<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> from numpy import overridable as unp<o:p></o:p></pre>
<pre> import numpy as np<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>or::<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> import numpy as np<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> # Use unumpy via np.overridable<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Duck-array coercion<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>There are inherent problems about returning objects that are not NumPy arrays<o:p></o:p></pre>
<pre>from ``numpy.array`` or ``numpy.asarray``, particularly in the context of C/C++<o:p></o:p></pre>
<pre>or Cython code that may get an object with a different memory layout than the<o:p></o:p></pre>
<pre>one it expects. However, we believe this problem may apply not only to these<o:p></o:p></pre>
<pre>two functions but all functions that return NumPy arrays. For this reason,<o:p></o:p></pre>
<pre>overrides are opt-in for the user, by using the submodule ``numpy.overridable``<o:p></o:p></pre>
<pre>rather than ``numpy``. NumPy will continue to work unaffected by anything in<o:p></o:p></pre>
<pre>``numpy.overridable``.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>If the user wishes to obtain a NumPy array, there are two ways of doing it:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>1. Use ``numpy.asarray`` (the non-overridable version).<o:p></o:p></pre>
<pre>2. Use ``numpy.overridable.asarray`` with the NumPy backend set and coercion<o:p></o:p></pre>
<pre> enabled<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Related Work<o:p></o:p></pre>
<pre>------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Other override mechanisms<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* NEP-18, the ``__array_function__`` protocol. [2]_<o:p></o:p></pre>
<pre>* NEP-13, the ``__array_ufunc__`` protocol. [3]_<o:p></o:p></pre>
<pre>* NEP-30, the ``__duck_array__`` protocol. [9]_<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Existing NumPy-like array implementations<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* Dask: <a href="https://dask.org/">https://dask.org/</a><o:p></o:p></pre>
<pre>* CuPy: <a href="https://cupy.chainer.org/">https://cupy.chainer.org/</a><o:p></o:p></pre>
<pre>* PyData/Sparse: <a href="https://sparse.pydata.org/">https://sparse.pydata.org/</a><o:p></o:p></pre>
<pre>* Xnd: <a href="https://xnd.readthedocs.io/">https://xnd.readthedocs.io/</a><o:p></o:p></pre>
<pre>* Astropy's Quantity: <a href="https://docs.astropy.org/en/stable/units/">https://docs.astropy.org/en/stable/units/</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Existing and potential consumers of alternative arrays<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* Dask: <a href="https://dask.org/">https://dask.org/</a><o:p></o:p></pre>
<pre>* scikit-learn: <a href="https://scikit-learn.org/">https://scikit-learn.org/</a><o:p></o:p></pre>
<pre>* xarray: <a href="https://xarray.pydata.org/">https://xarray.pydata.org/</a><o:p></o:p></pre>
<pre>* TensorLy: <a href="http://tensorly.org/">http://tensorly.org/</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Existing alternate dtype implementations<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* ``ndtypes``: <a href="https://ndtypes.readthedocs.io/en/latest/">https://ndtypes.readthedocs.io/en/latest/</a><o:p></o:p></pre>
<pre>* Datashape: <a href="https://datashape.readthedocs.io">https://datashape.readthedocs.io</a><o:p></o:p></pre>
<pre>* Plum: <a href="https://plum-py.readthedocs.io/">https://plum-py.readthedocs.io/</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Implementation<o:p></o:p></pre>
<pre>--------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The implementation of this NEP will require the following steps:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* Implementation of ``uarray`` multimethods corresponding to the<o:p></o:p></pre>
<pre> NumPy API, including classes for overriding ``dtype``, ``ufunc``<o:p></o:p></pre>
<pre> and ``array`` objects, in the ``unumpy`` repository.<o:p></o:p></pre>
<pre>* Moving backends from ``unumpy`` into the respective array libraries.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>``uarray`` Primer<o:p></o:p></pre>
<pre>~~~~~~~~~~~~~~~~~<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>**Note:** *This section will not attempt to go into too much detail about<o:p></o:p></pre>
<pre>uarray, that is the purpose of the uarray documentation.* [1]_<o:p></o:p></pre>
<pre>*However, the NumPy community will have input into the design of<o:p></o:p></pre>
<pre>uarray, via the issue tracker.*<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>``unumpy`` is the interface that defines a set of overridable functions<o:p></o:p></pre>
<pre>(multimethods) compatible with the numpy API. To do this, it uses the<o:p></o:p></pre>
<pre>``uarray`` library. ``uarray`` is a general purpose tool for creating<o:p></o:p></pre>
<pre>multimethods that dispatch to one of multiple different possible backend<o:p></o:p></pre>
<pre>implementations. In this sense, it is similar to the ``__array_function__``<o:p></o:p></pre>
<pre>protocol but with the key difference that the backend is explicitly installed<o:p></o:p></pre>
<pre>by the end-user and not coupled into the array type.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Decoupling the backend from the array type gives much more flexibility to<o:p></o:p></pre>
<pre>end-users and backend authors. For example, it is possible to:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* override functions not taking arrays as arguments<o:p></o:p></pre>
<pre>* create backends out of source from the array type<o:p></o:p></pre>
<pre>* install multiple backends for the same array type<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This decoupling also means that ``uarray`` is not constrained to dispatching<o:p></o:p></pre>
<pre>over array-like types. The backend is free to inspect the entire set of<o:p></o:p></pre>
<pre>function arguments to determine if it can implement the function e.g. ``dtype``<o:p></o:p></pre>
<pre>parameter dispatching.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Defining backends<o:p></o:p></pre>
<pre>^^^^^^^^^^^^^^^^^<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>``uarray`` consists of two main protocols: ``__ua_convert__`` and<o:p></o:p></pre>
<pre>``__ua_function__``, called in that order, along with ``__ua_domain__``.<o:p></o:p></pre>
<pre>``__ua_convert__`` is for conversion and coercion. It has the signature<o:p></o:p></pre>
<pre>``(dispatchables, coerce)``, where ``dispatchables`` is an iterable of<o:p></o:p></pre>
<pre>``ua.Dispatchable`` objects and ``coerce`` is a boolean indicating whether or<o:p></o:p></pre>
<pre>not to force the conversion. ``ua.Dispatchable`` is a simple class consisting<o:p></o:p></pre>
<pre>of three simple values: ``type``, ``value``, and ``coercible``.<o:p></o:p></pre>
<pre>``__ua_convert__`` returns an iterable of the converted values, or<o:p></o:p></pre>
<pre>``NotImplemented`` in the case of failure.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>``__ua_function__`` has the signature ``(func, args, kwargs)`` and defines<o:p></o:p></pre>
<pre>the actual implementation of the function. It recieves the function and its<o:p></o:p></pre>
<pre>arguments. Returning ``NotImplemented`` will cause a move to the default<o:p></o:p></pre>
<pre>implementation of the function if one exists, and failing that, the next<o:p></o:p></pre>
<pre>backend.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Here is what will happen assuming a ``uarray`` multimethod is called:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>1. We canonicalise the arguments so any arguments without a default<o:p></o:p></pre>
<pre> are placed in ``*args`` and those with one are placed in ``**kwargs``.<o:p></o:p></pre>
<pre>2. We check the list of backends.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> a. If it is empty, we try the default implementation.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>3. We check if the backend's ``__ua_convert__`` method exists. If it exists:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> a. We pass it the output of the dispatcher,<o:p></o:p></pre>
<pre> which is an iterable of ``ua.Dispatchable`` objects.<o:p></o:p></pre>
<pre> b. We feed this output, along with the arguments,<o:p></o:p></pre>
<pre> to the argument replacer. ``NotImplemented`` means we move to 3<o:p></o:p></pre>
<pre> with the next backend.<o:p></o:p></pre>
<pre> c. We store the replaced arguments as the new arguments.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>4. We feed the arguments into ``__ua_function__``, and return the output, and<o:p></o:p></pre>
<pre> exit if it isn't ``NotImplemented``.<o:p></o:p></pre>
<pre>5. If the default implementation exists, we try it with the current backend.<o:p></o:p></pre>
<pre>6. On failure, we move to 3 with the next backend. If there are no more<o:p></o:p></pre>
<pre> backends, we move to 7.<o:p></o:p></pre>
<pre>7. We raise a ``ua.BackendNotImplementedError``.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Defining overridable multimethods<o:p></o:p></pre>
<pre>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>To define an overridable function (a multimethod), one needs a few things:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>1. A dispatcher that returns an iterable of ``ua.Dispatchable`` objects.<o:p></o:p></pre>
<pre>2. A reverse dispatcher that replaces dispatchable values with the supplied<o:p></o:p></pre>
<pre> ones.<o:p></o:p></pre>
<pre>3. A domain.<o:p></o:p></pre>
<pre>4. Optionally, a default implementation, which can be provided in terms of<o:p></o:p></pre>
<pre> other multimethods.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>As an example, consider the following::<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> import uarray as ua<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> def full_argreplacer(args, kwargs, dispatchables):<o:p></o:p></pre>
<pre> def full(shape, fill_value, dtype=None, order='C'):<o:p></o:p></pre>
<pre> return (shape, fill_value), dict(<o:p></o:p></pre>
<pre> dtype=dispatchables[0],<o:p></o:p></pre>
<pre> order=order<o:p></o:p></pre>
<pre> )<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> return full(*args, **kwargs)<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> @ua.create_multimethod(full_argreplacer, domain="numpy")<o:p></o:p></pre>
<pre> def full(shape, fill_value, dtype=None, order='C'):<o:p></o:p></pre>
<pre> return (ua.Dispatchable(dtype, np.dtype),)<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>A large set of examples can be found in the ``unumpy`` repository, [8]_.<o:p></o:p></pre>
<pre>This simple act of overriding callables allows us to override:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* Methods<o:p></o:p></pre>
<pre>* Properties, via ``fget`` and ``fset``<o:p></o:p></pre>
<pre>* Entire objects, via ``__get__``.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Examples for NumPy<o:p></o:p></pre>
<pre>^^^^^^^^^^^^^^^^^^<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>A library that implements a NumPy-like API will use it in the following<o:p></o:p></pre>
<pre>manner (as an example)::<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> import numpy.overridable as unp<o:p></o:p></pre>
<pre> _ua_implementations = {}<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> __ua_domain__ = "numpy"<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> def __ua_function__(func, args, kwargs):<o:p></o:p></pre>
<pre> fn = _ua_implementations.get(func, None)<o:p></o:p></pre>
<pre> return fn(*args, **kwargs) if fn is not None else NotImplemented<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> def implements(ua_func):<o:p></o:p></pre>
<pre> def inner(func):<o:p></o:p></pre>
<pre> _ua_implementations[ua_func] = func<o:p></o:p></pre>
<pre> return func<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> return inner<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> @implements(unp.asarray)<o:p></o:p></pre>
<pre> def asarray(a, dtype=None, order=None):<o:p></o:p></pre>
<pre> # Code here<o:p></o:p></pre>
<pre> # Either this method or __ua_convert__ must<o:p></o:p></pre>
<pre> # return NotImplemented for unsupported types,<o:p></o:p></pre>
<pre> # Or they shouldn't be marked as dispatchable.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> # Provides a default implementation for ones and zeros.<o:p></o:p></pre>
<pre> @implements(unp.full)<o:p></o:p></pre>
<pre> def full(shape, fill_value, dtype=None, order='C'):<o:p></o:p></pre>
<pre> # Code here<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Backward compatibility<o:p></o:p></pre>
<pre>----------------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>There are no backward incompatible changes proposed in this NEP.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Alternatives<o:p></o:p></pre>
<pre>------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The current alternative to this problem is a combination of NEP-18 [2]_,<o:p></o:p></pre>
<pre>NEP-13 [4]_ and NEP-30 [9]_ plus adding more protocols (not yet specified)<o:p></o:p></pre>
<pre>in addition to it. Even then, some parts of the NumPy API will remain<o:p></o:p></pre>
<pre>non-overridable, so it's a partial alternative.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The main alternative to vendoring ``unumpy`` is to simply move it into NumPy<o:p></o:p></pre>
<pre>completely and not distribute it as a separate package. This would also achieve<o:p></o:p></pre>
<pre>the proposed goals, however we prefer to keep it a separate package for now,<o:p></o:p></pre>
<pre>for reasons already stated above.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The third alternative is to move ``unumpy`` into the NumPy organisation and<o:p></o:p></pre>
<pre>develop it as a NumPy project. This will also achieve the said goals, and is<o:p></o:p></pre>
<pre>also a possibility that can be considered by this NEP. However, the act of<o:p></o:p></pre>
<pre>doing an extra ``pip install`` or ``conda install`` may discourage some users<o:p></o:p></pre>
<pre>from adopting this method.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Discussion<o:p></o:p></pre>
<pre>----------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>* ``uarray`` blogpost: <a href="https://labs.quansight.org/blog/2019/07/uarray-update-api-changes-overhead-and-comparison-to-__array_function__/">https://labs.quansight.org/blog/2019/07/uarray-update-api-changes-overhead-and-comparison-to-__array_function__/</a><o:p></o:p></pre>
<pre>* The discussion section of NEP-18: <a href="https://numpy.org/neps/nep-0018-array-function-protocol.html#discussion">https://numpy.org/neps/nep-0018-array-function-protocol.html#discussion</a><o:p></o:p></pre>
<pre>* NEP-22: <a href="https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html">https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html</a><o:p></o:p></pre>
<pre>* Dask issue #4462: <a href="https://github.com/dask/dask/issues/4462">https://github.com/dask/dask/issues/4462</a><o:p></o:p></pre>
<pre>* PR #13046: <a href="https://github.com/numpy/numpy/pull/13046">https://github.com/numpy/numpy/pull/13046</a><o:p></o:p></pre>
<pre>* Dask issue #4883: <a href="https://github.com/dask/dask/issues/4883">https://github.com/dask/dask/issues/4883</a><o:p></o:p></pre>
<pre>* Issue #13831: <a href="https://github.com/numpy/numpy/issues/13831">https://github.com/numpy/numpy/issues/13831</a><o:p></o:p></pre>
<pre>* Discussion PR 1: <a href="https://github.com/hameerabbasi/numpy/pull/3">https://github.com/hameerabbasi/numpy/pull/3</a><o:p></o:p></pre>
<pre>* Discussion PR 2: <a href="https://github.com/hameerabbasi/numpy/pull/4">https://github.com/hameerabbasi/numpy/pull/4</a><o:p></o:p></pre>
<pre>* Discussion PR 3: <a href="https://github.com/numpy/numpy/pull/14389">https://github.com/numpy/numpy/pull/14389</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>References and Footnotes<o:p></o:p></pre>
<pre>------------------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [1] uarray, A general dispatch mechanism for Python: <a href="https://uarray.readthedocs.io">https://uarray.readthedocs.io</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [2] NEP 18 — A dispatch mechanism for NumPy’s high level array functions: <a href="https://numpy.org/neps/nep-0018-array-function-protocol.html">https://numpy.org/neps/nep-0018-array-function-protocol.html</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [3] NEP 22 — Duck typing for NumPy arrays – high level overview: <a href="https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html">https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [4] NEP 13 — A Mechanism for Overriding Ufuncs: <a href="https://numpy.org/neps/nep-0013-ufunc-overrides.html">https://numpy.org/neps/nep-0013-ufunc-overrides.html</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [5] Reply to Adding to the non-dispatched implementation of NumPy methods: <a href="http://numpy-discussion.10968.n7.nabble.com/Adding-to-the-non-dispatched-implementation-of-NumPy-methods-tp46816p46874.html">http://numpy-discussion.10968.n7.nabble.com/Adding-to-the-non-dispatched-implementation-of-NumPy-methods-tp46816p46874.html</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [6] Custom Dtype/Units discussion: <a href="http://numpy-discussion.10968.n7.nabble.com/Custom-Dtype-Units-discussion-td43262.html">http://numpy-discussion.10968.n7.nabble.com/Custom-Dtype-Units-discussion-td43262.html</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [7] The epic dtype cleanup plan: <a href="https://github.com/numpy/numpy/issues/2899">https://github.com/numpy/numpy/issues/2899</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [8] unumpy: NumPy, but implementation-independent: <a href="https://unumpy.readthedocs.io">https://unumpy.readthedocs.io</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [9] NEP 30 — Duck Typing for NumPy Arrays - Implementation: <a href="https://www.numpy.org/neps/nep-0030-duck-array-protocol.html">https://www.numpy.org/neps/nep-0030-duck-array-protocol.html</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>.. [10] <a href="http://scipy.github.io/devdocs/fft.html#backend-control">http://scipy.github.io/devdocs/fft.html#backend-control</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Copyright<o:p></o:p></pre>
<pre>---------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>This document has been placed in the public domain.<o:p></o:p></pre>
<p class="MsoNormal">_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
<o:p></o:p></p>
</div>
</body>
</html>