<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";
mso-fareast-language:EN-GB;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="DE" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Hello everyone, I’ve improved upon the content of NEP 31 to make it simpler, and also according to the new NEP template, only part of the NEP is being sent out to the mailing list. For the full
nep, please see <a href="https://github.com/numpy/numpy/pull/14793">PR 14793</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">============================================================<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">NEP 31 — Context-local and global overrides of the NumPy API<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">============================================================<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">:Author: Hameer Abbasi <habbasi@quansight.com><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">:Author: Ralf Gommers <rgommers@quansight.com><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">:Author: Peter Bell <pbell@quansight.com><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">:Status: Draft<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">:Type: Standards Track<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">:Created: 2019-08-22<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Abstract<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">--------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This NEP proposes to make all of NumPy's public API overridable via an<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">extensible backend mechanism.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Acceptance of this NEP means NumPy would provide global and context-local<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">overrides in a separate namespace, as well as a dispatch mechanism similar<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">to NEP-18 [2]_. First experiences with ``__array_function__`` show that it<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">is necessary to be able to override NumPy functions that *do not take an<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">array-like argument*, and hence aren't overridable via<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">``__array_function__``. The most pressing need is array creation and coercion<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">functions, such as ``numpy.zeros`` or ``numpy.asarray``; see e.g. NEP-30 [9]_.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This NEP proposes to allow, in an opt-in fashion, overriding any part of the<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">NumPy API. It is intended as a comprehensive resolution to NEP-22 [3]_, and<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">obviates the need to add an ever-growing list of new protocols for each new<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">type of function or object that needs to become overridable.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Motivation and Scope<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">--------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">The primary end-goal of this NEP is to make the following possible:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">.. code:: python<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # On the library side<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> import numpy.overridable as unp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> def library_function(array):<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> array = unp.asarray(array)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # Code using unumpy as usual<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> return array<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # On the user side:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> import numpy.overridable as unp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> import uarray as ua<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> import dask.array as da<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> ua.register_backend(da) # Can be done within Dask itself<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> library_function(dask_array) # works and returns dask_array<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> with unp.set_backend(da):<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> library_function([1, 2, 3, 4]) # actually returns a Dask array.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Here, ``backend`` can be any compatible object defined either by NumPy or an<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">external library, such as Dask or CuPy. Ideally, it should be the module<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">``dask.array`` or ``cupy`` itself.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">These kinds of overrides are useful for both the end-user as well as library<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">authors. End-users may have written or wish to write code that they then later<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">speed up or move to a different implementation, say PyData/Sparse. They can do<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">this simply by setting a backend. Library authors may also wish to write code<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">that is portable across array implementations, for example ``sklearn`` may wish<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">to write code for a machine learning algorithm that is portable across array<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">implementations while also using array creation functions.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This NEP takes a holistic approach: It assumes that there are parts of<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">the API that need to be overridable, and that these will grow over time. It<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">provides a general framework and a mechanism to avoid a design of a new<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">protocol each time this is required. This was the goal of ``uarray``: to<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">allow for overrides in an API without needing the design of a new protocol.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This NEP proposes the following: That ``unumpy`` [8]_ becomes the<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">recommended override mechanism for the parts of the NumPy API not yet covered<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">by ``__array_function__`` or ``__array_ufunc__``, and that ``uarray`` is<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">vendored into a new namespace within NumPy to give users and downstream<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">dependencies access to these overrides. This vendoring mechanism is similar<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">to what SciPy decided to do for making ``scipy.fft`` overridable (see [10]_).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">The motivation behind ``uarray`` is manyfold: First, there have been several<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">attempts to allow dispatch of parts of the NumPy API, including (most<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">prominently), the ``__array_ufunc__`` protocol in NEP-13 [4]_, and the<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">``__array_function__`` protocol in NEP-18 [2]_, but this has shown the need<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">for further protocols to be developed, including a protocol for coercion (see<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">[5]_, [9]_). The reasons these overrides are needed have been extensively<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">discussed in the references, and this NEP will not attempt to go into the<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">details of why these are needed; but in short: It is necessary for library<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">authors to be able to coerce arbitrary objects into arrays of their own types,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">such as CuPy needing to coerce to a CuPy array, for example, instead of<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">a NumPy array. In simpler words, one needs things like ``np.asarray(...)`` or<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">an alternative to "just work" and return duck-arrays.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Usage and Impact<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">----------------<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This NEP allows for global and context-local overrides, as well as<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">automatic overrides a-la ``__array_function__``.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Here are some use-cases this NEP would enable, besides the
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">first one stated in the motivation section:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">The first is allowing alternate dtypes to return their<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">respective arrays.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">.. code:: python<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # Returns an XND array<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> x = unp.ones((5, 5), dtype=xnd_dtype) # Or torch dtype<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">The second is allowing overrides for parts of the API.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This is to allow alternate and/or optimised implementations<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">for ``np.linalg``, BLAS, and ``np.random``.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">.. code:: python<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> import numpy as np<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> import pyfftw # Or mkl_fft<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # Makes pyfftw the default for FFT<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> np.set_global_backend(pyfftw)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # Uses pyfftw without monkeypatching<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> np.fft.fft(numpy_array)
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> with np.set_backend(pyfftw) # Or mkl_fft, or numpy<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">
</span><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"># Uses the backend you specified<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> np.fft.fft(numpy_array)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This will allow an official way for overrides to work with NumPy without<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">monkeypatching or distributing a modified version of NumPy.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Here are a few other use-cases, implied but not already<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">stated:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">.. code:: python<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> data = da.from_zarr('myfile.zarr')<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # result should still be dask, all things being equal<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> result = library_function(data)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> result.to_zarr('output.zarr')<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">This second one would work if ``magic_library`` was built<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">on top of ``unumpy``.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">.. code:: python<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> from dask import array as da<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> from magic_library import pytorch_predict<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> data = da.from_zarr('myfile.zarr')<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> # normally here one would use e.g. data.map_overlap<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> result = pytorch_predict(data)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> result.to_zarr('output.zarr')<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">Backward compatibility<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">----------------------<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">There are no backward incompatible changes proposed in this NEP.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>