<div dir="ltr">Hello All, <div><br></div><div>Yesterday I opened PR <a href="https://github.com/numpy/numpy/pull/4998" target="_blank">#4889</a> to solve a problem I have been having w.r.t. xdress and Nathaniel asked me bring the issue up here. The PR itself is quite small (6 lines?) and is easy to review.</div>


<div><br></div><div>The opening text of my PR is pasted below because I believe that is a pretty good description of the issue. But briefly, pulling user defined dtypes out of an array do not behave idiomatically because you get a numpy scalar rather than a more representative Python object. For user-defined dtypes - which are typically more complex and possibly stateful than the builtin dtypes, I believe that it makes much more sense to get actual Python representation back a la the getitem() function.</div>


<div><br></div><div>In fact, I think that this case also applies to the object dtype. However, changing that usage would likely break downstream code and would be inconsistent with how other builtin types are returned. In future major versions of numpy it would be ideal if the dtypes themselves could flag how they wished to be returned - either as a scalar or as the Python item.</div>


<div><br></div><div>Thoughts?</div><div><br></div><div>Be Well</div><div>Anthony</div><div><br></div><div><p style="margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px;margin-top:0px!important">

This updates what is effectively the <code style="font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;padding:0.2em 0px;margin:0px;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;background-color:rgba(0,0,0,0.0392157)">__getitem__()</code> method. For arrays such that the dtype is a user defined type, you receive the return that dtype's <code style="font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;padding:0.2em 0px;margin:0px;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;background-color:rgba(0,0,0,0.0392157)">getitem()</code> rather than a numpy scalar of the dtype. This allow the custom type to present a single Python API as well as an associated dtype. It also prevents users from having to subclass ndarray to get the appropriate behaviour.</p>

<p style="margin-top:0px;margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px">For example, suppose that we have a dtype representing a C++ <code style="font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;padding:0.2em 0px;margin:0px;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;background-color:rgba(0,0,0,0.0392157)">std::vector<int></code> and then we had a numpy array of this dtype. From Python, it might look like</p>

<div class="" style="margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px;overflow:visible!important;background-image:initial;background-repeat:initial">

<pre style="overflow:auto;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;margin-top:0px;margin-bottom:0px;padding:16px;line-height:1.45;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;word-wrap:normal;word-break:normal;background-color:rgb(247,247,247)">

<span class="" style="font-weight:bold">>>></span> <span class="">arr</span>
<span class="">array</span><span class="">([</span><span class="">array</span><span class="">([</span><span class="" style="color:rgb(0,153,153)">0</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">0</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">0</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">0</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">0</span><span class="">],</span> <span class="">dtype</span><span class="" style="font-weight:bold">=</span><span class="">int32</span><span class="">),</span>
       <span class="">array</span><span class="">([</span><span class="" style="color:rgb(0,153,153)">0</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">1</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">2</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">3</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">4</span><span class="">],</span> <span class="">dtype</span><span class="" style="font-weight:bold">=</span><span class="">int32</span><span class="">),</span>
       <span class="">array</span><span class="">([</span><span class="" style="color:rgb(0,153,153)">0</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">2</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">4</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">6</span><span class="">,</span> <span class="" style="color:rgb(0,153,153)">8</span><span class="">],</span> <span class="">dtype</span><span class="" style="font-weight:bold">=</span><span class="">int32</span><span class="">)],</span> <span class="">dtype</span><span class="" style="font-weight:bold">=</span><span class="" style="color:rgb(221,17,68)">'xd_vector_int'</span><span class="">)</span>
</pre></div><p style="margin-top:0px;margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px">Without this PR, you'd have to do the following to access the most deeply nested elements:</p>

<div class="" style="margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px;overflow:visible!important;background-image:initial;background-repeat:initial">

<pre style="overflow:auto;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;margin-top:0px;margin-bottom:0px;padding:16px;line-height:1.45;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;word-wrap:normal;word-break:normal;background-color:rgb(247,247,247)">

<span class="" style="font-weight:bold">>>></span> <span class="">arr</span><span class="" style="font-weight:bold">.</span><span class="">item</span><span class="">(</span><span class="" style="color:rgb(0,153,153)">2</span><span class="">)[</span><span class="" style="color:rgb(0,153,153)">4</span><span class="">]</span>
<span class="" style="color:rgb(0,153,153)">8</span>
</pre></div><p style="margin-top:0px;margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px">This is because you cannot index a scalar:</p>

<div class="" style="margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px;overflow:visible!important;background-image:initial;background-repeat:initial">

<pre style="overflow:auto;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;margin-top:0px;margin-bottom:0px;padding:16px;line-height:1.45;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;word-wrap:normal;word-break:normal;background-color:rgb(247,247,247)">

<span class="" style="font-weight:bold">>>></span> <span class="">arr</span><span class="">[</span><span class="" style="color:rgb(0,153,153)">2</span><span class="">][</span><span class="" style="color:rgb(0,153,153)">4</span><span class="">]</span>
<span class="" style="color:rgb(153,0,0);font-weight:bold">IndexError</span><span class="">:</span> <span class="">invalid</span> <span class="">index</span> <span class="">to</span> <span class="">scalar</span> <span class="">variable</span>
</pre></div><p style="margin-top:0px;margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px">With this PR, the idiomatic expression is now allowable because <code style="font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;padding:0.2em 0px;margin:0px;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;background-color:rgba(0,0,0,0.0392157)">arr[2]</code> is the associated Python type:</p>

<div class="" style="margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px;overflow:visible!important;background-image:initial;background-repeat:initial">

<pre style="overflow:auto;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;margin-top:0px;margin-bottom:0px;padding:16px;line-height:1.45;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;word-wrap:normal;word-break:normal;background-color:rgb(247,247,247)">

<span class="" style="font-weight:bold">>>></span> <span class="">arr</span><span class="">[</span><span class="" style="color:rgb(0,153,153)">2</span><span class="">][</span><span class="" style="color:rgb(0,153,153)">4</span><span class="">]</span>
<span class="" style="color:rgb(0,153,153)">8</span>
</pre></div><p style="margin-top:0px;margin-bottom:16px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px">This is a pretty big deal for <a href="http://xdress.org/" style="color:rgb(65,131,196);text-decoration:none;background:transparent">xdress</a> which creates many custom dtypes and provided a Python interface into those. See <a href="https://github.com/xdress/xdress/pull/265" class="" title="demo of xd vv error" style="color:rgb(65,131,196);text-decoration:none;background:transparent">xdress/xdress#265</a> for what prompted this.</p>

<p style="margin-top:0px;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif;font-size:14px;line-height:22.399999618530273px;margin-bottom:0px!important">Thanks for considering!</p>

</div></div>