[Numpy-discussion] Setting custom dtypes and 1.14

[Allan Haldane]

On 01/26/2018 03:38 PM, Chris Barker wrote:
> I was hoping it would dig down to the inner structures looking for a
> match to the dtype, rather than looking at the type of the top level. Oh
> well.
> 
> So yeah, not sure where you would go from tuple to list -- probably at
> the bottom level, but that may not always be unambiguous.

As I remember, numpy has some fairly convoluted code for array creation
which tries to make sense of various nested lists/tuples/ndarray
combinations. It makes a difference for structured arrays and object
arrays. I don't remember the details right now, but I know in some cases
the rule is "If it's a Python list, recurse, otherwise assume it is an
object array".

While numpy does try to be lenient, I think we should guide the user to
assume that if they want to specify a structured element, they should
only use a tuple or a structured scalar, and if they want to specify a
new dimension of elements, they should use a list. I expect less
headaches that way.

>     These points make me think that instead of a `.totuple` method, this
>     might be more suitable as a new function in np.lib.recfunctions.
> 
> I don't seem to have that module -- and I'm running 1.14.0 -- is this a
> new idea?

Sorry, I didn't specify it correctly. It is "numpy.lib.recfunctions".

It is actually quite old, but has never been officially documented. I
think that is because it has been considered "provisional" for a long
time. See https://github.com/numpy/numpy/issues/5008
https://github.com/numpy/numpy/issues/2805

I still hesitate to make it more official now, since I'm not sure
that structured arrays are yet bug-free enough to encourage more complex
uses. Also, the functions in that module encourage "pandas-like" use of
structured arrays, but I'm not sure they should be used that way. I've
been thinking they should be primarily used for binary interfaces
with/to numpy, eg to talk to C programs or to read complicated binary files.

> However, I didn't actually need tuples, I needed something I could pack
> into a stuctarray, and this does work, without the tolist:
> 
> full = np.array(zip(time, pack_last_axis(uv)), dtype=dt)
> 
> 
> So maybe that is the way to go.

Right, that was my feeling: That we didn't really need `.totuple`, what
we actually wanted is a special function for packing a
nonstructured-array as a structured-array.

> I'm not sure I'd have thought to look for this function, but what can
> you do?
> 
> Thanks for your attention to this,
> 
> -CHB
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
> 
> Chris.Barker at noaa.gov <mailto:Chris.Barker at noaa.gov>
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>