[Numpy-discussion] How to concatenate 2 recarrays into a single recarray

[Isaac Gerg]

It looks like some of the problem I was seeing has to do with reading in
hdf5 files from matlab that contain strings for column types.  when they
are read in using h5py the type is reported as 'O' which is object.  The
problem is I can't reproduce the problem in small script because I can't
get hdf5 to create these files with 'O' type; I have to go through matlab.

On Fri, May 12, 2017 at 9:59 AM, Isaac Gerg <isaac.gerg at gergltd.com> wrote:

> I was able to fix my problem. One of the types in the recarray was of type
> object when it should have been S27.  Type object was the result of reading
> in a matlab generated hdf5 file which had a string for one of the matrix
> columns.  I converted this column manually to a S27.
>
> I am reverting my code back to where i had the systemerror to post.
>
> On Thu, May 11, 2017 at 5:17 PM, Isaac Gerg <isaac.gerg at gergltd.com>
> wrote:
>
>> Will do. I'm working on my side to get a solution that works.  It seems
>> that when i use append_field i dont get a recarray back and when i call
>> asrecarray=True, i get an error.  once i fix this.  i will replicate the
>> error and post to github.
>>
>> On Thu, May 11, 2017 at 5:07 PM, Eric Wieser <wieser.eric+numpy at gmail.com
>> > wrote:
>>
>>> Even if you solve your own problem, please do - a SystemError is 100% a
>>> mistake in numpy, and should never be raised from python code, even if you
>>> call a numpy function with the wrong inputs.
>>>
>>> Eric
>>>
>>>
>>> On Thu, 11 May 2017 at 19:35 Isaac Gerg <isaac.gerg at gergltd.com> wrote:
>>>
>>>> Sure.
>>>>
>>>> On Thu, May 11, 2017 at 2:31 PM, Eric Wieser <
>>>> wieser.eric+numpy at gmail.com> wrote:
>>>>
>>>>> If that's the case, can you file an issue on github along with a
>>>>> minimal example that produces the error, and the full stack trace?
>>>>>
>>>>> On Thu, 11 May 2017 at 19:29 Isaac Gerg <isaac.gerg at gergltd.com>
>>>>> wrote:
>>>>>
>>>>>> newtable = np.lib.recfunctions.merge_arrays([a, b], asrecarray=True)
>>>>>>
>>>>>> yeilds:
>>>>>>
>>>>>> builtins.SystemError: ..\Objects\dictobject.c:2054: bad argument to
>>>>>> internal function
>>>>>>
>>>>>> On Thu, May 11, 2017 at 2:02 PM, Benjamin Root <ben.v.root at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Check in numpy.recfunctions. I know there is merge_arrays() and
>>>>>>> stack_arrays(). I forget which one does what.
>>>>>>>
>>>>>>> Cheers!
>>>>>>> Ben Root
>>>>>>>
>>>>>>>
>>>>>>> On Thu, May 11, 2017 at 1:51 PM, Isaac Gerg <isaac.gerg at gergltd.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I'd prefer to stay in numpy land if possible.
>>>>>>>>
>>>>>>>> On Thu, May 11, 2017 at 1:17 PM, Isaac Xin Pei <isaacpei at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Check Pandas pd.concate ?
>>>>>>>>> On Thu, May 11, 2017 at 12:45 PM Isaac Gerg <
>>>>>>>>> isaac.gerg at gergltd.com> wrote:
>>>>>>>>>
>>>>>>>>>> I have 2 arrays, a and b which are rec arrays of length 10.  Each
>>>>>>>>>> array has 5 columns.
>>>>>>>>>>
>>>>>>>>>> I would like to combine all the columns into a single recarray
>>>>>>>>>> with 10 columns and length 10.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Isaac
>>>>>>>>>> _______________________________________________
>>>>>>>>>> NumPy-Discussion mailing list
>>>>>>>>>> NumPy-Discussion at python.org
>>>>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> NumPy-Discussion mailing list
>>>>>>>>> NumPy-Discussion at python.org
>>>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> NumPy-Discussion mailing list
>>>>>>>> NumPy-Discussion at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> NumPy-Discussion mailing list
>>>>>>> NumPy-Discussion at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> NumPy-Discussion mailing list
>>>>>> NumPy-Discussion at python.org
>>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20170512/02fc4426/attachment-0001.html>