[AstroPy] ValueError in vstacking tables

Fri Jun 19 08:43:48 EDT 2015

Hi Brian,

Could you let us know what table1.dtype and table3.dtype are equal to,
and how many rows are in each table?

Thanks!
Tom

On 18 June 2015 at 16:34, Brian Cherinka <cherinka at di.utoronto.ca> wrote:
> Anymore thoughts on this?  My table length issue wasn’t the problem.  I’m
> stilling having a problem with my resulting hstacks and vstacks.  The stacks
> happen without error.  I just lose the ability to index by row.
>
> Cheers, Brian
>
> ---------------------------------------
> Brian Cherinka, Ph.D
> Dunlap Institute for Astronomy & Astrophysics
> University of Toronto
> Toronto, ON, Canada M5S 3H4
> phone: 416-978-7299
> email: cherinka at di.utoronto.ca
> ----------------------------------------
>
> On Jun 17, 2015, at 3:25 PM, Brian Cherinka <cherinka at di.utoronto.ca> wrote:
>
> Hi Thomas,
>
> Your example does indeed work for me.  I cannot yet guess why it doesn’t
> work in my full example.  However it’s not quite a complete reconstruction
> of what I’m doing.  Technically, I’m hstacking two tables together, and then
> vstacking that result with another table.  Something like this, which works
> just fine.
>
> In [382]: t1 = Table([[1], [2]], names=['a', 'b'])
>
> In [383]: t2 = Table([[3], [4]], names=['c','d'])
>
> In [384]: t3 = hstack([t1,t2],join_type='outer')
>
> In [386]: t4 = Table([[5], [6]], names=['a', 'b'])
>
> In [387]: t34 = vstack([t3,t4],join_type='outer')
>
> In [388]: t34
> Out[388]:
> <Table masked=True length=2>
>   a     b     c     d
> int64 int64 int64 int64
> ----- ----- ----- -----
>     1     2     3     4
>     5     6    --    --
>
> In [389]: t34[0]
> Out[389]:
> <Row 0 of table
>  values=(1, 2, 3, 4)
>  dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8'), ('d', '<i8')]>
>
> In my full example, I did find though that I can row index the individual
> tables t1 and t2, but cannot row index table t3 after hstacking.  I get the
> same ValueError.  When I attempt to vstack with table t4, that error
> propagates forward.  I can row index t4 but not t34.  Looking at it a bit
> more, it might be that my rows in the initial tables t1, t2 do not match up
> with one another, and I appear to have one extra row in table t2, than I
> should.   I’ll dig a bit more.
>
> Cheers, Brian
>
> ---------------------------------------
> Brian Cherinka, Ph.D
> Dunlap Institute for Astronomy & Astrophysics
> University of Toronto
> Toronto, ON, Canada M5S 3H4
> phone: 416-978-7299
> email: cherinka at di.utoronto.ca
> ----------------------------------------
>
> On Jun 17, 2015, at 2:20 PM, Aldcroft, Thomas
> <aldcroft at head.cfa.harvard.edu> wrote:
>
> Hi Brian,
>
> Below is what I think is a minimal example of what you are describing.  Can
> you look at this and first reproduce that this works for you, and second,
> can you guess what might be different in your full example vs. my minimal
> example?
>
> In [19]: t1 = Table([[1], [2]], names=['a', 'b'])
>
> In [20]: t2 = Table([[3], [4]], names=['b', 'c'])
>
> In [23]: t12 = vstack([t1, t2], join_type='outer')
>
> In [24]: t12
> Out[24]:
> <Table masked=True length=2>
>   a     b     c
> int64 int64 int64
> ----- ----- -----
>     1     2    --
>    --     3     4
>
> In [25]: t12[0]
> Out[25]:
> <Row 0 of table
>  values=(1, 2, --)
>  dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')]>
>
>
> On Wed, Jun 17, 2015 at 2:08 PM, Brian Cherinka <cherinka at di.utoronto.ca>
> wrote:
>>
>> Hi,
>>
>> I’m trying to vstack (vertically stack) two Astropy tables with different
>> numbers of columns.  The tables appear to stack correctly without error.
>> However, upon attempting to index a row in the table, it throws me this
>> error
>>
>> <repr(<astropy.table.row.Row at 0x10c72ed50>) failed: ValueError: setting
>> an array element with a sequence.>
>>
>> I’m not trying to do anything fancy with mixed columns, or Quantities.
>> Just stackin’ two good ol’ natural Tables together.     One table has 84
>> columns, and another has 57, which are all a part of the 84 in the first
>> table.  At first glance I would guess it’s a mismatch in the dtypes of a
>> particular column that I’m trying to merge.  However, the columns all have
>> the same dtype, format, and they are in the same order.  I get no messages
>> regarding any kind of merge errors.
>>
>> Any ideas on this?
>>
>> Thanks, Brian
>>
>> P.S. Full print of traceback below
>>
>> table3 = vstack([table1, table3],join_type=‘outer’)
>> print table3[0]
>>
>> ---------------------------------------------------------------------------
>> ValueError                                Traceback (most recent call
>> last)
>> <ipython-input-278-a0c75e8d3ef3> in <module>()
>> ----> 1 print tmp3[0]
>>
>> /Users/Brian/anaconda/lib/python2.7/site-packages/astropy/table/row.pyc in
>> __repr__(self)
>>     176     def __repr__(self):
>>     177         return "<{3} {0} of table\n values={1!r}\n
>> dtype={2}>".format(
>> --> 178             self.index, self.as_void(), self.dtype,
>> self.__class__.__name__)
>>     179
>>     180
>>
>> /Users/Brian/anaconda/lib/python2.7/site-packages/numpy/ma/core.pyc in
>> __repr__(self)
>>    5699             return self._data.__repr__()
>>    5700         m = tuple(m)
>> -> 5701         if not any(m):
>>    5702             return self._data.__repr__()
>>    5703         p = masked_print_option
>>
>> /Users/Brian/anaconda/lib/python2.7/site-packages/numpy/ma/core.pyc in
>> __call__(self, a, *args, **params)
>>    6089         method = getattr(MaskedArray, method_name, None)
>>    6090         if method is not None:
>> -> 6091             return method(MaskedArray(a), *args, **params)
>>    6092         # Still here ? OK, let's call the corresponding np
>> function
>>    6093         method = getattr(np, method_name)
>>
>> /Users/Brian/anaconda/lib/python2.7/site-packages/numpy/ma/core.pyc in
>> __new__(cls, data, mask, dtype, copy, subok, ndmin, fill_value, keep_mask,
>> hard_mask, shrink, **options)
>>    2649         """
>>    2650         # Process data............
>> -> 2651         _data = np.array(data, dtype=dtype, copy=copy, subok=True,
>> ndmin=ndmin)
>>    2652         _baseclass = getattr(data, '_baseclass', type(_data))
>>    2653         # Check that we're not erasing the mask..........
>>
>> ValueError: setting an array element with a sequence.
>>
>>
>>
>>
>> ---------------------------------------
>> Brian Cherinka, Ph.D
>> Dunlap Institute for Astronomy & Astrophysics
>> University of Toronto
>> Toronto, ON, Canada M5S 3H4
>> phone: 416-978-7299
>> email: cherinka at di.utoronto.ca
>> ----------------------------------------
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>