[Tutor] Adding a new row to the dataframe with datetime as index

[Asif Iqbal]

On Mon, May 21, 2018 at 4:59 PM, Peter Otten <__peter__ at web.de> wrote:

> Asif Iqbal wrote:
>
> > On Mon, May 21, 2018 at 9:28 AM, Peter Otten <__peter__ at web.de> wrote:
> >
> >> Asif Iqbal wrote:
> >>
> >> > Hi,
> >> >
> >> > I am trying to add a new row to a new date in the dataframe like below
> >> >
> >> >    df.loc['2018-01-24'] = [0,1,2,3,4,5]
> >> >
> >> > And I am getting the following error
> >> >
> >> >   ValueError: cannot set using a list-like indexer with a different
> >> length
> >> > than the value
> >> >
> >> > I do have the right number of columns and I can lookup a row by the
> >> > date
> >> >
> >> >   df.loc['2018-01-23']
> >> >
> >> >   df.shape
> >> >   (8034, 6)
> >> >
> >> >   df.index
> >> >   DatetimeIndex(['2018-01-23', '2018-01-22', '2018-01-19',
> >> >   '2018-01-18',
> >> >                '2018-01-17', '2018-01-16', '2018-01-12', '2018-01-11',
> >> >                '2018-01-10', '2018-01-09',
> >> >                ...
> >> >                '1986-03-25', '1986-03-24', '1986-03-21', '1986-03-20',
> >> >                '1986-03-19', '1986-03-18', '1986-03-17', '1986-03-14',
> >> >                '1986-03-13', '2018-01-24'],
> >> >               dtype='datetime64[ns]', name='date', length=8034,
> >> freq=None)
> >> >
> >> > Any idea how to add a new row to a new date?
> >>
> >> My experiments indicate that there may be multiple values with the same
> >> key:
> >>
> >> > >>> import pandas as pd
> >> >>> df = pd.DataFrame([[1,2], [3,4], [5,6], [7,8]], index=["a", "b",
> "a",
> >> "a"])
> >> >>> df.loc["a"]
> >>    0  1
> >> a  1  2
> >> a  5  6
> >> a  7  8
> >>
> >> [3 rows x 2 columns]
> >> >>> df.loc["a"] = [10, 20]
> >> Traceback (most recent call last):
> >>   File "<stdin>", line 1, in <module>
> >>   File "/usr/lib/python3/dist-packages/pandas/core/indexing.py", line
> 98,
> >> in
> >> __setitem__
> >>     self._setitem_with_indexer(indexer, value)
> >>   File "/usr/lib/python3/dist-packages/pandas/core/indexing.py", line
> >> 422,
> >> in _setitem_with_indexer
> >>     self.obj._data = self.obj._data.setitem(indexer, value)
> >>   File "/usr/lib/python3/dist-packages/pandas/core/internals.py", line
> >> 2396,
> >> in setitem
> >>     return self.apply('setitem', *args, **kwargs)
> >>   File "/usr/lib/python3/dist-packages/pandas/core/internals.py", line
> >> 2376,
> >> in apply
> >>     applied = getattr(blk, f)(*args, **kwargs)
> >>   File "/usr/lib/python3/dist-packages/pandas/core/internals.py", line
> >> 615,
> >> in setitem
> >>     raise ValueError("cannot set using a list-like indexer "
> >> ValueError: cannot set using a list-like indexer with a different length
> >> than the value
> >>
> >> If found two ways to resolve this,
> >>
> >> (1) the obvious, ensure that the lengths are the same:
> >>
> >> >>> df.loc["a"] = [[10, 20], [30, 40], [50, 60]]
> >> >>> df
> >>     0   1
> >> a  10  20
> >> b   3   4
> >> a  30  40
> >> a  50  60
> >>
> >> (2) pass the key as a tuple:
> >>
> >> >>> df.loc["a",] = [1000, 2000]
> >> >>> df
> >>       0     1
> >> a  1000  2000
> >> b     3     4
> >> a  1000  2000
> >> a  1000  2000
> >>
> >> [4 rows x 2 columns]
> >>
> >> I suspect that you want neither, and instead avoid duplicate keys.
> >
> >
> >
> > I want to overwrite the row
> >
> >   print ( df.loc['2018-01-24'] )
> >   2018-01-24 0.0 1.0 2.0 3.0 4.0 NaN
> >
> >
> >   df.loc['2018-01-24'] = [0,1,2,3,4,5]
> >   ValueError: cannot set using a list-like indexer with a different
> length
> > than the value
>
> Can you post a self-contained example, i. e. a small script that also
> creates a -- hopefully small -- DataFrame and then triggers the ValueError?
>
>
It is working after I ran a df = df.sort_index()

I was looping through new dates and feeding predicted data to new row for
next day, but I was going in the wrong direction.


-- 
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?