[Pandas-dev] [pydata] Sparse data structures in pandas: refactor - feedback welcome!

[Tom Augspurger]

Just to be clear, the current sparse datatframe stores each column independently. There’s no memory saving over a DataFrame or sparse columns.


________________________________
From: pydata at googlegroups.com on behalf of Pietro Battiston <me at pietrobattiston.it>
Sent: Friday, November 16, 2018 16:48
To: pydata at googlegroups.com; pandas-dev at python.org
Subject: Re: [pydata] Sparse data structures in pandas: refactor - feedback welcome!

Hi Joris,

thanks for the recap...

Il giorno ven, 16/11/2018 alle 21.07 +0100, Joris Van den Bossche ha
scritto:
> [...]
> - Since a normal pandas Series and DataFrame can hold sparse data,
> there may be no need for the dedicated SparseSeries and
> SparseDataFrame subclasses. Therefore, we are planning to deprecate
> those subclasses, and the specific sparse functionality will be
> accessible on normal Series/DataFrame with the `sparse` accessor.
>   However, this might have complications we didn't think about, so we
> need your feedback!

>From the last dev discussion I thought we had decided to not provide
(at least immediately) any actual replacement for SparseDataFrame class
(in the sense of supporting 2d sparse structures, i.e. skipping
columns). Unless I misunderstood, this is probably the change that
users (of sparse structures) should be most aware.

(On the other hand, it is true that in most cases transposing the
DataFrame will probably still allow for exactly the same data structure
...)

Pietro

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe at googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pandas-dev/attachments/20181117/797be0c4/attachment.html>