Getting Pandas NaT to propagate like NaN
Skip Montanaro
skip.montanaro at gmail.com
Tue Oct 30 08:59:05 EDT 2018
I'm trying to take the min and max of a couple Pandas Series objects in the
face of NaT. np.minimum and np.maximum work the way I want if the elements
are floats. For example:
>>> s1
0 0.0
1 1.8
2 3.6
3 5.4
dtype: float64
>>> s2
0 10.0
1 17.0
2 NaN
3 14.0
dtype: float64
>>> np.maximum(s1, s2)
0 10.0
1 17.0
2 NaN
3 14.0
dtype: float64
>>> np.minimum(s1, s2)
0 0.0
1 1.8
2 NaN
3 5.4
dtype: float64
This doesn't work if s1 and s2 are datetime64[ns] objects:
>>> s1
0 2199-12-31
1 2199-12-31
2 2199-12-31
3 2199-12-31
dtype: datetime64[ns]
>>> s2
0 NaT
1 2018-10-30
2 NaT
3 NaT
dtype: datetime64[ns]
>>> np.maximum(s1, s2)
0 2199-12-31
1 2199-12-31
2 2199-12-31
3 2199-12-31
dtype: datetime64[ns]
>>> np.minimum(s1, s2)
0 2199-12-31
1 2018-10-30
2 2199-12-31
3 2199-12-31
dtype: datetime64[ns]
After doing a bit of reading, I came to realize NaT is only approximately
NaN, the latter having a proper floating point representation. Further
reading suggested no simple way to have NaT "pollute" these comparisons.
What's the correct way to get NaT to propagate in min/max comparisons the
way NaN does in a floating point context?
Thanks,
Skip
More information about the Python-list
mailing list