Operator execution in pandas 2.0
I spent a little time brainstorming about the design of operators (anything from arithmetic, to math operations, string operations, etc.) in pandas 2.0 and the use cases we'll want to be able to support: https://docs.google.com/document/d/1YmsV48iO6YNSxCIC84Xig5z-i9g4g7rzYTmCgUAx... Comments and other ideas welcome. There's no hard code yet, but I wanted to identify some of the requirements before setting down to hack. For backwards compatibility, most functions in Series and DataFrame will keep doing what they've been doing: eager evaluation. We can make some of them faster through multithreading (similar to what numexpr does in some cases). If and when we create a small deferred expression API for a supported subset of pandas operations, we'll be able to avail ourselves of other kinds of reasonably straightforward optimizations (like reusing temporary arrays between operations rather than allocating new ones). In any case, I'm hopeful this will help inform the implementation of libpandas. - Wes
participants (1)
-
Wes McKinney