Consider the following MWE ``` import pandas as pd print(pd.__version__) df = pd.DataFrame({"x": [1]}) class F: def __iter__(self): yield 0 def __call__(self, x): return x f = F() df["x"].apply(lambda e: f(e)) # ok df["x"].apply(f.__call__) # ok df["x"].apply(f) # AssertionError: f not callable ``` In Pandas < 1.3, the three calls all return the same value, namely a Series containing only [1]. Starting from Pandas 1.3, the last of the three calls raises an AssertionError. This is a backwards incompatible change that happened in 1.3. However, 1. I have realized that I should use Series.map rather than apply. This fixes my issue. 2. The documentation for Series.apply says that it expects a "Python function" or "Numpy ufunc". Apparently, it doesn't support "Python callables". The only output from Pandas is "AssertionError", and in the code (pandas/core/apply.py:1046 [1106 in trunk]), it simply says "assert callable(f)": ``` # string, list-like, and dict-like are entirely handled in super assert callable(f) ``` This is weird, since _my f_ clearly is callable. Would it be possible to at least add a string to the assert to make it easier for users to understand what's going on? Here is the output: ``` (e39) [~]$ python applytest.py 1.2.5 (e39) [~]$ pip install -q pandas==1.3 (e39) [~]$ python applytest.py 1.3.0 Traceback (most recent call last): File "/home/p/applytest.py", line 14, in <module> df["x"].apply(f) # AssertionError: f not callable File "/home/p/e39/lib/python3.9/site-packages/pandas/core/series.py", line 4356, in apply return SeriesApply(self, func, convert_dtype, args, kwargs).apply() [...] File "/home/p/e39/lib/python3.9/site-packages/pandas/core/apply.py", line 1046, in agg assert callable(f) AssertionError ``` Best regards, Pål GD
Thanks for reporting this Pål. I think it's probably better if you open an issue for this. I think it'll be easier to keep of all the discussion easily accessible, and not get this lost. The first thing that comes to my mind is is if callable(F()) returns True. I guess it does, then I wonder why assert callable(f) in the pandas code, I guess f is not the original parameter. Maybe you can have a look at that. Also, maybe git blame on that assert line points you to the breaking change, and the PR and issue give you an idea of why the change, and whether this was intentional. On Thu, Jun 30, 2022, 17:35 Pål Grønås Drange <paal.drange@gmail.com> wrote:
Consider the following MWE
``` import pandas as pd print(pd.__version__) df = pd.DataFrame({"x": [1]})
class F: def __iter__(self): yield 0 def __call__(self, x): return x
f = F() df["x"].apply(lambda e: f(e)) # ok df["x"].apply(f.__call__) # ok df["x"].apply(f) # AssertionError: f not callable ```
In Pandas < 1.3, the three calls all return the same value, namely a Series containing only [1].
Starting from Pandas 1.3, the last of the three calls raises an AssertionError.
This is a backwards incompatible change that happened in 1.3.
However, 1. I have realized that I should use Series.map rather than apply. This fixes my issue. 2. The documentation for Series.apply says that it expects a "Python function" or "Numpy ufunc". Apparently, it doesn't support "Python callables".
The only output from Pandas is "AssertionError", and in the code (pandas/core/apply.py:1046 [1106 in trunk]), it simply says "assert callable(f)":
``` # string, list-like, and dict-like are entirely handled in super assert callable(f) ```
This is weird, since _my f_ clearly is callable. Would it be possible to at least add a string to the assert to make it easier for users to understand what's going on?
Here is the output:
``` (e39) [~]$ python applytest.py 1.2.5 (e39) [~]$ pip install -q pandas==1.3 (e39) [~]$ python applytest.py 1.3.0 Traceback (most recent call last): File "/home/p/applytest.py", line 14, in <module> df["x"].apply(f) # AssertionError: f not callable File "/home/p/e39/lib/python3.9/site-packages/pandas/core/series.py", line 4356, in apply return SeriesApply(self, func, convert_dtype, args, kwargs).apply() [...] File "/home/p/e39/lib/python3.9/site-packages/pandas/core/apply.py", line 1046, in agg assert callable(f) AssertionError ```
Best regards, Pål GD _______________________________________________ Pandas-dev mailing list Pandas-dev@python.org https://mail.python.org/mailman/listinfo/pandas-dev
participants (2)
-
Marc Garcia -
Pål Grønås Drange