Modeling functions with attributes

Consider this decorator from django: def csrf_exempt(view_func): """Mark function as being exempt from the CSRF view protection.""" # wrapping not strictly necessary here but is good practice @wraps(view_func) def wrapper_view(*args, **kwargs): return view_func(*args, **kwargs) wrapper_view.csrf_exempt = True # set function attribute return wrapper_view This kind of pattern is quite common in Python since PEP 232 added the ability to assign attributes to functions. But how should this be typed? If we annotate the return type simply as "Callable", we lose the information that `.csrf_exempt` has been set. The current way to annotate this is to use a callback protocol: P = ParamSpec("P") R = TypeVar("R", covariant=True) class CSRFExempt(Protocol[P, R]): csrf_exempt: bool # here is the attribute def __call__(self, *args: P.args, **kwargs: P.kwargs) -> R: ... def csrf_exempt(view_func: Callable[P, R]) -> CSRFExempt[P, R]: @wraps(view_func) def wrapper_view(*args: P.args, **kwargs: P.kwargs) -> R: return view_func(*args, **kwargs) wrapper_view.csrf_exempt = True return cast(CSRFExempt[P, R], wrapper_view) This will result in the right types when decorating *functions*, but it will not result in the right types for *methods*. That is because callable classes are not bound as methods; only real functions are. But in our annotation, the return type of csrf_exempt() is not a function but a callable object. So, type checkers will mistakenly think that the function is not bound as a method when it actually is. You can try to solve this problem by making csrf_exempt() return a descriptor object when called on a method, but that leads to other problems as discussed here for example: https://mail.python.org/archives/list/typing-sig@python.org/thread/QZX5WN4NL... Besides, at runtime there aren't actually any descriptor objects involved, so it would be nice if we could actually describe what happens at runtime. My proposal is to introduce a new `FunctionProtocol` type (alternative names `AttributedFunction` or `FunctionWithAttributes`) to accurately describe *functions* (not callable objects) with attributes. For the above example, it would look like this: class CSRFExempt(FunctionProtocol[P, R]): # now `FunctionProtocol` csrf_exempt: bool @staticmethod def __call__(*args: P.args, **kwargs: P.kwargs) -> R: ... A FunctionProtocol describes only real functions that can optionally have attributes. It is similar to TypedDict in that it cannot be used at runtime in any way. So, in contrast to Protocol, not even subclasses of FunctionProtocol can be instantiated and there will be no @runtime_checkable decorator. One problem is that there is no syntax in Python for specifying attribues on a function in the definition of the function itself. That is, we have to write: def a() -> None: pass a.publish = True instead of something like def a() with (publish = True) -> None: pass so, type checkers will unfortunately have to do extra work in keeping track of attributes: class Publishable(FunctionProtocol): publish: bool @staticmethod def __call__() -> None: ... def a() -> None: pass a.publish = True # type checker has to note that an attribute was set f: Publishable = a # this assignment is valid This creates a kind of spooky action at a distance, but I don't see a good way to avoid this (except to introduce new syntax). The example from above then becomes: class CSRFExempt(FunctionProtocol[P, R]): csrf_exempt: bool @staticmethod def __call__(*args: P.args, **kwargs: P.kwargs) -> R: ... def csrf_exempt(view_func: Callable[P, R]) -> CSRFExempt[P, R]: @wraps(view_func) def wrapper_view(*args: P.args, **kwargs: P.kwargs) -> R: return view_func(*args, **kwargs) wrapper_view.csrf_exempt = True return wrapper_view # type checker can verify that this is valid And this decorator will then work as expected with methods as well as functions. FunctionProtocol will inherit all the attributes that functions have: ('__module__', '__name__', '__qualname__', '__doc__', '__annotations__') Just as with TypedDict, you can create subtypes, that add even more attributes. One could even re-use the NotRequired[] annotation from TypedDicts to express that an attribute might not be set (but if it was set, it would have the specified type). Callable objects are incompatible with FunctionProtocol: class SimpleFunc(FunctionProtocol): # you can use `@staticmethod` here or add a `self`; both are fine def __call__(self) -> None: ... class MyCallable: def __call__(self) -> None: print("Hello, world!") f: SimpleFunc = MyCallable() # type error! g: SimpleFunc = lambda: None # OK The main concern I see with this proposal is the spooky action at a distance that type checkers will have to implement. I can't really say how difficult it would be. Best, Thomas

IMO the best solution to this problem would be to use an Intersection type (which should hopefully be in 3.13 or 3.14). As it stands I'm not a big fan of adding new special forms/types to solve a very specific problem.

```py class HasCSRFExempt(Protocol): csrf_exempt: bool def csrf_exempt(view_func: Callable[P, R]) -> Callable[P, R] & HasCSRFExempt: view_func.csrf_exempt = True return view_func ``` From a static analysis point of view I'm not entirely convinced this would ever work without a cast though just cause this seems like a pain to verify (maybe someone can correct me on this though). To make this work with bound methods you would also need types.FunctionType to be subscriptable (and types.MethodType) but I think that's something that should be made possible at some point anyway.
participants (2)
-
James H-B
-
Thomas Kehrenberg