This post doesn't have any particular aim, just a design
report. I'm
curious if anyone else had done this and has experience to
share, or
maybe there's a nicer solution for this in Python.
### Motivation
In pytest we have plugin system where external plugins may
implement
various hooks. For the sake of example let's say a plugin
wants to
do something on each test ("item") setup and teardown:
def pytest_runtest_setup(item: pytest.Item) -> None:
...
def pytest_runtest_teardown(item: pytest.Item) ->
None: ...
One thing plugins often want to do is to store/associate some
data with
the item on setup, then use it on teardown.
The traditional way plugins did this was to use an attribute
on the
`item`:
def pytest_runtest_setup(item: pytest.Item) -> None:
item.my_plugin_attr = MyPluginAttrType(...)
def pytest_runtest_teardown(item: pytest.Item) ->
None:
use_my_attr(item.my_plugin_attr)
# Maybe
del item.my_plugin_attr
The problem with this is that type checkers are not aware of
`my_plugin_attr` and they complain. To make things clean for
type
checkers, and type safe (no casts/asserts) an alternative
solution is
needed.
### Rejected solution 1
We can add a dict to each item on the pytest side:
class Item:
def __init__(self, ...) -> None:
...
self.store: Final[Dict[str, ???]] = {}
which plugins will use as
def pytest_runtest_setup(item: pytest.Item) -> None:
item.store["my_plugin_attr"] = MyPluginAttrType(...)
This fixes the "undefined attribute" issue, but as the `???`
indicates,
it's still not good. We can either use `object` and force
casts/asserts,
or `Any` and lose type safety.
### Rejected solution 2
We can imagine a typing feature which allows a 3rd-party to
"overlay" a
type it did not define with extra fields/methods. I think
some languages
support this (TypeScript?). But it does not seem like a
desirable feature
to me.
### Solution we ended up with
We define a `Stash` type (because plugins "stash" their data
there), which
is used as follows:
# In pytest
class Item:
def __init__(self, ...) -> None:
...
self.stash: Final[Stash] = Stash()
# In plugin
my_plugin_attr: Final =
pytest.StashKey[MyPluginAttrType]()
def pytest_runtest_setup(item: pytest.Item) -> None:
item.stash[my_plugin_attr] = MyPluginAttrType(...)
def pytest_runtest_teardown(item: pytest.Item) ->
None:
use_my_attr(item.stash[my_plugin_attr])
# Maybe
del item.stash[my_plugin_attr]
Each plugin defines `StashKey`s for its attributes, and the
stash key
carries the attribute type which makes things type safe.
The `Stash` is defined like this (see [0] for full
implementation in pytest):
T = TypeVar("T")
D = TypeVar("D")
class StashKey(Generic[T]):
__slots__ = ()
class Stash:
__slots__ = ("_storage",)
def __init__(self) -> None:
self._storage: Dict[StashKey[Any], object] = {}
def __setitem__(self, key: StashKey[T], value: T)
-> None:
self._storage[key] = value
def __getitem__(self, key: StashKey[T]) -> T:
return cast(T, self._storage[key])
def __delitem__(self, key: StashKey[T]) -> None:
del self._storage[key]
# .. other mutable mapping methods -- same idea.
### Advantages
No undefined attributes. Type safe. The `StashKey`s are
namespaced
objects, which makes conflicts between plugins impossible,
unlike with
string keys.
### Disadvantages
Not standard, users are not familiar with idea and are
confused, try to
use it as a dict.
Less ergonomic, need to define the `StashKey`s at the module
level.
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: dev@reggx.eu