[pytest-dev] Proposal: python namespacign for markings/marker objects
Ronny Pfannschmidt
opensource at ronnypfannschmidt.de
Thu Oct 20 08:55:35 EDT 2016
HI Bruno
On 20.10.2016 14:20, Bruno Oliveira wrote:
> On Thu, Oct 20, 2016 at 7:12 AM Ronny Pfannschmidt
> <opensource at ronnypfannschmidt.de
> <mailto:opensource at ronnypfannschmidt.de>> wrote:
>
>
> On 20.10.2016 10:56, holger krekel wrote:
> > Hey Ronny,
> >
> > i
> >> while trying to turn various internal markers of a work project
> >> into public plugins, i noticed a very plain problems -
> different other
> >> plugins used the same generic marker name for different
> purposes/intents
> >>
> >> such a flat namespace really doesn't scale
> > The more plugins there are the more it causes potential clashes
> of marker and fixture names. We've had some discussions at the
> sprint about it IIRC. I suggest that whatever namespacing we come
> up with it should be a) backward-compatible b) work for both
> markers and fixtures.
> python packages are pretty perfect for name-spacing markers, and its
> backward compatible
>
>
> Perhaps we should clearly state the points of Ronny's proposal so we
> can have a more structured discussion. Also, I suggest we try to
> discuss the proposal using an example which we are all familiar with,
> like the "skipif" marker. This also lets us see how we could
> eventually replace the current marker's implementation in the core
> (hopefully improving them by making them easier to understand) while
> also maintaining backward compatibility.
>
> * Problem 1: Flat namespace for markers
>
> This seems to be the main point of Ronny's proposal, the fact that a
> flat namespace for markers doesn't scale, with the potential of
> different plugins using markers with the same name causing problems
> and confusion.
>
> Ronny's proposal: import and use "marker objects"
>
> from myplugin import SkipIf
>
> @pytest.mark(SkipIf(sys.platform != 'win32', reason='win32 only'))
> def test_foo():
> pass
>
> * Problem 2: How plugins use markers and handle their arguments
>
> Currently plugins using markers have to deal with the *args and
> **kwargs parameters themselves, which is often messy and done incorrectly.
>
> Currently:
>
> def pytest_collection_modifyitems(items):
> for item in items:
> m = item.getmarker('skipif')
> condition = m.args[0] if skip.args else m.kwargs['condition']
> ...
>
> If getmarker also supports a `type` as parameter, this becomes possible:
>
> def pytest_collection_modifyitems(items):
> for item in items:
> skipif = item.getmarker(SkipIf)
> m.condition # naturally available as an attribute
> ...
>
> IMO the API `item.getmarker(x)` can be kept backward compatible by
> accepting either a mark "global name" or a marker `type`. More on how
> to declare/register markers below.
>
> Some questions:
>
> 1. How to declare those markers?
>
> a) One of Ronny's suggestion is a new entry point: not entirely clear
> to me how it would be done. I personally don't like this idea, entry
> points are used to declare *plugins* only, and plugins use other
> mechanisms to declare fixtures, markers, hooks, etc. I don't think we
> should overload this.
>
> b) Another suggestion from Ronny: running collection and getting used
> marker names. I'm not entirely sure what this means exactly, since the
> original proposal doesn't use "names" at all, only the objects directly.
in my proposal the "name" of a marker would be either the current string
or a type
>
> One mechanism for declaring markers that I believe we discussed in the
> sprint was to provide a new hook which could be used to declare new
> markers. The hook would return a dict of marker names and opaque
> types, which could then be accessed by plugins using the current
> `item.getmarker(name)` API. For example, using "skipif":
>
> class SkipIf:
> def __init__(self, condition, *, reason=None):
> pass
>
> The hook would look like this:
>
> def pytest_register_markers(config):
> return {'skipif': SkipIf}
>
> This is similar to what Floris mentioned:
>
> @pytest.marker
> def skipif(condition, *, reason=None):
> return SkipIf(condition, reason=reason)
>
> In both cases, `@pytest.mark.skipif` would instantiate a new `SkipIf`
> instance with the parameters given and make it available to items
> using the `item.getmarker(name)` API.
another important part of my proposal is, that i want to decouple from
the namespace we put into pytest.mark,
after all there is now a multitude of plugins that register certain
markers, sometimes for *different* usages and incompatible signatures
>
> I really like both the hook idea as well as the `@pytest.marker`
> decorator as a way to improve how the current markers work because
> they fix the "handle *args and **kwargs" problem nicely.
>
> This of course doesn`t address the "flat namespace" problem, but I
> think they both can co-exist.
>
> 2. How to declare markers at the module and class level.
>
> Ronny suggests using the `pytestmark` mechanism for that, which I
> think works and is backward compatible.
>
> I would like to comment that marks are often used in test suites (as
> opposed to in plugins) to mark some tests for test selection using
> `-m`, for example `@pytest.mark.integration` or `@pytest.mark.units`.
> In this use case, the user usually doesn't pass any arguments and
> doesn't formally declare markers. But I think this is still compatible
> with the proposal so far, because `item.getmarker` in those cases
> could return a general `Marker(*args, **kwargs)` object when
> `pytest.mark.integration` is called and no "integration" marker has
> been registered.
>
> ---
>
> So far I think we can address both the "flat namespace" and "argument
> handling" aspects of markers in a compatible way, which can even be
> implemented and deployed separately in a backward compatible way.
>
> IMHO the discussion by email at this point is a little hard to digest
> and to track all points/replies/proposals. Perhaps we should move this
> discussion to a different venue? I propose an issue on GitHub because
> the main issue containing the proposal can be updated as the
> discussion progresses, although I'm not sure if it would be any easier
> to track the discussion itself. Perhaps using the Wiki would also be
> possible?
>
i could make a wiki page
cheers, Ronny
> Cheers,
> Bruno.
>
>
> as for fixture namespaces, if we base them in plain string names and
> continue doing so,
> i feel absolutely certain it will break in a messy way and it has
> different referencing needs than markers
>
> so making 2 different things use the same mechanism is a guarantee
> that
> we end up in a broken mess :(
>
> i'd much rather start a discussion about using different means
> even for
> fixtures,
> but thats for a different topic that i hope to start this weekend (and
> it would solve lot of massive headaches wrt getting rid of
> py.path.local)
>
>
> >> as such i would like to propose having marker types and objects as
> >> importable objects
> >>
> >>
> >> for example
> >>
> >>
> >>
> >> import pytest
> >> from pytest_bugzilla import Blocker
> >>
> >>
> >> @pytest.mark(Blocker(123))
> >> def test_fun():
> >> pass
> >>
> >> that way we can do both, reuse python name-spacing *and* use more
> >> meaningful objects for markings
> > a few questions:
> >
> > - How would you integrate this with interactive help such as
> "py.test --markers"?
> i see 2 sensible paths
> a) a new entry-point to make them discover-able
> b) running collection and getting a set of the used marker names
>
> >
> > - how would you access the Blocker marker from a hook? New API?
> basically a mark name is either a identifier or a type
>
> getmarker(Blocker) ->mark collection of Blocker objects
>
> this api break is needed anyway because what we have now is
> demonstrably broken and unusable
> in various situations, usage and parameters we get different
> kinds of
> objects with different behaviors
>
> >
> > - how would you apply it at module or class level?
> pytestmark = [ Blocker(123) ]
>
>
> >
> > holger
> > _______________________________________________
> > pytest-dev mailing list
> > pytest-dev at python.org <mailto:pytest-dev at python.org>
> > https://mail.python.org/mailman/listinfo/pytest-dev
> -- Ronny
>
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org <mailto:pytest-dev at python.org>
> https://mail.python.org/mailman/listinfo/pytest-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20161020/1d920575/attachment-0001.html>
More information about the pytest-dev
mailing list