[pytest-dev] (new) pytest plugins / note to plugin authors

[Bruno Oliveira]

Hi Holger,

On Fri, Oct 4, 2013 at 6:25 PM, holger krekel <holger at merlinux.eu> wrote:

> > Are you interested in displaying compatibility to several py.test
> versions,
> > or only to the latest one?
>
> ideally latest release and trunk ...
>

Fair enough. :)


> > and some way to re-generate this information (especially the test
> status).
> > > I guess we could start with considering plugins that have a tox.ini
> > > and then write a script to download the plugin, unpack it, run tests
> etc.
> > >
> >
> > I considered this, but then I realize we might have to implement several
> > features to support that, like installing dependencies: pytest-qt for
> > instance requires PySide, which must be installed for it to run its own
> > tests. Other plugins might have other requirements to run its tests (like
> > having a browser open for headless testing), which would have to be
> > supported as well and so on. I fear we would have to implement a lot of
> > infrastructure similar to what travis-ci.org or d
> > <http://crate.io>rone.iodoes for example.
> >
> > Perhaps we could adopt a solution similar to coveralls.io, in which the
> > author is responsible to upload coverage data, and the site collects and
> > displays that information. We could provide a script or code snippet to
> be
> > included by the plugin author in he's C.I of choice that would generate
> and
> > upload pytest-compatibility information. This could be easily hosted in a
> > PaaS like Heroku.
> >
> > We would have to count on plugin authors' contribution to the system for
> it
> > to work on this scheme, of course.
>
> I wonder how difficult most plugins really are to setup.
> PyQT might be difficult but OTOH many others should just be fine.
>
> And pushing the responsibility for performing tests and uploading results
> still would not get the benefit of testing current plugin releases
> against a new core pytest release (which can break many plugins whereas
> a new plugin release can just break that plugin :).


Yes, having to rely on package authors to update their CI system to use a
new pytest version would defeat the purpose of the status page. :/


> So i'd think it's best to check how many of the plugins even provide tests.
>

Hmm, of that I have no idea...

But even just collecting the other information i mentioned (without
> the test status / running questions) would already be helpful, btw.
>

That seems simple enough, a script that queries pypi and extracts this
information could be used to generate a page and/or statistics.

>From my understanding you would like a page to accomplish two things:

1. A list of known py-test plugins and their py27/py33 status.
2. If the plugins work against latest pytest and against trunk, so pytest
developers know if they're breaking existing plugins (intentionally or not).

Is that correct?

I'm trying to figure out a semi automatic way to accomplish both. :)

Cheers,
Bruno


> cheers,
> holger
>
> > The other part is how to present this info, but e.g. a CSV file
> > > can be rendered by sphinx and the main docs could include it
> > > or we make a separate page if need be.
> >
> >
> > > thanks for considering to help!
> > >
> >
> > Glad to. :)
> >
> > Best Regards,
> > Bruno
> >
> > >
> > > holger
> > >
> > > > Cheers,
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Oct 4, 2013 at 10:09 AM, holger krekel <holger at merlinux.eu>
> > > wrote:
> > > >
> > > > >
> > > > > I just did three pytest plugin releases to get out a few minor
> > > > > fixes and improvements. see below.  As far as i am aware most
> > > > > existing plugins (as well as the new releases here) should be
> > > > > compatible to pytest-2.4.2.  Some plugins like pytest-capturelog
> > > > > have seen their latest release three years ago but remain
> functional.
> > > > > A good sign that the plugin system is reasonable i guess.
> > > > >
> > > > > If someone wants to help with organizing plugin compatibility
> > > > > tests and maintaining a status page, i'd really appreciate that.
> > > > > With over 50 existing plugins it's not easy to stay on top.
> > > > > The current pytest.org page is only a start:
> > > > >
> > > > >     http://pytest.org/latest/plugins.html#extplugins
> > > > >
> > > > > and here is the full pypi list:
> > > > >
> > > > >
> > > > >
> > >
> https://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search
> > > > >
> > > > > there are also some github/bitbucket-only plugins, in addition.
> > > > >
> > > > > And finally a note to plugin authors: please be subscribed
> > > > > to pytest-dev and post new plugin releases here.
> > > > >
> > > > > best,
> > > > > holger
> > > > >
> > > > > pytest-xdist-1.9:
> > > > >
> > > > > - changed LICENSE to MIT
> > > > >
> > > > > - fix duplicate reported test ids with --looponfailing
> > > > >   (thanks Jeremy Thurgood)
> > > > >
> > > > > - fix pytest issue41: re-run tests on all file changes, not just
> > > > >   randomly select ones like .py/.c.
> > > > >
> > > > > - fix pytest issue347: slaves running on top of Python3.2
> > > > >   will set PYTHONDONTWRITEYBTECODE to 1 to avoid import concurrency
> > > > >   bugs.
> > > > >
> > > > > pytest-xprocess-0.8:
> > > > >
> > > > > - support python3 (tested on linux/win32)
> > > > >
> > > > > - split out pytest independent process support into xprocess.py
> > > > >
> > > > > - add info.kill() method and some smaller refactoring
> > > > >
> > > > > - fix various windows related Popen / killing details
> > > > >
> > > > > - add tests
> > > > >
> > > > >
> > > > > pytest-pep8:
> > > > >
> > > > > - use pytest-2.4.2 node.add_marker() API for adding "pep8" marker
> > > > >
> > > > > _______________________________________________
> > > > > Pytest-dev mailing list
> > > > > Pytest-dev at python.org
> > > > > https://mail.python.org/mailman/listinfo/pytest-dev
> > > > >
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20131004/e8510e32/attachment-0001.html>