[pytest-dev] plugin status page

[Bruno Oliveira]

Hi Holger, all,

On Tue, Nov 12, 2013 at 7:25 AM, holger krekel <holger at merlinux.eu> wrote:

> > I'm thinking we can use those URLs to embed status images for the latest
>  > pytest version into the plugin index page, similar to the summary table
> at
> > the start of the pytest-plugs heroku app.
> >
> > As I noted earlier, the tests reported for pytest-2.3.5 are actually for
> > 2.4.2 since I didn't overwrite the environment in the tox.ini files. I
> will
> > try the approach of replacing the "pytest" dependency manually, unless
> you
> > have another suggestion.
>
> Are the tests run via creating a ``tox.ini`` yourself, overriding any
> tox.ini that might be contained with the package?
>

I'm creating a tox.ini file if there is none, or if there is already a
tox.ini inplace I try to use a simple regex to replace "pytest" occurrences
to "pytest==$PYTEST_VERSION" in an attempt to enforce the pytest version
that should be used for running the tests. This has worked for most
plugins, but some of them setup the dependencies in a slightly different
manner than what my simple regex-replace trick could handle:

* Some packages declare a specific pytest a version as dependency, like
"pytest==2.4.0" or "pytest>=2.0".
* One of the packages installed the dependencies from a requirements file,
using "-r requirements-tests.txt" instead of listing the dependencies on
the tox.ini file itself.

I could of course make my regex more complex in an attempt to cover all the
cases, but I feel this would mean a lot of duplicated work that is already
done by tox itself. Any suggestions?

In any case, it would be useful to everyone (plugin authors, users,
> pytest core devs) to see what is actually breaking (or working), i.e.
> the actual test failure/success log.  Is that available somewhere already?
>

Hmm good point. It is available from the travis-ci output log, but that is
not very friendly. I will change the travis-ci jobs to also post the
results from "tox --result-json" which contains all the details from that
test session, and make that available from the webpage as well.

So I'm thinking this should be the next steps:

1. Update plugins index page to include the status images;
2. Post test results to the pytest-plugs page and make it available from
there;
3. Find a nice way to ensure we are running "tox" with the pytest version
we want;

About 3), I just had an idea... what if we install a devpi server that
would serve only the pytest version we want (say, 2.3.5) and force tox to
use that devpi server by using the "-i" option? Would that be possible?
("Shadowed Release Files" seems to be what would be required for this to
work).

Cheers,
Bruno.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20131112/f71794c7/attachment.html>