[pytest-dev] plugin status page

[holger krekel]

Hi Bruno,

On Wed, Oct 23, 2013 at 21:51 -0200, Bruno Oliveira wrote:
> Hi Holger, all,
> 
> On Sun, Oct 20, 2013 at 11:06 AM, Bruno Oliveira <nicoddemus at gmail.com>wrote:
> 
> >
> > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel <holger at merlinux.eu> wrote:
> >
> >>
> >> What about generating a "plugin details " page for each plugin so we don't
> >> have to think too hard about what to put in the index page?
> >>
> >
> > That's a possibility, for sure.
> >
> >
> >> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the
> >> script to automate the download/unpack/test/post-results steps?
> >>
> >
> > I didn't use devpi at that point because of my limited knowledge of devpi
> > at the moment, but that is certainly the next step.
> >
> > I will work on that next, and also work on generating a default tox.ini
> > file for plugins that don't have one as you suggested.
> >
> 
> I have updated the script to create a tox.ini file that just uses "py.test
> --help" as a test command for those packages that don't have one. Now the
> script tests all pytest plugins in pypi, please take a look at the summary
> at the bottom of the page:
> 
> https://www.travis-ci.org/nicoddemus/pytest-plugs/jobs/12959702

looks good!  I guess having a py27 and py33 row would be great
to also determine py3 compatibility/installability.

> I guess it would be useful to identify which plugins are using just
> "py.test --help" as smoke test and which ones have real tests, since the
> former doesn't guarantee that the plugin actually works on that
> python/pytest combination.
> 
> I looked into devpi to simplify the download/unpack/test process and
> stumbled in a minor roadblock: is there anyway to provide a tox.ini file
> for those plugins that don't have one? This is the only issue I see right
> now with using devpi for this particular application.

Now, but i am going to add a "devpi test -c path-to-ini" option today
and plan to release devpi-1.2 real soon now.  I'll get back here when it's
implemented.
 
> >  | > I guess we can implement in devpi a concept similar to travis build
> > status
> >
> >>  > imagens, which we could then use in the plugins_index page.
> >> >
> >> > Against python 2.6 the script is failing, I have yet to investigate.
> >>
> >
> So, what do you think of the idea of using devpi.net to host test results
> and add an endpoint to the API to provide test status images? Or should I
> create another app to do this?

What do you mean with "test status images"?
In general i am open to extending devpi-server to provide more test-status
info -- just didn't have a chance to think about it in detail yet.
I guess we could take the plugin testing bits as a starting point.

best,
holger

> Cheers,
> Bruno.
> 
> 
> > I found the reason for that (http vs https) and now the job runs against
> > Python 2.6, 2.7, 3.2 and 3.2.
> >
> > Thanks!  Got back but caught a flue ...
> >>
> >
> > It happens, hope you get better soon. :)
> >
> > Cheers,
> > Bruno.
> >
> >
> > On Sun, Oct 20, 2013 at 3:52 AM, holger krekel <holger at merlinux.eu> wrote:
> >
> >> On Mon, Oct 14, 2013 at 23:52 -0300, Bruno Oliveira wrote:
> >> > > > >
> >> > > > > - i'd collapse NAME and VERSION columns to save space, i.e.
> >> > > > >   "pytest-bdd-0.6.1".
> >> > > > >
> >> > > > > - what about adding download numbers?
> >> > > > >
> >> > > > > - as to code organisation: you can leave it as is for now or
> >> (maybe
> >> > > > >   better) put all code and generated files into a dedicated
> >> directory
> >> > > >
> >> > >
> >> >
> >> > Done, opened a PR for discussion and reviewing. :)
> >>
> >> only got to it now, looks good so far!
> >>
> >> > > > > - we should try to collect repository locations. maybe parsing
> >> > > > >   for github/bitbucket urls would yield most of them
> >> automatically?
> >> > > > >   Maybe we need to add some manually.
> >> > > > >
> >> > > >
> >> > > > May I ask why we would need the repository locations? I mean, to
> >> work on
> >> > > > the compatibility feature we can work directly with packages in
> >> pypi or
> >> > > > devpi...
> >> > >
> >> > > purely for the person reading the page and wanting to look at the code
> >> > > or do a PR.
> >> > >
> >> >
> >> > Oh I see! Well, parsing the urls seems problematic, because we also
> >> have to
> >> > take in account the username ("hpk42/pytest" for example). I looked at
> >> the
> >> > GitHub and BitBucket's rest APIs to see if we could use them for
> >> searching,
> >> > but it seems only GitHub at the moment has a search API (and in beta
> >> stage
> >> > at that).
> >> >
> >> > I think we will have to maintain the repository list manually. Package
> >> > authors can help as well by simply opening PRs adding repositories for
> >> > their plugins I guess.
> >>
> >> What about generating a "plugin details " page for each plugin so we don't
> >> have to think too hard about what to put in the index page?
> >>
> >> > > A very interesting bit will be the "2.4.2 compat" and "dev" compat
> >> > > > <snip>
> >> > >
> >> >
> >> > Some late night quick hacking and I have come up with this proof of
> >> concept:
> >> >
> >> > https://www.travis-ci.org/nicoddemus/pytest-plugs
> >> >
> >> > As it is right now it serves only as a prove of concept, of course. The
> >> > idea is using travis to handle running things for us, driving it using a
> >> > script that downloads and run tests using tox. We can then collect json
> >> > information from that and POST those results back to devpi. Also we can
> >> use
> >> > its build matrix to test against different pytest versions, for example.
> >>
> >> Hum, interesting.  Is this mainly to make use of travis-ci's build hosts?
> >> What is the advantage over just invoking "devpi test PLUGIN-NAME" from the
> >> script to automate the download/unpack/test/post-results steps?
> >>
> >> > I guess we can implement in devpi a concept similar to travis build
> >> status
> >> > imagens, which we could then use in the plugins_index page.
> >> >
> >> > Against python 2.6 the script is failing, I have yet to investigate.
> >> >
> >> > great, i am travelling to Pycon DE tomorrow morning but should be online
> >> > >  from time to time.
> >> > >
> >> >
> >> > Nice tripe and good PyCon!
> >>
> >> Thanks!  Got back but caught a flue ...
> >>
> >> cheers,
> >> holger
> >>
> >
> >