[Distutils] test coverage for setuptools

[David Lyon]

> On Mon, Jan 4, 2010 at 2:53 PM, anatoly techtonik <techtonik at gmail.com>
> wrote:

>>>> Where can I see test coverage results for setuptools package?
>>>> I've got two tracebacks in last week after upgrading and that seems to
>>>> be too much.
>>>> I'd like to see if there are any tests for failing piece to see why
>>>> they might fail before digging further.

We, as in the python community aren't currently doing any
'all-of-python-packages' tests. Like CPAN does, with CPANTS. I think we
tried with
python pants but it got laughed off the planet (kindof sad - as it
was fully in keeping with monty python - chuckle)

Back to serious, in a way, setuptools should be like every other
package on pypi and have code/test analysis run on it.

I've just recently started resurrecting cheesecake putting together a
testbot to do such reports on all the packages on pypi. I'm trying to
correct the output section so that it outputs to a template engine
(cheetah) so the results are less noisy.

The URL is http://bitbucket.org/djlyon/pypi-package-testbot/

In a few weeks, when it is working a bit better I will move it to it's own
domain and publish the reports.

My idea is to build a celery based server farm and use various analysis
tools to measure everything and produce nice reports such as you have
linked to in your email.

It's no trivial task however. There's so many gotcha's in python packaging
that just installing all the packages is a bumpy process.

>> By test coverage results I mean publicly available reports about
>> percentage of code covered by tests that were run by testing
>> buildbots. See examples:
>> http://www.eclipse.org/pde/pde-api-tools/coverage/20091126/eclemma.html
>> http://eigenclass.org/static/rcov-sample-report/
>> http://web.archive.org/web/20070403133248/http://seb.dbzteam.com/pub/urllib2.py.html

We need stuff in python like that.. there's no doubt.

As Tarek says, coverage looks like a good tool. I'd never
seen it till he mentioned it.

If you want to break my bikeshed by joining in you (anybody)
would be more than welcome. I have the time and facilities to
run a testbot but there are specific areas like html reporting
and so forth where I really could use some help. If you need
bribing in beer (well heck - it worked for CPAN), that can
be arranged too. I live in a country with endless beer supplies
and posting to the world is easy..

Regards

David