[Python-Dev] Conventions for functional tests, PyUnit?

[Walter Dörwald]

John J Lee wrote:

> On Tue, 9 Dec 2003, [ISO-8859-1] Walter D?rwald wrote:
> [...]
> 
>>>One problem, though: doesn't putting functional tests in Lib/test throw
>>>off Walter Dorwald's unit test coverage numbers?  Perhaps there should be
>>>a 'functional' resource for test_support.use_resources (so the tests can
>>>be run with -uall, -functional for coverage measurements)?
>>
>>What exactly do you mean with 'functional' tests? We certainly don't
>>need non-functional tests! ;) And why would additional test scripts
>>throw off the coverage numbers?
> 
> 
> http://c2.com/cgi/wiki?FunctionalTest
> 
> I had assumed you were measuring unit test coverage, but maybe there are
> lots of functional tests in Lib/test too.

What I'm doing is compiling the Python source with profiling options
and running Lib/test/regrtest.py via Lib/trace.py.

After that the script calls gcov for every .c file to get C coverage
info and checks the coverage files generated by trace.py to get Python
coverage info.

> BTW, I had a look at your coverage page, and it looked as if it would be
> useful and interesting, if I understood what all those little graphs &c.
> were. :-/

I guess a little explanation would help. There are four types of source
code lines:

1) Unknown: The status of the line can not be determined, because
    gcov or trace.py didn't generate a coverage file
    (color black)
2) Uncoverable: Can not be executed (Comment or empty line)
    (color grey)
3) Uncovered (but coverable): Has not been executed (but is executable)
    (color red)
4) Covered: Has been executed at least once during the test run.
    (color green)

For files:

1) Unknown: A coverage file hasn't been created.
2) Uncoverable: The file consists only of uncoverable lines
    (e.g. a file with comments only).
3) Uncovered: The file contains coverable lines, but none of
    these lines have been executed.
4) Covered: The file contains at least one covered line.


Note that the web application isn't exactly fast. The machine is
rather old and the Postgres database seems to have a few problems
with tables that contain 10 million records. :-/

Bye,
    Walter Dörwald