[Python-Dev] unittest Suggestions

[Olemis Lang]

> >
> > > The goal
> > > should be to produce something like a unittest-ng, distribute it via
> > > the Cheeseshop, and gather consensus around it for possible inclusion
> > > in Python 2.7/3.1.
> > >
> >

IMO test support in Python needs some enhancements. I also think there
are great libraries available today in Python (stdlib)... but you
know... other frameworks (e.g. JUnit, NUnit, ...) have evolved in
order to support more effective practices and also to satisfy new
requirements present in more complex test environments and ... and
improvements are really necessary.

That's why...

> unittest.asserts
> ----------------
>
> Currently all assertions reside as methods on the TestCase class. This
> means that all testing functions, including utility functions, need to be
> on a TestCase-derived class. This seems wrong to me, especially since it
> combines different concerns on a class: Testing the functionality of a
> SUT, running a test suite, and providing test functions.

I agree...

> This proposal
> will not try to separate the first two concerns, but only the third one.

I think this is cool...

> In the long term I think it's worthwhile to make tests work on any
> class, not only on classes derived from TestCase, but as I said, this is
> not part of this proposal.
>

I am not very sure about this... even though I love doctest, and you
can define that kind of tests in almost every class... but I think
this case is different since doctests have an additional value...
Users can quickly and easily see example usage while they are looking
for help.

IMO today (I would like you to make me change my mind) this could lead
to test code mixed with the functionality of (for example) a class...
and at least now I don't see the benefits.

> JUnit 4 moved the assertions to a static class in a separate module. In
> Python I'd propose to use functions in a separate module.

I agree 100% with this... However, I think that TestCase assert* (or
fail*, as you wish... that's another thread) methods should remain
available for a while so as to avoid breaking third party (and perhaps
stdlib) test code.

But conceptually I think this is ok. It'd be precise to evaluate how
much convinient it is in practice.

About the rest... I will have further comments once I see this... in action ;-).

> > There is already unittest, nose, py.test, trial... perhaps others I don't
> know
> > of. I fear writing yet another testing framework from the ground-up will
> lead to
> > more bikeshedding and less focussed discussion (see some testing-in-python
> > threads for an example :-)).
> >

I think there should be a consensus about which are the most relevant
improvements needed nowadays in this field. The weaknesses and strong
points. And also what will be considered standard and what will be
considered third party modules (because including every library code
available, or a particular framework as a whole, "might" not be a good
idea). And this does not mean that some code that is available and
distributed under "appropriate" terms will (can) not be included in
the future, nor that a new test framework ought to be built from
scratch. I mean that both kinds of decisions should be made
considering some kind of "requirements" or "milestones" which clearly
state what should (needs to) be done... and perhaps this could be the
first milestone to avoid chaos.

What I do think is that we need a mail list (or equivalent) dedicated
to discuss such things... because (again IMO) python-dev might not be
the best place to talk about these topics. Or what is the same...
there should be a place (preferently here in python.org) to talk about
this topics in detail, without interfering with the development of the
language itself.

The relevant decisions and ideas might be communicated somehow (e.g.
informational / ST / Process PEPs, guidelines) in order to "keep
focused".

> Incremental
>  improvements with backwards compatibility are a great thing.

I agree...

>  I very
>  strongly encourage that course of action.  It has already happened a number
>  of times in this thread that some proposed functionality already exists in
>  some third-party unittest extension and could easily be moved into the
>  stdlib unittest module.  That's a good thing: it shows that the
> functionality
>  is actually valuable and it makes it easy to include, since it's already
>  implemented.
>

Yes... this is right when "we all" agree on what "we" want and find
out that the library turns "our dreams" into "reality".


-- 
Regards,

Olemis.