[soc2008-general] unit testing the standard library

[Brett Cannon]

On Tue, Mar 25, 2008 at 3:10 PM, Christian Heimes <lists at cheimes.de> wrote:
> Brett Cannon schrieb:
>
>  > Possibly. We need:
>  >
>  > * A function that imports the module being tested so that if it isn't
>  > imported the test is skipped, but all other import failures are an
>  > error.
>  > * Some way to specify what OSs the test can be run on, both expected
>  > and optionally.
>  > * A way to denote whitebox tests specific for CPython.
>  > * Rewrite regrtest to use all of this new stuff.
>  >
>  > That's everything I ever had an idea for. Most of it, though, is not
>  > overly complex (maybe the regrtest rewrite since that file has grown
>  > organically).
>
>  I also like to see:
>
>  * a way to execute a single test easily, for example
>
>  $ python regrtest.py test_sys:SysModuleTest.test_original_displayhook
>
>  * A simple way to record all failing unit tests and re-test just the
>  failing test functions
>
>  $ python regrtest.py --record
>  $ python regrtest.py --replay
>

Also having the feature to replay all tests run in the same order,
fail or not, is useful when a random test order is used and passing
tests play a role in the failure (but that should be the exception not
the default).

>  * a bullet proof and reliable infrastructure to set up socket based and
>  threaded tests

Michael Foord and Jerry Seutter were working on some mock
implementations of sockets at PyCon this year. Should be on the
tracker somewhere.

>
>  * infrastructure for common operations like creating and removing a file

E.g., a context manager that creates a file and then deletes it upon exit.

-Brett