[Patches] [ python-Patches-662807 ] Port tests to unittest

SourceForge.net noreply@sourceforge.net
Thu, 01 May 2003 10:58:07 -0700


Patches item #662807, was opened at 2003-01-05 21:50
Message generated for change (Comment added) made by doerwalter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=662807&group_id=5470

Category: Tests
Group: Python 2.3
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Walter Dörwald (doerwalter)
Summary: Port tests to unittest

Initial Comment:
This patch ports the three tests test_pow.py, 
test_charmapcodec.py and test_userdict.py to unittest.

----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-05-01 19:58

Message:
Logged In: YES 
user_id=89016

OK, I've changed the new run_unittest() to call run_suite()
with a second argument if exactly one testcase is passed. I
hope that's OK.

Checked in as:
test_StringIO.py 1.16
test___all__.py 1.30
test_base64.py 1.6
test_bisect.py 1.9
test_bool.py 1.12
test_builtin.py 1.19
test_bz2.py 1.14
test_calendar.py 1.5
test_call.py 1.4
test_cfgparser.py 1.20
test_charmapcodec.py 1.7
test_codeccallbacks.py 1.12
test_codecs.py 1.9
test_copy.py 1.8
test_cpickle.py 1.15
test_csv.py 1.6
test_dummy_thread.py 1.4
test_enumerate.py 1.4
test_filecmp.py 1.2
test_getargs2.py 1.5
test_grp.py 1.16
test_hexoct.py 1.4
test_hmac.py 1.7
test_isinstance.py 1.7
test_itertools.py 1.6
test_optparse.py 1.2
test_os.py 1.18
test_parser.py 1.17
test_pep277.py 1.6
test_pickle.py 1.18
test_posix.py 1.6
test_pow.py 1.18
test_profilehooks.py 1.10
test_pwd.py 1.17
test_re.py 1.42
test_richcmp.py 1.9
test_sets.py 1.24
test_shelve.py 1.5
test_shutil.py 1.2
test_socket.py 1.64
test_str.py 1.3
test_string.py 1.25
test_strptime.py 1.13
test_support.py 1.52
test_sys.py 1.7
test_tarfile.py 1.8
test_tempfile.py 1.14
test_textwrap.py 1.19
test_timeout.py 1.14
test_trace.py 1.9
test_ucn.py 1.14
test_unicode.py 1.83
test_unicodedata.py 1.9
test_univnewlines.py 1.5
test_urllib.py 1.13
test_urllibnet.py 1.2
test_userdict.py 1.15
test_userlist.py 1.9
test_userstring.py 1.11
test_weakref.py 1.23
test_xpickle.py 1.4
test_zipimport.py 1.8
test_zlib.py 1.23

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-05-01 19:17

Message:
Logged In: YES 
user_id=80475

My preference was to keep run_classtests() as a separate 
function.  The run_unittest() function was unique in that it 
called run_suite() with two arguments instead of one.  That 
accounts for the additional verbosity you are seeing.  While 
I prefer the separate run_classtests(), it's up to you to do 
whichever seems best.

Otherwise, all looks okay.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-05-01 18:54

Message:
Logged In: YES 
user_id=89016

I've changed test_support.run_unittest() to work like your 
run_classtests() and changed all test scripts to use this.
(see diff2.txt)

I compared the output of
./python Lib/test/regrtest.py -v 2>&1 | grep -v "Ran.*test.*in"
and apart from the fact that test_csv seems to be more verbose
now, there seem to be no regression bugs. Raymond, can you 
check, if this is OK?

BTW, there's another check for memory leaks in
test_datetime.py. Perhaps it would make sense to combine
both and use them globally in regrtest.py (or test_support.py)?

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-29 23:39

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/test_richcmp.py 1.8
Lib/test/output/test_richcmp delete
which a few additional tests and comments.

I'll probably work on using run_classtests() tomorrow, but
wouldn't it be simpler to add the multiple arguments to the
already existing function run_unittest() instead of
introducing the new run_classtests()?

> BTW, why is the int to int comparison labeled as useless?
Because it would check whether comparing two ints gives the
same result as comparing the same two ints. ;)


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-04-28 21:10

Message:
Logged In: YES 
user_id=80475

Yes.  test_support.run_classtests() ought to be used 
everywhere it is applicable.  There are a couple of places 
like test_sets.py where the "suite" variable is used again for 
other purposed (testing for memory leaks).  In those 
cases, it ought to be left alone until I can write a function 
that encapsulates the essense of the combined test.

test_richcmp.py looks good.  Please add comment lines 
describing the point of classes and tests whose names are 
not self explanatory (i.e. misbehavin, weird, etc).

BTW, why is the int to int comparison labeled as useless?

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-28 20:42

Message:
Logged In: YES 
user_id=89016

Here's the next one: test_richcmp.py with a few additional
tests to increase code coverage in object.c and listobject.c.

BTW, should we change all tests to use the new
test_support.run_classtests()?

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-25 13:40

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/test_bool.py 1.9

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-04-23 23:00

Message:
Logged In: YES 
user_id=6380

I suggest not to mess with unittest.py; it is still
externally maintained as PyUnit. Maybe ask the PyUnit developer.

No time to review the test, sorry; if it works, I suggest to
just check it in. ;-)

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-23 22:33

Message:
Logged In: YES 
user_id=89016

Here's the next one: test_bool.py ported to PyUnit with a
few additional tests. Code coverage for boolobject.c is at 100%.

Would it make sense to move assertIs() and assertIsNot() to
unittest.py?


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-15 13:30

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/output/test_pwd delete
Lib/test/test_pwd.py 1.13
Lib/test/test_grp.py 1.11
Lib/test/test___all__.py 1.29
(with
if __name == '__main__': 
      test_main()
added to test_pwd)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-04-15 01:09

Message:
Logged In: YES 
user_id=33168

Sorry, you are correct.  I removed the output/test_pwd and
the test passes.  This is on redhat 9 now.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-14 14:31

Message:
Logged In: YES 
user_id=89016

Neal, are you sure the error you get isn't just from the
output comparison? Can you post a traceback? test_pwd works
for me with Redhat 7.3.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-13 11:07

Message:
Logged In: YES 
user_id=89016

> question why do you use 3 lines in test_main,
> rather than the old way? --
> test_support.run_unittest(Test)

The example in Lib/test/README seems to suggest that it 
should be done this way.

I'll look into the test_pwd problem tomorrow.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-04-12 03:54

Message:
Logged In: YES 
user_id=33168

question why do you use 3 lines in test_main, rather than
the old way? -- test_support.run_unittest(Test)

test___all__ is fine
test_grp is fine
test_pwd fails for me on Redhat 7.2.  It also needs an
  if __name == '__main__': 
      test_main()


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-10 11:10

Message:
Logged In: YES 
user_id=89016

Here's an updated test_grp that executes tests equivalent to
those in test_pwd

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-04-10 10:46

Message:
Logged In: YES 
user_id=89016

Here's the next one: test_pwd

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-03-11 20:15

Message:
Logged In: YES 
user_id=89016

Here's the next one: test___all__.py ported to PyUnit and
updated.

A better solution might be to replace __builtin__.__import__
in regrtest.py and test for the __all__ attribute there.
Additionally this might allow us to check which modules are
imported by regrtest.py and which are not and require
additional tests.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-26 16:08

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/test_ucn.py 1.12
Lib/test/test_unicodedata.py 1.7
Lib/test/output/test_ucn delete
Lib/test/output/test_unicodedata delete


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-26 14:42

Message:
Logged In: YES 
user_id=38388

test_ucn and test_unicodedata look OK.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-25 18:53

Message:
Logged In: YES 
user_id=89016

OK, here are the next few ports: test_ucn and
test_unicodedata. I'm not actually sure, whether changing
test_unicodedata (which uses the comparison of generated
output with expected output) is a good thing, as now updates
to the database require manual changes. I've added a few
error checks which increase coverage in unicodedata.c from
87% to 95%.

Marc-André can you check if this is OK?

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 14:05

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/string_tests.py 1.27
Lib/test/test_str.py 1.1
Lib/test/test_string.py 1.24
Lib/test/test_unicode.py 1.79
Lib/test/test_userstring.py 1.10
Lib/test/output/test_string delete

I've removed the sets import and renamed the mixin tests to
contain the relevant class/module names (e.g.
MixinStrStringUserStringTest)

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 04:39

Message:
Logged In: YES 
user_id=80475

* test_string.py imports sets but does not use it.

* the names of the mixin classes could possibly
   be made clearer so I won't have to search into
   the comments to find-out which mixins are
   appropriate for each class.

Overall, it looks like a nice factoring job and ought
to go a long ways toward keeping these guys 
in sync in the future.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-17 19:29

Message:
Logged In: YES 
user_id=89016

Here is the next bunch of ports: the string tests have been
ported to PyUnit and made as reusable as possible. Tests are
now shared between str, unicode, UserString and the string
module. As a result of reusing a part of the unicode tests
for str, the coverage in stringobject.c goes from 83% to
86%. Furthermore it should help keep the API consistent
between str and unicode (Example: "%c" % 0xffffffff raises
OverflowError, u"%c" % 0xffffffff raises ValueError)

Raymond can look look through the scripts and check that
everything is OK?

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-16 10:33

Message:
Logged In: YES 
user_id=89016

I'm currently working on a PyUnit port of the string tests (i.e. 
str, unicode, UserString and the string module). Uploading 
the result to this patch would be easier, as it already has a 
establsihed audience: But I can open a new patch for that if 
you want.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-14 21:11

Message:
Logged In: YES 
user_id=33168

Walter, can this patch be closed now?

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-14 12:30

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/output/test_charmapcodec delete
Lib/test/test_charmapcodec.py 1.6


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-14 09:52

Message:
Logged In: YES 
user_id=38388

test_charmapcodec looks OK. Just remove
the DOS-lineends before checking it in.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-13 19:16

Message:
Logged In: YES 
user_id=89016

OK, checked in as test_userlist.py 1.7.

Assigned back to MAL for the review of test_charmapcodec.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 19:08

Message:
Logged In: YES 
user_id=6380

Walter, feel free to check in test_userlist.py!

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-13 19:02

Message:
Logged In: YES 
user_id=89016

Here's another one: test_userlist has been ported to PyUnit
and a few tests have been added to increase coverage.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-13 04:12

Message:
Logged In: YES 
user_id=33168

MAL, could you look at the test_charmapcodec.py?  I think
that's the only file outstanding from this patch.  It's a
pretty straightforward test.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 00:13

Message:
Logged In: YES 
user_id=89016

OK, test_sys.py is checked in.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 23:56

Message:
Logged In: YES 
user_id=6380

I think you can check this in -- if it fails with Jython,
Finn or Samuele will quickly patch it. :-)

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-03 23:44

Message:
Logged In: YES 
user_id=89016

OK, here's a new test_sys.py

> test_sys.py:
>
> - I agree that it's not worth testing the code 
> paths that will invoke a custom __displayhook__ or
> __excepthook__, but I regret it nevertheless. :-)
> maybe this deserves a comment?

Testing a custom displayhook is now done (via compile(...,
"single")/exec). Testing a custom excepthook seems to be
trickier. This could probably be done by calling the
interpreter recursively via os.system() or os.popen(). I've
added a comment for now that this isn't tested.
Unfortunately this leaves a large block in
Python/pythonrun.c uncovered.

> - sys.exit() should also be callable with a string

OK, done.

> - you could check that the value of the SystemExit exception
> has the right exit code

Done.

- Have you checked this with Jython? I don't know if it
implements all
   of these; in particular I doubt it has getrefcount().

I haven't tested Jython yet, but I guess test_sys.py will
have to many many exceptions for Jython. I'll try this tomorrow.

- I presume you've tested this on Windows?

Linux & Windows

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 22:10

Message:
Logged In: YES 
user_id=6380

test_sys.py:

- I agree that it's not worth testing the code paths that
will invoke a custom
   __displayhook__ or __excepthook__, but I regret it
nevertheless. :-)
   maybe this deserves a comment?

- sys.exit() should also be callable with a string

- you could check that the value of the SystemExit exception
has the right
   exit code

- Have you checked this with Jython? I don't know if it
implements all
   of these; in particular I doubt it has getrefcount().

- I presume you've tested this on Windows?

Sorry, I can't help you with charmapcodec


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-03 21:36

Message:
Logged In: YES 
user_id=89016

Here's a new one: test_sys.py tests Python/sysmodule.c.
Coverage goes from 68% to 77%.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-19 15:46

Message:
Logged In: YES 
user_id=80475

All are approved except test_charmapcodec.py -- 
someone else should look at that one.

Be sure to follow GvR's advice and replace assertEquals 
with assertEqual.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-16 21:47

Message:
Logged In: YES 
user_id=89016

test_unicode is ported and enhanced (coverage goes from
80.81% to 85.05%)

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-10 18:17

Message:
Logged In: YES 
user_id=89016

> In general, don't do tests that hardwire implementation 
details

So should we remove
self.assertEquals(reduce(42, "1"), "1")
self.assertEquals(reduce(42, "", "1"), "1")
from test_filter?

BTW, you should look at test_builtin first, as the others
are still simply ports to PyUnit.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-10 18:03

Message:
Logged In: YES 
user_id=80475

Good to hear the news on increasing the coverage.

In general, don't do tests that hardwire implementation 
details.  Test it if it is a documented variable,  exposed 
through __all__, is a key constact (like the magic numbers 
in random.py), or a variable that a module user is likely to 
be relying upon.  Otherwise, no -- it should be possible to 
improve an implementation without crashing the suite.

I'll try to review a few of these over the next few days.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-10 17:53

Message:
Logged In: YES 
user_id=89016

test_builtin.py is now updated to test more error
situations. This increases the coverage of bltinmodule.c
from 75.13% to 92.20%, and it actually revealed one or two
potential bugs:
http://www.python.org/sf/665761 and
http://www.python.org/sf/665835

I'm not 100% sure that test_intern() and test_execfile() do
the right thing.

I'm not sure, whether the test script should check for
undocumented implementation artefacts, like:

a = 1
self.assert_(min(a, 1L) is a)

but in this way at least we get notified if something is
changed unintentionally.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-08 20:05

Message:
Logged In: YES 
user_id=89016

test_b1 and test_b2 are combined into test_builtin now

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-08 15:03

Message:
Logged In: YES 
user_id=6380

Two random suggestions:

- a blank line before each method, even trivial ones, even
the first one

- use assertEqual, not assertEquals

BTW, I see you've picked up on the convention that unit test
methods should not have doc strings. Good! (But they may
have comments.)

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-07 17:37

Message:
Logged In: YES 
user_id=89016

test_b1.py has been ported too.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-05 21:56

Message:
Logged In: YES 
user_id=89016

The patch is hard to read, so I'll upload all three test scripts.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=662807&group_id=5470