[ python-Bugs-1529297 ] unrelated variable messing up doctests
SourceForge.net
noreply at sourceforge.net
Fri Jul 28 02:46:11 CEST 2006
Bugs item #1529297, was opened at 2006-07-26 15:02
Message generated for change (Comment added) made by macquigg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1529297&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: David MacQuigg (macquigg)
Assigned to: Nobody/Anonymous (nobody)
Summary: unrelated variable messing up doctests
Initial Comment:
Version 2.5b2 with IDLE 1.2b2 running on Windows XP
Home, fully updated.
Unpack the attached zip file.
Open program.py, test_suite.py, in separate IDLE
windows.
Run test_suite.py.
8 passed and 0 failed.
Now add a variable at the top of program.py, any name
will do.
xyz = 2
Re-run test_suite.py.
7 passed and 1 failed !!
I haven't been able to isolate the problem, but it is
repeatable. Taking out blocks of unrelated statements
changes the behavior. Feels like a memory allocation
error, or something strange.
----------------------------------------------------------------------
>Comment By: David MacQuigg (macquigg)
Date: 2006-07-27 17:46
Message:
Logged In: YES
user_id=676422
That fixed it!! Nice catch.
I've been aware for some time of the sensitivity to order
in the doctests, but it has never been a problem until now,
and I've been writing programs with doctests since 2.3.
When I add a test, and it breaks a later test, it has
always been easy to nail down the piece that changed.
This "behind the scenes" change of test order seems like
the kind of problem that rarely occurs, but can cause
serious problems when it does.
Thanks again for an excellent followup.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2006-07-27 17:01
Message:
Logged In: YES
user_id=31435
DocTestFinder has been changed to sort test names again, in
revs 50882 (on the trunk, for 2.5) and 50886 (on the 2.4
branch, for 2.4.4).
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2006-07-27 16:21
Message:
Logged In: YES
user_id=31435
I think I know what's happening here. doctest was largely
rewritten for Python 2.4, and it lost a subtle intended
feature: it used to (<= 2.3) sort tests by name before
running them, but in 2.4 started running them in whatever
order they happen to appear in the hash-code-based module
__dict__.
Changing the namespace in program.py in seemingly random
ways matters because test_suite.py starts with
from program import *
and so everything in program.py's module namespace
"pollutes" test_suite.py's namespace too. That can change
the order tests run in. And as I noted before, whether or
not the doctest for _envrcpt() passes or fails depends on
whether or not the doctests for _hello() run before them,
due to side effects on the shared `ci` instance. Look at
your output carefully, and I expect you'll find that the
order of tests /does/ change depending on crap variable
names you stick in (or remove from) program.py.
If you're with me so far, try changing the end of
doctest.py's DocTestFinder.find() from
return tests
to
tests.sort()
return tests
----------------------------------------------------------------------
Comment By: David MacQuigg (macquigg)
Date: 2006-07-27 14:48
Message:
Logged In: YES
user_id=676422
Some observations:
If I give the test variable a private name, the problem
goes away.
. _xyz = 2
If I assign the variable to an object already in memory,
the problem doesn't go away.
. xyz = LocalADDRs
It doesn't matter that the test variable is never used. The
problem persists, even if I add a second statement:
. xyz = 2
. if xyz == 2: pass
It doesn't matter where in the module the statement is
placed, as long as it is module level, not inside a class
or function.
----------------------------------------------------------------------
Comment By: David MacQuigg (macquigg)
Date: 2006-07-27 14:48
Message:
Logged In: YES
user_id=676422
Some observations:
If I give the test variable a private name, the problem
goes away.
. _xyz = 2
If I assign the variable to an object already in memory,
the problem doesn't go away.
. xyz = LocalADDRs
It doesn't matter that the test variable is never used. The
problem persists, even if I add a second statement:
. xyz = 2
. if xyz == 2: pass
It doesn't matter where in the module the statement is
placed, as long as it is module level, not inside a class
or function.
----------------------------------------------------------------------
Comment By: David MacQuigg (macquigg)
Date: 2006-07-27 10:32
Message:
Logged In: YES
user_id=676422
The failure is not the specific output of doctest, but the
fact that it *changes* when I add an unrelated variable
xyz=2 in the module program.py. I am not able to reproduce
this failure with Tim's stripped down test_suite.py.
I agree the problem is not in IDLE. I can do the same from
a bash command line with
$ python2.4 test_suite.py
but not with
$ python2.3 test_suite.py
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2006-07-27 09:14
Message:
Logged In: YES
user_id=31435
I see exactly the same failure akuchling reported, on
Windows using current trunk, running from a command shell
(not IDLE -- didn't try IDLE). The attached test_suite.py
throws away everything not needed to see the same failure;
in particular, it gets rid of doctest, the logging module,
and most of the functions in test_suite.py.
Reduced to this, I don't see a reason to suspect a Python
bug. ci.IDrating is initialized to 100, and the
_envrcpt('zz1 at open-mail.org')
line produces (the expected)
CONTINUE [['zz1 at open-mail.org', 50, 2, 75, 101, '']]
if you /leave ci.IDrating alone/. However, if you do
ci.hello('zz_IP_192.168.0.8_mailout7.simple.tld')
first, that has a (apparently deliberate) side effect of
changing ci.IDrating to None, and then this path is /not/ taken:
# . . Whitelist pass
if self.IDrating > whitelvl:
rcptinfo[4] = 101 # ham > 100% to ignore later
spamscore ###2e
self.R.append(rcptinfo)
The 101 vanishes from the expected output then, and I get
CONTINUE [['zz1 at open-mail.org', 50, 2, 75, 50, '']]
instead.
That exactly reproduces the only failure Andrew & I have
seen, and appears to be what the code is telling it to do
(via side effects on the shared `ci` instance).
----------------------------------------------------------------------
Comment By: David MacQuigg (macquigg)
Date: 2006-07-27 08:22
Message:
Logged In: YES
user_id=676422
On my Redhat Linux machine, I see the failure in 2.4.1, but
not in 2.3.4.
Trying to isolate the problem, I've commented out some
blocks of unrelated code, but I can't get very far with
this, because many of the deletions make the problem go
away. It's as if the problem depends on the exact position
in memory of the test variable.
I did manage to delete the DNS queries, so now you can run
the tests without intermittent failures due to DNS timeouts.
I'll keep the latest at http://www.open-
mail.org/test/test_0607nn.tgz
Thanks for your efforts.
----------------------------------------------------------------------
Comment By: A.M. Kuchling (akuchling)
Date: 2006-07-27 05:15
Message:
Logged In: YES
user_id=11375
On a Linux machine, both 2.4.3 and SVN HEAD report 7 tests
passed, one failed.
For both versions, the failure is:
Failed example:
_envrcpt('zz1 at open-mail.org')
Expected:
CONTINUE [['zz1 at open-mail.org', 50, 2, 75, 101, '']]
Got:
CONTINUE [['zz1 at open-mail.org', 50, 2, 75, 50, '']]
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2006-07-26 20:59
Message:
Logged In: YES
user_id=33168
Can you try with SVN head? There was an unitialized read
that the doctests could trigger. It has been fixed.
Perhaps you are running into that?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1529297&group_id=5470
More information about the Python-bugs-list
mailing list