[Python-Dev] Suspect intermittent failure in buildbots

[Guido van Rossum]

Module globals are also reset when the module *object* is
garbage-collected (e.g. it's removed from sys.modules and not
referenced elsewhere), but the module *dict* is still referenced. This
can happen if all uses of the module is of the form "from <module>
import <something>" where the <something> is a class or function, and
at least one of the users survives the garbage-collected module.

I suspect that something is resetting part of sys.modules content. It
is a known problem (in some parts) that encodings modules cannot be
reset that way. I suspect that there is code in the regrtest.py
framework that does this (resetting part of sys.modules) in order to
restore a clean environment in some cases. Or perhaps it's one of the
tests that does this.

On Fri, Oct 3, 2008 at 3:56 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> I've noticed an error that comes up from time to time in python 3.0 buildbots.
> The error is always similar to this one:
>
> Traceback (most recent call last):
>  File "E:\cygwin\home\db3l\buildarea\3.0.bolen-windows\build\lib\test\test_io.py",
> line 900, in testBasicIO
>    self.assertEquals(f.write("abc"), 3)
>  File "E:\cygwin\home\db3l\buildarea\3.0.bolen-windows\build\lib\io.py",
> line 1486, in write
>    b = encoder.encode(s)
>  File "E:\cygwin\home\db3l\buildarea\3.0.bolen-windows\build\lib\encodings\ascii.py",
> line 22, in encode
>    return codecs.ascii_encode(input, self.errors)[0]
> AttributeError: 'NoneType' object has no attribute 'ascii_encode'
>
> The most recent is here:
> http://www.python.org/dev/buildbot/3.0/AMD64%20W2k8%203.0/builds/843/step-test/0
>
> but this already happened on various buildbots:
> http://www.google.fr/search?q=%27NoneType%27+object+has+no+attribute+encode+site:mail.python.org/pipermail/python-checkins&filter=0
>
> "x86 XP-4 3.0", "amd64 gentoo 3.0", "AMD64 W2k8 3.0", "x86 W2k8 3.0",
> "g4 osx.4 3.0", "OS X x86 3.0"
> "x86 XP-3 trunk"
>
> yes, even on trunk!
> Every time, a "codecs" global module variable has been reset to None,
> either in a codec module (encoding/ascii.py, encoding/mac_roman.py) or
> in test_io.py.
> Every time, io.py is not far (which may be normal, it must have the
> larger usage of encodings written in a .py)
>
> I know that modules globals are reset to None on interpreter shutdown,
> but it does not seem to be the case here: the unit test fail, and
> fails again when run in verbose mode at the end.
>
> I checked that the "codecs" name is a module global: the disassembler
> shows a LOAD_GLOBAL opcode followed by LOAD_ATTR:
>   0 LOAD_GLOBAL              0 (codecs)
>   3 LOAD_ATTR                1 (ascii_decode)
>   ...
>
> I fail to imagine a reason, apart from a creeping memory error (in
> dictionary lookup; chilling idea).
> Thoughts?
>
> --
> Amaury Forgeot d'Arc
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)