[py-dev] intermittent bugs in pytest/python on osx lion ?

[holger krekel]

Hi Ian,

On Fri, Jun 15, 2012 at 20:32 -0400, Ian Miers wrote:
> Hi, I just started using pytest. It's lovely.
> The TLDR on this is we are getting intermittent non reproducible errors
>  that change and sometimes disappear between test runs like the following :
> >   import os.path
> E   TypeError: __import__() argument 1 must be str without null bytes, not
> str

Looks odd.  pytest does not override __import__ but it does by default
use a PEP302 compliant module loader.  If you use

    py.test --assert=reinterpret  # or --assert=plain

do the errors go away?  If so that points to a problem in pytest's module
loader or some interaction problem with python3.2.

If changing the assertion mode still leads to errors then i strongly
suspect it's other parts of the code you are running.  I'd then suggest
to check if something in your environment modifies __import__
e.g. by writing a test that checks/prints out __import__?

best,
holger

> Longer version:
> We've been getting buggy results out of test runs on OSX Lion with python
> 3.2.3  and  py.test version 2.2.4. Specifically we've been getting what
> appear to be false-positive  test failures that change from run to run  and
> cannot be reproduced by running some code ourselves or in the case of
> doctest manually running  python3.2 -m doctest file.
> 
> Some of these errors will stay around for multiple test runs even after
> make clean, etc. Some will change from run to run. All of them
> eventually disappeared  temporarily and we got a clean test pass, though
> how I don't know. Morever, the problems cropped up again. All of this was
> with no code changes and code known to work on ubuntu and partially
> manually tested on OSX.
> 
> Ordinarly I'd say there was something wrong with our code. However, some of
> the errors are vanishingly unlikely. Claims that modules don't exist when
> they do and are importable via python3.2 -c "from foo.bar.baz import narf"
> and such.
> 
> The most glaringly, however, is this gem:
> charm/toolbox/pairinggroup.py:3: in <module>
> >   import os.path
> E   TypeError: __import__() argument 1 must be str without null bytes, not
> str
> 
> We also got a lovely bug where it appeared __pycache__  was corrupted
> during test runs.  On an initial run, we could import a function from a
> python c extension. On subsequent runs, it didn't exist. The function was
> still in the .so file, as shown by nm, however help(module) returned
> function_name#$@^%#$% Function description. Deleting __pychache__ folders
>  resolved it for the next test run but then it came back. It too
> disappeared after a couple of test runs never to be seen since.
> 
> Has anyone seen anyhting like this? Are their known issues on OSX with
> python ? With pytest? Does anyone have any idea how I might get a better
> idea whats going on?
> 
> As an addendum, the latest error I am getting is now :
> >   test = [hashFn(struct.pack(">%dsI" % (len(seed)), seed, i)) for i in
> ran]
> E   UnicodeEncodeError: 'ascii' codec can't encode character '\x9e' in
> position 0: ordinal not in range(128
> This actually might be in our code,though again it works on ubuntu and at
> one point on OSX.  Given that its a pattern that points to python or pytest
> doing something  to binaries,  I'm including it anyway.
> 
> The project is charm, you can see the code on this branch here via
> https://github.com/JHUISI/charm/tree/dev. Python was installed via fenc ( I
> think, its not my box). The errors happened both with python3.2 -m pytest
>  and with python3.2 setup.py test.  Though it appears more so with the
> later.
> 
> Thanks,
> 
> Ian

> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev