[Pytest-commit] Issue #501: xdist process crashes with ImportError (hpk42/pytest)

Andreas Pelme issues-reply at bitbucket.org
Fri Apr 11 17:00:13 CEST 2014


New issue 501: xdist process crashes with ImportError
https://bitbucket.org/hpk42/pytest/issue/501/xdist-process-crashes-with-importerror

Andreas Pelme:

I encountered this issue when originally trying to debug another issue. I might have something screwed up on my machine, but I cannot figure out what it is, and any pointers would be helpful.

```
~ $ mkvirtualenv tmp1234
New python executable in tmp1234/bin/python
Installing setuptools, pip...done.
[tmp1234] ~ $ cd code/pytest
[tmp1234] ~/code/pytest $ hg log|head -n1
changeset:   3585:1bbe10ccc558
[tmp1234] ~/code/pytest $ pip install -e .
Obtaining file:///Users/andreas/code/pytest
  Running setup.py (path:/Users/andreas/code/pytest/setup.py) egg_info for package from file:///Users/andreas/code/pytest

Downloading/unpacking py>=1.4.19 (from pytest==2.5.2.dev1)
  http://localhost:3141/root/pypi/+simple/py/ uses an insecure transport scheme (http). Consider using https if localhost:3141 has it available
  Downloading py-1.4.20.tar.gz (185kB): 185kB downloaded
  Running setup.py (path:/Users/andreas/.virtualenvs/tmp1234/build/py/setup.py) egg_info for package py

Installing collected packages: py, pytest
  Running setup.py install for py

  Running setup.py develop for pytest

    Creating /Users/andreas/.virtualenvs/tmp1234/lib/python2.7/site-packages/pytest.egg-link (link to .)
    Adding pytest 2.5.2.dev1 to easy-install.pth file
    Installing py.test script to /Users/andreas/.virtualenvs/tmp1234/bin
    Installing py.test-2.7 script to /Users/andreas/.virtualenvs/tmp1234/bin

    Installed /Users/andreas/code/pytest
Successfully installed py pytest
Cleaning up...
[tmp1234] ~/code/pytest $ cd ~/code/pytest-xdist
[tmp1234] ~/code/pytest-xdist $ hg log|head -n1
changeset:   174:81566ffbb271
[tmp1234] ~/code/pytest-xdist $ pip install -e .
Obtaining file:///Users/andreas/code/pytest-xdist
  Running setup.py (path:/Users/andreas/code/pytest-xdist/setup.py) egg_info for package from file:///Users/andreas/code/pytest-xdist

    no previously-included directories found matching '.hg'
Downloading/unpacking execnet>=1.1 (from pytest-xdist==1.11.dev1)
  http://localhost:3141/root/pypi/+simple/execnet/ uses an insecure transport scheme (http). Consider using https if localhost:3141 has it available
  Downloading execnet-1.2.0.tar.gz (163kB): 163kB downloaded
  Running setup.py (path:/Users/andreas/.virtualenvs/tmp1234/build/execnet/setup.py) egg_info for package execnet

    warning: no files found matching 'conftest.py'
Requirement already satisfied (use --upgrade to upgrade): pytest>=2.4.2 in /Users/andreas/code/pytest (from pytest-xdist==1.11.dev1)
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.19 in /Users/andreas/.virtualenvs/tmp1234/lib/python2.7/site-packages (from pytest>=2.4.2->pytest-xdist==1.11.dev1)
Installing collected packages: execnet, pytest-xdist
  Running setup.py install for execnet

    warning: no files found matching 'conftest.py'
  Running setup.py develop for pytest-xdist

    no previously-included directories found matching '.hg'
    Creating /Users/andreas/.virtualenvs/tmp1234/lib/python2.7/site-packages/pytest-xdist.egg-link (link to .)
    Adding pytest-xdist 1.11.dev1 to easy-install.pth file

    Installed /Users/andreas/code/pytest-xdist
Successfully installed execnet pytest-xdist
Cleaning up...
```

Things seems to be installed properly:
```
[tmp1234] ~ $ python
Python 2.7.5 (default, Sep 12 2013, 21:33:34)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytest;pytest
<module 'pytest' from '/Users/andreas/code/pytest/pytest.pyc'>
>>> import _pytest.config;_pytest.config
<module '_pytest.config' from '/Users/andreas/code/pytest/_pytest/config.pyc'>
>>> import xdist;xdist
<module 'xdist' from '/Users/andreas/code/pytest-xdist/xdist/__init__.pyc'>
>>>

[tmp1234] ~ $ py.test --help
usage: py.test [options] [file_or_dir] [file_or_dir] [...]

positional arguments:
  file_or_dir

general:
  -k EXPRESSION         only run tests which match the given substring
                        expression. An expression is a python evaluatable
                        expression where all names are substring-matched
                        against test names and their parent classes. Example:
                        -k 'test_method or test other' matches all test
                        functions and classes whose name contains
                        'test_method' or 'test_other'. Additionally keywords
                        are matched to classes and functions containing extra
                        names in their 'extra_keyword_matches' set, as well as
                        functions which have names assigned directly to them.
  -m MARKEXPR           only run tests matching given mark expression.
                        example: -m 'mark1 and not mark2'.
  --markers             show markers (builtin, plugin and per-project ones).
  -x, --exitfirst       exit instantly on first error or failed test.
  --maxfail num         exit after first num failures or errors.
  --strict              run pytest in strict mode, warnings become errors.
  --fixtures, --funcargs
                        show available fixtures, sorted by plugin appearance
  --pdb                 start the interactive Python debugger on errors.
  --capture method      per-test capturing method: one of fd (default)|sys|no.
  -s                    shortcut for --capture=no.
  --runxfail            run tests even if they are marked xfail

reporting:
  -v, --verbose         increase verbosity.
  -q, --quiet           decrease verbosity.
  -r chars              show extra test summary info as specified by chars
                        (f)ailed, (E)error, (s)skipped, (x)failed, (X)passed.
  -l, --showlocals      show locals in tracebacks (disabled by default).
  --report opts         (deprecated, use -r)
  --tb style            traceback print mode (long/short/line/native/no).
  --full-trace          don't cut any tracebacks (default is to cut).
  --color color         color terminal output (yes/no/auto).
  --durations N         show N slowest setup/test durations (N=0 for all).
  --pastebin mode       send failed|all info to bpaste.net pastebin service.
  --junit-xml path      create junit-xml style report file at given path.
  --junit-prefix str    prepend prefix to classnames in junit-xml output
  --result-log path     path for machine-readable result log.

collection:
  --collect-only        only collect tests, don't execute them.
  --pyargs              try to interpret all arguments as python packages.
  --ignore path         ignore path during collection (multi-allowed).
  --confcutdir dir      only load conftest.py's relative to specified dir.
  --doctest-modules     run doctests in all .py modules
  --doctest-glob pat    doctests file matching pattern, default: test*.txt

test session debugging and configuration:
  --basetemp dir        base temporary directory for this test run.
  --version             display pytest lib version and import information.
  -h, --help            show help message and configuration info
  -p name               early-load given plugin (multi-allowed). To avoid
                        loading of plugins, use the `no:` prefix, e.g.
                        `no:doctest`.
  --trace-config        trace considerations of conftest.py files.
  --debug               store internal tracing debug information in
                        'pytestdebug.log'.
  --assert MODE         control assertion debugging tools. 'plain' performs no
                        assertion debugging. 'reinterp' reinterprets assert
                        statements after they failed to provide assertion
                        expression information. 'rewrite' (the default)
                        rewrites assert statements in test modules on import
                        to provide assert expression information.
  --no-assert           DEPRECATED equivalent to --assert=plain
  --no-magic            DEPRECATED equivalent to --assert=plain
  --genscript path      create standalone pytest script at given target path.

distributed and subprocess testing:
  -f, --looponfail      run tests in subprocess, wait for modified files and
                        re-run failing test set until all pass.
  -n numprocesses       shortcut for '--dist=load --tx=NUM*popen'
  --boxed               box each test run in a separate process (unix)
  --dist distmode       set mode for distributing tests to exec environments.
                        each: send each test to each available environment.
                        load: send each test to available environment.
                        (default) no: run tests inprocess, don't distribute.
  --tx xspec            add a test execution environment. some examples: --tx
                        popen//python=python2.5 --tx socket=192.168.1.102:8888
                        --tx ssh=user at codespeak.net//chdir=testcache
  -d                    load-balance tests. shortcut for '--dist=load'
  --rsyncdir DIR        add directory for rsyncing to remote tx nodes.
  --rsyncignore GLOB    add expression for ignores when rsyncing to remote tx
                        nodes.


[pytest] ini-options in the next pytest.ini|tox.ini|setup.cfg file:

  markers (linelist)       markers for test functions
  norecursedirs (args)     directory patterns to avoid for recursion
  usefixtures (args)       list of default fixtures to be used with this project
  python_files (args)      glob-style file patterns for Python test module disco
  python_classes (args)    prefixes for Python test class discovery
  python_functions (args)  prefixes for Python test function and method discover
  addopts (args)           extra command line options
  minversion (string)      minimally required pytest version
  rsyncdirs (pathlist)     list of (relative) paths to be rsynced for remote dis
  rsyncignore (pathlist)   list of (relative) glob-style paths to be ignored for
  looponfailroots (pathlist) directories to check for changes


to see available markers type: py.test --markers
to see available fixtures type: py.test --fixtures
```


However, any invocations of py.test with xdist crashes like this:

```
[tmp1234] ~ $ py.test -n1
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- pytest-2.5.2.dev1
plugins: xdist
gw0 C[gw0] node down: Traceback (most recent call last):
  File "/Users/andreas/.virtualenvs/tmp1234/lib/python2.7/site-packages/execnet/gateway_base.py", line 1029, in executetask
    do_exec(co, loc) # noqa
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 139, in <module>
  File "<remote exec>", line 115, in remote_initconfig
  File "/Users/andreas/code/pytest/_pytest/config.py", line 4, in <module>
    import pytest
  File "/Users/andreas/code/pytest/pytest.py", line 14, in <module>
    from _pytest.config import main, UsageError, _preloadplugins, cmdline
ImportError: cannot import name main


===============================  in 0.03 seconds ===============================

```




More information about the pytest-commit mailing list