Hi experts !
I am struggling with Pylint with an issue that might be a problem coming
from my code. Since I am relatively new to python (~1year) I would like to
have your input on the matter.
I am trying to split a big python project I am doing for my company into
several packages, allowing me to update each part without impacting the
others. Doing that, we decided to go with Jenkins to improve our code
quality which lead me to implement pylint for code violations checks.
Currently my build process is the following :
- Create a new virtualenv with Python version 2.7.6
- Install all dependencies needed by my package
- Download the last version of the package from the master branch in our
internal gitlab
- Run nosetests with coverage reports
- Run pylint and export the result to have some violations reports in
Jenkins.
At first I had a lot of pylint errors since I wasn't installing all
dependencies, resulting in something like 200 import errors ... Now it is
reduced to some 60 errors but when I look a the build output log, I can see
that pylint is actually crashing in the middle of the process.
************* Module company.project.core.security.app
company/project/core/security/app.py:1: [C0111(missing-docstring), ]
Missing module docstring
load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')()
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/__init__.py",
line 21, in run_pylint
Run(sys.argv[1:])
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/lint.py",
line 1051, in __init__
linter.check(args)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/lint.py",
line 626, in check
self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/lint.py",
line 712, in check_astroid_module
walker.walk(astroid)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/utils.py",
line 715, in walk
self.walk(child)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/utils.py",
line 712, in walk
cb(astroid)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/checkers/imports.py",
line 269, in visit_from
self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name))
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/checkers/imports.py",
line 302, in _add_imported_module
importedmodname = get_module_part(importedmodname)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py",
line 352, in get_module_part
path=path, context_file=context_file)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py",
line 297, in file_from_modpath
return _file_from_modpath(modpath, path, context)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py",
line 556, in _file_from_modpath
mtype, mp_filename = _module_file(modpath, path)
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py",
line 636, in _module_file
return _search_zip(modpath, pic)[:2]
File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py",
line 577, in _search_zip
raise ImportError('No module named %s' % '.'.join(modpath))
ImportError: No module named project
This is how my package is constructed :
company/
-- __init__.py
-- project/
-- __init__.py
-- core/
-- __init__.py
-- app.py
-- security/
-- __init__.py
-- app.py
In company.project.core.security.app I am trying to import
company.project.core.app which contains some BaseClass for my project using:
from ..app import BaseApplication
This is working fine with python, although pylint is crying out loud on
this specific line. I tried to replace the relative import by core.app
which worked fine with pylint but my code is not working anymore. I tried
also to use the full path (company.project.core.app) but I have the exact
same import error.
Be aware that both company and project are just namespace packages.
Also I am forced to use the path to run pylint because it looks like my
top-level folder is not a package (even if it contains a __init__ file).
pylint company.project.core
No module named project.core (fatal)
pylint company/project/core is working.
When I try to run pylint directly from the top-level package, I have the
exact same result as from the path.
A workaround I can try is to install the package before running pylint on
it but I reckon since my cwd is supposed to be added to the path I can't
see how it would help (although I have other company packages installed).
What do you think I should do ? Is my design a poorly chosen one and I
don't have any solution ?
Thanks,
François
I am using pip to install all dependencies of new packages and am
running into a few issues (see error at end of email).
The following patch seems to remove the issue. Not sure why imports of
names space packages had to be greater
than one, but it seems like pip installs namespaces differently than
easy_install did.
$ hg diff
diff --git a/modutils.py b/modutils.py
--- a/modutils.py
+++ b/modutils.py
@@ -612,11 +612,14 @@
except AttributeError:
checkeggs = False
# pkg_resources support (aka setuptools namespace packages)
- if pkg_resources is not None and modpath[0] in pkg_resources._namespace_packages and len(modpath) > 1:
+# if pkg_resources is not None and modpath[0] in pkg_resources._namespace_packages and len(modpath) > 1:
+ if pkg_resources is not None and modpath[0] in pkg_resources._namespace_packages:#
# setuptools has added into sys.modules a module object with proper
# __path__, get back information from there
module = sys.modules[modpath.pop(0)]
path = module.__path__
+ mtype = PKG_DIRECTORY
+ mp_filename = path[0]
imported = []
while modpath:
modname = modpath[0]
=======================================================================
$ paver pylint bqserver/bq/module_service/controllers/module_server.py
---> pavement.pylint
PYTHONPATH=bqcore/bq:bqserver/bq:bqengine/bq:bqfeature/bq pylint bqserver/bq/module_service/controllers/module_server.py --rcfile=bqcore/pylint.rc
************* Module bq.module_service.controllers.module_server
I0011:330,0: Locally disabling dangerous-default-value (W0102)
Traceback (most recent call last):
File "/home/kgk/work/bisque/bq055/bqenv/bin/pylint", line 9, in <module>
load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')()
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/__init__.py", line 21, in run_pylint
Run(sys.argv[1:])
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/lint.py", line 1051, in __init__
linter.check(args)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/lint.py", line 626, in check
self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/lint.py", line 712, in check_astroid_module
walker.walk(astroid)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/utils.py", line 715, in walk
self.walk(child)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/utils.py", line 712, in walk
cb(astroid)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/checkers/imports.py", line 269, in visit_from
self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name))
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/checkers/imports.py", line 302, in _add_imported_module
importedmodname = get_module_part(importedmodname)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 352, in get_module_part
path=path, context_file=context_file)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 297, in file_from_modpath
return _file_from_modpath(modpath, path, context)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 556, in _file_from_modpath
mtype, mp_filename = _module_file(modpath, path)
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 636, in _module_file
return _search_zip(modpath, pic)[:2]
File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 577, in _search_zip
raise ImportError('No module named %s' % '.'.join(modpath))
ImportError: No module named paste
Captured Task Output:
I'm trying to whitelist checkers from a pylintrc file for pylint, but when
I have:
[MESSAGES CONTROL]
disable=all
enable=old-style-class
nothing is outputted. When I comment out the `disable` line then everything
is outputted. And if I set `disable` to something like `no-init` then that
works, so I know the pylintrc file is working properly.
How can I get the equivalent of --disable=all specified in a pylintrc file
so I can select exactly what checkers I want to run?
