Hi Sylvain,
First of all thanks for the quick answer. I tried to create a dummy project
(the real one is confidential and not mine to give to the community) and it
helped me narrow down my two issues.
The ImportError: No module named project is linked to the relative import,
and I could not reproduce the error in a dummy project. I finally found it,
it has to do with my namespaces packages.
What I mean by namespaces is in my python library I dispose of several
packages like this :
company.mongodb
company.redis
company.hbase
And also :
company.project.core
company.project.app.dashboard
company.project.app.security
Where company and project are all the same things, this allows me and my
team to easily identify each package and what it is supposed to do. We try
to foster the open-source spirit in my company, hence the company top-level
namespace. It is still internal but at least we share it across all teams :)
Going back to the issue, if one package of the namespace is installed, then
apparently pylint is looking directly inside the python path and can't find
my package since it is not yet installed. I have the same issue with the
"python setup.py develop" command, so I guess I am doing namespacing wrong,
or it is not well defined in python ?
The only thing I do is adding
*_import__('pkg_resources').declare_namespace(__name__)
*in the __init__.py of the namespace package.
If I run pylint in another environment without any packages installed, it
works like a charm (except I have a bunch of violations for missing
imports).
Do you want me to open an issue on bitbucket explaining all that or do you
think I am doing something I am not supposed to do ?
Cheers
François
2014-05-16 14:02 GMT+02:00 Sylvain Thénault
On 16 mai 11:16, François Vanderkelen wrote:
Hi experts !
Hi François,
I am struggling with Pylint with an issue that might be a problem coming from my code. Since I am relatively new to python (~1year) I would like to have your input on the matter.
I am trying to split a big python project I am doing for my company into several packages, allowing me to update each part without impacting the others. Doing that, we decided to go with Jenkins to improve our code quality which lead me to implement pylint for code violations checks.
Currently my build process is the following :
- Create a new virtualenv with Python version 2.7.6 - Install all dependencies needed by my package - Download the last version of the package from the master branch in our internal gitlab - Run nosetests with coverage reports - Run pylint and export the result to have some violations reports in Jenkins.
At first I had a lot of pylint errors since I wasn't installing all dependencies, resulting in something like 200 import errors ... Now it is reduced to some 60 errors but when I look a the build output log, I can see that pylint is actually crashing in the middle of the process.
************* Module company.project.core.security.app company/project/core/security/app.py:1: [C0111(missing-docstring), ] Missing module docstring load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')() File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/__init__.py", line 577, in _search_zip raise ImportError('No module named %s' % '.'.join(modpath)) ImportError: No module named project
Would you create an issue in pylint's bitbucket tracker for this? In any case it shouldn't crash like that?
This is how my package is constructed :
company/ -- __init__.py -- project/ -- __init__.py -- core/ -- __init__.py -- app.py -- security/ -- __init__.py -- app.py
In company.project.core.security.app I am trying to import company.project.core.app which contains some BaseClass for my project using:
from ..app import BaseApplication
This is probably another issue that deserves a ticket as well.
This is working fine with python, although pylint is crying out loud on this specific line. I tried to replace the relative import by core.app which worked fine with pylint but my code is not working anymore. I tried also to use the full path (company.project.core.app) but I have the exact same import error.
Be aware that both company and project are just namespace packages.
what do you call namespace package ? Do you mean part of them are installed in different directories?
Also I am forced to use the path to run pylint because it looks like my top-level folder is not a package (even if it contains a __init__ file).
pylint company.project.core No module named project.core (fatal)
and this work if you set a PYTHONPATH?
pylint company/project/core is working.
When I try to run pylint directly from the top-level package, I have the exact same result as from the path.
A workaround I can try is to install the package before running pylint on it but I reckon since my cwd is supposed to be added to the path I can't see how it would help (although I have other company packages installed).
It should work.
[syt@orion ~]$ mkdir x [syt@orion ~]$ mkdir x/y [syt@orion ~]$ touch x/__init__.py [syt@orion ~]$ touch x/y/__init__.py [syt@orion ~]$ pylint -rn x.y ************* Module x.y C: 1, 0: Missing module docstring (missing-docstring)
What do you think I should do ? Is my design a poorly chosen one and I don't have any solution ?
At a quick glance I would say you seem to abuse a bit from packages, leading to a deep structure which may be over-complicated. For instance, do you really want a 'company' first level package? Anyway, that doesn't seem to be the cause of your problems with pylint. At least, it shouldn't.
-- Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations Développement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org