[code-quality] Namespaced packages and relative imports

[Sylvain Thénault]

Hi François,

by chance, would you be in EuroPython next week ? I intend to work on this
namespace package problem there.

Cheers

On 01 juillet 11:22, François Vanderkelen wrote:
> Hi Sylvain, experts.
> 
> I tried to work a little more on this and I found out why it "worked" with
> setuptools 0.9.8.
> 
> Turns out it did not, it is just when you start with a clean environement
> (not containing any of your namespaced packages) pylint is running properly
> and flagging every import of related namespace packages as missing.
> 
> Although the good part is that it is matching properly relatives import,
> you cannot have multiple dependencies to packages from the same namespace.
> 
> I am pretty sure that namespace usage is "erratic" in python since even
> Setuptools does not seem to respect PEP420, but I am not sure the fact the
> Pylint block on this comes from setuptools.
> 
> I think is is worth fixing but I am not sure where to start :)
> 
> 
> 2014-05-16 15:57 GMT+02:00 François Vanderkelen <
> vanderkelen.francois at gmail.com>:
> 
> > I tried multiple things, first even if I install the package before
> > running pylint it won't work except on one of my machines.
> >
> > So I tried to narrow it down with setuptools :
> >
> > Working with 0.9.8
> > Not working with 2.2 (ImportError)
> > Not working with 3.6 (ImportError)
> >
> > I guess setuptools changed something concerning namespaces packages, bug
> > or not this is another story. I'll keep looking.
> >
> > Thanks a lot Sylvain
> >
> >
> > 2014-05-16 15:01 GMT+02:00 Sylvain Thénault <sylvain.thenault at logilab.fr>:
> >
> > On 16 mai 14:42, François Vanderkelen wrote:
> >> > Hi Sylvain,
> >> >
> >> > First of all thanks for the quick answer. I tried to create a dummy
> >> project
> >> > (the real one is confidential and not mine to give to the community)
> >> and it
> >> > helped me narrow down my two issues.
> >> >
> >> > The ImportError: No module named project is linked to the relative
> >> import,
> >> > and I could not reproduce the error in a dummy project. I finally found
> >> it,
> >> > it has to do with my namespaces packages.
> >> >
> >> > What I mean by namespaces is in my python library I dispose of several
> >> > packages like this :
> >> >
> >> > company.mongodb
> >> > company.redis
> >> > company.hbase
> >> >
> >> > And also :
> >> >
> >> > company.project.core
> >> > company.project.app.dashboard
> >> > company.project.app.security
> >> >
> >> > Where company and project are all the same things, this allows me and my
> >> > team to easily identify each package and what it is supposed to do. We
> >> try
> >> > to foster the open-source spirit in my company, hence the company
> >> top-level
> >> > namespace. It is still internal but at least we share it across all
> >> teams :)
> >> >
> >> > Going back to the issue, if one package of the namespace is installed,
> >> then
> >> > apparently pylint is looking directly inside the python path and can't
> >> find
> >> > my package since it is not yet installed. I have the same issue with the
> >> > "python setup.py develop" command, so I guess I am doing namespacing
> >> wrong,
> >> > or it is not well defined in python ?
> >> >
> >> > The only thing I do is adding
> >> > *_import__('pkg_resources').declare_namespace(__name__)
> >> > *in the __init__.py of the namespace package.
> >> >
> >> > If I run pylint in another environment without any packages installed,
> >> it
> >> > works like a charm (except I have a bunch of violations for missing
> >> > imports).
> >> >
> >> > Do you want me to open an issue on bitbucket explaining all that or do
> >> you
> >> > think I am doing something I am not supposed to do ?
> >>
> >> I think what you're doing is right.
> >>
> >> The crash has already been supported:
> >>
> >> https://bitbucket.org/logilab/pylint/issue/203/importing-namespace-packages-crashes
> >>
> >> It seems to me that the namespace package issue is related to recent
> >> version of
> >> setuptools and has also been reported but can't find it right away.
> >>
> >> --
> >> Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse
> >> (05.62.17.16.42)
> >> Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations
> >> Développement logiciel sur mesure:       http://www.logilab.fr/services
> >> CubicWeb, the semantic web framework:    http://www.cubicweb.org
> >>
> >
> >

-- 
Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42)
Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations
Développement logiciel sur mesure:       http://www.logilab.fr/services
CubicWeb, the semantic web framework:    http://www.cubicweb.org