Pylint with threading
Hello guys, I came from http://docs.pylint.org/contribute.html#mailing-lists Im the author of some code-quality libraries such as https://github.com/klen/pylama. I use that in my VIM plugin https://github.com/klen/python-mode. Pylama is simular to flake8, but have some additional features and works also with pylint. Today I would like to make code checking with pylama is more asynchronously. I haven't any problems with pyflake, pep8, pep257 and mccabe but cannot make pylint works. When I run pylint with a few threads I get errors like this: https://gist.github.com/klen/5721176. When one thread try to run pylint a more from one time I got this error. I think the problem in ASTNG MANAGER and I try to clean that cache. I tried to clear the cache in several ways https://gist.github.com/klen/5721225 but not have success. Can anybody help me? PS: Sorry for my English, its far from well I now. Kirill Klenov web developer ______________________________________________________________________ horneds@gmail.com | http://klen.github.io mobile: +7 906 7723620
On 06 juin 20:47, horneds@gmail.com wrote:
Hello guys,
Hi,
Im the author of some code-quality libraries such as https://github.com/klen/pylama. I use that in my VIM plugin https://github.com/klen/python-mode. Pylama is simular to flake8, but have some additional features and works also with pylint.
Today I would like to make code checking with pylama is more asynchronously. I haven't any problems with pyflake, pep8, pep257 and mccabe but cannot make pylint works. When I run pylint with a few threads I get errors like this: https://gist.github.com/klen/5721176. When one thread try to run pylint a more from one time I got this error.
I think the problem in ASTNG MANAGER and I try to clean that cache.
I tried to clear the cache in several ways https://gist.github.com/klen/5721225 but not have success.
Can anybody help me?
nor pylint nor astng have been written with multi-threading in mind so I'm not surprised such pb arises. You're right that the one you're issuing are much probably due to the astng's MANAGER singleton. Though IMO it'll be hard if possible to remove that singleton/borg and your attempt to solve the pb are not in the good direction. We should rather ensure that the manager and other underlying code is thread safe. Quickly digging in the code shown me the unsafety you're encountering: the rebuilder is set as a class attribute on the builder, hence shared among thread. See attached patch for an attempt to fix this. There may be other pb on the way, and the best thing to do is to open an issue for that on bitbucket so we can continue this discussion there. On the pylint side, I assume you have a pylint instance per thread? -- Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations Développement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org
the patch... -- Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations Développement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org
participants (2)
-
horneds@gmail.com
-
Sylvain Thénault