Re: [code-quality] Make pylint faster by caching modules that passed without warnings

(I'm sorry, I accidentally had mail delivery turned off, so I reply to myself)

...

This sounds like a great feature. Have you considered contributing it as a pull request into pylint?

I will be glad to - you can use the code for whatever you like - but I think that a member of the team should review my code and say what he thinks and how it should be integrated into the main code.

...

I tried this on my large project (Open edX), and it generally worked great, but was one crash:

I updated the gist so now I hope it would work - can you check? Thanks, the crash is fixed. On our code, which has a ton of violations

On 11/7/15 4:04 PM, Noam Yorav-Raphael wrote: that we are slowly working through, it's about the same speed as regular pylint though. I think we need to shift the balance so that more files have no violations before the caching pays off. Thanks, this is very interesting, and I hope it's pursued as a pylint feature. --Ned.

Cheers, Noam

On Thu, Nov 5, 2015 at 12:21 PM, Noam Yorav-Raphael mailto:noamraph@gmail.com> wrote:

Hi,

Take a look at cached_pylint.py at https://gist.github.com/noamraph/c933b32deb8304ac7ccd

It records module dependencies, and when a module passes with no warnings, it records it together with the sha1 of the contents of each of its dependencies, so when run again it could pass without checking it.

Dependencies are recorded by using infer and by using ImportsChecker, so if module A imports something from module B that was actually defined in module C, both B and C will be recorded as dependencies of A.

We have a medium sized code repository that took two minutes for each check. Now pylint only checks the modified modules, and it takes two seconds.

I tested it with pylint 1.4.3.

Hope this helps someone,

Noam