Mailman 3 November 2015 - code-quality

Moving pylint/astroid to GitHub
by Florian Bruhin 07 Dec '15

07 Dec '15

Hey, what do you think about moving pylint and astroid from Bitbucket/hg to GitHub/git? I've briefly talked with Claudiu about this, and he seems open to the idea and suggested I write this mail :) If people agree, I'll be able to move issues (including the correct issue numbers) and the repository (including tags/branches). I recently helped doing the same for pytest: https://bitbucket.org/pytest-dev/pytest/issues/699/ Unfortunately, I didn't find a good way to migrate pull requests, so those would've been to be recreated by their authors. Some rationales, from my point of view: - GitHub has the bigger exposure, and more people are familiar with it than with Bitbucket - this potentially means more contributors. - Contributing is IMHO cumbersome with Bitbucket. Apparently, one's supposed to use bookmarks instead of branches (because branches can't be deleted, IIRC), but Bitbucket doesn't support doing pull requests with them - the "solution" Bitbucket uses when editing things with the online editor is to create a new fork for every change, by the way... I also tried to find documentation for basic stuff like "how do I contribute to a project with branches and PRs", and found it very difficult. Sure, I'm used to git, but getting started with git was a lot easier than with hg/bitbucket. - Travis CI[1] is a lot better than drone.io, the current CI system in use. For example, one is able to test PRs, which is something essential in a CI IMHO (and also makes contributing easier). I don't think I need to elaborate on that :) - There's also a lot of other potentially useful tooling which is available for GitHub, but not BitBucket - see point 1 (bigger exposure). [1] https://travis-ci.org/ My plan would be to create a pylint organization on GitHub and move the repos and issues there. Opinions? Florian -- http://www.the-compiler.org | me(a)the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/

4 8

Pylint 1.5.0 / Astroid 1.4.1 released
by Claudiu Popa 01 Dec '15

01 Dec '15

Hello, I'm happy to announce you the release of Pylint 1.5.0, respectively Astroid 1.4.1. It's been over a year since the last major release and the amount of changes that were brought into pylint in this time is humongous, with over 30 new checks and tons of bug fixes. I would like to use this occasion for thanking for their contributions the new committers who joined pylint's team in the past months, ceridwen and Dmitry, as well as thanking the contributors that made this release possible. Here are some of the major changes of this release: - We finally support Python 3.5 in both projects. - We stopped depending on logilab-common, which means that users of the popular testing framework pytest will finally stop being confused by having two executables with the same name doing different things. - Almost 30 new checks you might enjoy not having in your project. These checks fall into multiple categories, regarding maintainability, readability, recommendations for improving the code, as well as new type-checking capabilities. They are: 'wrong-import-order', 'ungrouped-imports', 'wrong-import-position', 'unneeded-not', 'simplifiable-if-condition', 'too-many-boolean-expressions' 'too-many-nested-blocks', 'multiple-imports', 'duplicate-except', 'using-constant-test', 'confusing-with-statement', 'singleton-comparison', 'misplaced-comparison-constant', 'consider-using-enumerate', 'nonlocal-and-global', 'continue-in-finally', 'misplaced-bare-raise', 'nonlocal-without-binding', 'yield-inside-async-function', 'too-many-star-expressions', 'invalid-star-assignment-target', 'import-star-module-level, 'star-needs-assignment', 'unexpected-special-method-signature', 'repeated-keyword'. Some new type checks for finding violations of the type system are 'unsubscriptable-object', 'unsupported-membership-test', 'not-an-iterable', 'not-context-manager', 'not-async-context-manager', 'duplicate-bases' and 'inconsistent-mro'. - We also added a new 'extensions' component, which contains optional checkers that needs to be activated explicitly. These includes 'extensions.check_docs', which verifies a bunch of properties of the docstrings, such as checking that all function, method and constructor parameters are mentioned in the params and types part of the docstring. Also, it checks that there are no naming inconsistencies between the signature and the documentation, i.e. also report documented parameters that are missing in the signature. This is important to find cases where parameters are renamed only in the code, not in the documentation. Activate this checker with: --load-plugins=pylint.extensions.check_docs Most of the work was put into astroid though, which got more capable in this period of time. Unfortunately, more than half of the work that went into it was postponed for astroid 1.6 and astroid 2.0, since it wasn't deemed stable enough for this release. New features worth mentioning are: - Python 3.5 support - some of the nodes were renamed in order to be more similar to builtin's ast module, such as Class to ClassDef, Function to FunctionDef Getattr to Attribute etc, the old names being slated for removal in astroid 2.0. This affects the plugins as well, since it means that the visit methods should use the new names instead (so visit_classdef for class). Old names are still supported in visit methods for a while, until pylint 2.0. Activating the warnings when running should result in spurious PendingDeprecationWarnings when using the old node names. - add proper grammatical names for `infered` and `ass_type` methods, namely `inferred` and `assign_type`. The old methods will raise PendingDeprecationWarning, being slated for removal in astroid 2.0. - we added a new convenience API, `astroid.parse`, which can be used to retrieve an astroid AST from a source code string, similar to how ast.parse can be used to obtain a Python AST from a source string. - There's a new separate step for transforms. Until now, the transforms were applied at the same time the tree was being built. This was problematic if the transform functions were using inference, since the inference was executed on a partially constructed tree, which led to failures when post-building information was needed. - Better support for understanding builtins. We're understanding a bunch of new builtins, but unfortunately most of them weren't released, since they weren't stable enough for now, due to some inherent problems that astroid has. The most important that's released though is the understanding of super, which means that we can now detect problems of sort: class A(B, C, D): def __init__(self): super(A, self).call_missing_method() You can read the complete changelog here https://bitbucket.org/logilab/astroid/raw/4f45b6fc4c23b80b4f7154add1b9d9cee… and https://bitbucket.org/logilab/pylint/raw/2871c4d7084478b5db02f384e4e5167641… Now with every release there's going to be new false positives or regressions or things that could be improved. Don't hesitate to open an issue or to send a PR if you notice something's amiss and we'll try to have new bug fix releases as soon as possible (you might have noticed that's astroid 1.4.1..) As a final note, we're planning to switch to GitHub soon, which will mean an improved CI for us, resulting in less maintenance pain. Thank you for reading all this and enjoy pylinting! Claudiu

3 3

some arguable pylint PR
by Sylvain Thénault 23 Nov '15

23 Nov '15

Hi there, we (Logilab) have been recently working on pylint for a customer, and provided at this occasion a couple of pull requests. I would like your opinion about some of them, not really technically but rather if you think the functionnality should go in pylint or not. The first one is about a message for 'else: if <cond>' that could be turned into 'elif <cond>: https://bitbucket.org/logilab/pylint/pull-requests/300/check-for-use-of-els… The other one is about a message about avoiding unnecessary usage of 'not' identifier. This is explained in https://bitbucket.org/logilab/pylint/issues/670/warning-upon-unneeded-negat… and there is a discussion about whether we should add a special case for class implementin '__neq__'. See https://bitbucket.org/logilab/pylint/pull-requests/293/check-for-nots-too-m… You might also be interested about this one : https://bitbucket.org/logilab/pylint/pull-requests/299/check-if-the-type-of… which checks for assigment of several types to a single identifier. Depending on the community (you) opinion, I'll integrate those in pylint or in a separated checker we've done for our customer. Regards -- Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations Développement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org

1 0

Make pylint faster by caching modules that passed without warnings
by Noam Yorav-Raphael 08 Nov '15

08 Nov '15

Hi, Take a look at cached_pylint.py at https://gist.github.com/noamraph/c933b32deb8304ac7ccd It records module dependencies, and when a module passes with no warnings, it records it together with the sha1 of the contents of each of its dependencies, so when run again it could pass without checking it. Dependencies are recorded by using infer and by using ImportsChecker, so if module A imports something from module B that was actually defined in module C, both B and C will be recorded as dependencies of A. We have a medium sized code repository that took two minutes for each check. Now pylint only checks the modified modules, and it takes two seconds. I tested it with pylint 1.4.3. Hope this helps someone, Noam

2 4