I need help debugging a strange test failure in pylint-django that started
happening after the upgrade to pylint/astroid 2.0.
For all I can tell we've already adjusted the code and this failure is happening
randomly. pylint-django's test used uses the test_functional.LintModuleTest
class from pylint (I know it isn't public interface) and executes against input
files comparing the linter results with expected.
The error that we get is:
E Failed: Wrong results for file "func_noerror_foreignkeys":
E Unexpected in testdata:
E 27: no-member
The strange thing is that on one test job in Travis this passed with Python 3.4
and Django 2.0 (see job #447.3):
and on the next commit this failed on Python 3.4/Django 2.0 (see #448.3):
however it passed on 3.5 + Django 2.0.
The two test jobs use the same code (minor differences in commits due to
When the linter is executed manually against the offending file it produces a
If I modify the test suite like so:
@@ -43,6 +43,8 @@ def get_tests(input_dir='input', sort=False):
suite = 
for fname in os.listdir(input_dir):
+ if fname.find('func_noerror_foreign') == -1:
if fname != '__init__.py' and fname.endswith('.py'):
then I get only a subset of the tests executed (including the offending one) and
this time it passes.
Anyone seen such erratic behaviour ? I think this is a flaky test but I have no
idea how to approach debugging that.
For the record this is blocking us from releasing a new version of pylint-django
so any help is appreciated.
I think this is not the first time this has happened to me. I am
running PyLint on Travis, and as of recently, I fixed the PyLint
version to be ==2.0.0 which is still up to date. However, now it
installs Astroid 2.0.1 which causes this:
nuitka/nodes/NodeBases.py:273 E0237 assigning-non-slot
NodeBase.setCompatibleSourceReference Assigning to attribute
'effective_source_ref' not defined in class slots
nuitka/nodes/NodeBases.py:272 I0021 useless-suppression Useless
suppression of 'attribute-defined-outside-init'
The name or kind of message that PyLint outputs changed here it seems.
Plus I am getting what I consider a false alarm:
nuitka/Errors.py:37 E1133 not-an-iterable NuitkaNodeError.__str__
Non-iterable value self.args is used in an iterating context
This is self.args from Exception, as can be seen in the code here:
# Try to output more information about nodes passed.
from nuitka.codegen.Indentation import indented
parts = [""]
for arg in self.args:
if hasattr(arg, "asXmlText"):
parts.append(indented("\n%s\n" % arg.asXmlText()))
parts.append("The above information should be included in a
And then another regression:
nuitka/nodes/NodeMetaClasses.py:66 E1121 too-many-function-args
NodeCheckMetaClass.__new__ Too many positional arguments for
return ABCMeta.__new__(cls, name, bases, dictionary)
But my question basically it, do I need to also fixate the astroid
version, so that I can be sure that a PyLint 2.0.0 is behaving like
another PyLint 2.0.0 ? This has been driving me nuts this morning,
so I have added PyLint checks to travis. It nicely makes sure that PRs
I get submitted become clean.
However, for each PyLint release, e.g. 2.0.0, there will be a bunch of
findings that are new, and will to be found, leading to the Travis
builds to fail due to PyLint, all the time now.
How to approach that. Is there a way I could have enforced using 1.9.2
on the "master" branch for Travis (and probably "develop" too), and
use the latest greatest PyLint for "factory" only. Anyone know if that
is possible, somehow, anybody doing something like that? Or maybe
install a given version, but make it an error if that version is not
the most recent on only the factory branch, what are people doing in
The pylint team is happy to announce the release of pylint 2.0 and astroid 2.0!
This release only works with Python 3.4+, while older pylint releases
are still maintained for Python 2 compatibility, at least until next year.
You can find more details about what's new in this release over here:
Thanks and enjoy linting!
Claudiu & all the Pylint contributors
I wrote a Pylint plugin to detect less than optimal usage of unittest
assertions, for example writing self.assertEqual(x, None) instead of
self.assertIsNone(x) or using deprecated aliases. It's published in
PyPI under the name pylint-unittest and is also available on GitHub
I would love to hear your feedback on it. Also, it took me longer than
I expected to put together the necessary boilerplate for doing
test-driven development on this plugin, so I hope it can serve as a
good base for others too. I feel that the way pylint-django tests are
structured is too awkward for newcomers.
The team at Software Testing Board organizes a monthly meetup where one of
the testing professionals will talk about the new Testing Framework,
Testing Strategy, any new Testing Tool or anything related to Software
These meetups are not tutorials or workshops. Just another tester is
presenting you the new tools, framework or an idea they are working on. You
can pick it up and explore it further. You can ask your questions to the
presenters and they will try to answer with the best of their knowledge.
You can even provide your feedback or suggestions for speakers or to us.
These live meetups are happening through YouTube and Google Hangout every
4th Saturday of the month at 5:00 PM IST.
To join the upcoming meetup, go to YouTube and search "Software Testing
Board". Click subscribe and press the bell icon to never miss a
notification from us.
To know more about the meetup, check out About section of the channel.
If you have any questions, feel free to write back or connect with me on