code-quality October 2021

code-quality@python.org

10 participants
9 discussions

Consistent handling of left and right inputs for selected operators (with astroid?)

by Markus Elfring

Hello, Some operators are supported also by the Python programming language. There are operators which depend on two input parameters. Their representation in grammars and programming interfaces triggers special software development challenges. Examples: A) A few classes represent assignments. But their base class “AssignTypeMixin” does not provide methods for simple access to required parameters at the moment. https://github.com/PyCQA/astroid/blame/e79b0b361da0d277c11af73213b6aadc4854… B) The class “Compare” was designed with the usage of only the base class “NodeNG” so far. https://github.com/PyCQA/astroid/blame/6463dee0f5819ddfa344628aa504f226198f… How do you think about to support powerful data processing for left and right inputs together with base methods by such operator types so that source code analyses and transformations will become more convenient? Regards, Markus

1 month

Prospector on Talk Python podcast?

by Michael Kennedy

Hi folks, I'm contemplating whether it would make sense to cover prospector on the podcast. What do you all think? Interested in having one or a few people join the show? Best, Michael Sent via Superhuman <https://sprh.mn/?vip=michael@mkennedy.tech> Sent via Superhuman <https://sprh.mn/?vip=michael@talkpython.fm>

1 month, 1 week

Checking type determination with astroid for comparison operators

by Markus Elfring

Hello, The following information is provided by a documentation. https://pylint.pycqa.org/projects/astroid/en/latest/inference.html#crash-co… “… astroid offers a relatively similar API to the builtin ast module, that is, you can do astroid.parse(string) to get an AST out of the given string: >>> tree = astroid.parse('a + b') … >>> print(tree.repr_tree()) … body=[Expr(value=BinOp( op='+', left=Name(name='a'), right=Name(name='b')))]) The repr_tree() is super useful to inspect how a tree actually looks. …” Another information is provided by a related documentation. https://docs.python.org/3/library/ast.html “… cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn …” I guess that the identifier at the beginning of such a line from the abstract grammar should be interpreted as a comparison operator. Additional command examples: > python3 >>> import astroid >>> tree = astroid.parse('x == y') >>> print(tree.repr_tree()) … body=[Expr(value=Compare( left=Name(name='x'), ops=[['==', Name(name='y')]]))]) >>> tree = astroid.parse('1 in (2, 3)') >>> print(tree.repr_tree()) … body=[Expr(value=Compare( left=Const(value=1), ops=[['in', Tuple( ctx=<Context.Load: 1>, elts=[Const(value=2), Const(value=3)])]]))]) Now I am looking for more documentation so that the astroid programming interface can be safely applied also together with the shown variations of data representations. How will the chances evolve to clarify the affected implementation details a bit more? Regards, Markus

1 month, 1 week

Checking pytest dependencies for astroid

by Markus Elfring

Hello, I was informed that I should perform some software tests. https://github.com/PyCQA/astroid#test Thus I tried the following command out. [Markus_Elfring@fedora lokal]$ pytest astroid Failed to open '/proc/1/fd/2': Permission denied Traceback (most recent call last): File "/usr/bin/pytest", line 33, in <module> sys.exit(load_entry_point('pytest==6.2.2', 'console_scripts', 'pytest')()) … File "/usr/lib64/python3.9/argparse.py", line 1596, in _handle_conflict_error raise ArgumentError(action, message % conflict_string) argparse.ArgumentError: argument --no-cov: conflicting option string: --no-cov How should the mentioned system dependencies be fixed? Regards, Markus

1 month, 1 week

Checking test dependencies according to tox for astroid

by Markus Elfring

Hello, I was informed that I should perform some software tests. https://github.com/PyCQA/astroid#test Thus I tried the following command out. [Markus_Elfring@fedora lokal]$ tox -r -e py39 GLOB sdist-make: /home/altes_Heim2/elfring/Projekte/astroid/lokal/setup.py py39 recreate: /home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/py39 py39 installdeps: -rrequirements_test_min.txt, -rrequirements_test_brain.txt, coverage<5 py39 inst: /home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/.tmp/package/1/astroid-2.8.4.dev0.zip py39 installed: astroid @ file:///home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/.tmp/package/1/astroid-2.8.4.dev0.zip,attrs==21.2.0,coverage==4.5.4,iniconfig==1.1.1,lazy-object-proxy==1.6.0,nose==1.3.7,numpy==1.21.3,packaging==21.0,pluggy==1.0.0,py==1.10.0,pyparsing==3.0.0,pytest==6.2.5,python-dateutil==2.8.2,six==1.16.0,toml==0.10.2,types-attrs==19.1.0,types-python-dateutil==2.8.2,types-six==1.16.2,typing-extensions==3.10.0.2,wrapt==1.13.2 py39 run-test-pre: PYTHONHASHSEED='3647029265' py39 run-test: commands[0] | python -Wi /home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/py39/lib/python3.9/site-packages/coverage run -m pytest --pyargs tests /home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/py39/bin/python: can't open file '/home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/py39/lib/python3.9/site-packages/coverage': [Errno 2] No such file or directory ERROR: InvocationError for command /home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/py39/bin/python -Wi /home/altes_Heim2/elfring/Projekte/astroid/lokal/.tox/py39/lib/python3.9/site-packages/coverage run -m pytest --pyargs tests (exited with code 2) … ERROR: py39: commands failed How should the mentioned system dependencies be fixed? Regards, Markus

1 month, 1 week

Pylint

by Elmore, Charita R. (GSFC-705.0)[TELOPHASE CORP]

Hello, my name is Charita Elmore and I am a Supply Chain Risk Management Coordinator at NASA.  As such, I ensure that all NASA acquisitions of Covered Articles comply with Section 208 of the Further Consolidated Appropriations Act, 2020, Public Law 116-94, enacted December 20, 2019.  To do so, the Country of Origin (CoO) information must be obtained from the company that develops, produces, manufactures, or assembles the product(s).  Specifically, identify the country where each of the following products were developed, manufactured, and assembled: • Pylint  If the CoO is outside the United States, please provide any information you may have stating that testing is performed in the United States prior to supplying products to customers. Additionally, if available, please identify all authorized resellers of the product(s) in question. Lastly, as required by Section 889 of the Fiscal Year 2019 National Defense Authorization Act (NDAA) please 1.) advise if the product(s) in question is/are not manufactured by, contain components manufactured by or substantial influence from prohibited entities – Huawei, ZTE, Hytera, Hikvision, and Dahua and their subsidiaries and affiliates, and, 2.) advise if your organization has the covered telecommunications and/or video surveillance equipment or services as a substantial or essential component of any system, or as critical technology as part of any system within the organization. Ref. RITM1134907 [cid:image001.png@01D7C5AB.68B28B50]

1 month, 1 week

Pylint transform plugin and transforming imported module

by David Rajaratnam

Hi all, I'm trying to write a transform plugin to better work with a library that I maintain. For background my library extends a third-party library and uses some dynamic typing to add new features and wrappers on top of the third-party library. Because it uses dynamically created classes pylint complains about having missing members (E1101) and unexpected keyword arguments in the constructor (E1123). The class that is being dynamically generated only gets generated once when the module is imported and provides a proxy/wrapper around an existing class from the third-party library. The wrapper mirrors the member functions from the original class, as well as overriding and adding some new member functions. So the structure is something like: tpmodule.py # third-party library class Control: .... mymodule.py # my library import tpmodule def _make_wrapper(towrap): .... # dynamically create new class Control = _make_wrapper(tpmodule.Control) user.py # user code to run pylint on import mymodule ctrl = Control(...) ctrl.original_func(...) ctrl.new_func(...) My plan with the transform plugin is to use 'astroid.MANAGER.register_transform()' to detect the 'astroid.Module' calls and turn 'mymodule.py' into something more palatable for pylint; in particular to replace 'mymodule.Control' with an 'astroid.ClassDef' that has the correct initializer and member functions. Unfortunately, things are not working the way I would like and I'm still getting the missing member and bad constructor arguments errors, even though, as far as I can see, I am returning the appropriately modified module. However, there are many things that I don't understand: 1) Relating to the above example, pylint seems to be treating the 'ctrl' variable in 'user.py' 'ctrl' as being of type 'tpmodule.Control' so doesn't complain about things that are common to 'tpmodule.Control' and 'mymodule.Control'. But I cannot understand how pylint could possibly detect the relationship between 'tpmodule.Control' and 'mymodule.Control' without getting into the guts of the 'mymodule._make_wrapper()' function. 2) It is not clear to me how pylint treats imports from external libraries? In my example tpmodule.py and mymodule.py are not things that I want pylint to report on. But presumably it still needs to follow some of the chain of dependencies to work out the signature of what is exposed by the module so I understand why it would parse 'mymodule.py', but I don't understand why it would parse 'tpmodule.py'. In fact, another curious thing is that I created a cut down example to highlight the problem and while it has some of the same errors it also behaves slightly differently. In the cut-down example pylint DOESN'T actually parse 'tpmodule.py' so 'tpmodule.Control' doesn't trigger the 'register_transform()' the way it does in the real system. Even though this makes more sense to me than what is happening in the actual code, it is unfortunate because I wanted to dynamically construct the signature of 'tpmodule.Control' so was relying on pylint parsing 'tpmodule.py'. Note for clarification, in the real system 'mymodule.py' and 'tpmodule.py' are more complex and both use functions from their respective internal modules. 'tpmodule.py' is actually written in C++ and is using CFFI to generate the Python API, so maybe that can explain some difference in the behaviour to the pylint expert? Anyway, I'm sorry for the long winded questions. However, I'm hoping someone has someone insight or can point me to some docs or a specific example that solves a similar issue. I can also post a zip file of my cut down example if it is appropriate to do so. It consists of four short files ('mymodule.py', 'tpmodule.py', 'user.py', and 'plugin.py'). Regards, Dave To be clear I'm not using pylint on the library itself, instead I want pylint to work nicely with users of the library; although at some point I will try to configure pylint to help me with refactoring some of the code as well. My library extends an existing library with some new feature. and does some dynamic typing. I'm new to the group and

1 month, 4 weeks

Pylint traceback on any file, using CPython 3.6

by Dan Stromberg

The traceback looks like: $ python3.6 -m pylint keepercommander/commands/record_common.py cmd output started 2021 Mon Oct 04 03:54:33 PM PDT Traceback (most recent call last): File "/usr/local/cpython-3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/local/cpython-3.6/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/__main__.py", line 9, in <module> pylint.run_pylint() File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/__init__.py", line 24, in run_pylint PylintRun(sys.argv[1:]) File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/lint/run.py", line 268, in __init__ linter.load_default_plugins() File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/lint/pylinter.py", line 547, in load_default_plugins checkers.initialize(self) File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/checkers/__init__.py", line 84, in initialize register_plugins(linter, __path__[0]) File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/utils/utils.py", line 195, in register_plugins module.register(linter) File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/checkers/refactoring/__init__.py", line 50, in register linter.register_checker(RefactoringChecker(linter)) File "/Users/dstromberg/.local/lib/python3.6/site-packages/pylint/checkers/refactoring/refactoring_checker.py", line 468, in __init__ self._consider_using_with_stack = ConsiderUsingWithStack() TypeError: __new__() missing 3 required positional arguments: 'module_scope', 'class_scope', and 'function_scope' It doesn't appear to matter what file I try to lint; pylint gives that same traceback. Further detail: $ python3.6 -m pylint --version cmd output started 2021 Mon Oct 04 03:55:06 PM PDT pylint 2.11.1 astroid 2.8.0 Python 3.6.0 (default, Oct 4 2021, 15:45:33) [GCC Apple LLVM 12.0.5 (clang-1205.0.22.11)] $ python3.6 --version cmd output started 2021 Mon Oct 04 04:06:05 PM PDT Python 3.6.0 Pypi seems to believe that pylint still supports CPython 3.6 though. The same pylint version of the same input file with CPython 3.7 works as I'm accustomed to. Is there any possibility of a fix, or workaround? Thanks! I really like pylint. -- Dan Stromberg | Senior Software Engineer Mobile +1.949.342.6502 <https://keepersecurity.com/> ** This email is confidential and is intended for the recipient(s) addressed herein **

1 month, 4 weeks

McCabe complexity for just changed files in a commit?

by Dan Stromberg

Hi folks. Is there a way of getting the McCabe Complexity of just the functions and methods (in Python) changed in a git commit? I found radon, and it looks good. But I think it wants to do entire files, no? Thanks!

2 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

code-quality October 2021