On 2/10/2015 12:31 PM, Thomas Kluyver wrote:

Maybe we should consider including functionality like that in IDLE, but that would require giving one static analysis tool the official blessing of inclusion in the standard library, which would probably be controversial.

There was a tracker proposal to include one particular checker in Idle. I rejected that for multiple reasons and suggested instead a framework for letting a user run *any* checker on editor contents. This expanded proposal was part of last summer's GSOC project but needs a bit more work. The inclusion of pip makes downloading 3rd party modules easier than it used to be. -- Terry Jan Reedy

On Tue, Feb 10, 2015 at 12:07 PM, Chris Barker wrote:

... simply not known *at* run-time.

Do you mean "simply not known *until* run-time"? Cheers, Skip

Well, Perl also dynamic language, but except the usage of the additional analyser tools he provides 'use warning' directive and many Perl's developers use this feature. Why you do not want to add such feature in Python ? If developer do not want to use this feature and want to use static analyser tools - okay. But what if someone wants to checks errata (not even an errors) without the s.a tools ? To my mind some people simply do not know that even dynamic language can do some checks.. I am, as a developer which came from Perl programming language really do not understand how you can agree with this behaviour and why you are really do not understand me.. Where it is written that dynamic language should not checks such things ? Seems very strange that Python checks ordinary blocks like 'if', 'else', 'def' statement and others..We could just have a big amount of tools to check this. I do not want to create C++ from Python. I just want to correct this situation, when you can not say to interpreter "please check at least something!!" Many thanks! - Eduard 2015-02-10 20:55 GMT+02:00 Chris Barker :

On Tue, Feb 10, 2015 at 10:51 AM, Skip Montanaro

...

wrote:

...

On Tue, Feb 10, 2015 at 12:07 PM, Chris Barker wrote:

...

... simply not known *at* run-time.

Do you mean "simply not known *until* run-time"?

indeed -- the stuff we don't even know at run time is a whole different matter ;-)

-Chris

...

Cheers,

Skip

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

Eduard, if you're going to suggest changes to Python, you really ought to first learn the current version, 3.4, and use it in your examples, rather than 2.7. (And use PEP8 style, too.) In simple cases like this, it doesn't make any real difference, and people can understand what you're suggesting--but it may still make people subconsciously less receptive (as in, "if he's not even willing to keep up with Python and see what we've improved in the last decade, what are the chances he'll have any good insight that's worth paying attention to that we haven't already thought of?"). Also, you keep comparing Python to, along with perl, C++. But in C++, many people _do_ use linters and static analyzers. Whether things like the cpplint, clang SA, the Xcode/Visual Studio/Eclipse/etc. IDE features, etc. count as "third party" is kind of a tricky question (for many developers, "comes with VS" or "comes with Xcode" probably means "not third party"), but I don't think it's really an important question; they're features that aren't part of the language, or of every implementation, that many people rely on--just like pylint, flakes8, the PyCharm IDE features, etc. in Python. They can catch things that are usually but not always wrong, or legal but stylistically bad, or illegal but a conforming compiler isn't allowed to issue an error, or expensive to check for, etc. On Feb 10, 2015, at 12:27, Ian Cordasco wrote:

On Tue, Feb 10, 2015 at 1:53 PM, Eduard Bondarenko wrote:

...

Well, Perl also dynamic language, but except the usage of the additional analyser tools he provides 'use warning' directive and many Perl's developers use this feature.

First of all, Python is not a "he". Python is an "it".

I don't know; as the mother of other languages like Boo, maybe Python as is "she". Like the mighty Crimson Permanent Assurance, she sails, striking terror into the hearts of all those born from the C, especially off the coast of Java.

On Feb 10, 2015, at 14:15, Chris Angelico wrote:

On Wed, Feb 11, 2015 at 9:09 AM, Andrew Barnert wrote:

...

(for many developers, "comes with VS" or "comes with Xcode" probably means "not third party")

For many, "comes with PyCharm" probably has the same feeling as regards Python code.

Sure, and there are probably others for whom "comes with Anaconda" does (one user on Stack Overflow refused to accept that IPython wasn't part of standard Python, although he knew the qtconsole mode wasn't). So I guess you're right, it's an even trickier question than I said. :)

On Tue, Feb 10, 2015 at 11:53 AM, Eduard Bondarenko wrote:

Well, Perl also dynamic language, but except the usage of the additional analyser tools he provides 'use warning' directive and many Perl's developers use this feature.

Why you do not want to add such feature in Python ?

I don't think anyone is saying that pre-run-time static analysis isn't useful. And I'd bet that most of the folks on this list use some or all of the tools mentioned. So the question is -- should some small subset of such analysis be built-in to the interpreter? -- and I. for one, don't think it should, at least at this point. Seems very strange that Python checks ordinary blocks like 'if', 'else',

'def' statement and others..We could just have a big amount of tools to check this.

Python checks syntax before it compiles (or while it compiles), because, well, it literally can't compile the code without correct syntax. It doesn't check things like names existing because those names could be created at run time. If a line of code never runs during testing, then it can only be assumed to be incorrect. If it does run during tests, then the really quick and easy stuff like mis-typed names will be caught right away. As there is no long compile-link step, there isn't much to be gained by the interpreter catching these things (that may not actually be errors) in the compile step. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Eduard Bondarenko writes:

Well, Perl also dynamic language, but except the usage of the additional analyser tools he provides 'use warning' directive and many Perl's developers use this feature.

Why you do not want to add such feature in Python ?

One reason would be because it doesn't exist in Perl either: $ cat test.pl use warnings; $foo = <STDIN>; if($foo eq "bar") { print("hello"); } else { print(undefined_variable) } $ perl test.pl Name "main::undefined_variable" used only once: possible typo at test.pl line 7. foo print() on unopened filehandle undefined_variable at test.pl line 6, <STDIN> line 1. Note that you only get the warning about the undefined variable when the code actually enters that branch. You are getting a warning at compile time because you used the variable only once, but I don't think that's what you'd like to add to Python. Or is it? Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

Andrew, Eduard, if you're going to suggest changes to Python, you really ought to

first learn the current version, 3.4, and use it in your examples, rather than 2.7. (And use PEP8 style, too.)

In simple cases like this, it doesn't make any real difference, and people can understand what you're suggesting--but it may still make people subconsciously less receptive (as in, "if he's not even willing to keep up with Python and see what we've improved in the last decade, what are the chances he'll have any good insight that's worth paying attention to that we haven't already thought of?").

I am agree with you. It was my fault. 2015-02-11 4:30 GMT+02:00 Stephen J. Turnbull :

Eduard Bondarenko writes:

...

So I can type whatever I want in else block and someone can detect it after the several years..

That's a shame. However, such errors are not the only way for severe errors to occur in rarely used branches. As long as your static tool can't handle *all* bugs, there's a cost/benefit question of whether the (1) programmer effort to add and maintain the functionality (including updating it for new features, especially keywords, and formerly illegal syntax), and (2) the additional time to run it are justified by the likely benefit in reduced bugginess.

The cost/benefit issue is of course always present, but it becomes discouraging here because the only way to detect non-syntax bugs before runtime is careful code review, and that is likely to catch most syntax bugs -- the benefit to additional compile-time checking is small.

Note that I'm not sure that the Python compiler currently knows enough to provide the warnings you want. Consider:

def doit(): print(a) a = "It works!" doit()

Because Python is a dynamic language, the names 'a' and 'doit' (and 'print', for that matter) must be looked up at runtime, returning objects. This means that compiling that program does not require that the compiler know whether 'a', 'doit', and 'print' are defined! So adding those warnings might require major changes to the compiler.

...

And the first thing that I have to do programming in Python is to download analyser tool..

The SEI would say "No no no! The first thing you need to do is *review your code*!" IDE developers would say "why aren't you using identifier completion?", etc, etc. This "need" for warnings from the python interpreter is very much a personal preference on your part. There's nothing wrong with having such preferences, and nothing wrong with expressing them here. You might come up with something with wide applicability to new users or to less frequent users that the heavy users who are the bulk of developers would miss.

However, I think that for both typical new users (at least those whose job title is marketer or sociologist or lawyer) and less frequent users the non-syntax G'IGO problem (where G' = good and G = garbage) is much larger than the occasional uncaught NameError. Such users don't branch as heavily as library module authors, typically branch only for easily conceptualized cases or cases observed early in development, and so often are quite well-served by "all errors are fatal at runtime". And of course this is all personal opinion, too, but I suspect similar ones are shared by many core Python developers (and I'm not one of those! so nothing I say is authoritative <wink />).

...

I understand the importance of static analyser tools, but in this case (to check simple things like a variables and functions names) it is like "To use a sledge-hammer to crack a nut.".

Again, this is a personal preference (and again, there's nothing wrong with having one or expressing it). The CPython community has generally agreed that the virtual machine (including the compiler and interpreter) should be kept as simple and clean of extra features as possible. A change to the compiler merely to warn about suspicious constructs is considered the "sledgehammer" here. On the other hand, because the constructs used by the compiler are exposed as ordinary classes, writing separate tools is fairly easy.

Also, I don't really understand your sledgehammer analogy. It's true that catching undefined names could be done separately, but running a static analysis tool will catch many other kinds of bugs that any given warning filter would not. There are many simple (one-pass, line-by-line) "linters" that only look within a single logical line for suspicious constructs, others that will check things like assignment before use. You can pick the level of analysis you like.

...

PS. Within C++ ^ *** Warning *** Every line of this program contains obscure and DANGEROUS constructs!

<wink />

...

or other languages I should use static analyser tools to check real bug-prone situation, but in new, modern Python I should use third- party tools to check the real simple errata.

I suppose from your mention of "third-party" tools in the case of C++ you mean "vendor-provided" static analyzer tools, though you didn't write that. The answer to the implied question is "yes and no". C++ compiler optimizations are notoriously buggy and backward- incompatible, and implemented differently by each vendor. And there's a tradition (maintained for "political" reasons in the case of GCC, leading to RMS's resistance to a plugin architecture and exporting the AST) of packing all functionality into a single command, even a single program. Vendor-specific, vendor-implemented checkers make a heck of a lot of sense. By contrast with GCC, LLVM's C and C++ tools look a lot like Python, with competing implementations of quite a few modules in and out of the LLVM project proper.

Now, in Python as I mentioned, it's relatively easy to write checkers, and there are many of them. There is some core agreement on the required features, but the degree of agreement is considered insufficient to justify adding a module to the standard library, especially as many of the various checkers are in active development getting new features. This isn't a blanket opposition; years ago support for "function annotations" was added, and just recently a specific syntax for type hinting has been decided (pretty much) and the PEP will probably be approved shortly, addressing both improved diagnostics and optimization needs.

Finally (and I think nobody has mentioned this yet), downloading third- party tools has always been fairly easy in Python due to the package repository PyPI, and continuous effort has been expended over the years on making it easier. In fact, for many practical purposes the "batteries included" philosophy has become impractical to support. So before you can begin to write a program, you need to download a module (and its documentation) to support functionality needed by your program. This is true for almost everybody nowadays. In response, the download and installation tool "pip" was recently added to the core distribution of both Python 2 and Python 3. That kind of generally useful effort is valued more in Python than simple addition of features that most likely would not be used by most Python programmers, not even implicitly in the standard library. And the community deliberately aims at such improvements.

Your mileage may vary, of course, but having observed the community for over a decade now I believe the above to be a fairly accurate expression of some of the core values of this community.

Regards,

Steve