[Idle-dev] IDLE extension to integrate 3rd party checkers and analyzers

Terry Reedy tjreedy at udel.edu
Wed Jul 2 21:07:58 CEST 2014

On 7/2/2014 2:57 AM, Saimadhav Heblikar wrote:
> Preface: The existing approach of using subprocess
> http://bugs.python.org/issue21880
> did not yield good result on 3 major platforms.

Are there problems on Mac? I have not completely given up on subprocess 
on Windows for other uses, but for checking code in a Text widget, I 
think a direct approach is better anyway.

> In this post, I am outlining an alternate approach
> The proposed workflow is as follows
> IDLE  <---> Middleware Generic API <---> Middleware Checker specific
> API <--> Checker

'Middleware Generic API' == Checker API
'Middleware Checker specific API' == set of adapters

> IDLE and middleware generic api will go into Lib/idlelib. Middleware
> checker specific api will go into site-package(usually a single file).
> Checker, depends, but currently most checkers go into site-packages
> anyways.
> So, what happens is:
> Whenever IDLE receives a request to check a file with a checker, it

I think requests to check the editor buffer and requests to check a path 
should be kept separate. I prefer to start with the first.

> requests the middleware generic api, passing on all parameters as
> necessary.  The middleware generic api's only function is to pass on
> the requests to the middleware checker specific api and return the
> results back to IDLE.

For editor requests, the flow might be in the other direction.

> The middleware checker specific api, will *import* the checker and do
> the processing. It will return the result,(after formatting if
> necessary).

Or have the specific api pass the specifics back to idle to use.

Here is how trivial the interfacing is for pyflakes.
class CW:  # CheckerWindow with .write, for instance
     def __init__(self):
         # self.text = Text() (or passed in param)
         self.text = []

     def write(self, chunk):
         # self.text.insert('insert', chunk)
     def get(self):  # for this demo
         # self.text.get('1.end') or whatever
         return ''.join(self.text)
cw = CW()

# Specific interface: import and call
from pyflakes import api, reporter

# Call as it mightbe configured.
# "api.check({code}, {fname}, reporter.Reporter({stream}, {stream}))"
# where 'stream' is an object with a .write method.
# Let ew be the EditorWindow being checked. Then use above with
# .format(code=ew.text.get('1.end'), fname=ew.???, stream = cw)
# The two args to Reporter are warning and error streams.
# Errors incluce compile() SyntaxErrors, as in second example below.
# A separate error stream could 'color' error text if desired.
# It could also separate counting of warnings from errors

api.check('a = b\n', 'tem.py', reporter.Reporter(cw, cw))
api.check('a =\n', 'tem.py', reporter.Reporter(cw, cw))

print(cw.get())  # view result in CW
tem.py:1: undefined name 'b'
tem.py:1:4: invalid syntax
a =

I believe that any properly written Python-coded checker/analyzer should 
provide such an interface, so the input and output can be passed back 
and forth as Python strings without encoding to or decoding from bytes 
(in Py3) or writing to or reading from external files.

I would like to see a revised patch on 21880 that runs pyflakes this way 
so we have a working example for discussion of the ui, the dialogs and 
output window.

> Getting the checker specific api file into the users system could be
> done through existing framework: pip
> To ensure that the checker is present on the users system, we could
> mark it as a dependency when uploading to PyPI.
> For the middleware generic api to detect existing checkers: Create a
> idlelib specific folder in site-packages(i.e. every middleware checker
> specific api file to be placed into site-packages/idlelib/ to enable
> detection)

I dislike creating lots of little 100 bytes files. This is what I like 
about the config.def files.

> Installation of middleware checker api could be done via command
> line.(We could also support from inside IDLE, but that is a separate
> issue altogether)
> It could be something like : pip install idle-checker-x etc.

> My proposal will avoid 'favoritism' in the checkers supported as well
> as not hardcode any checker(and its interface to IDLE) into idlelib

As I said on email, I am willing to favor programs that provide what i 
consider to be a proper interface. I would like to see some research 
into other programs and whether they have apis anything like the one for 
pyflakes, or what adaptor would be needed.

Terry Jan Reedy

More information about the IDLE-dev mailing list