[Python-checkins] CVS: python/dist/src/Tools/scripts fixdiv.py,1.1,1.2
Guido van Rossum
gvanrossum@users.sourceforge.net
Sat, 01 Sep 2001 21:49:38 -0700
Update of /cvsroot/python/python/dist/src/Tools/scripts
In directory usw-pr-cvs1:/tmp/cvs-serv19145
Modified Files:
fixdiv.py
Log Message:
Added more text to the docstring, updated the way the exit status is
percolated out, and some general cleanup. The output is still the
same, except it now prints "Index: <file>" instead of "Processing:
<file>", so that the output can be used as input for patch (but only
the diff-style parts of it).
Index: fixdiv.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Tools/scripts/fixdiv.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -d -r1.1 -r1.2
*** fixdiv.py 2001/09/01 21:55:58 1.1
--- fixdiv.py 2001/09/02 04:49:36 1.2
***************
*** 6,12 ****
This runs the script `yourscript.py' while writing warning messages
about all uses of the classic division operator to the file
! `warnings'. (The warnings are written to stderr, so you must use `2>'
! for the I/O redirect. I don't yet know how to do this on Windows.)
Then run `python fixdiv.py warnings'. This first reads the warnings,
looking for classic division warnings, and sorts them by file name and
--- 6,23 ----
This runs the script `yourscript.py' while writing warning messages
about all uses of the classic division operator to the file
! `warnings'. The warnings look like this:
!
! <file>:<line>: DeprecationWarning: classic <type> division
!
! The warnings are written to stderr, so you must use `2>' for the I/O
! redirect. I know of no way to redirect stderr on Windows in a DOS
! box, so you will have to modify the script to set sys.stderr to some
! kind of log file if you want to do this on Windows.
+ The warnings are not limited to the script; modules imported by the
+ script may also trigger warnings. In fact a useful technique is to
+ write a test script specifically intended to exercise all code in a
+ particular module or set of modules.
+
Then run `python fixdiv.py warnings'. This first reads the warnings,
looking for classic division warnings, and sorts them by file name and
***************
*** 15,19 ****
operators found in the source code. If it is successful, it writes a
recommendation to stdout in the form of a context diff. If it is not
! successful, it writes recommendations to stdout instead.
"""
--- 26,95 ----
operators found in the source code. If it is successful, it writes a
recommendation to stdout in the form of a context diff. If it is not
! successful, it writes observations to stdout instead.
!
! There are several possible recommendations and observations:
!
! - A / operator was found that can remain unchanged. This is the
! recommendation when only float and/or complex arguments were seen.
!
! - A / operator was found that should be changed to //. This is the
! recommendation when only int and/or long arguments were seen.
!
! - A / operator was found for which int or long as well as float or
! complex arguments were seen. This is highly unlikely; if it occurs,
! you may have to restructure the code to keep the classic semantics,
! or maybe you don't care about the classic semantics.
!
! - A / operator was found for which no warnings were seen. This could
! be code that was never executed, or code that was only executed with
! with user-defined objects as arguments. You will have to
! investigate further. Note that // can be overloaded separately from
! /, using __floordiv__. True division can also be separately
! overloaded, using __truediv__. Classic division should be the same
! as either of those. (XXX should I add a warning for division on
! user-defined objects, to disambiguate this case from code that was
! never executed?)
!
! - A warning was seen for a line not containing a / operator. This is
! an anomaly that shouldn't happen; the most likely cause is a change
! to the file between the time the test script was run to collect
! warnings and the time fixdiv was run.
!
! - More than one / operator was found on one line, or in a statement
! split across multiple lines. Because the warnings framework doesn't
! (and can't) show the offset within the line, and the code generator
! doesn't always give the correct line number for operations in a
! multi-line statement, it's not clear whether both were executed. In
! practice, they usually are, so the default action is make the same
! recommendation for all / operators, based on the above criteria.
!
! Notes:
!
! - The augmented assignment operator /= is handled the same way as the
! / operator.
!
! - This tool never looks at the // operator; no warnings are ever
! generated for use of this operator.
!
! - This tool never looks at the / operator when a future division
! statement is in effect; no warnings are generated in this case, and
! because the tool only looks at files for which at least one classic
! division warning was seen, it will never look at files containing a
! future division statement.
!
! - Warnings may be issued for code not read from a file, but executed
! using an exec statement or the eval() function. These will have
! <string> in the filename position. The fixdiv script will attempt
! and fail to open a file named "<string>", and issue a warning about
! this failure. You're on your own to deal with this. You could make
! all recommended changes and add a future division statement to all
! affected files, and then re-run the test script; it should not issue
! any warnings. If there are any, and you have a hard time tracking
! down where they are generated, you can use the -Werror option to
! force an error instead of a first warning, generating a traceback.
!
! - The tool should be run from the same directory as that from which
! the original script was run, otherwise it won't be able to open
! files given by relative pathnames.
"""
***************
*** 28,55 ****
opts, args = getopt.getopt(sys.argv[1:], "h")
except getopt.error, msg:
! usage(2, msg)
for o, a in opts:
if o == "-h":
! help()
if not args:
! usage(2, "at least one file argument is required")
if args[1:]:
sys.stderr.write("%s: extra file arguments ignored\n", sys.argv[0])
! readwarnings(args[0])
! def usage(exit, msg=None):
! if msg:
! sys.stderr.write("%s: %s\n" % (sys.argv[0], msg))
sys.stderr.write("Usage: %s warnings\n" % sys.argv[0])
sys.stderr.write("Try `%s -h' for more information.\n" % sys.argv[0])
- sys.exit(exit)
! def help():
! print __doc__
! sys.exit(0)
def readwarnings(warningsfile):
! pat = re.compile(
! "^(.+?):(\d+): DeprecationWarning: classic ([a-z]+) division$")
try:
f = open(warningsfile)
--- 104,142 ----
opts, args = getopt.getopt(sys.argv[1:], "h")
except getopt.error, msg:
! usage(msg)
! return 2
for o, a in opts:
if o == "-h":
! print __doc__
! return
if not args:
! usage("at least one file argument is required")
! return 2
if args[1:]:
sys.stderr.write("%s: extra file arguments ignored\n", sys.argv[0])
! warnings = readwarnings(args[0])
! if warnings is None:
! return 1
! files = warnings.keys()
! if not files:
! print "No classic division warnings read from", args[0]
! return
! files.sort()
! exit = None
! for file in files:
! x = process(file, warnings[file])
! exit = exit or x
! return exit
! def usage(msg):
! sys.stderr.write("%s: %s\n" % (sys.argv[0], msg))
sys.stderr.write("Usage: %s warnings\n" % sys.argv[0])
sys.stderr.write("Try `%s -h' for more information.\n" % sys.argv[0])
! PATTERN = ("^(.+?):(\d+): DeprecationWarning: "
! "classic (int|long|float|complex) division$")
def readwarnings(warningsfile):
! prog = re.compile(PATTERN)
try:
f = open(warningsfile)
***************
*** 62,66 ****
if not line:
break
! m = pat.match(line)
if not m:
if line.find("division") >= 0:
--- 149,153 ----
if not line:
break
! m = prog.match(line)
if not m:
if line.find("division") >= 0:
***************
*** 73,97 ****
list.append((int(lineno), intern(what)))
f.close()
! files = warnings.keys()
! files.sort()
! for file in files:
! process(file, warnings[file])
def process(file, list):
print "-"*70
! if not list:
! sys.stderr.write("no division warnings for %s\n" % file)
! return
try:
fp = open(file)
except IOError, msg:
sys.stderr.write("can't open: %s\n" % msg)
! return
! print "Processing:", file
f = FileContext(fp)
list.sort()
index = 0 # list[:index] has been processed, list[index:] is still to do
- orphans = [] # subset of list for which no / operator was found
- unknown = [] # lines with / operators for which no warnings were seen
g = tokenize.generate_tokens(f.readline)
while 1:
--- 160,177 ----
list.append((int(lineno), intern(what)))
f.close()
! return warnings
def process(file, list):
print "-"*70
! assert list # if this fails, readwarnings() is broken
try:
fp = open(file)
except IOError, msg:
sys.stderr.write("can't open: %s\n" % msg)
! return 1
! print "Index:", file
f = FileContext(fp)
list.sort()
index = 0 # list[:index] has been processed, list[index:] is still to do
g = tokenize.generate_tokens(f.readline)
while 1:
***************
*** 100,106 ****
--- 180,189 ----
break
assert startlineno <= endlineno is not None
+ orphans = []
while index < len(list) and list[index][0] < startlineno:
orphans.append(list[index])
index += 1
+ if orphans:
+ reportphantomwarnings(orphans, f)
warnings = []
while index < len(list) and list[index][0] <= endlineno:
***************
*** 110,114 ****
pass
elif slashes and not warnings:
! report(slashes, "Unexecuted code")
elif warnings and not slashes:
reportphantomwarnings(warnings, f)
--- 193,197 ----
pass
elif slashes and not warnings:
! report(slashes, "No conclusive evidence")
elif warnings and not slashes:
reportphantomwarnings(warnings, f)
***************
*** 223,227 ****
if token in ("/", "/="):
slashes.append((start, line))
- ## if type in (tokenize.NEWLINE, tokenize.NL, tokenize.COMMENT):
if type == tokenize.NEWLINE:
break
--- 306,309 ----
***************
*** 235,237 ****
if __name__ == "__main__":
! main()
--- 317,319 ----
if __name__ == "__main__":
! sys.exit(main())