[issue30440] document peephole optimizer effects
Andrew Dalke
report at bugs.python.org
Tue May 23 08:52:21 EDT 2017
New submission from Andrew Dalke:
The peephole optimizer is an overall benefit to Python but it has some side-effects that occasionally cause problems. These are well-known in the issue tracker, but there is no other documentation which will help a Python programmer figure out which constructs may be a problem.
1) it will compute large integer constants and save them in the .pyc file. The following results in a 19M .pyc file.
def compute_largest_known_prime():
return 2**74207281 - 1
As an example of someone affected by the compile-time evaluation, see https://stackoverflow.com/questions/34113609/why-does-python-preemptively-hang-when-trying-to-calculate-a-very-large-number/ . Note the many people who struggled to find definitive documentation.
2) it will create and discard large strings. Consider this variant of the code in test_zlib.py, where I have replaced the imported module variable "_1G" with its value:
@bigmemtest(size=_4G + 4, memuse=1, dry_run=False)
def test_big_buffer(self, size):
data = b"nyan" * (2**30 + 1) # the original uses "_1G"
self.assertEqual(zlib.crc32(data), 1044521549)
self.assertEqual(zlib.adler32(data), 2256789997)
The byte compiler will create the ~4GB string then discard it, even though the function will not be called on machines with insufficient RAM.
As an example of how I was affected by this, see #21074 .
3) The optimizer affects control flow such that the coverage.py gives false positives about unreached code.
As examples of how people are affected, see #2506 and https://bitbucket.org/ned/coveragepy/issues/198/continue-marked-as-not-covered . The last item on the coverage.py tracker asks "Is this limitation documented anywhere?"
I do not believe that the current peephole optimizer should be changed to support these use cases, only that there should be documentation about how the optimizer may negatively affect real-world code.
The domain expert on this topic is Raymond Hettinger. He does not feel safe in issues where I am involved. As I believe my continued presence on this issue will inhibit the documentation changes which I think are needed, I will remove my name from this issue and not be further involved.
----------
assignee: docs at python
components: Documentation
messages: 294248
nosy: dalke, docs at python
priority: normal
severity: normal
status: open
title: document peephole optimizer effects
versions: Python 3.7
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30440>
_______________________________________
More information about the Python-bugs-list
mailing list