Hi Oscar, Thanks for the feedback. On 07/12/2019 7:37 pm, Oscar Benjamin wrote:
On Sat, 7 Dec 2019 at 06:29, Steven D'Aprano <steve@pearwood.info> wrote:
A million seems reasonable for lines of source code, if we're prepared to tell people using machine generated code to split their humongous .py files into multiple scripts. A small imposition on a small subset of Python users, for the benefit of all. I'm okay with that.
I recently hit on a situation that created a one million line code file: https://github.com/pytest-dev/pytest/issues/4406#issuecomment-439629715
The original file (which is included in SymPy) has 3000 lines averaging 500 characters per line so that the total file is 1.5MB. Since it is a test file pytest rewrites the corresponding pyc file and adds extra lines to annotate the intermediate results in the large expressions. The pytest-rewritten code has just over a million lines.
There are two possible solutions here (in the context of PEP 611) 1. Split the original SymPy test file into two or more files and the test function into many smaller functions. 2. Up the line limit to two million and the bytecode limit to many million. Note that changing pytest to output fewer lines won't work as we will just hit the bytecode limit instead. 1. Is this difficult? I wouldn't expect it to be so. 2. Simple, but with a performance impact. The simplest solution appears to be to just up the limits, but the problem with that is any costs we incur are incurred by all Python programs forever. Fixing the test is a one off cost.
When I first tried pytest with this file it lead to a CPython segfault. It seems that the crash in CPython was fixed in 3.7.1 though so subsequent versions can work fine with this (although it is slow).
The tests in the file are skipped anyway so I just made sure that the file was blacklisted in SymPy's pytest configuration.
-- Oscar _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IUG6EBER... Code of Conduct: http://python.org/psf/codeofconduct/