[New-bugs-announce] [issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[Miro Hrončok]

New submission from Miro Hrončok <miro at hroncok.cz>:

In Fedora, when building Python 3.11.0a2 with Python 3.11.0a2 for regen, we found that regen-frozen fails.

It can be reproduced either on the v3.11.0a2 tag or on the main branch.

On tag v3.11.0a2 with 3.11.0a2 installed in $PATH:

# in root of cpython local clone:
$ git clean -fdx
$ mkdir -p build/optimized
$ cd build/optimized
$ ../../configure
$ make regen-all PYTHON_FOR_REGEN=python3.11
...
ERROR: missing _freeze_module


On the main branch:

# in root of cpython local clone:
$ git clean -fdx
$ mkdir -p build/optimized
$ cd build/optimized
$ ../../configure
$ make
$ make regen-all PYTHON_FOR_REGEN=./python
(success)

However, the working tree is dirty:

$ git diff
diff --git a/Parser/parser.c b/Parser/parser.c
index b3aa35989ed..c26cd6eeb05 100644
--- a/Parser/parser.c
+++ b/Parser/parser.c
@@ -1,4 +1,4 @@
-// @generated by pegen from ./Grammar/python.gram
+// @generated by pegen from ../../Grammar/python.gram
 #include "pegen.h"
 
 #if defined(Py_DEBUG) && defined(Py_BUILD_CORE)
diff --git a/Tools/peg_generator/pegen/grammar_parser.py b/Tools/peg_generator/pegen/grammar_parser.py
index 6e9f7d3d11d..fbbbfad76b0 100644
--- a/Tools/peg_generator/pegen/grammar_parser.py
+++ b/Tools/peg_generator/pegen/grammar_parser.py
@@ -1,5 +1,5 @@
 #!/usr/bin/env python3.8
-# @generated by pegen from ./Tools/peg_generator/pegen/metagrammar.gram
+# @generated by pegen from ../../Tools/peg_generator/pegen/metagrammar.gram
 
 import ast
 import sys


And if we install Python somewhere:

$ make
$ make install DESTDIR=/tmp/python

And use that one again for regen:

$ cd ../..
$ git clean -fdx
$ mkdir -p build/optimized
$ cd build/optimized
$ ../../configure
$ make regen-all PYTHON_FOR_REGEN=/tmp/python/usr/local/bin/python3.11
...
# Regenerate Lib/keyword.py from Grammar/python.gram and Grammar/Tokens
# using Tools/peg_generator/pegen
PYTHONPATH=../../Tools/peg_generator /tmp/python/usr/local/bin/python3.11 -m pegen.keywordgen \
	../../Grammar/python.gram \
	../../Grammar/Tokens \
	../../Lib/keyword.py.new
/tmp/python/usr/local/bin/python3.11 ../../Tools/scripts/update_file.py ../../Lib/keyword.py ../../Lib/keyword.py.new
/tmp/python/usr/local/bin/python3.11 ../../Tools/scripts/freeze_modules.py
ERROR: missing _freeze_module
make: *** [Makefile:1259: regen-frozen] Error 1



It fails. I've isolated the failure to:

$ make regen-frozen PYTHON_FOR_REGEN=/tmp/python/usr/local/bin/python3.11
/tmp/python/usr/local/bin/python3.11 ../../Tools/scripts/freeze_modules.py
ERROR: missing _freeze_module
make: *** [Makefile:1259: regen-frozen] Error 1

----------
components: Build
messages: 406764
nosy: eric.snow, hroncok, petr.viktorin, vstinner
priority: normal
severity: normal
status: open
title: Out of tree build of Python 3.11.0a2+ breaks regen-frozen
type: compile error
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45866>
_______________________________________