[Python-Dev] "make touch" replaced with "make regen-all"

[Ryan Gonzalez]

FWIW this will also make cross-compiling a lot easier, since you can't
accidentally overwrite the cross-compiled pgen as easily.

On Thu, May 4, 2017 at 12:13 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> tl;dr Are you ok to backport my change replacing "make touch" with
> "make regen-all"? (commit a5c62a8e)
>
>
> Since the creation of CPython, generated files were regenerated
> depending on file modification time. For development, that's a
> convenient feature. But in practice, it caused a long list of pain
> points. It caused me many issues in my experience:
>
> * On Solaris, Python failed to regenerated the AST because only the
> system Python was Python 2.6 and the script required Python 2.7 or
> newer. The "make touch" workaround didn't help, also because of the
> old version of the system Python.
>
> * On FreeBSD, generated files require "python" but only "python2.7" or
> "python3.6" programs are available. In The build system was enhanced
> to try pythonX.Y and then "python".
>
> * "make touch" workaround requires Mercurial, but also a specific
> version of Mercurial: more than once, "make touch" failed because my
> Mercurial version was too old.
>
> * Since CPython migrated to Git, "make touch" doesn't work anymore.
> Sorry, I didn't check why exactly, but I would prefer to not depend on
> Git *and* Mercurial.
>
>
> For all these reasons, it was decided to modify the CPython (UNIX/BSD)
> build system to not regenerate generated files based on file
> modification time anymore, but require an explicit action: "make
> regen-all". I already pushed my change to the master branch:
>
> https://github.com/python/cpython/commit/a5c62a8e9f0de6c4133825a5710984a3cd5e102b
>
> ---
> commit a5c62a8e9f0de6c4133825a5710984a3cd5e102b
> Author: Victor Stinner <victor.stinner at gmail.com>
> Date:   Wed May 3 18:21:48 2017 +0200
>
> bpo-23404: make touch becomes make regen-all (#1405)
>
> Don't rebuild generated files based on file modification time
> anymore, the action is now explicit. Replace "make touch"
> with "make regen-all".
>
> Changes:
>
> * Remove "make touch", Tools/hg/hgtouch.py and .hgtouch
> * Add a new "make regen-all" command to rebuild all generated files
> * Add subcommands to only generate specific files:
>
>   - regen-ast: Include/Python-ast.h and Python/Python-ast.c
>   - regen-grammar: Include/graminit.h and Python/graminit.c
>   - regen-importlib: Python/importlib_external.h and Python/importlib.h
>   - regen-opcode: Include/opcode.h
>   - regen-opcode-targets: Python/opcode_targets.h
>   - regen-typeslots: Objects/typeslots.inc
>
> * Rename PYTHON_FOR_GEN to PYTHON_FOR_REGEN
> * pgen is now only built by by "make regen-grammar"
> * Add $(srcdir)/ prefix to paths to source files to handle correctly
>   compilation outside the source directory
>
> Note: $(PYTHON_FOR_REGEN) is no more used nor needed by "make"
> default target building Python.
> ---
>
>
> See the issue for the full rationale:
>
>    http://bugs.python.org/issue23404
>
> My commit fixed the two remaining FreeBSD buildbots which were broken
> because of broken "make touch". All FreeBSD buildbots are repaired on
> master!
>
> Ok, now the question is: are you ok to backport this change to Python
> 2.7, 3.5 and 3.6?
>
> I started with a backport to 3.6:
>
> https://github.com/python/cpython/pull/1461
>
>
> See also "Test somehow that generated files are up to date: run make
> regen-all" issue:
> http://bugs.python.org/issue30259
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com



-- 
Ryan (ライアン)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://refi64.com/