[New-bugs-announce] [issue39235] Generator expression has wrong line/col info when inside a Call object

[Lysandros Nikolaou]

New submission from Lysandros Nikolaou <lisandrosnik at gmail.com>:

A normal generator expression like (i for i in a) produces the following AST:

Module(
    body=[
        Expr(
            value=GeneratorExp(
                elt=Name(
                    id="i", ctx=Load(), lineno=1, col_offset=1, end_lineno=1, end_col_offset=2
                ),
                generators=[
                    comprehension(
                        target=Name(
                            id="i",
                            ctx=Store(),
                            lineno=1,
                            col_offset=7,
                            end_lineno=1,
                            end_col_offset=8,
                        ),
                        iter=Name(
                            id="a",
                            ctx=Load(),
                            lineno=1,
                            col_offset=12,
                            end_lineno=1,
                            end_col_offset=13,
                        ),
                        ifs=[],
                        is_async=0,
                    )
                ],
                lineno=1,
                *col_offset=0,*
                end_lineno=1,
                *end_col_offset=14,*
            ),
            lineno=1,
            col_offset=0,
            end_lineno=1,
            end_col_offset=14,
        )
    ],
    type_ignores=[],
)

But when calling a function with a generator expression as an argument, something is off:

Module(
    body=[
        Expr(
            value=Call(
                func=Name(
                    id="f", ctx=Load(), lineno=1, col_offset=0, end_lineno=1, end_col_offset=1
                ),
                args=[
                    GeneratorExp(
                        elt=Name(
                            id="i",
                            ctx=Load(),
                            lineno=1,
                            col_offset=2,
                            end_lineno=1,
                            end_col_offset=3,
                        ),
                        generators=[
                            comprehension(
                                target=Name(
                                    id="i",
                                    ctx=Store(),
                                    lineno=1,
                                    col_offset=8,
                                    end_lineno=1,
                                    end_col_offset=9,
                                ),
                                iter=Name(
                                    id="a",
                                    ctx=Load(),
                                    lineno=1,
                                    col_offset=13,
                                    end_lineno=1,
                                    end_col_offset=14,
                                ),
                                ifs=[],
                                is_async=0,
                            )
                        ],
                        lineno=1,
                        *col_offset=1,*
                        end_lineno=1,
                        *end_col_offset=2,*
                    )
                ],
                keywords=[],
                lineno=1,
                col_offset=0,
                end_lineno=1,
                end_col_offset=15,
            ),
            lineno=1,
            col_offset=0,
            end_lineno=1,
            end_col_offset=15,
        )
    ],
    type_ignores=[],
)


I'm not sure if this is intentional or not, because there is a call to copy_location in Python/ast.c:3149. If this call to copy_location is removed, the inconsistency goes away.

----------
components: Interpreter Core
messages: 359454
nosy: lys.nikolaou
priority: normal
severity: normal
status: open
title: Generator expression has wrong line/col info when inside a Call object
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39235>
_______________________________________