[New-bugs-announce] [issue39721] Fix constness of members of tok_state struct.

[Andy Lester]

New submission from Andy Lester <andy at petdance.com>:

The function PyTokenizer_FromUTF8 from Parser/tokenizer.c had a comment:

    /* XXX: constify members. */

This patch addresses that.

In the tok_state struct:
    * end and start were non-const but could be made const
    * str and input were const but should have been non-const

Changes to support this include:
    * decode_str() now returns a char * since it is allocated.
    * PyTokenizer_FromString() and PyTokenizer_FromUTF8() each creates a
        new char * for an allocate string instead of reusing the input
        const char *.
    * PyTokenizer_Get() and tok_get() now take const char ** arguments.
    * Various local vars are const or non-const accordingly.

I was able to remove five casts that cast away constness.

----------
components: Interpreter Core
messages: 362441
nosy: petdance
priority: normal
severity: normal
status: open
title: Fix constness of members of tok_state struct.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39721>
_______________________________________