[New-bugs-announce] [issue46565] Multiple modules leak `for` loop variables into module's namespace

[Nikita Sobolev]

New submission from Nikita Sobolev <mail at sobolevn.me>:

Some variables created as `for X in ...` leak into module's namespace, where the loop is defined.

I wrote a simple `flake8` plugin to find names that are used in `ast.Module` in `ast.For`, but not under `if __name__ == '__main__'` and are not used in `del` afterwards.

Here's what I got:

- Lib/inspect.py:157
- Lib/locale.py:746
- Lib/sysconfig.py:186
- Lib/tokenize.py:141 - 151
- Lib/multiprocessing/process.py:427
- Lib/multiprocessing/managers.py:55 
- Lib/json/encoder.py:30
- Lib/http/cookiejar.py:93 
- Lib/email/contentmanager.py:73
- Lib/email/contentmanager.py:79
- Lib/email/contentmanager.py:247
- Lib/email/quoprimime.py:60
- Lib/email/quoprimime.py:149
- Lib/_compat_pickle
- Lib/lib2to3/pgen2/grammar.py

I think, that we need to remove these names. Why?
1. They complicate typeshed typing, we have to annotate them in typeshed, or write custom ignore rules for our test suite. Ref: https://github.com/python/typeshed/blob/56aa2088aada530400b6fdddf0f1d17ca3aaa86f/tests/stubtest_allowlists/py3_common.txt#L448

2. They are in `dir()`, example:

```
>>> import inspect
>>> 'k' in dir(inspect)
True
```

3. They are listed in `help()`, let's use `json.encoder` as an example:

```
DATA
    ESCAPE = re.compile('[\\x00-\\x1f\\\\"\\b\\f\\n\\r\\t]')
    ESCAPE_ASCII = re.compile('([\\\\"]|[^\\ -~])')
    ESCAPE_DCT = {'\x00': r'\u0000', '\x01': r'\u0001', '\x02': r'\u0002',...
    HAS_UTF8 = re.compile(b'[\x80-\xff]')
    INFINITY = inf
    i = 31
```

4. We also have to exclude them sometimes in tests, like https://github.com/python/cpython/blob/db77bcd6092f3c174ae855522411ab83854d65a8/Lib/test/test_inspect.py#L111


I think that adding `del X` somewhere in these modules is a good thing:
1. Not hard to backport
2. Fixes multiple issues above
3. Does not store useless objects in memory
4. Does not confuse people
5. Some modules already delete unused intermediate vars, so it is not something new to CPython, for example: `multiprocessing.process` https://github.com/python/cpython/blob/db77bcd6092f3c174ae855522411ab83854d65a8/Lib/multiprocessing/process.py#L419

PR is on its way!

----------
components: Library (Lib)
messages: 412006
nosy: sobolevn
priority: normal
severity: normal
status: open
title: Multiple modules leak `for` loop variables into module's namespace
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46565>
_______________________________________