[New-bugs-announce] [issue38937] NameError in list comprehension within .pth file

[Chris Billington]

New submission from Chris Billington <chrisjbillington at gmail.com>:

The following one-liner works fine in a regular Python interpreter:

$ python -c 'import sys; x = 5; [print(x + i) for i in range(5)]'
5
6
7
8
9

But in a .pth file, it raises a NameError:

$ echo 'import sys; x = 5; [print(x + i) for i in range(5)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
$ python
Error processing line 1 of /usr/lib/python3.8/site-packages/test.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.8/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "<string>", line 1, in <listcomp>
  NameError: name 'x' is not defined

Remainder of file ignored
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Since site.py uses exec() to exec each line of a .pth file, I thought I'd compare with that. It also works fine:

$ python -c 'exec("import sys; x = 5; [print(x + i) for i in range(5)]")'
5
6
7
8
9

This slight modification (the variable being used in the next statement still, but not within the loop of the comprehension) does not raise a NameError:

$ echo 'import sys; x = 5; [print(i) for i in range(x)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; x = 5; [print(i) for i in range(x)]
$ python
0
1
2
3
4
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

I know .pth file processing is very early in interpreter startup such that many things aren't working yet, but I wouldn't expect using a name defined outside a list comprehension within the loop body of said list comprehension not to work.

The following is fine also, showing that using names from outside the list comprehension doesn't always break:

$ echo 'import sys; [print(sys) for i in range(5)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; [print(sys) for i in range(5)]
$ python
<module 'sys' (built-in)>
<module 'sys' (built-in)>
<module 'sys' (built-in)>
<module 'sys' (built-in)>
<module 'sys' (built-in)>
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

This is fine too:

$ echo 'import sys; [print(str(sys) * i) for i in range(5)]' | sudo tee /usr/lib/python3.8/site-packages/test.pth
import sys; [print(str(sys) * i) for i in range(5)]
$ python

<module 'sys' (built-in)>
<module 'sys' (built-in)><module 'sys' (built-in)>
<module 'sys' (built-in)><module 'sys' (built-in)><module 'sys' (built-in)>
<module 'sys' (built-in)><module 'sys' (built-in)><module 'sys' (built-in)><module 'sys' (built-in)>
Python 3.8.0 (default, Oct 23 2019, 18:51:26) 
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 


My use case is looping over subdirs of a directory and adding them all to sys.path to provide similar functionality to python setup.py develop, with all python vcs repositories within a specific directory being prepended to sys.path, rather than having to add them one-by-one. I probably won't end up doing what I'm doing this way, but in any case the above seems like it's a bug, unless I'm grossly misunderstanding something.

----------
components: Interpreter Core
messages: 357627
nosy: Chris Billington
priority: normal
severity: normal
status: open
title: NameError in list comprehension within .pth file
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38937>
_______________________________________