[ python-Bugs-931336 ] RecenFilesList mysteriously containing str with NULL bytes

[SourceForge.net]

Bugs item #931336, was opened at 2004-04-07 16:25
Message generated for change (Comment added) made by kbk
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=931336&group_id=5470

Category: IDLE
Group: Irreproducible
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Gustavo Tabares (gustabares)
Assigned to: Kurt B. Kaiser (kbk)
Summary: RecenFilesList mysteriously containing str with NULL bytes

Initial Comment:
Please note that I have only seen this a few (3) times. I 
am unable to reproduce this. 

When I attempt to start IDLE from within Windows it 
does not bring up any window. When I tried to start it 
from the command line, I received this error:

C:\Python23\Lib\idlelib>python idle.py
Traceback (most recent call last):
  File "idle.py", line 23, in ?
    idlelib.PyShell.main()
  File "C:\Python23\lib\idlelib\PyShell.py", line 1271, in 
main
    flist.new()
  File "C:\Python23\lib\idlelib\FileList.py", line 62, in new
    return self.EditorWindow(self, filename)
  File "C:\Python23\lib\idlelib\PyShell.py", line 80, in 
__init__
    EditorWindow.__init__(self, *args)
  File "C:\Python23\lib\idlelib\EditorWindow.py", line 186, 
in __init__
    self.UpdateRecentFilesList()
  File "C:\Python23\lib\idlelib\EditorWindow.py", line 595, 
in UpdateRecentFilesList
    rfList=self.__CleanRecentFiles(rfList)
  File "C:\Python23\lib\idlelib\EditorWindow.py", line 620, 
in __CleanRecentFiles
    if not os.path.exists(path[0:-1]):
  File "C:\Python23\lib\ntpath.py", line 256, in exists
    st = os.stat(path)
TypeError: stat() argument 1 must be (encoded string 
without NULL bytes), not str

With a little print statement of what "path" is in this 
case, I found out that it is the following:

['\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00']

I have worked around the problem by clearing out rfList 
on line 619 of EditorWindow.py. I'm guessing I'm getting 
into some sort of weird state but it is worth mentioning.

----------------------------------------------------------------------

>Comment By: Kurt B. Kaiser (kbk)
Date: 2004-04-10 22:36

Message:
Logged In: YES 
user_id=149084

Modify EditorWindow.update_recent_files_list()
to remove paths containing NULL.

Source of corruption in OP's .idlerc/recent-files.lst is
unknown.

EditorWindow.py 1.57



----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2004-04-08 16:41

Message:
Logged In: YES 
user_id=21627

The posix module is behaving correctly here, I cannot see a
problem. Posix does not allow file names with embedded NUL
bytes, as open(2), stat(2) etc. take a null-terminated
string, so it is *impossible* to use a string with embedded
null bytes as a path name.

If the posix module would allow embedded NUL bytes, the
system would silently truncate the file name, and
create/stat a file with a different name. Therefore, the
posix module must perform the test and raise an exception.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2004-04-08 16:20

Message:
Logged In: YES 
user_id=149084

Error is produced by 

>>> import os
>>> os.stat("foo\0.py")
TypeError: stat() argument 1 must be (encoded string 
without NULL bytes), not str

Error trace in reverse (using my posix box):
Python/getargs.c:	 
  convertsimple()
  convertitem()
  vgetargs1()
  PyArg_ParseTuple(path, "et:stat", ...)
Modules/posixmodule.c:	 
  posix_do_stat(self, path, "et:stat", STAT, NULL, NULL)
  posix_stat(self, path)

posix_do_stat() does path validity checks before calling the
system stat function.  The "et" format doesn't allow embedded
nulls, and the error is a TypeError from convertsimple(), rather
than the os.error [no path] that ntpath.py: exist() is
expecting.

Nulls in a path cause a lot of functions in os.path to break.  
1. Should this be fixed?
2. If so, in posix_do_stat() ?
3. Or in [ntpath, posixpath, macpath] by 
      s/os.error/(os.error, TypeError)/ ?
4. Or just put a test in IDLE?

Assigning to Martin for an opinion.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2004-04-07 21:42

Message:
Logged In: YES 
user_id=149084

Next time this happens, could you post the contents of your
~/.idlerc/recent-files.lst?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=931336&group_id=5470