[ python-Bugs-1306484 ] compile() converts "filename" parameter to StringType

[SourceForge.net]

Bugs item #1306484, was opened at 2005-09-28 04:49
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1306484&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Vágvölgyi Attila (wigy)
Assigned to: Martin v. Löwis (loewis)
Summary: compile() converts "filename" parameter to StringType

Initial Comment:
The builtin compile() signature looks like:

  compile(string, filename, kind[, flags[, dont_inherit]])

The string parameter can be either StringType or
UnicodeType, but the filename parameter will be
converted to StringType, so if there are non-ascii
characters in the unicode object passed, it raises
UnicodeEncodeError.

This can be an issue on filesystems having utf-8
filenames, or when using non-English names for the
backtrace beautification.

The attached file contains a unit test that will
succeed when the bug is resolved. I saw the error in
2.3 and 2.4, maybe it is there for all releases?

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2005-09-29 06:34

Message:
Logged In: YES 
user_id=849994

Sounds sound. :)

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2005-09-29 06:20

Message:
Logged In: YES 
user_id=21627

Why couldn't co_filename just be the Unicode string? I think
one would have to change:
- code_repr, to convert the filename into a byte string
(preferably using 'ascii', 'replace')
- tb_printinternal (not sure what to do here)
- code_new, to accept either strings or unicode strings
- builtin_compile, which probably indeed needs to convert
the string using the file system encoding, and then patch
the resulting code object to point to the unicode object
originally passed (unless we can accept more pythonrun
functions).

----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-09-28 12:54

Message:
Logged In: YES 
user_id=1188172

Should compile() use the Py_FileSystemEncoding?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1306484&group_id=5470