[Python-bugs-list] [ python-Feature Requests-617979 ] need easy way of decoding a literal

[SourceForge.net]

Feature Requests item #617979, was opened at 2002-10-03 10:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=617979&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
>Priority: 3
Submitted By: Michael Hudson (mwh)
Assigned to: M.-A. Lemburg (lemburg)
Summary: need easy way of decoding a literal

Initial Comment:
Especially since PEP 263, we badly need a way of
turning a Python string literal into the appropriate
object.

The immediate need is to get the compiler package
working again, but I'm sure there are other places it
could be used.

I thought Martin already had a patch somewhere doing
this, but I can't find it.  Maybe he remembers?

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-01-16 13:28

Message:
Logged In: YES 
user_id=6656

For a bunch of reasons, I now care less about this so I'm
going to reduce the priority.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2002-10-04 12:15

Message:
Logged In: YES 
user_id=38388

MWH: My first paragraph was refering to Martin's suggestion
to use eval() for mapping literal strings to objects.

Please take this to the python-dev mailing list.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-10-04 10:00

Message:
Logged In: YES 
user_id=6656

MAL: I don't understand your first paragraph at all.

The {sys.,}source_encoding variable might work, but it would
be a dreadful hack for my purposes.  Using eval() is already
a nasty hack, IMHO.

I agree a mailing list is a more civilised interface than a
web browser...


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-03 20:21

Message:
Logged In: YES 
user_id=21627

That would not be correct. If you have a (plain) string
literal, it would end up encoded as UTF-8, whereas the
original source encoding was, say, Latin-1. People will be
confused if string literals don't come out in the expected
encoding in interactive mode.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2002-10-03 19:11

Message:
Logged In: YES 
user_id=38388

I believe the correct way to handle this would be to have
eval() accept Unicode and then convert it to UTF-8 for
the parser to process. The PEP only talks about compile()
which should accept Unicode objects, but eval() is certainly
a candidate too.

About the source_encoding idea: wouldn't it be better to
define this for interactive use in the sys module ? A Python
startup script could then be used to set this variable in
interactive mode.

BTW, it's better to discuss these things on python-dev than
in these small textareas... IMHO, at least.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-03 18:55

Message:
Logged In: YES 
user_id=21627

I see.

For a different reason (interactive mode), I have been
contemplating to recognize a variable source_encoding in
globals(), which would be set in interactive mode to the
terminal's encoding, so that input of non-ASCII Unicode
literals would "work", without having to type an encoding
declaration each time.

I believe such a mechanism could also solve this problem;
callers of eval would have to set the global.

What do you think?


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-10-03 17:45

Message:
Logged In: YES 
user_id=6656

How do you get eval to take notice of the declared encoding?

What I want is the functionality of compile.c:parsestr
available from Python.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-03 17:35

Message:
Logged In: YES 
user_id=21627

Not sure what you are asking. eval() would certainly be an
easy way, no?
In any case, I'm not aware of an alternative.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=617979&group_id=5470