[ python-Bugs-979739 ] compile of code with incorrect encoding yields MemoryError

[SourceForge.net]

Bugs item #979739, was opened at 2004-06-25 14:45
Message generated for change (Comment added) made by mwh
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=979739&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Anthony Tuininga (atuining)
Assigned to: Nobody/Anonymous (nobody)
Summary: compile of code with incorrect encoding yields MemoryError

Initial Comment:
The following code will fail in both Python 2.3 and
Python 2.4, raising a MemoryError exception, when run
on any platform except Windows:

compile("# -*- coding: mbcs -*-", "test.py", "exec")

This has been reproduced on the following platforms:

Linux x86
HP-UX
Solaris
Tru64 Unix

I realize this is an invalid encoding but it would be
nice if something other than MemoryError was raised!

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2004-08-07 22:33

Message:
Logged In: YES 
user_id=6656

Here's a simple and seemingly effective patch.  I'm not sure
it's "the right thing", though.  The whole issue of whether
the parser should or should not set exceptions is a
frightful mess.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2004-06-25 20:09

Message:
Logged In: YES 
user_id=11105

I assume the behaviour occurrs when an unknown encoding is
specified.  It can be reproduced on Windows:

compile("# -*- coding: xxx -*-", "test.py", "exec")

It should probably raise a SyntaxError, as trying to import
a module containing this encoding does.

The problem seems that when
PyParser_ParseStringFlagsFilename() calls
PyTokenizer_FromString(), and when the latter fails the
error is set to E_NOMEM.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=979739&group_id=5470