[ python-Bugs-868864 ] 8-bit string literal with iso8859 coding => crash

[SourceForge.net]

Bugs item #868864, was opened at 2004-01-01 18:30
Message generated for change (Settings changed) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=868864&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: L. Peter Deutsch (lpd)
Assigned to: Martin v. Löwis (loewis)
Summary: 8-bit string literal with iso8859 coding => crash

Initial Comment:
Environment:
Python 2.3.3, built from unmodified out-of-the-tarball
sources
(./configure, make, make install)
Red Hat Linux 7.3, SMP build from source (2 Athlon CPUs)

Symptom:
python out.py
Segmentation fault


----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2005-08-25 09:30

Message:
Logged In: YES 
user_id=21627

This bug is now fixed in the CVS, for 2.5 and 2.4.2.

----------------------------------------------------------------------

Comment By: Mike Rozhnov (rozhnov)
Date: 2004-04-22 14:00

Message:
Logged In: YES 
user_id=1026824

Is it possible to fix this bug before Python 2.4 release?

For now choices when using 8-bit literals are:
 - always receive warning about PEP-263
 - possible crash after editing any source module with 8-bit
literals in any time

Sorry. I can't fix this bug myself (don't have and use
Visual C).

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2004-02-16 15:44

Message:
Logged In: YES 
user_id=21627

It's much worse than off-by-one. fp_readl reads size-1 bytes
from the input stream, reencodes them as UTF-8, and tries to
fit them into size bytes. In the example, it tries to put
8860 bytes into space for 8192 bytes.

I believe fp_readl should extend the buffer, using code
similar to tok_nextc.

Alternatively, fp_readl could, conservatively, request only
size/4 bytes from the underlying input stream, knowing that,
for most encodings, converting to UTF-8 will increase the
size at worst by a factor of 4. The assertion that the
result still fits should then become a FatalError.

----------------------------------------------------------------------

Comment By: Jarek Zgoda (zgoda)
Date: 2004-01-02 09:58

Message:
Logged In: YES 
user_id=92222

Python 2.3.2 (#49, Oct  2 2003, 20:02:00) [MSC v.1200 32 bit
(Intel)] on win32
Windows 2kSP4
Effect is the same - Python crashes with "memory can not be
written".

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2004-01-01 20:53

Message:
Logged In: YES 
user_id=33168

Looks like there may be an off-by-1 error in the
implementation of PEP263.  Attached is a valgrind report. 
Martin, do you have any ideas?

----------------------------------------------------------------------

Comment By: L. Peter Deutsch (lpd)
Date: 2004-01-01 18:32

Message:
Logged In: YES 
user_id=8861

'make test' ran with no problems.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=868864&group_id=5470