[ python-Bugs-1532726 ] incorrect behaviour of PyUnicode_EncodeMBCS?

[SourceForge.net]

Bugs item #1532726, was opened at 2006-08-01 23:20
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1532726&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Jan-Willem (jwnmulder)
Assigned to: Nobody/Anonymous (nobody)
Summary: incorrect behaviour of PyUnicode_EncodeMBCS?

Initial Comment:
Using python 2.4.3
This behaviour is not reproducable on a window or 
linux machine. I found the bug when trying to find a 
problem on python 2.4.3 ported to the xbox.

running the next two commands

  test_string = 'encode me'
  print repr(test_string.encode('mbcs'))

results on windows in : 'encode me'
and on the xbox : 'encode me\\x00'

The problem is that 'PyUnicode_EncodeMBCS' returns an 
PyStringObject that contains the data 'encode me' but 
with an object size of 10.
string_repr(test_string) assumes the string contains 
a 0 character and encodes it as '\\x00'

looking at the function 'PyUnicode_EncodeMBCS(const 
Py_UNICODE *p, int size, const char *errors)' there 
are basicly two functions

{
  mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, 
NULL, 0, NULL, NULL);
  repr = PyString_FromStringAndSize(NULL, mbcssize);
}

WideCharToMultiByte returns the nummer of bytes 
needed for the buffer, because of the string 
termination this functions returns 10.
PyString_FromStringAndSize assumes its second 
argument to be the number of needed characters, not 
bytes. So an easy fix would be
to change
  repr = PyString_FromStringAndSize(NULL, mbcssize);
in
  repr = PyString_FromStringAndSize(NULL, mbcssize - 
1);

Just checked the 2.4.3 svn trunk and it contains the 
same bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1532726&group_id=5470