[ python-Bugs-824977 ] Memory error on AIX in email.Utils._qdecode

[SourceForge.net]

Bugs item #824977, was opened at 2003-10-16 13:40
Message generated for change (Comment added) made by bwarsaw
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=824977&group_id=5470

Category: Extension Modules
Group: Python 2.2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart D. Gathman (customdesigned)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: Memory error on AIX in email.Utils._qdecode

Initial Comment:
The following scriptlet works correctly on RedHat
python2-2.2.3, but gets a MemoryError on AIX
python-2.2.3.  This is the only anomoly for AIX python
I have seen.

import email

fp = open('analfail')
msg = email.message_from_file(fp)
for part in msg.walk():
  if part.get_main_type() == 'text':
    txt = part.get_payload(decode=True)
    #del msg["content-transfer-encoding"]
    msg.set_payload(txt)
fp.close()
print msg.as_string()

It doesn't matter whether the 'del' is there.

----------------------------------------------------------------------

>Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-11-21 16:29

Message:
Logged In: YES 
user_id=12800

Whoops, you're right.   I don't think PyMem_MALLOC
guarantees that, so the patch isn't quite right (we'd have
to zero it out ourselves).

----------------------------------------------------------------------

Comment By: Stuart D. Gathman (customdesigned)
Date: 2003-11-21 16:08

Message:
Logged In: YES 
user_id=142072

Are you sure that the following code doesn't depend on
calloc clearing the memory to 0?  Does PyMem_MALLOC clear to 0?

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-11-21 14:56

Message:
Logged In: YES 
user_id=12800

Or at least PyMem_MALLOC, right?  See attached patch
candidate.  If this fixes the problem for you (and if it
passes Michael's muster), I'll commit this to the 2.3cvs.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-10-17 07:46

Message:
Logged In: YES 
user_id=6656

Why does binascii call calloc directly?  Shouldn't it be
calling PyMem_New?

----------------------------------------------------------------------

Comment By: Stuart D. Gathman (customdesigned)
Date: 2003-10-16 16:14

Message:
Logged In: YES 
user_id=142072

Might as well fix b2a_qp while we are at it:
--- ./Modules/binascii.c.aix    Mon Mar 17 06:34:43 2003
+++ ./Modules/binascii.c        Thu Oct 16 16:08:25 2003
@@ -1036,7 +1036,7 @@
        /* We allocate the output same size as input, this
is overkill */
        odata = (unsigned char *) calloc(1, datalen);

-       if (odata == NULL) {
+       if (odata == NULL && datalen > 0) {
                PyErr_NoMemory();
                return NULL;
        }
@@ -1206,7 +1206,7 @@

        odata = (unsigned char *) calloc(1, odatalen);

-       if (odata == NULL) {
+       if (odata == NULL && odatalen > 0) {
                PyErr_NoMemory();
                return NULL;
        }


----------------------------------------------------------------------

Comment By: Stuart D. Gathman (customdesigned)
Date: 2003-10-16 16:00

Message:
Logged In: YES 
user_id=142072

Fixed binascii as follows:
--- ./Modules/binascii.c.bms    Mon Mar 17 06:34:43 2003
+++ ./Modules/binascii.c        Thu Oct 16 15:55:34 2003
@@ -1036,7 +1036,7 @@
        /* We allocate the output same size as input, this
is overkill */
        odata = (unsigned char *) calloc(1, datalen);
 
-       if (odata == NULL) {
+       if (odata == NULL && datalen > 0) {
                PyErr_NoMemory();
                return NULL;
        }

This bug will manifest not just on AIX, but any system which
returns NULL from malloc or calloc when allocated size is 0.

----------------------------------------------------------------------

Comment By: Stuart D. Gathman (customdesigned)
Date: 2003-10-16 13:47

Message:
Logged In: YES 
user_id=142072

It looks like the problem is with a2b_qp in the binascii
extension module.  So I am changing the Category.

----------------------------------------------------------------------

Comment By: Stuart D. Gathman (customdesigned)
Date: 2003-10-16 13:43

Message:
Logged In: YES 
user_id=142072

I forgot to include the traceback.

$ python t.py
Traceback (most recent call last):
  File "t.py", line 8, in ?
    txt = part.get_payload(decode=True)
  File "/usr/local/lib/python2.2/email/Message.py", line
197, in get_payload
    return Utils._qdecode(payload)
  File "/usr/local/lib/python2.2/quopri.py", line 161, in
decodestring
    return a2b_qp(s, header = header)
MemoryError


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=824977&group_id=5470