IMAP4 example in docs causes memory error for me (OS X 10.3, python 2.3) - help?
t-meyer at ihug.co.nz
Wed Sep 15 08:35:30 CEST 2004
> I am running the IMAP4 example in the documentation:
> with two small changes (different username, comment out print
> statement). The inbox I am accessing is 42 MB and has
> 80-someodd messages, so the messages are fairly big (all of
> them have attachments). When I run this program, I get a
> malloc error.
Background: I don't have access to a mac at the moment to test this there,
so this is theory based. However, I do look after the IMAP filter script
that's part of SpamBayes, so get a fair bit of mail about that, including
from mac users. Some of those have reported much the same error, and
googling about for information gives some, although not a lot.
The information that I found at the time indicated that this was the result
of a change introduced with OS X 10.3. The problem always occurs at line
301 of socket.py, which is basically just retrieving the data from the
socket. Under the hood (i.e. in the C of the socket module, around line
2024), all it's really doing is allocating memory for the amount to be
received, and receiving it.
With SpamBayes, I just wormed around the problem (i.e. I skipped those
messages). If you do need to get hold of them, then I imagine that this
will be rather annoying to fix. My ideas at the moment are:
* I suspect that you could retrieve chunks of the message (with the
appropriate IMAP commands) and then put it all together afterwards.
* You could modify imaplib (or have a subclass) that (in _getresponse())
alters the way that literals are read (i.e. change it to chunks).
* Otherwise, particularly if you are able to run with a modified Python,
you could try fixing up socketmodule.c to do this more nicely on OS X.
I plan to look into it more at some point, but have no idea when I'll have
If you do find a workable solution, please post it to the list, as it would
definitely be of interest.
> Now, I'm really new to python programming, but
> it was my understanding that memory management was not
> something I had to explicitly do. I'd appreciate some help in
> understanding what I've done wrong.
FWIW, you shouldn't ever need worry about the malloc'ing with Python (unless
you're writing extensions etc). This is a very odd case, caused by some OS
X 10.3 oddity. I'm fairly confident that if you wrote a (correct) little C
program that asked for the same amount of memory, OS X would fail there,
More information about the Python-list