[Python-bugs-list] [ python-Bugs-619789 ] segmentation fault importing huge source

[noreply@sourceforge.net]

Bugs item #619789, was opened at 2002-10-07 17:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=619789&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Gerson Kurz (gersonkurz)
Assigned to: Jason Tishler (jlt63)
Summary: segmentation fault importing huge source

Initial Comment:
I posted a problem with importing a very large sourcefile 
in python 2.2.1 to clp, see 
<news:3da070ec.18868296@news.t-online.de>. 
Someone suggested that 2.3 would handle this situation 
much better, so I got the current version from CVS, built 
it with cygwin and tested it. That version raises a 
segmentation fault. The stackdump shows an 
ACCESS_VIOLATION somewhere inside python2.3.exe.

You can download the source that causes this at

http://p-nand-q.com/python/toobigfor2.3.zip

but beware, the zipped source is 613k large (unzipped 
around 4.2 mb).


----------------------------------------------------------------------

>Comment By: Gerson Kurz (gersonkurz)
Date: 2002-10-19 05:16

Message:
Logged In: YES 
user_id=367261

you are right, it works with cygwin 1.3.13 - the allocation 
succeeds. you can close the bug report. 

----------------------------------------------------------------------

Comment By: Jason Tishler (jlt63)
Date: 2002-10-18 20:14

Message:
Logged In: YES 
user_id=86216

Yes, this is the same Cygwin malloc problem.

Gerson, please upgrade to Cygwin 1.3.13.

IMO, the following is the easiest way to verify that the
problem has been corrected:

$ python -c 'import sys; list(xrange(sys.maxint / 4))'
Traceback (most recent call last):
  File "<string>", line 1, in ?
MemoryError

Gerson, please let me know if it's OK to close this
bug report.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-10-18 19:35

Message:
Logged In: YES 
user_id=33168

Jason can you shed any more light on this problem?  Is this
the same cygwin malloc problem or a new problem?

----------------------------------------------------------------------

Comment By: Gerson Kurz (gersonkurz)
Date: 2002-10-13 15:47

Message:
Logged In: YES 
user_id=367261

It is the latest version of cygwin I know of, the cygwin1.dll is 
dated 2002-07-06 02:16. uname -a says

CYGWIN_NT-5.0 DREIZEHN 1.3.12(0.54/3/2) 2002-07-06 
02:16 i686 unknown

so I guess its the 1.3.12 you mentioned.

I can reproduce the crash using the following very simple 
script

b = {}
for i in range(900000):
    b[i] = [0] * 60

when I use range(900000), I get a crash. When I use range
(800000), I see this

Traceback (most recent call last):
  File "test2.py", line 3, in ?
    b[i] = [0] * 60

and then a crash. When I use (500000), it works. It smells 
like some 256mb limit or something; the memory for the 
process goes up to 264.000k which comes close, and then 
crashes. 

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-10-11 01:43

Message:
Logged In: YES 
user_id=33168

Do you know where python crashed?  My guess was after a
failed malloc.  I tried testing this, but I don't have
enough memory/cpu/patience to wait for loading such a large
file.  Can you make the test case smaller?  What is the
minimum size that still causes the seg fault?  What version
of cygwin?  There is a known malloc problem in 1.3.12, see
http://sources.redhat.com/ml/newlib/2002/msg00369.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=619789&group_id=5470