[Python-bugs-list] [ python-Bugs-561858 ] Assertion with very long lists
noreply@sourceforge.net
noreply@sourceforge.net
Thu, 13 Jun 2002 17:14:12 -0700
Bugs item #561858, was opened at 2002-05-29 06:12
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=561858&group_id=5470
Category: Python Interpreter Core
Group: Python 2.1.2
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Jerome Alet (jalet)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Assertion with very long lists
Initial Comment:
I've tried to import a module which only define a list
of 170000 elements (a word list) which defines an
element per line, so the module also has 170000 lines
in it. Here's the error :
--- CUT ---
Python 2.1.3 (#1, Apr 20 2002, 10:14:34)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more
information.
>>> import dictionnaire
python: ../Python/ceval.c:695: eval_code2: Assertion
`(stack_pointer - f->f_valuestack) <= f->f_stacksize'
failed.
Abandon
--- CUT ---
this seems to be the same problem as bug id 210647 but
there's no segfault.
the module is attached to this bug report.
----------------------------------------------------------------------
>Comment By: Guido van Rossum (gvanrossum)
Date: 2002-06-13 20:14
Message:
Logged In: YES
user_id=6380
This looks good to me. You might want to increase the size
of the list in the testcase so that it actually crashes for
non-debug Python without the fix; I could only get it to
fail an assertion in debug mode.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-06-13 18:08
Message:
Logged In: YES
user_id=33168
Guido, could you review this patch before checking in?
Thanks.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-06-04 13:20
Message:
Logged In: YES
user_id=33168
Tim, could you also review long-list.diff. This is the
patch against CVS version.
It changes what is marshalled to disk. Thanks.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-06-01 14:28
Message:
Logged In: YES
user_id=33168
Thanks Tim.
Checked in for 2.1/2.2 as 2.175.2.3 and 2.192.6.2.
Could you or someone also review the other patch for current
CVS?
That patch has a test and is different ([rw]_short ->
[rw]_long).
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-06-01 14:14
Message:
Logged In: YES
user_id=31435
Assuming the patch you're talking about is the one named
long-list-2.2.diff, looks good to me, so Accepted and
assigned back to you. Thanks!
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-06-01 10:56
Message:
Logged In: YES
user_id=33168
Added a patch for 2.2 (2.1) is basically the same.
No test for 2.2/2.1 though.
Could someone please review before checkin?
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-05-31 22:04
Message:
Logged In: YES
user_id=33168
Final patch attached.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-05-31 20:17
Message:
Logged In: YES
user_id=33168
Got a fix. Patch attached.
It will probably have to be changed,
since I don't remember what else needs to be
done when the MAGIC is changed.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-05-31 20:06
Message:
Logged In: YES
user_id=33168
I found at least part of the problem, perhaps everything.
The co_stacksize > 32767 and only a short value is marshalled
to the .pyc.
See Python/marshal.c line 246.
Not sure how to fix yet.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-05-31 18:32
Message:
Logged In: YES
user_id=33168
I think there are several problems here.
I was able to narrow down at least one problem to a long list.
It doesn't seem to have anything to do w/backslashes.
There seems to be an interaction with importing the module.
If the long list is in __main__, the problem does not occur.
The problem only occurs when the list is in an imported module
and only after importing it the second time (after the .pyc
is created).
If a list has 32768+ items there seems to be a memory problem.
There is an exception, then a crash. Here's the exception:
TypeError: unsupported operand type(s) for &: 'str' and 'str'
Here's a stack trace against CVS version:
#0 chunk_free (ar_ptr=0x401b1620, p=0x8156a10) at malloc.c:3180
#1 0x400fdbf4 in __libc_free (mem=0x8156a18) at malloc.c:3154
#2 0x0805c0db in list_dealloc (op=0x401c4eac) at
Objects/listobject.c:214
#3 0x080c7618 in PyDict_SetItem (op=0x401bd79c,
key=0x401c4ec0,
value=0x80e185c) at Objects/dictobject.c:373
#4 0x080c93f9 in PyDict_SetItemString (v=0x401bd79c,
key=0x80d9070 "path",
item=0x80e185c) at Objects/dictobject.c:1895
#5 0x08099ad8 in PyImport_Cleanup () at Python/import.c:286
#6 0x080a266d in Py_Finalize () at Python/pythonrun.c:228
#7 0x08053766 in Py_Main (argc=2, argv=0xbffff904) at
Modules/main.c:379
----------------------------------------------------------------------
Comment By: Jerome Alet (jalet)
Date: 2002-05-29 08:42
Message:
Logged In: YES
user_id=97214
I've done more tests, the bug only occurs if every line has
a backslash at its end :
liste = [ \
"item0", \
"item1", \
# ... many many lines
"itemN" ]
and it works if I remove the \ from each end of line.
hth.
----------------------------------------------------------------------
Comment By: Jerome Alet (jalet)
Date: 2002-05-29 06:14
Message:
Logged In: YES
user_id=97214
module too long to be attached. ask it by email at
alet@librelogiciel.com if needed.
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=561858&group_id=5470