Hey Folks, I just got this message from a developer inside of ILM today. Before I start digging, can anyone tell me if this has been dealt with in any version of python more recent than 1.5.2? (and where I might look for this info myself- I'm very unfamiliar with sourceforge and its bug tracker)... thanks! Garrick Meeker writes: | ******** is starting to produce very long project files (over 32K lines) | which it currently can't read. (Because curves are imbedded in the | files, it's not difficult to hit that limit.) | | Here's what I've found so far (in both python 1.4 and 1.5): | | ******** read the file by calling PyRun_SimpleFile. This byte compiles | the entire file into memory and then runs that (which is probably not | ideal). | | The file is tokenized into nodes that have a 'short' for the line | number, so this wraps around to negative numbers. | | The line number is entered into the byte code with a call to | 'com_addint', which calls: | | com_addbyte(c, x & 0xff); | com_addbyte(c, x >> 8); /* XXX x should be positive */ | | (notice that 'x >> 8' for a negative number will be a negative number). | | com_addbyte has a sanity check: | | if (byte < 0 || byte > 255) { | com_error(c, PyExc_SystemError, "com_addbyte: byte out of range"); | | This is what prevents ******** from loading long files. The options I | see are: | | Fix our build of python to fix this problem. We can't change the line | number to 'int' because we'd be changing the byte code, but we could | stop the number from wrapping around. | | Manually read each line from the file and feed it to the interpreter as | if we were interactive. We'd lose the line number information if | there's an error in the file (but we can't save numbers over 32K | anyway). I think it would also be friendlier on memory because it | wouldn't byte compile the entire file first. There is a call to pass | strings in this manner, but I don't know if it will actually work this | way. | | Tommy, have you heard of this before? I know I still missing part of | the story because I can create a simple file with 40K lines of: | | a = 0 | | and python accepts it without error.
On Thu, Sep 06, 2001 at 12:28:14PM -0700, The Intermission Elf wrote:
I just got this message from a developer inside of ILM today. Before I start digging, can anyone tell me if this has been dealt with in any version of python more recent than 1.5.2? (and where I might look for this info myself- I'm very unfamiliar with sourceforge and its bug tracker)...
Yes, it's fixed. As for how to find it, well, that can prove a bit challenging, as there are already hundreds of closed bugreports, and only a few of them deal with this ;P The easiest way is to use CVS. You know the code is in compile.c, so a quick browse of the CVS log might show it: ---------------------------- revision 2.133 date: 2000/08/24 00:32:09; author: fdrake; state: Exp; lines: +32 -2 Charles G. Waldman <cgw@fnal.gov>: Add the EXTENDED_ARG opcode to the virtual machine, allowing 32-bit arguments to opcodes instead of being forced to stick to the 16-bit limit. This is especially useful for machine-generated code, which can be too long for the SET_LINENO parameter to fit into 16 bits. This closes the implementation portion of SourceForge patch #100893. ---------------------------- (Checking the version numbers you'll see that it was fixed between Python 1.6-final and Python 2.0b1. The fix should be pretty backportable, if you want to insert it into 1.5.2 or older.... but that would kind of defeat the point of releases.) Or you can browse the bug or patch database online, first the open bugs/patches, and then the closed bugs/patches... but though in this case there's an entry in both the (closed) lists, it isn't true for all bugs, so checking the CVS logs is more fool-proof. Frankly, asking someone here might be the easiest solution :-) Of course, the best test is to copy one of those large files over to a machine with a recent Python, and checking to see if it works. Maybe ******** generates files so big, or big in a different way, that it still doesn't work.
Garrick Meeker writes: | ******** is starting to produce very long project files (over 32K lines) | which it currently can't read. (Because curves are imbedded in the | files, it's not difficult to hit that limit.)
-- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
Hey Folks,
I just got this message from a developer inside of ILM today. Before I start digging, can anyone tell me if this has been dealt with in any version of python more recent than 1.5.2? (and where I might look for this info myself- I'm very unfamiliar with sourceforge and its bug tracker)...
thanks!
Line numbers > 32K? Yes, we do that since 2.0. Another reason to upgrade. --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (3)
-
Guido van Rossum -
The Intermission Elf -
Thomas Wouters