[Patches] [ python-Patches-770280 ] PyMarshal_ReadLastObjectFromFile

SourceForge.net noreply@sourceforge.net
Mon, 14 Jul 2003 09:11:03 -0700


Patches item #770280, was opened at 2003-07-12 16:42
Message generated for change (Comment added) made by paulfelix
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=770280&group_id=5470

Category: Core (C code)
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Paul Felix (paulfelix)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyMarshal_ReadLastObjectFromFile

Initial Comment:
This is a new implementation of the performance boost
in PyMarshal_ReadLastObjectFromFile where the pyc file
data is loaded into memory before parsing.

The original logic is as follows:
- Get the file size.
- If the file size is 16K or less, allocate a 16k char
buffer on the call stack and load the file data into
the buffer.
- Otherwise, if file size is 256k or less, malloc a
temp char buffer from the heap and load the file data
into the buffer.
- Otherwise, don't use a buffer.

Allocating 16k on the call stack can cause a stack
overflow when doing nested imports in a secondary
thread.  The default thread stack size on some systems
is quite small (64k on HP-UX!).  Also,
allocating/freeing memory on the heap for this purpose
may not be warranted.

This patch proposes a new implentation that allocates a
static 256k char buffer and uses a thread lock to keep
it thread-safe.  The lock/unlock function logic was
taken from import.c.

Note: performance tests show an improvment on some
platforms when the static char buffer is defined at the
file level instead of inside the function.

----------------------------------------------------------------------

>Comment By: Paul Felix (paulfelix)
Date: 2003-07-14 12:11

Message:
Logged In: YES 
user_id=819103

Adding 256k to the core footprint is probably not acceptable
either. How about 16k? Patch 2 is a compromise that uses the
original logic, but allocates the 16k buffer from static memory.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=770280&group_id=5470