[Patches] [Patch #103252] Startup optimize: read *.pyc as string, not with getc()

[noreply@sourceforge.net]

Patch #103252 has been updated. 

Project: python
Category: core (C code)
Status: Open
Submitted by: pj99
Assigned to : tim_one
Summary: Startup optimize: read *.pyc as string, not with getc()

Follow-Ups:

Date: 2001-Jan-16 22:15
By: tim_one

Comment:
I'm afraid this patch needs major work before we could use it.  The big
thing is that stat isn't std C, and as far as I can tell we don't even
define a config symbol saying whether it's available.  Even when it is
available, on at least one oddball platform it apparently needs oddball
declarations (see the STAT/FSTAT/etc macros in posixfile.c).  And the
Macintosh code isn't in the CVS tree (Jack Jansen owns it), so no telling
how it's done there.  *Standard* C doesn't have a good way to determine the
size of a file, either.  While Python's os.path.getsize() is supported on
all platforms, that's done in platform-dependent ways in platform-dependent
files, and there's no Python C API for it.  So the first step would be to
define and implement a PyOS_GetFileSize() C API, akin to the existing
PyOS_GetLastModificationTime() (in getmtime.c).

Beyond that, there are small things that need fixing.  Like:

+ malloc's return should be cast to char*

+ but malloc shouldn't be called directly to begin with (see pymem.h)

+ ditto free()

+ the pointer test (buf > 0) doesn't make sense (buf != NULL was intended)

Once we get beyond all that <wink>, I think a three-tiered strategy would
be worth implementing:  also declare a 16K stack buffer.  If a file is
small enough, read into that; else if less than 256K (as Guido asked)
malloc a buffer as is done now; else give up.  75% of the .pyc files on my
box are under 16K (90% of IDLE's), so that would save the bulk of the
malloc/free calls due to this.

-------------------------------------------------------

Date: 2001-Jan-16 13:36
By: pj99

Comment:
No problem, Tim (on the alledged abruptness).  In any case,
it was useful - in that it helped motivate /f into proposing
a fix, me into coding it, and you to consider applying
it.  All in all a useful result.  Most thanks, Tim.

-------------------------------------------------------

Date: 2001-Jan-15 20:26
By: tim_one

Comment:
Indeed, IDLE takes more than a full second to come up on my box, *much*
longer than Notepad.  Now that we have a clear goal, I assigned this patch
to me <wink>.  (OK, it's really that I was abrupt with Paul when this came
up on c.l.py, and left him hanging there -- I owe him on this one).
-------------------------------------------------------

Date: 2001-Jan-15 20:18
By: gvanrossum

Comment:
Oh, but I do care about the load time of Tkinter. :-)  Anything to keep
IDLE ahead of Komodo in the race. :-)
-------------------------------------------------------

Date: 2001-Jan-15 20:05
By: tim_one

Comment:
I like the idea, but gimmicks like this make porting to tiny platforms
harder when pushed "too far".

People concerned about startup time seem mostly to deal with apps that e.g.
fire up Python 1000s of times from shell scripts or cgi.  They're not going
to care about monster modules (like Tkinter and grail).  So I think Paul's
original 64K is a good compromise.

-------------------------------------------------------

Date: 2001-Jan-15 19:39
By: gvanrossum

Comment:
Not a bad idea.

I'd up the limit to 256K so Tkinter.pyc (comfortably) fits.
-------------------------------------------------------

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/patch/?func=detailpatch&patch_id=103252&group_id=5470