[Patches] [ python-Patches-1353872 ] a faster Modulefinder

[SourceForge.net]

Patches item #1353872, was opened at 2005-11-11 12:51
Message generated for change (Comment added) made by theller
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1353872&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Performance
Group: Python 2.5
>Status: Closed
>Resolution: Out of Date
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Nobody/Anonymous (nobody)
Summary: a faster Modulefinder

Initial Comment:
py2exe uses Python's modulefinder to find the modules
and packages that belong to one or more scripts.

For not too small projects, the runtime of modulefinder
is quite long.  On my system, the time to find all 533
modules my project needs is around 48 seconds.

So, I profiled the Python 2.4 modulefinder, and patched
it for a speedup of a factor of ~2.5 - the time
required to find the modules drops to around 19 seconds.


----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2006-10-17 21:56

Message:
Logged In: YES 
user_id=11105

This patch is out of date.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2005-11-11 13:02

Message:
Logged In: YES 
user_id=11105

Here is a description of the changes in the patch:

Modulefinder's scan_code method did call ord() on each
character of the co.co_code string, that took the most time,
and it built the argument (again with ord() calls) of each
bytecode that had one, even if it was never used.

The patch changes the code to
- work on the characters of the co.co_code string, avoiding
the calls to ord() altogether
- create the bytecodes argument only when needed,
- create the bytecode with struct.pack which is faster.

I did not stop there, so other changes were that the objects
that scan_code needs most are passed as default arguments to
the functions instead of looking them up in the global
namespace.

This patch will probably be in the next py2exe release, so
it will undergo some testing.

I would appreciate comments on the patch.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1353872&group_id=5470