[Patches] [ python-Patches-416704 ] More robust freeze

[noreply@sourceforge.net]

Patches item #416704, was opened at 2001-04-17 07:24
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=416704&group_id=5470

Category: None
Group: None
Status: Open
Resolution: Out of Date
Priority: 5
Submitted By: Toby Dickenson (htrd)
Assigned to: Guido van Rossum (gvanrossum)
Summary: More robust freeze

Initial Comment:
This patch addresses three issues, all relating to 
robustness of frozen programs.

Specifically, this patch allows explicit and complete 
control over which modules may be loaded from source 
on the filesystem of the host system where the frozen 
program is run, and which may not.

Without this patch it is impossible to create a non-
trivial frozen program which will *never* load a 
module from source on the filesystem.



1. A patch to correct bug #404545 (frozen package 
import uses wrong files). Under this change, 
submodules of a frozen package must themselves be 
frozen modules. Previously, the import machinery may 
also try to import submodules from curiously named 
files (packagename.modulename.py) from directories in 
sys.path


2. A patch to add an extra command line option -E to 
freeze.py, which forces freeze to terminate with an 
error message if there are modules that it can not 
locate.

If this switch is not specified then the default 
behaviour is unchanged: modules which can not be found 
by freeze will not be included in the frozen program, 
and the import machinery will try to load them from 
source on sys.path when the frozen program is run.

In practice we have found that a missing module is 
probably an error (and it is a fairly frequent error 
too!). The -E switch can be used to detect this error; 
any missing modules will cause freeze.py to fail.

In the rare case of a frozen module importing a non-
frozen one (ie one which should be loaded from source 
when the program is run), the non-frozen module must 
be excluded from the freeze using the -x option.


3. A patch to add an extra command line option -X to 
freeze.py, which indicates that a specified module is 
excluded from the freeze, and also that the frozen 
program should not try to load the module from 
sys.path when it is imported. Importing the specified 
module will always trigger an ImportError.

This is useful if a module used by a frozen program 
can optionally use a submodule...
try:
    import optional_submodule
except ImportError:
    pass

It may be preferable for the frozen program's 
behaviour to not depend on whether optional_submodule 
happens to be installed on the host system, and that 
the 'import optional_submodule' should always fail 
with an ImportError. This can be achieved using the '-
X optional_submodule' command line switch to freeze.py

This is implemented by including the excluded module 
in the frozen imports table (_PyImport_FrozenModules), 
with the code pointer set to NULL.




----------------------------------------------------------------------

>Comment By: Toby Dickenson (htrd)
Date: 2001-10-03 03:34

Message:
Logged In: YES 
user_id=46460

Attached is an updated patch against the current CVS.

I have verified each aspect of this patch with 
Py_VerboseFlag set to 2, so that import.c traces out which 
files it is checking during the import process.

(Im not aware of an easy way to set Py_VerboseFlag to 2 in 
a frozen program.... Ive added a new patch #467455 to 
enhance the PYTHONVERBOSE environment variable to support 
this)

I can confirm that the current CVS (rougly 2.2a4) still 
demonstrates the problem from bug #404545. The details are 
slightly different to my original bug report; Im not sure 
if this is due to an incidental change in python since 
1.5.2, or if I messed up that bug report. Anyway, using 
PyVerbose=2 clearly shows that before this patch it is 
looking for files it shouldnt (and it uses those files if 
they exist). After this patch, it definitely only looks for 
the right files.

The other aspect of this patch, adding -E and -X to 
freeze.py, is exactly as before.


----------------------------------------------------------------------

Comment By: Toby Dickenson (htrd)
Date: 2001-09-10 07:20

Message:
Logged In: YES 
user_id=46460

OK, I should get round to this soon.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-05 08:50

Message:
Logged In: YES 
user_id=6380

I'm willing to look at this, but right now the patch doesn't
apply cleanly. From the headers in the diff file it looks
like you're patching an early beta for 2.0.

If you can rework this for current CVS or Python 2.2a2 or
2.2a3, that would be great.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-06-07 15:26

Message:
Logged In: YES 
user_id=21627

Why is this assigned to Mark? I cannot see anything 
windows-specific in it. Mark, if you are not interested in 
reviewing this patch, I recommend to unassign this from 
yourself.



----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=416704&group_id=5470