The DirtyExeT proposal

[Ionel Simionescu]

DirtyExeT is a proposal I hereby submit to your attention.


Objective:
----------
Package a Python application that (comprises | depends upon) several Python
source files into a single EXE file. This process should be automatic,
customizable and largely independent of the concrete Python application at
hand.


Context:
--------
One has a working Python application.

He wants to demonstrate it to a potential client, or in some other context.
In order to do that he should make sure that Python is installed on the
client's machine, as well as many of the libraries he use, and he should
also check the environment settings.

Because it is rarely the case that everything is setup on the target
machine, one must often undertake the effort of packing up his working
environment, dress it up with smart install scripts and hope it will work
when unfolded elsewhere.

An automated and elegant solution is needed. Ideally, a certain command
should be available in Python that transforms an application into a
selfcontained, standalone exe file. Size is not important in many
situations, but functionality and time to market/client.


Solution:
---------
A Python application, APP, is basically defined by its main module, app.py.

The task of identifying the dependencies of APP, can be automated by
recursively looking at the import statements.

APP is thus constituted by a set of named files; more specifically, we shall
consider the set of p-code files.

-------------------------------------------------------------------
The basic idea of the DirtyExe method is to (attach|append) these files to
the body of a special Python.exe interpreter, and then fix the result to
remain a valid exe.
-------------------------------------------------------------------

This approach avoids a true compilation step, is fast, can be quite
effective and it has been already proved in practice [1].

I will hereby suggest what modifications to the Python interpreter are
necessary and how it might work.

On Win32, an EXE may contain resources and there is a mechanism to use them.
In order to allow the interpreter to accept and use p-code patches one needs
to augment its implementation for the <import> routine.

The modified import will look for the files not just on the disk but firstly
in its own table of resources.

Moreover, the interpreter should always start by executing a p-code resource
string called, for example, "PYTHONSTARTUP".

By now, the EXE is already dirty as it contains not only host specific
object code but also Python specific object code (the content of the
PYTHONSTARTUP string).

Such an EXE may be changed to contain arbitrary Python applications in its
resources table. It is left to the patching phase to actually manipulate
these resources and thus create custom applications.

---

On other platforms than Win32, the p-code files pending to an apllication
could, in principle, be appended to PYTHON.EXE as new global variables.
Being able to actually use this code requires that the EXE uses a global
PYTHONSTARTUP string containing p-code, and this string can be replaced by
the code of the new, custom application.

When there is no distinction between object code and some form of resources,
it might be a bit more difficult to handle the symbol table of the exe in
order to add new global variables. Yet, it should be doable because it's
just appending new records and shifting around some addresses for some of
the existing ones.

(Python.exe is thus turned into a willing host for python applications. :-)

---

Conclusion:
-----------
A method is suggested by which an EXE file that contains a Python
interpreter can be augmented with pseudo code files representing a custom
application.

The net effect is that developers will be able to get standalone
executables.



Comments:
---------
1.
I hope the wizards of the Python community will find an easy way to
implement this idea and give us the tool to create bulky standalone
applications with a click of the mouse.
2.
This method really fits Python: just think about python(.exe) eating up your
p-code, :-)
3.
For the sophisticated, several kinds of python.exe builds may be designed,
as some libraries are included or not.
4.
For the casual user, and, attention, for the students preparing their term
projects, such a tool would be quite a compelling reason to use Python,
because it hides its actual source code. For example, they might claim to
have coded it in C++... They should not, of course, do such things. :-)
5.
For the professionals, they would have an easier time with clients locked in
their respect for the "compiled" languages. The clients will not be able to
tell a Pythonish composit from a "genuine" exe.
6.
There is some other work to be done that I did not even mention: checking
for the distinction btw. console and windowed applications, accompanying the
resulting exe with binary files that might be required, etc.

...


ionel



----------------------------------------------------------------------------
-----

[1] For a supposed implementation of the DirtyExe idea, see the enviroment
"Dialect Standard Edition", at http://www.aristar.com/DeveloperTools.html.

The DirtyExe method is nowhere explicitly pointed to, but if you made an exe
with their tool you'll see that there is no other way to do it.

I was annoyed of that little bastard, Dialect, not mentioning its parents,
i.e. Python, but I was impressed of its ease in creating EXEs. This posting
is the resolution of this conflicting thoughts.