Using emacs' unexec to speed Python startup (was Re: [Python-Dev] Startup time)

[Daniel Berlin]

On Monday, May 19, 2003, at 08:06  AM, Jeff Epler wrote:

> O Sun, May 18, 2003 at 11:28:08PM -0400, Barry Warsaw wrote:
>> Since it looks like you implemented the meat of it as a module, I
>> wonder if it couldn't be cleaned up (with the interrupt reset either
>> pulled in the extension or exposed to Python) and added to Python 2.3?
>
> First off, I sure doubt that this feature could be truly made
> "non-experimental" before 2.3 is released.  There was one "strange 
> bug" so
> far (the signal thing), though that was quickly solved (with another
> change to the core Python source code).
>
> Secondly, forcing all allocations to come from the heap instead of 
> mmap'd
> space may hurt performance.
>
> Thirdly, the files implementing unexec itself, which come from fsf 
> emacs,
> are covered by the GNU GPL, which I think makes them unsuitable for
> compiling into Python. (There's something called "dynodump" in Emacs 
> that
> appears to apply to ELF binaries which bears this license:
>  * This source code is a product of Sun Microsystems, Inc. and is 
> provided
>  * for unrestricted use provided that this legend is included on all 
> tape
>  * media and as a part of the software program in whole or part.  Users
>  * may copy or modify this source code without charge, but are not 
> authorized
>  * to license or distribute it to anyone else except as part of a 
> product or
>  * program developed by the user.
> I wish I understood what "except as part of a product or program 
> developed
> by the user" meant--does that mean that Alice can't download Python
> then give it to Bob if it includes dynodump?  After all, Alice didn't
> develop it, she simply downloaded it.  The other dumpers in xemacs
> seem to be GPL, and I think that the "portable undump" mentioned by
> another poster is a placeholder for a project that isn't written yet:
> http://www.xemacs.org/Architecting-XEmacs/unexec.html)
It was written and is on by default  since 21.2 came out, the website 
is out of date.

See http://www.xemacs.org/Releases/Public-21.2/projects/pdump.html

It's probably too xemacs specific, however.

The file you want is dumper.c.

>
> Fourthly, we'd have to duplicate whatever machinery chooses the correct
> unexec implementation for the platform you're running on---there are 
> lots to
> choose from:

Only if you do undumping the same way. The portable dumper way was to 
not make an executable, instead putting it in a seperate file, and 
storing it in a neutral format that was architected to make loading 
fast.
It's still faster than loading byte-compiled files, since nothing needs 
to be executed as we are just recreating the in-memory representation.

> 	unexaix.c     unexconvex.c  unexenix.c     unexnext.c    unexw32.c
> 	unexalpha.c   unexec.c      unexhp9k800.c  unexsni.c
> 	unexapollo.c  unexelf.c     unexmips.c     unexsunos4.c
> (Of course, it's well known that only elf and win32 matter in these 
> modern
> times)
>
> I'd be excited to see "my work" in Python, though the fact of the 
> matter
> is that I just tried this out because I was bored on a Sunday 
> afternoon.
>
> Jeff
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev