After writing some code, I see that the library file needs to be a little different. We need to add the file size so all fseek()'s can be done relative to the end. That enables the file to be appended to the executable. The offsets written to the file are from ftell() and are relative to the beginning. The length is subtracted on access.
And I introduced a main dictionary TOC and a different dictionary for the *.pyc files pyc_dict. Although that may seem a little fancy, it is actually easier. And it is much more extensible. Data may be freely written to the main dict TOC without damaging the code for reading *.pyc using pyc_dict. And another section (I'm thinking of Gordon's installer) can be added as yet another dict. In another variation, core Python files are in pyc_dict, but encoded *.pyc files are in a different dict, and imputils is used to import either.
The Python library file format now is:
Data... consisting of *.pyc files including their 8-byte header; Possibly other data... TOC, a marshal'd dictionary table of contents; a twelve byte ascii decimal seek offset to the start of TOC, a NULL byte; a twelve byte ascii decimal total file size, a NULL byte; a 16-byte magic number to identify the file as a valid Python library file.
The TOC (table of contents) is a dictionary containing the following keys: "TIME" The time the library was created, time.time(). "ASCTIME" The ascii time the library was created. "VERSION" The version of the library file format, "1". "PYC*VERSION" The ascii version number of the PYC section, "19990720". "PYC" A dictionary with keys "name", and values a three-tuple "tup". The "name" is the module name. This is a plain name such as "os" or a dotted name such as "foo.bar". The "tup" is the seek offset to the data, which is the bytes of the module .pyc file including the 8-byte header. The "tup" is the full path of the .pyc file. The "tup" is an integer used as flag bits. Flag bit 0x01 is one if the module name refers to a package, in which case the file is package.__init__.pyc.
This format is designed to be extensible. Just add atoms of data to TOC. To add a another whole section, add another name like "PYC".
I have almost finished makepyl.py, a Python program which creates a PYL file or lists its contents. And C-code changes to import.c to be able to import from a PYL file. However, I am a little short of time because I am leaving for two weeks vacation on this Saturday.