[Python-Dev] PYC Magic

[Guido van Rossum]

[GMcM]
> I can't be 100% sure of the cause, but I *have*
> seen this (a bad .pyc file that had to be
> deleted before the module would import). The .pyc
> was woefully short but passed the magic
> test. I think this was 2.1, maybe 2.0. 

Hm...  Here's the code responsible for writing .pyc files:

static void
write_compiled_module(PyCodeObject *co, char *cpathname, long mtime)
{
	[...]
	PyMarshal_WriteLongToFile(pyc_magic, fp);
	/* First write a 0 for mtime */
	PyMarshal_WriteLongToFile(0L, fp);
	PyMarshal_WriteObjectToFile((PyObject *)co, fp);
	if (ferror(fp)) {
		/* Don't keep partial file */
		fclose(fp);
		(void) unlink(cpathname);
		return;
	}
	/* Now write the true mtime */
	fseek(fp, 4L, 0);
	PyMarshal_WriteLongToFile(mtime, fp);
	fflush(fp);
	fclose(fp);
	[...]
}

It's been like this for a very long time.  It always writes the magic
number, but withholds the mtime until it's done writing without
errors.  And if the mtime doesn't match, the .pyc is ignored (unless
there's no .py file...).

The only way this could write the correct mtime but not all the
marshalled data would be if ferror(fp) doesn't actually indicate an
error after a write failure due to a disk full condition.  And that's
a stdio quality of implementation issue.

I'm not sure if there's anything I could do differently to make this
more robust.  (I guess I could write the correct magic number at the
end too.)

--Guido van Rossum (home page: http://www.python.org/~guido/)