[Python-Dev] importlib is now bootstrapped (and what that means)

[Georg Brandl]

On 16.04.2012 18:15, R. David Murray wrote:
> On Mon, 16 Apr 2012 11:21:34 -0400, Brett Cannon<brett at python.org>  wrote:
>>  On Mon, Apr 16, 2012 at 10:07, "Martin v. Löwis"<martin at v.loewis.de>  wrote:
>>
>>  >  >  We have other instances of this (e.g. the Objects/typeslots.inc file
>>  >  >  is generated and checked in), but in the case of importlib, we have
>>  >  >  to use the ./python binary for freezing to avoid bytecode
>>  >  >  incompatibilities, which obviously is a problem if ./python isn't
>>  >  >  built yet.
>>  >
>>  >  As for dependencies on byte code: we could consider using Cython instead
>>  >  of freeze (not sure whether Cython would build the bootstrap correctly;
>>  >  it may need to be fixed first). With that, we would get semi-readable
>>  >  source code, which should also play more nicely with hg diffs. On the
>>  >  down side, we would depend on Cython for evolving .

That is an interesting idea.  We already depend on external tools, e.g.
autotools, for updating checked-in files, so why not Cython.

>>  We could also just store the raw source code and use that if we are all
>>  willing to pay the performance cost of parsing and compiling the code at
>>  every startup.
>
> I don't see how depending on Cython is better than depending on having
> an existing Python.

No, it's not just an existing Python, it is (at least currently) the same
version of Python being built.  Therefore I wrote about the bootstrapping
problems when bytecode changes.

Depending on Cython is better in that it breaks the bootstrapping cycle,
but on the other hand the C code may need to be regenerated when the C API
changes in an incompatible way.

>  If the only benefit is semi-readable code, surely
> we do have source code for the pre-frozen module, and it is just a matter
> of convincing hg that the bytecode is binary, not text?

The benefit is also (presumably) better performance.

Georg