[Python-Dev] Does Zip Importer have to be Special?

[Nick Coghlan]

On 25 July 2014 19:33, Phil Thompson <phil at riverbankcomputing.com> wrote:
> On 24/07/2014 9:42 pm, Nick Coghlan wrote:
>> As you say, though it's a niche problem compared to something like
>> packaging, which is why it got bumped down my personal priority list. I
>> haven't even got back to the first preparatory step I identified which is
>> to separate out our main functions to a separate "Programs" directory so
>> it's easier to distinguish "embeds Python" sections of the code from the
>> more typical "is part of Python" and "extends Python" code.
>
>
> Is there any way for somebody you don't trust :) to be able to help move it
> forward?

This thread prompted me to finally commit one of the smaller pieces of
preparatory refactoring, moving the 3 applications we have that embed
the CPython runtime out to a separate directory:
http://bugs.python.org/issue18093 (that seems like a trivial change,
but I found it made a surprisingly big difference when trying to keep
the various moving parts of the initialisation sequence straight in my
head)

The other preparatory refactoring would be to split the monster
pythonrun.c file in 2, by creating a separate "lifecycle.c" file. In
my original PEP 432 branch I split it into 3 (pythonrun.c,
bootstrap.c, shutdown.c) but that's actually quite an intrusive change
- you end up have to expose a lot of otherwise static variables to the
linker so the startup and shutdown code can both see them. Splitting
in two should achieve most of the same benefits (i.e. separating the
lifecycle management of the interpreter itself from the normal runtime
operation code) without having to expose so much additional
information to the linker (and hence change the names to include the
_Py prefix).

The origin of those refactorings is the fact that attempting to merge
the default branch into my PEP 432 development branch
(https://bitbucket.org/ncoghlan/cpython_sandbox/branch/pep432_modular_bootstrap)
was generally a pain due to the merge conflicts around the structural
changes. Doing the structural refactorings *first* makes it more
feasible to work on the patch and do regular merges in from default.
Since these are areas that aren't likely to change in a maintenance
release, the risk of merge conflicts when merging forward from 3.4 to
default is low even with code moved around on default. By contrast, I
regularly hit significant problems when trying to merge from default
to the feature branch.

The existing feature branch is dated enough now (more than 18 months
since the last commit!) that I wouldn't try to use it directly.
Instead, I'd recommend starting a new clone based on the GitHub or
BitBucket mirror (according to version control system and hosting
service preference), and then use the current PEP draft and my old
feature branch as a point of reference for starting another
implementation attempt. (You may also be able to find some interested
collaborators on http://bugs.python.org/issue13533, as I suspect PEP
432 is a prerequisite to resolving their issues as well)

Cheers,
Nick.

P.S. I'm also starting to think that PEP 432 may pave the way for a
locale independent startup sequence, which would let us offer a "-X
utf8" option to tell the interpreter to ignore the OS locale settings
entirely when deciding which encodings to use for various things. That
would be a possible future enhancement rather than something to pursue
in the initial implementation, though.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia