[Python-Dev] Packaging in Python 2 anyone ?

[Éric Araujo]

Hi Tarek,

> Doing an automated conversion turned out to be a nightmare, and I was
> about to go ahead and maintain a fork of the packaging package, with
> the few modules that are needed (sysconfig, etc) within a standalone
> release.

Can you give us more info?  Do you have a repo somewhere, or notes?

A related question: what is the minimum 2.x version that we should
support?  2.6 would be a dream, thanks to bytes literal and all that,
but I’m sure it’s not realistic; 2.5 would be nice for the with
statement and hashlib, otherwise 2.4 is okay.

When I talked with Łukasz in private email about backports and 3to2, we
agreed that there were some serious bugs in 3to2 and we wanted to work
on patches.  I also wanted to make the command-line driver more
flexible, so that it would be easy to run a command to apply only
3.3→3.2 fixes, then another for 3.2→2.7, etc.

Maybe your problems were caused by the state of the packaging codebase.
 The conversion to 3.x was a little messy: in some cases there were
parallel code branches for 2.x and 3.x, on other cases 2to3 was run, and
in many cases the conversion had to be cleaned up (esp. bytes/str
madness).  Even now that the code runs and the tests pass, there may
still be things in need of a cleanup in the codebase, and maybe they
trip up 3to2.

> I am looking for someone that has some free time and that is willing
> to lead this work.

Well, free time is scarce with all these distutils bugs on my plate, but
I am definitely interested in heading the backport, as I stated earlier.
 I think the key point is to avoid making the same work over and over
again, and I see a few ways of managing that.

The first way is to start with a 2.x-converted codebase (thanks Vinay!)
and manually port all cpython/packaging changesets to distutils2, like I
used to do.  This is just as annoying as backporting to 2.7, and just as
simple.

The second way is to work on a conversion tool instead of working on
changesets.  The idea is to make a robust tool based on 3to2 that copies
code and converts it.  This would not be the easiest way, as shown by
your experience, but surely the less cumbersome in the long term.

The third way is to use a new Mercurial repo converted from the cpython
repo, so that we can run “hg convert” again to pull new changesets.
Convert, test and commit.  The advantage is that it’s not required to
port each changeset: the convert-merge dance can be done once a month,
or just for new releases.

The fourth way is hybrid: start from a 2.x-converted codebase, and each
month, make a diff for cpython/Lib/packaging and apply to distutils2.  I
fear that such diffs would be painful to apply, and consist mostly of
rejects.  With idea #3, we get to use a merge tool, which is much better.

After writing out these ideas, I think the first one is certainly the
simplest thing that could work with minimum pain.


Le 18/08/2011 00:30, Vinay Sajip a écrit :
> stdlib dependency code is either moved to util.py or test/support.py as
> appropriate.
We need sysconfig, shutil, tarfile, hashlib... Surely that’s a lot to
put in util.py.

> I'm not sure if I'll have much more time to spend on this, but it's there in
> case someone else can look at the remaining test failures, plus Steps 4 and 5;
> hopefully I've broken the back of it :-)
I join my thanks to Tarek’s, and volunteer to follow on :)

Regards