[Python-Dev] Packaging in Python 2 anyone ?

[Tarek Ziadé]

On Thu, Aug 18, 2011 at 8:27 PM, Éric Araujo <merwok at netwok.org> wrote:
> Hi Tarek,
>
>> Doing an automated conversion turned out to be a nightmare, and I was
>> about to go ahead and maintain a fork of the packaging package, with
>> the few modules that are needed (sysconfig, etc) within a standalone
>> release.
>
> Can you give us more info?  Do you have a repo somewhere, or notes?

I tried using relative imports, but that made the whole thing
complicated and not working under older 2.x
then there are a lot of spots where the word 'packaging' is used for
other things than modules.

then there are spots when we needed to change the bytes/str behavior
depending on the py version, making everything complex to maintain

I guess it's the addition of the three that made it too complex :
transparent renaming + 3to2 + 3.xto3.x

>
> A related question: what is the minimum 2.x version that we should
> support?  2.6 would be a dream, thanks to bytes literal and all that,
> but I’m sure it’s not realistic; 2.5 would be nice for the with
> statement and hashlib, otherwise 2.4 is okay.

2.5 sounds good. I am sold on dropping 2.4 frankly. Maybe we can drop
2.5 in a few months ;)

>
> When I talked with Łukasz in private email about backports and 3to2, we
> agreed that there were some serious bugs in 3to2 and we wanted to work
> on patches.  I also wanted to make the command-line driver more
> flexible, so that it would be easy to run a command to apply only
> 3.3→3.2 fixes, then another for 3.2→2.7, etc.
>
> Maybe your problems were caused by the state of the packaging codebase.
>  The conversion to 3.x was a little messy: in some cases there were
> parallel code branches for 2.x and 3.x, on other cases 2to3 was run, and
> in many cases the conversion had to be cleaned up (esp. bytes/str
> madness).  Even now that the code runs and the tests pass, there may
> still be things in need of a cleanup in the codebase, and maybe they
> trip up 3to2.

I think that's not worth the effort frankly. keeping a clean fully py3
code without worrying about making it 3to2 friendly, make all
contributors life easier ihmo. The tradeoff is that we will have to
backport to distutils2 changes. That's what was done for a while
between the Python trunk and the Py3k branch, so I guess it's doable
-- if all packaging contributors agree to do this backport work.


>
>> I am looking for someone that has some free time and that is willing
>> to lead this work.
>
> Well, free time is scarce with all these distutils bugs on my plate, but
> I am definitely interested in heading the backport, as I stated earlier.
>  I think the key point is to avoid making the same work over and over
> again, and I see a few ways of managing that.
>
> The first way is to start with a 2.x-converted codebase (thanks Vinay!)
> and manually port all cpython/packaging changesets to distutils2, like I
> used to do.  This is just as annoying as backporting to 2.7, and just as
> simple.
>
> The second way is to work on a conversion tool instead of working on
> changesets.  The idea is to make a robust tool based on 3to2 that copies
> code and converts it.  This would not be the easiest way, as shown by
> your experience, but surely the less cumbersome in the long term.
>
> The third way is to use a new Mercurial repo converted from the cpython
> repo, so that we can run “hg convert” again to pull new changesets.
> Convert, test and commit.  The advantage is that it’s not required to
> port each changeset: the convert-merge dance can be done once a month,
> or just for new releases.
>
> The fourth way is hybrid: start from a 2.x-converted codebase, and each
> month, make a diff for cpython/Lib/packaging and apply to distutils2.  I
> fear that such diffs would be painful to apply, and consist mostly of
> rejects.  With idea #3, we get to use a merge tool, which is much better.
>
> After writing out these ideas, I think the first one is certainly the
> simplest thing that could work with minimum pain.

I think so too.  The automatic conversion sounded like a great thing,
but the nature of the project makes it too hard,

Cheers




-- 
Tarek Ziadé | http://ziade.org