[Python-Dev] Python "2migr8"

[Terry Reedy]

On 4/14/2014 1:19 PM, Guido van Rossum wrote:
> Some quick thoughts:
>
> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>
> - Are you sure this isn't better directed to python-ideas first? Most
> ideas have to prove their worth in that list before python-dev will give
> them the light of day.
>
> - When it comes to purely syntactic issues (e.g. "except x, y:") a
> linter or some other separate tool can handle this well (heck, you can
> build it into an import hook probably :-).
>
> - When it's about backported modules, a sumo distribution is probably
> the way to go; when it's about renamed stdlib modules, six (perhaps an
> extended version) should cover you.
>
> - Regarding warning about the changed dict API, I wonder how you plan to
> implement that if you allow passing dict object back and forth between
> code that has opted in to single-source and code that hasn't yet. Please
> think through some specific examples before responding.
>
> - But the biggest issue is of course bytes vs. text. You would have to
> first do a careful analysis of the *whole* problem before you can even
> think about proposing a solution. Too many people think their is an easy
> solution for this; but almost everybody is focused on only part of the
> problem (the part that causes them immediate pain) without realizing
> that other people's pain may be different.
>
> - As far as your assertion that it would have to come from python-dev
> because nobody outside is going to tackle it, I think it's the opposite:
> the core developers would prefer not to have to deal with this, while
> some folks outside the inner circles will not be discouraged by our
> opinions (e.g. Stackless is working on "Stackless 2.8").
>
> - Regarding open source projects having a reputation of "not taking
> contributions", I would guess that this is usually about those
> "contributions" violating the most basic rules of the project (and I
> don't mean the coding style). I do want to discourage discussions with

Did you mean 'don't want to discourage'?

> users like the company you referred to, but I think it would be much
> more useful if they laid out their problems for us instead of expecting
> they can buy acceptance for a "solution" they develop in-house. We could
> then hopefully have a productive dialog over many months where we
> iterate over possible approaches that could be acceptable both to Python
> and to the customer. But it will take a significant investment of time
> on both sides -- there is no shortcut. And it's not a particularly
> interesting problem (for most people) to work on -- things like
> designing a notation for optional type declarations are always much more
> fun. :-)
>
>
>
>
> On Mon, Apr 14, 2014 at 11:32 AM, Steve Dower <Steve.Dower at microsoft.com
> <mailto:Steve.Dower at microsoft.com>> wrote:
>
>
>     Just in case there's anyone out there who isn't yet sick of
>     discussing how to proceed with Python 2.7, I have some more inputs
>     to contribute.
>
>     To put it up front, I'm totally against "CPython 2.8" ever becoming
>     a real thing. Anything that comes out should be seen as a migration
>     path, not an upgrade path. I'll also admit I'm not heavily invested
>     in working on it myself, but I had a number of conversations during
>     PyCon (as well as being at the language summit) that puts me in a
>     position to share the ideas and concerns that have been raised.
>
>     The main trigger was a conversation I had with two employees of a
>     very large bank that has about 3000 Python users (not developers -
>     mostly financial analysts) and 16 million lines of code running on
>     2.7. They are keen to migrate to 3.x but cannot afford to stop work
>     entirely while their code is updated. (There was much more to the
>     conversation than I'm relating here - I'm keeping to the directly
>     relevant bits.)
>
>     In describing the approach they'd like to take, they made me realise
>     that there is definitely a place for a Python that is different but
>     mostly compatible with 2.7, in a way that 2.7.x could not be. For
>     the sake of having a name, I'll refer to this as "Python 2migr8"
>     (pronounced "to migrate" :) ).
>
>     The two important components of Python 2migr8 would be the ability
>     to disable 2.7-only features, and to do so on a module-by-module basis.
>
>     My best idea so far would be to have a magic comment (to ensure 2.7
>     compatibility better than a "from __future__ ...") near the top of
>     the file that marks that file as "must straddle 2.7 and 3.3". Adding
>     this comment causes (for example) the parser to treat "except x, y"
>     as a syntax error in this file, forces "from __future__ import ...",
>     hides "dict.iterkeys", undefines "basestring", etc., but only for
>     this file. (I haven't thought through all the possibilities or
>     implications - Eric Snow said he was going to sprint on this
>     today/tomorrow, so he'll soon have a better idea just what can be done.)
>
>     In effect, 2migr8 would be the version that *only* supports
>     "single-source" files. This allows large code bases to progressively
>     migrate modules from 2.x to single-source while continuing to run
>     against Python 2.7. As files are updated, they are marked as being
>     single-source. When all files have this marker, it should be
>     possible to flip the switch and run with Python 3.3 or later.
>
>     You could also think of this as enabling "-3 --warnings-as-errors"
>     for individual modules, though since the user has already opted in
>     to 2migr8, it isn't unreasonable to make more significant changes,
>     like having dict.keys returning a list that warns if it is mutated.
>     This sort of warning can only really be done by changing the
>     interpreter - static analysis just can't catch everything - and only
>     when users accept a potential performance hit and low probability of
>     breakage when they move to 2migr8 (followed by a not-quite-as-low
>     probability of breaking when they eventually move from 2migr8 to
>     3.x, but it's still better than guaranteed breakage).
>
>     As a fork, it would also be possible to bundle the modules that have
>     been backported, and possibly also to disallow importing deprecated
>     stdlib modules when 2.7 functionality is disabled. As I said, I
>     haven't thought through all the possibilities, but the general idea
>     is to take 2.7 and *remove* features so it becomes easier to migrate.
>
>     Where does python-dev come in? Obviously this is where a fork like
>     this would have to start - there has been such strong and public
>     opposition to any significant changes like this that you'd be hard
>     pressed to find someone willing to start and promote it from
>     outside. There is also a good opportunity to make a start and
>     directly invite those using it to contribute the rules or warnings
>     that they need - the 3000 Python "users" I mentioned earlier are
>     backed by a team of true developers who are more than capable of
>     contributing, and this would be a great opportunity to directly
>     invite them. However unfair and incorrect it may be, there is a
>     perception in some businesses that open-source projects do not want
>     contributions from them. I invited more than one business to have
>     someone join python-dev and get involved during PyCon, and I heard
>     that others did the same - it may not be at the level of employing a
>     core developer full time, but it's the starting point that some
>     companies will ne
>       ed to be able to become comfortable with employing a core dev.
>
>     I'm not pretending to have a full plan on how this will work. I was
>     privileged to have some private conversations during PyCon that are
>     directly relevant, so I'm bringing it here to promote the
>     discussion. Thanks to everyone I had a chance to chat to, and to
>     everyone generally for a great PyCon.
>
>     Cheers,
>     Steve
>
>
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>     https://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe:
>     https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>
>


-- 
Terry Jan Reedy