[Python-Dev] Python "2migr8"

[Guido van Rossum]

Some quick thoughts:

- I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)

- Are you sure this isn't better directed to python-ideas first? Most ideas
have to prove their worth in that list before python-dev will give them the
light of day.

- When it comes to purely syntactic issues (e.g. "except x, y:") a linter
or some other separate tool can handle this well (heck, you can build it
into an import hook probably :-).

- When it's about backported modules, a sumo distribution is probably the
way to go; when it's about renamed stdlib modules, six (perhaps an extended
version) should cover you.

- Regarding warning about the changed dict API, I wonder how you plan to
implement that if you allow passing dict object back and forth between code
that has opted in to single-source and code that hasn't yet. Please think
through some specific examples before responding.

- But the biggest issue is of course bytes vs. text. You would have to
first do a careful analysis of the *whole* problem before you can even
think about proposing a solution. Too many people think their is an easy
solution for this; but almost everybody is focused on only part of the
problem (the part that causes them immediate pain) without realizing that
other people's pain may be different.

- As far as your assertion that it would have to come from python-dev
because nobody outside is going to tackle it, I think it's the opposite:
the core developers would prefer not to have to deal with this, while some
folks outside the inner circles will not be discouraged by our opinions
(e.g. Stackless is working on "Stackless 2.8").

- Regarding open source projects having a reputation of "not taking
contributions", I would guess that this is usually about those
"contributions" violating the most basic rules of the project (and I don't
mean the coding style). I do want to discourage discussions with users like
the company you referred to, but I think it would be much more useful if
they laid out their problems for us instead of expecting they can buy
acceptance for a "solution" they develop in-house. We could then hopefully
have a productive dialog over many months where we iterate over possible
approaches that could be acceptable both to Python and to the customer. But
it will take a significant investment of time on both sides -- there is no
shortcut. And it's not a particularly interesting problem (for most people)
to work on -- things like designing a notation for optional type
declarations are always much more fun. :-)




On Mon, Apr 14, 2014 at 11:32 AM, Steve Dower <Steve.Dower at microsoft.com>wrote:

>
> Just in case there's anyone out there who isn't yet sick of discussing how
> to proceed with Python 2.7, I have some more inputs to contribute.
>
> To put it up front, I'm totally against "CPython 2.8" ever becoming a real
> thing. Anything that comes out should be seen as a migration path, not an
> upgrade path. I'll also admit I'm not heavily invested in working on it
> myself, but I had a number of conversations during PyCon (as well as being
> at the language summit) that puts me in a position to share the ideas and
> concerns that have been raised.
>
> The main trigger was a conversation I had with two employees of a very
> large bank that has about 3000 Python users (not developers - mostly
> financial analysts) and 16 million lines of code running on 2.7. They are
> keen to migrate to 3.x but cannot afford to stop work entirely while their
> code is updated. (There was much more to the conversation than I'm relating
> here - I'm keeping to the directly relevant bits.)
>
> In describing the approach they'd like to take, they made me realise that
> there is definitely a place for a Python that is different but mostly
> compatible with 2.7, in a way that 2.7.x could not be. For the sake of
> having a name, I'll refer to this as "Python 2migr8" (pronounced "to
> migrate" :) ).
>
> The two important components of Python 2migr8 would be the ability to
> disable 2.7-only features, and to do so on a module-by-module basis.
>
> My best idea so far would be to have a magic comment (to ensure 2.7
> compatibility better than a "from __future__ ...") near the top of the file
> that marks that file as "must straddle 2.7 and 3.3". Adding this comment
> causes (for example) the parser to treat "except x, y" as a syntax error in
> this file, forces "from __future__ import ...", hides "dict.iterkeys",
> undefines "basestring", etc., but only for this file. (I haven't thought
> through all the possibilities or implications - Eric Snow said he was going
> to sprint on this today/tomorrow, so he'll soon have a better idea just
> what can be done.)
>
> In effect, 2migr8 would be the version that *only* supports
> "single-source" files. This allows large code bases to progressively
> migrate modules from 2.x to single-source while continuing to run against
> Python 2.7. As files are updated, they are marked as being single-source.
> When all files have this marker, it should be possible to flip the switch
> and run with Python 3.3 or later.
>
> You could also think of this as enabling "-3 --warnings-as-errors" for
> individual modules, though since the user has already opted in to 2migr8,
> it isn't unreasonable to make more significant changes, like having
> dict.keys returning a list that warns if it is mutated. This sort of
> warning can only really be done by changing the interpreter - static
> analysis just can't catch everything - and only when users accept a
> potential performance hit and low probability of breakage when they move to
> 2migr8 (followed by a not-quite-as-low probability of breaking when they
> eventually move from 2migr8 to 3.x, but it's still better than guaranteed
> breakage).
>
> As a fork, it would also be possible to bundle the modules that have been
> backported, and possibly also to disallow importing deprecated stdlib
> modules when 2.7 functionality is disabled. As I said, I haven't thought
> through all the possibilities, but the general idea is to take 2.7 and
> *remove* features so it becomes easier to migrate.
>
> Where does python-dev come in? Obviously this is where a fork like this
> would have to start - there has been such strong and public opposition to
> any significant changes like this that you'd be hard pressed to find
> someone willing to start and promote it from outside. There is also a good
> opportunity to make a start and directly invite those using it to
> contribute the rules or warnings that they need - the 3000 Python "users" I
> mentioned earlier are backed by a team of true developers who are more than
> capable of contributing, and this would be a great opportunity to directly
> invite them. However unfair and incorrect it may be, there is a perception
> in some businesses that open-source projects do not want contributions from
> them. I invited more than one business to have someone join python-dev and
> get involved during PyCon, and I heard that others did the same - it may
> not be at the level of employing a core developer full time, but it's the
> starting point that some companies will ne
>  ed to be able to become comfortable with employing a core dev.
>
> I'm not pretending to have a full plan on how this will work. I was
> privileged to have some private conversations during PyCon that are
> directly relevant, so I'm bringing it here to promote the discussion.
> Thanks to everyone I had a chance to chat to, and to everyone generally for
> a great PyCon.
>
> Cheers,
> Steve
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/745de0df/attachment-0001.html>