[Python-Dev] Python "2migr8"
Guido van Rossum
guido at python.org
Mon Apr 14 22:26:45 CEST 2014
On Apr 14, 2014 2:42 PM, "Terry Reedy" <tjreedy at udel.edu> wrote:
>
> On 4/14/2014 1:19 PM, Guido van Rossum wrote:
>>
>> Some quick thoughts:
>>
>> - I'd prefer a name that plays on 2 and 3, not 2 and 8. :-)
>>
>> - Are you sure this isn't better directed to python-ideas first? Most
>> ideas have to prove their worth in that list before python-dev will give
>> them the light of day.
>>
>> - When it comes to purely syntactic issues (e.g. "except x, y:") a
>> linter or some other separate tool can handle this well (heck, you can
>> build it into an import hook probably :-).
>>
>> - When it's about backported modules, a sumo distribution is probably
>> the way to go; when it's about renamed stdlib modules, six (perhaps an
>> extended version) should cover you.
>>
>> - Regarding warning about the changed dict API, I wonder how you plan to
>> implement that if you allow passing dict object back and forth between
>> code that has opted in to single-source and code that hasn't yet. Please
>> think through some specific examples before responding.
>>
>> - But the biggest issue is of course bytes vs. text. You would have to
>> first do a careful analysis of the *whole* problem before you can even
>> think about proposing a solution. Too many people think their is an easy
>> solution for this; but almost everybody is focused on only part of the
>> problem (the part that causes them immediate pain) without realizing
>> that other people's pain may be different.
>>
>> - As far as your assertion that it would have to come from python-dev
>> because nobody outside is going to tackle it, I think it's the opposite:
>> the core developers would prefer not to have to deal with this, while
>> some folks outside the inner circles will not be discouraged by our
>> opinions (e.g. Stackless is working on "Stackless 2.8").
>>
>> - Regarding open source projects having a reputation of "not taking
>> contributions", I would guess that this is usually about those
>> "contributions" violating the most basic rules of the project (and I
>> don't mean the coding style). I do want to discourage discussions with
>
>
> Did you mean 'don't want to discourage'?
Yes. Sorry. Pylon brain fry...
>> users like the company you referred to, but I think it would be much
>> more useful if they laid out their problems for us instead of expecting
>> they can buy acceptance for a "solution" they develop in-house. We could
>> then hopefully have a productive dialog over many months where we
>> iterate over possible approaches that could be acceptable both to Python
>> and to the customer. But it will take a significant investment of time
>> on both sides -- there is no shortcut. And it's not a particularly
>> interesting problem (for most people) to work on -- things like
>> designing a notation for optional type declarations are always much more
>> fun. :-)
>>
>>
>>
>>
>> On Mon, Apr 14, 2014 at 11:32 AM, Steve Dower <Steve.Dower at microsoft.com
>> <mailto:Steve.Dower at microsoft.com>> wrote:
>>
>>
>> Just in case there's anyone out there who isn't yet sick of
>> discussing how to proceed with Python 2.7, I have some more inputs
>> to contribute.
>>
>> To put it up front, I'm totally against "CPython 2.8" ever becoming
>> a real thing. Anything that comes out should be seen as a migration
>> path, not an upgrade path. I'll also admit I'm not heavily invested
>> in working on it myself, but I had a number of conversations during
>> PyCon (as well as being at the language summit) that puts me in a
>> position to share the ideas and concerns that have been raised.
>>
>> The main trigger was a conversation I had with two employees of a
>> very large bank that has about 3000 Python users (not developers -
>> mostly financial analysts) and 16 million lines of code running on
>> 2.7. They are keen to migrate to 3.x but cannot afford to stop work
>> entirely while their code is updated. (There was much more to the
>> conversation than I'm relating here - I'm keeping to the directly
>> relevant bits.)
>>
>> In describing the approach they'd like to take, they made me realise
>> that there is definitely a place for a Python that is different but
>> mostly compatible with 2.7, in a way that 2.7.x could not be. For
>> the sake of having a name, I'll refer to this as "Python 2migr8"
>> (pronounced "to migrate" :) ).
>>
>> The two important components of Python 2migr8 would be the ability
>> to disable 2.7-only features, and to do so on a module-by-module
basis.
>>
>> My best idea so far would be to have a magic comment (to ensure 2.7
>> compatibility better than a "from __future__ ...") near the top of
>> the file that marks that file as "must straddle 2.7 and 3.3". Adding
>> this comment causes (for example) the parser to treat "except x, y"
>> as a syntax error in this file, forces "from __future__ import ...",
>> hides "dict.iterkeys", undefines "basestring", etc., but only for
>> this file. (I haven't thought through all the possibilities or
>> implications - Eric Snow said he was going to sprint on this
>> today/tomorrow, so he'll soon have a better idea just what can be
done.)
>>
>> In effect, 2migr8 would be the version that *only* supports
>> "single-source" files. This allows large code bases to progressively
>> migrate modules from 2.x to single-source while continuing to run
>> against Python 2.7. As files are updated, they are marked as being
>> single-source. When all files have this marker, it should be
>> possible to flip the switch and run with Python 3.3 or later.
>>
>> You could also think of this as enabling "-3 --warnings-as-errors"
>> for individual modules, though since the user has already opted in
>> to 2migr8, it isn't unreasonable to make more significant changes,
>> like having dict.keys returning a list that warns if it is mutated.
>> This sort of warning can only really be done by changing the
>> interpreter - static analysis just can't catch everything - and only
>> when users accept a potential performance hit and low probability of
>> breakage when they move to 2migr8 (followed by a not-quite-as-low
>> probability of breaking when they eventually move from 2migr8 to
>> 3.x, but it's still better than guaranteed breakage).
>>
>> As a fork, it would also be possible to bundle the modules that have
>> been backported, and possibly also to disallow importing deprecated
>> stdlib modules when 2.7 functionality is disabled. As I said, I
>> haven't thought through all the possibilities, but the general idea
>> is to take 2.7 and *remove* features so it becomes easier to migrate.
>>
>> Where does python-dev come in? Obviously this is where a fork like
>> this would have to start - there has been such strong and public
>> opposition to any significant changes like this that you'd be hard
>> pressed to find someone willing to start and promote it from
>> outside. There is also a good opportunity to make a start and
>> directly invite those using it to contribute the rules or warnings
>> that they need - the 3000 Python "users" I mentioned earlier are
>> backed by a team of true developers who are more than capable of
>> contributing, and this would be a great opportunity to directly
>> invite them. However unfair and incorrect it may be, there is a
>> perception in some businesses that open-source projects do not want
>> contributions from them. I invited more than one business to have
>> someone join python-dev and get involved during PyCon, and I heard
>> that others did the same - it may not be at the level of employing a
>> core developer full time, but it's the starting point that some
>> companies will ne
>> ed to be able to become comfortable with employing a core dev.
>>
>> I'm not pretending to have a full plan on how this will work. I was
>> privileged to have some private conversations during PyCon that are
>> directly relevant, so I'm bringing it here to promote the
>> discussion. Thanks to everyone I had a chance to chat to, and to
>> everyone generally for a great PyCon.
>>
>> Cheers,
>> Steve
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org <mailto:Python-Dev at python.org>
>>
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>>
>>
>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140414/7cc30db0/attachment.html>
More information about the Python-Dev
mailing list