[Python-Dev] Language Summit Follow-Up

[Nick Coghlan]

On 29 May 2014 08:26, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:

Thanks for the write-up!

> Here are some ideas for Python 2.7+.
>
> Add ensurepip to the installers.  Having pip reliably available increases
> the availability of libraries that help with porting, and will generally
> strengthen the broader ecosystem in the (increasingly long) transition
> period.

Agreed on this one - I only dropped it from 453 because I didn't think
I could continue to push for it and still make the beta deadline for
3.4, not because I changed my mind. This mainly needs a champion to
write the PEP to describe the exact details of what would be
backported (the tricky parts of PEP 453 relating to pyvenv don't
apply) and to actually do the work. Bundling the Python Launcher with
the Windows installer would also be desirable.

> Add some warnings about python 3 compatibility.
>
> It should at least be possible to get a warning for every single implicit
> string coercion.
> Old-style classes.
> Old-style division.
> Print statements.
> Old-style exception syntax.
> buffer().
> bytes(memoryview(b'abc'))
> Importing old locations from the stdlib (see point 4.)
> Long integer syntax.
> Use of variables beyond the lifetime of an 'except Exception as e' block or
> a list comprehension.

A few more for the "missing -3 warning" list:

    - calling str.encode
    - calling unicode.decode
    - returning str from str.decode
    - returning unicode from unicode.encode

The relevant TypeErrors in 3.x show where the latter two warnings need
to go in 2.7.

More controversially: warn for any calls to Py2 str methods that
aren't present on Py3 bytes.

> Backport 'yield from' to allow people to use Tulip and Tulip-compatible
> code, and to facilitate the development of Tulip-friendly libraries and a
> Tulip ecosystem.  A robust Tulip ecosystem requires the participation of
> people who are not yet using Python 3.

Given that the target audience here is our more conservative user
base, Tulip/Trollius backends for Twisted and gevent seem more
conducive to that than supporting Tulip's native coroutine syntax.
That said, it's a pure additive change without any significant
backwards compatibility implications, so I personally wouldn't object
if Guido said yes.

> Add aliases for the renamed modules in the stdlib.  This will allow people
> to "just write python 3" in a lot more circumstances.

This is what the module aliasing options provided by python-future are
designed to do - it's much closer to "just write Python 3" than six is
(although that comes at the price of being a bit more magical).

The python-future experience also shows there's a potential backwards
compatibility issue here - depending on exactly how you do it, you can
end up conflicting with other libraries trying to do the same thing.

> (re-)Enable warnings by default, including enabling -3 warnings.  Right now
> all warnings are silent by default, which greatly reduces discoverability of
> future compatibility issues.  I hope it's not controversial to say that most
> new Python code is still being written against Python 2.7 today; if people
> are writing that code in such a way that it's not 3-friendly, it should be a
> more immediately noticeable issue.

The reasons these were turned off haven't changed, and passing -3
already enables DeprecationWarnings generally (along with -Qwarn).
However, several of the warnings mentioned in the list above are
currently missing entirely, so they won't show up regardless of the
warning state.

> Get rid of 2to3. Particularly, of any discussion of using 2to3 in the
> documentation.  More than one very experienced, well-known Python developer
> in this discussion has told me that they thought 2to3 was the blessed way to
> port their code, and it's no surprise that they think so, given that the
> first technique <https://docs.python.org/3/howto/pyporting.html> mentions is
> still 2to3.  We should replace 2to3 with something like
> <https://github.com/mitsuhiko/python-modernize>. 2to3 breaks your code on
> python 2, and doesn't necessarily get it running on python 3.  A more
> conservative approach that reduced the amount of work to get your code 2/3
> compatible but was careful to leave everything working would be a lot more
> effective.

+1, although I think python-future provides a better "subset language"
experience than modernize+six does, and provides a 3->common subset
converter in addition to a 2->common subset (see
http://python-future.org/faq.html#relationship-between-python-future-and-other-compatibility-tools
for more on the difference between the two approaches).

The futurize script also has the nice property of supporting two
different stages, with "--stage1" being a pure syntactic update - no
new runtime dependencies needed.

"Big bang" migrations are highly unlikely to be the right choice for
anyone, and while the code produced by python-future will still have
some quirks (e.g. avoiding direct use of any mapping methods as PEP
469 ended up recommending, plus a bunch of import noise at the start
of the file)

> Add a new 'bytes' type that actually behaves like the Python 3 bytes type
> (bytes(5)).

This is available in python-future.

> We have rejected any changes for Python 3.5, simply because of the extremely
> long time to get those features into users hands.  Any changes for Python 3
> that we're proposing would need to get into a 3.4.x release, so that, for
> example, they can make their way into Ubuntu 14.04 LTS.
>
> Here are some ideas for Python 3.4.x:
>
> Usage of Python2 style syntax (for example, a print statement) or stdlib
> module names (for example, 'import urllib2') should result in a specific,
> informative warning, not a generic SyntaxError/ImportError.  This will
> really help new users.
> Add 'unicode' back as an alias for 'str'.  Just today I was writing some
> documentation where I had to resort to some awkward encoding tricks just to
> get a bytes object out without explaining the whole 2/3 dichotomy in some
> unrelated prose.

While I realise it's out of scope for your "near term changes" list,
there are still a few changes I think are worth exploring for the
Python 3.5+ time frame:

- the return of binary interpolation is already approved (but isn't
implemented yet)
- various other improvements to the Py3 binary data model (such as the
ideas in PEP 467)
- PEP 432 (cleaning up the startup sequence), or something along those
lines, has now been identified as a prerequisite for decoupling Python
3 from the locale encoding on Linux (and hence letting us more easily
ignore the entirely-inappropriate-for-the-21st-century C locale)
- I'd like to see someone take a serious tilt at proposing "NAME
<args>" call statement support at the language level. IPython already
supports that style of invocation for arbitrary callables (so print
statements still appear work even in Python 3 if you use IPython
rather than the default REPL or direct script execution), and there's
a proof of concept patch at http://bugs.python.org/issue18788 that
shows CPython's parser is able to handle the idea (see my final
comment for additional restrictions a full proposal would need)

For that last point, my interest is as much educational as it is in
easing the transition from Python 2. The parentheses in "print('Hello
world!')" mean introducing the idea of function calls early to explain
how it works, while being able to omit them makes it easier to gloss
over the distinction between statements and function calls initially
and then cover it later after the basics of flow control have been
nailed down.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia