[Python-ideas] Type Hinting - Performance booster ?

[Nick Coghlan]

On 22 December 2014 at 14:39, Guido van Rossum <guido at python.org> wrote:

> Ludovic,
>
> If I understand you correctly you would like type annotations to be used
> to speed up Python programs. You are not alone, but it's a really hard
> problem, because nobody wants their program to break. I think that
> eventually this will be possible, but in the short term, it would be a
> much, much bigger effort to create such a compiler than it is just to
> define (and reach agreement on) the syntax for annotations. I like to take
> smaller steps first, especially when it comes to defining syntax. Maybe
> once we have a standard type hinting notation someone will write a compiler.
>

I'd like to echo this particular sentiment, since my other replies could
easily give the impression I disagreed. I'm actually optimistic that once
this kind of annotation becomes popular, folks will find ways to make use
of it for performance improvements (even if it's just priming a JIT cache
to reduce warmup time).

The only idea I disagree with is that it's necessary to *wait* for this
feature for Pythonistais to start countering the "Python is slow" meme -
the 3.5 release is still almost a year away, and the migration of the
overall ecosystem from Python 2 to 3 is definitely still a work in
progress. As a result, I believe that in the near term, it's better to look
at the barriers to adoption for existing tools like PyPy, Numba and Cython
and start figuring out whether those barriers can be reduced in some way.

In the case of PyPy, deployment of web services using it can be awkward
relative to CPython. A prebuilt image on DockerHub that includes uWSGI +
the PyPy plugin + PyPy could be a good way to reduce barriers to entry for
Linux based deployments. A separate prebuilt image on DockerHub could also
be a good way to make it easy for folks to experiment with building web
services based on the PyPy STM branch.

Another barrier to PyPy adoption is the level of use of cffi vs hand
crafted C extensions. While the inclusion of pip with CPython makes cffi
more readily available, there's also in principle agreement to incorporate
it (and its dependencies) into the standard library. Folks interested in
increasing PyPy's popularity may want to get in touch with the cffi
developers and see what remaining barriers (other than available time)
there are to moving forward with that proposal as a PEP for Python 3.5.
(Standard library backports have more credibility as dependencies in many
situations, since the standard library inclusion acts as a clear
endorsement of the original project)

For Numba and Cython, challenges are more likely to be centred around the
process of getting started with the tools, and providing a suitable build
environment. Again, Docker images can help, but in this case, by providing
a preconfigured build environment that can either run in a local container
on Linux, or under a Linux VM on Mac OS X or Windows.

None of those Docker related ideas need to be filtered through the CPython
core development team, though, nor even through the development teams of
the individual projects. All it takes is someone sufficiently motivated to
figure out how to do it, publish their work, and then approach the relevant
project to say "Hey, I did this, does anyone else want to help promote and
maintain it, and perhaps give it an official project blessing?".

Unlike the Docker image ideas, putting together the cffi PEP *does* need
involvement from both the developers of the cffi project and the rest of
the CPython core development team, but if the cffi folks are amenable, it's
conceivable someone else could do the actual PEP wrangling.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141222/b2f5b1be/attachment.html>