[pypy-dev] PyPy support for Python 3 -- request for comments

[Yury Selivanov]

Hello Antonio,

And what are the rough time-estimates?

Thank you,

-Yury
On 2011-09-12, at 9:49 AM, Antonio Cuni wrote:

> Hello pypy-dev,
> 
> in the past weeks, I and the other core developers have talked a lot about
> supporting Python 3 in PyPy.  The task is huge and it's unlikely that it will
> be completed shortly based only on volunteer work, so we came up with the
> following proposal, which splits the work into several steps and sub-steps,
> togheter with an estimate of how much money is needed to complete each one.
> 
> The plan is to publish the proposal on the blog, and publicly ask for
> donations, with the hope to collect enough to cover the whole work.  However,
> before putting it on the blog we would like to ask for your comments/thoughts
> about it.  Any feedback will be appreciated!
> 
> Thank you,
> Antonio (on the behalf of the PyPy team)
> 
> 
> Python 3 on PyPy
> =================
> 
> The release of Python 3 has been a major undertaking for the Python community,
> both technically and socially.  So far PyPy implements only the version 2 of
> the Python language, which creates a very dangerous potential for a community
> split.  We believe that by supporting both versions of the language we will
> help to fill the gap.
> 
> This project should help both the part of the Python community which is
> reluctant to use PyPy because it does not support Python 3, and the part which
> is reluctant to move to Python 3 because they are already PyPy users.
> 
> However, porting PyPy to Python 3 requires a lot of work, and it will take a
> long time before we can complete it by relying only on volunteer work.  Thus,
> we are asking the community to help with funding the necessary work, to make
> it happen faster.
> 
> High level description
> -----------------------
> 
> The goal of this project is to write an interpreter that interprets version
> 3 of Python language. To be precise we would aim at having Python 3.2
> interpreter together in the same codebase as python 2.7 one.
> 
> At the end of the project, it will be possible to decide at translation time
> whether to build an interpreter which supports Python 2.7 or Python 3.2 and
> both versions will be nightly tested and available from nightly builds.
> 
> The focus of this project is on compatibility, not performance.  In
> particular, it might be possible that the resulting Python 3 interpreter will
> be slower than the Python 2 one.  If needed, optimizing and making it more JIT
> friendly will be the scope of a separate project.
> 
> Step 1: core language
> ----------------------
> 
> In this step, we implement all the changes to the core language,
> i.e. everything which is not in the extension modules.  This includes, but it
> is not necessarily limited to the following items, which are split into two
> big areas:
> 
> * **Sub-step 1.1**: string vs unicode and I/O:
> 
>    - adapt the existing testing infrastructure to support running Python 3 code
> 
>    - string vs bytes: the interpreter uses unicode strings everywhere.
> 
>    - the ``print`` function
> 
>    - ``open`` is now an alias for ``io.open``, removal of the old file type.
> 
>    - string formatting (for the part which is not already implemented in Python
>      2.7)
> 
>    - the _io module (for the part which is not already implemented in Python
>      2.7)
> 
>    - syntactic changes to make ``io.py`` importable (in particular:
>      ``metaclass=...`` in class declarations)
> 
>    - **Estimate cost**: 35.000 $
> 
> * **Sub-step 1.2**: other syntactic changes, builtin types and functions,
>    exceptions:
> 
>    - views and iterators instead of lists (e.g., ``dict.items()``, ``map``,
>      ``range`` & co.)
> 
>    - new rules for ordering comparisons
> 
>    - removal of old-style classes
> 
>    - int/long unification
> 
>    - function annotations
> 
>    - smaller syntax changes, such as keyword-only arguments, ``nonlocal``,
>      extended iterable unpacking, set literals, dict and set comprehension, etc.
> 
>    - changes to exceptions: ``__traceback__`` attribute, chained exceptions,
>      ``del e`` at the end of the except block, etc.
> 
>    - changes to builtins: ``super``, ``input``, ``next()``, etc.
> 
>    - improved ``with`` statement
> 
>    - **Estimate cost**: 25.000 $
> 
> 
> Note that the distinction between sub-steps 1.1 and 1.2 is blur, and it might be
> possible that during the development we will decide to move items between the
> two sub-steps, as needed.
> 
> For more information, look at the various "What's new" documents:
> 
> - http://docs.python.org/py3k/whatsnew/3.0.html
> 
> - http://docs.python.org/py3k/whatsnew/3.1.html
> 
> - http://docs.python.org/py3k/whatsnew/3.2.html
> 
> **Total estimate cost**: 60.000 $
> 
> 
> Step 2: extension modules
> --------------------------
> 
> In this step, we implement all the changes to the extension modules which are
> written in C in CPython.  This includes, but it is not necessarily limited to:
> 
> - ``collections``, ``gzip``, ``bz2``, ``decimal``, ``itertools``, ``re``,
>  ``functools``, ``pickle``, ``_elementtree``, ``math``, etc.
> 
> **Estimate cost**: this is hard to do at this point, we will be able to give a
> more precise estimate as soon as Step 1 is completed.  As a reference, it
> should be possible to complete it with 35.000 $
> 
> Step 3: cpyext
> --------------
> 
> The ``cpyext`` module allows to load CPython C extensions in PyPy.  Since the
> C API changed a lot between Python 2.7 and Python 3.2, ``cpyext`` will not
> work out of the box in the Python 3 PyPy interpreter.  In this step, we will
> adapt it to work with Python 3 as well.
> 
> Note that, even for Python 2, ``cpyext`` is still in a beta state.  In
> particular, not all extension modules compile and load correctly.  As a
> consequence, the same will be true for Python 3 as well.  As a general rule,
> we expect that if a Python 2 module works with ``cpyext``, the corresponding
> Python 3 version will also work when this step is completed, although the
> details might vary depending on the exact C extension module.
> 
> **Estimate cost**: 10.000 $
> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev