[pypy-dev] Porting PyPy/rpython to Python 3

[VanL]

Just bundled up a few of the more mechanical changes into a PR and sent it
upstream.

On Thu, Apr 16, 2015 at 8:55 AM, VanL <van.lindberg at gmail.com> wrote:

> Hi Maciej,
>
> On Thu, Apr 16, 2015 at 3:48 AM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>
>> Hi Van.
>>
>> First of all I'm really sorry if we ever gave an impression that
>> working on porting RPython to Python 3 would not be welcomed and I
>> would like to strongly disagree with that.
>>
>> What we did say (or wanted to say) is that we're unlikely to put a
>> significant effort into doing the porting ourselves, however reviewing
>> the pull requests is definitely within the scope of the work someone
>> will do (if they languish, feel free to poke me personally).
>>
>
> That is good to hear, and consistent with the conversations I had at
> PyCon. I wasn't discouraged from doing the work (as you can see from this
> thread), but I just figured I needed to do it in a private fork that
> wouldn't be accepted upstream. With this encouragement, you will start to
> see PRs from me.
>
> So it might be useful to explain what I am about, as that will give
> context for what I am doing.
>
> I am creating a language which I call spy, for "sub-python," that is a
> strict semantic and syntactic subset of Python 3 that is AOT-compilable.
> You can think of this as an evolved version of rpython, but in Python 3,
> fully specified, and with a slightly different compilation model.
>
> Why? I have lots of reasons, but the initial seed of the idea came out of
> following a number of the "compile Python" projects over the years. What I
> determined was that, regardless of the approach used, there was a ceiling
> that each project reached *at almost exactly the same place.* For example,
> the subset of Python that can be handled in rpython, shedskin, and
> starkiller is almost feature-for-feature identical, despite the fact that
> each of these projects used a different approach. In conversations with the
> Numba folks, they are approaching this same boundary. Thus, there is a
> natural AOT subset of Python... that isn't too far from full Python. What's
> more, you all - the PyPy project - implemented full Python in this subset.
>
> A few Q&As:
>
> Why this project:  For fun. But I also have professional interest in a
> couple things that I think this would fix.
>
> First, deployment. There are some applications that it would *really* help
> me to be able to drop a binary on a server and go. AOT compilation gives me
> that, if I am willing to restrict myself to the spy subset.
>
> Second, libraries loadable in either cPython, PyPy, or even Jython (all
> via cffi).
>
> Third, I have an interest in mobile, where I believe that this approach
> would work better (it is similar to what Unity does, for example).
>
> Why Py3?:  I like Py3 better.  I want to use function annotations to
> provide information to the inferencing process. The annotations would
> provide new roots for a the type inferencing process - and importantly,
> would allow me to stop that process at module boundaries efficiently.
>
> Type inferencing? The types in typing.py don't work for us:  Yes, but we
> don't need to be restricted to those types only. There is no reason not to
> declare the types that we need - for example to have UInt32 as a possible
> type in a function annotation. This allows us to get rid of a fair amount
> of "noise" is the rpython implementation (or at least to sequester it
> better).
>
> What do you mean fully specified?  Well, I want to have a spec as to what
> is within this AOT subset. I think I can even detect unsupported language
> constructs on import and throw a SyntaxError (or whichever error is
> appropriate) if a non-AOT feature is used. If someone figures out how to
> AOT compile a new feature, then it is added to the subset. I want to allow
> for  different implementations.
>
> What changes in the compilation model?   One big one: Be able to
> effectively do type inferencing on a smaller piece of the program than "the
> whole program." I would like to stop/start at either function boundaries or
> at module boundaries. Declaring appropriate type information would let me
> do that.
>
> As for modules, I would just require that anything in __all__ be
> annotated. Only functions in __all__ would be exported, and it would be an
> error to access anything else. From a pypy/rpython perspective, this would
> allow builtin modules (like sre) to be separately compiled and not have to
> recompile the world when a change was made somewhere else.
>
> Why bother with rpython at all?   It seemed the fastest way to get to
> where I wanted to go:
>
> 1. Port rpython to Py3.
> 2. Reuse the PyPy3 target, but strip out everything that can't be AOT
> compiled.
> 3. Evolve the rpython syntax to py3/spy, make it nice, doc it, start work
> on new compilation model, etc.
>
> Thanks,
> Van
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20150416/902ff76d/attachment.html>