Build Pypy with different Interpreters
Is it possible to build different interpreters like Stackless, IronPython or Jython? If not - why? If yes - is it (in theory) possible to gain a speedup on those without GIL? (Is there multithreading at all the in translation process?) Thanks, Jan
On 8 September 2016 at 19:40, Jan Brohl <janbrohl@t-online.de> wrote:
Is it possible to build different interpreters like Stackless, IronPython or Jython?
That was actually the original motivation for creating pypy - maintaining all those different python implementations was a lot of unnecessary work. Stackless support is enabled by default. Support for translating to CLI and the JVM was eventually dropped for lack of interest. If someone wanted to re-add that support, they could learn from the mistakes that the previous implementation used. http://doc.pypy.org/en/latest/stackless.html
If not - why?
If yes - is it (in theory) possible to gain a speedup on those without GIL? (Is there multithreading at all the in translation process?)
Translation can't be done concurrently at the moment. I probably should have expanded upon this in my previous email, and maybe I will; there are a number of global structures, registries, and work lists that would need to be refactored before the translation work could be distributed. If that's the route the pypy team go, we will consider it after pypy itself supports parallelism. There's another route, which is to support separate compilation, and then to hand off the translation of built-in modules to different executors. This is itself quite a bit of work due to some inherent properties of rpython. -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.
Sorry, for the typo - I was asking if it is possible to build pypy *with* different interpreters instead of just cpython and pypy. (using eg "ipy64" instead of "pypy" or "python" in the translation step described at http://doc.pypy.org/en/latest/build.html#run-the-translation ) Am 08.09.2016 um 14:07 schrieb William ML Leslie:
On 8 September 2016 at 19:40, Jan Brohl <janbrohl@t-online.de <mailto:janbrohl@t-online.de>>wrote:
Is it possible to build different interpreters like Stackless, IronPython or Jython?
That was actually the original motivation for creating pypy - maintaining all those different python implementations was a lot of unnecessary work. Stackless support is enabled by default. Support for translating to CLI and the JVM was eventually dropped for lack of interest. If someone wanted to re-add that support, they could learn from the mistakes that the previous implementation used.
http://doc.pypy.org/en/latest/stackless.html
If not - why?
If yes - is it (in theory) possible to gain a speedup on those without GIL? (Is there multithreading at all the in translation process?)
Translation can't be done concurrently at the moment. I probably should have expanded upon this in my previous email, and maybe I will; there are a number of global structures, registries, and work lists that would need to be refactored before the translation work could be distributed. If that's the route the pypy team go, we will consider it after pypy itself supports parallelism.
There's another route, which is to support separate compilation, and then to hand off the translation of built-in modules to different executors. This is itself quite a bit of work due to some inherent properties of rpython.
-- William Leslie
Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.
On 8 September 2016 at 22:38, Jan Brohl <janbrohl@t-online.de> wrote:
Sorry, for the typo - I was asking if it is possible to build pypy *with* different interpreters instead of just cpython and pypy.
That makes sense, it fits with your other question (: The translator operates on functions that have been byte-compiled by the host compiler. IronPython and Jython compile directly to their corresponding virtual machine, so there's no cpython or pypy bytecode available.
(using eg "ipy64" instead of "pypy" or "python" in the translation step described at http://doc.pypy.org/en/latest/build.html#run-the-translation )
-- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.
On 8 September 2016 at 22:38, Jan Brohl <janbrohl@t-online.de> wrote:
Sorry, for the typo - I was asking if it is possible to build pypy *with* different interpreters instead of just cpython and pypy.
(using eg "ipy64" instead of "pypy" or "python" in the translation step described at http://doc.pypy.org/en/latest/build.html#run-the-translation )
I didn't say anything about Stackless; maybe I should. Stackless is CPython with a few extra features. Translation probably works on it, but given that Stackless has a GIL, and that rpython doesn't make use of any Stackless APIs, it's probably just the same as translating with CPython. -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.
Hi Jan The short answer is - all of those interpreters are a lot slower than pypy. Having even relatively good parallelization (and only some steps can be parallelized) would yield no improvements over using pure pypy for the most examples On Thu, Sep 8, 2016 at 2:38 PM, Jan Brohl <janbrohl@t-online.de> wrote:
Sorry, for the typo - I was asking if it is possible to build pypy *with* different interpreters instead of just cpython and pypy.
(using eg "ipy64" instead of "pypy" or "python" in the translation step described at http://doc.pypy.org/en/latest/build.html#run-the-translation )
Am 08.09.2016 um 14:07 schrieb William ML Leslie:
On 8 September 2016 at 19:40, Jan Brohl <janbrohl@t-online.de <mailto:janbrohl@t-online.de>>wrote:
Is it possible to build different interpreters like Stackless, IronPython or Jython?
That was actually the original motivation for creating pypy - maintaining all those different python implementations was a lot of unnecessary work. Stackless support is enabled by default. Support for translating to CLI and the JVM was eventually dropped for lack of interest. If someone wanted to re-add that support, they could learn from the mistakes that the previous implementation used.
http://doc.pypy.org/en/latest/stackless.html
If not - why?
If yes - is it (in theory) possible to gain a speedup on those without GIL? (Is there multithreading at all the in translation process?)
Translation can't be done concurrently at the moment. I probably should have expanded upon this in my previous email, and maybe I will; there are a number of global structures, registries, and work lists that would need to be refactored before the translation work could be distributed. If that's the route the pypy team go, we will consider it after pypy itself supports parallelism.
There's another route, which is to support separate compilation, and then to hand off the translation of built-in modules to different executors. This is itself quite a bit of work due to some inherent properties of rpython.
-- William Leslie
Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
participants (3)
-
Jan Brohl -
Maciej Fijalkowski -
William ML Leslie