Rough draft of x86-64 JIT backend GSoC proposal
Hi Guys, I am a student planning on applying to GSoC to create an x86-64 backend for the JIT. I have a (very) rough draft of my proposal, and I was hoping to get some feedback on it. Specifically, I'm wondering about operating system support. I've written the proposal as if I would support Linux/Mac OS X/Windows. I would be developing on Linux, so I think we can assume that would be fairly well supported, but obviously I would like it to work on Mac OS X and Windows as well. (I'm not sure if I would have the time/motivation to care about obscure BSDs). OTOH, if there are already a lot of outstanding issues on one of those platforms, I don't know that I would be able to get it working. So what do you think would be a reasonable goal here? Secondly, my timeline is pretty vague. The PSF proposal template recommends a week-by-week timeline, but honestly, I'm not sure how the time usage would break down. Any comments on that would be greatly appreciated. Here's the draft: === Proposal === The PyPy JIT, which has shown substantial performance improvements over CPython, often several times faster, does not currently support the x86-64 instruction set, making it impractical to use on 64-bit x86 systems. My proposal is to extend the existing x86 JIT backend to support x86-64 as well. === Deliverables === Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged into PyPy trunk. === Implementation plan === This is not a research proposal. The goal is simply to have a PyPy JIT that works out of the box on 64-bit CPUs, implemented as conservatively as possible. As such, I will attempt to reuse as much of the existing x86 backend that I can. In fact, the architectural similarities between x86 and x86-64 are large enough that I hope to implement a unified x86/x86-64 backend with the majority of the code working for either platform. There is an existing branch that, while very incomplete, has the beginnings of a unified x86/x86-64 instruction encoding module. I intend to use that branch as a starting point. Rough outline: 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a basis for instruction encoding. 2. Port the existing 32-bit backend to use the new instruction encoding scheme. 3. Add 64-bit support to the backend, A) Modify register allocator to use new general purpose and floating point registers. B) Port "ResOperation" operations to 64-bit C) Port guard failure handling to 64-bit 4. Test 64-bit on Mac OS X and Windows and fix inevitable issues. === About Me === I am a first-year Computer Science student at Flathead Valley Community College planning to transfer to Montana State University. I have several years of professional development experience. I am comfortable programming in Python, C and x86 assembly. Starting May 17th, I will be able to commit 40 hours/week to the project until the end of August. I may travel for a few weeks at some point in the summer, but I will have a laptop with me with the expectation of continuing full-time work. === Contact information === Email: jcreigh@gmail.com IRC: "jcreigh" on Freenode Phone: (will be given on request, but not preferred)
On Mon, Mar 29, 2010 at 9:12 PM, Jason Creighton <jcreigh@gmail.com> wrote:
That would be great to have, but I would be happy enough to have it on linux. Let's say that if pypy continues to work on os x and windows you'll fix remaining 64bit JIT bugs.
It's not like pypy works on anything else than linux os x and windows.
I think week based planning is ridiculous. That may be done for some simpler stuff, but not for this. I would say have some milestones, but avoid precise timing.
and over unladen swallow.
There is also part about asmgcc.
Proposal is rather short, but I don't see how to expand it. You might want to add that in case you finish this stuff earlier you would do this and that (or just general JIT work or not do anything and drink beer).
Starting from May 1 I leave my current job and I'm returning full time at university to work on my MSc project with PyPy. A FreeBSD 7.3 x86_64 buildbot from me is already available for the PyPy project. I also work under MacOS X (Snow Leopard) for my MSc project. I could probably help you for the particularities of these two platforms. Keep me updated! Gabriel Lavoie 2010/3/30 Maciej Fijalkowski <fijall@gmail.com>
-- Gabriel Lavoie glavoie@gmail.com
I agree with fijal that it seems short, but I don't know what else to put in. Updated draft: === Proposal === The PyPy JIT, which has shown substantial performance improvements over other implementations of Python, including CPython and Unladen Swallow, does not currently support the x86-64 instruction set, making it impractical to use on 64-bit x86 systems. My proposal is to extend the existing x86 JIT backend to support x86-64 as well. === Deliverables === Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged into PyPy trunk. === Implementation plan === This is not a research proposal. The goal is simply to have a PyPy JIT that works out of the box on 64-bit CPUs, implemented as conservatively as possible. As such, I will attempt to reuse as much of the existing x86 backend that I can. In fact, the architectural similarities between x86 and x86-64 are large enough that I hope to implement a unified x86/x86-64 backend with the majority of the code working for either platform. There is an existing branch that, while very incomplete, has the beginnings of a unified x86/x86-64 instruction encoding module, which can encode instructions for either x86 or x86-64 in a fairly seamless manner. I intend to use that branch as a starting point. Milestones: 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a basis for instruction encoding. Modify the existing 32-bit backend to use the new instruction encoding scheme. 2. Add 64-bit support to the backend, A) Add tests to ensure that 64-bit instructions are being generated correctly. B) Modify register allocator to use new general purpose and floating point registers. C) Port 32-bit specific portions of the JIT (for example, guard failure handling) to 64-bit 3. Test 64-bit on Mac OS X and Windows, and fix inevitable issues. If the 64-bit JIT is completed before the end of the summer, I will spend the remainder of the time working on other 64-bit or JIT-related aspects of PyPy, for example adapting the "asmgcc" garbage collection strategy to 64-bit. === Development Workflow === The PyPy project makes extensive use of Subversion branches for development, so I will follow that convention and develop the 64-bit JIT in one or more branches. For unit testing, I will use the py.test framework (already used throughout PyPy), aiming for 100% test coverage. === About Me === I am a first-year Computer Science student at Flathead Valley Community College planning to transfer to Montana State University. I have several years of professional development experience. I am comfortable programming in Python, C and x86 assembly. Starting May 17th, I will be able to commit 40 hours/week to the project until the end of August. I may travel for a few weeks at some point in the summer, but I will have a laptop with me with the expectation of continuing full-time work. === Contact information === Email: jcreigh@gmail.com IRC: "jcreigh" on Freenode Blog: http://jcreigh.blogspot.com Phone: <will be included in actual proposal, but pypy-dev is publicly archived...>
On Mon, Mar 29, 2010 at 9:12 PM, Jason Creighton <jcreigh@gmail.com> wrote:
That would be great to have, but I would be happy enough to have it on linux. Let's say that if pypy continues to work on os x and windows you'll fix remaining 64bit JIT bugs.
It's not like pypy works on anything else than linux os x and windows.
I think week based planning is ridiculous. That may be done for some simpler stuff, but not for this. I would say have some milestones, but avoid precise timing.
and over unladen swallow.
There is also part about asmgcc.
Proposal is rather short, but I don't see how to expand it. You might want to add that in case you finish this stuff earlier you would do this and that (or just general JIT work or not do anything and drink beer).
Starting from May 1 I leave my current job and I'm returning full time at university to work on my MSc project with PyPy. A FreeBSD 7.3 x86_64 buildbot from me is already available for the PyPy project. I also work under MacOS X (Snow Leopard) for my MSc project. I could probably help you for the particularities of these two platforms. Keep me updated! Gabriel Lavoie 2010/3/30 Maciej Fijalkowski <fijall@gmail.com>
-- Gabriel Lavoie glavoie@gmail.com
I agree with fijal that it seems short, but I don't know what else to put in. Updated draft: === Proposal === The PyPy JIT, which has shown substantial performance improvements over other implementations of Python, including CPython and Unladen Swallow, does not currently support the x86-64 instruction set, making it impractical to use on 64-bit x86 systems. My proposal is to extend the existing x86 JIT backend to support x86-64 as well. === Deliverables === Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged into PyPy trunk. === Implementation plan === This is not a research proposal. The goal is simply to have a PyPy JIT that works out of the box on 64-bit CPUs, implemented as conservatively as possible. As such, I will attempt to reuse as much of the existing x86 backend that I can. In fact, the architectural similarities between x86 and x86-64 are large enough that I hope to implement a unified x86/x86-64 backend with the majority of the code working for either platform. There is an existing branch that, while very incomplete, has the beginnings of a unified x86/x86-64 instruction encoding module, which can encode instructions for either x86 or x86-64 in a fairly seamless manner. I intend to use that branch as a starting point. Milestones: 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a basis for instruction encoding. Modify the existing 32-bit backend to use the new instruction encoding scheme. 2. Add 64-bit support to the backend, A) Add tests to ensure that 64-bit instructions are being generated correctly. B) Modify register allocator to use new general purpose and floating point registers. C) Port 32-bit specific portions of the JIT (for example, guard failure handling) to 64-bit 3. Test 64-bit on Mac OS X and Windows, and fix inevitable issues. If the 64-bit JIT is completed before the end of the summer, I will spend the remainder of the time working on other 64-bit or JIT-related aspects of PyPy, for example adapting the "asmgcc" garbage collection strategy to 64-bit. === Development Workflow === The PyPy project makes extensive use of Subversion branches for development, so I will follow that convention and develop the 64-bit JIT in one or more branches. For unit testing, I will use the py.test framework (already used throughout PyPy), aiming for 100% test coverage. === About Me === I am a first-year Computer Science student at Flathead Valley Community College planning to transfer to Montana State University. I have several years of professional development experience. I am comfortable programming in Python, C and x86 assembly. Starting May 17th, I will be able to commit 40 hours/week to the project until the end of August. I may travel for a few weeks at some point in the summer, but I will have a laptop with me with the expectation of continuing full-time work. === Contact information === Email: jcreigh@gmail.com IRC: "jcreigh" on Freenode Blog: http://jcreigh.blogspot.com Phone: <will be included in actual proposal, but pypy-dev is publicly archived...>
participants (3)
-
Gabriel Lavoie -
Jason Creighton -
Maciej Fijalkowski