Mailman 3 More Push towards Python 3.x? - pypy-dev

More Push towards Python 3.x?

older
Re: [pypy-dev] Dead loop occurs...

Phyo Arkar

6 Jan 2016 6 Jan '16

12:43 p.m.

Pypy, Python 3.x is lagging behind a lot. Python 3.5 is a version we should be Evolving to so Is there any planned timeline for Pypy Python 3.x ?

Attachments:

attachment.html (text/html — 209 bytes)

Show replies by date

I don't have a timeline (and as far as I know pypy-dev doesn't either), but I do intend to help push this forwards however I can. (I am brand new at this though) Work is progressing on 3.3. There's no point in trying to immediately work on 3.5, because all changes needed for 3.3 will mostly all need to be made for 3.5 anyway. As always in open source, if you want it faster, feel free to jump in and help. : )

Manuel Jacob

5:22 p.m.

Hi, On 2016-01-06 13:43, Phyo Arkar wrote:

...

Pypy, Python 3.x is lagging behind a lot. Python 3.5 is a version we should be Evolving to so Is there any planned timeline for Pypy Python 3.x ?

There is no official timeline. My personal roadmap, as one of the main contributors in the last months, is the following: 1) Regularly merge default into the py3k branch, adapting code to Python 3.x as necessary, trying to keep the buildbots green. 2) Implement missing features / fixing bugs in the py3.3 branch when I have time and motivation. 3) In any case, the py3.3 branch will be merged back into py3k and closed before or at the beginning of the Leysin sprint. A new branch py3.5 will be opened to create many opportunities for newcomers at the sprint. Many people have asked why we have a py3k branch, currently implementing Python 3.2, a version almost nobody uses. The point is to see easily whether a test failure resulted from the last merge from the default branch, or was failing a potentially long time before. This helps making sure we don't accumulate failing tests over time. -Manuel

Richard Plangger

14 Jan 14 Jan

5:32 p.m.

New subject: s390x libffi issue

Hi, so far this issue is not resolved. Sadly the argument is that I/we probably do not handle the return type of libffi correctly. Let me explain the problem (again). For instance in the test [1] the case lltype.FuncType([rffi.Short, rffi.Short], rffi.Short). Here is what happens step by step (if not readable in the mail see [3]): jit compiles trace: <trace call> -- parameter (1213,1213) both 64-bit | v ffi_closure_SYSV | v ffi_closure_helper_SYSV (1) | v < enters ctypes> closure_fcn | v _CallPythonObject | v (2) | v (3) contents of the buffer of (1) before the call: 0xdeadbeefdeadbeef contents of the buffer of (1) after the call: 0xdeadbeefdead097a || vv returns every frame back to <trace call> and saves 0xdeadbeefdead097a in the variable e.g. i42 = call_i(..., 1213, 1213) (1) provides the stack location (3) writes the result to (2) leads into ll2types.py internal_callback and returns 2426L My understanding is that: libffi expects that <trace call> should, just after returning the 64 bit value, cast the result to a 16 bit value. We cannot do that! AFAIK the jit has no notion of a narrower integer type than the machine register. The only exception is loading and storing from/to memory that are later sign extended. Any suggestions? Cheers, Richard [1] rpython/jit/backend/test/runner_test.py:test_call [2] https://github.com/python/cpython/blob/1fe0fd9feb6a4472a9a1b186502eb9c0b2366... [3] http://paste.pound-python.org/show/hvcqL68eqUApEH0PhnuG/

Armin Rigo

15 Jan 15 Jan

8:33 a.m.

New subject: s390x libffi issue

Hi Richard, On Thu, Jan 14, 2016 at 6:32 PM, Richard Plangger wrote:

...

<trace call> -- parameter (1213,1213) both 64-bit (2)

Do you mean in both cases 16-bit integer instead of 64-bit integer?

...

returns every frame back to <trace call> and saves 0xdeadbeefdead097a

...

My understanding is that: libffi expects that <trace call> should, just after returning the 64 bit value, cast the result to a 16 bit value. We cannot do that!

If I understand correctly, you can do exactly that. After a call instruction to a function that returns a 16 bits result, simply add another instruction to sign- or zero-extend the result to a full 64-bit value. Surely it is not a performance problem to add a single simple instruction after some rare calls? In more details: my point of view is that libffi is *documented* to return the value 0x000000000000097a, but instead it returns 0xdeadbeefdead097a. It's a bug in libffi, but maybe one that is not going to be fixed promptly. In that case, you can simply work around it. Specifically, after a "call_i" instruction that should return a 16-bit number: the official ABI says it should return 0x000000000000097a; when called via ctypes it returns 0xdeadbeefdead097a instead; so go for the careful solution and only assume that the last 16 bits are valid, and emit an instruction just after the call to sign- or zero-extend the result from 16 bits to 64 bits. Then you can leave the issue in the hands of libffi for s390x and not be annoyed if it doesn't get fixed. A bientôt, Armin.

David Edelsohn

1:28 p.m.

New subject: s390x libffi issue

On Fri, Jan 15, 2016 at 3:33 AM, Armin Rigo wrote:

...

Hi Richard,

On Thu, Jan 14, 2016 at 6:32 PM, Richard Plangger wrote:

...
<trace call> -- parameter (1213,1213) both 64-bit (2)

Do you mean in both cases 16-bit integer instead of 64-bit integer?

...
returns every frame back to <trace call> and saves 0xdeadbeefdead097a

...
My understanding is that: libffi expects that <trace call> should, just after returning the 64 bit value, cast the result to a 16 bit value. We cannot do that!

If I understand correctly, you can do exactly that. After a call instruction to a function that returns a 16 bits result, simply add another instruction to sign- or zero-extend the result to a full 64-bit value. Surely it is not a performance problem to add a single simple instruction after some rare calls?

In more details: my point of view is that libffi is *documented* to return the value 0x000000000000097a, but instead it returns 0xdeadbeefdead097a. It's a bug in libffi, but maybe one that is not going to be fixed promptly. In that case, you can simply work around it. Specifically, after a "call_i" instruction that should return a 16-bit number: the official ABI says it should return 0x000000000000097a; when called via ctypes it returns 0xdeadbeefdead097a instead; so go for the careful solution and only assume that the last 16 bits are valid, and emit an instruction just after the call to sign- or zero-extend the result from 16 bits to 64 bits. Then you can leave the issue in the hands of libffi for s390x and not be annoyed if it doesn't get fixed.

libffi is *documented* to return the non sign-extended value. - David

Richard Plangger

1:59 p.m.

New subject: s390x libffi issue

Hi,

...

libffi is *documented* to return the non sign-extended value.

I have fixed this issue at the call site. The caller sign/zero extends narrower integer types. The reason I did not change it to that in the first place is: I thought that it is not easy to determine this information at the callsite (because it is not done in any other backend). Apparently it is available. Cheers, Richard

Armin Rigo

6:06 p.m.

New subject: s390x libffi issue

Hi Richard, On Fri, Jan 15, 2016 at 2:59 PM, Richard Plangger wrote:

...

I have fixed this issue at the call site. The caller sign/zero extends narrower integer types. The reason I did not change it to that in the first place is: I thought that it is not easy to determine this information at the callsite (because it is not done in any other backend). Apparently it is available.

It is done in the x86 backend. See load_result() in x86/callbuilder.py, which abuses load_from_mem() to emit an instruction MOV(S|Z)X(bits) for sign- or zero extension. It is also done in the arm backend, where load_result() calls self._ensure_result_bit_extension(). A bientôt, Armin.

Armin Rigo

6:01 p.m.

New subject: s390x libffi issue

Hi David, On Fri, Jan 15, 2016 at 2:28 PM, David Edelsohn wrote:

...

...
In more details: my point of view is that libffi is *documented* to return the value 0x000000000000097a, but instead it returns 0xdeadbeefdead097a.

libffi is *documented* to return the non sign-extended value.

Ok. I think my confusion came from the fact that we have to tell ffi whether values are signed or unsigned. As far as I can tell, this would be only useful in order to sign- or zero-extend the result value, if it did. (The docs included with libffi-3.2.1.tar.gz don't seem to say anything about that, which would make us both wrong---this specific behavior is not documented at all. I may be missing a point; please correct me in that case.) A bientôt, Armin.

3023

Age (days ago)

3032

Last active (days ago)

List overview

Download

8 comments

6 participants

participants (6)

Armin Rigo
David Edelsohn
Manuel Jacob
marky1991 .
Phyo Arkar
Richard Plangger

More Push towards Python 3.x?

tags

participants (6)