[pypy-commit] pypy release-pypy2.7-5.x: merge default into release
mattip
pypy.commits at gmail.com
Tue Oct 3 06:55:49 EDT 2017
Author: Matti Picus <matti.picus at gmail.com>
Branch: release-pypy2.7-5.x
Changeset: r92566:84a2f3e6a7f8
Date: 2017-10-03 13:49 +0300
http://bitbucket.org/pypy/pypy/changeset/84a2f3e6a7f8/
Log: merge default into release
diff --git a/pypy/doc/release-v5.9.0.rst b/pypy/doc/release-v5.9.0.rst
--- a/pypy/doc/release-v5.9.0.rst
+++ b/pypy/doc/release-v5.9.0.rst
@@ -10,15 +10,18 @@
This new PyPy2.7 release includes the upstream stdlib version 2.7.13, and
PyPy3.5 includes the upstream stdlib version 3.5.3.
-NumPy and Pandas now work on PyPy2.7. Issues that appeared as excessive memory
+NumPy and Pandas now work on PyPy2.7 (together with Cython 0.27.1). Issues
+that appeared as excessive memory
use were cleared up and other incompatibilities were resolved. The C-API
compatibility layer does slow down code which crosses the python-c interface
often, we have ideas on how it could be improved, and still recommend
using pure python on PyPy or interfacing via CFFI_. Many other modules
based on C-API exentions now work on PyPy as well.
-Cython 0.27 (released last week) should support more projects with PyPy, both
-on PyPy2.7 and PyPy3.5 beta.
+Cython 0.27.1 (released very recently) supports more projects with PyPy, both
+on PyPy2.7 and PyPy3.5 beta. Note version **0.27.1** is now the minimum
+version that supports this version of PyPy, due to some interactions with
+updated C-API interface code.
We optimized the JSON parser for recurring string keys, which should decrease
memory use to 50% and increase parsing speed by up to 15% for large JSON files
@@ -148,6 +151,7 @@
* Issue 2590_: fix the bounds in the GC when allocating a lot of objects with finalizers
* Replace magical NOT RPYTHON comment with a decorator
* Implement ``socket.sendmsg()``/``.recvmsg()`` for py3.5
+ * Add ``memory_pressure`` for ``_SSLSocket`` objects
* Degredations
diff --git a/pypy/doc/whatsnew-head.rst b/pypy/doc/whatsnew-head.rst
--- a/pypy/doc/whatsnew-head.rst
+++ b/pypy/doc/whatsnew-head.rst
@@ -1,87 +1,7 @@
-==========================
-What's new in PyPy2.7 5.9+
-==========================
+===========================
+What's new in PyPy2.7 5.10+
+===========================
-.. this is a revision shortly after release-pypy2.7-v5.8.0
-.. startrev: 558bd00b3dd8
+.. this is a revision shortly after release-pypy2.7-v5.9.0
+.. startrev:d56dadcef996
-In previous versions of PyPy, ``instance.method`` would return always
-the same bound method object, when gotten out of the same instance (as
-far as ``is`` and ``id()`` can tell). CPython doesn't do that. Now
-PyPy, like CPython, returns a different bound method object every time.
-For ``type.method``, PyPy2 still returns always the same *unbound*
-method object; CPython does it for built-in types but not for
-user-defined types.
-
-.. branch: cffi-complex
-.. branch: cffi-char16-char32
-
-The two ``cffi-*`` branches are part of the upgrade to cffi 1.11.
-
-.. branch: ctypes_char_indexing
-
-Indexing into char* behaves differently than CPython
-
-.. branch: vmprof-0.4.8
-
-Improve and fix issues with vmprof
-
-.. branch: issue-2592
-
-CPyext PyListObject.pop must return the value
-
-.. branch: cpyext-hash_notimpl
-
-If ``tp_hash`` is ``PyObject_HashNotImplemented``, set ``obj.__dict__['__hash__']`` to None
-
-.. branch: cppyy-packaging
-
-Renaming of ``cppyy`` to ``_cppyy``.
-The former is now an external package installable with ``pip install cppyy``.
-
-.. branch: Enable_PGO_for_clang
-
-.. branch: nopax
-
-At the end of translation, run ``attr -q -s pax.flags -V m`` on
-PAX-enabled systems on the produced binary. This seems necessary
-because PyPy uses a JIT.
-
-.. branch: pypy_bytearray
-
-Improve ``bytearray`` performance (backported from py3.5)
-
-.. branch: gc-del-limit-growth
-
-Fix the bounds in the GC when allocating a lot of objects with finalizers,
-fixes issue #2590
-
-.. branch: arrays-force-less
-
-Small improvement to optimize list accesses with constant indexes better by
-throwing away information about them less eagerly.
-
-
-.. branch: getarrayitem-into-bridges
-
-More information is retained into a bridge: knowledge about the content of
-arrays (at fixed indices) is stored in guards (and thus available at the
-beginning of bridges). Also, some better feeding of information about known
-fields of constant objects into bridges.
-
-.. branch: cpyext-leakchecking
-
-Add support for leakfinder in cpyext tests (disabled for now, due to too many
-failures).
-
-.. branch: pypy_swappedbytes
-
-Added ``_swappedbytes_`` support for ``ctypes.Structure``
-
-.. branch: pycheck-macros
-
-Convert many Py*_Check cpyext functions into macros, like CPython.
-
-.. branch: py_ssize_t
-
-Explicitly use Py_ssize_t as the Signed type in pypy c-api
diff --git a/pypy/doc/whatsnew-pypy2-5.9.0.rst b/pypy/doc/whatsnew-pypy2-5.9.0.rst
--- a/pypy/doc/whatsnew-pypy2-5.9.0.rst
+++ b/pypy/doc/whatsnew-pypy2-5.9.0.rst
@@ -85,3 +85,12 @@
.. branch: py_ssize_t
Explicitly use Py_ssize_t as the Signed type in pypy c-api
+
+.. branch: cpyext-jit
+
+Differentiate the code to call METH_NOARGS, METH_O and METH_VARARGS in cpyext:
+this allows to write specialized code which is much faster than previous
+completely generic version. Moreover, let the JIT to look inside the cpyext
+module: the net result is that cpyext calls are up to 7x faster. However, this
+is true only for very simple situations: in all real life code, we are still
+much slower than CPython (more optimizations to come)
diff --git a/pypy/module/_cffi_backend/cffi1_module.py b/pypy/module/_cffi_backend/cffi1_module.py
--- a/pypy/module/_cffi_backend/cffi1_module.py
+++ b/pypy/module/_cffi_backend/cffi1_module.py
@@ -1,4 +1,5 @@
from rpython.rtyper.lltypesystem import lltype, rffi
+from rpython.rlib import jit
from pypy.interpreter.error import oefmt
from pypy.interpreter.module import Module
@@ -15,7 +16,7 @@
INITFUNCPTR = lltype.Ptr(lltype.FuncType([rffi.VOIDPP], lltype.Void))
-
+ at jit.dont_look_inside
def load_cffi1_module(space, name, path, initptr):
# This is called from pypy.module.cpyext.api.load_extension_module()
from pypy.module._cffi_backend.call_python import get_ll_cffi_call_python
diff --git a/pypy/module/_cffi_backend/ctypeptr.py b/pypy/module/_cffi_backend/ctypeptr.py
--- a/pypy/module/_cffi_backend/ctypeptr.py
+++ b/pypy/module/_cffi_backend/ctypeptr.py
@@ -156,10 +156,11 @@
class W_CTypePtrBase(W_CTypePtrOrArray):
# base class for both pointers and pointers-to-functions
- _attrs_ = ['is_void_ptr', 'is_voidchar_ptr']
- _immutable_fields_ = ['is_void_ptr', 'is_voidchar_ptr']
+ _attrs_ = ['is_void_ptr', 'is_voidchar_ptr', 'is_onebyte_ptr']
+ _immutable_fields_ = ['is_void_ptr', 'is_voidchar_ptr', 'is_onebyte_ptr']
is_void_ptr = False
is_voidchar_ptr = False
+ is_onebyte_ptr = False
def convert_to_object(self, cdata):
ptrdata = rffi.cast(rffi.CCHARPP, cdata)[0]
@@ -179,12 +180,20 @@
if self.is_void_ptr or other.is_void_ptr:
pass # cast from or to 'void *'
elif self.is_voidchar_ptr or other.is_voidchar_ptr:
- space = self.space
- msg = ("implicit cast from '%s' to '%s' "
- "will be forbidden in the future (check that the types "
- "are as you expect; use an explicit ffi.cast() if they "
- "are correct)" % (other.name, self.name))
- space.warn(space.newtext(msg), space.w_UserWarning)
+ # for backward compatibility, accept "char *" as either
+ # source of target. This is not what C does, though,
+ # so emit a warning that will eventually turn into an
+ # error. The warning is turned off if both types are
+ # pointers to single bytes.
+ if self.is_onebyte_ptr and other.is_onebyte_ptr:
+ pass # no warning
+ else:
+ space = self.space
+ msg = ("implicit cast from '%s' to '%s' "
+ "will be forbidden in the future (check that the types "
+ "are as you expect; use an explicit ffi.cast() if they "
+ "are correct)" % (other.name, self.name))
+ space.warn(space.newtext(msg), space.w_UserWarning)
else:
raise self._convert_error("compatible pointer", w_ob)
@@ -214,6 +223,7 @@
self.is_void_ptr = isinstance(ctitem, ctypevoid.W_CTypeVoid)
self.is_voidchar_ptr = (self.is_void_ptr or
isinstance(ctitem, ctypeprim.W_CTypePrimitiveChar))
+ self.is_onebyte_ptr = (ctitem.size == 1)
W_CTypePtrBase.__init__(self, space, size, extra, 2, ctitem)
def newp(self, w_init, allocator):
diff --git a/pypy/module/_cffi_backend/test/_backend_test_c.py b/pypy/module/_cffi_backend/test/_backend_test_c.py
--- a/pypy/module/_cffi_backend/test/_backend_test_c.py
+++ b/pypy/module/_cffi_backend/test/_backend_test_c.py
@@ -2099,7 +2099,8 @@
if sys.platform.startswith("linux"):
BWChar = new_primitive_type("wchar_t")
assert sizeof(BWChar) == 4
- assert int(cast(BWChar, -1)) == -1 # signed, on linux
+ # wchar_t is often signed on Linux, but not always (e.g. on ARM)
+ assert int(cast(BWChar, -1)) in (-1, 4294967295)
def test_char16():
BChar16 = new_primitive_type("char16_t")
@@ -3903,9 +3904,11 @@
BCharP = new_pointer_type(new_primitive_type("char"))
BIntP = new_pointer_type(new_primitive_type("int"))
BVoidP = new_pointer_type(new_void_type())
+ BUCharP = new_pointer_type(new_primitive_type("unsigned char"))
z1 = cast(BCharP, 0)
z2 = cast(BIntP, 0)
z3 = cast(BVoidP, 0)
+ z4 = cast(BUCharP, 0)
with warnings.catch_warnings(record=True) as w:
newp(new_pointer_type(BIntP), z1) # warn
assert len(w) == 1
@@ -3919,6 +3922,12 @@
assert len(w) == 2
newp(new_pointer_type(BIntP), z3) # fine
assert len(w) == 2
+ newp(new_pointer_type(BCharP), z4) # fine (ignore signedness here)
+ assert len(w) == 2
+ newp(new_pointer_type(BUCharP), z1) # fine (ignore signedness here)
+ assert len(w) == 2
+ newp(new_pointer_type(BUCharP), z3) # fine
+ assert len(w) == 2
# check that the warnings are associated with lines in this file
assert w[1].lineno == w[0].lineno + 4
diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -1316,7 +1316,7 @@
if not ctx:
raise ssl_error(space, "failed to allocate SSL context")
- rgc.add_memory_pressure(10 * 1024)
+ rgc.add_memory_pressure(10 * 1024 * 1024)
self = space.allocate_instance(_SSLContext, w_subtype)
self.ctx = ctx
self.check_hostname = False
diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -449,6 +449,11 @@
if func.__name__ in FUNCTIONS_BY_HEADER[header]:
raise ValueError("%s already registered" % func.__name__)
func._always_inline_ = 'try'
+ #
+ # XXX: should we @jit.dont_look_inside all the @cpython_api functions,
+ # or we should only disable some of them?
+ func._jit_look_inside_ = False
+ #
api_function = ApiFunction(
argtypes, restype, func,
error=_compute_error(error, restype), gil=gil,
@@ -558,6 +563,7 @@
'PyObject_GetBuffer', 'PyBuffer_Release',
'PyBuffer_FromMemory', 'PyBuffer_FromReadWriteMemory', 'PyBuffer_FromObject',
'PyBuffer_FromReadWriteObject', 'PyBuffer_New', 'PyBuffer_Type', '_Py_get_buffer_type',
+ '_Py_setfilesystemdefaultencoding',
'PyCObject_FromVoidPtr', 'PyCObject_FromVoidPtrAndDesc', 'PyCObject_AsVoidPtr',
'PyCObject_GetDesc', 'PyCObject_Import', 'PyCObject_SetVoidPtr',
@@ -653,7 +659,11 @@
'PyClass_Type': 'space.gettypeobject(W_ClassObject.typedef)',
'PyStaticMethod_Type': 'space.gettypeobject(StaticMethod.typedef)',
'PyCFunction_Type': 'space.gettypeobject(cpyext.methodobject.W_PyCFunctionObject.typedef)',
- 'PyWrapperDescr_Type': 'space.gettypeobject(cpyext.methodobject.W_PyCMethodObject.typedef)'
+ 'PyClassMethodDescr_Type': 'space.gettypeobject(cpyext.methodobject.W_PyCClassMethodObject.typedef)',
+ 'PyGetSetDescr_Type': 'space.gettypeobject(cpyext.typeobject.W_GetSetPropertyEx.typedef)',
+ 'PyMemberDescr_Type': 'space.gettypeobject(cpyext.typeobject.W_MemberDescr.typedef)',
+ 'PyMethodDescr_Type': 'space.gettypeobject(cpyext.methodobject.W_PyCMethodObject.typedef)',
+ 'PyWrapperDescr_Type': 'space.gettypeobject(cpyext.methodobject.W_PyCWrapperObject.typedef)',
}.items():
register_global(cpyname, 'PyTypeObject*', pypyexpr, header=pypy_decl)
@@ -1045,11 +1055,17 @@
get_capsule_type = rffi.llexternal('_%s_get_capsule_type' % prefix,
[], PyTypeObjectPtr,
compilation_info=eci, _nowrapper=True)
+ setdefenc = rffi.llexternal('_%s_setfilesystemdefaultencoding' % prefix,
+ [rffi.CCHARP], lltype.Void,
+ compilation_info=eci, _nowrapper=True)
def init_types(space):
from pypy.module.cpyext.typeobject import py_type_ready
+ from pypy.module.sys.interp_encoding import getfilesystemencoding
py_type_ready(space, get_buffer_type())
py_type_ready(space, get_cobject_type())
py_type_ready(space, get_capsule_type())
+ s = space.text_w(getfilesystemencoding(space))
+ setdefenc(rffi.str2charp(s, track_allocation=False)) # "leaks"
INIT_FUNCTIONS.append(init_types)
from pypy.module.posix.interp_posix import add_fork_hook
_reinit_tls = rffi.llexternal('%sThread_ReInitTLS' % prefix, [],
@@ -1315,17 +1331,20 @@
for decl in FORWARD_DECLS:
decls[pypy_decl].append("%s;" % (decl,))
decls[pypy_decl].append("""
- /* hack for https://bugs.python.org/issue29943 */
- PyAPI_FUNC(int) %s(PySliceObject *arg0,
- Signed arg1, Signed *arg2,
- Signed *arg3, Signed *arg4, Signed *arg5);
- static int PySlice_GetIndicesEx(PySliceObject *arg0, Py_ssize_t arg1,
- Py_ssize_t *arg2, Py_ssize_t *arg3, Py_ssize_t *arg4,
- Py_ssize_t *arg5) {
- return %s(arg0, arg1, arg2, arg3,
- arg4, arg5);
- }
- """ % ((mangle_name(prefix, 'PySlice_GetIndicesEx'),)*2))
+/* hack for https://bugs.python.org/issue29943 */
+
+PyAPI_FUNC(int) %s(PySliceObject *arg0,
+ Signed arg1, Signed *arg2,
+ Signed *arg3, Signed *arg4, Signed *arg5);
+#ifdef __GNUC__
+__attribute__((__unused__))
+#endif
+static int PySlice_GetIndicesEx(PySliceObject *arg0, Py_ssize_t arg1,
+ Py_ssize_t *arg2, Py_ssize_t *arg3, Py_ssize_t *arg4,
+ Py_ssize_t *arg5) {
+ return %s(arg0, arg1, arg2, arg3,
+ arg4, arg5);
+}""" % ((mangle_name(prefix, 'PySlice_GetIndicesEx'),)*2))
for header_name, header_functions in FUNCTIONS_BY_HEADER.iteritems():
header = decls[header_name]
diff --git a/pypy/module/cpyext/include/descrobject.h b/pypy/module/cpyext/include/descrobject.h
--- a/pypy/module/cpyext/include/descrobject.h
+++ b/pypy/module/cpyext/include/descrobject.h
@@ -1,34 +1,6 @@
#ifndef Py_DESCROBJECT_H
#define Py_DESCROBJECT_H
-#define PyDescr_COMMON \
- PyObject_HEAD \
- PyTypeObject *d_type; \
- PyObject *d_name
-
-typedef struct {
- PyDescr_COMMON;
-} PyDescrObject;
-
-typedef struct {
- PyDescr_COMMON;
- PyMethodDef *d_method;
-} PyMethodDescrObject;
-
-typedef struct {
- PyDescr_COMMON;
- struct PyMemberDef *d_member;
-} PyMemberDescrObject;
-
-typedef struct {
- PyDescr_COMMON;
- PyGetSetDef *d_getset;
-} PyGetSetDescrObject;
-
-typedef struct {
- PyDescr_COMMON;
- struct wrapperbase *d_base;
- void *d_wrapped; /* This can be any function pointer */
-} PyWrapperDescrObject;
+#include "cpyext_descrobject.h"
#endif
diff --git a/pypy/module/cpyext/include/fileobject.h b/pypy/module/cpyext/include/fileobject.h
--- a/pypy/module/cpyext/include/fileobject.h
+++ b/pypy/module/cpyext/include/fileobject.h
@@ -1,1 +1,2 @@
-#define Py_FileSystemDefaultEncoding NULL
+PyAPI_DATA(const char *) Py_FileSystemDefaultEncoding;
+PyAPI_FUNC(void) _Py_setfilesystemdefaultencoding(const char *);
diff --git a/pypy/module/cpyext/methodobject.py b/pypy/module/cpyext/methodobject.py
--- a/pypy/module/cpyext/methodobject.py
+++ b/pypy/module/cpyext/methodobject.py
@@ -1,4 +1,5 @@
-from rpython.rtyper.lltypesystem import lltype, rffi, llmemory
+from rpython.rtyper.lltypesystem import lltype, rffi
+from rpython.rlib import jit
from pypy.interpreter.baseobjspace import W_Root
from pypy.interpreter.error import OperationError, oefmt
@@ -10,8 +11,8 @@
from pypy.module.cpyext.api import (
CONST_STRING, METH_CLASS, METH_COEXIST, METH_KEYWORDS, METH_NOARGS, METH_O,
METH_STATIC, METH_VARARGS, PyObject, bootstrap_function,
- cpython_api, generic_cpy_call, CANNOT_FAIL,
- PyTypeObjectPtr, slot_function, cts)
+ cpython_api, generic_cpy_call, CANNOT_FAIL, slot_function, cts,
+ build_type_checkers)
from pypy.module.cpyext.pyobject import (
Py_DecRef, from_ref, make_ref, as_pyobj, make_typedescr)
@@ -42,8 +43,8 @@
from pypy.module.cpyext.object import _dealloc
_dealloc(space, py_obj)
-
class W_PyCFunctionObject(W_Root):
+ # TODO create a slightly different class depending on the c_ml_flags
def __init__(self, space, ml, w_self, w_module=None):
self.ml = ml
self.name = rffi.charp2str(rffi.cast(rffi.CCHARP,self.ml.c_ml_name))
@@ -56,7 +57,7 @@
w_self = self.w_self
flags = rffi.cast(lltype.Signed, self.ml.c_ml_flags)
flags &= ~(METH_CLASS | METH_STATIC | METH_COEXIST)
- if space.is_true(w_kw) and not flags & METH_KEYWORDS:
+ if not flags & METH_KEYWORDS and space.is_true(w_kw):
raise oefmt(space.w_TypeError,
"%s() takes no keyword arguments", self.name)
@@ -96,6 +97,20 @@
else:
return space.w_None
+class W_PyCFunctionObjectNoArgs(W_PyCFunctionObject):
+ def call(self, space, w_self, w_args, w_kw):
+ # Call the C function
+ if w_self is None:
+ w_self = self.w_self
+ func = self.ml.c_ml_meth
+ return generic_cpy_call(space, func, w_self, None)
+
+class W_PyCFunctionObjectSingleObject(W_PyCFunctionObject):
+ def call(self, space, w_self, w_o, w_kw):
+ if w_self is None:
+ w_self = self.w_self
+ func = self.ml.c_ml_meth
+ return generic_cpy_call(space, func, w_self, w_o)
class W_PyCMethodObject(W_PyCFunctionObject):
w_self = None
@@ -109,7 +124,7 @@
return self.space.unwrap(self.descr_method_repr())
def descr_method_repr(self):
- w_objclass = self.w_objclass
+ w_objclass = self.w_objclass
assert isinstance(w_objclass, W_TypeObject)
return self.space.newtext("<method '%s' of '%s' objects>" % (
self.name, w_objclass.name))
@@ -117,7 +132,7 @@
def descr_call(self, space, __args__):
args_w, kw_w = __args__.unpack()
if len(args_w) < 1:
- w_objclass = self.w_objclass
+ w_objclass = self.w_objclass
assert isinstance(w_objclass, W_TypeObject)
raise oefmt(space.w_TypeError,
"descriptor '%s' of '%s' object needs an argument",
@@ -125,7 +140,7 @@
w_instance = args_w[0]
# XXX: needs a stricter test
if not space.isinstance_w(w_instance, self.w_objclass):
- w_objclass = self.w_objclass
+ w_objclass = self.w_objclass
assert isinstance(w_objclass, W_TypeObject)
raise oefmt(space.w_TypeError,
"descriptor '%s' requires a '%s' object but received a '%T'",
@@ -137,6 +152,10 @@
ret = self.call(space, w_instance, w_args, w_kw)
return ret
+# PyPy addition, for Cython
+_, _ = build_type_checkers("MethodDescr", W_PyCMethodObject)
+
+
@cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL)
def PyCFunction_Check(space, w_obj):
from pypy.interpreter.function import BuiltinFunction
@@ -163,6 +182,7 @@
(self.name, self.w_objclass.getname(self.space)))
+
class W_PyCWrapperObject(W_Root):
def __init__(self, space, pto, method_name, wrapper_func,
wrapper_func_kwds, doc, func, offset=None):
@@ -210,6 +230,7 @@
(self.method_name,
self.w_objclass.name))
+ at jit.dont_look_inside
def cwrapper_descr_call(space, w_self, __args__):
self = space.interp_w(W_PyCWrapperObject, w_self)
args_w, kw_w = __args__.unpack()
@@ -220,10 +241,22 @@
space.setitem(w_kw, space.newtext(key), w_obj)
return self.call(space, w_self, w_args, w_kw)
+def cfunction_descr_call_noargs(space, w_self):
+ # special case for calling with flags METH_NOARGS
+ self = space.interp_w(W_PyCFunctionObjectNoArgs, w_self)
+ return self.call(space, None, None, None)
+def cfunction_descr_call_single_object(space, w_self, w_o):
+ # special case for calling with flags METH_O
+ self = space.interp_w(W_PyCFunctionObjectSingleObject, w_self)
+ return self.call(space, None, w_o, None)
+
+ at jit.dont_look_inside
def cfunction_descr_call(space, w_self, __args__):
+ # specialize depending on the W_PyCFunctionObject
self = space.interp_w(W_PyCFunctionObject, w_self)
args_w, kw_w = __args__.unpack()
+ # XXX __args__.unpack is slow
w_args = space.newtuple(args_w)
w_kw = space.newdict()
for key, w_obj in kw_w.items():
@@ -231,6 +264,7 @@
ret = self.call(space, None, w_args, w_kw)
return ret
+ at jit.dont_look_inside
def cclassmethod_descr_call(space, w_self, __args__):
self = space.interp_w(W_PyCFunctionObject, w_self)
args_w, kw_w = __args__.unpack()
@@ -271,6 +305,26 @@
)
W_PyCFunctionObject.typedef.acceptable_as_base_class = False
+W_PyCFunctionObjectNoArgs.typedef = TypeDef(
+ 'builtin_function_or_method', W_PyCFunctionObject.typedef,
+ __call__ = interp2app(cfunction_descr_call_noargs),
+ __doc__ = GetSetProperty(W_PyCFunctionObjectNoArgs.get_doc),
+ __module__ = interp_attrproperty_w('w_module', cls=W_PyCFunctionObjectNoArgs),
+ __name__ = interp_attrproperty('name', cls=W_PyCFunctionObjectNoArgs,
+ wrapfn="newtext_or_none"),
+ )
+W_PyCFunctionObjectNoArgs.typedef.acceptable_as_base_class = False
+
+W_PyCFunctionObjectSingleObject.typedef = TypeDef(
+ 'builtin_function_or_method', W_PyCFunctionObject.typedef,
+ __call__ = interp2app(cfunction_descr_call_single_object),
+ __doc__ = GetSetProperty(W_PyCFunctionObjectSingleObject.get_doc),
+ __module__ = interp_attrproperty_w('w_module', cls=W_PyCFunctionObjectSingleObject),
+ __name__ = interp_attrproperty('name', cls=W_PyCFunctionObjectSingleObject,
+ wrapfn="newtext_or_none"),
+ )
+W_PyCFunctionObjectSingleObject.typedef.acceptable_as_base_class = False
+
W_PyCMethodObject.typedef = TypeDef(
'method_descriptor',
__get__ = interp2app(cmethod_descr_get),
@@ -333,11 +387,15 @@
def PyClassMethod_New(space, w_func):
return ClassMethod(w_func)
- at cpython_api([PyTypeObjectPtr, lltype.Ptr(PyMethodDef)], PyObject)
+ at cts.decl("""
+ PyObject *
+ PyDescr_NewClassMethod(PyTypeObject *type, PyMethodDef *method)""")
def PyDescr_NewMethod(space, w_type, method):
return W_PyCMethodObject(space, method, w_type)
- at cpython_api([PyObject, lltype.Ptr(PyMethodDef)], PyObject)
+ at cts.decl("""
+ PyObject *
+ PyDescr_NewClassMethod(PyTypeObject *type, PyMethodDef *method)""")
def PyDescr_NewClassMethod(space, w_type, method):
return W_PyCClassMethodObject(space, method, w_type)
diff --git a/pypy/module/cpyext/modsupport.py b/pypy/module/cpyext/modsupport.py
--- a/pypy/module/cpyext/modsupport.py
+++ b/pypy/module/cpyext/modsupport.py
@@ -1,11 +1,13 @@
from rpython.rtyper.lltypesystem import rffi, lltype
from pypy.module.cpyext.api import cpython_api, cpython_struct, \
- METH_STATIC, METH_CLASS, METH_COEXIST, CANNOT_FAIL, CONST_STRING
+ METH_STATIC, METH_CLASS, METH_COEXIST, CANNOT_FAIL, CONST_STRING, \
+ METH_NOARGS, METH_O
from pypy.module.cpyext.pyobject import PyObject, as_pyobj
from pypy.interpreter.module import Module
from pypy.module.cpyext.methodobject import (
W_PyCFunctionObject, PyCFunction_NewEx, PyDescr_NewMethod,
- PyMethodDef, PyDescr_NewClassMethod, PyStaticMethod_New)
+ PyMethodDef, PyDescr_NewClassMethod, PyStaticMethod_New,
+ W_PyCFunctionObjectNoArgs, W_PyCFunctionObjectSingleObject)
from pypy.module.cpyext.pyerrors import PyErr_BadInternalCall
from pypy.module.cpyext.state import State
from pypy.interpreter.error import oefmt
@@ -79,6 +81,13 @@
space.newtext(rffi.charp2str(doc)))
return w_mod # borrowed result kept alive in PyImport_AddModule()
+def _create_pyc_function_object(space, method, w_self, w_name, flags):
+ flags &= ~(METH_CLASS | METH_STATIC | METH_COEXIST)
+ if flags == METH_NOARGS:
+ return W_PyCFunctionObjectNoArgs(space, method, w_self, w_name)
+ if flags == METH_O:
+ return W_PyCFunctionObjectSingleObject(space, method, w_self, w_name)
+ return W_PyCFunctionObject(space, method, w_self, w_name)
def convert_method_defs(space, dict_w, methods, w_type, w_self=None, name=None):
w_name = space.newtext_or_none(name)
@@ -98,7 +107,8 @@
raise oefmt(space.w_ValueError,
"module functions cannot set METH_CLASS or "
"METH_STATIC")
- w_obj = W_PyCFunctionObject(space, method, w_self, w_name)
+ w_obj = _create_pyc_function_object(space, method, w_self,
+ w_name, flags)
else:
if methodname in dict_w and not (flags & METH_COEXIST):
continue
diff --git a/pypy/module/cpyext/parse/cpyext_descrobject.h b/pypy/module/cpyext/parse/cpyext_descrobject.h
new file mode 100644
--- /dev/null
+++ b/pypy/module/cpyext/parse/cpyext_descrobject.h
@@ -0,0 +1,29 @@
+#define PyDescr_COMMON \
+ PyObject_HEAD \
+ PyTypeObject *d_type; \
+ PyObject *d_name
+
+typedef struct {
+ PyDescr_COMMON;
+} PyDescrObject;
+
+typedef struct {
+ PyDescr_COMMON;
+ PyMethodDef *d_method;
+} PyMethodDescrObject;
+
+typedef struct {
+ PyDescr_COMMON;
+ struct PyMemberDef *d_member;
+} PyMemberDescrObject;
+
+typedef struct {
+ PyDescr_COMMON;
+ PyGetSetDef *d_getset;
+} PyGetSetDescrObject;
+
+typedef struct {
+ PyDescr_COMMON;
+ struct wrapperbase *d_base;
+ void *d_wrapped; /* This can be any function pointer */
+} PyWrapperDescrObject;
diff --git a/pypy/module/cpyext/pyobject.py b/pypy/module/cpyext/pyobject.py
--- a/pypy/module/cpyext/pyobject.py
+++ b/pypy/module/cpyext/pyobject.py
@@ -14,7 +14,7 @@
from rpython.rlib.objectmodel import specialize
from rpython.rlib.objectmodel import keepalive_until_here
from rpython.rtyper.annlowlevel import llhelper
-from rpython.rlib import rawrefcount
+from rpython.rlib import rawrefcount, jit
from rpython.rlib.debug import fatalerror
@@ -151,6 +151,7 @@
class InvalidPointerException(Exception):
pass
+ at jit.dont_look_inside
def create_ref(space, w_obj, w_userdata=None, immortal=False):
"""
Allocates a PyObject, and fills its fields with info from the given
@@ -190,6 +191,7 @@
w_marker_deallocating = W_Root()
+ at jit.dont_look_inside
def from_ref(space, ref):
"""
Finds the interpreter object corresponding to the given reference. If the
@@ -227,6 +229,7 @@
assert isinstance(w_type, W_TypeObject)
return get_typedescr(w_type.layout.typedef).realize(space, ref)
+ at jit.dont_look_inside
def as_pyobj(space, w_obj, w_userdata=None, immortal=False):
"""
Returns a 'PyObject *' representing the given intepreter object.
diff --git a/pypy/module/cpyext/slotdefs.py b/pypy/module/cpyext/slotdefs.py
--- a/pypy/module/cpyext/slotdefs.py
+++ b/pypy/module/cpyext/slotdefs.py
@@ -429,23 +429,6 @@
from rpython.rlib.nonconst import NonConstant
-SLOTS = {}
-
- at specialize.memo()
-def get_slot_tp_function(space, typedef, name):
- """Return a description of the slot C function to use for the built-in
- type for 'typedef'. The 'name' is the slot name. This is a memo
- function that, after translation, returns one of a built-in finite set.
- """
- key = (typedef, name)
- try:
- return SLOTS[key]
- except KeyError:
- slot_func = build_slot_tp_function(space, typedef, name)
- api_func = slot_func.api_func if slot_func else None
- SLOTS[key] = api_func
- return api_func
-
def build_slot_tp_function(space, typedef, name):
w_type = space.gettypeobject(typedef)
@@ -793,7 +776,7 @@
def __init__(self, method_name, slot_name, function, wrapper1, wrapper2, doc):
self.method_name = method_name
self.slot_name = slot_name
- self.slot_names = ("c_" + slot_name).split(".")
+ self.slot_names = tuple(("c_" + slot_name).split("."))
self.slot_func = function
self.wrapper_func = wrapper1
self.wrapper_func_kwds = wrapper2
diff --git a/pypy/module/cpyext/src/missing.c b/pypy/module/cpyext/src/missing.c
--- a/pypy/module/cpyext/src/missing.c
+++ b/pypy/module/cpyext/src/missing.c
@@ -27,3 +27,7 @@
int Py_Py3kWarningFlag = 0;
int Py_HashRandomizationFlag = 0;
+const char *Py_FileSystemDefaultEncoding; /* filled when cpyext is imported */
+void _Py_setfilesystemdefaultencoding(const char *enc) {
+ Py_FileSystemDefaultEncoding = enc;
+}
diff --git a/pypy/module/cpyext/test/foo.c b/pypy/module/cpyext/test/foo.c
--- a/pypy/module/cpyext/test/foo.c
+++ b/pypy/module/cpyext/test/foo.c
@@ -83,6 +83,35 @@
return cls;
}
+// for CPython
+#ifndef PyMethodDescr_Check
+int PyMethodDescr_Check(PyObject* method)
+{
+ PyObject *meth = PyObject_GetAttrString((PyObject*)&PyList_Type, "append");
+ if (!meth) return 0;
+ int res = PyObject_TypeCheck(method, meth->ob_type);
+ Py_DECREF(meth);
+ return res;
+}
+#endif
+
+PyObject* make_classmethod(PyObject* method)
+{
+ // adapted from __Pyx_Method_ClassMethod
+ if (PyMethodDescr_Check(method)) {
+ PyMethodDescrObject *descr = (PyMethodDescrObject *)method;
+ PyTypeObject *d_type = descr->d_type;
+ return PyDescr_NewClassMethod(d_type, descr->d_method);
+ }
+ else if (PyMethod_Check(method)) {
+ return PyClassMethod_New(PyMethod_GET_FUNCTION(method));
+ }
+ else {
+ PyErr_SetString(PyExc_TypeError, "unknown method kind");
+ return NULL;
+ }
+}
+
static PyObject *
foo_unset(fooobject *self)
{
@@ -95,6 +124,7 @@
{"copy", (PyCFunction)foo_copy, METH_NOARGS, NULL},
{"create", (PyCFunction)foo_create, METH_NOARGS|METH_STATIC, NULL},
{"classmeth", (PyCFunction)foo_classmeth, METH_NOARGS|METH_CLASS, NULL},
+ {"fake_classmeth", (PyCFunction)foo_classmeth, METH_NOARGS, NULL},
{"unset_string_member", (PyCFunction)foo_unset, METH_NOARGS, NULL},
{NULL, NULL} /* sentinel */
};
@@ -167,19 +197,19 @@
/* copied from numpy scalartypes.c for inherited classes */
if (t->tp_bases && (PyTuple_GET_SIZE(t->tp_bases) > 1))
{
- PyTypeObject *sup;
- /* We are inheriting from a Python type as well so
+ PyTypeObject *sup;
+ /* We are inheriting from a Python type as well so
give it first dibs on conversion */
sup = (PyTypeObject *)PyTuple_GET_ITEM(t->tp_bases, 1);
- /* Prevent recursion */
- if (new_fooType != sup->tp_new)
+ /* Prevent recursion */
+ if (new_fooType != sup->tp_new)
{
o = sup->tp_new(t, args, kwds);
return o;
}
}
o = t->tp_alloc(t, 0);
- return o;
+ return o;
};
static PyMemberDef foo_members[] = {
@@ -714,7 +744,7 @@
"foo",
"Module Doc",
-1,
- foo_functions,
+ foo_functions,
NULL,
NULL,
NULL,
@@ -748,6 +778,7 @@
#endif
{
PyObject *d;
+ PyObject *fake_classmeth, *classmeth;
#if PY_MAJOR_VERSION >= 3
PyObject *module = PyModule_Create(&moduledef);
#else
@@ -805,6 +836,12 @@
INITERROR;
gettype2 = PyObject_New(PyObject, &GetType2);
+ fake_classmeth = PyDict_GetItemString((PyObject *)fooType.tp_dict, "fake_classmeth");
+ classmeth = make_classmethod(fake_classmeth);
+ if (classmeth == NULL)
+ INITERROR;
+ if (PyDict_SetItemString((PyObject *)fooType.tp_dict, "fake_classmeth", classmeth) < 0)
+ INITERROR;
d = PyModule_GetDict(module);
if (d == NULL)
diff --git a/pypy/module/cpyext/test/test_fileobject.py b/pypy/module/cpyext/test/test_fileobject.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/cpyext/test/test_fileobject.py
@@ -0,0 +1,13 @@
+from pypy.module.cpyext.test.test_cpyext import AppTestCpythonExtensionBase
+
+
+class AppTestFileObject(AppTestCpythonExtensionBase):
+ def test_defaultencoding(self):
+ import sys
+ module = self.import_extension('foo', [
+ ("defenc", "METH_NOARGS",
+ """
+ return PyString_FromString(Py_FileSystemDefaultEncoding);
+ """),
+ ])
+ assert module.defenc() == sys.getfilesystemencoding()
diff --git a/pypy/module/cpyext/test/test_typeobject.py b/pypy/module/cpyext/test/test_typeobject.py
--- a/pypy/module/cpyext/test/test_typeobject.py
+++ b/pypy/module/cpyext/test/test_typeobject.py
@@ -130,6 +130,12 @@
assert repr(descr) == "<method 'copy' of 'foo.foo' objects>"
raises(TypeError, descr, None)
+ def test_cython_fake_classmethod(self):
+ module = self.import_module(name='foo')
+ print(module.fooType.fake_classmeth)
+ print(type(module.fooType.fake_classmeth))
+ assert module.fooType.fake_classmeth() is module.fooType
+
def test_new(self):
# XXX cpython segfaults but if run singly (with -k test_new) this passes
module = self.import_module(name='foo')
@@ -1356,6 +1362,52 @@
assert Asize == Bsize
assert Asize > basesize
+ def test_multiple_inheritance_bug1(self):
+ module = self.import_extension('foo', [
+ ("get_type", "METH_NOARGS",
+ '''
+ Py_INCREF(&Foo_Type);
+ return (PyObject *)&Foo_Type;
+ '''
+ ), ("forty_two", "METH_O",
+ '''
+ return PyInt_FromLong(42);
+ '''
+ )], prologue='''
+ static PyTypeObject Foo_Type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ "foo.foo",
+ };
+ static PyObject *dummy_new(PyTypeObject *t, PyObject *a,
+ PyObject *k)
+ {
+ abort(); /* never actually called in CPython */
+ }
+ ''', more_init = '''
+ Foo_Type.tp_base = (PyTypeObject *)PyExc_Exception;
+ Foo_Type.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
+ Foo_Type.tp_new = dummy_new;
+ if (PyType_Ready(&Foo_Type) < 0) INITERROR;
+ ''')
+ Foo = module.get_type()
+ class A(Foo, SyntaxError):
+ pass
+ assert A.__base__ is SyntaxError
+ A(42) # assert is not aborting
+
+ class Bar(Exception):
+ __new__ = module.forty_two
+
+ class B(Bar, SyntaxError):
+ pass
+
+ assert B() == 42
+
+ # aaaaa even more hackiness
+ class C(A):
+ pass
+ C(42) # assert is not aborting
+
class AppTestHashable(AppTestCpythonExtensionBase):
def test_unhashable(self):
diff --git a/pypy/module/cpyext/typeobject.py b/pypy/module/cpyext/typeobject.py
--- a/pypy/module/cpyext/typeobject.py
+++ b/pypy/module/cpyext/typeobject.py
@@ -23,7 +23,6 @@
Py_TPFLAGS_TYPE_SUBCLASS,
Py_TPFLAGS_INT_SUBCLASS, Py_TPFLAGS_STRING_SUBCLASS, # change on py3
)
-from pypy.module.cpyext.cparser import parse_source
from pypy.module.cpyext.methodobject import (W_PyCClassMethodObject,
W_PyCWrapperObject, PyCFunction_NewEx, PyCFunction, PyMethodDef,
W_PyCMethodObject, W_PyCFunctionObject)
@@ -32,7 +31,7 @@
PyObject, make_ref, from_ref, get_typedescr, make_typedescr,
track_reference, Py_DecRef, as_pyobj)
from pypy.module.cpyext.slotdefs import (
- slotdefs_for_tp_slots, slotdefs_for_wrappers, get_slot_tp_function,
+ slotdefs_for_tp_slots, slotdefs_for_wrappers, build_slot_tp_function,
llslot)
from pypy.module.cpyext.state import State
from pypy.module.cpyext.structmember import PyMember_GetOne, PyMember_SetOne
@@ -42,6 +41,8 @@
from pypy.objspace.std.typeobject import W_TypeObject, find_best_base
+cts.parse_header(parse_dir / "cpyext_descrobject.h")
+
#WARN_ABOUT_MISSING_SLOT_FUNCTIONS = False
PyType_Check, PyType_CheckExact = build_type_checkers_flags("Type")
@@ -116,57 +117,24 @@
)
assert not W_MemberDescr.typedef.acceptable_as_base_class # no __new__
-PyDescrObject = lltype.ForwardReference()
-PyDescrObjectPtr = lltype.Ptr(PyDescrObject)
-PyDescrObjectFields = PyObjectFields + (
- ("d_type", PyTypeObjectPtr),
- ("d_name", PyObject),
- )
-cpython_struct("PyDescrObject", PyDescrObjectFields,
- PyDescrObject)
-
-PyMemberDescrObjectStruct = lltype.ForwardReference()
-PyMemberDescrObject = lltype.Ptr(PyMemberDescrObjectStruct)
-PyMemberDescrObjectFields = PyDescrObjectFields + (
- ("d_member", lltype.Ptr(PyMemberDef)),
- )
-cpython_struct("PyMemberDescrObject", PyMemberDescrObjectFields,
- PyMemberDescrObjectStruct, level=2)
-
-PyGetSetDescrObjectStruct = lltype.ForwardReference()
-PyGetSetDescrObject = lltype.Ptr(PyGetSetDescrObjectStruct)
-PyGetSetDescrObjectFields = PyDescrObjectFields + (
- ("d_getset", lltype.Ptr(PyGetSetDef)),
- )
-cpython_struct("PyGetSetDescrObject", PyGetSetDescrObjectFields,
- PyGetSetDescrObjectStruct, level=2)
-
-PyMethodDescrObjectStruct = lltype.ForwardReference()
-PyMethodDescrObject = lltype.Ptr(PyMethodDescrObjectStruct)
-PyMethodDescrObjectFields = PyDescrObjectFields + (
- ("d_method", lltype.Ptr(PyMethodDef)),
- )
-cpython_struct("PyMethodDescrObject", PyMethodDescrObjectFields,
- PyMethodDescrObjectStruct, level=2)
-
@bootstrap_function
def init_memberdescrobject(space):
make_typedescr(W_MemberDescr.typedef,
- basestruct=PyMemberDescrObject.TO,
+ basestruct=cts.gettype('PyMemberDescrObject'),
attach=memberdescr_attach,
realize=memberdescr_realize,
)
make_typedescr(W_GetSetPropertyEx.typedef,
- basestruct=PyGetSetDescrObject.TO,
+ basestruct=cts.gettype('PyGetSetDescrObject'),
attach=getsetdescr_attach,
)
make_typedescr(W_PyCClassMethodObject.typedef,
- basestruct=PyMethodDescrObject.TO,
+ basestruct=cts.gettype('PyMethodDescrObject'),
attach=methoddescr_attach,
realize=classmethoddescr_realize,
)
make_typedescr(W_PyCMethodObject.typedef,
- basestruct=PyMethodDescrObject.TO,
+ basestruct=cts.gettype('PyMethodDescrObject'),
attach=methoddescr_attach,
realize=methoddescr_realize,
)
@@ -176,7 +144,7 @@
Fills a newly allocated PyMemberDescrObject with the given W_MemberDescr
object. The values must not be modified.
"""
- py_memberdescr = rffi.cast(PyMemberDescrObject, py_obj)
+ py_memberdescr = cts.cast('PyMemberDescrObject*', py_obj)
# XXX assign to d_dname, d_type?
assert isinstance(w_obj, W_MemberDescr)
py_memberdescr.c_d_member = w_obj.member
@@ -195,7 +163,7 @@
Fills a newly allocated PyGetSetDescrObject with the given W_GetSetPropertyEx
object. The values must not be modified.
"""
- py_getsetdescr = rffi.cast(PyGetSetDescrObject, py_obj)
+ py_getsetdescr = cts.cast('PyGetSetDescrObject*', py_obj)
if isinstance(w_obj, GetSetProperty):
py_getsetdef = make_GetSet(space, w_obj)
assert space.isinstance_w(w_userdata, space.w_type)
@@ -207,7 +175,7 @@
py_getsetdescr.c_d_getset = w_obj.getset
def methoddescr_attach(space, py_obj, w_obj, w_userdata=None):
- py_methoddescr = rffi.cast(PyMethodDescrObject, py_obj)
+ py_methoddescr = cts.cast('PyMethodDescrObject*', py_obj)
# XXX assign to d_dname, d_type?
assert isinstance(w_obj, W_PyCFunctionObject)
py_methoddescr.c_d_method = w_obj.ml
@@ -258,7 +226,30 @@
dict_w[name] = w_descr
i += 1
+SLOTS = {}
+ at specialize.memo()
+def get_slot_tp_function(space, typedef, name):
+ """Return a description of the slot C function to use for the built-in
+ type for 'typedef'. The 'name' is the slot name. This is a memo
+ function that, after translation, returns one of a built-in finite set.
+ """
+ key = (typedef, name)
+ try:
+ return SLOTS[key]
+ except KeyError:
+ slot_func = build_slot_tp_function(space, typedef, name)
+ api_func = slot_func.api_func if slot_func else None
+ SLOTS[key] = api_func
+ return api_func
+
missing_slots={}
+def warn_missing_slot(space, method_name, slot_name, w_type):
+ if not we_are_translated():
+ if slot_name not in missing_slots:
+ missing_slots[slot_name] = w_type.getname(space)
+ print "missing slot %r/%r, discovered on %r" % (
+ method_name, slot_name, w_type.getname(space))
+
def update_all_slots(space, w_type, pto):
# fill slots in pto
# Not very sure about it, but according to
@@ -266,87 +257,78 @@
# overwrite slots that are already set: these ones are probably
# coming from a parent C type.
- if w_type.is_heaptype():
- typedef = None
- search_dict_w = w_type.dict_w
- else:
- typedef = w_type.layout.typedef
- search_dict_w = None
-
for method_name, slot_name, slot_names, slot_apifunc in slotdefs_for_tp_slots:
slot_func_helper = None
- if search_dict_w is None:
- # built-in types: expose as many slots as possible, even
- # if it happens to come from some parent class
- slot_apifunc = None # use get_slot_tp_function
+ w_descr = w_type.dict_w.get(method_name, None)
+ if w_descr:
+ # use the slot_apifunc (userslots) to lookup at runtime
+ pass
+ elif len(slot_names) ==1:
+ # 'inherit' from tp_base
+ slot_func_helper = getattr(pto.c_tp_base, slot_names[0])
else:
- # For heaptypes, w_type.layout.typedef will be object's typedef, and
- # get_slot_tp_function will fail
- w_descr = search_dict_w.get(method_name, None)
- if w_descr:
- # use the slot_apifunc (userslots) to lookup at runtime
- pass
- elif len(slot_names) ==1:
- # 'inherit' from tp_base
- slot_func_helper = getattr(pto.c_tp_base, slot_names[0])
- else:
- struct = getattr(pto.c_tp_base, slot_names[0])
- if struct:
- slot_func_helper = getattr(struct, slot_names[1])
+ struct = getattr(pto.c_tp_base, slot_names[0])
+ if struct:
+ slot_func_helper = getattr(struct, slot_names[1])
if not slot_func_helper:
- if typedef is not None:
- if slot_apifunc is None:
- slot_apifunc = get_slot_tp_function(space, typedef, slot_name)
if not slot_apifunc:
- if not we_are_translated():
- if slot_name not in missing_slots:
- missing_slots[slot_name] = w_type.getname(space)
- print "missing slot %r/%r, discovered on %r" % (
- method_name, slot_name, w_type.getname(space))
+ warn_missing_slot(space, method_name, slot_name, w_type)
continue
slot_func_helper = slot_apifunc.get_llhelper(space)
+ fill_slot(space, pto, w_type, slot_names, slot_func_helper)
- # XXX special case wrapper-functions and use a "specific" slot func
+def update_all_slots_builtin(space, w_type, pto):
+ typedef = w_type.layout.typedef
+ for method_name, slot_name, slot_names, slot_apifunc in slotdefs_for_tp_slots:
+ slot_apifunc = get_slot_tp_function(space, typedef, slot_name)
+ if not slot_apifunc:
+ warn_missing_slot(space, method_name, slot_name, w_type)
+ continue
+ slot_func_helper = slot_apifunc.get_llhelper(space)
+ fill_slot(space, pto, w_type, slot_names, slot_func_helper)
- if len(slot_names) == 1:
- if not getattr(pto, slot_names[0]):
- setattr(pto, slot_names[0], slot_func_helper)
- elif ((w_type is space.w_list or w_type is space.w_tuple) and
- slot_names[0] == 'c_tp_as_number'):
- # XXX hack - how can we generalize this? The problem is method
- # names like __mul__ map to more than one slot, and we have no
- # convenient way to indicate which slots CPython have filled
- #
- # We need at least this special case since Numpy checks that
- # (list, tuple) do __not__ fill tp_as_number
- pass
- elif (space.issubtype_w(w_type, space.w_basestring) and
- slot_names[0] == 'c_tp_as_number'):
- # like above but for any str type
- pass
- else:
- assert len(slot_names) == 2
- struct = getattr(pto, slot_names[0])
- if not struct:
- #assert not space.config.translating
- assert not pto.c_tp_flags & Py_TPFLAGS_HEAPTYPE
- if slot_names[0] == 'c_tp_as_number':
- STRUCT_TYPE = PyNumberMethods
- elif slot_names[0] == 'c_tp_as_sequence':
- STRUCT_TYPE = PySequenceMethods
- elif slot_names[0] == 'c_tp_as_buffer':
- STRUCT_TYPE = PyBufferProcs
- elif slot_names[0] == 'c_tp_as_mapping':
- STRUCT_TYPE = PyMappingMethods
- else:
- raise AssertionError(
- "Structure not allocated: %s" % (slot_names[0],))
- struct = lltype.malloc(STRUCT_TYPE, flavor='raw', zero=True)
- setattr(pto, slot_names[0], struct)
+ at specialize.arg(3)
+def fill_slot(space, pto, w_type, slot_names, slot_func_helper):
+ # XXX special case wrapper-functions and use a "specific" slot func
+ if len(slot_names) == 1:
+ if not getattr(pto, slot_names[0]):
+ setattr(pto, slot_names[0], slot_func_helper)
+ elif ((w_type is space.w_list or w_type is space.w_tuple) and
+ slot_names[0] == 'c_tp_as_number'):
+ # XXX hack - how can we generalize this? The problem is method
+ # names like __mul__ map to more than one slot, and we have no
+ # convenient way to indicate which slots CPython have filled
+ #
+ # We need at least this special case since Numpy checks that
+ # (list, tuple) do __not__ fill tp_as_number
+ pass
+ elif (space.issubtype_w(w_type, space.w_basestring) and
+ slot_names[0] == 'c_tp_as_number'):
+ # like above but for any str type
+ pass
+ else:
+ assert len(slot_names) == 2
+ struct = getattr(pto, slot_names[0])
+ if not struct:
+ #assert not space.config.translating
+ assert not pto.c_tp_flags & Py_TPFLAGS_HEAPTYPE
+ if slot_names[0] == 'c_tp_as_number':
+ STRUCT_TYPE = PyNumberMethods
+ elif slot_names[0] == 'c_tp_as_sequence':
+ STRUCT_TYPE = PySequenceMethods
+ elif slot_names[0] == 'c_tp_as_buffer':
+ STRUCT_TYPE = PyBufferProcs
+ elif slot_names[0] == 'c_tp_as_mapping':
+ STRUCT_TYPE = PyMappingMethods
+ else:
+ raise AssertionError(
+ "Structure not allocated: %s" % (slot_names[0],))
+ struct = lltype.malloc(STRUCT_TYPE, flavor='raw', zero=True)
+ setattr(pto, slot_names[0], struct)
- if not getattr(struct, slot_names[1]):
- setattr(struct, slot_names[1], slot_func_helper)
+ if not getattr(struct, slot_names[1]):
+ setattr(struct, slot_names[1], slot_func_helper)
def add_operators(space, dict_w, pto):
from pypy.module.cpyext.object import PyObject_HashNotImplemented
@@ -426,6 +408,10 @@
ptr = get_new_method_def(space)
ptr.c_ml_meth = rffi.cast(PyCFunction, llslot(space, tp_new_wrapper))
+ at jit.dont_look_inside
+def is_tp_new_wrapper(space, ml):
+ return ml.c_ml_meth == rffi.cast(PyCFunction, llslot(space, tp_new_wrapper))
+
def add_tp_new_wrapper(space, dict_w, pto):
if "__new__" in dict_w:
return
@@ -452,7 +438,7 @@
pto.c_tp_flags |= Py_TPFLAGS_INT_SUBCLASS
elif space.issubtype_w(w_obj, space.w_long):
pto.c_tp_flags |= Py_TPFLAGS_LONG_SUBCLASS
- elif space.issubtype_w(w_obj, space.w_bytes):
+ elif space.issubtype_w(w_obj, space.w_bytes):
pto.c_tp_flags |= Py_TPFLAGS_STRING_SUBCLASS # STRING->BYTES on py3
elif space.issubtype_w(w_obj, space.w_unicode):
pto.c_tp_flags |= Py_TPFLAGS_UNICODE_SUBCLASS
@@ -562,25 +548,18 @@
pto = obj.c_ob_type
base = pto
this_func_ptr = llslot(space, subtype_dealloc)
- w_obj = from_ref(space, rffi.cast(PyObject, base))
# This wrapper is created on a specific type, call it w_A.
# We wish to call the dealloc function from one of the base classes of w_A,
# the first of which is not this function itself.
# w_obj is an instance of w_A or one of its subclasses. So climb up the
# inheritance chain until base.c_tp_dealloc is exactly this_func, and then
# continue on up until they differ.
- #print 'subtype_dealloc, start from', rffi.charp2str(base.c_tp_name)
while base.c_tp_dealloc != this_func_ptr:
base = base.c_tp_base
assert base
- #print ' ne move to', rffi.charp2str(base.c_tp_name)
- w_obj = from_ref(space, rffi.cast(PyObject, base))
while base.c_tp_dealloc == this_func_ptr:
base = base.c_tp_base
assert base
- #print ' eq move to', rffi.charp2str(base.c_tp_name)
- w_obj = from_ref(space, rffi.cast(PyObject, base))
- #print ' end with', rffi.charp2str(base.c_tp_name)
dealloc = base.c_tp_dealloc
# XXX call tp_del if necessary
generic_cpy_call(space, dealloc, obj)
@@ -807,7 +786,10 @@
if pto.c_tp_itemsize < pto.c_tp_base.c_tp_itemsize:
pto.c_tp_itemsize = pto.c_tp_base.c_tp_itemsize
- update_all_slots(space, w_type, pto)
+ if w_type.is_heaptype():
+ update_all_slots(space, w_type, pto)
+ else:
+ update_all_slots_builtin(space, w_type, pto)
if not pto.c_tp_new:
base_object_pyo = make_ref(space, space.w_object)
base_object_pto = rffi.cast(PyTypeObjectPtr, base_object_pyo)
diff --git a/pypy/module/pypyjit/policy.py b/pypy/module/pypyjit/policy.py
--- a/pypy/module/pypyjit/policy.py
+++ b/pypy/module/pypyjit/policy.py
@@ -14,7 +14,7 @@
return True
if '.' in modname:
modname, rest = modname.split('.', 1)
- if modname in ['unicodedata', 'gc', '_minimal_curses', 'cpyext']:
+ if modname in ['unicodedata', 'gc', '_minimal_curses']:
return False
else:
rest = ''
diff --git a/pypy/objspace/std/typeobject.py b/pypy/objspace/std/typeobject.py
--- a/pypy/objspace/std/typeobject.py
+++ b/pypy/objspace/std/typeobject.py
@@ -626,6 +626,12 @@
if w_newdescr is None: # see test_crash_mro_without_object_1
raise oefmt(space.w_TypeError, "cannot create '%N' instances",
self)
+ #
+ # issue #2666
+ if space.config.objspace.usemodules.cpyext:
+ w_newtype, w_newdescr = self.hack_which_new_to_call(
+ w_newtype, w_newdescr)
+ #
w_newfunc = space.get(w_newdescr, self)
if (space.config.objspace.std.newshortcut and
not we_are_jitted() and
@@ -646,6 +652,46 @@
"__init__() should return None")
return w_newobject
+ def hack_which_new_to_call(self, w_newtype, w_newdescr):
+ # issue #2666: for cpyext, we need to hack in order to reproduce
+ # an "optimization" of CPython that actually changes behaviour
+ # in corner cases.
+ #
+ # * Normally, we use the __new__ found in the MRO in the normal way.
+ #
+ # * If by chance this __new__ happens to be implemented as a C
+ # function, then instead, we discard it and use directly
+ # self.__base__.tp_new.
+ #
+ # * Most of the time this is the same (and faster for CPython), but
+ # it can fail if self.__base__ happens not to be the first base.
+ #
+ from pypy.module.cpyext.methodobject import W_PyCFunctionObject
+
+ if isinstance(w_newdescr, W_PyCFunctionObject):
+ return self._really_hack_which_new_to_call(w_newtype, w_newdescr)
+ else:
+ return w_newtype, w_newdescr
+
+ def _really_hack_which_new_to_call(self, w_newtype, w_newdescr):
+ # This logic is moved in yet another helper function that
+ # is recursive. We call this only if we see a
+ # W_PyCFunctionObject. That's a performance optimization
+ # because in the common case, we won't call any function that
+ # contains the stack checks.
+ from pypy.module.cpyext.methodobject import W_PyCFunctionObject
+ from pypy.module.cpyext.typeobject import is_tp_new_wrapper
+
+ if (isinstance(w_newdescr, W_PyCFunctionObject) and
+ w_newtype is not self and
+ is_tp_new_wrapper(self.space, w_newdescr.ml)):
+ w_bestbase = find_best_base(self.bases_w)
+ if w_bestbase is not None:
+ w_newtype, w_newdescr = w_bestbase.lookup_where('__new__')
+ return w_bestbase._really_hack_which_new_to_call(w_newtype,
+ w_newdescr)
+ return w_newtype, w_newdescr
+
def descr_repr(self, space):
w_mod = self.get_module()
if w_mod is None or not space.isinstance_w(w_mod, space.w_text):
More information about the pypy-commit
mailing list