[pypy-commit] pypy py3k: hg merge default
antocuni
noreply at buildbot.pypy.org
Mon Jun 11 17:20:06 CEST 2012
Author: Antonio Cuni <anto.cuni at gmail.com>
Branch: py3k
Changeset: r55590:b706ec75d2f4
Date: 2012-06-11 17:12 +0200
http://bitbucket.org/pypy/pypy/changeset/b706ec75d2f4/
Log: hg merge default
diff --git a/lib-python/2.7/distutils/sysconfig_pypy.py b/lib-python/2.7/distutils/sysconfig_pypy.py
--- a/lib-python/2.7/distutils/sysconfig_pypy.py
+++ b/lib-python/2.7/distutils/sysconfig_pypy.py
@@ -39,11 +39,10 @@
If 'prefix' is supplied, use it instead of sys.prefix or
sys.exec_prefix -- i.e., ignore 'plat_specific'.
"""
- if standard_lib:
- raise DistutilsPlatformError(
- "calls to get_python_lib(standard_lib=1) cannot succeed")
if prefix is None:
prefix = PREFIX
+ if standard_lib:
+ return os.path.join(prefix, "lib-python", get_python_version())
return os.path.join(prefix, 'site-packages')
diff --git a/pypy/doc/conf.py b/pypy/doc/conf.py
--- a/pypy/doc/conf.py
+++ b/pypy/doc/conf.py
@@ -45,9 +45,9 @@
# built documents.
#
# The short X.Y version.
-version = '1.8'
+version = '1.9'
# The full version, including alpha/beta/rc tags.
-release = '1.8'
+release = '1.9'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff --git a/pypy/doc/cpython_differences.rst b/pypy/doc/cpython_differences.rst
--- a/pypy/doc/cpython_differences.rst
+++ b/pypy/doc/cpython_differences.rst
@@ -85,13 +85,6 @@
_winreg
- Note that only some of these modules are built-in in a typical
- CPython installation, and the rest is from non built-in extension
- modules. This means that e.g. ``import parser`` will, on CPython,
- find a local file ``parser.py``, while ``import sys`` will not find a
- local file ``sys.py``. In PyPy the difference does not exist: all
- these modules are built-in.
-
* Supported by being rewritten in pure Python (possibly using ``ctypes``):
see the `lib_pypy/`_ directory. Examples of modules that we
support this way: ``ctypes``, ``cPickle``, ``cmath``, ``dbm``, ``datetime``...
diff --git a/pypy/doc/getting-started-python.rst b/pypy/doc/getting-started-python.rst
--- a/pypy/doc/getting-started-python.rst
+++ b/pypy/doc/getting-started-python.rst
@@ -103,10 +103,12 @@
executable. The executable behaves mostly like a normal Python interpreter::
$ ./pypy-c
- Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 19:41:03)
- [PyPy 1.8.0 with GCC 4.4.3] on linux2
+ Python 2.7.2 (341e1e3821ff, Jun 07 2012, 15:40:31)
+ [PyPy 1.9.0 with GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
- And now for something completely different: ``this sentence is false''
+ And now for something completely different: ``RPython magically makes you rich
+ and famous (says so on the tin)''
+
>>>> 46 - 4
42
>>>> from test import pystone
diff --git a/pypy/doc/getting-started.rst b/pypy/doc/getting-started.rst
--- a/pypy/doc/getting-started.rst
+++ b/pypy/doc/getting-started.rst
@@ -53,10 +53,10 @@
PyPy is ready to be executed as soon as you unpack the tarball or the zip
file, with no need to install it in any specific location::
- $ tar xf pypy-1.8-linux.tar.bz2
- $ ./pypy-1.8/bin/pypy
- Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 19:41:03)
- [PyPy 1.8.0 with GCC 4.4.3] on linux2
+ $ tar xf pypy-1.9-linux.tar.bz2
+ $ ./pypy-1.9/bin/pypy
+ Python 2.7.2 (341e1e3821ff, Jun 07 2012, 15:40:31)
+ [PyPy 1.9.0 with GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``it seems to me that once you
settle on an execution / object model and / or bytecode format, you've already
@@ -76,14 +76,14 @@
$ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
- $ ./pypy-1.8/bin/pypy distribute_setup.py
+ $ ./pypy-1.9/bin/pypy distribute_setup.py
- $ ./pypy-1.8/bin/pypy get-pip.py
+ $ ./pypy-1.9/bin/pypy get-pip.py
- $ ./pypy-1.8/bin/pip install pygments # for example
+ $ ./pypy-1.9/bin/pip install pygments # for example
-3rd party libraries will be installed in ``pypy-1.8/site-packages``, and
-the scripts in ``pypy-1.8/bin``.
+3rd party libraries will be installed in ``pypy-1.9/site-packages``, and
+the scripts in ``pypy-1.9/bin``.
Installing using virtualenv
---------------------------
diff --git a/pypy/doc/index.rst b/pypy/doc/index.rst
--- a/pypy/doc/index.rst
+++ b/pypy/doc/index.rst
@@ -15,7 +15,7 @@
* `FAQ`_: some frequently asked questions.
-* `Release 1.8`_: the latest official release
+* `Release 1.9`_: the latest official release
* `PyPy Blog`_: news and status info about PyPy
@@ -75,7 +75,7 @@
.. _`Getting Started`: getting-started.html
.. _`Papers`: extradoc.html
.. _`Videos`: video-index.html
-.. _`Release 1.8`: http://pypy.org/download.html
+.. _`Release 1.9`: http://pypy.org/download.html
.. _`speed.pypy.org`: http://speed.pypy.org
.. _`RPython toolchain`: translation.html
.. _`potential project ideas`: project-ideas.html
@@ -120,9 +120,9 @@
Windows, on top of .NET, and on top of Java.
To dig into PyPy it is recommended to try out the current
Mercurial default branch, which is always working or mostly working,
-instead of the latest release, which is `1.8`__.
+instead of the latest release, which is `1.9`__.
-.. __: release-1.8.0.html
+.. __: release-1.9.0.html
PyPy is mainly developed on Linux and Mac OS X. Windows is supported,
but platform-specific bugs tend to take longer before we notice and fix
diff --git a/pypy/doc/release-1.9.0.rst b/pypy/doc/release-1.9.0.rst
--- a/pypy/doc/release-1.9.0.rst
+++ b/pypy/doc/release-1.9.0.rst
@@ -5,7 +5,7 @@
We're pleased to announce the 1.9 release of PyPy. This release brings mostly
bugfixes, performance improvements, other small improvements and overall
progress on the `numpypy`_ effort.
-It also brings an improved situation on windows and OS X.
+It also brings an improved situation on Windows and OS X.
You can download the PyPy 1.9 release here:
@@ -28,8 +28,8 @@
.. _`pypy 1.9 and cpython 2.7.2`: http://speed.pypy.org
-Thanks to our donators
-======================
+Thanks to our donors
+====================
But first of all, we would like to say thank you to all people who
donated some money to one of our four calls:
@@ -46,7 +46,7 @@
programmers to get funded like that, at least for some
time. We want to include this thank you in the present release
announcement even though most of the work is not finished yet. More
-precisely, neither Py3k nor STM are ready to make it an official release
+precisely, neither Py3k nor STM are ready to make it in an official release
yet: people interested in them need to grab and (attempt to) translate
PyPy from the corresponding branches (respectively ``py3k`` and
``stm-thread``).
@@ -58,8 +58,7 @@
Highlights
==========
-* This release still implements Python 2.7, the standard library has been
- upgraded to CPython 2.7.2.
+* This release still implements Python 2.7.2.
* Many bugs were corrected for Windows 32 bit. This includes new
functionality to test the validity of file descriptors; and
@@ -68,12 +67,12 @@
and Amaury Forgeot d'Arc.
* Improvements in ``cpyext``, our emulator for CPython C extension modules.
- For example PyOpenSSL should now work.
+ For example PyOpenSSL should now work. We thank various people for help.
* Sets now have strategies just like dictionaries. This means for example
that a set containing only ints will be more compact (and faster).
-* A lot of progress on various aspects of ``numpypy``. See `numpy-status`_
+* A lot of progress on various aspects of ``numpypy``. See the `numpy-status`_
page for the automatic report.
* It is now possible to create and manipulate C-like structures using the
@@ -86,22 +85,27 @@
* The non-x86 backends for the JIT are progressing but are still not
merged (ARMv7 and PPC64).
-* JIT hooks for inspecting the created assembler code has been improved.
+* JIT hooks for inspecting the created assembler code have been improved.
See `JIT hooks documentation`_ for details.
-* ``select.kqueue`` has been added.
+* ``select.kqueue`` has been added (BSD).
* Handling of keyword arguments has been drastically improved in the best-case
- scenario.
+ scenario: proxy functions which simply forwards ``*args`` and ``**kwargs``
+ to another function now performs much better with the JIT.
* List comprehension has been improved.
+.. _`numpy-status`: http://buildbot.pypy.org/numpy-status/latest.html
+.. _`JIT hooks documentation`: http://doc.pypy.org/en/latest/jit-hooks.html
+
JitViewer
=========
There is a corresponding 1.9 release of JitViewer which is guaranteed to work
-with PyPy 1.9. See `JitViewer docs`_ for details.
+with PyPy 1.9. See the `JitViewer docs`_ for details.
-.. _`numpy status`: http://buildbot.pypy.org/numpy-status/latest.html
.. _`JitViewer docs`: http://bitbucket.org/pypy/jitviewer
-.. _`JIT hooks documentation`: http://doc.pypy.org/en/latest/jit-hooks.html
+
+Cheers,
+The PyPy Team
diff --git a/pypy/interpreter/astcompiler/test/test_astbuilder.py b/pypy/interpreter/astcompiler/test/test_astbuilder.py
--- a/pypy/interpreter/astcompiler/test/test_astbuilder.py
+++ b/pypy/interpreter/astcompiler/test/test_astbuilder.py
@@ -1118,6 +1118,18 @@
s = ast_from_node(space, tree, info).body[0].value
assert isinstance(s, ast.Str)
assert space.eq_w(s.s, space.wrap(japan))
+
+ def test_string_bug(self):
+ py3k_skip('fixme')
+ space = self.space
+ source = '# -*- encoding: utf8 -*-\nstuff = "x \xc3\xa9 \\n"\n'
+ info = pyparse.CompileInfo("<test>", "exec")
+ tree = self.parser.parse_source(source, info)
+ assert info.encoding == "utf8"
+ s = ast_from_node(space, tree, info).body[0].value
+ assert isinstance(s, ast.Str)
+ expected = ['x', ' ', chr(0xc3), chr(0xa9), ' ', '\n']
+ assert space.eq_w(s.s, space.wrap(''.join(expected)))
def test_number(self):
def get_num(s):
diff --git a/pypy/interpreter/pyparser/parsestring.py b/pypy/interpreter/pyparser/parsestring.py
--- a/pypy/interpreter/pyparser/parsestring.py
+++ b/pypy/interpreter/pyparser/parsestring.py
@@ -94,7 +94,8 @@
return space.wrap(v)
need_encoding = (encoding is not None and
- encoding != "utf-8" and encoding != "iso-8859-1")
+ encoding != "utf-8" and encoding != "utf8" and
+ encoding != "iso-8859-1")
assert 0 <= ps <= q
substr = s[ps : q]
if rawmode or '\\' not in s[ps:]:
@@ -126,19 +127,18 @@
builder = StringBuilder(len(s))
ps = 0
end = len(s)
- while 1:
- ps2 = ps
- while ps < end and s[ps] != '\\':
+ while ps < end:
+ if s[ps] != '\\':
+ # note that the C code has a label here.
+ # the logic is the same.
if recode_encoding and ord(s[ps]) & 0x80:
w, ps = decode_utf8(space, s, ps, end, recode_encoding)
+ # Append bytes to output buffer.
builder.append(w)
- ps2 = ps
else:
+ builder.append(s[ps])
ps += 1
- if ps > ps2:
- builder.append_slice(s, ps2, ps)
- if ps == end:
- break
+ continue
ps += 1
if ps == end:
diff --git a/pypy/interpreter/pyparser/test/test_parsestring.py b/pypy/interpreter/pyparser/test/test_parsestring.py
--- a/pypy/interpreter/pyparser/test/test_parsestring.py
+++ b/pypy/interpreter/pyparser/test/test_parsestring.py
@@ -84,3 +84,10 @@
s = '"""' + '\\' + '\n"""'
w_ret = parsestring.parsestr(space, None, s)
assert space.str_w(w_ret) == ''
+
+ def test_bug1(self):
+ space = self.space
+ expected = ['x', ' ', chr(0xc3), chr(0xa9), ' ', '\n']
+ input = ["'", 'x', ' ', chr(0xc3), chr(0xa9), ' ', chr(92), 'n', "'"]
+ w_ret = parsestring.parsestr(space, 'utf8', ''.join(input))
+ assert space.str_w(w_ret) == ''.join(expected)
diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py
--- a/pypy/jit/backend/test/runner_test.py
+++ b/pypy/jit/backend/test/runner_test.py
@@ -1110,6 +1110,79 @@
def test_virtual_ref_finish(self):
pass # VIRTUAL_REF_FINISH must not reach the backend nowadays
+ def test_arguments_to_execute_token(self):
+ # this test checks that execute_token() can be called with any
+ # variant of ints and floats as arguments
+ if self.cpu.supports_floats:
+ numkinds = 2
+ else:
+ numkinds = 1
+ seed = random.randrange(0, 10000)
+ print 'Seed is', seed # or choose it by changing the previous line
+ r = random.Random()
+ r.seed(seed)
+ for nb_args in range(50):
+ print 'Passing %d arguments to execute_token...' % nb_args
+ #
+ inputargs = []
+ values = []
+ for k in range(nb_args):
+ kind = r.randrange(0, numkinds)
+ if kind == 0:
+ inputargs.append(BoxInt())
+ values.append(r.randrange(-100000, 100000))
+ else:
+ inputargs.append(BoxFloat())
+ values.append(longlong.getfloatstorage(r.random()))
+ #
+ looptoken = JitCellToken()
+ faildescr = BasicFailDescr(42)
+ operations = []
+ retboxes = []
+ retvalues = []
+ #
+ ks = range(nb_args)
+ random.shuffle(ks)
+ for k in ks:
+ if isinstance(inputargs[k], BoxInt):
+ newbox = BoxInt()
+ x = r.randrange(-100000, 100000)
+ operations.append(
+ ResOperation(rop.INT_ADD, [inputargs[k],
+ ConstInt(x)], newbox)
+ )
+ y = values[k] + x
+ else:
+ newbox = BoxFloat()
+ x = r.random()
+ operations.append(
+ ResOperation(rop.FLOAT_ADD, [inputargs[k],
+ constfloat(x)], newbox)
+ )
+ y = longlong.getrealfloat(values[k]) + x
+ y = longlong.getfloatstorage(y)
+ kk = r.randrange(0, len(retboxes)+1)
+ retboxes.insert(kk, newbox)
+ retvalues.insert(kk, y)
+ #
+ operations.append(
+ ResOperation(rop.FINISH, retboxes, None, descr=faildescr)
+ )
+ print inputargs
+ for op in operations:
+ print op
+ self.cpu.compile_loop(inputargs, operations, looptoken)
+ #
+ fail = self.cpu.execute_token(looptoken, *values)
+ assert fail.identifier == 42
+ #
+ for k in range(len(retvalues)):
+ if isinstance(retboxes[k], BoxInt):
+ got = self.cpu.get_latest_value_int(k)
+ else:
+ got = self.cpu.get_latest_value_float(k)
+ assert got == retvalues[k]
+
def test_jump(self):
# this test generates small loops where the JUMP passes many
# arguments of various types, shuffling them around.
diff --git a/pypy/jit/backend/x86/tool/viewcode.py b/pypy/jit/backend/x86/tool/viewcode.py
--- a/pypy/jit/backend/x86/tool/viewcode.py
+++ b/pypy/jit/backend/x86/tool/viewcode.py
@@ -253,7 +253,7 @@
self.logentries[addr] = pieces[3]
elif line.startswith('SYS_EXECUTABLE '):
filename = line[len('SYS_EXECUTABLE '):].strip()
- if filename != self.executable_name:
+ if filename != self.executable_name and filename != '??':
self.symbols.update(load_symbols(filename))
self.executable_name = filename
diff --git a/pypy/module/_ffi/interp_funcptr.py b/pypy/module/_ffi/interp_funcptr.py
--- a/pypy/module/_ffi/interp_funcptr.py
+++ b/pypy/module/_ffi/interp_funcptr.py
@@ -15,6 +15,51 @@
from pypy.rlib.objectmodel import we_are_translated
from pypy.module._ffi.type_converter import FromAppLevelConverter, ToAppLevelConverter
+import os
+if os.name == 'nt':
+ def _getfunc(space, CDLL, w_name, w_argtypes, w_restype):
+ argtypes_w, argtypes, w_restype, restype = unpack_argtypes(
+ space, w_argtypes, w_restype)
+ if space.isinstance_w(w_name, space.w_str):
+ name = space.str_w(w_name)
+ try:
+ func = CDLL.cdll.getpointer(name, argtypes, restype,
+ flags = CDLL.flags)
+ except KeyError:
+ raise operationerrfmt(
+ space.w_AttributeError,
+ "No symbol %s found in library %s", name, CDLL.name)
+
+ return W_FuncPtr(func, argtypes_w, w_restype)
+ elif space.isinstance_w(w_name, space.w_int):
+ ordinal = space.int_w(w_name)
+ try:
+ func = CDLL.cdll.getpointer_by_ordinal(
+ ordinal, argtypes, restype,
+ flags = CDLL.flags)
+ except KeyError:
+ raise operationerrfmt(
+ space.w_AttributeError,
+ "No ordinal %d found in library %s", ordinal, CDLL.name)
+ return W_FuncPtr(func, argtypes_w, w_restype)
+ else:
+ raise OperationError(space.w_TypeError, space.wrap(
+ 'function name must be a string or integer'))
+else:
+ @unwrap_spec(name=str)
+ def _getfunc(space, CDLL, w_name, w_argtypes, w_restype):
+ name = space.str_w(w_name)
+ argtypes_w, argtypes, w_restype, restype = unpack_argtypes(
+ space, w_argtypes, w_restype)
+ try:
+ func = CDLL.cdll.getpointer(name, argtypes, restype,
+ flags = CDLL.flags)
+ except KeyError:
+ raise operationerrfmt(
+ space.w_AttributeError,
+ "No symbol %s found in library %s", name, CDLL.name)
+
+ return W_FuncPtr(func, argtypes_w, w_restype)
def unwrap_ffitype(space, w_argtype, allow_void=False):
res = w_argtype.get_ffitype()
@@ -271,19 +316,8 @@
raise operationerrfmt(space.w_OSError, '%s: %s', self.name,
e.msg or 'unspecified error')
- @unwrap_spec(name=str)
- def getfunc(self, space, name, w_argtypes, w_restype):
- argtypes_w, argtypes, w_restype, restype = unpack_argtypes(space,
- w_argtypes,
- w_restype)
- try:
- func = self.cdll.getpointer(name, argtypes, restype,
- flags = self.flags)
- except KeyError:
- raise operationerrfmt(space.w_AttributeError,
- "No symbol %s found in library %s", name, self.name)
-
- return W_FuncPtr(func, argtypes_w, w_restype)
+ def getfunc(self, space, w_name, w_argtypes, w_restype):
+ return _getfunc(space, self, w_name, w_argtypes, w_restype)
@unwrap_spec(name=str)
def getaddressindll(self, space, name):
@@ -291,8 +325,9 @@
address_as_uint = rffi.cast(lltype.Unsigned,
self.cdll.getaddressindll(name))
except KeyError:
- raise operationerrfmt(space.w_ValueError,
- "No symbol %s found in library %s", name, self.name)
+ raise operationerrfmt(
+ space.w_ValueError,
+ "No symbol %s found in library %s", name, self.name)
return space.wrap(address_as_uint)
@unwrap_spec(name='str_or_None', mode=int)
diff --git a/pypy/module/_ffi/interp_struct.py b/pypy/module/_ffi/interp_struct.py
--- a/pypy/module/_ffi/interp_struct.py
+++ b/pypy/module/_ffi/interp_struct.py
@@ -56,8 +56,7 @@
class W__StructDescr(Wrappable):
- def __init__(self, space, name):
- self.space = space
+ def __init__(self, name):
self.w_ffitype = W_FFIType('struct %s' % name, clibffi.FFI_TYPE_NULL,
w_structdescr=self)
self.fields_w = None
@@ -69,7 +68,6 @@
raise operationerrfmt(space.w_ValueError,
"%s's fields has already been defined",
self.w_ffitype.name)
- space = self.space
fields_w = space.fixedview(w_fields)
# note that the fields_w returned by compute_size_and_alignement has a
# different annotation than the original: list(W_Root) vs list(W_Field)
@@ -104,11 +102,11 @@
return W__StructInstance(self, allocate=False, autofree=True, rawmem=rawmem)
@jit.elidable_promote('0')
- def get_type_and_offset_for_field(self, name):
+ def get_type_and_offset_for_field(self, space, name):
try:
w_field = self.name2w_field[name]
except KeyError:
- raise operationerrfmt(self.space.w_AttributeError, '%s', name)
+ raise operationerrfmt(space.w_AttributeError, '%s', name)
return w_field.w_ffitype, w_field.offset
@@ -116,7 +114,7 @@
@unwrap_spec(name=str)
def descr_new_structdescr(space, w_type, name, w_fields=None):
- descr = W__StructDescr(space, name)
+ descr = W__StructDescr(name)
if w_fields is not space.w_None:
descr.define_fields(space, w_fields)
return descr
@@ -185,13 +183,15 @@
@unwrap_spec(name=str)
def getfield(self, space, name):
- w_ffitype, offset = self.structdescr.get_type_and_offset_for_field(name)
+ w_ffitype, offset = self.structdescr.get_type_and_offset_for_field(
+ space, name)
field_getter = GetFieldConverter(space, self.rawmem, offset)
return field_getter.do_and_wrap(w_ffitype)
@unwrap_spec(name=str)
def setfield(self, space, name, w_value):
- w_ffitype, offset = self.structdescr.get_type_and_offset_for_field(name)
+ w_ffitype, offset = self.structdescr.get_type_and_offset_for_field(
+ space, name)
field_setter = SetFieldConverter(space, self.rawmem, offset)
field_setter.unwrap_and_do(w_ffitype, w_value)
diff --git a/pypy/module/_ffi/test/test_funcptr.py b/pypy/module/_ffi/test/test_funcptr.py
--- a/pypy/module/_ffi/test/test_funcptr.py
+++ b/pypy/module/_ffi/test/test_funcptr.py
@@ -632,4 +632,17 @@
types.void, FUNCFLAG_STDCALL)
sleep(10)
-
+ def test_by_ordinal(self):
+ if not self.iswin32:
+ skip("windows specific")
+ """
+ int DLLEXPORT AAA_first_ordinal_function()
+ {
+ return 42;
+ }
+ """
+ from _ffi import CDLL, types
+ libfoo = CDLL(self.libfoo_name)
+ f_name = libfoo.getfunc('AAA_first_ordinal_function', [], types.sint)
+ f_ordinal = libfoo.getfunc(1, [], types.sint)
+ assert f_name.getaddr() == f_ordinal.getaddr()
diff --git a/pypy/module/_ffi/test/test_ztranslation.py b/pypy/module/_ffi/test/test_ztranslation.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ffi/test/test_ztranslation.py
@@ -0,0 +1,4 @@
+from pypy.objspace.fake.checkmodule import checkmodule
+
+def test__ffi_translates():
+ checkmodule('_ffi', '_rawffi')
diff --git a/pypy/module/_ssl/__init__.py b/pypy/module/_ssl/__init__.py
--- a/pypy/module/_ssl/__init__.py
+++ b/pypy/module/_ssl/__init__.py
@@ -32,5 +32,6 @@
def startup(self, space):
from pypy.rlib.ropenssl import init_ssl
init_ssl()
- from pypy.module._ssl.interp_ssl import setup_ssl_threads
- setup_ssl_threads()
+ if space.config.objspace.usemodules.thread:
+ from pypy.module._ssl.thread_lock import setup_ssl_threads
+ setup_ssl_threads()
diff --git a/pypy/module/_ssl/interp_ssl.py b/pypy/module/_ssl/interp_ssl.py
--- a/pypy/module/_ssl/interp_ssl.py
+++ b/pypy/module/_ssl/interp_ssl.py
@@ -983,38 +983,3 @@
libssl_X509_free(x)
finally:
libssl_BIO_free(cert)
-
-# this function is needed to perform locking on shared data
-# structures. (Note that OpenSSL uses a number of global data
-# structures that will be implicitly shared whenever multiple threads
-# use OpenSSL.) Multi-threaded applications will crash at random if
-# it is not set.
-#
-# locking_function() must be able to handle up to CRYPTO_num_locks()
-# different mutex locks. It sets the n-th lock if mode & CRYPTO_LOCK, and
-# releases it otherwise.
-#
-# filename and line are the file number of the function setting the
-# lock. They can be useful for debugging.
-_ssl_locks = []
-
-def _ssl_thread_locking_function(mode, n, filename, line):
- n = intmask(n)
- if n < 0 or n >= len(_ssl_locks):
- return
-
- if intmask(mode) & CRYPTO_LOCK:
- _ssl_locks[n].acquire(True)
- else:
- _ssl_locks[n].release()
-
-def _ssl_thread_id_function():
- from pypy.module.thread import ll_thread
- return rffi.cast(rffi.LONG, ll_thread.get_ident())
-
-def setup_ssl_threads():
- from pypy.module.thread import ll_thread
- for i in range(libssl_CRYPTO_num_locks()):
- _ssl_locks.append(ll_thread.allocate_lock())
- libssl_CRYPTO_set_locking_callback(_ssl_thread_locking_function)
- libssl_CRYPTO_set_id_callback(_ssl_thread_id_function)
diff --git a/pypy/module/_ssl/thread_lock.py b/pypy/module/_ssl/thread_lock.py
new file mode 100644
--- /dev/null
+++ b/pypy/module/_ssl/thread_lock.py
@@ -0,0 +1,78 @@
+from pypy.rlib.ropenssl import *
+from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.translator.tool.cbuild import ExternalCompilationInfo
+
+# CRYPTO_set_locking_callback:
+#
+# this function is needed to perform locking on shared data
+# structures. (Note that OpenSSL uses a number of global data
+# structures that will be implicitly shared whenever multiple threads
+# use OpenSSL.) Multi-threaded applications will crash at random if
+# it is not set.
+#
+# locking_function() must be able to handle up to CRYPTO_num_locks()
+# different mutex locks. It sets the n-th lock if mode & CRYPTO_LOCK, and
+# releases it otherwise.
+#
+# filename and line are the file number of the function setting the
+# lock. They can be useful for debugging.
+
+
+# This logic is moved to C code so that the callbacks can be invoked
+# without caring about the GIL.
+
+separate_module_source = """
+
+#include <openssl/crypto.h>
+
+static unsigned int _ssl_locks_count = 0;
+static struct RPyOpaque_ThreadLock *_ssl_locks;
+
+static unsigned long _ssl_thread_id_function(void) {
+ return RPyThreadGetIdent();
+}
+
+static void _ssl_thread_locking_function(int mode, int n, const char *file,
+ int line) {
+ if ((_ssl_locks == NULL) ||
+ (n < 0) || ((unsigned)n >= _ssl_locks_count))
+ return;
+
+ if (mode & CRYPTO_LOCK) {
+ RPyThreadAcquireLock(_ssl_locks + n, 1);
+ } else {
+ RPyThreadReleaseLock(_ssl_locks + n);
+ }
+}
+
+int _PyPy_SSL_SetupThreads(void)
+{
+ unsigned int i;
+ _ssl_locks_count = CRYPTO_num_locks();
+ _ssl_locks = calloc(_ssl_locks_count, sizeof(struct RPyOpaque_ThreadLock));
+ if (_ssl_locks == NULL)
+ return 0;
+ for (i=0; i<_ssl_locks_count; i++) {
+ if (RPyThreadLockInit(_ssl_locks + i) == 0)
+ return 0;
+ }
+ CRYPTO_set_locking_callback(_ssl_thread_locking_function);
+ CRYPTO_set_id_callback(_ssl_thread_id_function);
+ return 1;
+}
+"""
+
+
+eci = ExternalCompilationInfo(
+ separate_module_sources=[separate_module_source],
+ export_symbols=['_PyPy_SSL_SetupThreads'],
+)
+
+_PyPy_SSL_SetupThreads = rffi.llexternal('_PyPy_SSL_SetupThreads',
+ [], rffi.INT,
+ compilation_info=eci)
+
+def setup_ssl_threads():
+ result = _PyPy_SSL_SetupThreads()
+ if rffi.cast(lltype.Signed, result) == 0:
+ raise MemoryError
diff --git a/pypy/module/imp/importing.py b/pypy/module/imp/importing.py
--- a/pypy/module/imp/importing.py
+++ b/pypy/module/imp/importing.py
@@ -263,7 +263,7 @@
w_mod = check_sys_modules_w(space, rel_modulename)
if w_mod is not None and space.is_w(w_mod, space.w_None):
# if we already find space.w_None, it means that we
- # already tried and failed and falled back to the
+ # already tried and failed and fell back to the
# end of this function.
w_mod = None
else:
@@ -283,10 +283,16 @@
return w_mod
def absolute_import(space, modulename, baselevel, fromlist_w, tentative):
- # Short path: check in sys.modules
- w_mod = absolute_import_try(space, modulename, baselevel, fromlist_w)
- if w_mod is not None and not space.is_w(w_mod, space.w_None):
- return w_mod
+ # Short path: check in sys.modules, but only if there is no conflict
+ # on the import lock. In the situation of 'import' statements
+ # inside tight loops, this should be true, and absolute_import_try()
+ # should be followed by the JIT and turned into not much code. But
+ # if the import lock is currently held by another thread, then we
+ # have to wait, and so shouldn't use the fast path.
+ if not getimportlock(space).lock_held_by_someone_else():
+ w_mod = absolute_import_try(space, modulename, baselevel, fromlist_w)
+ if w_mod is not None and not space.is_w(w_mod, space.w_None):
+ return w_mod
return absolute_import_with_lock(space, modulename, baselevel,
fromlist_w, tentative)
@@ -741,6 +747,9 @@
self.lockowner = None
self.lockcounter = 0
+ def lock_held_by_someone_else(self):
+ return self.lockowner is not None and not self.lock_held()
+
def lock_held(self):
me = self.space.getexecutioncontext() # used as thread ident
return self.lockowner is me
diff --git a/pypy/module/imp/test/test_import.py b/pypy/module/imp/test/test_import.py
--- a/pypy/module/imp/test/test_import.py
+++ b/pypy/module/imp/test/test_import.py
@@ -1272,3 +1272,45 @@
"objspace.usepycfiles": True,
"objspace.lonepycfiles": True
}
+
+
+class AppTestMultithreadedImp(object):
+ def setup_class(cls):
+ #if not conftest.option.runappdirect:
+ # py.test.skip("meant as an -A test")
+ cls.space = gettestobjspace(usemodules=['thread', 'time'])
+ tmpfile = udir.join('test_multithreaded_imp.py')
+ tmpfile.write('''if 1:
+ x = 666
+ import time
+ for i in range(1000): time.sleep(0.001)
+ x = 42
+ ''')
+ cls.w_tmppath = cls.space.wrap(str(udir))
+
+ def test_multithreaded_import(self):
+ import sys, thread, time
+ oldpath = sys.path[:]
+ try:
+ sys.path.insert(0, self.tmppath)
+ got = []
+
+ def check():
+ import test_multithreaded_imp
+ got.append(getattr(test_multithreaded_imp, 'x', '?'))
+
+ for i in range(5):
+ thread.start_new_thread(check, ())
+
+ for n in range(100):
+ for i in range(105): time.sleep(0.001)
+ if len(got) == 5:
+ break
+ else:
+ raise AssertionError("got %r so far but still waiting" %
+ (got,))
+
+ assert got == [42] * 5, got
+
+ finally:
+ sys.path[:] = oldpath
diff --git a/pypy/module/posix/app_posix.py b/pypy/module/posix/app_posix.py
--- a/pypy/module/posix/app_posix.py
+++ b/pypy/module/posix/app_posix.py
@@ -64,15 +64,10 @@
self.__dict__['st_ctime'] = self[9]
if osname == 'posix':
- def _validate_fd(fd):
- try:
- import fcntl
- except ImportError:
- return
- try:
- fcntl.fcntl(fd, fcntl.F_GETFD)
- except IOError as e:
- raise OSError(e.errno, e.strerror, e.filename)
+ # POSIX: we want to check the file descriptor when fdopen() is called,
+ # not later when we read or write data. So we call fstat(), letting
+ # it raise if fd is invalid.
+ _validate_fd = posix.fstat
else:
_validate_fd = validate_fd
diff --git a/pypy/module/signal/interp_signal.py b/pypy/module/signal/interp_signal.py
--- a/pypy/module/signal/interp_signal.py
+++ b/pypy/module/signal/interp_signal.py
@@ -148,28 +148,27 @@
n = pypysig_poll()
if n < 0:
break
- if self.reissue_signal_action is None:
- # no threads: we can report the signal immediately
+ self.perform_signal(executioncontext, n)
+
+ def perform_signal(self, executioncontext, n):
+ if self.reissue_signal_action is None:
+ # no threads: we can report the signal immediately
+ self.report_signal(n)
+ else:
+ main_ec = self.space.threadlocals.getmainthreadvalue()
+ if executioncontext is main_ec:
+ # running in the main thread: we can report the
+ # signal immediately
self.report_signal(n)
else:
- main_ec = self.space.threadlocals.getmainthreadvalue()
- if executioncontext is main_ec:
- # running in the main thread: we can report the
- # signal immediately
- self.report_signal(n)
- else:
- # running in another thread: we need to hack a bit
- self.pending_signals[n] = None
- self.reissue_signal_action.fire_after_thread_switch()
+ # running in another thread: we need to hack a bit
+ self.pending_signals[n] = None
+ self.reissue_signal_action.fire_after_thread_switch()
def set_interrupt(self):
"Simulates the effect of a SIGINT signal arriving"
- n = cpy_signal.SIGINT
- if self.reissue_signal_action is None:
- self.report_signal(n)
- else:
- self.pending_signals[n] = None
- self.reissue_signal_action.fire_after_thread_switch()
+ ec = self.space.getexecutioncontext()
+ self.perform_signal(ec, cpy_signal.SIGINT)
def report_signal(self, n):
try:
diff --git a/pypy/objspace/fake/checkmodule.py b/pypy/objspace/fake/checkmodule.py
--- a/pypy/objspace/fake/checkmodule.py
+++ b/pypy/objspace/fake/checkmodule.py
@@ -2,13 +2,14 @@
from pypy.config.pypyoption import get_pypy_config
-def checkmodule(modname):
+def checkmodule(*modnames):
config = get_pypy_config(translating=True)
space = FakeObjSpace(config)
- mod = __import__('pypy.module.%s' % modname, None, None, ['__doc__'])
- # force computation and record what we wrap
- module = mod.Module(space, W_Root())
- for name in module.loaders:
- module._load_lazily(space, name)
+ for modname in modnames:
+ mod = __import__('pypy.module.%s' % modname, None, None, ['__doc__'])
+ # force computation and record what we wrap
+ module = mod.Module(space, W_Root())
+ for name in module.loaders:
+ module._load_lazily(space, name)
#
space.translates(**{'translation.list_comprehension_operations':True})
diff --git a/pypy/rlib/libffi.py b/pypy/rlib/libffi.py
--- a/pypy/rlib/libffi.py
+++ b/pypy/rlib/libffi.py
@@ -415,6 +415,11 @@
return Func(name, argtypes, restype, dlsym(self.lib, name),
flags=flags, keepalive=self)
+ def getpointer_by_ordinal(self, name, argtypes, restype,
+ flags=FUNCFLAG_CDECL):
+ return Func('by_ordinal', argtypes, restype,
+ dlsym_byordinal(self.lib, name),
+ flags=flags, keepalive=self)
def getaddressindll(self, name):
return dlsym(self.lib, name)
@@ -423,6 +428,11 @@
def getpointer(self, name, argtypes, restype, flags=FUNCFLAG_STDCALL):
return Func(name, argtypes, restype, dlsym(self.lib, name),
flags=flags, keepalive=self)
+ def getpointer_by_ordinal(self, name, argtypes, restype,
+ flags=FUNCFLAG_STDCALL):
+ return Func(name, argtypes, restype, dlsym_byordinal(self.lib, name),
+ flags=flags, keepalive=self)
+
# ======================================================================
@jit.oopspec('libffi_struct_getfield(ffitype, addr, offset)')
diff --git a/pypy/rlib/test/test_libffi.py b/pypy/rlib/test/test_libffi.py
--- a/pypy/rlib/test/test_libffi.py
+++ b/pypy/rlib/test/test_libffi.py
@@ -222,7 +222,7 @@
if meth.__doc__ is not None and '{' in meth.__doc__:
snippets.append(meth.__doc__)
import re
- for match in re.finditer(" ([a-z_]+)\(", meth.__doc__):
+ for match in re.finditer(" ([A-Za-z_]+)\(", meth.__doc__):
exports.append(match.group(1))
#
c_file.write(STANDARD_DEFINES + str(py.code.Source('\n'.join(snippets))))
@@ -557,10 +557,10 @@
if os.name == 'nt':
def test_stdcall_simple(self):
"""
- int __stdcall std_diff_xy(int x, Signed y)
- {
- return x - y;
- }
+ int __stdcall std_diff_xy(int x, Signed y)
+ {
+ return x - y;
+ }
"""
libfoo = self.get_libfoo()
func = (libfoo, 'std_diff_xy', [types.sint, types.signed], types.sint)
@@ -575,5 +575,36 @@
else:
assert 0, 'wrong calling convention should have raised'
+ def test_by_ordinal(self):
+ """
+ int AAA_first_ordinal_function()
+ {
+ return 42;
+ }
+ """
+ libfoo = self.get_libfoo()
+ f_by_name = libfoo.getpointer('AAA_first_ordinal_function' ,[],
+ types.uint)
+ f_by_ordinal = libfoo.getpointer_by_ordinal(1 ,[], types.uint)
+ print dir(f_by_name)
+ assert f_by_name.funcsym == f_by_ordinal.funcsym
+
+ def test_by_ordinal2(self):
+ """
+ int __stdcall BBB_second_ordinal_function()
+ {
+ return 24;
+ }
+ """
+ from pypy.rlib.libffi import WinDLL
+ dll = WinDLL(self.libfoo_name)
+ f_by_name = dll.getpointer('BBB_second_ordinal_function' ,[],
+ types.uint)
+ f_by_ordinal = dll.getpointer_by_ordinal(2 ,[], types.uint)
+ print dir(f_by_name)
+ assert f_by_name.funcsym == f_by_ordinal.funcsym
+ chain = ArgChain()
+ assert 24 == f_by_ordinal.call(chain, lltype.Signed, is_struct=False)
+
diff --git a/pypy/rpython/module/ll_os.py b/pypy/rpython/module/ll_os.py
--- a/pypy/rpython/module/ll_os.py
+++ b/pypy/rpython/module/ll_os.py
@@ -1362,7 +1362,8 @@
os_isatty = self.llexternal(underscore_on_windows+'isatty', [rffi.INT], rffi.INT)
def isatty_llimpl(fd):
- rposix.validate_fd(fd)
+ if not rposix.is_valid_fd(fd):
+ return False
res = rffi.cast(lltype.Signed, os_isatty(rffi.cast(rffi.INT, fd)))
return res != 0
diff --git a/pypy/rpython/module/test/test_ll_os.py b/pypy/rpython/module/test/test_ll_os.py
--- a/pypy/rpython/module/test/test_ll_os.py
+++ b/pypy/rpython/module/test/test_ll_os.py
@@ -268,6 +268,14 @@
expected = -signal.SIGTERM
assert proc.wait() == expected
+def test_isatty():
+ try:
+ f = getllimpl(os.isatty)
+ except:
+ skip('No isatty in os')
+ assert f(-1) == False
+
+
class ExpectTestOs:
def setup_class(cls):
if not hasattr(os, 'ttyname'):
diff --git a/pypy/translator/c/funcgen.py b/pypy/translator/c/funcgen.py
--- a/pypy/translator/c/funcgen.py
+++ b/pypy/translator/c/funcgen.py
@@ -214,6 +214,10 @@
myblocknum = self.blocknum[block]
yield ''
yield 'block%d:' % myblocknum
+ if block in self.innerloops:
+ for line in self.gen_while_loop_hack(block):
+ yield line
+ continue
for i, op in enumerate(block.operations):
for line in self.gen_op(op):
yield line
@@ -236,9 +240,6 @@
assert len(block.exits) == 1
for op in self.gen_link(block.exits[0]):
yield op
- elif block in self.innerloops:
- for line in self.gen_while_loop_hack(block):
- yield line
else:
assert block.exitswitch != c_last_exception
# block ending in a switch on a value
@@ -341,11 +342,11 @@
# decision is) we produce code like this:
#
# headblock:
- # ...headblock operations...
- # while (cond) {
+ # while (1) {
+ # ...headblock operations...
+ # if (!cond) break;
# goto firstbodyblock;
- # headblock_back:
- # ...headblock operations...
+ # headblock_back: ;
# }
#
# The real body of the loop is not syntactically within the
@@ -366,19 +367,19 @@
i = list(headblock.exits).index(enterlink)
exitlink = headblock.exits[1 - i]
+ yield 'while (1) {'
+
+ for i, op in enumerate(headblock.operations):
+ for line in self.gen_op(op):
+ yield '\t' + line
+
expr = self.expr(headblock.exitswitch)
- if enterlink.exitcase == False:
+ if enterlink.exitcase == True:
expr = '!' + expr
- yield 'while (%s) {' % expr
+ yield '\tif (%s) break;' % expr
for op in self.gen_link(enterlink):
yield '\t' + op
- # the semicolon after the colon is needed in case no operation
- # produces any code after the label
- yield '\t block%d_back: ;' % self.blocknum[headblock]
- if headblock.operations:
- for i, op in enumerate(headblock.operations):
- for line in self.gen_op(op):
- yield '\t' + line
+ yield ' block%d_back: ;' % self.blocknum[headblock]
yield '}'
for op in self.gen_link(exitlink):
yield op
diff --git a/pypy/translator/c/gcc/trackgcroot.py b/pypy/translator/c/gcc/trackgcroot.py
--- a/pypy/translator/c/gcc/trackgcroot.py
+++ b/pypy/translator/c/gcc/trackgcroot.py
@@ -476,15 +476,17 @@
# floating-point operations cannot produce GC pointers
'f',
'cvt', 'ucomi', 'comi', 'subs', 'subp' , 'adds', 'addp', 'xorp',
- 'movap', 'movd', 'movlp', 'sqrtsd', 'movhpd',
+ 'movap', 'movd', 'movlp', 'sqrt', 'rsqrt', 'movhpd',
'mins', 'minp', 'maxs', 'maxp', 'unpck', 'pxor', 'por', # sse2
'shufps', 'shufpd',
# arithmetic operations should not produce GC pointers
'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
- 'bswap', 'bt', 'rdtsc',
- 'punpck', 'pshufd', 'pcmp', 'pand', 'psllw', 'pslld', 'psllq',
- 'paddq', 'pinsr', 'pmul', 'psrl',
+ 'bswap', 'bt', 'rdtsc', 'rounds',
+ 'pabs', 'pack', 'padd', 'palign', 'pand', 'pavg', 'pcmp', 'pextr',
+ 'phadd', 'phsub', 'pinsr', 'pmadd', 'pmax', 'pmin', 'pmovmsk',
+ 'pmul', 'por', 'psadb', 'pshuf', 'psign', 'psll', 'psra', 'psrl',
+ 'psub', 'punpck', 'pxor',
# all vectors don't produce pointers
'v',
# sign-extending moves should not produce GC pointers
@@ -492,7 +494,7 @@
# zero-extending moves should not produce GC pointers
'movz',
# locked operations should not move GC pointers, at least so far
- 'lock',
+ 'lock', 'pause',
])
# a partial list is hopefully good enough for now; it's all to support
diff --git a/pypy/translator/c/src/thread_nt.h b/pypy/translator/c/src/thread_nt.h
--- a/pypy/translator/c/src/thread_nt.h
+++ b/pypy/translator/c/src/thread_nt.h
@@ -31,6 +31,7 @@
VOID DeleteNonRecursiveMutex(PNRMUTEX mutex);
DWORD EnterNonRecursiveMutex(PNRMUTEX mutex, DWORD milliseconds);
BOOL LeaveNonRecursiveMutex(PNRMUTEX mutex);
+int RPyThreadLockInit(struct RPyOpaque_ThreadLock *lock);
void RPyOpaqueDealloc_ThreadLock(struct RPyOpaque_ThreadLock *lock);
int RPyThreadAcquireLock(struct RPyOpaque_ThreadLock *lock, int waitflag);
RPyLockStatus RPyThreadAcquireLockTimed(struct RPyOpaque_ThreadLock *lock,
@@ -151,7 +152,7 @@
{
}
-int RPyThreadLockInit(struct RPyOpaque_ThreadLock * lock)
+int RPyThreadLockInit(struct RPyOpaque_ThreadLock *lock)
{
return InitializeNonRecursiveMutex(lock);
}
diff --git a/pypy/translator/goal/app_main.py b/pypy/translator/goal/app_main.py
--- a/pypy/translator/goal/app_main.py
+++ b/pypy/translator/goal/app_main.py
@@ -252,6 +252,24 @@
sys.stderr = sys.__stderr__ = create_stdio(
2, True, "<stderr>", encoding, errors, unbuffered)
+def set_stdio_encodings(ignore_environment):
+ import os
+ readenv = not ignore_environment
+ io_encoding = readenv and os.getenv("PYTHONIOENCODING")
+ if io_encoding:
+ errors = None
+ if ":" in io_encoding:
+ io_encoding, errors = io_encoding.split(":", 1)
+ set_io_encoding(io_encoding, io_encoding, errors, True)
+ else:
+ if IS_WINDOWS:
+ import __pypy__
+ io_encoding, io_encoding_output = __pypy__.get_console_cp()
+ else:
+ io_encoding = io_encoding_output = sys.getfilesystemencoding()
+ if io_encoding:
+ set_io_encoding(io_encoding, io_encoding_output, None, False)
+
def create_stdio(fd, writing, name, encoding, errors, unbuffered):
import io
@@ -468,13 +486,13 @@
if PYTHON26 and not options["ignore_environment"]:
if os.getenv('PYTHONNOUSERSITE'):
- options["no_user_site"] = True
+ options["no_user_site"] = 1
if os.getenv('PYTHONDONTWRITEBYTECODE'):
- options["dont_write_bytecode"] = True
+ options["dont_write_bytecode"] = 1
if (options["interactive"] or
(not options["ignore_environment"] and os.getenv('PYTHONINSPECT'))):
- options["inspect"] = True
+ options["inspect"] = 1
if PYTHON26 and we_are_translated():
flags = [options[flag] for flag in sys_flags]
@@ -527,22 +545,9 @@
except:
print("'import site' failed", file=sys.stderr)
+ set_stdio_encodings(ignore_environment)
+
readenv = not ignore_environment
- io_encoding = readenv and os.getenv("PYTHONIOENCODING")
- if io_encoding:
- errors = None
- if ":" in io_encoding:
- io_encoding, errors = io_encoding.split(":", 1)
- set_io_encoding(io_encoding, io_encoding, errors, True)
- else:
- if IS_WINDOWS:
- import __pypy__
- io_encoding, io_encoding_output = __pypy__.get_console_cp()
- else:
- io_encoding = io_encoding_output = sys.getfilesystemencoding()
- if io_encoding:
- set_io_encoding(io_encoding, io_encoding_output, None, False)
-
pythonwarnings = readenv and os.getenv('PYTHONWARNINGS')
if pythonwarnings:
warnoptions.extend(pythonwarnings.split(','))
More information about the pypy-commit
mailing list