[pypy-commit] pypy default: merged upstream
alex_gaynor
noreply at buildbot.pypy.org
Thu Feb 9 15:13:30 CET 2012
Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch:
Changeset: r52294:3478844da220
Date: 2012-02-09 09:12 -0500
http://bitbucket.org/pypy/pypy/changeset/3478844da220/
Log: merged upstream
diff --git a/lib_pypy/numpypy/core/numeric.py b/lib_pypy/numpypy/core/numeric.py
--- a/lib_pypy/numpypy/core/numeric.py
+++ b/lib_pypy/numpypy/core/numeric.py
@@ -1,6 +1,7 @@
from _numpypy import array, ndarray, int_, float_, bool_ #, complex_# , longlong
from _numpypy import concatenate
+import math
import sys
import _numpypy as multiarray # ARGH
from numpypy.core.arrayprint import array2string
@@ -311,6 +312,11 @@
little_endian = (sys.byteorder == 'little')
Inf = inf = infty = Infinity = PINF = float('inf')
+NINF = float('-inf')
+PZERO = 0.0
+NZERO = -0.0
nan = NaN = NAN = float('nan')
False_ = bool_(False)
True_ = bool_(True)
+e = math.e
+pi = math.pi
\ No newline at end of file
diff --git a/pypy/doc/jit-hooks.rst b/pypy/doc/jit-hooks.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/jit-hooks.rst
@@ -0,0 +1,66 @@
+JIT hooks in PyPy
+=================
+
+There are several hooks in the `pypyjit` module that may help you with
+understanding what's pypy's JIT doing while running your program. There
+are three functions related to that coming from the `pypyjit` module:
+
+* `set_optimize_hook`::
+
+ Set a compiling hook that will be called each time a loop is optimized,
+ but before assembler compilation. This allows to add additional
+ optimizations on Python level.
+
+ The hook will be called with the following signature:
+ hook(jitdriver_name, loop_type, greenkey or guard_number, operations)
+
+ jitdriver_name is the name of this particular jitdriver, 'pypyjit' is
+ the main interpreter loop
+
+ loop_type can be either `loop` `entry_bridge` or `bridge`
+ in case loop is not `bridge`, greenkey will be a tuple of constants
+ or a string describing it.
+
+ for the interpreter loop` it'll be a tuple
+ (code, offset, is_being_profiled)
+
+ Note that jit hook is not reentrant. It means that if the code
+ inside the jit hook is itself jitted, it will get compiled, but the
+ jit hook won't be called for that.
+
+ Result value will be the resulting list of operations, or None
+
+* `set_compile_hook`::
+
+ Set a compiling hook that will be called each time a loop is compiled.
+ The hook will be called with the following signature:
+ hook(jitdriver_name, loop_type, greenkey or guard_number, operations,
+ assembler_addr, assembler_length)
+
+ jitdriver_name is the name of this particular jitdriver, 'pypyjit' is
+ the main interpreter loop
+
+ loop_type can be either `loop` `entry_bridge` or `bridge`
+ in case loop is not `bridge`, greenkey will be a tuple of constants
+ or a string describing it.
+
+ for the interpreter loop` it'll be a tuple
+ (code, offset, is_being_profiled)
+
+ assembler_addr is an integer describing where assembler starts,
+ can be accessed via ctypes, assembler_lenght is the lenght of compiled
+ asm
+
+ Note that jit hook is not reentrant. It means that if the code
+ inside the jit hook is itself jitted, it will get compiled, but the
+ jit hook won't be called for that.
+
+* `set_abort_hook`::
+
+ Set a hook (callable) that will be called each time there is tracing
+ aborted due to some reason.
+
+ The hook will be called as in: hook(jitdriver_name, greenkey, reason)
+
+ Where reason is the reason for abort, see documentation for set_compile_hook
+ for descriptions of other arguments.
diff --git a/pypy/doc/jit/index.rst b/pypy/doc/jit/index.rst
--- a/pypy/doc/jit/index.rst
+++ b/pypy/doc/jit/index.rst
@@ -21,6 +21,9 @@
- Notes_ about the current work in PyPy
+- Hooks_ debugging facilities available to a python programmer
+
.. _Overview: overview.html
.. _Notes: pyjitpl5.html
+.. _Hooks: ../jit-hooks.html
diff --git a/pypy/doc/release-1.8.0.rst b/pypy/doc/release-1.8.0.rst
--- a/pypy/doc/release-1.8.0.rst
+++ b/pypy/doc/release-1.8.0.rst
@@ -2,11 +2,11 @@
PyPy 1.8 - business as usual
============================
-We're pleased to announce the 1.8 release of PyPy. As became a habit, this
-release brings a lot of bugfixes, performance and memory improvements over
+We're pleased to announce the 1.8 release of PyPy. As has become a habit, this
+release brings a lot of bugfixes, and performance and memory improvements over
the 1.7 release. The main highlight of the release is the introduction of
list strategies which makes homogenous lists more efficient both in terms
-of performance and memory. Otherwise it's "business as usual" in the sense
+of performance and memory. This release also upgrades us from Python 2.7.1 compatibility to 2.7.2. Otherwise it's "business as usual" in the sense
that performance improved roughly 10% on average since the previous release.
You can download the PyPy 1.8 release here:
@@ -20,7 +20,8 @@
due to its integrated tracing JIT compiler.
This release supports x86 machines running Linux 32/64, Mac OS X 32/64 or
-Windows 32. Windows 64 work is ongoing, but not yet natively supported.
+Windows 32. Windows 64 work has been stalled, we would welcome a volunteer
+to handle that.
.. _`pypy 1.8 and cpython 2.7.1`: http://speed.pypy.org
@@ -34,7 +35,7 @@
strategies for unicode and string lists.
* As usual, numerous performance improvements. There are many examples
- of python constructs that now should behave faster; too many to list them.
+ of python constructs that now should be faster; too many to list them.
* Bugfixes and compatibility fixes with CPython.
@@ -52,11 +53,40 @@
* a lot of other minor changes
+ Right now the `numpy` module is available under both `numpy` and `numpypy`
+ names. However, because it's incomplete, you have to `import numpypy` first
+ before doing any imports from `numpy`.
+
+* New JIT hooks that allow you to hook into the JIT process from your python
+ program. There is a `brief overview`_ of what they offer.
+
* Since the last release there was a significant breakthrough in PyPy's
fundraising. We now have enough funds to work on first stages of `numpypy`_
- and `py3k`_
+ and `py3k`_. We would like to thank again to everyone who donated.
+ It's also probably worth noting, we're considering donations for the STM
+ project.
+
+* Standard library upgrade from 2.7.1 to 2.7.2.
+
+Ongoing work
+============
+
+As usual, there is quite a bit of ongoing work that either didn't make it to
+the release or is not ready yet. Highlights include:
+
+* Non-x86 backends for the JIT: ARMv7 (almost ready) and PPC64 (in progress)
+
+* Specialized type instances - allocate instances as efficient as C structs,
+ including type specialization
+
+* More numpy work
+
+* Software Transactional Memory, you can read more about `our plans`_
+
+.. _`brief overview`: http://doc.pypy.org/en/latest/jit-hooks.html
.. _`numpy status page`: http://buildbot.pypy.org/numpy-status/latest.html
.. _`numpy status update blog report`: http://morepypy.blogspot.com/2012/01/numpypy-status-update.html
.. _`numpypy`: http://pypy.org/numpydonate.html
.. _`py3k`: http://pypy.org/py3donate.html
+.. _`our plans`: http://morepypy.blogspot.com/2012/01/transactional-memory-ii.html
diff --git a/pypy/module/micronumpy/__init__.py b/pypy/module/micronumpy/__init__.py
--- a/pypy/module/micronumpy/__init__.py
+++ b/pypy/module/micronumpy/__init__.py
@@ -31,7 +31,7 @@
'concatenate': 'interp_numarray.concatenate',
'set_string_function': 'appbridge.set_string_function',
-
+
'count_reduce_items': 'interp_numarray.count_reduce_items',
'True_': 'types.Bool.True',
@@ -111,8 +111,5 @@
'min': 'app_numpy.min',
'identity': 'app_numpy.identity',
'max': 'app_numpy.max',
- 'inf': 'app_numpy.inf',
- 'e': 'app_numpy.e',
- 'pi': 'app_numpy.pi',
'arange': 'app_numpy.arange',
}
diff --git a/pypy/module/micronumpy/app_numpy.py b/pypy/module/micronumpy/app_numpy.py
--- a/pypy/module/micronumpy/app_numpy.py
+++ b/pypy/module/micronumpy/app_numpy.py
@@ -3,11 +3,6 @@
import _numpypy
-inf = float("inf")
-e = math.e
-pi = math.pi
-
-
def average(a):
# This implements a weighted average, for now we don't implement the
# weighting, just the average part!
@@ -59,7 +54,7 @@
if not hasattr(a, "max"):
a = _numpypy.array(a)
return a.max(axis)
-
+
def arange(start, stop=None, step=1, dtype=None):
'''arange([start], stop[, step], dtype=None)
Generate values in the half-interval [start, stop).
diff --git a/pypy/module/micronumpy/test/test_module.py b/pypy/module/micronumpy/test/test_module.py
--- a/pypy/module/micronumpy/test/test_module.py
+++ b/pypy/module/micronumpy/test/test_module.py
@@ -21,13 +21,3 @@
from _numpypy import array, max
assert max(range(10)) == 9
assert max(array(range(10))) == 9
-
- def test_constants(self):
- import math
- from _numpypy import inf, e, pi
- assert type(inf) is float
- assert inf == float("inf")
- assert e == math.e
- assert type(e) is float
- assert pi == math.pi
- assert type(pi) is float
diff --git a/pypy/module/micronumpy/test/test_numarray.py b/pypy/module/micronumpy/test/test_numarray.py
--- a/pypy/module/micronumpy/test/test_numarray.py
+++ b/pypy/module/micronumpy/test/test_numarray.py
@@ -579,7 +579,7 @@
def test_div(self):
from math import isnan
- from _numpypy import array, dtype, inf
+ from _numpypy import array, dtype
a = array(range(1, 6))
b = a / a
@@ -600,15 +600,15 @@
a = array([-1.0, 0.0, 1.0])
b = array([0.0, 0.0, 0.0])
c = a / b
- assert c[0] == -inf
+ assert c[0] == float('-inf')
assert isnan(c[1])
- assert c[2] == inf
+ assert c[2] == float('inf')
b = array([-0.0, -0.0, -0.0])
c = a / b
- assert c[0] == inf
+ assert c[0] == float('inf')
assert isnan(c[1])
- assert c[2] == -inf
+ assert c[2] == float('-inf')
def test_div_other(self):
from _numpypy import array
diff --git a/pypy/module/micronumpy/test/test_ufuncs.py b/pypy/module/micronumpy/test/test_ufuncs.py
--- a/pypy/module/micronumpy/test/test_ufuncs.py
+++ b/pypy/module/micronumpy/test/test_ufuncs.py
@@ -312,9 +312,9 @@
def test_arcsinh(self):
import math
- from _numpypy import arcsinh, inf
+ from _numpypy import arcsinh
- for v in [inf, -inf, 1.0, math.e]:
+ for v in [float('inf'), float('-inf'), 1.0, math.e]:
assert math.asinh(v) == arcsinh(v)
assert math.isnan(arcsinh(float("nan")))
@@ -367,7 +367,7 @@
b = add.reduce(a, 0, keepdims=True)
assert b.shape == (1, 4)
assert (add.reduce(a, 0, keepdims=True) == [12, 15, 18, 21]).all()
-
+
def test_bitwise(self):
from _numpypy import bitwise_and, bitwise_or, arange, array
@@ -416,7 +416,7 @@
assert count_reduce_items(a) == 24
assert count_reduce_items(a, 1) == 3
assert count_reduce_items(a, (1, 2)) == 3 * 4
-
+
def test_true_divide(self):
from _numpypy import arange, array, true_divide
assert (true_divide(arange(3), array([2, 2, 2])) == array([0, 0.5, 1])).all()
diff --git a/pypy/module/posix/interp_posix.py b/pypy/module/posix/interp_posix.py
--- a/pypy/module/posix/interp_posix.py
+++ b/pypy/module/posix/interp_posix.py
@@ -543,10 +543,16 @@
dirname = FileEncoder(space, w_dirname)
result = rposix.listdir(dirname)
w_fs_encoding = getfilesystemencoding(space)
- result_w = [
- space.call_method(space.wrap(s), "decode", w_fs_encoding)
- for s in result
- ]
+ len_result = len(result)
+ result_w = [None] * len_result
+ for i in range(len_result):
+ w_bytes = space.wrap(result[i])
+ try:
+ result_w[i] = space.call_method(w_bytes,
+ "decode", w_fs_encoding)
+ except OperationError, e:
+ # fall back to the original byte string
+ result_w[i] = w_bytes
else:
dirname = space.str0_w(w_dirname)
result = rposix.listdir(dirname)
diff --git a/pypy/module/posix/test/test_posix2.py b/pypy/module/posix/test/test_posix2.py
--- a/pypy/module/posix/test/test_posix2.py
+++ b/pypy/module/posix/test/test_posix2.py
@@ -29,6 +29,7 @@
mod.pdir = pdir
unicode_dir = udir.ensure('fi\xc5\x9fier.txt', dir=True)
unicode_dir.join('somefile').write('who cares?')
+ unicode_dir.join('caf\xe9').write('who knows?')
mod.unicode_dir = unicode_dir
# in applevel tests, os.stat uses the CPython os.stat.
@@ -308,14 +309,22 @@
'file2']
def test_listdir_unicode(self):
+ import sys
unicode_dir = self.unicode_dir
if unicode_dir is None:
skip("encoding not good enough")
posix = self.posix
result = posix.listdir(unicode_dir)
- result.sort()
- assert result == [u'somefile']
- assert type(result[0]) is unicode
+ typed_result = [(type(x), x) for x in result]
+ assert (unicode, u'somefile') in typed_result
+ try:
+ u = "caf\xe9".decode(sys.getfilesystemencoding())
+ except UnicodeDecodeError:
+ # Could not decode, listdir returned the byte string
+ assert (str, "caf\xe9") in typed_result
+ else:
+ assert (unicode, u) in typed_result
+
def test_access(self):
pdir = self.pdir + '/file1'
More information about the pypy-commit
mailing list