[pypy-svn] pypy arm-backend-2: Merge default
bivab
commits-noreply at bitbucket.org
Wed Dec 22 14:27:19 CET 2010
Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r40185:a1b990780990
Date: 2010-12-22 11:53 +0100
http://bitbucket.org/pypy/pypy/changeset/a1b990780990/
Log: Merge default
diff --git a/pypy/doc/release-1.4.0.txt b/pypy/doc/release-1.4.1.txt
copy from pypy/doc/release-1.4.0.txt
copy to pypy/doc/release-1.4.1.txt
--- a/pypy/doc/release-1.4.0.txt
+++ b/pypy/doc/release-1.4.1.txt
@@ -1,59 +1,84 @@
===============================
-PyPy 1.4: Ouroboros in practice
+PyPy 1.4.1
===============================
-We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough
-in our long journey, as PyPy 1.4 is the first PyPy release that can translate
-itself faster than CPython. Starting today, we are using PyPy more for
-our every-day development. So may you :) You can download it here:
+We're pleased to announce the 1.4.1 release of PyPy. This
+release consolidates all the bug fixes that occurred since the
+previous release. To everyone that took the trouble to report
+them, we want to say thank you.
http://pypy.org/download.html
What is PyPy
============
-PyPy is a very compliant Python interpreter, almost a drop-in replacement
-for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison)
+PyPy is a very compliant Python interpreter, almost a drop-in
+replacement for CPython. Note that it still only emulates Python
+2.5 by default; the ``fast-forward`` branch with Python 2.7
+support is slowly getting ready but will only be integrated in
+the next release.
-Among its new features, this release includes numerous performance improvements
-(which made fast self-hosting possible), a 64-bit JIT backend, as well
-as serious stabilization. As of now, we can consider the 32-bit and 64-bit
-linux versions of PyPy stable enough to run `in production`_.
+In two words, the advantage of trying out PyPy instead of CPython
+(the default implementation of Python) is, for now, the
+performance. Not all programs are faster in PyPy, but we are
+confident that any CPU-intensive task will be much faster, at
+least if it runs for long enough (the JIT has a slow warm-up
+phase, which can take several seconds or even one minute on the
+largest programs).
-Numerous speed achievements are described on `our blog`_. Normalized speed
-charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython 2.6`_
-are available on benchmark website. For the impatient: yes, we got a lot faster!
+Note again that we do support compiling and using C extension
+modules from CPython (``pypy setup.py install``). However, this
+is still an alpha feature, and the most complex modules typically
+fail for various reasons; others work (e.g. ``PIL``) but take a
+serious performance hit. Also, for Mac OS X see below.
+
+Please note also that PyPy's performance was optimized almost
+exclusively on Linux. It seems from some reports that on Windows
+as well as Mac OS X (probably for different reasons) the
+performance might be lower. We did not investigate much so far.
+
More highlights
===============
-* PyPy's built-in Just-in-Time compiler is fully transparent and
- automatically generated; it now also has very reasonable memory
- requirements. The total memory used by a very complex and
- long-running process (translating PyPy itself) is within 1.5x to
- at most 2x the memory needed by CPython, for a speed-up of 2x.
+* We migrated to Mercurial (thanks to Ronny Pfannschmidt and
+ Antonio Cuni) for the effort) and moved to bitbucket. The new
+ command to check out a copy of PyPy is::
-* More compact instances. All instances are as compact as if
- they had ``__slots__``. This can give programs a big gain in
- memory. (In the example of translation above, we already have
- carefully placed ``__slots__``, so there is no extra win.)
+ hg clone http://bitbucket.org/pypy/pypy
-* `Virtualenv support`_: now PyPy is fully compatible with virtualenv_: note that
- to use it, you need a recent version of virtualenv (>= 1.5).
+* In long-running processes, the assembler generated by old
+ JIT-compilations is now freed. There should be no more leak,
+ however long the process runs.
-* Faster (and JITted) regular expressions - huge boost in speeding up
- the `re` module.
+* Improve a lot the performance of the ``binascii`` module, and
+ of ``hashlib.md5`` and ``hashlib.sha``.
-* Other speed improvements, like JITted calls to functions like map().
+* Made sys.setrecursionlimit() a no-op. Instead, we rely purely
+ on the built-in stack overflow detection mechanism, which also
+ gives you a RuntimeError -- just not at some exact recursion
+ level.
-.. _virtualenv: http://pypi.python.org/pypi/virtualenv
-.. _`Virtualenv support`: http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html
-.. _`in production`: http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software.html
-.. _`our blog`: http://morepypy.blogspot.com
-.. _`pypy 1.4 and pypy 1.3`: http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=1%2B41&chart=normal+bars
-.. _`pypy 1.4 and cpython 2.6`: http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=2%2B35&chart=normal+bars
+* Fix argument processing (now e.g. ``pypy -OScpass`` works like
+ it does on CPython --- if you have a clue what it does there
+ ``:-)`` )
+
+* cpyext on Mac OS X: it still does not seem to work. I get
+ systematically a segfault in dlopen(). Contributions welcome.
+
+* Fix two corner cases in the GC (one in minimark, one in
+ asmgcc+JIT). This notably prevented "pypy translate.py -Ojit"
+ from working on Windows, leading to crashes.
+
+* Fixed a corner case in the JIT's optimizer, leading to "Fatal
+ RPython error: AssertionError".
+
+* Added some missing built-in functions into the 'os' module.
+
+* Fix ctypes (it was not propagating keepalive information from
+ c_void_p).
+
Cheers,
-Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski,
-Amaury Forgeot d'Arc, Armin Rigo and the PyPy team
+Armin Rigo, for the rest of the team
diff --git a/pypy/tool/release/package.py b/pypy/tool/release/package.py
--- a/pypy/tool/release/package.py
+++ b/pypy/tool/release/package.py
@@ -78,7 +78,13 @@
old_dir = os.getcwd()
try:
os.chdir(str(builddir))
- os.system("strip -x " + str(archive_pypy_c)) # ignore errors
+ #
+ # 'strip' fun: see https://codespeak.net/issue/pypy-dev/issue587
+ if sys.platform == 'darwin':
+ os.system("strip -x " + str(archive_pypy_c)) # ignore errors
+ else:
+ os.system("strip " + str(archive_pypy_c)) # ignore errors
+ #
if USE_TARFILE_MODULE:
import tarfile
tf = tarfile.open(str(builddir.join(name + '.tar.bz2')), 'w:bz2')
diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -939,7 +939,9 @@
if os.sep not in path:
path = os.curdir + os.sep + path # force a '/' in the path
state = space.fromcache(State)
- state.package_context = name
+ if state.find_extension(name, path) is not None:
+ return
+ state.package_context = name, path
try:
from pypy.rlib import rdynload
try:
@@ -964,7 +966,8 @@
generic_cpy_call(space, initfunc)
state.check_and_raise_exception()
finally:
- state.package_context = None
+ state.package_context = None, None
+ state.fixup_extension(name, path)
@specialize.ll()
def generic_cpy_call(space, func, *args):
diff --git a/pypy/interpreter/argument.py b/pypy/interpreter/argument.py
--- a/pypy/interpreter/argument.py
+++ b/pypy/interpreter/argument.py
@@ -105,14 +105,11 @@
make_sure_not_resized(self.arguments_w)
if w_stararg is not None:
self._combine_starargs_wrapped(w_stararg)
- if w_starstararg is not None:
- self._combine_starstarargs_wrapped(w_starstararg)
- # if we have a call where **args are used at the callsite
- # we shouldn't let the JIT see the argument matching
- self._dont_jit = True
- else:
- self._dont_jit = False
-
+ # if we have a call where **args are used at the callsite
+ # we shouldn't let the JIT see the argument matching
+ self._dont_jit = (w_starstararg is not None and
+ self._combine_starstarargs_wrapped(w_starstararg))
+
def __repr__(self):
""" NOT_RPYTHON """
name = self.__class__.__name__
@@ -160,10 +157,20 @@
raise OperationError(space.w_TypeError,
space.wrap("argument after ** must be "
"a dictionary"))
- keywords_w = [None] * space.int_w(space.len(w_starstararg))
- keywords = [None] * space.int_w(space.len(w_starstararg))
+ if space.is_true(w_starstararg):
+ self._do_combine_starstarargs_wrapped(w_starstararg)
+ return True
+ else:
+ return False # empty dict; don't disable the JIT
+
+ def _do_combine_starstarargs_wrapped(self, w_starstararg):
+ space = self.space
+ keys_w = space.unpackiterable(w_starstararg)
+ length = len(keys_w)
+ keywords_w = [None] * length
+ keywords = [None] * length
i = 0
- for w_key in space.unpackiterable(w_starstararg):
+ for w_key in keys_w:
try:
key = space.str_w(w_key)
except OperationError, e:
diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -1126,6 +1126,11 @@
buffer = self.buffer_w(w_obj)
return buffer.as_str()
+ def str_or_None_w(self, w_obj):
+ if self.is_w(w_obj, self.w_None):
+ return None
+ return self.str_w(w_obj)
+
def realstr_w(self, w_obj):
# Like str_w, but only works if w_obj is really of type 'str'.
if not self.is_true(self.isinstance(w_obj, self.w_str)):
diff --git a/pypy/module/cpyext/test/test_cpyext.py b/pypy/module/cpyext/test/test_cpyext.py
--- a/pypy/module/cpyext/test/test_cpyext.py
+++ b/pypy/module/cpyext/test/test_cpyext.py
@@ -222,6 +222,12 @@
else:
return os.path.dirname(mod)
+ def reimport_module(self, mod, name):
+ api.load_extension_module(self.space, mod, name)
+ return self.space.getitem(
+ self.space.sys.get('modules'),
+ self.space.wrap(name))
+
def import_extension(self, modname, functions, prologue=""):
methods_table = []
codes = []
@@ -261,6 +267,7 @@
self.imported_module_names = []
self.w_import_module = self.space.wrap(self.import_module)
+ self.w_reimport_module = self.space.wrap(self.reimport_module)
self.w_import_extension = self.space.wrap(self.import_extension)
self.w_compile_module = self.space.wrap(self.compile_module)
self.w_record_imported_module = self.space.wrap(
@@ -709,3 +716,43 @@
p = mod.get_programname()
print p
assert 'py' in p
+
+ def test_no_double_imports(self):
+ import sys, os
+ try:
+ init = """
+ static int _imported_already = 0;
+ FILE *f = fopen("_imported_already", "w");
+ fprintf(f, "imported_already: %d\\n", _imported_already);
+ fclose(f);
+ _imported_already = 1;
+ if (Py_IsInitialized()) {
+ Py_InitModule("foo", NULL);
+ }
+ """
+ self.import_module(name='foo', init=init)
+ assert 'foo' in sys.modules
+
+ f = open('_imported_already')
+ data = f.read()
+ f.close()
+ assert data == 'imported_already: 0\n'
+
+ f = open('_imported_already', 'w')
+ f.write('not again!\n')
+ f.close()
+ m1 = sys.modules['foo']
+ m2 = self.reimport_module(m1.__file__, name='foo')
+ assert m1 is m2
+ assert m1 is sys.modules['foo']
+
+ f = open('_imported_already')
+ data = f.read()
+ f.close()
+ assert data == 'not again!\n'
+
+ finally:
+ try:
+ os.unlink('_imported_already')
+ except OSError:
+ pass
diff --git a/pypy/tool/release/make_release.py b/pypy/tool/release/make_release.py
--- a/pypy/tool/release/make_release.py
+++ b/pypy/tool/release/make_release.py
@@ -4,7 +4,8 @@
into release packages. Note: you must run apropriate buildbots first and
make sure there are no failures. Use force-builds.py from the same directory.
-Usage: make_release.py release/<release name> release_version
+Usage: make_release.py <branchname> <version>
+ e.g.: make_release.py release-1.4.1 1.4.1
"""
import autopath
@@ -20,6 +21,7 @@
import shutil
BASEURL = 'http://buildbot.pypy.org/nightly/'
+PAUSE = False
def browse_nightly(branch,
baseurl=BASEURL,
@@ -32,12 +34,12 @@
dom = minidom.parseString(xml)
refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')
if 'pypy' in node.getAttribute('href')]
- # all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2
- r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$')
+ # all refs are of form: pypy-c-{type}-{revnum}-{hghash}-{platform}.tar.bz2
+ r = re.compile('pypy-c-([\w\d]+)-(\d+)-([0-9a-f]+)-([\w\d]+).tar.bz2$')
d = {}
for ref in refs:
- kind, rev, platform = r.match(ref).groups()
- rev = int(rev)
+ kind, revnum, hghash, platform = r.match(ref).groups()
+ rev = int(revnum)
try:
lastrev, _ = d[(kind, platform)]
except KeyError:
@@ -51,8 +53,10 @@
tmpdir = udir.join('download')
tmpdir.ensure(dir=True)
alltars = []
+ olddir = os.getcwd()
try:
os.chdir(str(tmpdir))
+ print 'Using tmpdir', str(tmpdir)
for (kind, platform), (rev, name) in to_download.iteritems():
if platform == 'win32':
print 'Ignoring %s, windows unsupported' % name
@@ -64,20 +68,23 @@
t = tarfile.open(str(tmpdir.join(name)))
dirname = t.getmembers()[0].name
t.extractall(path=str(tmpdir))
- os.system('mv %s %s' % (str(tmpdir.join(dirname)),
- str(tmpdir.join('pypy-%s' % release))))
if kind == 'jit':
kind = ''
else:
kind = '-' + kind
- olddir = os.getcwd()
- name = 'pypy-%s-%s%s.tar.bz2' % (release, platform, kind)
+ topdirname = 'pypy-%s-%s%s' % (release, platform, kind)
+ os.system('mv %s %s' % (str(tmpdir.join(dirname)),
+ str(tmpdir.join(topdirname))))
+ if PAUSE:
+ print 'Pausing, press Enter...'
+ raw_input()
+ name = '%s.tar.bz2' % topdirname
print "Building %s" % name
t = tarfile.open(name, 'w:bz2')
- t.add('pypy-%s' % release)
+ t.add(topdirname)
alltars.append(name)
t.close()
- shutil.rmtree(str(tmpdir.join('pypy-' + release)))
+ shutil.rmtree(str(tmpdir.join(topdirname)))
for name in alltars:
print "Uploading %s" % name
os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name)
diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -6,6 +6,7 @@
syntax: regexp
^testresult$
^site-packages$
+^bin$
^pypy/module/cpyext/src/.+\.o$
^pypy/bin/pypy-c
^pypy/translator/jvm/src/pypy/.+\.class$
More information about the Pypy-commit
mailing list