[pypy-svn] pypy arm-backend-2: Merge default

bivab commits-noreply at bitbucket.org
Wed Dec 22 14:27:19 CET 2010


Author: David Schneider <david.schneider at picle.org>
Branch: arm-backend-2
Changeset: r40185:a1b990780990
Date: 2010-12-22 11:53 +0100
http://bitbucket.org/pypy/pypy/changeset/a1b990780990/

Log:	Merge default

diff --git a/pypy/doc/release-1.4.0.txt b/pypy/doc/release-1.4.1.txt
copy from pypy/doc/release-1.4.0.txt
copy to pypy/doc/release-1.4.1.txt
--- a/pypy/doc/release-1.4.0.txt
+++ b/pypy/doc/release-1.4.1.txt
@@ -1,59 +1,84 @@
 ===============================
-PyPy 1.4: Ouroboros in practice
+PyPy 1.4.1
 ===============================
 
-We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough
-in our long journey, as PyPy 1.4 is the first PyPy release that can translate
-itself faster than CPython.  Starting today, we are using PyPy more for
-our every-day development.  So may you :) You can download it here:
+We're pleased to announce the 1.4.1 release of PyPy.  This
+release consolidates all the bug fixes that occurred since the
+previous release.  To everyone that took the trouble to report
+them, we want to say thank you.
 
     http://pypy.org/download.html
 
 What is PyPy
 ============
 
-PyPy is a very compliant Python interpreter, almost a drop-in replacement
-for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison)
+PyPy is a very compliant Python interpreter, almost a drop-in
+replacement for CPython.  Note that it still only emulates Python
+2.5 by default; the ``fast-forward`` branch with Python 2.7
+support is slowly getting ready but will only be integrated in
+the next release.
 
-Among its new features, this release includes numerous performance improvements
-(which made fast self-hosting possible), a 64-bit JIT backend, as well
-as serious stabilization. As of now, we can consider the 32-bit and 64-bit
-linux versions of PyPy stable enough to run `in production`_.
+In two words, the advantage of trying out PyPy instead of CPython
+(the default implementation of Python) is, for now, the
+performance.  Not all programs are faster in PyPy, but we are
+confident that any CPU-intensive task will be much faster, at
+least if it runs for long enough (the JIT has a slow warm-up
+phase, which can take several seconds or even one minute on the
+largest programs).
 
-Numerous speed achievements are described on `our blog`_. Normalized speed
-charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython 2.6`_
-are available on benchmark website. For the impatient: yes, we got a lot faster!
+Note again that we do support compiling and using C extension
+modules from CPython (``pypy setup.py install``).  However, this
+is still an alpha feature, and the most complex modules typically
+fail for various reasons; others work (e.g. ``PIL``) but take a
+serious performance hit.  Also, for Mac OS X see below.
+
+Please note also that PyPy's performance was optimized almost
+exclusively on Linux.  It seems from some reports that on Windows
+as well as Mac OS X (probably for different reasons) the
+performance might be lower.  We did not investigate much so far.
+
 
 More highlights
 ===============
 
-* PyPy's built-in Just-in-Time compiler is fully transparent and
-  automatically generated; it now also has very reasonable memory
-  requirements.  The total memory used by a very complex and
-  long-running process (translating PyPy itself) is within 1.5x to
-  at most 2x the memory needed by CPython, for a speed-up of 2x.
+* We migrated to Mercurial (thanks to Ronny Pfannschmidt and
+  Antonio Cuni) for the effort) and moved to bitbucket.  The new
+  command to check out a copy of PyPy is::
 
-* More compact instances.  All instances are as compact as if
-  they had ``__slots__``.  This can give programs a big gain in
-  memory.  (In the example of translation above, we already have
-  carefully placed ``__slots__``, so there is no extra win.)
+        hg clone http://bitbucket.org/pypy/pypy
 
-* `Virtualenv support`_: now PyPy is fully compatible with virtualenv_: note that
-  to use it, you need a recent version of virtualenv (>= 1.5).
+* In long-running processes, the assembler generated by old
+  JIT-compilations is now freed.  There should be no more leak,
+  however long the process runs.
 
-* Faster (and JITted) regular expressions - huge boost in speeding up
-  the `re` module.
+* Improve a lot the performance of the ``binascii`` module, and
+  of ``hashlib.md5`` and ``hashlib.sha``.
 
-* Other speed improvements, like JITted calls to functions like map().
+* Made sys.setrecursionlimit() a no-op.  Instead, we rely purely
+  on the built-in stack overflow detection mechanism, which also
+  gives you a RuntimeError -- just not at some exact recursion
+  level.
 
-.. _virtualenv: http://pypi.python.org/pypi/virtualenv
-.. _`Virtualenv support`: http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html
-.. _`in production`: http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software.html
-.. _`our blog`: http://morepypy.blogspot.com
-.. _`pypy 1.4 and pypy 1.3`: http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=1%2B41&chart=normal+bars
-.. _`pypy 1.4 and cpython 2.6`: http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=2%2B35&chart=normal+bars
+* Fix argument processing (now e.g. ``pypy -OScpass`` works like
+  it does on CPython --- if you have a clue what it does there
+  ``:-)`` )
+
+* cpyext on Mac OS X: it still does not seem to work.  I get
+  systematically a segfault in dlopen().  Contributions welcome.
+
+* Fix two corner cases in the GC (one in minimark, one in
+  asmgcc+JIT).  This notably prevented "pypy translate.py -Ojit"
+  from working on Windows, leading to crashes.
+
+* Fixed a corner case in the JIT's optimizer, leading to "Fatal
+  RPython error: AssertionError".
+
+* Added some missing built-in functions into the 'os' module.
+
+* Fix ctypes (it was not propagating keepalive information from
+  c_void_p).
+
 
 Cheers,
 
-Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski,
-Amaury Forgeot d'Arc, Armin Rigo and the PyPy team
+Armin Rigo, for the rest of the team

diff --git a/pypy/tool/release/package.py b/pypy/tool/release/package.py
--- a/pypy/tool/release/package.py
+++ b/pypy/tool/release/package.py
@@ -78,7 +78,13 @@
     old_dir = os.getcwd()
     try:
         os.chdir(str(builddir))
-        os.system("strip -x " + str(archive_pypy_c))    # ignore errors
+        #
+        # 'strip' fun: see https://codespeak.net/issue/pypy-dev/issue587
+        if sys.platform == 'darwin':
+            os.system("strip -x " + str(archive_pypy_c))    # ignore errors
+        else:
+            os.system("strip " + str(archive_pypy_c))    # ignore errors
+        #
         if USE_TARFILE_MODULE:
             import tarfile
             tf = tarfile.open(str(builddir.join(name + '.tar.bz2')), 'w:bz2')

diff --git a/pypy/module/cpyext/api.py b/pypy/module/cpyext/api.py
--- a/pypy/module/cpyext/api.py
+++ b/pypy/module/cpyext/api.py
@@ -939,7 +939,9 @@
     if os.sep not in path:
         path = os.curdir + os.sep + path      # force a '/' in the path
     state = space.fromcache(State)
-    state.package_context = name
+    if state.find_extension(name, path) is not None:
+        return
+    state.package_context = name, path
     try:
         from pypy.rlib import rdynload
         try:
@@ -964,7 +966,8 @@
         generic_cpy_call(space, initfunc)
         state.check_and_raise_exception()
     finally:
-        state.package_context = None
+        state.package_context = None, None
+    state.fixup_extension(name, path)
 
 @specialize.ll()
 def generic_cpy_call(space, func, *args):

diff --git a/pypy/interpreter/argument.py b/pypy/interpreter/argument.py
--- a/pypy/interpreter/argument.py
+++ b/pypy/interpreter/argument.py
@@ -105,14 +105,11 @@
         make_sure_not_resized(self.arguments_w)
         if w_stararg is not None:
             self._combine_starargs_wrapped(w_stararg)
-        if w_starstararg is not None:
-            self._combine_starstarargs_wrapped(w_starstararg)
-            # if we have a call where **args are used at the callsite
-            # we shouldn't let the JIT see the argument matching
-            self._dont_jit = True
-        else:
-            self._dont_jit = False
-        
+        # if we have a call where **args are used at the callsite
+        # we shouldn't let the JIT see the argument matching
+        self._dont_jit = (w_starstararg is not None and
+                          self._combine_starstarargs_wrapped(w_starstararg))
+
     def __repr__(self):
         """ NOT_RPYTHON """
         name = self.__class__.__name__
@@ -160,10 +157,20 @@
             raise OperationError(space.w_TypeError,
                                  space.wrap("argument after ** must be "
                                             "a dictionary"))
-        keywords_w = [None] * space.int_w(space.len(w_starstararg))
-        keywords = [None] * space.int_w(space.len(w_starstararg))
+        if space.is_true(w_starstararg):
+            self._do_combine_starstarargs_wrapped(w_starstararg)
+            return True
+        else:
+            return False    # empty dict; don't disable the JIT
+
+    def _do_combine_starstarargs_wrapped(self, w_starstararg):
+        space = self.space
+        keys_w = space.unpackiterable(w_starstararg)
+        length = len(keys_w)
+        keywords_w = [None] * length
+        keywords = [None] * length
         i = 0
-        for w_key in space.unpackiterable(w_starstararg):
+        for w_key in keys_w:
             try:
                 key = space.str_w(w_key)
             except OperationError, e:

diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
--- a/pypy/interpreter/baseobjspace.py
+++ b/pypy/interpreter/baseobjspace.py
@@ -1126,6 +1126,11 @@
             buffer = self.buffer_w(w_obj)
             return buffer.as_str()
 
+    def str_or_None_w(self, w_obj):
+        if self.is_w(w_obj, self.w_None):
+            return None
+        return self.str_w(w_obj)
+
     def realstr_w(self, w_obj):
         # Like str_w, but only works if w_obj is really of type 'str'.
         if not self.is_true(self.isinstance(w_obj, self.w_str)):

diff --git a/pypy/module/cpyext/test/test_cpyext.py b/pypy/module/cpyext/test/test_cpyext.py
--- a/pypy/module/cpyext/test/test_cpyext.py
+++ b/pypy/module/cpyext/test/test_cpyext.py
@@ -222,6 +222,12 @@
         else:
             return os.path.dirname(mod)
 
+    def reimport_module(self, mod, name):
+        api.load_extension_module(self.space, mod, name)
+        return self.space.getitem(
+            self.space.sys.get('modules'),
+            self.space.wrap(name))
+
     def import_extension(self, modname, functions, prologue=""):
         methods_table = []
         codes = []
@@ -261,6 +267,7 @@
         self.imported_module_names = []
 
         self.w_import_module = self.space.wrap(self.import_module)
+        self.w_reimport_module = self.space.wrap(self.reimport_module)
         self.w_import_extension = self.space.wrap(self.import_extension)
         self.w_compile_module = self.space.wrap(self.compile_module)
         self.w_record_imported_module = self.space.wrap(
@@ -709,3 +716,43 @@
         p = mod.get_programname()
         print p
         assert 'py' in p
+
+    def test_no_double_imports(self):
+        import sys, os
+        try:
+            init = """
+            static int _imported_already = 0;
+            FILE *f = fopen("_imported_already", "w");
+            fprintf(f, "imported_already: %d\\n", _imported_already);
+            fclose(f);
+            _imported_already = 1;
+            if (Py_IsInitialized()) {
+                Py_InitModule("foo", NULL);
+            }
+            """
+            self.import_module(name='foo', init=init)
+            assert 'foo' in sys.modules
+
+            f = open('_imported_already')
+            data = f.read()
+            f.close()
+            assert data == 'imported_already: 0\n'
+
+            f = open('_imported_already', 'w')
+            f.write('not again!\n')
+            f.close()
+            m1 = sys.modules['foo']
+            m2 = self.reimport_module(m1.__file__, name='foo')
+            assert m1 is m2
+            assert m1 is sys.modules['foo']
+
+            f = open('_imported_already')
+            data = f.read()
+            f.close()
+            assert data == 'not again!\n'
+
+        finally:
+            try:
+                os.unlink('_imported_already')
+            except OSError:
+                pass

diff --git a/pypy/tool/release/make_release.py b/pypy/tool/release/make_release.py
--- a/pypy/tool/release/make_release.py
+++ b/pypy/tool/release/make_release.py
@@ -4,7 +4,8 @@
 into release packages. Note: you must run apropriate buildbots first and
 make sure there are no failures. Use force-builds.py from the same directory.
 
-Usage: make_release.py release/<release name> release_version
+Usage: make_release.py  <branchname>  <version>
+ e.g.: make_release.py  release-1.4.1  1.4.1
 """
 
 import autopath
@@ -20,6 +21,7 @@
 import shutil
 
 BASEURL = 'http://buildbot.pypy.org/nightly/'
+PAUSE = False
 
 def browse_nightly(branch,
                    baseurl=BASEURL,
@@ -32,12 +34,12 @@
     dom = minidom.parseString(xml)
     refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')
             if 'pypy' in node.getAttribute('href')]
-    # all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2
-    r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$')
+    # all refs are of form: pypy-c-{type}-{revnum}-{hghash}-{platform}.tar.bz2
+    r = re.compile('pypy-c-([\w\d]+)-(\d+)-([0-9a-f]+)-([\w\d]+).tar.bz2$')
     d = {}
     for ref in refs:
-        kind, rev, platform = r.match(ref).groups()
-        rev = int(rev)
+        kind, revnum, hghash, platform = r.match(ref).groups()
+        rev = int(revnum)
         try:
             lastrev, _ = d[(kind, platform)]
         except KeyError:
@@ -51,8 +53,10 @@
     tmpdir = udir.join('download')
     tmpdir.ensure(dir=True)
     alltars = []
+    olddir = os.getcwd()
     try:
         os.chdir(str(tmpdir))
+        print 'Using tmpdir', str(tmpdir)
         for (kind, platform), (rev, name) in to_download.iteritems():
             if platform == 'win32':
                 print 'Ignoring %s, windows unsupported' % name
@@ -64,20 +68,23 @@
                 t = tarfile.open(str(tmpdir.join(name)))
                 dirname = t.getmembers()[0].name
                 t.extractall(path=str(tmpdir))
-                os.system('mv %s %s' % (str(tmpdir.join(dirname)),
-                                        str(tmpdir.join('pypy-%s' % release))))
                 if kind == 'jit':
                     kind = ''
                 else:
                     kind = '-' + kind
-                olddir = os.getcwd()
-                name = 'pypy-%s-%s%s.tar.bz2' % (release, platform, kind)
+                topdirname = 'pypy-%s-%s%s' % (release, platform, kind)
+                os.system('mv %s %s' % (str(tmpdir.join(dirname)),
+                                        str(tmpdir.join(topdirname))))
+                if PAUSE:
+                    print 'Pausing, press Enter...'
+                    raw_input()
+                name = '%s.tar.bz2' % topdirname
                 print "Building %s" % name
                 t = tarfile.open(name, 'w:bz2')
-                t.add('pypy-%s' % release)
+                t.add(topdirname)
                 alltars.append(name)
                 t.close()
-                shutil.rmtree(str(tmpdir.join('pypy-' + release)))
+                shutil.rmtree(str(tmpdir.join(topdirname)))
         for name in alltars:
             print "Uploading %s" % name
             os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name)

diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -6,6 +6,7 @@
 syntax: regexp
 ^testresult$
 ^site-packages$
+^bin$
 ^pypy/module/cpyext/src/.+\.o$
 ^pypy/bin/pypy-c
 ^pypy/translator/jvm/src/pypy/.+\.class$


More information about the Pypy-commit mailing list