Mailman 3 February 2012 - pypy-issue

[issue833] missing functions in the os module
by teepark 13 May '14

13 May '14

New submission from teepark <travis.parker(a)gmail.com>: I ran into this when I needed os.fchmod, but there are quite a few names not present in pypy's os module (I'm on x86_64 linux, and I'm sure many of these aren't guaranteed to be there for any platform, but pypy on x86_64 linux should probably include them): - getlogin - confstr - getresgid - tcsetpgrp - statvfs - setresgid - initgroups - fstatvfs - getresuid - fchmod - setresuid - tcgetpgrp - confstr_names - setgroups - pathconf - fchown - openpty - ctermid - statvfs_result Please change the priority if "feature" isn't appropriate - it doesn't feel right but I don't know that cpython compatibility issues are being treated as bugs. ---------- messages: 2975 nosy: pypy-issue, teepark priority: feature status: unread title: missing functions in the os module ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue833> ________________________________________

6 13

[issue644] Add option parser to package.py
by Dan 09 May '14

09 May '14

New submission from Dan <dcolish(a)gmail.com>: I've been using package.py a lot more often now and I find an option parser interface to be nice for command line scripts. I've written one for package.py, but I believe it will require buildbot updates since buildbot is calling the old interface. ---------- effort: ??? files: package.diff messages: 2169 nosy: dcolish, pypy-issue priority: wish release: ??? status: in-progress title: Add option parser to package.py _______________________________________________________ PyPy development tracker <pypy-dev-issue(a)codespeak.net> <https://codespeak.net/issue/pypy-dev/issue644> _______________________________________________________

2 1

[issue662] get_config_vars has nothing for CC, while expected internally
by Martijn 08 May '14

08 May '14

New submission from Martijn <ss(a)kleptog.org>: Currently distutils.sysconfig.get_config_vars() has no value for CC. This means that when you try to build a C extension using distutils it breaks, because distutils internally uses get_config_vars() to determine the compiler (distutils/ccompiler.py). Attached is a patch that works for me, with this C extensions build normally (excluding cpyext problems). But this may be a layering violation, so I'll leave it to you guys to determine the correct fix. In any case it's an improvement which improves compatibility. ---------- effort: ??? files: distutils.patch messages: 2267 nosy: kleptog, pypy-issue priority: feature release: ??? status: unread title: get_config_vars has nothing for CC, while expected internally _______________________________________________________ PyPy development tracker <pypy-dev-issue(a)codespeak.net> <https://codespeak.net/issue/pypy-dev/issue662> _______________________________________________________

4 3

[issue955] move struct.Struct to interplevel
by Alex Gaynor 06 May '14

06 May '14

New submission from Alex Gaynor <alex.gaynor(a)gmail.com>: There are a few reasons and issues to be fixed: 1) format should be immutable on the Struct instance 2) format should be promoted so that unpack/pack loops are unrolled correctly 3) it should work nicer with buffers Discussion from IRC: [6:24 PM] <Alex_Gaynor> arigato: so struct.unpack is very fast because the JIT checks if the format str is constant and if it is unrolsl the loops [6:24 PM] <arigato> ah, bah [6:24 PM] <Alex_Gaynor> arigato: however if you have x = Struct("format str"); for i n xrange(100): x.unpack(i) or so, the string isn't constant [6:24 PM] <Alex_Gaynor> it'd be ok if Struct was a global or something [6:24 PM] <Alex_Gaynor> I would if Struct instances should just promote their format and be done with it [6:24 PM] <arigato> and even not, because x.format is not a constant attribute [6:25 PM] <Alex_Gaynor> is it constant on CPython? [6:25 PM] <arigato> I suppose, but right now it's written at app-level [6:25 PM] <arigato> in pypy [6:25 PM] ⇐ lucian (~lucian@cpc1-newc15-2-0- cust84.gate.cable.virginmedia.com) quit: Ping timeout: 240 seconds [6:25 PM] <Alex_Gaynor> arigato: yes, I guess it might make sense to move it to interplevel so we can make it const and do the promotion [6:25 PM] <Alex_Gaynor> I think promote makes sense here [6:25 PM] <arigato> yes and no, as usual [6:26 PM] <Alex_Gaynor> indeed [6:26 PM] <arigato> e.g. what if someone does x = Struct("i" * n) [6:27 PM] <arigato> well I suppose that he's punished and it's his fault, or something [6:27 PM] <Alex_Gaynor> indeed, as always the mess of heuristics :/ [6:27 PM] <arigato> yes [6:27 PM] <exarkun> I did that this afternoon. [6:27 PM] <arigato> ah no, it's potentially a bad idea to promote a string, anyway [6:27 PM] ⇐ JaRoel|4d (~jaroel|4d@2001:980:7133:1:317d:e87f:5636:fdbe) quit: Remote host closed the connection [6:28 PM] <arigato> I think it's promoting the pointer, not the actual string content [6:28 PM] <Alex_Gaynor> arigato: err, yes, we have string_promote nowadays though [6:28 PM] <arigato> ah, ok Benchmark: http://pastebin.com/brfj0Aq9 ---------- messages: 3546 nosy: agaynor, pypy-issue priority: performance bug status: unread title: move struct.Struct to interplevel ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue955> ________________________________________

2 2

[issue818] MySQLdb-python 1.2.3 segmentation fault on PyPy 1.5 OSX 64bit
by Didip Kerabat 25 Apr '14

25 Apr '14

New submission from Didip Kerabat <didipk(a)gmail.com>: I'm not sure if this would help, but I documented the crash report here: https://gist.github.com/1120902 ---------- messages: 2901 nosy: didip, pypy-issue priority: bug status: unread title: MySQLdb-python 1.2.3 segmentation fault on PyPy 1.5 OSX 64bit ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue818> ________________________________________

3 2

[issue1022] dict.update significantly slower than series of dict.__setitem__
by Xavier Morel 24 Apr '14

24 Apr '14

New submission from Xavier Morel <bugs.pypy.org(a)masklinn.net>: In CPython, as the number of keys grow dict.update settles at roughly half the speed of an equivalent bunch of dict[key]=value sequence: > python2.7 -m timeit -s 'd = {}' 'd["foo"] = 3' 'd["bar"] = 4' 'd["baz"] = 5' 'd["qux"] = 6' 'd["quux"] = 7' 1000000 loops, best of 3: 0.395 usec per loop > python2.7 -m timeit -s 'd = {}' 'd.update(foo=3, bar=4, baz=5, qux=6, quux=7)' 1000000 loops, best of 3: 0.657 usec per loop In Pypy, on the other hand, the difference in speed is on the order of 20x, even while setting 5 keys and dict.update is ~30% the speed of the equivalent call in CPython at almost 2µsec/loop to CPython's 0.66µsec/loop, while pypy's __setitem__ is more than 4 times faster than CPython: > pypy-c -m timeit -s 'd = {}' 'd["foo"] = 3' 'd["bar"] = 4' 'd["baz"] = 5' 'd["qux"] = 6' 'd["quux"] = 7' 10000000 loops, best of 3: 0.0877 usec per loop > pypy-c -m timeit -s 'd = {}' 'd.update(foo=3, bar=4, baz=5, qux=6, quux=7)' 100000 loops, best of 3: 1.95 usec per loop that kind-of bothers me, as I tend to use dict.update over long sequences of setitem for readability purpose. Now of course half a µsec per call is not a big difference in absolute, but still… Versions information: * OSX 10.6.8 * CPython 2.7.2 (Macports) * Pypy 1.7.0 Python 2.7.1 (?, Nov 24 2011, 10:57:50) (Macports) ---------- messages: 3814 nosy: masklinn, pypy-issue priority: performance bug status: unread title: dict.update significantly slower than series of dict.__setitem__ ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue1022> ________________________________________

4 3

[issue824] Memory usage in time of pytest.py running under pypy
by Piotr Skamruk 24 Apr '14

24 Apr '14

New submission from Piotr Skamruk <piotr.skamruk(a)gmail.com>: pypy-c (jit version) translated around 2011-08-02 machine with 4G ram, with ubuntu oneiric onboard python2.7 pytest.py pypy/module/_ssl/test after less than minute - done max ram usage around 700M pypy pytest.py pypy/module/_ssl/test after several minutes pypy eated whole ram, and kswapd goes crazy ---------- messages: 2913 nosy: jell, pypy-issue priority: bug status: unread title: Memory usage in time of pytest.py running under pypy ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue824> ________________________________________

2 1

[issue1029] cpython test_subprocess.py fails
by Garen Parham 23 Apr '14

23 Apr '14

New submission from Garen Parham <garen.p(a)gmail.com>: On Ubuntu Linux 11.10, pypy trunk currently fails one of the modified test_subproces.py unit tests like so: garen@ubuntu:~/dev/bins$ ~/dev/bins/pypy-c-jit-51842-457e2166648c-linux/bin/pypy ~/dev/bins//pypy-c-jit-51842-457e2166648c-linux/lib-python/modified- 2.7/test/test_subprocess.py ... ====================================================================== FAIL: test_executable_without_cwd (__main__.ProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/garen/dev/bins//pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/test/test_subprocess.py", line 163, in test_executable_without_cwd self.assertEqual(p.returncode, 47) AssertionError: 1 != 47 ====================================================================== FAIL: test_executable_without_cwd (__main__.ProcessTestCaseNoPoll) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/garen/dev/bins//pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/test/test_subprocess.py", line 163, in test_executable_without_cwd self.assertEqual(p.returncode, 47) AssertionError: 1 != 47 Also, the test_subprocess.py shipped with the latest cpython 2.7.2.5 has several new tests that also fail: ====================================================================== ERROR: test_communicate_epipe_only_stdin (__main__.ProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 617, in test_communicate_epipe_only_stdin p.communicate("x" * 2**20) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 729, in communicate self.stdin.write(input) IOError: [Errno 32] Broken pipe: '<fdopen>' ====================================================================== ERROR: test_close_fd_0 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 830, in test_close_fd_0 self.check_close_std_fds([0]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_close_fd_1 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 833, in test_close_fd_1 self.check_close_std_fds([1]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_close_fd_2 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 836, in test_close_fd_2 self.check_close_std_fds([2]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_close_fds_0_1 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 839, in test_close_fds_0_1 self.check_close_std_fds([0, 1]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_close_fds_0_1_2 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 850, in test_close_fds_0_1_2 self.check_close_std_fds([0, 1, 2]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_close_fds_0_2 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 842, in test_close_fds_0_2 self.check_close_std_fds([0, 2]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_close_fds_1_2 (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 845, in test_close_fds_1_2 self.check_close_std_fds([1, 2]) File "./Python-2.7.2/Lib/test/test_subprocess.py", line 821, in check_close_std_fds err = test_support.strip_python_stderr(err) AttributeError: 'module' object has no attribute 'strip_python_stderr' ====================================================================== ERROR: test_communicate_epipe (__main__.ProcessTestCaseNoPoll) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 609, in test_communicate_epipe p.communicate("x" * 2**20) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 740, in communicate return self._communicate(input) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 1263, in _communicate stdout, stderr = self._communicate_with_select(input) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 1365, in _communicate_with_select bytes_written = os.write(self.stdin.fileno(), chunk) OSError: [Errno 32] Broken pipe ====================================================================== ERROR: test_communicate_epipe_only_stdin (__main__.ProcessTestCaseNoPoll) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 617, in test_communicate_epipe_only_stdin p.communicate("x" * 2**20) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 729, in communicate self.stdin.write(input) IOError: [Errno 32] Broken pipe: '<fdopen>' ====================================================================== FAIL: test_executable_without_cwd (__main__.ProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 163, in test_executable_without_cwd self.assertEqual(p.returncode, 47) AssertionError: 1 != 47 ====================================================================== FAIL: test_wait_when_sigchild_ignored (__main__.POSIXProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 860, in test_wait_when_sigchild_ignored " non-zero with this error:\n%s" % stderr) AssertionError: sigchild_ignore.py exited non-zero with this error: Traceback (most recent call last): File "app_main.py", line 51, in run_toplevel File "/home/garen/dev/bins/Python- 2.7.2/Lib/test/subprocessdata/sigchild_ignore.py", line 6, in <module> subprocess.Popen([sys.executable, '-c', 'print("albatross")']).wait() File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 1247, in wait pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/subprocess.py", line 471, in _eintr_retry_call return func(*args) OSError: [Errno 10] No child processes ====================================================================== FAIL: test_executable_without_cwd (__main__.ProcessTestCaseNoPoll) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Python-2.7.2/Lib/test/test_subprocess.py", line 163, in test_executable_without_cwd self.assertEqual(p.returncode, 47) AssertionError: 1 != 47 ---------------------------------------------------------------------- Ran 127 tests in 23.897s FAILED (failures=3, errors=10, skipped=14) Traceback (most recent call last): File "app_main.py", line 51, in run_toplevel File "./Python-2.7.2/Lib/test/test_subprocess.py", line 1053, in <module> test_main() File "./Python-2.7.2/Lib/test/test_subprocess.py", line 1049, in test_main test_support.run_unittest(*unit_tests) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/test/test_support.py", line 1138, in run_unittest _run_suite(suite) File "/home/garen/dev/bins/pypy-c-jit-51842-457e2166648c-linux/lib- python/modified-2.7/test/test_support.py", line 1092, in _run_suite raise TestFailed(err) TestFailed: multiple errors occurred At least a few of these appear to be related to some previously filed issues: https://bugs.pypy.org/issue934 https://bugs.pypy.org/issue828 ---------- messages: 3837 nosy: Garen, pypy-issue priority: bug release: ??? status: unread title: cpython test_subprocess.py fails ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue1029> ________________________________________

4 4

[issue1031] Speed of a tiny arithmetic program
by Armin Rigo 23 Apr '14

23 Apr '14

New submission from Armin Rigo <armin.rigo(a)gmail.com>: http://paste.pocoo.org/show/543141/ This program is not much faster with pypy. Even if I get rid of the obvious inefficiencies using divmod(..,2) and replace 2**r*d with d << r, the program ends up only twice as fast. Why? ---------- messages: 3845 nosy: arigo, pypy-issue priority: performance bug status: unread title: Speed of a tiny arithmetic program ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue1031> ________________________________________

4 4

[issue1027] Do something with longs
by Fijal 23 Apr '14

23 Apr '14

New submission from Fijal <fijall(a)gmail.com>: A bit no clue what, but performance of longs is fairly disastrous right now, so I suppose we should do something with it (not attaching any benchmarks, but *anything* is slow with them). ---------- messages: 3832 nosy: fijal, pypy-issue priority: performance bug status: unread title: Do something with longs ________________________________________ PyPy bug tracker <tracker(a)bugs.pypy.org> <https://bugs.pypy.org/issue1027> ________________________________________

2 1