pypy 2.1 pip install rbtree
>>>>>>>>>>>>>>>>>>>>>>>>> Downloading/unpacking rbtree Running setup.py egg_info for package rbtree
Installing collected packages: rbtree Running setup.py install for rbtree building 'rbtree' extension cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree_impl.c -o build/temp.linux-x86_64-2.7/src/rbtree_impl.o cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree.c -o build/temp.linux-x86_64-2.7/src/rbtree.o src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': src/rbtree.c:2427:48: error: 'PyRange_Type' undeclared (first use in this function) src/rbtree.c:2427:48: note: each undeclared identifier is reported only once for each function it appears in src/rbtree.c: In function '__Pyx_GetException': src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4996:11: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4997:11: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4998:11: error: 'PyThreadState' has no member named 'exc_traceback' error: command 'cc' failed with exit status 1 Complete output from command /pypy/bin/pypy -c "import setuptools;__file__='/pypy/build/rbtree/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-s2EnAa-record/install-record.txt --install-headers /pypy/include/site/python2.7: running install running build running build_ext building 'rbtree' extension cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree_impl.c -o build/temp.linux-x86_64-2.7/src/rbtree_impl.o cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree.c -o build/temp.linux-x86_64-2.7/src/rbtree.o src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': src/rbtree.c:2427:48: error: 'PyRange_Type' undeclared (first use in this function) src/rbtree.c:2427:48: note: each undeclared identifier is reported only once for each function it appears in src/rbtree.c: In function '__Pyx_GetException': src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4996:11: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4997:11: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4998:11: error: 'PyThreadState' has no member named 'exc_traceback' error: command 'cc' failed with exit status 1 ---------------------------------------- Command /pypy/bin/pypy -c "import setuptools;__file__='/pypy/build/rbtree/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-s2EnAa-record/install-record.txt --install-headers /pypy/include/site/python2.7 failed with error code 1 in /pypy/build/rbtree Storing complete log in /home/chenjiaming/.pip/pip.log
Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bientôt, Armin.
Tks! but
>>>>>>>>> This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). >>>>>>>>>
why? ------------------ Original ------------------ From: "Armin Rigo";<arigo@tunes.org>; Date: Thu, Oct 31, 2013 04:53 PM To: "KaShining"<mail2shine@qq.com>; Cc: "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bientôt, Armin.
2013/10/31 KaShining <mail2shine@qq.com>
Tks!
but
>>>>>>>>>> This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). >>>>>>>>>>
why?
See this blog post: http://morepypy.blogspot.ch/2011/05/numpy-follow-up.html specially the paragraph starting with "cpyext is slow"
------------------ Original ------------------ *From: * "Armin Rigo";<arigo@tunes.org>; *Date: * Thu, Oct 31, 2013 04:53 PM *To: * "KaShining"<mail2shine@qq.com>; ** *Cc: * "pypy-dev"<pypy-dev@python.org>; ** *Subject: * Re: [pypy-dev] pip install rbtree fail in pypy env!
Hi KaShining,
On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself.
Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version.
A bientôt,
Armin.
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Amaury Forgeot d'Arc
On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo@tunes.org> wrote:
Hi KaShining,
On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky.
Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such.
tks,but
test-red_black_dict_mod.py under pypy-2.1. consume :3.89574193954 import sys import red_black_dict_mod import time import random
t = red_black_dict_mod.RedBlackTree() begin = time.time() for i in range(100000): k = random.randint(0, 10000000) t[k] = k end = time.time() print end - begin while
test-rbtree.py under python2.7.3 .consume:0.35432600975 import sys import rbtree import time import random
t = rbtree.rbtree() begin = time.time() for i in range(100000): k = random.randint(0, 10000000) t[k] = k end = time.time() print end - begin ------------------ Original ------------------ From: "Dan Stromberg";<drsalists@gmail.com>; Date: Fri, Nov 1, 2013 06:29 AM To: "Armin Rigo"<arigo@tunes.org>; Cc: "KaShining"<mail2shine@qq.com>; "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo@tunes.org> wrote: Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such.
On Thu, Oct 31, 2013 at 8:07 PM, KaShining <mail2shine@qq.com> wrote:
tks,but
test-red_black_dict_mod.py under pypy-2.1. consume :3.89574193954 test-rbtree.py under python2.7.3 .consume:0.35432600975s@gmail.com<drsalists@gmail.com> ;
Interesting. Do you need good performance? You might consider a treap: https://pypi.python.org/pypi/treap It has a cython version, as well as pure python. Supposedly treaps outperform red black trees on average, but give a larger standard deviation in operation times. I've used this treap code on CPython 2.x, CPython 3.x, Pypy, Pypy3 (beta) and Jython.
Hi KaShining, Yes, the issue then is that pure Python equivalents are often slower than their C version --- for example, this red_black_tree_mod file is huge for the job (more than 1000 lines), written in a Java-ish style with tons of small overridden methods, but full of properties, recursive function invocations, and so on. Getting 10 times slower than the C version is not good, but I guess that the C version was more optimized. Often we can hope to get at most a few times slower. PyPy is not faster at everything. Usually there is no way it can be faster than a program that exercices only a CPython C extension module. In some cases like this one, the solutions you have with PyPy are all a lot slower --- both some pure Python version, and the cpyext version. The pure Python version could be rewritten with performance-on-PyPy in mind (just like the C version was written with performance-on-CPython in mind). Note that the point of red_black_tree_mod is very unclear to me. It seems to replace plain dicts. Why can't you use plain dicts? A bientôt, Armin.
I found bintrees-fastrbtree is the best choice： （some test code： def test_dict(tree, msg): begin = time.time() for i in range(100000): k = random.randint(0, 10000000) tree[k] = k end = time.time() print msg,end - begin ） bintrees_rbtree under cpython 0.393133878708 bintrees_fast_rbtree under pypy 0.357424020767 ------------------ Original ------------------ From: "Armin Rigo";<arigo@tunes.org>; Date: Fri, Nov 1, 2013 05:06 PM To: "KaShining"<mail2shine@qq.com>; Cc: "Dan Stromberg"<drsalists@gmail.com>; "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, Yes, the issue then is that pure Python equivalents are often slower than their C version --- for example, this red_black_tree_mod file is huge for the job (more than 1000 lines), written in a Java-ish style with tons of small overridden methods, but full of properties, recursive function invocations, and so on. Getting 10 times slower than the C version is not good, but I guess that the C version was more optimized. Often we can hope to get at most a few times slower. PyPy is not faster at everything. Usually there is no way it can be faster than a program that exercices only a CPython C extension module. In some cases like this one, the solutions you have with PyPy are all a lot slower --- both some pure Python version, and the cpyext version. The pure Python version could be rewritten with performance-on-PyPy in mind (just like the C version was written with performance-on-CPython in mind). Note that the point of red_black_tree_mod is very unclear to me. It seems to replace plain dicts. Why can't you use plain dicts? A bientôt, Armin.
participants (4)
-
Amaury Forgeot d'Arc -
Armin Rigo -
Dan Stromberg -
KaShining