pypy 2.1 pip install rbtree
Installing collected packages: rbtree Running setup.py install for rbtree building 'rbtree' extension cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree_impl.c -o build/temp.linux-x86_64-2.7/src/rbtree_impl.o cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree.c -o build/temp.linux-x86_64-2.7/src/rbtree.o src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': src/rbtree.c:2427:48: error: 'PyRange_Type' undeclared (first use in this function) src/rbtree.c:2427:48: note: each undeclared identifier is reported only once for each function it appears in src/rbtree.c: In function '__Pyx_GetException': src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4996:11: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4997:11: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4998:11: error: 'PyThreadState' has no member named 'exc_traceback' error: command 'cc' failed with exit status 1 Complete output from command /pypy/bin/pypy -c "import setuptools;__file__='/pypy/build/rbtree/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-s2EnAa-record/install-record.txt --install-headers /pypy/include/site/python2.7: running install running build running build_ext building 'rbtree' extension cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree_impl.c -o build/temp.linux-x86_64-2.7/src/rbtree_impl.o cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree.c -o build/temp.linux-x86_64-2.7/src/rbtree.o src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': src/rbtree.c:2427:48: error: 'PyRange_Type' undeclared (first use in this function) src/rbtree.c:2427:48: note: each undeclared identifier is reported only once for each function it appears in src/rbtree.c: In function '__Pyx_GetException': src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4996:11: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4997:11: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4998:11: error: 'PyThreadState' has no member named 'exc_traceback' error: command 'cc' failed with exit status 1 ---------------------------------------- Command /pypy/bin/pypy -c "import setuptools;__file__='/pypy/build/rbtree/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-s2EnAa-record/install-record.txt --install-headers /pypy/include/site/python2.7 failed with error code 1 in /pypy/build/rbtree Storing complete log in /home/chenjiaming/.pip/pip.log
Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bientôt, Armin.
Tks! but
why? ------------------ Original ------------------ From: "Armin Rigo";<arigo@tunes.org>; Date: Thu, Oct 31, 2013 04:53 PM To: "KaShining"<mail2shine@qq.com>; Cc: "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bientôt, Armin.
2013/10/31 KaShining <mail2shine@qq.com>
See this blog post: http://morepypy.blogspot.ch/2011/05/numpy-follow-up.html specially the paragraph starting with "cpyext is slow"
-- Amaury Forgeot d'Arc
On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo@tunes.org> wrote:
Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such.
tks,but
t = red_black_dict_mod.RedBlackTree() begin = time.time() for i in range(100000): k = random.randint(0, 10000000) t[k] = k end = time.time() print end - begin while
t = rbtree.rbtree() begin = time.time() for i in range(100000): k = random.randint(0, 10000000) t[k] = k end = time.time() print end - begin ------------------ Original ------------------ From: "Dan Stromberg";<drsalists@gmail.com>; Date: Fri, Nov 1, 2013 06:29 AM To: "Armin Rigo"<arigo@tunes.org>; Cc: "KaShining"<mail2shine@qq.com>; "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo@tunes.org> wrote: Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such.
On Thu, Oct 31, 2013 at 8:07 PM, KaShining <mail2shine@qq.com> wrote:
Interesting. Do you need good performance? You might consider a treap: https://pypi.python.org/pypi/treap It has a cython version, as well as pure python. Supposedly treaps outperform red black trees on average, but give a larger standard deviation in operation times. I've used this treap code on CPython 2.x, CPython 3.x, Pypy, Pypy3 (beta) and Jython.
Hi KaShining, Yes, the issue then is that pure Python equivalents are often slower than their C version --- for example, this red_black_tree_mod file is huge for the job (more than 1000 lines), written in a Java-ish style with tons of small overridden methods, but full of properties, recursive function invocations, and so on. Getting 10 times slower than the C version is not good, but I guess that the C version was more optimized. Often we can hope to get at most a few times slower. PyPy is not faster at everything. Usually there is no way it can be faster than a program that exercices only a CPython C extension module. In some cases like this one, the solutions you have with PyPy are all a lot slower --- both some pure Python version, and the cpyext version. The pure Python version could be rewritten with performance-on-PyPy in mind (just like the C version was written with performance-on-CPython in mind). Note that the point of red_black_tree_mod is very unclear to me. It seems to replace plain dicts. Why can't you use plain dicts? A bientôt, Armin.
I found bintrees-fastrbtree is the best choice： （some test code： def test_dict(tree, msg): begin = time.time() for i in range(100000): k = random.randint(0, 10000000) tree[k] = k end = time.time() print msg,end - begin ） bintrees_rbtree under cpython 0.393133878708 bintrees_fast_rbtree under pypy 0.357424020767 ------------------ Original ------------------ From: "Armin Rigo";<arigo@tunes.org>; Date: Fri, Nov 1, 2013 05:06 PM To: "KaShining"<mail2shine@qq.com>; Cc: "Dan Stromberg"<drsalists@gmail.com>; "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, Yes, the issue then is that pure Python equivalents are often slower than their C version --- for example, this red_black_tree_mod file is huge for the job (more than 1000 lines), written in a Java-ish style with tons of small overridden methods, but full of properties, recursive function invocations, and so on. Getting 10 times slower than the C version is not good, but I guess that the C version was more optimized. Often we can hope to get at most a few times slower. PyPy is not faster at everything. Usually there is no way it can be faster than a program that exercices only a CPython C extension module. In some cases like this one, the solutions you have with PyPy are all a lot slower --- both some pure Python version, and the cpyext version. The pure Python version could be rewritten with performance-on-PyPy in mind (just like the C version was written with performance-on-CPython in mind). Note that the point of red_black_tree_mod is very unclear to me. It seems to replace plain dicts. Why can't you use plain dicts? A bientôt, Armin.
Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bientôt, Armin.
Tks! but
why? ------------------ Original ------------------ From: "Armin Rigo";<arigo@tunes.org>; Date: Thu, Oct 31, 2013 04:53 PM To: "KaShining"<mail2shine@qq.com>; Cc: "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bientôt, Armin.
2013/10/31 KaShining <mail2shine@qq.com>
See this blog post: http://morepypy.blogspot.ch/2011/05/numpy-follow-up.html specially the paragraph starting with "cpyext is slow"
-- Amaury Forgeot d'Arc
On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo@tunes.org> wrote:
Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such.
tks,but
t = red_black_dict_mod.RedBlackTree() begin = time.time() for i in range(100000): k = random.randint(0, 10000000) t[k] = k end = time.time() print end - begin while
t = rbtree.rbtree() begin = time.time() for i in range(100000): k = random.randint(0, 10000000) t[k] = k end = time.time() print end - begin ------------------ Original ------------------ From: "Dan Stromberg";<drsalists@gmail.com>; Date: Fri, Nov 1, 2013 06:29 AM To: "Armin Rigo"<arigo@tunes.org>; Cc: "KaShining"<mail2shine@qq.com>; "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo <arigo@tunes.org> wrote: Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining <mail2shine@qq.com> wrote:
src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset':
This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such.
On Thu, Oct 31, 2013 at 8:07 PM, KaShining <mail2shine@qq.com> wrote:
Interesting. Do you need good performance? You might consider a treap: https://pypi.python.org/pypi/treap It has a cython version, as well as pure python. Supposedly treaps outperform red black trees on average, but give a larger standard deviation in operation times. I've used this treap code on CPython 2.x, CPython 3.x, Pypy, Pypy3 (beta) and Jython.
Hi KaShining, Yes, the issue then is that pure Python equivalents are often slower than their C version --- for example, this red_black_tree_mod file is huge for the job (more than 1000 lines), written in a Java-ish style with tons of small overridden methods, but full of properties, recursive function invocations, and so on. Getting 10 times slower than the C version is not good, but I guess that the C version was more optimized. Often we can hope to get at most a few times slower. PyPy is not faster at everything. Usually there is no way it can be faster than a program that exercices only a CPython C extension module. In some cases like this one, the solutions you have with PyPy are all a lot slower --- both some pure Python version, and the cpyext version. The pure Python version could be rewritten with performance-on-PyPy in mind (just like the C version was written with performance-on-CPython in mind). Note that the point of red_black_tree_mod is very unclear to me. It seems to replace plain dicts. Why can't you use plain dicts? A bientôt, Armin.
I found bintrees-fastrbtree is the best choice： （some test code： def test_dict(tree, msg): begin = time.time() for i in range(100000): k = random.randint(0, 10000000) tree[k] = k end = time.time() print msg,end - begin ） bintrees_rbtree under cpython 0.393133878708 bintrees_fast_rbtree under pypy 0.357424020767 ------------------ Original ------------------ From: "Armin Rigo";<arigo@tunes.org>; Date: Fri, Nov 1, 2013 05:06 PM To: "KaShining"<mail2shine@qq.com>; Cc: "Dan Stromberg"<drsalists@gmail.com>; "pypy-dev"<pypy-dev@python.org>; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, Yes, the issue then is that pure Python equivalents are often slower than their C version --- for example, this red_black_tree_mod file is huge for the job (more than 1000 lines), written in a Java-ish style with tons of small overridden methods, but full of properties, recursive function invocations, and so on. Getting 10 times slower than the C version is not good, but I guess that the C version was more optimized. Often we can hope to get at most a few times slower. PyPy is not faster at everything. Usually there is no way it can be faster than a program that exercices only a CPython C extension module. In some cases like this one, the solutions you have with PyPy are all a lot slower --- both some pure Python version, and the cpyext version. The pure Python version could be rewritten with performance-on-PyPy in mind (just like the C version was written with performance-on-CPython in mind). Note that the point of red_black_tree_mod is very unclear to me. It seems to replace plain dicts. Why can't you use plain dicts? A bientôt, Armin.
participants (4)
-
Amaury Forgeot d'Arc -
Armin Rigo -
Dan Stromberg -
KaShining