Hi, I am using pypy in Debian: ondra@fuji:~/sympy$ python2.4 Python 2.4.5 (#2, Jul 20 2008, 20:55:34) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
Do you know what this error means and how should we fix sympy so that it works on top of pypy? Ondrej
Works for me: fijal@zygfryd:~$ pypy-c Python 2.4.1 (pypy 1.0.0 build 57237) on linux2 Type "help", "copyright", "credits" or "license" for more information. Welcome to rlcompleter2 0.96 for nice experiences hit <tab> multiple times And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
import sympy
If you have old pypy build try pypy-c --oldstyle, otherwise you can retranslate pypy. On Sun, Aug 24, 2008 at 4:08 PM, Ondrej Certik <ondrej@certik.cz> wrote:
On Mon, Aug 25, 2008 at 10:23 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hm, doesn't seem to work for me: $ svn co http://codespeak.net/svn/pypy/dist pypy-dist $ cd sympy/ $ python ~/repos/pypy-dist/pypy/bin/py.py faking <type 'module'> PyPy 1.0.0 in StdObjSpace on top of Python 2.5.2 (startuptime: 10.83 secs)
If you have old pypy build try pypy-c --oldstyle, otherwise you can retranslate pypy.
Old pypy + oldstyle doesn't seem to help: $ pypy --oldstyle Python 2.4.1 (pypy 1.0.0 build 55235) on linux2 Type "help", "copyright", "credits" or "license" for more information. ``I'm sorry, could you please not agree with the cat as well?''
So now I am trying to compile pypy-svn, but I doubt it will work if it doesn't work on top of python. Which sympy version are you using? Ondrej
On Mon, Aug 25, 2008 at 11:19 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Ok, I can reproduce it with newer version of sympy. Not sure what's wrong yet.
Exactly. That's what I wanted to ask you what's wrong. :) I think we are using some stupid hacks, but if you could enlighten me how to fix sympy so that it works on pypy, it'd be awesome. Ondrej
I'm not sure whether it's sympy's fault or pypy's fault. It might be that pypy is too strict about that. Arguments look like this: (Pdb++) bases[0].__mro__ (Function, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) bases[1].__mro__ (<class 'sympy.core.basic.Atom'>, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) cls <class 'sympy.core.function.FunctionClass'> (Pdb++) cls.__mro__ (<class 'sympy.core.function.FunctionClass'>, <class 'sympy.core.basic.BasicMeta'>, <class 'sympy.core.basic.BasicType'>, <type 'type'>, <type 'object'>) (Pdb++) And it seems that bases[0] and bases[1] are incompatible (why?) On Mon, Aug 25, 2008 at 11:45 AM, Ondrej Certik <ondrej@certik.cz> wrote:
On Mon, Aug 25, 2008 at 12:08 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Don't know. This patch fixes it: diff --git a/sympy/core/function.py b/sympy/core/function.py index 76a0f5b..a99c21d 100644 --- a/sympy/core/function.py +++ b/sympy/core/function.py @@ -362,7 +362,7 @@ class Function(Basic): x = sympify(x) return cls(x).diff(x, n).subs(x, 0) * x**n / C.Factorial(n) -class WildFunction(Function, Atom): +class WildFunction(Function): """ WildFunction() matches any expression but another WildFunction() XXX is this as intended, does it work ? But now there is this problem:
Any idea what's wrong here? Here is the code: @threaded() def cancel(f, *symbols): """Cancel common factors in a given rational function. and: def threaded(**flags): Seems a-ok to me. Is pypy using some strict things? I don't know, but this patch fixes it: diff --git a/sympy/utilities/decorator.py b/sympy/utilities/decorator.py index 0043de1..f812b51 100644 --- a/sympy/utilities/decorator.py +++ b/sympy/utilities/decorator.py @@ -3,7 +3,7 @@ from sympy.core.add import Add from sympy.core.sympify import sympify from sympy.core.relational import Relational -def threaded(**flags): +def threaded(*args, **flags): Now it imports. Wow!
Guys, this is awesome! I did some timings: $ ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``Python 300''
Compare to python: $ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
So pypy is roughly 4.7x slower on this particular thing. Do you have any plans to release pypy? I think it's getting very useful. Ondrej
Yes, we have some plans to have a release at some point soon. There are no details though. Note that if you do this: from sympy import * var('x') from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 100); t = clock()-t; print t var('y') t = clock(); a= sin(y).series(y, 0, 100); t = clock()-t; print gap between pypy and cpython is getting smaller (some startup time is particularly large). I can do a bit profiling if you like. Regarding threaded - I think you should not assume that your argument is passed as keyword arg when using decorator I think. The other one looks like bug in pypy, but I'm not 100% sure. Cheers, fijal
On Mon, Aug 25, 2008 at 1:04 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Use SYMPY_USE_CACHE=no when doing repeated profiling, otherwise you are just measuring how fast our cache is. ondra@pc232:~/repos/sympy$ SYMPY_USE_CACHE=no ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``- PyPy status blog: http://morepypy.blogspot.com/''
ondra@pc232:~/repos/sympy$ SYMPY_USE_CACHE=no python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
We are in the process of speeding up sympy, you can also play with our experimental core that doesn't use caching: $ git clone git://github.com/certik/sympyx.git $ cd sympyx $ ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
$ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
So for that pypy is only 2x to 3x slower. BTW, when you compile our core using Cython, it get's way faster: $ make cython --convert-range sympy_pyx.pyx gcc -I/usr/include/python2.5 -I/usr/include/python2.5 -g -O0 -fPIC -c -o sympy_pyx.o sympy_pyx.c gcc -shared sympy_pyx.o -o sympy_pyx.so $ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
Do you have any plans for supporting writing C extensions to pypy? Because as you can see above, the speed is imho only possible when writing it in C. Well, if RPython could produce as optimized code as Cython, then it could be an option. Any ideas on that? Ondrej
There are no immediate plans for supporting C extensions to pypy besides via ctypes or rpython. RPython is fairly fast these days, try it out if you like (big fat warning, rpython is an obscure language). Also usecases like sympy one should see relatively good speedups via JIT. Did you try using psyco? Cheers, fijal
On Mon, Aug 25, 2008 at 10:26 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Psyco gives pretty good results (but not as good as Cython), but as I understood, it is not developed anymore as I thought pypy is the way to go. Yes, Cython is also a new language basically, but it's very easy to learn and easy to debug, easy to compile and one is able to blend C and Python very easily and naturally, so it's my choice. But on the other hand, I really think that one should just write what he wants in Python, maybe give it couple hints and pypy (or anything else) should be clever enough to optimize it. Cython is not perfect, but one can get C level speed now. They plan to allow a pure python syntax, so that the same code actually also runs in Python. But anyway, pypy has my thumbs up. 3x slower than CPython is not *that* bad, so imho if you could release, it'd be awesome. Ondrej
On Tue, Aug 26, 2008 at 10:27 PM, Simon Burton <simon@arrowtheory.com> wrote:
Does it? Wow, didn't know about it. How do you write this in pure python syntax? def f(int i, int j): return i*j ? What I mean is to use some decorator, or some other (python) syntax to represent the "int" in there. Ondrej
Hi Ondrej, On Sun, Aug 24, 2008 at 04:08:29PM +0200, Ondrej Certik wrote:
TypeError: instance layout conflicts in multiple inheritance
Not completely surprizingly, our multiple inheritance "instance layout conflict" condition is not 100% the same as CPython's (as the latter is really obscure and based on conditions like the size of C-level structures). Still, I think that this case should work, so it looks like a bug in PyPy. Filed issue #390: https://codespeak.net/issue/pypy-dev/issue390 Armin
On Mon, Aug 25, 2008 at 4:04 PM, Armin Rigo <arigo@tunes.org> wrote:
Thanks Armin for looking into this. Btw, if it helps, ironpython also fails on conflict1.py from #390: $ ipy conflict1.py Traceback (most recent call last): File conflict1, line unknown, in Initialize File mscorlib, line unknown, in get_Item KeyError: The given key was not present in the dictionary. Also I don't like mutliple inheritance, so we should just fix sympy and be done with it. Unfortunately IronPython fails later too and Jython as well: $ /home/ondra/ext/jython/jython Jython 2.5a1+ (asm:4943:4945, Jul 15 2008, 15:30:04) [Java HotSpot(TM) Server VM (Sun Microsystems Inc.)] on java1.5.0_15 Type "help", "copyright", "credits" or "license" for more information.
So this suggests that sympy code is not standard. Ondrej
Hi Ondrej, On Mon, Aug 25, 2008 at 04:20:40PM +0200, Ondrej Certik wrote:
That's obscure. It probably points to a bug in Jython and IronPython, or just some feature that is really obscure but that PyPy implements like CPython by chance. I fixed the multiple inheritance bug as well as the bug shown by the @threaded() decorator. Since rev 57656, pypy-c can import sympy. (I didn't try to do anything with it, though.) A bientot, Armin.
Works for me: fijal@zygfryd:~$ pypy-c Python 2.4.1 (pypy 1.0.0 build 57237) on linux2 Type "help", "copyright", "credits" or "license" for more information. Welcome to rlcompleter2 0.96 for nice experiences hit <tab> multiple times And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
import sympy
If you have old pypy build try pypy-c --oldstyle, otherwise you can retranslate pypy. On Sun, Aug 24, 2008 at 4:08 PM, Ondrej Certik <ondrej@certik.cz> wrote:
On Mon, Aug 25, 2008 at 10:23 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hm, doesn't seem to work for me: $ svn co http://codespeak.net/svn/pypy/dist pypy-dist $ cd sympy/ $ python ~/repos/pypy-dist/pypy/bin/py.py faking <type 'module'> PyPy 1.0.0 in StdObjSpace on top of Python 2.5.2 (startuptime: 10.83 secs)
If you have old pypy build try pypy-c --oldstyle, otherwise you can retranslate pypy.
Old pypy + oldstyle doesn't seem to help: $ pypy --oldstyle Python 2.4.1 (pypy 1.0.0 build 55235) on linux2 Type "help", "copyright", "credits" or "license" for more information. ``I'm sorry, could you please not agree with the cat as well?''
So now I am trying to compile pypy-svn, but I doubt it will work if it doesn't work on top of python. Which sympy version are you using? Ondrej
On Mon, Aug 25, 2008 at 11:19 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Ok, I can reproduce it with newer version of sympy. Not sure what's wrong yet.
Exactly. That's what I wanted to ask you what's wrong. :) I think we are using some stupid hacks, but if you could enlighten me how to fix sympy so that it works on pypy, it'd be awesome. Ondrej
I'm not sure whether it's sympy's fault or pypy's fault. It might be that pypy is too strict about that. Arguments look like this: (Pdb++) bases[0].__mro__ (Function, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) bases[1].__mro__ (<class 'sympy.core.basic.Atom'>, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) cls <class 'sympy.core.function.FunctionClass'> (Pdb++) cls.__mro__ (<class 'sympy.core.function.FunctionClass'>, <class 'sympy.core.basic.BasicMeta'>, <class 'sympy.core.basic.BasicType'>, <type 'type'>, <type 'object'>) (Pdb++) And it seems that bases[0] and bases[1] are incompatible (why?) On Mon, Aug 25, 2008 at 11:45 AM, Ondrej Certik <ondrej@certik.cz> wrote:
On Mon, Aug 25, 2008 at 12:08 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Don't know. This patch fixes it: diff --git a/sympy/core/function.py b/sympy/core/function.py index 76a0f5b..a99c21d 100644 --- a/sympy/core/function.py +++ b/sympy/core/function.py @@ -362,7 +362,7 @@ class Function(Basic): x = sympify(x) return cls(x).diff(x, n).subs(x, 0) * x**n / C.Factorial(n) -class WildFunction(Function, Atom): +class WildFunction(Function): """ WildFunction() matches any expression but another WildFunction() XXX is this as intended, does it work ? But now there is this problem:
Any idea what's wrong here? Here is the code: @threaded() def cancel(f, *symbols): """Cancel common factors in a given rational function. and: def threaded(**flags): Seems a-ok to me. Is pypy using some strict things? I don't know, but this patch fixes it: diff --git a/sympy/utilities/decorator.py b/sympy/utilities/decorator.py index 0043de1..f812b51 100644 --- a/sympy/utilities/decorator.py +++ b/sympy/utilities/decorator.py @@ -3,7 +3,7 @@ from sympy.core.add import Add from sympy.core.sympify import sympify from sympy.core.relational import Relational -def threaded(**flags): +def threaded(*args, **flags): Now it imports. Wow!
Guys, this is awesome! I did some timings: $ ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``Python 300''
Compare to python: $ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
So pypy is roughly 4.7x slower on this particular thing. Do you have any plans to release pypy? I think it's getting very useful. Ondrej
Yes, we have some plans to have a release at some point soon. There are no details though. Note that if you do this: from sympy import * var('x') from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 100); t = clock()-t; print t var('y') t = clock(); a= sin(y).series(y, 0, 100); t = clock()-t; print gap between pypy and cpython is getting smaller (some startup time is particularly large). I can do a bit profiling if you like. Regarding threaded - I think you should not assume that your argument is passed as keyword arg when using decorator I think. The other one looks like bug in pypy, but I'm not 100% sure. Cheers, fijal
On Mon, Aug 25, 2008 at 1:04 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Use SYMPY_USE_CACHE=no when doing repeated profiling, otherwise you are just measuring how fast our cache is. ondra@pc232:~/repos/sympy$ SYMPY_USE_CACHE=no ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``- PyPy status blog: http://morepypy.blogspot.com/''
ondra@pc232:~/repos/sympy$ SYMPY_USE_CACHE=no python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
We are in the process of speeding up sympy, you can also play with our experimental core that doesn't use caching: $ git clone git://github.com/certik/sympyx.git $ cd sympyx $ ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
$ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
So for that pypy is only 2x to 3x slower. BTW, when you compile our core using Cython, it get's way faster: $ make cython --convert-range sympy_pyx.pyx gcc -I/usr/include/python2.5 -I/usr/include/python2.5 -g -O0 -fPIC -c -o sympy_pyx.o sympy_pyx.c gcc -shared sympy_pyx.o -o sympy_pyx.so $ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
Do you have any plans for supporting writing C extensions to pypy? Because as you can see above, the speed is imho only possible when writing it in C. Well, if RPython could produce as optimized code as Cython, then it could be an option. Any ideas on that? Ondrej
There are no immediate plans for supporting C extensions to pypy besides via ctypes or rpython. RPython is fairly fast these days, try it out if you like (big fat warning, rpython is an obscure language). Also usecases like sympy one should see relatively good speedups via JIT. Did you try using psyco? Cheers, fijal
On Mon, Aug 25, 2008 at 10:26 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Psyco gives pretty good results (but not as good as Cython), but as I understood, it is not developed anymore as I thought pypy is the way to go. Yes, Cython is also a new language basically, but it's very easy to learn and easy to debug, easy to compile and one is able to blend C and Python very easily and naturally, so it's my choice. But on the other hand, I really think that one should just write what he wants in Python, maybe give it couple hints and pypy (or anything else) should be clever enough to optimize it. Cython is not perfect, but one can get C level speed now. They plan to allow a pure python syntax, so that the same code actually also runs in Python. But anyway, pypy has my thumbs up. 3x slower than CPython is not *that* bad, so imho if you could release, it'd be awesome. Ondrej
On Tue, Aug 26, 2008 at 10:27 PM, Simon Burton <simon@arrowtheory.com> wrote:
Does it? Wow, didn't know about it. How do you write this in pure python syntax? def f(int i, int j): return i*j ? What I mean is to use some decorator, or some other (python) syntax to represent the "int" in there. Ondrej
Hi Ondrej, On Sun, Aug 24, 2008 at 04:08:29PM +0200, Ondrej Certik wrote:
TypeError: instance layout conflicts in multiple inheritance
Not completely surprizingly, our multiple inheritance "instance layout conflict" condition is not 100% the same as CPython's (as the latter is really obscure and based on conditions like the size of C-level structures). Still, I think that this case should work, so it looks like a bug in PyPy. Filed issue #390: https://codespeak.net/issue/pypy-dev/issue390 Armin
On Mon, Aug 25, 2008 at 4:04 PM, Armin Rigo <arigo@tunes.org> wrote:
Thanks Armin for looking into this. Btw, if it helps, ironpython also fails on conflict1.py from #390: $ ipy conflict1.py Traceback (most recent call last): File conflict1, line unknown, in Initialize File mscorlib, line unknown, in get_Item KeyError: The given key was not present in the dictionary. Also I don't like mutliple inheritance, so we should just fix sympy and be done with it. Unfortunately IronPython fails later too and Jython as well: $ /home/ondra/ext/jython/jython Jython 2.5a1+ (asm:4943:4945, Jul 15 2008, 15:30:04) [Java HotSpot(TM) Server VM (Sun Microsystems Inc.)] on java1.5.0_15 Type "help", "copyright", "credits" or "license" for more information.
So this suggests that sympy code is not standard. Ondrej
Hi Ondrej, On Mon, Aug 25, 2008 at 04:20:40PM +0200, Ondrej Certik wrote:
That's obscure. It probably points to a bug in Jython and IronPython, or just some feature that is really obscure but that PyPy implements like CPython by chance. I fixed the multiple inheritance bug as well as the bug shown by the @threaded() decorator. Since rev 57656, pypy-c can import sympy. (I didn't try to do anything with it, though.) A bientot, Armin.
participants (4)
-
Armin Rigo -
Maciej Fijalkowski -
Ondrej Certik -
Simon Burton