Hi, I am using pypy in Debian: ondra@fuji:~/sympy$ python2.4 Python 2.4.5 (#2, Jul 20 2008, 20:55:34) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import sympy
ondra@fuji:~/sympy$ pypy Python 2.4.1 (pypy 1.0.0 build 55235) on linux2 Type "help", "copyright", "credits" or "license" for more information. ``the 'super' keyword is not that huggable''
import sympy Traceback (most recent call last): File "<console>", line 1, in <interactive> File "/home/ondra/sympy/sympy/__init__.py", line 16, in <module> from sympy.core import * File "/home/ondra/sympy/sympy/core/__init__.py", line 13, in <module> from function import Lambda, WildFunction, Derivative, diff, FunctionClass, \ File "/home/ondra/sympy/sympy/core/function.py", line 361, in <module> class WildFunction(Function, Atom): File "/home/ondra/sympy/sympy/core/function.py", line 68, in __new__ return type.__new__(cls, name, bases, attrdict) TypeError: instance layout conflicts in multiple inheritance
Do you know what this error means and how should we fix sympy so that it works on top of pypy? Ondrej
Works for me: fijal@zygfryd:~$ pypy-c Python 2.4.1 (pypy 1.0.0 build 57237) on linux2 Type "help", "copyright", "credits" or "license" for more information. Welcome to rlcompleter2 0.96 for nice experiences hit <tab> multiple times And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
import sympy
If you have old pypy build try pypy-c --oldstyle, otherwise you can retranslate pypy. On Sun, Aug 24, 2008 at 4:08 PM, Ondrej Certik <ondrej@certik.cz> wrote:
Hi,
I am using pypy in Debian:
ondra@fuji:~/sympy$ python2.4 Python 2.4.5 (#2, Jul 20 2008, 20:55:34) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import sympy
ondra@fuji:~/sympy$ pypy Python 2.4.1 (pypy 1.0.0 build 55235) on linux2 Type "help", "copyright", "credits" or "license" for more information. ``the 'super' keyword is not that huggable''
import sympy Traceback (most recent call last): File "<console>", line 1, in <interactive> File "/home/ondra/sympy/sympy/__init__.py", line 16, in <module> from sympy.core import * File "/home/ondra/sympy/sympy/core/__init__.py", line 13, in <module> from function import Lambda, WildFunction, Derivative, diff, FunctionClass, \ File "/home/ondra/sympy/sympy/core/function.py", line 361, in <module> class WildFunction(Function, Atom): File "/home/ondra/sympy/sympy/core/function.py", line 68, in __new__ return type.__new__(cls, name, bases, attrdict) TypeError: instance layout conflicts in multiple inheritance
Do you know what this error means and how should we fix sympy so that it works on top of pypy?
Ondrej _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Mon, Aug 25, 2008 at 10:23 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Works for me:
fijal@zygfryd:~$ pypy-c Python 2.4.1 (pypy 1.0.0 build 57237) on linux2 Type "help", "copyright", "credits" or "license" for more information. Welcome to rlcompleter2 0.96 for nice experiences hit <tab> multiple times And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
import sympy
Hm, doesn't seem to work for me: $ svn co http://codespeak.net/svn/pypy/dist pypy-dist $ cd sympy/ $ python ~/repos/pypy-dist/pypy/bin/py.py faking <type 'module'> PyPy 1.0.0 in StdObjSpace on top of Python 2.5.2 (startuptime: 10.83 secs)
import sympy Traceback (application-level): File "<inline>", line 1 in <interactive> import sympy File "/home/ondra/sympy/sympy/__init__.py", line 16 in <module> from sympy.core import * File "/home/ondra/sympy/sympy/core/__init__.py", line 13 in <module> from function import Lambda, WildFunction, Derivative, diff, FunctionClass, \ File "/home/ondra/sympy/sympy/core/function.py", line 361 in <module> class WildFunction(Function, Atom): File "/home/ondra/sympy/sympy/core/function.py", line 68 in __new__ return type.__new__(cls, name, bases, attrdict) TypeError: instance layout conflicts in multiple inheritance
If you have old pypy build try pypy-c --oldstyle, otherwise you can retranslate pypy.
Old pypy + oldstyle doesn't seem to help: $ pypy --oldstyle Python 2.4.1 (pypy 1.0.0 build 55235) on linux2 Type "help", "copyright", "credits" or "license" for more information. ``I'm sorry, could you please not agree with the cat as well?''
import sympy Traceback (most recent call last): File "<console>", line 1, in <interactive> File "/home/ondra/sympy/sympy/__init__.py", line 16, in <module> from sympy.core import * File "/home/ondra/sympy/sympy/core/__init__.py", line 13, in <module> from function import Lambda, WildFunction, Derivative, diff, FunctionClass, \ File "/home/ondra/sympy/sympy/core/function.py", line 361, in <module> class WildFunction(Function, Atom): File "/home/ondra/sympy/sympy/core/function.py", line 68, in __new__ return type.__new__(cls, name, bases, attrdict) TypeError: instance layout conflicts in multiple inheritance
So now I am trying to compile pypy-svn, but I doubt it will work if it doesn't work on top of python. Which sympy version are you using? Ondrej
Ok, I can reproduce it with newer version of sympy. Not sure what's wrong yet. On Mon, Aug 25, 2008 at 10:51 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
So now I am trying to compile pypy-svn, but I doubt it will work if it doesn't work on top of python. Which sympy version are you using?
Ondrej
0.4.2-1, I'll try newer one.
On Mon, Aug 25, 2008 at 11:19 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Ok, I can reproduce it with newer version of sympy. Not sure what's wrong yet.
Exactly. That's what I wanted to ask you what's wrong. :) I think we are using some stupid hacks, but if you could enlighten me how to fix sympy so that it works on pypy, it'd be awesome. Ondrej
I'm not sure whether it's sympy's fault or pypy's fault. It might be that pypy is too strict about that. Arguments look like this: (Pdb++) bases[0].__mro__ (Function, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) bases[1].__mro__ (<class 'sympy.core.basic.Atom'>, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) cls <class 'sympy.core.function.FunctionClass'> (Pdb++) cls.__mro__ (<class 'sympy.core.function.FunctionClass'>, <class 'sympy.core.basic.BasicMeta'>, <class 'sympy.core.basic.BasicType'>, <type 'type'>, <type 'object'>) (Pdb++) And it seems that bases[0] and bases[1] are incompatible (why?) On Mon, Aug 25, 2008 at 11:45 AM, Ondrej Certik <ondrej@certik.cz> wrote:
On Mon, Aug 25, 2008 at 11:19 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Ok, I can reproduce it with newer version of sympy. Not sure what's wrong yet.
Exactly. That's what I wanted to ask you what's wrong. :)
I think we are using some stupid hacks, but if you could enlighten me how to fix sympy so that it works on pypy, it'd be awesome.
Ondrej
On Mon, Aug 25, 2008 at 12:08 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
I'm not sure whether it's sympy's fault or pypy's fault. It might be that pypy is too strict about that.
Arguments look like this:
(Pdb++) bases[0].__mro__ (Function, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) bases[1].__mro__ (<class 'sympy.core.basic.Atom'>, <class 'sympy.core.basic.Basic'>, <class 'sympy.core.assumptions.AssumeMeths'>, <type 'object'>) (Pdb++) cls <class 'sympy.core.function.FunctionClass'> (Pdb++) cls.__mro__ (<class 'sympy.core.function.FunctionClass'>, <class 'sympy.core.basic.BasicMeta'>, <class 'sympy.core.basic.BasicType'>, <type 'type'>, <type 'object'>) (Pdb++)
And it seems that bases[0] and bases[1] are incompatible (why?)
Don't know. This patch fixes it: diff --git a/sympy/core/function.py b/sympy/core/function.py index 76a0f5b..a99c21d 100644 --- a/sympy/core/function.py +++ b/sympy/core/function.py @@ -362,7 +362,7 @@ class Function(Basic): x = sympify(x) return cls(x).diff(x, n).subs(x, 0) * x**n / C.Factorial(n) -class WildFunction(Function, Atom): +class WildFunction(Function): """ WildFunction() matches any expression but another WildFunction() XXX is this as intended, does it work ? But now there is this problem:
import sympy Traceback (most recent call last): File "<console>", line 1, in <interactive> File "/home/ondra/repos/sympy/sympy/__init__.py", line 22, in <module> from concrete import * File "/home/ondra/repos/sympy/sympy/concrete/__init__.py", line 2, in <module> from products import product, Product File "/home/ondra/repos/sympy/sympy/concrete/products.py", line 5, in <module> from sympy.simplify import powsimp File "/home/ondra/repos/sympy/sympy/simplify/__init__.py", line 10, in <module> from rewrite import cancel, trim, apart File "/home/ondra/repos/sympy/sympy/simplify/rewrite.py", line 22, in <module> def cancel(f, *symbols): TypeError: threaded() takes no non-keyword argument (1 given)
Any idea what's wrong here? Here is the code: @threaded() def cancel(f, *symbols): """Cancel common factors in a given rational function. and: def threaded(**flags): Seems a-ok to me. Is pypy using some strict things? I don't know, but this patch fixes it: diff --git a/sympy/utilities/decorator.py b/sympy/utilities/decorator.py index 0043de1..f812b51 100644 --- a/sympy/utilities/decorator.py +++ b/sympy/utilities/decorator.py @@ -3,7 +3,7 @@ from sympy.core.add import Add from sympy.core.sympify import sympify from sympy.core.relational import Relational -def threaded(**flags): +def threaded(*args, **flags): Now it imports. Wow!
import sympy from sympy import var var("x y z") (x, y, z) x**2 x**2 from sympy import sin e = sin(x) e.series(x, 0, 5) x - 1/6*x**3 + O(x**5)
Guys, this is awesome! I did some timings: $ ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``Python 300''
from sympy import * var("x") x from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 3.31447815895
Compare to python: $ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from sympy import * var("x") x from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 0.704444885254
So pypy is roughly 4.7x slower on this particular thing. Do you have any plans to release pypy? I think it's getting very useful. Ondrej
So pypy is roughly 4.7x slower on this particular thing.
Do you have any plans to release pypy? I think it's getting very useful.
Ondrej
Yes, we have some plans to have a release at some point soon. There are no details though. Note that if you do this: from sympy import * var('x') from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 100); t = clock()-t; print t var('y') t = clock(); a= sin(y).series(y, 0, 100); t = clock()-t; print gap between pypy and cpython is getting smaller (some startup time is particularly large). I can do a bit profiling if you like. Regarding threaded - I think you should not assume that your argument is passed as keyword arg when using decorator I think. The other one looks like bug in pypy, but I'm not 100% sure. Cheers, fijal
On Mon, Aug 25, 2008 at 1:04 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
So pypy is roughly 4.7x slower on this particular thing.
Do you have any plans to release pypy? I think it's getting very useful.
Ondrej
Yes, we have some plans to have a release at some point soon. There are no details though. Note that if you do this:
from sympy import * var('x') from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 100); t = clock()-t; print t var('y') t = clock(); a= sin(y).series(y, 0, 100); t = clock()-t; print
gap between pypy and cpython is getting smaller (some startup time is particularly large). I can do a bit profiling if you like.
Use SYMPY_USE_CACHE=no when doing repeated profiling, otherwise you are just measuring how fast our cache is. ondra@pc232:~/repos/sympy$ SYMPY_USE_CACHE=no ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``- PyPy status blog: http://morepypy.blogspot.com/''
from sympy import * var('x') x from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 3.56681609154 t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 3.2662088871 t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 3.22522711754 t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 3.24221301079
ondra@pc232:~/repos/sympy$ SYMPY_USE_CACHE=no python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from sympy import * var('x') x from timeit import default_timer as clock t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 0.83996295929 t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 0.739042043686 t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 0.73944401741 t = clock(); a= sin(x).series(x, 0, 1000); t = clock()-t; print t 0.739789009094
We are in the process of speeding up sympy, you can also play with our experimental core that doesn't use caching: $ git clone git://github.com/certik/sympyx.git $ cd sympyx $ ~/repos/pypy-dist/pypy/translator/goal/pypy-c Python 2.4.1 (pypy 1.0.0 build 57617) on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``"peephope" optimizations are what an optimistic Compiler uses''
from sympy import var I: import sympy_pyx ... fail (No module named sympy_pyx) W: can't import sympy_pyx -- will be pure python var("x y z") (x, y, z) e = (x+y+z+1)**40 from timeit import default_timer as clock t = clock(); a= e.expand(); t = clock()-t; print t 2.6539349556 t = clock(); a= e.expand(); t = clock()-t; print t 2.73331212997 t = clock(); a= e.expand(); t = clock()-t; print t 2.76934981346
$ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from sympy import var I: import sympy_pyx ... fail (No module named sympy_pyx) W: can't import sympy_pyx -- will be pure python var("x y z") (x, y, z) e = (x+y+z+1)**40 from timeit import default_timer as clock t = clock(); a= e.expand(); t = clock()-t; print t 0.867521047592 t = clock(); a= e.expand(); t = clock()-t; print t 1.13780593872 t = clock(); a= e.expand(); t = clock()-t; print t 1.13062214851 t = clock(); a= e.expand(); t = clock()-t; print t 1.14683890343 t = clock(); a= e.expand(); t = clock()-t; print t 1.17529797554
So for that pypy is only 2x to 3x slower. BTW, when you compile our core using Cython, it get's way faster: $ make cython --convert-range sympy_pyx.pyx gcc -I/usr/include/python2.5 -I/usr/include/python2.5 -g -O0 -fPIC -c -o sympy_pyx.o sympy_pyx.c gcc -shared sympy_pyx.o -o sympy_pyx.so $ python Python 2.5.2 (r252:60911, Aug 8 2008, 09:22:44) [GCC 4.3.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from sympy import var I: import sympy_pyx ... ok var("x y z") (Symbol(x), Symbol(y), Symbol(z)) e = (x+y+z+1)**40 from timeit import default_timer as clock t = clock(); a= e.expand(); t = clock()-t; print t 0.214432001114 t = clock(); a= e.expand(); t = clock()-t; print t 0.271343946457 t = clock(); a= e.expand(); t = clock()-t; print t 0.239308834076
Do you have any plans for supporting writing C extensions to pypy? Because as you can see above, the speed is imho only possible when writing it in C. Well, if RPython could produce as optimized code as Cython, then it could be an option. Any ideas on that? Ondrej
Do you have any plans for supporting writing C extensions to pypy? Because as you can see above, the speed is imho only possible when writing it in C. Well, if RPython could produce as optimized code as Cython, then it could be an option. Any ideas on that?
Ondrej
There are no immediate plans for supporting C extensions to pypy besides via ctypes or rpython. RPython is fairly fast these days, try it out if you like (big fat warning, rpython is an obscure language). Also usecases like sympy one should see relatively good speedups via JIT. Did you try using psyco? Cheers, fijal
On Mon, Aug 25, 2008 at 10:26 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Do you have any plans for supporting writing C extensions to pypy? Because as you can see above, the speed is imho only possible when writing it in C. Well, if RPython could produce as optimized code as Cython, then it could be an option. Any ideas on that?
Ondrej
There are no immediate plans for supporting C extensions to pypy besides via ctypes or rpython. RPython is fairly fast these days, try it out if you like (big fat warning, rpython is an obscure language). Also usecases like sympy one should see relatively good speedups via JIT. Did you try using psyco?
Psyco gives pretty good results (but not as good as Cython), but as I understood, it is not developed anymore as I thought pypy is the way to go. Yes, Cython is also a new language basically, but it's very easy to learn and easy to debug, easy to compile and one is able to blend C and Python very easily and naturally, so it's my choice. But on the other hand, I really think that one should just write what he wants in Python, maybe give it couple hints and pypy (or anything else) should be clever enough to optimize it. Cython is not perfect, but one can get C level speed now. They plan to allow a pure python syntax, so that the same code actually also runs in Python. But anyway, pypy has my thumbs up. 3x slower than CPython is not *that* bad, so imho if you could release, it'd be awesome. Ondrej
On Mon, 25 Aug 2008 23:29:03 +0200 "Ondrej Certik" <ondrej@certik.cz> wrote:
Cython is not perfect, but one can get C level speed now. They plan to allow a pure python syntax, so that the same code actually also runs in Python.
What do you mean here ? Cython already allows pure python syntax. Simon.
On Tue, Aug 26, 2008 at 10:27 PM, Simon Burton <simon@arrowtheory.com> wrote:
On Mon, 25 Aug 2008 23:29:03 +0200 "Ondrej Certik" <ondrej@certik.cz> wrote:
Cython is not perfect, but one can get C level speed now. They plan to allow a pure python syntax, so that the same code actually also runs in Python.
What do you mean here ? Cython already allows pure python syntax.
Does it? Wow, didn't know about it. How do you write this in pure python syntax? def f(int i, int j): return i*j ? What I mean is to use some decorator, or some other (python) syntax to represent the "int" in there. Ondrej
Hi Ondrej, On Sun, Aug 24, 2008 at 04:08:29PM +0200, Ondrej Certik wrote:
TypeError: instance layout conflicts in multiple inheritance
Not completely surprizingly, our multiple inheritance "instance layout conflict" condition is not 100% the same as CPython's (as the latter is really obscure and based on conditions like the size of C-level structures). Still, I think that this case should work, so it looks like a bug in PyPy. Filed issue #390: https://codespeak.net/issue/pypy-dev/issue390 Armin
On Mon, Aug 25, 2008 at 4:04 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi Ondrej,
On Sun, Aug 24, 2008 at 04:08:29PM +0200, Ondrej Certik wrote:
TypeError: instance layout conflicts in multiple inheritance
Not completely surprizingly, our multiple inheritance "instance layout conflict" condition is not 100% the same as CPython's (as the latter is really obscure and based on conditions like the size of C-level structures). Still, I think that this case should work, so it looks like a bug in PyPy.
Filed issue #390: https://codespeak.net/issue/pypy-dev/issue390
Thanks Armin for looking into this. Btw, if it helps, ironpython also fails on conflict1.py from #390: $ ipy conflict1.py Traceback (most recent call last): File conflict1, line unknown, in Initialize File mscorlib, line unknown, in get_Item KeyError: The given key was not present in the dictionary. Also I don't like mutliple inheritance, so we should just fix sympy and be done with it. Unfortunately IronPython fails later too and Jython as well: $ /home/ondra/ext/jython/jython Jython 2.5a1+ (asm:4943:4945, Jul 15 2008, 15:30:04) [Java HotSpot(TM) Server VM (Sun Microsystems Inc.)] on java1.5.0_15 Type "help", "copyright", "credits" or "license" for more information.
import sympy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "sympy/__init__.py", line 16, in <module> from sympy.core import * File "sympy/core/__init__.py", line 4, in <module> from basic import Basic, S, C, sympify File "sympy/core/basic.py", line 236, in <module> class Basic(AssumeMeths): TypeError: Error when calling the metaclass bases mro() returned base with unsuitable layout ('Basic')
So this suggests that sympy code is not standard. Ondrej
Hi Ondrej, On Mon, Aug 25, 2008 at 04:20:40PM +0200, Ondrej Certik wrote:
class Basic(AssumeMeths): TypeError: Error when calling the metaclass bases mro() returned base with unsuitable layout ('Basic')
So this suggests that sympy code is not standard.
That's obscure. It probably points to a bug in Jython and IronPython, or just some feature that is really obscure but that PyPy implements like CPython by chance. I fixed the multiple inheritance bug as well as the bug shown by the @threaded() decorator. Since rev 57656, pypy-c can import sympy. (I didn't try to do anything with it, though.) A bientot, Armin.
On Fri, Aug 29, 2008 at 6:00 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi Ondrej,
On Mon, Aug 25, 2008 at 04:20:40PM +0200, Ondrej Certik wrote:
class Basic(AssumeMeths): TypeError: Error when calling the metaclass bases mro() returned base with unsuitable layout ('Basic')
So this suggests that sympy code is not standard.
That's obscure. It probably points to a bug in Jython and IronPython, or just some feature that is really obscure but that PyPy implements like CPython by chance.
Yeah. We should try to fix sympy, so that we don't use obscure features.
I fixed the multiple inheritance bug as well as the bug shown by the @threaded() decorator. Since rev 57656, pypy-c can import sympy. (I didn't try to do anything with it, though.)
Thanks for fixing it. Ondrej
participants (4)
-
Armin Rigo -
Maciej Fijalkowski -
Ondrej Certik -
Simon Burton