Hi, I'm exploring the llvm backend of pypy. Does it work without llvm-gcc ? I tried the example at doc/getting-started.html and it seems to first compile to c code then invoke llvm-gcc.
f = t.compile_llvm() (calls to llvm-gcc)
But I'm also looking at the source for the llvm translator and there is code there for generating llva directly. Now I'm trying to run the demos in pypy/translator/llvm/demo. I assume I can use my regular python to run these.. somehow.. $ python run.py Traceback (most recent call last): ... File "/mnt/hda9/simonb/local/pypy-svn/pypy/annotation/bookkeeper.py", line 16, in ? from pypy.annotation.listdef import ListDef, MOST_GENERAL_LISTDEF ImportError: cannot import name ListDef ? Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
On Wed, 24 May 2006 14:02:07 +1000 Simon Burton <simon@arrowtheory.com> wrote:
But I'm also looking at the source for the llvm translator and there is code there for generating llva directly.
I'm gradually finding more pieces of the puzzle: 1) there is a wrapper to LLVM's ExecutionEngine.ParseAssemblyString 2) there are calls to ee.parse in pyllvm/test/test_ee.py, but only with test snippets but then: 3) there is code for generating llva in codewriter.py 4) calls to codewriter come from *node.py modules, these stitch together the pypy flow and type objects into llva code 5) genllvm.py coordinates all of this and then calls to buildllvm It seems that steps 1 and 2 are yet to be used as buildllvm uses the commandline llvm-as at the moment. Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
Hi Simon, llvm-gcc is used to compile the external function C code (that genllvm shares with genc) to a .ll file. The code in llvm/demo does not seem to work at the moment, sorry for that :( It looks like a circular dependency issue. You assumption about running the files in ..../demo with regular python is correct, however the run.py is respronsible for the invokation of genc/genllvm in this case so you would have to do something like: "python bpnn.py". About your other mail about genllvm: We indeed explored the usage of llvm's jit feature with ExecutionEngine.ParseAssemblyString but the use of this is currently limited to pypy/jit/codegen/llvm , where genllvm generates .ll code that gets added to the llvm jit with the execution engine. The steps the genllvm follows are: - (when necessary) create a .ll file out of genc's external function c code (one day to be replaced by (r)ctypes code) - generate and attach our source file - run llvm-as to convert the .ll to a .bc file - run opt to optimize with llvm's transformation - generate (a) assembly with llvm's native (x86) backend or (b) c code again with llvm's c backend - in case of b we run this code through gcc to further (profile based) optimize and create the final executable. I can image this information is a bit to dense so if you have any more questions please ask them here. It's good for us to know what people require for getting a better overview. cheers Eric On May 24, 2006, at 6:02 AM, Simon Burton wrote:
Hi Simon, On Wed, May 24, 2006 at 02:02:07PM +1000, Simon Burton wrote:
Oups, mea culpa. I added an import at the top of run.py without checking that it was still working. Importing pypy.annotation.listdef before importing the other modules of that package creates a circular import bug. As a workaround to this particular cycle, I added an import line in the __init__ of the package. A bientot, Armin
... On Wed, May 24, 2006 at 10:36:32AM +0200, Armin Rigo wrote:
As a workaround to this particular cycle, I added an import line in the __init__ of the package.
Now that I think about it, there are a few places (including this cycle) where modules import other modules only for the side-effects of having something registered; e.g. pypy.annotation.model imports pypy.annotation.unaryop and pypy.annotation.binaryop, and pypy.rpython.rtyper imports many of the pypy.rpython.r* modules. There is the same problem with every place that uses the extregistry (currently only rctypes and some ootypesystem functions and types). Maybe we could reorganize that by moving the necessary import statements to the __init__ of the appropriate packages. It certainly looks clearer and easier to control than these import loops. (It would also require us to finally do the long-pending clean-up of dividing the 'rpython' package in two: one for the rtyper and one for the modules of stuff that RPython programs are allowed to call, e.g. rarithmetic and objectmodel.) A bientot, Armin
Simon, hi! Something I have to add to my last email... The profile based optimization stuff is only the the nightly benchmark cronjob (pypy/translator/goal/bench-cronjob.py) but could in principle (with proper support from genc and genllvm) be added as an option to ./translate.py . cheers Eric
On Wed, 24 May 2006 14:02:07 +1000 Simon Burton <simon@arrowtheory.com> wrote:
But I'm also looking at the source for the llvm translator and there is code there for generating llva directly.
I'm gradually finding more pieces of the puzzle: 1) there is a wrapper to LLVM's ExecutionEngine.ParseAssemblyString 2) there are calls to ee.parse in pyllvm/test/test_ee.py, but only with test snippets but then: 3) there is code for generating llva in codewriter.py 4) calls to codewriter come from *node.py modules, these stitch together the pypy flow and type objects into llva code 5) genllvm.py coordinates all of this and then calls to buildllvm It seems that steps 1 and 2 are yet to be used as buildllvm uses the commandline llvm-as at the moment. Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
Hi Simon, llvm-gcc is used to compile the external function C code (that genllvm shares with genc) to a .ll file. The code in llvm/demo does not seem to work at the moment, sorry for that :( It looks like a circular dependency issue. You assumption about running the files in ..../demo with regular python is correct, however the run.py is respronsible for the invokation of genc/genllvm in this case so you would have to do something like: "python bpnn.py". About your other mail about genllvm: We indeed explored the usage of llvm's jit feature with ExecutionEngine.ParseAssemblyString but the use of this is currently limited to pypy/jit/codegen/llvm , where genllvm generates .ll code that gets added to the llvm jit with the execution engine. The steps the genllvm follows are: - (when necessary) create a .ll file out of genc's external function c code (one day to be replaced by (r)ctypes code) - generate and attach our source file - run llvm-as to convert the .ll to a .bc file - run opt to optimize with llvm's transformation - generate (a) assembly with llvm's native (x86) backend or (b) c code again with llvm's c backend - in case of b we run this code through gcc to further (profile based) optimize and create the final executable. I can image this information is a bit to dense so if you have any more questions please ask them here. It's good for us to know what people require for getting a better overview. cheers Eric On May 24, 2006, at 6:02 AM, Simon Burton wrote:
Hi Simon, On Wed, May 24, 2006 at 02:02:07PM +1000, Simon Burton wrote:
Oups, mea culpa. I added an import at the top of run.py without checking that it was still working. Importing pypy.annotation.listdef before importing the other modules of that package creates a circular import bug. As a workaround to this particular cycle, I added an import line in the __init__ of the package. A bientot, Armin
... On Wed, May 24, 2006 at 10:36:32AM +0200, Armin Rigo wrote:
As a workaround to this particular cycle, I added an import line in the __init__ of the package.
Now that I think about it, there are a few places (including this cycle) where modules import other modules only for the side-effects of having something registered; e.g. pypy.annotation.model imports pypy.annotation.unaryop and pypy.annotation.binaryop, and pypy.rpython.rtyper imports many of the pypy.rpython.r* modules. There is the same problem with every place that uses the extregistry (currently only rctypes and some ootypesystem functions and types). Maybe we could reorganize that by moving the necessary import statements to the __init__ of the appropriate packages. It certainly looks clearer and easier to control than these import loops. (It would also require us to finally do the long-pending clean-up of dividing the 'rpython' package in two: one for the rtyper and one for the modules of stuff that RPython programs are allowed to call, e.g. rarithmetic and objectmodel.) A bientot, Armin
Simon, hi! Something I have to add to my last email... The profile based optimization stuff is only the the nightly benchmark cronjob (pypy/translator/goal/bench-cronjob.py) but could in principle (with proper support from genc and genllvm) be added as an option to ./translate.py . cheers Eric
participants (3)
-
Armin Rigo -
Eric van Riet Paap -
Simon Burton