Richard Emslie <rxe@ukshells.co.uk> writes:

Hi,

I hope the past sprint was fun and I was wondering if any kind person from it would like to give an overview of activities, developments etc.... please? That would be really great!!

Hmm, I was there and I could do with the same! :-) Reading pypy-svn is probably the best way to get an idea of what happened. Most of the work was on the annontator/translator, although there was some appspace stuff -- md5, sha-1 hashes and files, although files are not plumbed into the rest of pypy yet -- and the usual polishing of the standard objects space. On the annotator side, there seems to be a general feeling that we have roughly the right approach now, although there are still an unknown number of hopefully small details to work out. The translator basically works, but there are some nasty hacks knocking about and we haven't really tackled the gnarly issue of how to make use of type annotations (currently the annotator is only used to work out *which* functions are called, and the knowledge of *how* they are called is ignored). Evolution, not revolution, I'd say. Cheers, mwh -- MGM will not get your whites whiter or your colors brighter. It will, however, sit there and look spiffy while sucking down a major honking wad of RAM. -- http://www.xiph.org/mgm/

Hi, For a number of us, the sprint was a full week of running translate_pypy.py and wait for the next crash, then debugging and thinking and fixing and workarounding the issue. Then start again at step #1. In the last days it generally took some time (and lots of memory) before it ran into a problem and crashed. It also reported increasingly large amounts of warnings... well, more precisely, we turned increasingly many errors into warnings :-) These ones report problems that should eventually be fixed. This mostly occupied Marius, Holger, Michael and myself. (Cheers to Marius, btw, for having caught up so quickly with PyPy!) Some time in the middle of it all, the basic annotation-less 'translate_pypy.py -no-a' was able to compile and run (-6)*(-7) and successfully gave us the ultimate answer. I guess the next step here is to try to change the 'entry_point' so that 'translate_pypy' can actually compile and run the interactive command-line interpreter. Bob worked on more specific pieces of code: the user interface of the Pygame graph viewer (Marius did bits of that too), and the genc.py C code generation, which now generates code with tons of debugging features. The best one is probably that if the generated C code fails with a Python exception, you can use pdb.pm() to debug it! You see the C frames with the content of their variables, with line numbers set correctly. pdb will display extracts of the C source code in the tracebacks :-) Christian worked on using the flow graph analysis to understand application-level code and transform them to interpreter-level code. For example, each + in the source becomes an add operation in the flow graph (using the existing almost-unmodified flow object space), which then (this is Christian's work) gets translated as a space.add() by genrpy.py. This is useful for application-level helper code, for example string formatting, which is currently written in _formatting.py as regular, app-level Python code. The goal is to turn this nice piece of code into its obfuscated interp-level equivalent automatically. This is essentially an optimization only, which will allow the code to run a bit faster because PyPy's own interpreter doesn't have to interpret the app-level _formatting.py every time it has to do a string formatting operation. It also allows future more advanced optimizations. Jacob and Laura, who didn't feel like digging into the PyPy internals too deepdy, focused on reimplementing some of the C functionality in plain Python. We now have MD5 and SHA-1 and a 'class file' replacement based on low-level primitives (generally the 'os' module). You can play with the 'file' class by importing 'appspace/_file.py' in CPython. The general guidelines to integrate this kind of pure Python module with PyPy is: (1) try the module in py.py when it works in CPython, and (2) plug it -- for the 'file' class, it's done by inserting 'from _file import file' in 'module/__builtin__module.py'. Warning, this will probably be quite slow :-) Finally, we got news from the EU. In all likelyhood the project will really start on the 1st of December. Yippee! End of the phase 1 of the administrative overload! Now we have to plan for more closely-packed sprints. The next one might take place somewhere around the end of January in Switzerland. Last but not least, thanks to the Programmers of Vilnius for the excellent sprint organization! See you soon, Armin.

Richard Emslie <rxe@ukshells.co.uk> writes:

Hi,

I hope the past sprint was fun and I was wondering if any kind person from it would like to give an overview of activities, developments etc.... please? That would be really great!!

Hmm, I was there and I could do with the same! :-) Reading pypy-svn is probably the best way to get an idea of what happened. Most of the work was on the annontator/translator, although there was some appspace stuff -- md5, sha-1 hashes and files, although files are not plumbed into the rest of pypy yet -- and the usual polishing of the standard objects space. On the annotator side, there seems to be a general feeling that we have roughly the right approach now, although there are still an unknown number of hopefully small details to work out. The translator basically works, but there are some nasty hacks knocking about and we haven't really tackled the gnarly issue of how to make use of type annotations (currently the annotator is only used to work out *which* functions are called, and the knowledge of *how* they are called is ignored). Evolution, not revolution, I'd say. Cheers, mwh -- MGM will not get your whites whiter or your colors brighter. It will, however, sit there and look spiffy while sucking down a major honking wad of RAM. -- http://www.xiph.org/mgm/

Hi, For a number of us, the sprint was a full week of running translate_pypy.py and wait for the next crash, then debugging and thinking and fixing and workarounding the issue. Then start again at step #1. In the last days it generally took some time (and lots of memory) before it ran into a problem and crashed. It also reported increasingly large amounts of warnings... well, more precisely, we turned increasingly many errors into warnings :-) These ones report problems that should eventually be fixed. This mostly occupied Marius, Holger, Michael and myself. (Cheers to Marius, btw, for having caught up so quickly with PyPy!) Some time in the middle of it all, the basic annotation-less 'translate_pypy.py -no-a' was able to compile and run (-6)*(-7) and successfully gave us the ultimate answer. I guess the next step here is to try to change the 'entry_point' so that 'translate_pypy' can actually compile and run the interactive command-line interpreter. Bob worked on more specific pieces of code: the user interface of the Pygame graph viewer (Marius did bits of that too), and the genc.py C code generation, which now generates code with tons of debugging features. The best one is probably that if the generated C code fails with a Python exception, you can use pdb.pm() to debug it! You see the C frames with the content of their variables, with line numbers set correctly. pdb will display extracts of the C source code in the tracebacks :-) Christian worked on using the flow graph analysis to understand application-level code and transform them to interpreter-level code. For example, each + in the source becomes an add operation in the flow graph (using the existing almost-unmodified flow object space), which then (this is Christian's work) gets translated as a space.add() by genrpy.py. This is useful for application-level helper code, for example string formatting, which is currently written in _formatting.py as regular, app-level Python code. The goal is to turn this nice piece of code into its obfuscated interp-level equivalent automatically. This is essentially an optimization only, which will allow the code to run a bit faster because PyPy's own interpreter doesn't have to interpret the app-level _formatting.py every time it has to do a string formatting operation. It also allows future more advanced optimizations. Jacob and Laura, who didn't feel like digging into the PyPy internals too deepdy, focused on reimplementing some of the C functionality in plain Python. We now have MD5 and SHA-1 and a 'class file' replacement based on low-level primitives (generally the 'os' module). You can play with the 'file' class by importing 'appspace/_file.py' in CPython. The general guidelines to integrate this kind of pure Python module with PyPy is: (1) try the module in py.py when it works in CPython, and (2) plug it -- for the 'file' class, it's done by inserting 'from _file import file' in 'module/__builtin__module.py'. Warning, this will probably be quite slow :-) Finally, we got news from the EU. In all likelyhood the project will really start on the 1st of December. Yippee! End of the phase 1 of the administrative overload! Now we have to plan for more closely-packed sprints. The next one might take place somewhere around the end of January in Switzerland. Last but not least, thanks to the Programmers of Vilnius for the excellent sprint organization! See you soon, Armin.