[pypy-svn] r11003 - pypy/dist/pypy/documentation

[hpk at codespeak.net]

Author: hpk
Date: Thu Apr 21 22:47:59 2005
New Revision: 11003

Modified:
   pypy/dist/pypy/documentation/coding-style.txt
   pypy/dist/pypy/documentation/objspace.txt
Log:
updating and improving some RPython chapters, 
they often refered to something as being in
the future than in the present.  I also introduced 
some links to source code and starkiller. 
feel free to review and further enhance. 

I note that the "integer" chapter is both
somewhat outdated with respect to the new 
ovfcheck() and is sometimes written in the 'I' 
form.  

Would be great if someone can state 
some authoritate things about the current
integer situation and refactor the 'I' into 
a 'we' and follow the style of our other 
documentation some more. 



Modified: pypy/dist/pypy/documentation/coding-style.txt
==============================================================================
--- pypy/dist/pypy/documentation/coding-style.txt	(original)
+++ pypy/dist/pypy/documentation/coding-style.txt	Thu Apr 21 22:47:59 2005
@@ -10,73 +10,78 @@
 
 We are writing a Python interpreter in Python, using Python's well known ability 
 to step behind the algorithmic problems as language. At first glance, one might 
-think this achieves nothing but a better understanding for everybody how the 
-interpreter works. This alone would make it worth doing, but we have much larger 
-goals. 
+think this achieves nothing but a better understanding how the interpreter works. 
+This alone would make it worth doing, but we have much larger goals. 
 
 
 CPython vs. PyPy
 -------------------
 
-Compared to the CPython implementation, Python takes the role of the C Code. So 
-actually, we describe something by Python, which has been coded in C already, 
-with all the restrictions that are implied by C. We are not trying to make the 
-structures of the CPython interpreter more flexible by rewriting things in C, 
-but we want to use Python to give an alternative description of the interpreter.
+Compared to the CPython implementation, Python takes the role of the C
+Code. We rewrite the CPython interpreter in Python itself.  We could
+also aim at writing a more flexible interpreter at C level but but we
+want to use Python to give an alternative description of the interpreter.
 
-The clear advantage is that this description is probably shorter and simpler to 
+The clear advantage is that such a description is shorter and simpler to 
 read, and many implementation details vanish. The drawback of this approach is 
-that this interpreter will be unbearably slow.
+that this interpreter will be unbearably slow as long as it is run on top
+of CPython.
 
-To get to a useful interpreter again, we need to apply some mappings to the 
-implementation, later. One rather straight-forward is to do a whole program 
-analysis of the PyPy interpreter and create a C source, again. There are many 
-other ways, but let's stick with the easiest approach, first.
-
-In order to make a C code generator simple, we restrict ourselves to a subset of 
-the Python language, and we adhere to some rules, which make code generation 
-obvious and easy.
-
-
-Restricted Python is Runtime Python
--------------------------------------
-
-Restricted Python describes a runnable Python interpreter implementation. This 
-is a quite static object that can be suitably described by RPython. But the 
-restrictions do not apply during the startup phase.
-
-
-PyPy Bootstrap
--------------------
-
-When the PyPy interpreter is started as a CPython program, it can use all of 
-CPython for a while, until it reaches runtime. That is, all executable code will 
-be executed using the full power of Python.
-
-An example can be found in the implementation, which is quite elegant: For the 
-definition of all the opcodes of the Python interpreter, the module dis is 
-imported and used. This saves us from adding extra modules to PyPy. The import 
-code is run at startup time, and we are allowed to use the CPython builtin 
-import function.
-
-When the startup code is done, all resulting objects, functions, code blocks 
-etc. must adhere to the runtime restrictions. All initialized modules are 
-written out in a persistent manner. Our current idea is to emit a huge C source 
-file which contains everything created so far. During this process, a whole 
-program analysis is performed, which makes use of the restrictions defined in 
-RPython. This enables the code generator to emit efficient replacements for pure 
-integer objects, for instance.
-
-
-RPython Definition
---------------------
-
-It might make sense to define a sub-language of Python called RPython, with the 
-restrictions depicted below. This is an evolving topic, and we're just 
-collecting things which come up during trying to code the interpreter, so this 
-is no language at all, but an arbitrary set of rules, which are about to be 
-changed all day.
+To get to a useful interpreter again, we need to translate our
+high-level description of Python to a lower level one.  One rather
+straight-forward way is to do a whole program analysis of the PyPy
+interpreter and create a C source, again. There are many other ways, 
+but let's stick with this somewhat canonical approach.  
+
+our runtime interpreter is "restricted python" 
+----------------------------------------------
+
+In order to make a C code generator feasible we restrict ourselves to a
+subset of the Python language, and we adhere to some rules which make
+translation to lower level langauges more obvious. 
+
+Unlike source-to-source translations (like e.g. Starkiller_) we start
+translation from live python code objects which constitute our Python
+interpreter.   When doing its work of interpreting bytecode our Python
+implementation must behave in a static way often referenced as 
+"RPythonic". 
+
+.. _Starkiller: http://www.python.org/pycon/dc2004/papers/1/paper.pdf
+
+However, when the PyPy interpreter is started as a Python program, it
+can use all of the Python language until it reaches interpretation
+runtime. That is, during initialisation our program is free to use the
+full dynamism of Python, including dynamic code generation. 
+
+An example can be found in the current implementation which is quite
+elegant: For the definition of all the opcodes of the Python
+interpreter, the module ``dis`` is imported and used to initialize our
+bytecode interpreter.  (See ``__initclass_`` in `pyopcode.py`_).  This
+saves us from adding extra modules to PyPy. The import code is run at
+startup time, and we are allowed to use the CPython builtin import
+function.
+
+After the startup code is finished, all resulting objects, functions,
+code blocks etc. must adhere to certain runtime restrictions which we
+describe further below.  Here is some background for why this is so:
+during translation, a whole program analysis ("type inference") is
+performed, which makes use of the restrictions defined in RPython. This
+enables the code generator to emit efficient machine level replacements
+for pure integer objects, for instance.   
+
+.. _`pyopcode.py`: http://codespeak.net/svn/pypy/dist/pypy/interpreter/pyopcode.py 
+
+RPython Definition, not
+-----------------------
+
+The list and exact details of the "RPython" restrictions are a somewhat 
+evolving topic.  In particular, we have no formal language definition
+as we find it more practical to discuss and evolve the set of 
+restrictions while working on the whole program analysis.  If you
+have any questions about the restrictions below then please feel
+free to mail us at pypy-dev at codespeak net. 
 
+.. _`wrapped object`: objspace.html#wrapping-rules
 
 Object restrictions
 -------------------------
@@ -87,7 +92,7 @@
   
   the same variable in the same context can receive values of different types, 
   at a possible overhead cost. For example, a variable that can contain a 
-  wrapped object or None is efficiently implemented as a PyObject* pointer that 
+  `wrapped object`_ or None is efficiently implemented as a PyObject* pointer that 
   can be NULL, but a variable that can contain either an integer or a float must 
   be implemented as a union with a type tag in C.
 
@@ -308,10 +313,45 @@
 
 .. _`py.test`: http://codespeak.net/py/current/doc/test.html 
 
-Writing a test
---------------
+Interpreter level tests 
+----------------------- 
+
+You can write test functions and methods like this:: 
 
-Currently the best reference is to go to some test files and look how they are done.
+    def test_something(space): 
+        # use space ... 
+        
+    class TestSomething: 
+        def test_some(self): 
+            # use 'self.space' here 
+
+Note that the prefix `test` for test functions and `Test` for test
+classes is mandatory.  In both cases you can import Python modules at
+module global level and use plain 'assert' statements thanks to the
+usage of the `py.test`_ tool.  
+
+Application Level tests 
+----------------------- 
+
+For testing the conformance and well-behavedness of PyPy it
+is often sufficient to write "normal" application-level 
+Python code that doesn't need to be aware of any particular
+coding style or restrictions.  If we have a choice we often
+use application level tests which usually look like this: 
+
+    def app_test_something():
+        # application level test code 
+
+    class AppTestSomething: 
+        def test_this(self): 
+            # application level test code 
+
+These application level test functions will run on top
+of PyPy, i.e. they have no access to interpreter details. 
+You cannot use imported modules from global level because
+they are imported at intepreter-level while you test code
+runs at application level. If you need to use modules
+you have to import them within the test function. 
 
 Command line tool test_all
 --------------------------
@@ -320,14 +360,15 @@
 
   python test_all.py
 
+which is a synonym for the general `py.test`_ utility. 
 For switches to modify test execution invoke "python test_all.py -h".
 
 Test conventions
 ----------------
 
-- adding features usually requires adding appropriate tests. 
-  (It often even makes sense to first write the tests so that you are
-  sure that they actually can fail.) 
+- adding features requires adding appropriate tests.  (It often even
+  makes sense to first write the tests so that you are sure that they
+  actually can fail.) 
 
 - All over the pypy source code there are test/ directories 
   which contain unittests.  Such scripts can usually be executed

Modified: pypy/dist/pypy/documentation/objspace.txt
==============================================================================
--- pypy/dist/pypy/documentation/objspace.txt	(original)
+++ pypy/dist/pypy/documentation/objspace.txt	Thu Apr 21 22:47:59 2005
@@ -14,6 +14,8 @@
 
 This document describes aspects of the implementation of PyPy's `Object Spaces`_.
 
+.. _`wrapping rules`: 
+
 Wrapping rules
 ==============