[pypy-svn] r16639 - pypy/release/0.7.x/pypy/doc

cfbolz at codespeak.net cfbolz at codespeak.net
Fri Aug 26 17:17:56 CEST 2005

Author: cfbolz
Date: Fri Aug 26 17:17:54 2005
New Revision: 16639

added section about external function calls. reviews welcome.

Modified: pypy/release/0.7.x/pypy/doc/index.txt
--- pypy/release/0.7.x/pypy/doc/index.txt	(original)
+++ pypy/release/0.7.x/pypy/doc/index.txt	Fri Aug 26 17:17:54 2005
@@ -36,7 +36,7 @@
 `license`_ contains licensing details (basically a straight MIT-license). 
-.. _FAQ: faq.html
+.. _`FAQ`: faq.html
 .. _parser: parser.html
 .. _`talks and related projects`: extradoc.html
 .. _`license`: http://codespeak.net/svn/pypy/dist/LICENSE 

Modified: pypy/release/0.7.x/pypy/doc/translation.txt
--- pypy/release/0.7.x/pypy/doc/translation.txt	(original)
+++ pypy/release/0.7.x/pypy/doc/translation.txt	Fri Aug 26 17:17:54 2005
@@ -1172,3 +1172,163 @@
 .. include:: _ref.txt
+External Function Calls
+There are some functions that we don't want to implement in Python for various
+reasons (e.g. if they need to make calls into the OS). These can be
+implemented by writing backend code by hand that either implements the
+functionality itself or calls appropriate libraries.
+Of course the annotator does not know what types these funtions return so we
+need a way to attach fixed annotations to such functions. Additionally for
+every such function we need a low level helper function that behaves like the
+hand-written backend code so that we can still test a call to such an external
+Annotating external function calls
+All the information about external functions that are important for the
+annotation process are written into `pypy/rpython/extfunctable.py`_. There is a
+function called ``declare`` that allows to tell the annotator the return type
+of these functions and to attach a low level dummy implementation::
+    declare(funct, annotation_factory, lowlevel_dummy)
+Here ``funct`` is the funtion that is supposed to be implemented in the
+backend, ``annotation_factory`` is a function that returns an appropriate
+annotation of the return value (an instance of a primitive type is also ok, so
+you can do ``declare(func, int...)``), lowlevel_dummy is a string of the form
+``module/ll_function`` that specifies where the low level dummy function is
+defined. Here ``module`` means a Python file in `pypy/rpython/module/`_ and
+``ll_function`` is a low level function defined in that file.
+If the annotator discovers a call to ``func`` it does not try to follow that
+call further (even though that might be possible if the function is written in
+Python) but annotates it with the given type immediately. The `RPython Typer`_
+replaces calls to ``func`` with calls to the function
+``module.ll_function``. The backend is supposed to replace calls to functions
+to ``module.ll_function`` by whatever code it finds appropriate.
+.. _`pypy/rpython/extfunctable.py`: ../rpython/extfunctable.py
+.. _`pypy/rpython/module/`: ../rpython/module
+Implementating low level replacement functions in Python
+The dummy low level replacement functions are there to implement the external
+function on the low level. In contrast to the original function they should
+take arguments that are of `low-level type`_. Most of the times they are
+implemented by calling appropriate low level to high level conversion
+functions and then calling the original funtion again.
+If the function is supposed to really be implemented by the backend then the
+low level function should have an attribute ``.suggested_primitive = True``
+attached. If this is not the case the low level function itself will be
+translated and used.
+Implementing the external functions in the C backend
+When the C-backend produces C code and finds a function call to one of the
+dummy functions it replaces the call to it by a call to a function written in
+C. This mapping is done in the file `pypy/translator/c/extfunc.py`_. In there
+is a dictionary called ``EXTERNALS`` which contains mappings from dummy
+functions to strings::
+    EXTERNAL = {
+        module.ll_function: "LL_c_name_of_function"
+        ...
+    }
+Here ``LL_c_name_of_function`` is the name of the C function that implements
+the functionality of the ``module.ll_function``. These C implementation are
+found in the directory `pypy/translator/c/src/`_.
+It sometimes neccessary to access a certain function or type that is written
+in Python from the C code you write by hand (for example for constructing the
+return value in the correct low level type). This can be a problem because the
+C backend mangels names to prevent clashes. To get a certain low level type
+under a fixed name the function ``predeclare_common_types`` needs to be
+changed. This function is a generator that yields tuples of the form
+``('c_name', LLTYPE)``. This makes the genc assign the name ``c_name`` to the
+type ``LLTYPE``. Similarly all the function defined in
+``predeclare_utility_functions`` are automacially made accessible under their
+name in C.
+.. _`pypy/translator/c/extfunc.py`: ../translator/c/extfunc.py
+.. _`pypy/translator/c/src/`: ../translator/c/src/
+To make this clearer the following shows all the relevant pieces that are
+needed for implementing os.open. It is implemented in the following way at
+interp-level in the `mixed posix module`_::
+    def open(space, fname, flag, mode=0777):
+        try: 
+            fd = os.open(fname, flag, mode)
+        except OSError, e: 
+            raise wrap_oserror(space, e) 
+        return space.wrap(fd)
+    open.unwrap_spec = [ObjSpace, str, int, int]
+If the annotator tries to annotate this function it will use the declaration
+it finds in the file `pypy/rpython/extfunctable.py`_::
+    declare(os.open, int , 'll_os/open')
+This means that the function returns an int and that the dummy low level
+function can be found in `pypy/rpython/module/ll_os.py`_::
+    def ll_os_open(fname, flag, mode):
+        return os.open(from_rstr(fname), flag, mode)
+    ll_os_open.suggested_primitive = True
+The ``suggested_primitive`` attribute of the ``ll_os_open`` is set to True,
+because it is reasonable that this function is written directly in the backend.
+If the C backend encounters a call to ``ll_os_open`` somewhere in the code it
+checks the ``EXTERNALS`` table in `pypy/translator/c/extfunc.py`_. The
+relevant part for ``os.open`` is::
+    from pypy.rpython.module import ll_os
+    EXTERNALS = {
+        ...
+        ll_os.ll_os_open:    'LL_os_open',
+        ...
+    }
+The `LL_os_open` function is implemented in the file
+    int LL_os_open(RPyString *filename, int flag, int mode)
+    {
+            char buf[PATH_MAX];
+            int fd, namelen = RPyString_Size(filename);
+            if (namelen >= PATH_MAX) {
+                    return -1;
+            }
+            else {
+                    memcpy(buf, RPyString_AsString(filename), namelen);
+                    buf[namelen] = 0;
+                    fd = open(buf, flag, mode);
+                    if (fd < 0)
+                            RPYTHON_RAISE_OSERROR(errno);
+                    return fd;
+            }
+    }
+.. _`mixed posix module`: ../module/posix/
+.. _`pypy/rpython/module/ll_os.py`: ../rpython/module/ll_os.py
+.. _`pypy/translator/c/src/ll_os.h`: ../translator/c/src/ll_os.h
\ No newline at end of file

More information about the Pypy-commit mailing list