[pypy-svn] r23496 - pypy/dist/pypy/doc

gromit at codespeak.net gromit at codespeak.net
Sun Feb 19 18:59:25 CET 2006


Author: gromit
Date: Sun Feb 19 18:59:23 2006
New Revision: 23496

Added:
   pypy/dist/pypy/doc/cpython-ctypes-behaviour.txt
   pypy/dist/pypy/doc/ctypes-integration.txt
Log:
ADD: Documentation about ctypes behavior on CPython and the integration plan.

Added: pypy/dist/pypy/doc/cpython-ctypes-behaviour.txt
==============================================================================
--- (empty file)
+++ pypy/dist/pypy/doc/cpython-ctypes-behaviour.txt	Sun Feb 19 18:59:23 2006
@@ -0,0 +1,93 @@
+===========================
+CPython's Ctype's Behaviour
+===========================
+
+.. contents::
+.. sectnum::
+
+This documents describes ctypes behaviour on CPython, as fas as it
+is know to the author and relevant for rctypes.
+
+Primitive Types
+===============
+All primitive types behave like their CPython counterparts.
+Some types like `c_long` and `c_int` are identical on some hardware platforms,
+depending on size of C's int type on those platforms.
+
+
+Heap Allocated Types
+====================
+Ctypes deals with heap allocated instances of types in a simple
+and straightforward manner. However documentationwise it 
+has some shady corners when it comes to heap allocated types.
+
+Structures
+----------
+Structures allocated by ctypes
+
+Structure instances in ctypes are actually proxies for C-compatible
+memory. The behaviour of such instances is illustrated by the following
+example structure throughout this document::
+
+    class TS( Structure ):
+        _fields_ = [ ( "f0", c_int ), ( "f1", c_float ) ]
+
+    ts0 = TS( 42, -42 )
+    ts1 = TS( -17, 17 )
+
+    p0 = pointer( ts0 )
+    p1 = pointer( ts1 )
+
+You can not assign structures by value as in C or C++.
+But ctypes provides a memmove-function just like C does.
+You do not need to pass a ctype's pointer to the structure type to
+memmove. Instead you can pass the structures directly as in the
+example below::
+
+    memmove( ts0, ts1, sizeof( ts0 ) )
+    assert ts0.f0 == -17
+    assert ts0.f1 == 17 
+
+Structures created from foreign pointers
+########################################
+
+The author is currently not shure, whether the text below
+is correct in all aspects, but carefully planned trials did
+not provide any evidence to the contrary.
+
+Structure instances can also be created by dereferencing pointers
+that where returned by a call to an external function.
+
+More on pointers in the next section.
+
+Pointers, especially pointers to Structures
+-------------------------------------------
+
+Pointer types are created by a factory function named `POINTER`.
+Pointers can be created by either calling and thus istanciating a
+pointer type or by calling another function named `pointer` that
+creates the neccessary pointer type on the fly and returns
+a pointer to the instance. 
+
+Pointers only implement one attribute named contents which 
+references the ctypes instance the pointer points to. Assigning to 
+this attribute only changes the pointers notion of the object it points
+to. The instances themselves are not touched, especially structures are
+not copied by value. 
+
+Pointers to Structures Allocated by The Ctypes
+##############################################
+
+Pointers to structures that where allocated by ctypes contain
+a reference to the structure in the contents attribute
+mentioned above. This reference is know to the garbage collector,
+which means that even if the last structure instance is deallocated,
+the C-compatible memory is not, provides a pointer still points to it.
+
+Pointers to Structures Allocated by Foreign Modules
+###################################################
+
+In this case the structure was probably allocated by the foreign module - at
+least ctypes must assume it was. In this case the pointer's reference to the 
+structure should not be made known to the GC. In the same sense the structure itself
+must record the fact that its C-compatible memory was not allocated by ctypes itself.

Added: pypy/dist/pypy/doc/ctypes-integration.txt
==============================================================================
--- (empty file)
+++ pypy/dist/pypy/doc/ctypes-integration.txt	Sun Feb 19 18:59:23 2006
@@ -0,0 +1,125 @@
+====================
+Bootstrapping Ctypes
+====================
+
+.. contents::
+.. sectnum::
+
+
+Abstract and Motivation
+=======================
+
+Bootstrapping Ctypes on PYPY is a 4 steps procedure.
+
+    1) Implement a restricted version of ctypes called rctypes 
+       at interpreter level.
+
+    2) Wrap the libffi provided by CPython's ctypes in a manner
+       that is compatible with CPython's ctypes using the existing
+       ctypes CPython. This means wrapping libffi with libffi.
+
+    3) Port the result of 2 to interpreter level by applying the
+       neccessary changes to make it rctypes compatible.
+
+    4) Refine ctypes and rctypes by adding callbacks, free unions and
+       other stuff needed. 
+
+The most appealing features of this approach are
+
+    -   the ability to implement step 1 and step 2 in parallel.
+
+    -   and the ability to wrap urgently needed modules, like the socket module
+        using the existing ctypes on CPython. Again this work
+        can be done in parallel.
+
+Of course all existing modules that wrap external libraries using ctypes
+will also be available for PyPy.
+
+Design
+======
+
+Restrictions
+------------
+
+Rctypes is desinged with the following restrictions.
+
+    -   All types are defined at module load time and
+        thus need not be rpython.
+
+    -   Free unions are not supported, because it is unclear
+        whether they can be properly annotated.
+
+    -   Callbacks are deferred to steps 4 and 5.
+
+    -   Functions that return structures and pointers with a mixed
+        allocation model are not supported in the initial rctypes version.
+
+    -   Ctypes custom allocators are not supported in the first 4 steps.
+    
+
+Annotation
+----------
+
+Ctypes on CPython tracks all the memory it has allocated by itself,
+may it be referenced by pointers and structures or only pointers.
+Thus memory allocated by ctypes is properly garbage collected and no
+dangling pointers should arise.
+
+Pointers to structures returned by an external function or passed
+to a callback are a different story. For such pointers we have to assume
+that were not be allocated by ctypes, even if they were actually allocated
+by ctypes.
+
+Thus the annotator tracks the memory state of each ctypes object. Pointers
+to structures are annotated differently when they are return by an external
+function. As stated abbove a mixed memory mode function result type is not
+considered rctypes compliant and therefore annotated as `SomeObject` [#].
+
+..[#] This restriction will be lifted in future ctypes versions. 
+
+Memory-Layout
+-------------
+
+Primitive Types
+~~~~~~~~~~~~~~~
+
+Ctypes' primitive types are mapped directly to the correspondending
+PyPy type.
+
+Structures
+~~~~~~~~~~
+Structures will have the following memory layout if they were allocated by ctypes::
+
+    Ptr( GcStruct( "CtypesGcStructure_<ClassName> 
+            ( "c_data" 
+                    (Struct "C-Data_<ClassName>
+                            *<Fieldefintions>) ) ) )
+
+We will try hard not to expose the "c-data" member of the structure
+at rpython level.
+
+Structures that result form dereferencing a pointer will have the following
+layout::
+
+    Ptr( GcStruct( "CtypesStructure_<ClassName>
+        ( "c_data"
+                Ptr( Struct( "C-Data_<ClassName>
+                             *<Fieldefintions>) ) ) ) )
+
+Pointers
+~~~~~~~~
+Pointers pointing to structures allocated by ctypes will have the following memory layout::
+
+    Ptr( GcStruct( "CtypesGCPointer_<ClassName>
+        "contents" Ptr( GcStruct( "CtypesGcStructure_<Name>" ... ) ) ) )
+
+
+Pointers pointing returned from external functions have the follwing layout if the
+point to a structure::
+
+    Ptr( GcStruct( "CtypesPointer_<ClassName>"
+        "contents" Ptr( Struct( "CtypesStructure_<Name>" ... ) ) ) )
+
+Currently it is not decided whether assiging a pointers `contents` attribute from
+a GC-pointer should be allowed. The other case will only become valid if we implement
+structures with mixed memory state.



More information about the Pypy-commit mailing list