[pypy-svn] rev 1760 - pypy/trunk/doc/funding

[arigo at codespeak.net]

Author: arigo
Date: Sun Oct 12 14:15:35 2003
New Revision: 1760

Added:
   pypy/trunk/doc/funding/translation.png   (contents, props changed)
Removed:
   pypy/trunk/doc/funding/B6.1_plan_introduction.txt
   pypy/trunk/doc/funding/B6.2_tech_aspects.txt
   pypy/trunk/doc/funding/B6.3_risks_minimise.txt
Modified:
   pypy/trunk/doc/funding/B6.0_detailed_implementation.txt
   pypy/trunk/doc/funding/FIXME
Log:
Done B6.0, B6.1, B6.2 (detailled implementation). Please review.
Still working on B6.3 (risks).
Moved all this text into a single file, B6.0.
Added a figure.



Modified: pypy/trunk/doc/funding/B6.0_detailed_implementation.txt
==============================================================================
--- pypy/trunk/doc/funding/B6.0_detailed_implementation.txt	(original)
+++ pypy/trunk/doc/funding/B6.0_detailed_implementation.txt	Sun Oct 12 14:15:35 2003
@@ -1,62 +1,234 @@
+Detailed Implementation Plan
+=============================
+
+
 :DELETE:BEGIN
 
-B.6 Detailed Implementation Plan
-       
-This section describes in detail the work planned to achieve the
-objectives for the full duration of the of the proposed project. The
-recommended length, excluding the forms specified below, is up to 15
-pages. An introduction should explain the structure of this workplan
-plan and how the plan will lead the participants to achieve the
-objectives. The workplan should be broken down according to types of
-activities: Research, technological development and innovation related
-activities, demonstration activities and project management
-activities. It should identify significant risks, and contingency
-plans for these. The plan must for each type of activity be broken
-down into workpackages (WPs) which should follow the logical phases of
-the project, and include management of the project and assessment of
-progress and results. 
-
-Essential elements of the plan are:
-       
-a) Detailed Implementation plan introduction ­ explaining the structure 
-   of this plan and the overall methodology used to achieve the objectives.
-       
-b) Work planning, showing the timing of the different WPs and their 
-   components (Gantt chart or similar)
-       
-c) Graphical presentation of the components showing their 
-   interdependencies (Pert diagram or similar)
-            
-d) Detailed work description broken down into workpackages:
-   Workpackage list (use Workpackage list form below);
-   Deliverables list (use Deliverables list form below);
-   Description of  each workpackage (use Workpackage description form 
-   below, one per workpackage):
-           
-Note: The number of workpackages used must be appropriate to the
-complexity of the work and the overall value of the proposed
-project. Each workpackage should be a major sub-division of the
-proposed project and should also have a verifiable end-point (normally
-a deliverable or an important milestone in the overall project).  The
-planning should be sufficiently detailed to justify the proposed
-effort and allow progress monitoring by the Commission ­ the
-day-to-day management of the project by the consortium may require a
-more detailed plan.
+B6.1
+
+:DELETE:END
+
+Workplan Introduction
+~~~~~~~~~~~~~~~~~~~~~
+
+
+The PyPy project can be roughly divided into three phases, the last two of which are somewhat independent from each other:
+
+o Phase 1: The core of PyPy itself must first be developed.
+
+o Phase 2: This code base can be used as an research/integration platform of choice.
+
+o Phase 3: Specific applications can be implemented and disseminated.
+
+Moreover, several project-long infrastructure tasks are of paramount importance. In particular, coordination is assured by the project coordinator in workpackage WP01_, with the help of the management and technical boards, as described in section B5. It involves the collection and monitoring of monthly status reports, reporting to the EU, organising sprints and maintaining an internal web site in collaboration with the maintenance workpackage.
+
 
-(recommended length, excluding the forms specified above,- up to 15 pages)
+
+:DELETE:BEGIN
+
+B6.2
 
 :DELETE:END
 
-Detailed Implementation Plan
-=============================
+Research and Technological Aspects and Options
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
+Phase 1
+-------
+
+The first goal is to develop a reasonably complete Python interpreter written in Python. It must be entierely compatible with the language specification. It consists of the following major parts:
+
+o A *bytecode compiler,* which translates Python source code into an internal intermediate representation, the *bytecode.*
+
+o A *bytecode interpreter,* which interprets bytecodes and manages the supporting internal structures (frames, exception tracebacks...). It considers objects as black boxes and delegates all individual operations on them to a library of built-in types, the *Object Space.*
+
+o An *Object Space,* which captures the semantics of the various types of the language.
+
+This subdivision is common among interpreter implementations, although we place special emphasis on the library and its separation from the bytecode interpreter. The implementation will closely follow the current reference C implementation (CPython). It is thus expected to be relatively straightforward, changing only a few design decisions (mainly with respect to the Object Space separation of concerns), and changing strictly nothing to the Python language itself.
+
+This task is done in workpackage WP04_. After the initial development, WP04_ will switch its focus to implementing the number of extension modules written in C that are either standard or widely-used in CPython. (The Python Standard Library is written in a large part in Python already, so that the problem only concerns C extension modules.) For each of these, two strategies can be followed: either the module is translated into a (regular) Python module, or it is written as an extension module of the PyPy interpreter (i.e. in Python too, but at the level of the interpreter.)
+
+The result so far can only run on top of another Python implementation like CPython. It still has advantages over the existing implementation, in education (as a much more compact, modular and readable piece of code than CPython), and in flexibility (as a basis to plug in alternate Object Spaces, alternate interpreters, or alternate compilers) -- more about it below.
+
+At this point, to make PyPy stand-alone (and running at a reasonable speed), we must have restricted key areas of the source to be written in a subset of the Python language, a sublanguage (RPython) in which staticness restrictions are enforced. This sublanguage is suitable for analysis and translation into a lower-level language. Its precise definition is a balance between the amount of dynamic desired to write PyPy and the amount of effort we put in the translation tools.
+
+Note that translation is not a one-shot process; the only source code for PyPy will be in Python or RPython, and translation can be repeated freely as part of a compilation process.
+
+We are giving translation an innovative emphasis (and thus a whole workpackage, WP05_) in the project. It is not merely an RPython-to-C translator; it is an essential piece towards the flexibility goals. Numerous aspects that were design decisions influencing the whole source code of the current CPython are now merely customizable behaviour of the translator. Indeed, instead of hard-coding such design decisions, we will keep the PyPy source as simple as possible and plug the required knowledge into the translator. For example, the high-level source need not be concerned about memory management issues (garbage collection, reference counting...); this aspect can be "weaved" into the low-level code by the translator. This point is essential for the separation of concerns. It has deep advantages over the classical monolithic approach, ranging from education (the main source base is not encumbered by details) to raw performance (choice of appropriate low-level models based on real-world context-dependent measures and comparisons). Also note its extreme adaptability: instead of generating C code, it is straightforward to target other runtime environments like Java or .NET. By contrast, today's costs of maintaining several evolving implementations (CPython, Jython for Java...) are very high.
+
+The translation process itself requires some kind of analysis of the RPython code. Among the various ways to perform this analysis we will most probably choose the one based on *abstract interpretation,* as opposed to source-level or bytecode-level analysis:
+
+.. image:: translation.png
+
+The basic idea is to write an alternative "abstract" Object Space which, instead of actually performing any operation between objects, records these operations and traces the control flow. The "abstract" Object Space will be plugged into the existing bytecode interpreter; these two components will then function as an abstract (or symbolic) interpreter in the common sense. The net result is that we can actually analyse RPython source code without writing any code specific to the language (!) given that we already have a bytecode interpreter which is flexible enough to accomodate a non-standard Object Space. In other words, the combination of PyPy and an "abstract" Object Space performs as the front-end of the translator and can be used to translate (for example) the regular PyPy interpreter and its standard Object Space. Note the two different roles of the bytecode interpreter in the diagram above.
+
+Another notable advantage of this approach is that instead of operating on static source code, it works on the result of loading and initializing the code into the existing CPython interpreter. (Python, unlike more static languages, allows arbitrary computations to be performed while loading modules, e.g. initializing caches or selecting components according to external parameters.) We are thus not restricted to RPython at initialization time, which is important to acheive the configurability goals.
+
+
+Phase 2
+-------
+
+The completion of the first translated stand-alone PyPy interpreter is where the project could potentially branch into a large number of directions. Numerous exciting applications can be foreseen; we will see some of them in more details in Phase 3.
+
+The Phase 2 is concerned about research and integration of research, based on the extreme flexibility provided by the PyPy platform.
+
+Part of Phase 2 focuses essentially on performance issues, which are important in helping to establish a language implementation, and to open the language to a wide range of applications for which it were previously thought to be unsuitable.
+
+The flexibility in PyPy allows a number of designs to be reconsidered; better yet, it allows different design decisions to coexist. Indeed, most "hard" issues in interpreters have no obvious best solution; they are all depend in complicated ways to the specific details of the runtime environment and on the particular application considered and its possibly evolving context. PyPy will provide a good platform to experiment with and compare empirically several different implementations for many of these issues. For example:
+
+o The language's core object types can have several implementations with different trade-offs. To experimented with this, we will write a collection of alternatives in the "standard" Object Space implementation and heuristics to select between them. This kind of research effort is common, but PyPy can provide a good platform for real-world comparisons, and to help isolate which particular choices have which effects in an otherwise unchanged environment. Such data is notoriously hard to obtain in monolithic interpreters. This is the focus of WP06_.
+
+o Similarily, as described above, pervasive design decisions can be experimented with by tailoring the translator. This is the focus of WP07_.
+
+We will in particular investigate in detail two specific ways to customize the translator:
+
+o Generating Continuation Passing Style (CPS) low-level code. This makes the advanced notion of continuation available for the programmer; but -- most importantly in our case -- it allows the development, with the help of an appropriate runtime system, to support massive parallelism. Indeed, in almost any OS, native threads are not appropriate for massive usage. Applications (e.g. web servers handling thousands of connections) have to somehow emulate parallelism explicitely. Soft-threads are an ideal target for language integration. This work (also part of WP07_) would consist of exploiting this idea, which has been first tried for Python in the Stackless project.
+
+o Generating a JIT compiler. Existing work in the Psyco project has shown that it would actually be possible to mostly generate a JIT compiler instead of having to write it from scratch. The basic idea is again abstract interpretation: we replace the "standard" Object Space with an Object Space which, instead of actually performing any operation between two objects, *generates* machine code that can perform the required operation. Again, no change to the bytecode interpreter is needed; all we need is an Object Space able to translate individual operations to processor instructions, and supporting runtime systems. This is defined by WP08_.
+
+These two key optimisations will lead to an efficient JIT-compiling interpreter. Note that some of these optimisations can be implemented either in the original source (e.g. by writing a new Object Space) or equivalently at translation time. We will have to choose which strategy is best on a case-by-case basis.
+
+Finally, another part of Phase 2 focuses on non-performance-oriented but research-level aspects. These are the extensions enabled by the flexible division (bytecode compiler, bytecode interpreter, Object Spaces, and translator) enabled in Phase 1. For example, it would be possible to write an interpreter for a completely different language (e.g. Prolog) using the same framework, leveraging the translator and thus obtaining an efficient JIT-enabled implementation with little efforts. (Doing so for the Prolog language has been adopted as an implementation strategy in a current EU-funded project.)
+
+In the context of the PyPy project, we will not replace the interpreter altogether, but experiment with extensions:
+
+o involving feedback from the Python community, we will experiment with the proposed language extensions that are deemed worthy by the CPython developers. Numerous PEPs (Python Extension Proposals), for example, need an experimental implementation for testing before they can be accepted or rejected for integration into CPython; not only is PyPy a good platform to test them on, but this would give PyPy an active part in the effort of keeping synchronised with the development of CPython. (The role of WP03_ is to keep track of the ongoing development of the Python language and its CPython implementation, for which we will also look for existing solutions to automate the "passive" part of this effort at least partially.)
+
+o WP09_ involves the research and development of techniques inspired from logic programming. An inference engine already exists in the python-logic libraries. Successful integration of these concepts, far from regular imperative programming, would provide a validation of the level of flexibility we achieved in Phase 1. It would also provide an essential starting point for WP10_, described below.
+
+
+Phase 3
+-------
+
+The third phase of the project is to implement selected key applications of the flexibility we have enabled so far.
 
-In this section, we describe in detail the work planned to achieve the
-objectives for the full duration of the proposed project. 
+Let us stress again that Phase 3 is not meant to run only after Phase 2, but partly in parallel with it. The core flexibility on top of which we will be building is provided by the PyPy interpreter as described in Phase 1.
 
-FIXME_CHRISTIAN: see B0.1 for phases
+The applications we have selected can be categorized into the following categories:
+
+o Language-level object models;
+
+o Language-level extensions;
+
+o Interpreter adaptations.
+
+This is also where we expect third-parties to build on top of our platform.
+
+
+Language-level object models
+++++++++++++++++++++++++++++
+
+This is the topic of WP12_. We will integrate with the language itself three middleware features: security, transparent distribution, and persistence.
+
+**Security** is an important and difficult aspect that requires knowledge and support at a wide number of levels to be effective. Programming languages can help contribute to security at their level. Different object security models are possible:
+
+o The language can be artificially restricted, with dangerous operations taken out and thus impossible to achieve by interpreted programs. In effect, this amount to building a stripped-down interpreter.
+
+o Access to some objects can be denied to only some (untrusted) parts of the interpreted program; or access can go though proxies performing some kind of access checks. This is a more flexible solution, but it is more difficult to implement efficiently. Explicit checks for each access consume time. In PyPy this could be implemented as an Object Space checking and delegating operations to another Object Space.
+
+For comparison, CPython used to implement security-by-absence-of-reference: untrusted parts of a program could supposedly not obtain a reference to a dangerous object. This model, called "RExec", has been recognized as fragile and hard to maintain. It has been removed from CPython.
+
+In PyPy, one or both of the alternatives previously described can be practically implemented.
+
+**Transparent distribution** of objects over a network is another middleware feature that would benefit from being implemented at the language level. This is a vast subject which has been studied extensively. There are several implementations already available in Python, with varying degrees of transparence. None however can be fully transparent by definition of the language itself, which allows introspection -- a program using introspection features could thus defeat the delicate mecanisms introduced by a network distribution library. Moreover, such libraries typically impose additional constrains, e.g. are only able to move objects over a network if they are of a class inheriting from a particular class.
+
+We will study which of these implementations is best suited for being moved into the interpreter itself. So far, a foreseen solution would be to design a proxying Object Space that delegates operations to a remote object server over the network. We will also study approaches enabling transparent distributed execution, i.e. in which the Object Space may decide to send not each operation individually, but instead the whole algorithm's bytecode, when numerous remote objects are involved.
+
+**Persistence** is related to distribution; it is essentially a way to move objects transparently between the persistent storage and the main memory. The same remarks apply, e.g. several implementations exist already (e.g. ZODB) but would benefit from better language integration.
+
+
+Language-level extensions
++++++++++++++++++++++++++
+
+WP10_ will base itself on the language extensions developed in WP09_.
+
+FIXME_ALASTAIR write this paragraph
+
+
+Interpreter adaptations
++++++++++++++++++++++++
+
+The PyPy interpreter as developed in Phases 1 and 2 will be particularly prone to adaptation to extremely various runtime environments, from embedded devices to number-crunching machines. In WP11_ we propose to study to specific case of embedded devices, which are often limited in processor speed and memory size. This either limits the power of software that is implemented for these platforms, or enforces use of low-level approaches like C/C++ or Java. PyPy is especially suited to support such platforms, since it can produce effective and compact code, while retaining the abstraction and ease-of-use of Python.
+
+Based on the specific needs of these devices, we will experiment with memory- or battery-efficient implementations of all the customizable aspects described in Phase 2. The PyPy code generator will need an extra platform specific support module, as well as interfaces to necessary device drivers. It may make sense to develop a PyPy simulator for the target platform.
+
+We will use feedback from actual hardware to compare the results obtained. If code space permits we will implement heuristics to switch to the most efficient implementations when these are context- or application-dependent.
+
+
+Integration and Configuration
++++++++++++++++++++++++++++++
+
+It is expected that most of the applications described above will still be relatively experimental when the project ends. It is also expected that the extreme flexibility will lead to a potentially large number of different end-user configurations of PyPy. Thus our essential goal, beyond validating our techniques and showing what can be done and how it benefits from integration with the language, is to make this knowledge available and easy-to-use for contributors and third-parties.
+
+To fulfill the technical aspect of this goal, we will not only release a range of different versions of PyPy with different trade-offs and including more or less of the features developed in Phase 3, but we will also build a complete toolchain to allow programmers to choose from the available features, various implementations, and runtime restrictions and build a custom PyPy version. This is the objective of WP13_.
+
+
+Documentation and Dissemination
+-------------------------------
+
+The documentational aspect of the dissemination goal is handled by one of the few workpackages that run for the whole duration of the project.
+
+Each sprint, as well as the regular progress of non-sprint development, will 
+produce technical novelties, some of which may afford immediate use and adoption 
+of the novel technologies being developed. WP14_ will periodically produce short 
+reports on the result of "sprints" and other development. Said reports will be 
+posted to the relevant mailing lists as well as archived on both the PyPy and 
+the Python Business Forum website for universal availability. 
+
+Python community members will be encouraged to keep current with the project 
+through the short reports, while community feedback on technical developments 
+will be gathered on the website and mailing lists to be used by the appropriate 
+project areas to further enhance the project's development efforts, in true Open 
+Source spirit.
+
+In addition to the periodically produced short reports, WP14_ will on occasion 
+present longer, detailed reports to the Commission, and to scientific committees 
+advising the Commission on technical issues. Technical papers and talks will be 
+submitted to scientific conferences which deal with the subject matters covered 
+by the project. When the advancement of the project warrants it, WP14_ will also 
+publish "popularization" articles and tutorial materials to help other 
+practitioners of software development to make practical use of the project's 
+results. Diagrams and schematics will be provided to illustrate fundamental 
+concepts, as appropriate to the audience and the subject matter. 
+
+
+Management
+----------
+
+PyPy's own development needs an infrastructure that must continuously be kept up-to-date and further developed. The role of WP02_ is decisive in choosing and adopting modern development environments. The main source repository is placed under control of Subversion, a Concurrent Versioning System. This workpackage also includes maintaining and possibly redesigning frameworks for unit-testing -- unit-tests are essential in the development process in sprints. The website makes heavy use of collaborative tools like Wiki pages, a content management system, and issue trackers.
+
+Other tasks include:
+
+o Subversion maintenance (e.g. notification hooks and pre/post-commit restrictions);
+
+o providing access over http/https and ssh server;
+
+o building extensions for automatic document extraction;
+
+o maintenance or setup of mailing lists;
+
+o implementing a search facility over all content;
+
+o maintaining or extending the issue tracker;
+
+o maintaining and enhancing the website infrastructure;
+
+o performing backups for all relevant information/code;
+
+o setting up a mirror repository which is kept up-to-date and which can be used readonly in case of failure of the main repository;
+  
+o taking care of regular repository backups, designing a strict backup policy suitable for the project, and adhering to it.
+
+
+:DELETE:BEGIN
+
+B6.3
+
+:DELETE:END
 
-Phase 1 is technical development of State of The Art interpreter
+Risks in the Project and Steps to Minimise
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Phase 2 is research and innovation to produce high performance interpreter
 
-Phase 3 is validation and dissemination through implementation of applications
+FIXME_ARMIN I'm working on it.

Deleted: /pypy/trunk/doc/funding/B6.1_plan_introduction.txt
==============================================================================
--- /pypy/trunk/doc/funding/B6.1_plan_introduction.txt	Sun Oct 12 14:15:35 2003
+++ (empty file)
@@ -1,398 +0,0 @@
-FIXME_CHRISTIAN: finish this
-
-.. include:: crossreferences.asc
-
-Workplan Introduction
-===============================
-
-The PyPy project addresses a number of topics, which
-can be categorized into the following working packages.
-Each group is further broken down into tasks as necessary.
-
-- Development of PyPy itself, as a Python written in Python.
-  This includes code generation for various targets, re-implementation
-  of all builtin Python objects and some extension modules, 
-  and the development of several object spaces, testing and documentation.
-  
-  - The PyPy Interpreter
-  
-  - The PyPy Compiler
-  
-  - Bootstrapping PyPy
-  
-  - High-performance PyPy-Python
-
-- Validation of PyPy in real applications.
-  Generating down-sized code for embedded systems, load balancing
-  in a distributed network, code generators optimized for number crunching
-  on some processor architectures and re-writing numerical Python
-  packages should be able to prove that PyPy is very appropriate 
-  for industrial strength applications.
-
-  - Supporting Embedded Devices
-  
-  - Load Balancing in a Multi-Processor environment
-  
-  - Numerical Applications
-  
-- Infra structure tasks carried out throughout the whole project
-
-  - Coordination and Management
-
-  - Project Documentation and Dissemination
-
-  - Maintenance of the development environment
-    
-  - Synchronisation with Standard Python
-  
-
-WP01_: Coordination and Management
--------------------------------------
-WP1 continues throughout the duration of the project and is carried out
-by the project coordinator. It involves the collection and monitoring of
-monthly status reports, reporting to the EU, organising meetings
-and maintaining an internal web site. The website will make heavy
-use of collaborative tools like Wiki pages, a content management system,
-and issue trackers.
-
-  
-WP04_: The PyPy Interpreter
----------------------------
-
-The goal is to make a complete Python interpreter that runs under any
-existing Python implementation.
-
-WPXXX21: Development and Completion of the PyPy Interpreter
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  The PyPy interpreter itself should be developed and completed
-  as a regular Python/RPython program. This package includes all
-  parts of CPython that we don't want to move to WPXXX22.
-  Further investigation is needed concerning the 
-  multimethod concepts of the standard object space,
-  and how to hook in the bytecode compiler.
-
-
-WPXXX22: Porting CPython C-sourcecode to regular Python
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  Port all parts of CPython which we don't want to
-  implement in WPXXX21 into regular Python libraries.
-  These ones should also work without PyPy, being just plain-Python
-  replacements for existing CPython functionality.
-  This includes the bytecode compiler, which definitely should
-  become a regular Python program, instead of being built
-  into the interpreter.
-
-
-WP05_: The PyPy Translation
----------------------------
-
-WPXXX31: Translation of RPython
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  RPython is a restricted version of Python, which is much
-  more statically defined than standard Python. It allows
-  to write code that is rather easily translated into something
-  else, like C source, Pyrex code, or even passed to
-  an assembly code generator.
-  The Definition of RPython is slightly in flux and should
-  be adjusted during WPXXX31.
-  
-  The goal is to be able to translate arbitrary RPython source code (e.g. 
-  the one produced in WPXXX21 into low-level code (C, Pyrex, Java, others). 
-  This includes making a stand-alone, not-PyPy-related tool for general
-  optimisation of arbitrary but suitably restricted Python application or
-  parts thereof.
-
-Bootstrapping PyPy
-~~~~~~~~~~~~~~~~~~~
-
-The goal is to put interpreter (WPXXX21, WPXXX22) and translator
-(WPXXX31) together.
-
-The interpreter is written as an RPython program. The translator
-has to translate this program into some low-level language.
-The resulting program then needs to be supported by a special
-runtime library. The work-flow of WPXXX32 is repetitive,
-since it will not be possible to "get it right" in the
-first attempt. Analysis and redesign will have to be
-repeated until the result is satisfactory.
-
-
-WPXXX32: Specific Analysis and Redesign
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  The global translation of PyPy is going to raise particular
-  problems, other than more general RPython programs do.
-  Since translation of RPython is the core idea of the
-  bootstrap process and the main target of the translator,
-  we need to investigate and isolate the
-  particular problems, and redesign specific parts of PyPy
-  to support translation, code generation and optimisation
-  better. This will also include a re-iteration of the interface
-  design between application level and interpreter level
-  until we reach overall convergence.
-  
-
-WPXXX33: Low Level PyPy Runtime
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  In order to give a working environment to the translated RPython
-  program, we need to build the low-level-specific **run-time**
-  components of PyPy.
-  Most notably are the object layout, the memory management,
-  and possibly threading support, as well as an
-  efficient multimethod dispatch.
-  The target language is not yet decided on. We might go different
-  paths at the same time.
-  If producing C code is a target, important
-  parts can be directly re-used from CPython.
-
-
-High-performance PyPy-Python
-------------------------------------------
-
-The goal is to optimize Bootstrapping in possibly various ways,
-building on its flexibility to go beyond CPython.
-The main lack of flexibility in CPython stems from the
-fact that all structures are hard-coded in C, and there is
-no abstraction layer. PyPy does provide this abstraction layer,
-since its RPython implementation is not meant to be
-executed directly, but goes though a code generator which
-produces the actual machine code. This layer is highly configurable.
-
-Integration of Existing Technology
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  There are existing projects, notably Psyco_ and Stackless_,
-  which have been traditionally dependent on closely
-  following CPython's code base.
-  
-  Both will be rewritten as a meta-component that hooks into the
-  translator plus a dedicated run-time component (WPXXX33).
-  
-  As a side effect, after successful completion of the
-  PyPy project, they no longer need to exist as stand-alone
-  projects.
-  
-
-WPXXX41: Integration of _`Stackless`
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  Stackless Python has implemented high speed multitasking in
-  a single thread for C Python in two different ways:
-  Continuation passing and stack switching. Both ways of
-  task switching can be integrated into PyPy. Furthermore,
-  pickling of running programs has been implemented in
-  Stackless Python and should enable PyPy for load-balancing
-  between different machines.
-
-
-WP06_: Core Optimisations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  Since the standard object space allows coexistence of different
-  implementations of the same type, we can develop
-  **several object implementations**.
-  We will develop heuristics to switch between different
-  implementations during execution.
-  The goal is to study the efficiency of different approaches,
-  with the possibility to change the default implementation
-  in favor of a different one, not known in CPython.
-  Some object layouts will further exist in parallel, if their
-  efficiency is highly application dependant.
-
-XXX: reformulate this.
-  The following objects are mostly independent from any particular low-level run-time 
-  environment.
-
-WP09_: Language Research and Extension
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-XXX write it.
-
-WP07_: Translator Optimisations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  It should be identified, which **optimisations** would benefit
-  from support from the translator. These are the optimisations
-  not easily available to CPython because they would require
-  large-scale code rewrites.
-  This includes design-considerations including the decision whether
-  to use reference counting together with garbage collection,
-  or whether to go for garbage collection, only.
-  
-  The meta-programming underlying WP07_
-  -- namely the work on the translator instead of on the resulting code -- is
-  what gives us the possibility of actually implementing several very
-  different schemes. The outcome of this effort is of course not unique,
-  but depends on the specific target of the optimisation.
-  There will be at least two efforts at the same time:
-  
-  - optimisation towards maximum performance of an integrated PyPy system
-  
-  - optimisation towards minimal code size for embedded systems.
-  
-
-WP06_: Integration of _`Psyco`
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  Psyco provides techniques to increase the performance of C
-  Python by generating specialized machine code at run-time. 
-  Developing C Psyco has proved that more flexibility would be of
-  paramount importance to the project. All prior knowledge of the
-  Psyco project will thus be integrated into PyPy, as Python and
-  RPython code, enabling yet more efficient optimisations and
-  allowing new processor architectures to be targeted.
-  
-
-WP11_: Validation of PyPy
----------------------------
-
-The goal is to provide a validation of the advancements of
-the state of the art and the real world applicability of these.
-
-Supporting Embedded Devices
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Embedded devices are often limited in processor speed and
-memory size, which either limits the power of software that
-is implemented for these platforms, or enforces use of
-low-level approaches like C/C++ or Java.
-PyPy is especially suited to support such platforms, since
-it can produce highly effective, compact code, while retaining
-the abstraction and ease-of-use of Python.
-
-
-WPXXX51: Porting PyPy to an Embedded Device
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  Dependant from the actual processor architecture, PyPy's
-  code generator needs an extra platform specific support
-  module. Interfaces to necessary device drivers are needed
-  as well as it makes sense to develop a PyPy simulator for
-  the target platform.
-
-WPXXX52: Evaluation whether to do a small OS in PyPy
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  
-  Implementing a small operating system in PyPy raises
-  some new questions and opens a new category of problems.
-  One of them is the possible need to write an IP stack
-  in PyPy, and a number of device drivers as well.
-  In this short task, we will figure out whether it is
-  feasible to carry on with the next task.
-
-WPXXX53: A small Operating System for an Embedded Device
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-  *we are thinking of a printer here, but don't have facts yet*
-  
-  In extent to WPXXX51, it makes sense to write the whole
-  operating system in PyPy. We need a low-level extension
-  to the code generator that allows to access I/O ports.
-  Interrupts need to be supported as well, and primitive
-  access to persistant storage.
-  Ideally, we can create a single threaded PyPy OS with
-  a prioritized scheduler that runs the OS parts together
-  with multiple Python application. The latter depends on
-  WPXXX41 very much.
-
-
-Numerical Applications
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-PyPy is extremely flexible, since it supports static
-and dynamic code generation by pluggable processor
-architecture modules. PyPy can use special hardware
-by itself, or it can cross-compile code for target
-machines which will run only a downsized PyPy
-instantiation which is runtime only.
-
-One advantage of using PyPy instead of other compiled
-libraries is its ability to not only support the
-special hardware, but also to adjust to the given
-machine properties, like cache sizes, number of
-parallel FPUs, memory access speed and memory size.
-PyPy is able to probe these parameters and to choose
-an optimum implementation for the particular hardware
-configuration during startup time.
-
-
-WPXXX54: Code Generator for SIMD Architecture
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  In order to support numerical applications better,
-  PyPy should be extended to support special hardware
-  like SIMD (Single Instruction Multiple Data) processors.
-  We are aiming to pick one SIMD architecture, like
-  SSE2 or AltiVec, and extend the code generator to support
-  and optimize for the new instructions.
-  We don't intend to support a completely new instruction
-  set, but prefer to choose an extension to an architecture
-  that we are already supporting.
-  
-  The existence of special instructions and other extensions
-  should be probed using run-time checks. Psyco should be
-  extended to be aware of parallel instructions, and be
-  enabled to emit optimized code for that.
-
-  PyPy needs to be extended to support a vectorized
-  data type from the Python level, as well.
-  The specification of that is part of this task.
-
-WPXXX55: Enhanced Numerical Package
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-  
-  Rewrite a numerical Python package like NumPy
-  using RPython. Identify the numerical operations
-  in the package which are candidates for parallelization.
-  Write an interface that allows to implement these operations
-  in either a traditional way or using SIMD instructions,
-  dependent from the available capabilities of the current
-  code generator.
-
-WPXXX56: Load Balancing in a Multi-Processor environment
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-*XXX add some general words here from the wp*
-
-
-WP12_: Integration and Configuration
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-XXX write it.
-
-Infrastructure
-------------------
-
-
-WP14_: Project Documentation and Dissemination
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-During the whole project, a set of documents will be maintained
-that documents the current status of the project, results of
-discussions, the planning of new sprints and their status reports,
-as well as the preparation of papers for presentation on
-various congresses.
-
-In extent, there is an ongoing information flow to external
-communities, like the Python Business Forum (PBF) and the Python
-developers list (python-dev at python.org).
-
-WP02_: Maintenance of Tools
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-PyPy's own development needs an infrastructure that must continuously
-be kept up-to-date and further developed. This includes maintenance
-of the Subversion package and tracking of its ongoing development,
-maintenance of the Subversion repository, extensions like automatic
-documentation extraction, and change notification via mailing lists.
-
-
-WP03_: Synchronisation with Standard Python
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Since Python is being further developed all the time, there is
-a continuous need of keeping PyPy in sync with the future developments
-of CPython, e.g. ways to relate pieces of PyPy source and pieces of CPython
-source. This work is carried out by hand at the moment. We
-do look for existing solutions which allow to automate this
-effort at least partially.
-
-

Deleted: /pypy/trunk/doc/funding/B6.2_tech_aspects.txt
==============================================================================
--- /pypy/trunk/doc/funding/B6.2_tech_aspects.txt	Sun Oct 12 14:15:35 2003
+++ (empty file)
@@ -1,8 +0,0 @@
-FIXME_LAURA FIXME_NICO: finish this
-
-.. include:: crossreferences.asc
-
-Research and Technological Aspects and Options
-==================================================
-
-FIXME something must go here.
\ No newline at end of file

Deleted: /pypy/trunk/doc/funding/B6.3_risks_minimise.txt
==============================================================================
--- /pypy/trunk/doc/funding/B6.3_risks_minimise.txt	Sun Oct 12 14:15:35 2003
+++ (empty file)
@@ -1,9 +0,0 @@
-FIXME_LAURA FIXME_NICO: finish this
-
-.. include:: crossreferences.asc
-
-Risks in the Project and Steps to Minimise
-=============================================
-
-FIXME: some text must go here.
-

Modified: pypy/trunk/doc/funding/FIXME
==============================================================================
--- pypy/trunk/doc/funding/FIXME	(original)
+++ pypy/trunk/doc/funding/FIXME	Sun Oct 12 14:15:35 2003
@@ -103,20 +103,20 @@
 
 B6.0_detailed_implementation
 Should be empty.
-NOT STARTED
+READY FOR REVIEW: Armin
 
 B6.1_plan_introduction
 Needs checking and editing.
 Material consistent with work packages?
-NOT STARTED
+DONE (merged into B6.0)
 
 B6.2_tech_aspects
 Needs writing.
-NOT STARTED
+DONE (merged into B6.0)
 
 B6.3_risks_minimise
 Needs writing.
-NOT STARTED
+STARTED: Armin
 
 B6.4_gantt
 Needs checking.

Added: pypy/trunk/doc/funding/translation.png
==============================================================================
Binary file. No diff available.