[pypy-commit] pypy online-transforms: hg merge default
rlamy
noreply at buildbot.pypy.org
Wed Oct 29 18:31:35 CET 2014
Author: Ronan Lamy <ronan.lamy at gmail.com>
Branch: online-transforms
Changeset: r74275:16b400a3b68d
Date: 2014-10-29 17:30 +0000
http://bitbucket.org/pypy/pypy/changeset/16b400a3b68d/
Log: hg merge default
diff too long, truncating to 2000 out of 36241 lines
diff --git a/.hgignore b/.hgignore
--- a/.hgignore
+++ b/.hgignore
@@ -89,6 +89,7 @@
^pypy/doc/image/lattice3\.png$
^pypy/doc/image/stackless_informal\.png$
^pypy/doc/image/parsing_example.+\.png$
+^rpython/doc/_build/.*$
^pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test\.o$
^compiled
^.git/
diff --git a/README.rst b/README.rst
--- a/README.rst
+++ b/README.rst
@@ -23,6 +23,7 @@
the pypy-dev team <pypy-dev at python.org>
+
Building
========
diff --git a/ctypes_configure/doc/configure.txt b/ctypes_configure/doc/configure.txt
--- a/ctypes_configure/doc/configure.txt
+++ b/ctypes_configure/doc/configure.txt
@@ -19,6 +19,4 @@
usage
=====
-`sample.py`_ explains in details how to use it.
-
-.. _`sample.py`: http://codespeak.net/svn/pypy/dist/ctypes_configure/doc/sample.py
+:source:`sample.py <ctypes_configure/doc/sample.py>` explains in details how to use it.
diff --git a/pypy/config/test/test_pypyoption.py b/pypy/config/test/test_pypyoption.py
--- a/pypy/config/test/test_pypyoption.py
+++ b/pypy/config/test/test_pypyoption.py
@@ -64,7 +64,7 @@
def check_file_exists(fn):
assert configdocdir.join(fn).check()
- from pypy.doc.config.confrest import all_optiondescrs
+ from pypy.doc.config.generate import all_optiondescrs
configdocdir = thisdir.dirpath().dirpath().join("doc", "config")
for descr in all_optiondescrs:
prefix = descr._name
diff --git a/pypy/doc/TODO b/pypy/doc/TODO
new file mode 100644
--- /dev/null
+++ b/pypy/doc/TODO
@@ -0,0 +1,40 @@
+Documentation TODO
+==================
+
+General
+-------
+
+* architecture documents don't really show the separation between PyPy and
+ RPython
+ * architecture.rst is duplicate (both pypy and rpython)
+* Consider moving information from pypy/doc/{build,windows}.rst to rpython/doc
+
+
+Cleanup
+~~~~~~~
+
+* remove documentation on removed features
+ * various object spaces
+* update / remove dead links
+
+
+Meta
+~~~~
+
+* work on configuration/options documentation generation
+
+
+PyPy
+----
+
+* Update coding guide
+* Move links from project-documentation.rst to index.rst and consider killing
+ it.
+
+
+RPython
+-------
+
+* make translation.rst a high-level overview and move details in their own
+ documents
+* redo various outdated pictures in translation.rst
diff --git a/pypy/doc/__pypy__-module.rst b/pypy/doc/__pypy__-module.rst
--- a/pypy/doc/__pypy__-module.rst
+++ b/pypy/doc/__pypy__-module.rst
@@ -1,20 +1,17 @@
-
.. comment: this document is very incomplete, should we generate
it automatically?
-=======================
The ``__pypy__`` module
=======================
The ``__pypy__`` module is the main entry point to special features provided
-by PyPy's standard interpreter. Its content depends on `configuration options`_
-which may add new functionality and functions whose existence or non-existence
-indicates the presence of such features.
+by PyPy's standard interpreter. Its content depends on :doc:`configuration options <config/index>`
+which may add new functionality and functions whose existence or non-existence
+indicates the presence of such features.
-.. _`configuration options`: config/index.html
Generally available functionality
-=================================
+---------------------------------
- ``internal_repr(obj)``: return the interpreter-level representation of an
object.
@@ -22,28 +19,22 @@
It works like a simplified array of characters (actually, depending on the
configuration the ``array`` module internally uses this).
+
Transparent Proxy Functionality
-===============================
+-------------------------------
-If `transparent proxies`_ are enabled (with :config:`objspace.std.withtproxy`)
+If :ref:`transparent proxies <tproxy>` are enabled (with :config:`objspace.std.withtproxy`)
the following functions are put into ``__pypy__``:
- ``tproxy(typ, controller)``: Return something that looks like it is of type
typ. Its behaviour is completely controlled by the controller. See the docs
- about `transparent proxies`_ for detail.
-
+ about :ref:`transparent proxies <tproxy>` for detail.
- ``get_tproxy_controller(obj)``: If obj is really a transparent proxy, return
its controller. Otherwise return None.
-.. _`transparent proxies`: objspace-proxies.html#tproxy
-
Functionality available on py.py (not after translation)
-========================================================
+--------------------------------------------------------
- ``isfake(obj)``: returns True if ``obj`` is faked.
-
- ``interp_pdb()``: start a pdb at interpreter-level.
-
-
-
diff --git a/pypy/doc/_ref.txt b/pypy/doc/_ref.txt
deleted file mode 100644
--- a/pypy/doc/_ref.txt
+++ /dev/null
@@ -1,114 +0,0 @@
-.. This file is generated automatically by makeref.py script,
- which in turn is run manually.
-
-.. _`ctypes_configure/doc/sample.py`: https://bitbucket.org/pypy/pypy/src/default/ctypes_configure/doc/sample.py
-.. _`dotviewer/`: https://bitbucket.org/pypy/pypy/src/default/dotviewer/
-.. _`lib-python/`: https://bitbucket.org/pypy/pypy/src/default/lib-python/
-.. _`lib-python/2.7/dis.py`: https://bitbucket.org/pypy/pypy/src/default/lib-python/2.7/dis.py
-.. _`lib_pypy/`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/
-.. _`lib_pypy/greenlet.py`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/greenlet.py
-.. _`lib_pypy/tputil.py`: https://bitbucket.org/pypy/pypy/src/default/lib_pypy/tputil.py
-.. _`pypy/bin/`: https://bitbucket.org/pypy/pypy/src/default/pypy/bin/
-.. _`pypy/bin/pyinteractive.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/bin/pyinteractive.py
-.. _`pypy/config/`: https://bitbucket.org/pypy/pypy/src/default/pypy/config/
-.. _`pypy/config/pypyoption.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/config/pypyoption.py
-.. _`pypy/doc/`: https://bitbucket.org/pypy/pypy/src/default/pypy/doc/
-.. _`pypy/doc/config/`: https://bitbucket.org/pypy/pypy/src/default/pypy/doc/config/
-.. _`pypy/doc/discussion/`: https://bitbucket.org/pypy/pypy/src/default/pypy/doc/discussion/
-.. _`pypy/goal/`: https://bitbucket.org/pypy/pypy/src/default/pypy/goal/
-.. _`pypy/interpreter`:
-.. _`pypy/interpreter/`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/
-.. _`pypy/interpreter/argument.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/argument.py
-.. _`pypy/interpreter/astcompiler`:
-.. _`pypy/interpreter/astcompiler/`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/
-.. _`pypy/interpreter/astcompiler/assemble.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/assemble.py
-.. _`pypy/interpreter/astcompiler/ast.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/ast.py
-.. _`pypy/interpreter/astcompiler/astbuilder.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/astbuilder.py
-.. _`pypy/interpreter/astcompiler/asthelpers.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/asthelpers.py
-.. _`pypy/interpreter/astcompiler/codegen.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/codegen.py
-.. _`pypy/interpreter/astcompiler/optimize.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/optimize.py
-.. _`pypy/interpreter/astcompiler/symtable.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/symtable.py
-.. _`pypy/interpreter/astcompiler/tools/Python.asdl`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/tools/Python.asdl
-.. _`pypy/interpreter/astcompiler/tools/asdl_py.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/astcompiler/tools/asdl_py.py
-.. _`pypy/interpreter/baseobjspace.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/baseobjspace.py
-.. _`pypy/interpreter/eval.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/eval.py
-.. _`pypy/interpreter/executioncontext.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/executioncontext.py
-.. _`pypy/interpreter/function.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/function.py
-.. _`pypy/interpreter/gateway.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/gateway.py
-.. _`pypy/interpreter/mixedmodule.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/mixedmodule.py
-.. _`pypy/interpreter/module.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/module.py
-.. _`pypy/interpreter/pyframe.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyframe.py
-.. _`pypy/interpreter/pyopcode.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyopcode.py
-.. _`pypy/interpreter/pyparser`:
-.. _`pypy/interpreter/pyparser/`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyparser/
-.. _`pypy/interpreter/pyparser/future.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyparser/future.py
-.. _`pypy/interpreter/pyparser/metaparser.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyparser/metaparser.py
-.. _`pypy/interpreter/pyparser/parser.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyparser/parser.py
-.. _`pypy/interpreter/pyparser/pyparse.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyparser/pyparse.py
-.. _`pypy/interpreter/pyparser/pytokenizer.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/pyparser/pytokenizer.py
-.. _`pypy/interpreter/typedef.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/interpreter/typedef.py
-.. _`pypy/module`:
-.. _`pypy/module/`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/
-.. _`pypy/module/__builtin__/__init__.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/__builtin__/__init__.py
-.. _`pypy/module/cppyy/capi/__init__.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/capi/__init__.py
-.. _`pypy/module/cppyy/capi/builtin_capi.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/capi/builtin_capi.py
-.. _`pypy/module/cppyy/include/capi.h`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/cppyy/include/capi.h
-.. _`pypy/module/test_lib_pypy/`: https://bitbucket.org/pypy/pypy/src/default/pypy/module/test_lib_pypy/
-.. _`pypy/objspace/`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/
-.. _`pypy/objspace/std`:
-.. _`pypy/objspace/std/`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/
-.. _`pypy/objspace/std/bytesobject.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/bytesobject.py
-.. _`pypy/objspace/std/multimethod.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/multimethod.py
-.. _`pypy/objspace/std/objspace.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/objspace.py
-.. _`pypy/objspace/std/proxy_helpers.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/proxy_helpers.py
-.. _`pypy/objspace/std/proxyobject.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/proxyobject.py
-.. _`pypy/objspace/std/strbufobject.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/strbufobject.py
-.. _`pypy/objspace/std/transparent.py`: https://bitbucket.org/pypy/pypy/src/default/pypy/objspace/std/transparent.py
-.. _`pypy/tool/`: https://bitbucket.org/pypy/pypy/src/default/pypy/tool/
-.. _`pypy/tool/pytest/`: https://bitbucket.org/pypy/pypy/src/default/pypy/tool/pytest/
-.. _`rpython/annotator`:
-.. _`rpython/annotator/`: https://bitbucket.org/pypy/pypy/src/default/rpython/annotator/
-.. _`rpython/annotator/annrpython.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/annotator/annrpython.py
-.. _`rpython/annotator/binaryop.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/annotator/binaryop.py
-.. _`rpython/annotator/builtin.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/annotator/builtin.py
-.. _`rpython/bin/translatorshell.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/bin/translatorshell.py
-.. _`rpython/config/`: https://bitbucket.org/pypy/pypy/src/default/rpython/config/
-.. _`rpython/config/translationoption.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/config/translationoption.py
-.. _`rpython/flowspace/`: https://bitbucket.org/pypy/pypy/src/default/rpython/flowspace/
-.. _`rpython/flowspace/model.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/flowspace/model.py
-.. _`rpython/memory/`: https://bitbucket.org/pypy/pypy/src/default/rpython/memory/
-.. _`rpython/memory/gc/generation.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/memory/gc/generation.py
-.. _`rpython/memory/gc/hybrid.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/memory/gc/hybrid.py
-.. _`rpython/memory/gc/minimarkpage.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/memory/gc/minimarkpage.py
-.. _`rpython/memory/gc/semispace.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/memory/gc/semispace.py
-.. _`rpython/rlib`:
-.. _`rpython/rlib/`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/
-.. _`rpython/rlib/listsort.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/listsort.py
-.. _`rpython/rlib/nonconst.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/nonconst.py
-.. _`rpython/rlib/objectmodel.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/objectmodel.py
-.. _`rpython/rlib/parsing/`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/parsing/
-.. _`rpython/rlib/parsing/tree.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/parsing/tree.py
-.. _`rpython/rlib/rarithmetic.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/rarithmetic.py
-.. _`rpython/rlib/rbigint.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/rbigint.py
-.. _`rpython/rlib/rrandom.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/rrandom.py
-.. _`rpython/rlib/rsocket.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/rsocket.py
-.. _`rpython/rlib/streamio.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/streamio.py
-.. _`rpython/rlib/test`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/test/
-.. _`rpython/rlib/unroll.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rlib/unroll.py
-.. _`rpython/rtyper`:
-.. _`rpython/rtyper/`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/
-.. _`rpython/rtyper/lltypesystem/`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/lltypesystem/
-.. _`rpython/rtyper/lltypesystem/lltype.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/lltypesystem/lltype.py
-.. _`rpython/rtyper/rint.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/rint.py
-.. _`rpython/rtyper/rlist.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/rlist.py
-.. _`rpython/rtyper/rmodel.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/rmodel.py
-.. _`rpython/rtyper/rtyper.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/rtyper.py
-.. _`rpython/rtyper/test/test_llinterp.py`: https://bitbucket.org/pypy/pypy/src/default/rpython/rtyper/test/test_llinterp.py
-.. _`rpython/tool/algo/`: https://bitbucket.org/pypy/pypy/src/default/rpython/tool/algo/
-.. _`rpython/translator`:
-.. _`rpython/translator/`: https://bitbucket.org/pypy/pypy/src/default/rpython/translator/
-.. _`rpython/translator/backendopt/`: https://bitbucket.org/pypy/pypy/src/default/rpython/translator/backendopt/
-.. _`rpython/translator/c/`: https://bitbucket.org/pypy/pypy/src/default/rpython/translator/c/
-.. _`rpython/translator/c/src/stacklet/`: https://bitbucket.org/pypy/pypy/src/default/rpython/translator/c/src/stacklet/
-.. _`rpython/translator/c/src/stacklet/stacklet.h`: https://bitbucket.org/pypy/pypy/src/default/rpython/translator/c/src/stacklet/stacklet.h
-.. _`rpython/translator/tool/`: https://bitbucket.org/pypy/pypy/src/default/rpython/translator/tool/
diff --git a/pypy/doc/architecture.rst b/pypy/doc/architecture.rst
--- a/pypy/doc/architecture.rst
+++ b/pypy/doc/architecture.rst
@@ -1,93 +1,29 @@
-==================================================
-Goals and Architecture Overview
-==================================================
+Goals and Architecture Overview
+===============================
.. contents::
+This document gives an overview of the goals and architecture of PyPy. If you're
+interested in :ref:`using PyPy <using-pypy>` or :ref:`hacking on it <developing-pypy>`,
+have a look at our :ref:`getting started <getting-started-index>` section.
-This document gives an overview of the goals and architecture of PyPy.
-See `getting started`_ for a practical introduction and starting points.
-Mission statement
-====================
+Mission statement
+-----------------
-We aim to provide:
+We aim to provide a compliant, flexible and fast implementation of the Python_
+Language which uses the RPython toolchain to enable new advanced high-level
+features without having to encode the low-level details. We call this PyPy.
- * a common translation and support framework for producing
- implementations of dynamic languages, emphasizing a clean
- separation between language specification and implementation
- aspects. We call this the `RPython toolchain`_.
+.. _Python: http://docs.python.org/reference/
- * a compliant, flexible and fast implementation of the Python_ Language
- which uses the above toolchain to enable new advanced high-level features
- without having to encode the low-level details. We call this PyPy.
-
-By separating concerns in this way, our implementation
-of Python - and other dynamic languages - is able to automatically
-generate a Just-in-Time compiler for any dynamic language. It also
-allows a mix-and-match approach to implementation decisions, including
-many that have historically been outside of a user's control, such as
-target platform, memory and
-threading models, garbage collection strategies, and optimizations applied,
-including whether or not to have a JIT in the first place.
High Level Goals
-=============================
-
-RPython - the Translation Toolchain
------------------------------------------------
-
-Traditionally, language interpreters are written in a target platform language
-such as C/Posix, Java or C#. Each implementation provides
-a fundamental mapping between application source code and the target
-environment. One of
-the goals of the "all-encompassing" environments, such as the .NET framework
-and to some extent the Java virtual machine, is to provide standardized
-and higher level functionalities in order to support language implementers
-for writing language implementations.
-
-PyPy is experimenting with a more ambitious approach. We are using a
-subset of the high-level language Python, called RPython_, in which we
-write languages as simple interpreters with few references to and
-dependencies on lower level details. The `RPython toolchain`_
-produces a concrete virtual machine for the platform of our choice by
-inserting appropriate lower level aspects. The result can be customized
-by selecting other feature and platform configurations.
-
-Our goal is to provide a possible solution to the problem of language
-implementers: having to write ``l * o * p`` interpreters for ``l``
-dynamic languages and ``p`` platforms with ``o`` crucial design
-decisions. PyPy aims at making it possible to change each of these
-variables independently such that:
-
-* ``l``: the language that we analyze can be evolved or entirely replaced;
-
-* ``o``: we can tweak and optimize the translation process to produce
- platform specific code based on different models and trade-offs;
-
-* ``p``: we can write new translator back-ends to target different
- physical and virtual platforms.
-
-By contrast, a standardized target environment - say .NET -
-enforces ``p=1`` as far as it's concerned. This helps making ``o`` a
-bit smaller by providing a higher-level base to build upon. Still,
-we believe that enforcing the use of one common environment
-is not necessary. PyPy's goal is to give weight to this claim - at least
-as far as language implementation is concerned - showing an approach
-to the ``l * o * p`` problem that does not rely on standardization.
-
-The most ambitious part of this goal is to `generate Just-In-Time
-Compilers`_ in a language-independent way, instead of only translating
-the source interpreter into an interpreter for the target platform.
-This is an area of language implementation that is commonly considered
-very challenging because of the involved complexity.
-
-
-PyPy - the Python Interpreter
---------------------------------------------
+----------------
Our main motivation for developing the translation framework is to
-provide a full featured, customizable, fast_ and `very compliant`_ Python
+provide a full featured, customizable, :ref:`fast <how-fast-is-pypy>` and
+:doc:`very compliant <cpython_differences>` Python
implementation, working on and interacting with a large variety of
platforms and allowing the quick introduction of new advanced language
features.
@@ -106,88 +42,22 @@
and fully orthogonal to the interpreter source code.
-PyPy Architecture
-===========================
-
-As you would expect from a project implemented using ideas from the world
-of `Extreme Programming`_, the architecture of PyPy has evolved over time
-and continues to evolve. Nevertheless, the high level architecture is
-stable. As described above, there are two rather independent basic
-subsystems: the `PyPy Python Interpreter`_ and the `RPython Translation Toolchain`_.
-
-.. _`translation framework`:
-
-RPython Translation Toolchain
--------------------------
-
-The job of the RPython toolchain is to translate RPython_ programs
-into an efficient version of that program for one of the various target
-platforms, generally one that is considerably lower-level than Python.
-
-The approach we have taken is to reduce the level of abstraction of the
-source RPython program in several steps, from the high level down to the
-level of the target platform, whatever that may be. Currently we
-support two broad flavours of target platforms: the ones that assume a
-C-like memory model with structures and pointers, and the ones that
-assume an object-oriented model with classes, instances and methods (as,
-for example, the Java and .NET virtual machines do).
-
-The RPython toolchain never sees the RPython source code or syntax
-trees, but rather starts with the *code objects* that define the
-behaviour of the function objects one gives it as input. It can be
-considered as "freezing" a pre-imported RPython program into an
-executable form suitable for the target platform.
-
-The steps of the translation process can be summarized as follows:
-
-* The code object of each source functions is converted to a `control
- flow graph` by the `Flow Object Space`_.
-
-* The control flow graphs are processed by the Annotator_, which
- performs whole-program type inference to annotate each variable of
- the control flow graph with the types it may take at run-time.
-
-* The information provided by the annotator is used by the RTyper_ to
- convert the high level operations of the control flow graphs into
- operations closer to the abstraction level of the target platform.
-
-* Optionally, `various transformations`_ can then be applied which, for
- example, perform optimizations such as inlining, add capabilities
- such as stackless-style concurrency, or insert code for the
- `garbage collector`_.
-
-* Then, the graphs are converted to source code for the target platform
- and compiled into an executable.
-
-This process is described in much more detail in the `document about
-the RPython toolchain`_ and in the paper `Compiling dynamic language
-implementations`_.
-
-.. _`control flow graph`: translation.html#the-flow-model
-.. _`Flow Object Space`: objspace.html#the-flow-object-space
-.. _Annotator: translation.html#the-annotation-pass
-.. _RTyper: rtyper.html#overview
-.. _`various transformations`: translation.html#the-optional-transformations
-.. _`document about the RPython toolchain`: translation.html
-.. _`garbage collector`: garbage_collection.html
-.. _`RPython toolchain`: translation.html
-.. _`standard interpreter`:
-.. _`python interpreter`:
+.. _python-interpreter:
PyPy Python Interpreter
--------------------------------------
+-----------------------
PyPy's *Python Interpreter* is written in RPython and implements the
full Python language. This interpreter very closely emulates the
behavior of CPython. It contains the following key components:
-- a bytecode compiler responsible for producing Python code objects
+- a bytecode compiler responsible for producing Python code objects
from the source code of a user application;
-- a `bytecode evaluator`_ responsible for interpreting
+- a :doc:`bytecode evaluator <interpreter>` responsible for interpreting
Python code objects;
-- a `standard object space`_, responsible for creating and manipulating
+- a :ref:`standard object space <standard-object-space>`, responsible for creating and manipulating
the Python objects seen by the application.
The *bytecode compiler* is the preprocessing phase that produces a
@@ -200,64 +70,6 @@
lists, as well as the operations between them, like addition or
truth-value-testing.
-This division between bytecode evaluator and object space is very
-important, as it gives a lot of flexibility. One can plug in
-different `object spaces`_ to get different or enriched behaviours
-of the Python objects. Additionally, a special more abstract object
-space, the `flow object space`_, allows us to reuse the bytecode
-evaluator for our translation framework.
-
-.. _`bytecode evaluator`: interpreter.html
-.. _`standard object space`: objspace.html#the-standard-object-space
-.. _`object spaces`: objspace.html
-.. _`flow object space`: objspace.html#the-flow-object-space
-
-.. _`the translation framework`:
-
-
-Further reading
-===============
-
-All of PyPy's documentation can be reached from the `documentation
-index`_. Of particular interest after reading this document might be:
-
- * `getting-started`_: a hands-on guide to getting involved with the
- PyPy source code.
-
- * `PyPy's approach to virtual machine construction`_: a paper
- presented to the Dynamic Languages Symposium attached to OOPSLA
- 2006.
-
- * `The translation document`_: a detailed description of our
- translation process.
-
- * `JIT Generation in PyPy`_, describing how we produce a Just-in-time
- Compiler from an interpreter.
-
- * A tutorial of how to use the `RPython toolchain`_ to `implement your own
- interpreter`_.
-
-.. _`documentation index`: index.html#project-documentation
-.. _`getting-started`: getting-started-dev.html
-.. _`PyPy's approach to virtual machine construction`: https://bitbucket.org/pypy/extradoc/raw/tip/talk/dls2006/pypy-vm-construction.pdf
-.. _`the translation document`: translation.html
-.. _`RPython toolchain`: translation.html
-.. _`Compiling dynamic language implementations`: https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D05.1_Publish_on_translating_a_very-high-level_description.pdf
-.. _`Technical reports`: index-report.html
-
-.. _`getting started`: getting-started.html
-.. _`Extreme Programming`: http://www.extremeprogramming.org/
-
-.. _fast: faq.html#how-fast-is-pypy
-.. _`very compliant`: cpython_differences.html
-
-.. _`RPython`: coding-guide.html#rpython
-
-.. _Python: http://docs.python.org/reference/
-.. _Psyco: http://psyco.sourceforge.net
-.. _`generate Just-In-Time Compilers`: jit/index.html
-.. _`JIT Generation in PyPy`: jit/index.html
-.. _`implement your own interpreter`: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html
-
-.. include:: _ref.txt
-
+This division between bytecode evaluator and object space gives a lot of
+flexibility. One can plug in different :doc:`object spaces <objspace>` to get
+different or enriched behaviours of the Python objects.
diff --git a/pypy/doc/arm.rst b/pypy/doc/arm.rst
deleted file mode 100644
--- a/pypy/doc/arm.rst
+++ /dev/null
@@ -1,166 +0,0 @@
-=========================
-Cross-translating for ARM
-=========================
-
-
-Here we describe the setup required and the steps needed to follow to translate
-an interpreter using the RPython translator to target ARM using a cross
-compilation toolchain.
-
-To translate an RPython program for ARM we can either
-translate directly on an ARM device following the normal translation steps. Unfortunately this is not really feasible on most ARM machines. The alternative is to cross-translate using a cross-compilation toolchain.
-
-To cross-translate we run the translation on a more powerful (usually
-x86) machine and generate a binary for ARM using a cross-compiler to compile
-the generated C code. There are several constraints when doing this. In
-particular we currently only support Linux as translation host and target
-platforms (tested on Ubuntu). Also we need a 32-bit environment to run the
-translation. This can be done either on a 32bit host or in 32bit chroot.
-
-
-Requirements
-------------
-
-The tools required to cross translate from a Linux based host to an ARM based Linux target are:
-
-- A checkout of PyPy (default branch).
-- The GCC ARM cross compiler (on Ubuntu it is the ``gcc-arm-linux-gnueabi package``) but other toolchains should also work.
-- Scratchbox 2, a cross-compilation engine (``scratchbox2`` Ubuntu package).
-- A 32-bit PyPy or Python.
-- And the following (or corresponding) packages need to be installed to create an ARM based chroot:
-
- * ``debootstrap``
- * ``schroot``
- * ``binfmt-support``
- * ``qemu-system``
- * ``qemu-user-static``
-
-- The dependencies above are in addition to the ones needed for a regular
- translation, `listed here`_.
-
-.. _`listed here`: getting-started-python.html#translating-the-pypy-python-interpreter
-
-
-Creating a Qemu based ARM chroot
---------------------------------
-
-First we will need to create a rootfs containing the packages and dependencies
-required in order to translate PyPy or other interpreters. We are going to
-assume, that the files will be placed in ``/srv/chroot/precise_arm``.
-
-Create the rootfs by calling:
-
-::
-
- mkdir -p /srv/chroot/precise_arm
- qemu-debootstrap --variant=buildd --arch=armel precise /srv/chroot/precise_arm/ http://ports.ubuntu.com/ubuntu-ports/
-
-Next, copy the qemu-arm-static binary to the rootfs.
-
-::
-
- cp /usr/bin/qemu-arm-static /srv/chroot/precise_arm/usr/bin/qemu-arm-static
-
-For easier configuration and management we will create a schroot pointing to
-the rootfs. We need to add a configuration block (like the one below) to the
-schroot configuration file in /etc/schroot/schroot.conf.
-
-
-::
-
- [precise_arm]
- directory=/srv/chroot/precise_arm
- users=USERNAME
- root-users=USERNAME
- groups=users
- aliases=default
- type=directory
-
-
-To verify that everything is working in the chroot, running ``schroot -c
-precise_arm`` should start a shell running in the schroot environment using
-qemu-arm to execute the ARM binaries. Running ``uname -m`` in the chroot should
-yeild a result like ``armv7l``. Showing that we are emulating an ARM system.
-
-Start the schroot as the user root in order to configure the apt sources and
-to install the following packages:
-
-
-::
-
- schroot -c precise_arm -u root
- echo "deb http://ports.ubuntu.com/ubuntu-ports/ precise main universe restricted" > /etc/apt/sources.list
- apt-get update
- apt-get install libffi-dev libgc-dev python-dev build-essential libncurses5-dev libbz2-dev
-
-
-Now all dependencies should be in place and we can exit the schroot environment.
-
-
-Configuring scratchbox2
------------------------
-
-To configure the scratchbox we need to cd into the root directory of the rootfs
-we created before. From there we can call the sb2 configuration tools which
-will take the current directory as the base directory for the scratchbox2
-environment.
-
-::
-
- cd /srv/chroot/precise_arm
- sb2-init -c `which qemu-arm` ARM `which arm-linux-gnueabi-gcc`
-
-This will create a scratchbox2 based environment called ARM that maps calls to
-gcc done within the scratchbox to the arm-linux-gnueabi-gcc outside the
-scratchbox. Now we should have a working cross compilation toolchain in place
-and can start cross-translating programs for ARM.
-
-Translation
------------
-
-Having performed all the preliminary steps we should now be able to cross
-translate a program for ARM. You can use this_ minimal
-target to test your setup before applying it to a larger project.
-
-Before starting the translator we need to set two environment variables, so the
-translator knows how to use the scratchbox environment. We need to set the
-**SB2** environment variable to point to the rootfs and the **SB2OPT** should
-contain the command line options for the sb2 command. If our rootfs is in the
-folder /srv/chroot/precise_arm and the scratchbox environment is called "ARM",
-the variables would be defined as follows.
-
-
-::
-
- export SB2=/srv/chroot/precise_arm
- export SB2OPT='-t ARM'
-
-Once this is set, you can call the translator. For example save this file
-
-::
-
- def main(args):
- print "Hello World"
- return 0
-
- def target(*args):
- return main, None
-
-and call the translator
-
-::
-
- pypy ~/path_to_pypy_checkout/rpython/bin/rpython -O1 --platform=arm target.py
-
-If everything worked correctly this should yield an ARM binary. Running this binary in the ARM chroot or on an ARM device should produce the output ``"Hello World"``.
-
-To translate the full python pypy interpreter with a jit, you can cd into pypy/goal and call
-
-::
-
- pypy <path to rpython>/rpython/bin/rpython -Ojit --platform=arm --gcrootfinder=shadowstack --jit-backend=arm targetpypystandalone.py
-
-The gcrootfinder option is needed to work around `issue 1377`_ and the jit-backend works around `issue 1376`_
-
-.. _`issue 1377`: https://bugs.pypy.org/issue1377
-.. _`issue 1376`: https://bugs.pypy.org/issue1376
diff --git a/pypy/doc/build.rst b/pypy/doc/build.rst
new file mode 100644
--- /dev/null
+++ b/pypy/doc/build.rst
@@ -0,0 +1,170 @@
+Building PyPy from Source
+=========================
+
+For building PyPy, we recommend installing a pre-built PyPy first (see
+:doc:`install`). It is possible to build PyPy with CPython, but it will take a
+lot longer to run -- depending on your architecture, between two and three
+times as long.
+
+Even when using PyPy to build PyPy, translation is time-consuming -- 30
+minutes on a fast machine -- and RAM-hungry. You will need **at least** 2 GB
+of memory on a 32-bit machine and 4GB on a 64-bit machine.
+
+
+Clone the repository
+--------------------
+
+If you prefer to compile your own PyPy, or if you want to modify it, you
+will need to obtain a copy of the sources. This can be done either by
+`downloading them from the download page`_ or by checking them out from the
+repository using mercurial. We suggest using mercurial if you want to access
+the current development.
+
+.. _downloading them from the download page: http://pypy.org/download.html
+
+You must issue the following command on your
+command line, DOS box, or terminal::
+
+ hg clone http://bitbucket.org/pypy/pypy pypy
+
+This will clone the repository and place it into a directory
+named ``pypy``, and will get you the PyPy source in ``pypy/pypy`` and
+documentation files in ``pypy/pypy/doc``.
+We try to ensure that the tip is always stable, but it might
+occasionally be broken. You may want to check out `our nightly tests`_:
+find a revision (12-chars alphanumeric string, e.g. "963e808156b3")
+that passed at least the
+``{linux32}`` tests (corresponding to a ``+`` sign on the
+line ``success``) and then, in your cloned repository, switch to this revision
+using::
+
+ hg up -r XXXXX
+
+where XXXXX is the revision id.
+
+.. _our nightly tests: http://buildbot.pypy.org/summary?branch=<trunk>
+
+
+Install build-time dependencies
+-------------------------------
+
+To build PyPy on Unix using the C translation backend, you need at least a C
+compiler and ``make`` installed. Further, some optional modules have additional
+dependencies:
+
+cffi, ctypes
+ libffi, pkg-config
+
+zlib
+ libz
+
+bz2
+ libbz2
+
+lzma (PyPy3 only)
+ liblzma
+
+sqlite3
+ libsqlite3
+
+curses
+ libncurses + cffi dependencies from above
+
+pyexpat
+ libexpat1
+
+_ssl
+ libssl
+
+Make sure to have these libraries (with development headers) installed before
+building PyPy, otherwise the resulting binary will not contain these modules.
+
+On Debian, this is the command to install all build-time dependencies::
+
+ apt-get install gcc make libffi-dev pkg-config libz-dev libbz2-dev \
+ libsqlite3-dev libncurses-dev libexpat1-dev libssl-dev
+
+For the optional lzma module on PyPy3 you will also need ``liblzma-dev``.
+
+On Fedora::
+
+ yum install gcc make libffi-devel pkgconfig zlib-devel bzip2-devel \
+ lib-sqlite3-devel ncurses-devel expat-devel openssl-devel
+
+For the optional lzma module on PyPy3 you will also need ``xz-devel``.
+
+On SLES11::
+
+ zypper install gcc make python-devel pkg-config \
+ zlib-devel libopenssl-devel libbz2-devel sqlite3-devel \
+ libexpat-devel libffi-devel python-curses
+
+For the optional lzma module on PyPy3 you will also need ``xz-devel``.
+
+On Mac OS X, most of these build-time dependencies are installed alongside
+the Developer Tools. However, note that in order for the installation to
+find them you may need to run::
+
+ xcode-select --install
+
+
+Run the translation
+-------------------
+
+Translate with JIT::
+
+ pypy rpython/bin/rpython --opt=jit pypy/goal/targetpypystandalone.py
+
+Translate without JIT::
+
+ pypy rpython/bin/rpython --opt=2 pypy/goal/targetpypystandalone.py
+
+(You can use ``python`` instead of ``pypy`` here, which will take longer
+but works too.)
+
+If everything works correctly this will create an executable ``pypy-c`` in the
+current directory. The executable behaves mostly like a normal Python
+interpreter (see :doc:`cpython_differences`).
+
+
+.. _translate-pypy:
+
+Translating with non-standard options
+-------------------------------------
+
+It is possible to have non-standard features enabled for translation,
+but they are not really tested any more. Look, for example, at the
+:doc:`objspace proxies <objspace-proxies>` document.
+
+
+
+Installation
+------------
+
+PyPy dynamically finds the location of its libraries depending on the location
+of the executable. The directory hierarchy of a typical PyPy installation
+looks like this::
+
+ ./bin/pypy
+ ./include/
+ ./lib_pypy/
+ ./lib-python/2.7
+ ./site-packages/
+
+The hierarchy shown above is relative to a PREFIX directory. PREFIX is
+computed by starting from the directory where the executable resides, and
+"walking up" the filesystem until we find a directory containing ``lib_pypy``
+and ``lib-python/2.7``.
+
+To install PyPy system wide on unix-like systems, it is recommended to put the
+whole hierarchy alone (e.g. in ``/opt/pypy``) and put a symlink to the
+``pypy`` executable into ``/usr/bin`` or ``/usr/local/bin``.
+
+If the executable fails to find suitable libraries, it will report ``debug:
+WARNING: library path not found, using compiled-in sys.path`` and then attempt
+to continue normally. If the default path is usable, most code will be fine.
+However, the ``sys.prefix`` will be unset and some existing libraries assume
+that this is never the case.
+
+
+.. TODO windows
diff --git a/pypy/doc/cleanup.rst b/pypy/doc/cleanup.rst
deleted file mode 100644
--- a/pypy/doc/cleanup.rst
+++ /dev/null
@@ -1,11 +0,0 @@
-Old documentation that needs review
------------------------------------
-
-.. The following stuff is old (and crufty?), and needs further investigation:
-
-.. doc-index: This needs merging somehow
-
-.. toctree::
-
- distribution.rst
-
diff --git a/pypy/doc/coding-guide.rst b/pypy/doc/coding-guide.rst
--- a/pypy/doc/coding-guide.rst
+++ b/pypy/doc/coding-guide.rst
@@ -1,21 +1,17 @@
-====================================
Coding Guide
-====================================
+============
.. contents::
This document describes coding requirements and conventions for
working with the PyPy code base. Please read it carefully and
ask back any questions you might have. The document does not talk
-very much about coding style issues. We mostly follow `PEP 8`_ though.
+very much about coding style issues. We mostly follow :pep:`8` though.
If in doubt, follow the style that is already present in the code base.
-.. _`PEP 8`: http://www.python.org/dev/peps/pep-0008/
-
-.. _`RPython`:
Overview and motivation
-========================
+------------------------
We are writing a Python interpreter in Python, using Python's well known
ability to step behind the algorithmic problems as a language. At first glance,
@@ -25,7 +21,7 @@
CPython vs. PyPy
--------------------
+~~~~~~~~~~~~~~~~
Compared to the CPython implementation, Python takes the role of the C
Code. We rewrite the CPython interpreter in Python itself. We could
@@ -44,20 +40,20 @@
but let's stick with this somewhat canonical approach.
-.. _`application-level`:
-.. _`interpreter-level`:
+.. _application-level:
+.. _interpreter-level:
Application-level and interpreter-level execution and objects
--------------------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Since Python is used for implementing all of our code base, there is a
-crucial distinction to be aware of: that between *interpreter-level* objects and
+crucial distinction to be aware of: that between *interpreter-level* objects and
*application-level* objects. The latter are the ones that you deal with
when you write normal python programs. Interpreter-level code, however,
cannot invoke operations nor access attributes from application-level
objects. You will immediately recognize any interpreter level code in
PyPy, because half the variable and object names start with a ``w_``, which
-indicates that they are `wrapped`_ application-level values.
+indicates that they are `wrapped`_ application-level values.
Let's show the difference with a simple example. To sum the contents of
two variables ``a`` and ``b``, one would write the simple application-level
@@ -80,10 +76,10 @@
interpreting.
-.. _`app-preferable`:
+.. _app-preferable:
-Application level is often preferable
--------------------------------------
+Application level is often preferable
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Application-level code is substantially higher-level, and therefore
correspondingly easier to write and debug. For example, suppose we want
@@ -113,11 +109,11 @@
space.setitem(w_self, w_key, w_value)
This interpreter-level implementation looks much more similar to the C
-source code. It is still more readable than its C counterpart because
-it doesn't contain memory management details and can use Python's native
-exception mechanism.
+source code. It is still more readable than its C counterpart because
+it doesn't contain memory management details and can use Python's native
+exception mechanism.
-In any case, it should be obvious that the application-level implementation
+In any case, it should be obvious that the application-level implementation
is definitely more readable, more elegant and more maintainable than the
interpreter-level one (and indeed, dict.update is really implemented at
applevel in PyPy).
@@ -129,10 +125,11 @@
level code is usually preferable. We have an abstraction (called the
'Gateway') which allows the caller of a function to remain ignorant of
whether a particular function is implemented at application or
-interpreter level.
+interpreter level.
+
Our runtime interpreter is "RPython"
-----------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to make a C code generator feasible all code on interpreter level has
to restrict itself to a subset of the Python language, and we adhere to some
@@ -159,7 +156,7 @@
elegant: For the definition of all the opcodes of the Python
interpreter, the module ``dis`` is imported and used to initialize our
bytecode interpreter. (See ``__initclass__`` in
-`pypy/interpreter/pyopcode.py`_). This
+:source:`pypy/interpreter/pyopcode.py`). This
saves us from adding extra modules to PyPy. The import code is run at
startup time, and we are allowed to use the CPython builtin import
function.
@@ -172,308 +169,13 @@
enables the code generator to emit efficient machine level replacements
for pure integer objects, for instance.
-RPython
-=================
-
-RPython Definition
-------------------
-
-RPython is a restricted subset of Python that is amenable to static analysis.
-Although there are additions to the language and some things might surprisingly
-work, this is a rough list of restrictions that should be considered. Note
-that there are tons of special cased restrictions that you'll encounter
-as you go. The exact definition is "RPython is everything that our translation
-toolchain can accept" :)
-
-.. _`wrapped object`: coding-guide.html#wrapping-rules
-
-Flow restrictions
--------------------------
-
-**variables**
-
- variables should contain values of at most one type as described in
- `Object restrictions`_ at each control flow point, that means for
- example that joining control paths using the same variable to
- contain both a string and a int must be avoided. It is allowed to
- mix None (basically with the role of a null pointer) with many other
- types: `wrapped objects`, class instances, lists, dicts, strings, etc.
- but *not* with int, floats or tuples.
-
-**constants**
-
- all module globals are considered constants. Their binding must not
- be changed at run-time. Moreover, global (i.e. prebuilt) lists and
- dictionaries are supposed to be immutable: modifying e.g. a global
- list will give inconsistent results. However, global instances don't
- have this restriction, so if you need mutable global state, store it
- in the attributes of some prebuilt singleton instance.
-
-
-
-**control structures**
-
- all allowed, ``for`` loops restricted to builtin types, generators
- very restricted.
-
-**range**
-
- ``range`` and ``xrange`` are identical. ``range`` does not necessarily create an array,
- only if the result is modified. It is allowed everywhere and completely
- implemented. The only visible difference to CPython is the inaccessibility
- of the ``xrange`` fields start, stop and step.
-
-**definitions**
-
- run-time definition of classes or functions is not allowed.
-
-**generators**
-
- generators are supported, but their exact scope is very limited. you can't
- merge two different generator in one control point.
-
-**exceptions**
-
-+ fully supported
-+ see below `Exception rules`_ for restrictions on exceptions raised by built-in operations
-
-
-Object restrictions
--------------------------
-
-We are using
-
-**integer, float, boolean**
-
- works.
-
-**strings**
-
- a lot of, but not all string methods are supported and those that are
- supported, not necesarilly accept all arguments. Indexes can be
- negative. In case they are not, then you get slightly more efficient
- code if the translator can prove that they are non-negative. When
- slicing a string it is necessary to prove that the slice start and
- stop indexes are non-negative. There is no implicit str-to-unicode cast
- anywhere. Simple string formatting using the ``%`` operator works, as long
- as the format string is known at translation time; the only supported
- formatting specifiers are ``%s``, ``%d``, ``%x``, ``%o``, ``%f``, plus
- ``%r`` but only for user-defined instances. Modifiers such as conversion
- flags, precision, length etc. are not supported. Moreover, it is forbidden
- to mix unicode and strings when formatting.
-
-**tuples**
-
- no variable-length tuples; use them to store or return pairs or n-tuples of
- values. Each combination of types for elements and length constitute
- a separate and not mixable type.
-
-**lists**
-
- lists are used as an allocated array. Lists are over-allocated, so list.append()
- is reasonably fast. However, if you use a fixed-size list, the code
- is more efficient. Annotator can figure out most of the time that your
- list is fixed-size, even when you use list comprehension.
- Negative or out-of-bound indexes are only allowed for the
- most common operations, as follows:
-
- - *indexing*:
- positive and negative indexes are allowed. Indexes are checked when requested
- by an IndexError exception clause.
-
- - *slicing*:
- the slice start must be within bounds. The stop doesn't need to, but it must
- not be smaller than the start. All negative indexes are disallowed, except for
- the [:-1] special case. No step. Slice deletion follows the same rules.
-
- - *slice assignment*:
- only supports ``lst[x:y] = sublist``, if ``len(sublist) == y - x``.
- In other words, slice assignment cannot change the total length of the list,
- but just replace items.
-
- - *other operators*:
- ``+``, ``+=``, ``in``, ``*``, ``*=``, ``==``, ``!=`` work as expected.
-
- - *methods*:
- append, index, insert, extend, reverse, pop. The index used in pop() follows
- the same rules as for *indexing* above. The index used in insert() must be within
- bounds and not negative.
-
-**dicts**
-
- dicts with a unique key type only, provided it is hashable. Custom
- hash functions and custom equality will not be honored.
- Use ``rpython.rlib.objectmodel.r_dict`` for custom hash functions.
-
-
-**list comprehensions**
-
- May be used to create allocated, initialized arrays.
-
-**functions**
-
-+ statically called functions may use defaults and a variable number of
- arguments (which may be passed as a list instead of a tuple, so write code
- that does not depend on it being a tuple).
-
-+ dynamic dispatch enforces the use of signatures that are equal for all
- possible called function, or at least "compatible enough". This
- concerns mainly method calls, when the method is overridden or in any
- way given different definitions in different classes. It also concerns
- the less common case of explicitly manipulated function objects.
- Describing the exact compatibility rules is rather involved (but if you
- break them, you should get explicit errors from the rtyper and not
- obscure crashes.)
-
-**builtin functions**
-
- A number of builtin functions can be used. The precise set can be
- found in `rpython/annotator/builtin.py`_ (see ``def builtin_xxx()``).
- Some builtin functions may be limited in what they support, though.
-
- ``int, float, str, ord, chr``... are available as simple conversion
- functions. Note that ``int, float, str``... have a special meaning as
- a type inside of isinstance only.
-
-**classes**
-
-+ methods and other class attributes do not change after startup
-+ single inheritance is fully supported
-+ use `rpython.rlib.objectmodel.import_from_mixin(M)` in a class
- body to copy the whole content of a class `M`. This can be used
- to implement mixins: functions and staticmethods are duplicated
- (the other class attributes are just copied unmodified).
-
-+ classes are first-class objects too
-
-**objects**
-
- Normal rules apply. The only special methods that are honoured are
- ``__init__``, ``__del__``, ``__len__``, ``__getitem__``, ``__setitem__``,
- ``__getslice__``, ``__setslice__``, and ``__iter__``. To handle slicing,
- ``__getslice__`` and ``__setslice__`` must be used; using ``__getitem__`` and
- ``__setitem__`` for slicing isn't supported. Additionally, using negative
- indices for slicing is still not support, even when using ``__getslice__``.
-
-This layout makes the number of types to take care about quite limited.
-
-
-Integer Types
--------------------------
-
-While implementing the integer type, we stumbled over the problem that
-integers are quite in flux in CPython right now. Starting with Python 2.4,
-integers mutate into longs on overflow. In contrast, we need
-a way to perform wrap-around machine-sized arithmetic by default, while still
-being able to check for overflow when we need it explicitly. Moreover, we need
-a consistent behavior before and after translation.
-
-We use normal integers for signed arithmetic. It means that before
-translation we get longs in case of overflow, and after translation we get a
-silent wrap-around. Whenever we need more control, we use the following
-helpers (which live in `rpython/rlib/rarithmetic.py`_):
-
-**ovfcheck()**
-
- This special function should only be used with a single arithmetic operation
- as its argument, e.g. ``z = ovfcheck(x+y)``. Its intended meaning is to
- perform the given operation in overflow-checking mode.
-
- At run-time, in Python, the ovfcheck() function itself checks the result
- and raises OverflowError if it is a ``long``. But the code generators use
- ovfcheck() as a hint: they replace the whole ``ovfcheck(x+y)`` expression
- with a single overflow-checking addition in C.
-
-**intmask()**
-
- This function is used for wrap-around arithmetic. It returns the lower bits
- of its argument, masking away anything that doesn't fit in a C "signed long int".
- Its purpose is, in Python, to convert from a Python ``long`` that resulted from a
- previous operation back to a Python ``int``. The code generators ignore
- intmask() entirely, as they are doing wrap-around signed arithmetic all the time
- by default anyway. (We have no equivalent of the "int" versus "long int"
- distinction of C at the moment and assume "long ints" everywhere.)
-
-**r_uint**
-
- In a few cases (e.g. hash table manipulation), we need machine-sized unsigned
- arithmetic. For these cases there is the r_uint class, which is a pure
- Python implementation of word-sized unsigned integers that silently wrap
- around. ("word-sized" and "machine-sized" are used equivalently and mean
- the native size, which you get using "unsigned long" in C.)
- The purpose of this class (as opposed to helper functions as above)
- is consistent typing: both Python and the annotator will propagate r_uint
- instances in the program and interpret all the operations between them as
- unsigned. Instances of r_uint are special-cased by the code generators to
- use the appropriate low-level type and operations.
- Mixing of (signed) integers and r_uint in operations produces r_uint that
- means unsigned results. To convert back from r_uint to signed integers, use
- intmask().
-
-
-Exception rules
----------------------
-
-Exceptions are by default not generated for simple cases.::
-
- #!/usr/bin/python
-
- lst = [1,2,3,4,5]
- item = lst[i] # this code is not checked for out-of-bound access
-
- try:
- item = lst[i]
- except IndexError:
- # complain
-
-Code with no exception handlers does not raise exceptions (after it has been
-translated, that is. When you run it on top of CPython, it may raise
-exceptions, of course). By supplying an exception handler, you ask for error
-checking. Without, you assure the system that the operation cannot fail.
-This rule does not apply to *function calls*: any called function is
-assumed to be allowed to raise any exception.
-
-For example::
-
- x = 5.1
- x = x + 1.2 # not checked for float overflow
- try:
- x = x + 1.2
- except OverflowError:
- # float result too big
-
-But::
-
- z = some_function(x, y) # can raise any exception
- try:
- z = some_other_function(x, y)
- except IndexError:
- # only catches explicitly-raised IndexErrors in some_other_function()
- # other exceptions can be raised, too, and will not be caught here.
-
-The ovfcheck() function described above follows the same rule: in case of
-overflow, it explicitly raise OverflowError, which can be caught anywhere.
-
-Exceptions explicitly raised or re-raised will always be generated.
-
-PyPy is debuggable on top of CPython
-------------------------------------
-
-PyPy has the advantage that it is runnable on standard
-CPython. That means, we can run all of PyPy with all exception
-handling enabled, so we might catch cases where we failed to
-adhere to our implicit assertions.
-
-.. _`wrapping rules`:
-.. _`wrapped`:
-
-
+.. _wrapped:
Wrapping rules
-==============
+--------------
Wrapping
----------
+~~~~~~~~
PyPy is made of Python source code at two levels: there is on the one hand
*application-level code* that looks like normal Python code, and that
@@ -500,7 +202,7 @@
structure.
For example, an application-level Python ``list``
-is implemented by the `standard object space`_ as an
+is implemented by the :ref:`standard object space <standard-object-space>` as an
instance of ``W_ListObject``, which has an instance attribute
``wrappeditems`` (an interpreter-level list which contains the
application-level list's items as wrapped objects).
@@ -509,7 +211,7 @@
Naming conventions
-------------------
+~~~~~~~~~~~~~~~~~~
* ``space``: the object space is only visible at
interpreter-level code, where it is by convention passed around by the name
@@ -529,14 +231,14 @@
Operations on ``w_xxx``
------------------------
+~~~~~~~~~~~~~~~~~~~~~~~
The core bytecode interpreter considers wrapped objects as black boxes.
It is not allowed to inspect them directly. The allowed
operations are all implemented on the object space: they are
called ``space.xxx()``, where ``xxx`` is a standard operation
name (``add``, ``getattr``, ``call``, ``eq``...). They are documented in the
-`object space document`_.
+:ref:`object space document <objspace-interface>`.
A short warning: **don't do** ``w_x == w_y`` or ``w_x is w_y``!
rationale for this rule is that there is no reason that two
@@ -548,12 +250,11 @@
use ``space.is_true(space.is_(w_x, w_y))`` or better
``space.is_w(w_x, w_y)``.
-.. _`object space document`: objspace.html#interface
-.. _`applevel-exceptions`:
+.. _applevel-exceptions:
Application-level exceptions
-----------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Interpreter-level code can use exceptions freely. However,
all application-level exceptions are represented as an
@@ -585,10 +286,10 @@
instances of subclasses.
-.. _`modules`:
+.. _modules:
Modules in PyPy
-===============
+---------------
Modules visible from application programs are imported from
interpreter or application level files. PyPy reuses almost all python
@@ -597,18 +298,19 @@
because they rely on implementation details of CPython.
If we don't just modify an original CPython module but need to rewrite
-it from scratch we put it into `lib_pypy/`_ as a pure application level
+it from scratch we put it into :source:`lib_pypy/` as a pure application level
module.
When we need access to interpreter-level objects we put the module into
-`pypy/module`_. Such modules use a `mixed module mechanism`_
+:source:`pypy/module`. Such modules use a `mixed module mechanism`_
which makes it convenient to use both interpreter- and application-level parts
for the implementation. Note that there is no extra facility for
pure-interpreter level modules, you just write a mixed module and leave the
application-level part empty.
+
Determining the location of a module implementation
----------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can interactively find out where a module comes from, when running py.py.
here are examples for the possible locations::
@@ -626,8 +328,9 @@
'/home/hpk/pypy-dist/lib-python/2.7/os.py'
>>>>
+
Module directories / Import order
----------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here is the order in which PyPy looks up Python modules:
@@ -650,10 +353,11 @@
The modified CPython library.
-.. _`modify modules`:
+
+.. _modify modules:
Modifying a CPython library module or regression test
--------------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Although PyPy is very compatible with CPython we sometimes need
to change modules contained in our copy of the standard library,
@@ -665,34 +369,31 @@
to see what is changed we have a branch called `vendor/stdlib`
wich contains the unmodified cpython stdlib
-.. _`mixed module mechanism`:
-.. _`mixed modules`:
+
+.. _mixed module mechanism:
+.. _mixed-modules:
Implementing a mixed interpreter/application level Module
----------------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If a module needs to access PyPy's interpreter level
then it is implemented as a mixed module.
-Mixed modules are directories in `pypy/module`_ with an `__init__.py`
+Mixed modules are directories in :source:`pypy/module` with an `__init__.py`
file containing specifications where each name in a module comes from.
Only specified names will be exported to a Mixed Module's applevel
namespace.
-Sometimes it is necessary to really write some functions in C (or
-whatever target language). See `rffi`_ and `external functions
-documentation`_ for details. The latter approach is cumbersome and
-being phased out and former has currently quite a few rough edges.
+Sometimes it is necessary to really write some functions in C (or whatever
+target language). See :ref:`rffi <rpython:rffi>` details.
-.. _`rffi`: rffi.html
-.. _`external functions documentation`: translation.html#extfunccalls
application level definitions
-.............................
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Application level specifications are found in the `appleveldefs`
dictionary found in ``__init__.py`` files of directories in ``pypy/module``.
-For example, in `pypy/module/__builtin__/__init__.py`_ you find the following
+For example, in :source:`pypy/module/__builtin__/__init__.py` you find the following
entry specifying where ``__builtin__.locals`` comes from::
...
@@ -703,12 +404,13 @@
interpreted at application level and the wrapped function value for ``locals``
will be extracted accordingly.
+
interpreter level definitions
-.............................
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Interpreter level specifications are found in the ``interpleveldefs``
dictionary found in ``__init__.py`` files of directories in ``pypy/module``.
-For example, in `pypy/module/__builtin__/__init__.py`_ the following
+For example, in :source:`pypy/module/__builtin__/__init__.py` the following
entry specifies where ``__builtin__.len`` comes from::
...
@@ -724,7 +426,7 @@
return space.len(w_obj)
Exposed interpreter level functions usually take a ``space`` argument
-and some wrapped values (see `wrapping rules`_) .
+and some wrapped values (see `Wrapping rules`_) .
You can also use a convenient shortcut in ``interpleveldefs`` dictionaries:
namely an expression in parentheses to specify an interpreter level
@@ -743,37 +445,40 @@
--withoutmod-mymodule (the latter being the default)) for py.py and
translate.py.
+
Testing modules in ``lib_pypy/``
---------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You can go to the `pypy/module/test_lib_pypy/`_ directory and invoke the testing tool
+You can go to the :source:`pypy/module/test_lib_pypy/` directory and invoke the testing tool
("py.test" or "python ../../pypy/test_all.py") to run tests against the
-lib_pypy hierarchy. Note, that tests in `pypy/module/test_lib_pypy/`_ are allowed
+lib_pypy hierarchy. Note, that tests in :source:`pypy/module/test_lib_pypy/` are allowed
and encouraged to let their tests run at interpreter level although
-`lib_pypy/`_ modules eventually live at PyPy's application level.
+:source:`lib_pypy/` modules eventually live at PyPy's application level.
This allows us to quickly test our python-coded reimplementations
against CPython.
+
Testing modules in ``pypy/module``
-----------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Simply change to ``pypy/module`` or to a subdirectory and `run the
tests as usual`_.
Testing modules in ``lib-python``
------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to let CPython's regression tests run against PyPy
-you can switch to the `lib-python/`_ directory and run
+you can switch to the :source:`lib-python/` directory and run
the testing tool in order to start compliance tests.
(XXX check windows compatibility for producing test reports).
+
Naming conventions and directory layout
-===========================================
+---------------------------------------
Directory and File Naming
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~
- directories/modules/namespaces are always **lowercase**
@@ -786,8 +491,9 @@
- keep filenames concise and completion-friendly.
+
Naming of python objects
-------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~
- class names are **CamelCase**
@@ -803,8 +509,9 @@
includes w_self. Don't use ``w_`` in application level
python only code.
+
Committing & Branching to the repository
------------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- write good log messages because several people
are reading the diffs.
@@ -815,22 +522,23 @@
``try1`` doesn't already exists) you should do::
hg branch try1
-
+
The branch will be recorded in the repository only after a commit. To switch
back to the default branch::
-
+
hg update default
-
+
For further details use the help or refer to the `official wiki`_::
-
+
hg help branch
-.. _`official wiki`: http://mercurial.selenic.com/wiki/Branch
+.. _official wiki: http://mercurial.selenic.com/wiki/Branch
-.. _`using development tracker`:
+
+.. _using-development-tracker:
Using the development bug/feature tracker
-=========================================
+-----------------------------------------
We have a `development tracker`_, based on Richard Jones'
`roundup`_ application. You can file bugs,
@@ -838,15 +546,14 @@
for the next milestone, both from an E-Mail and from a
web interface.
-.. _`development tracker`: https://bugs.pypy.org/
-.. _`roundup`: http://roundup.sourceforge.net/
+.. _development tracker: https://bugs.pypy.org/
+.. _roundup: http://roundup.sourceforge.net/
-.. _`testing in PyPy`:
-.. _`test-design`:
+.. _testing:
Testing in PyPy
-===============
+---------------
Our tests are based on the `py.test`_ tool which lets you write
unittests without boilerplate. All tests of modules
@@ -859,12 +566,11 @@
- **Application Level tests**. They run at application level which means
that they look like straight python code but they are interpreted by PyPy.
-.. _`standard object space`: objspace.html#standard-object-space
-.. _`objectspace`: objspace.html
-.. _`py.test`: http://pytest.org/
+.. _py.test: http://pytest.org/
+
Interpreter level tests
------------------------
+~~~~~~~~~~~~~~~~~~~~~~~
You can write test functions and methods like this::
@@ -880,8 +586,9 @@
module global level and use plain 'assert' statements thanks to the
usage of the `py.test`_ tool.
+
Application Level tests
------------------------
+~~~~~~~~~~~~~~~~~~~~~~~
For testing the conformance and well-behavedness of PyPy it
is often sufficient to write "normal" application-level
@@ -916,7 +623,8 @@
assert self.d["a"] == 1
assert self.d["b"] == 2
-.. _`run the tests as usual`:
+
+.. _run the tests as usual:
Another possibility is to use cls.space.appexec, for example::
@@ -959,7 +667,7 @@
Command line tool test_all
---------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~
You can run almost all of PyPy's tests by invoking::
@@ -969,8 +677,9 @@
located in the ``py/bin/`` directory. For switches to
modify test execution pass the ``-h`` option.
+
Coverage reports
-----------------
+~~~~~~~~~~~~~~~~
In order to get coverage reports the `pytest-cov`_ plugin is included.
it adds some extra requirements ( coverage_ and `cov-core`_ )
@@ -978,12 +687,13 @@
python test_all.py --cov file_or_direcory_to_cover file_or_directory
-.. _`pytest-cov`: http://pypi.python.org/pypi/pytest-cov
-.. _`coverage`: http://pypi.python.org/pypi/coverage
-.. _`cov-core`: http://pypi.python.org/pypi/cov-core
+.. _pytest-cov: http://pypi.python.org/pypi/pytest-cov
+.. _coverage: http://pypi.python.org/pypi/coverage
+.. _cov-core: http://pypi.python.org/pypi/cov-core
+
Test conventions
-----------------
+~~~~~~~~~~~~~~~~
- adding features requires adding appropriate tests. (It often even
makes sense to first write the tests so that you are sure that they
@@ -993,29 +703,28 @@
which contain unit tests. Such scripts can usually be executed
directly or are collectively run by pypy/test_all.py
-.. _`change documentation and website`:
+
+.. _change documentation and website:
Changing documentation and website
-==================================
+----------------------------------
documentation/website files in your local checkout
----------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Most of the PyPy's documentation is kept in `pypy/doc`.
You can simply edit or add '.rst' files which contain ReST-markuped
files. Here is a `ReST quickstart`_ but you can also just look
at the existing documentation and see how things work.
-.. _`ReST quickstart`: http://docutils.sourceforge.net/docs/user/rst/quickref.html
-
Note that the web site of http://pypy.org/ is maintained separately.
For now it is in the repository https://bitbucket.org/pypy/pypy.org
+.. _ReST quickstart: http://docutils.sourceforge.net/docs/user/rst/quickref.html
+
+
Automatically test documentation/website changes
-------------------------------------------------
-
-.. _`sphinx home page`:
-.. _`sphinx`: http://sphinx.pocoo.org/
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We automatically check referential integrity and ReST-conformance. In order to
run the tests you need sphinx_ installed. Then go to the local checkout
@@ -1035,5 +744,4 @@
which will check that remote URLs are reachable.
-
-.. include:: _ref.txt
+.. _sphinx: http://sphinx.pocoo.org/
diff --git a/pypy/doc/conf.py b/pypy/doc/conf.py
--- a/pypy/doc/conf.py
+++ b/pypy/doc/conf.py
@@ -40,9 +40,9 @@
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.intersphinx',
- 'sphinx.ext.todo', 'sphinx.ext.ifconfig', 'sphinx.ext.graphviz',
- 'pypyconfig']
+extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.extlinks',
+ 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.ifconfig',
+ 'sphinx.ext.graphviz', 'pypyconfig']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -215,7 +215,8 @@
# Example configuration for intersphinx: refer to the Python standard library.
-intersphinx_mapping = {'http://docs.python.org/': None}
+intersphinx_mapping = {'python': ('http://docs.python.org/2.7', None),
+ 'rpython': ('http://rpython.readthedocs.org/en/latest/', None)}
# -- Options for manpage output-------------------------------------------------
@@ -225,3 +226,4 @@
u'The PyPy Project', 1)
]
+extlinks = {'source': ('https://bitbucket.org/pypy/pypy/src/default/%s', '')}
diff --git a/pypy/doc/config/confrest.py b/pypy/doc/config/confrest.py
deleted file mode 100644
--- a/pypy/doc/config/confrest.py
+++ /dev/null
@@ -1,62 +0,0 @@
-from pypy.doc.confrest import *
-from pypy.config.makerestdoc import make_cmdline_overview
-from pypy.config import pypyoption
-from rpython.config.config import Config
-from rpython.config import translationoption
-
-
-all_optiondescrs = [pypyoption.pypy_optiondescription,
- translationoption.translation_optiondescription,
- ]
-start_to_descr = dict([(descr._name, descr) for descr in all_optiondescrs])
-
-class PyPyPage(PyPyPage):
- def fill(self):
- super(PyPyPage, self).fill()
- self.menubar[:] = html.div(
- html.a("general documentation", href="../index.html",
- class_="menu"), " ",
- html.a("config index", href="index.html",
- class_="menu"), " ",
- html.a("command-line overview", href="commandline.html",
- class_="menu"), " ",
- " ", id="menubar")
-
-class Project(Project):
- stylesheet = "../style.css"
- title = "PyPy Configuration"
- prefix_title = "PyPy Configuration"
- Page = PyPyPage
-
- def get_content(self, txtpath, encoding):
- if txtpath.basename == "commandline.rst":
- result = []
- for line in txtpath.read().splitlines():
- if line.startswith('.. GENERATE:'):
- start = line[len('.. GENERATE:'):].strip()
- descr = start_to_descr[start]
- line = make_cmdline_overview(descr, title=False).text()
- result.append(line)
- return "\n".join(result)
- fullpath = txtpath.purebasename
- start = fullpath.split(".")[0]
- path = fullpath.rsplit(".", 1)[0]
- basedescr = start_to_descr.get(start)
- if basedescr is None:
- return txtpath.read()
- if fullpath.count(".") == 0:
- descr = basedescr
- path = ""
- else:
- conf = Config(basedescr)
- subconf, step = conf._cfgimpl_get_home_by_path(
- fullpath.split(".", 1)[1])
- descr = getattr(subconf._cfgimpl_descr, step)
- text = unicode(descr.make_rest_doc(path).text())
- if txtpath.check(file=True):
- content = txtpath.read()
- if content:
- text += "\nDescription\n==========="
- return u"%s\n\n%s" % (text, unicode(txtpath.read(), encoding))
- return text
-
diff --git a/pypy/doc/config/generate.py b/pypy/doc/config/generate.py
--- a/pypy/doc/config/generate.py
+++ b/pypy/doc/config/generate.py
@@ -1,6 +1,5 @@
import py
from pypy.config import pypyoption, makerestdoc
-from pypy.doc.config.confrest import all_optiondescrs
from rpython.config import translationoption, config
all_optiondescrs = [pypyoption.pypy_optiondescription,
diff --git a/pypy/doc/configuration.rst b/pypy/doc/configuration.rst
--- a/pypy/doc/configuration.rst
+++ b/pypy/doc/configuration.rst
@@ -1,4 +1,3 @@
-=============================
PyPy's Configuration Handling
=============================
@@ -6,16 +5,17 @@
hand the necessary options to where they are actually used and even more
annoying to add new options. To circumvent these problems configuration
management was introduced. There all the necessary options are stored in a
-configuration object, which is available nearly everywhere in the `RPython
+configuration object, which is available nearly everywhere in the `RPython
toolchain`_ and in the standard interpreter so that adding new options becomes
trivial. Options are organized into a tree. Configuration objects can be
created in different ways, there is support for creating an optparse command
line parser automatically.
-_`RPython toolchain`: translation.html
+.. _RPython toolchain: https://rpython.readthedocs.org/
+
Main Assumption
-===============
+---------------
Configuration objects are produced at the entry points and handed down to
where they are actually used. This keeps configuration local but available
@@ -24,7 +24,7 @@
API Details
-===========
+-----------
The handling of options is split into two parts: the description of which
options are available, what their possible values and defaults are and how they
@@ -40,8 +40,9 @@
option group. The parts of the full name of the option are separated by dots:
e.g. ``config.translation.thread``.
+
Description of Options
-----------------------
+~~~~~~~~~~~~~~~~~~~~~~
All the constructors take a ``name`` and a ``doc`` argument as first arguments
to give the option or option group a name and to document it. Most constructors
@@ -70,6 +71,7 @@
``children`` is a list of option descriptions (including
``OptionDescription`` instances for nested namespaces).
+
``ChoiceOption``
++++++++++++++++
@@ -81,10 +83,11 @@
``requires`` is a dictionary mapping values to lists of of two-element
tuples.
+
``BoolOption``
++++++++++++++
-Represents a choice between ``True`` and ``False``.
+Represents a choice between ``True`` and ``False``.
``__init__(self, name, doc, default=None, requires=None, suggests=None, cmdline=DEFAULT, negation=True)``
``default`` specifies the default value of the option. ``requires`` is
@@ -103,7 +106,6 @@
Represents a choice of an integer.
``__init__(self, name, doc, default=None, cmdline=DEFAULT)``
-
``FloatOption``
@@ -112,7 +114,6 @@
Represents a choice of a floating point number.
``__init__(self, name, doc, default=None, cmdline=DEFAULT)``
-
``StrOption``
@@ -121,12 +122,10 @@
Represents the choice of a string.
``__init__(self, name, doc, default=None, cmdline=DEFAULT)``
-
-
Configuration Objects
----------------------
+~~~~~~~~~~~~~~~~~~~~~
``Config`` objects hold the chosen values for the options (of the default,
if no choice was made). A ``Config`` object is described by an
@@ -164,7 +163,7 @@
Production of optparse Parsers
-------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To produce an optparse parser use the function ``to_optparse``. It will create
an option parser using callbacks in such a way that the config object used for
@@ -182,14 +181,11 @@
The usage of config objects in PyPy
-===================================
+-----------------------------------
-The two large parts of PyPy, the Python interpreter and the `RPython
-toolchain`_
+The two large parts of PyPy, the Python interpreter and the RPython
toolchain, have two separate sets of options. The translation toolchain options
can be found on the ``config`` attribute of all ``TranslationContext``
-instances and are described in `rpython/config/translationoption.py`_. The interpreter options
+instances and are described in :source:`rpython/config/translationoption.py`. The interpreter options
are attached to the object space, also under the name ``config`` and are
-described in `pypy/config/pypyoption.py`_.
-
-.. include:: _ref.txt
+described in :source:`pypy/config/pypyoption.py`.
diff --git a/pypy/doc/confrest.py b/pypy/doc/confrest.py
deleted file mode 100644
--- a/pypy/doc/confrest.py
+++ /dev/null
@@ -1,69 +0,0 @@
-import py
-
More information about the pypy-commit
mailing list