[pypy-commit] pypy default: update build instructions for pyexpat

[mattip]

Author: Matti Picus <matti.picus at gmail.com>
Branch: 
Changeset: r92377:8f23ddc1eec0
Date: 2017-09-13 09:36 +0300
http://bitbucket.org/pypy/pypy/changeset/8f23ddc1eec0/

Log:	update build instructions for pyexpat

diff --git a/pypy/doc/windows.rst b/pypy/doc/windows.rst
--- a/pypy/doc/windows.rst
+++ b/pypy/doc/windows.rst
@@ -209,17 +209,85 @@
 The expat XML parser
 ~~~~~~~~~~~~~~~~~~~~
 
-Download the source code of expat on sourceforge:
-https://github.com/libexpat/libexpat/releases and extract it in the base directory.
-Version 2.1.1 is known to pass tests. Then open the project file ``expat.dsw``
-with Visual Studio; follow the instruction for converting the project files,
-switch to the "Release" configuration, use the ``expat_static`` project,
-reconfigure the runtime for Multi-threaded DLL (/MD) and build. Do the same for
-the ``expat`` project to build the ``expat.dll`` (for tests via ll2ctypes)
+CPython compiles expat from source as part of the build. PyPy uses the same
+code base, but expects to link to a static lib of expat. Here are instructions
+to reproduce the static lib in version 2.2.4.
 
-Then, copy the file ``win32\bin\release\libexpat.lib`` into
-LIB, and both ``lib\expat.h`` and ``lib\expat_external.h`` in
-INCLUDE, and ``win32\bin\release\libexpat.dll`` into PATH.
+Download the source code of expat: https://github.com/libexpat/libexpat. 
+``git checkout`` the proper tag, in this case ``R_2_2_4``. Run
+``vcvars.bat`` to set up the visual compiler tools, and CD into the source
+directory. Create a file ``stdbool.h`` with the content
+
+.. code-block:: c
+
+    #pragma once
+
+    #define false   0
+    #define true    1
+
+    #define bool int
+
+and put it in a place on the ``INCLUDE`` path, or create it in the local
+directory and add ``.`` to the ``INCLUDE`` path::
+
+    SET INCLUDE=%INCLUDE%;.
+
+Then compile all the ``*.c`` file into ``*.obj``::
+
+    cl.exe /nologo /MD  /O2 *c /c
+    rem for debug
+    cl.exe /nologo /MD  /O0 /Ob0 /Zi *c /c
+
+You may need to move some variable declarations to the beginning of the
+function, to be compliant with C89 standard. Here is the diff for version 2.2.4
+
+.. code-block:: diff
+
+    diff --git a/expat/lib/xmltok.c b/expat/lib/xmltok.c
+    index 007aed0..a2dcaad 100644
+    --- a/expat/lib/xmltok.c
+    +++ b/expat/lib/xmltok.c
+    @@ -399,19 +399,21 @@ utf8_toUtf8(const ENCODING *UNUSED_P(enc),
+       /* Avoid copying partial characters (due to limited space). */
+       const ptrdiff_t bytesAvailable = fromLim - *fromP;
+       const ptrdiff_t bytesStorable = toLim - *toP;
+    +  const char * fromLimBefore;
+    +  ptrdiff_t bytesToCopy;
+       if (bytesAvailable > bytesStorable) {
+         fromLim = *fromP + bytesStorable;
+         output_exhausted = true;
+       }
+
+       /* Avoid copying partial characters (from incomplete input). */
+    -  const char * const fromLimBefore = fromLim;
+    +  fromLimBefore = fromLim;
+       align_limit_to_full_utf8_characters(*fromP, &fromLim);
+       if (fromLim < fromLimBefore) {
+         input_incomplete = true;
+       }
+
+    -  const ptrdiff_t bytesToCopy = fromLim - *fromP;
+    +  bytesToCopy = fromLim - *fromP;
+       memcpy((void *)*toP, (const void *)*fromP, (size_t)bytesToCopy);
+       *fromP += bytesToCopy;
+       *toP += bytesToCopy;
+
+
+Create ``libexpat.lib`` (for translation) and ``libexpat.dll`` (for tests)::
+
+    cl /LD *.obj libexpat.def /Felibexpat.dll 
+    rem for debug
+    rem cl /LDd /Zi *.obj libexpat.def /Felibexpat.dll
+
+    rem this will override the export library created in the step above
+    rem but tests do not need the export library, they load the dll dynamically
+    lib *.obj /out:libexpat.lib
+
+Then, copy 
+
+- ``libexpat.lib`` into LIB
+- both ``lib\expat.h`` and ``lib\expat_external.h`` in INCLUDE
+- ``libexpat.dll`` into PATH
 
 
 The OpenSSL library
@@ -363,7 +431,7 @@
 It is probably not too much work if the goal is only to get a translated
 PyPy executable, and to run all tests before translation.  But you need
 to start somewhere, and you should start with some tests in
-rpython/translator/c/test/, like ``test_standalone.py`` and
+``rpython/translator/c/test/``, like ``test_standalone.py`` and
 ``test_newgc.py``: try to have them pass on top of CPython64/64.
 
 Keep in mind that this runs small translations, and some details may go
@@ -373,7 +441,7 @@
 should be something like ``long long``.
 
 What is more generally needed is to review all the C files in
-rpython/translator/c/src for the word ``long``, because this means a
+``rpython/translator/c/src`` for the word ``long``, because this means a
 32-bit integer even on Win64.  Replace it with ``Signed`` most of the
 times.  You can replace one with the other without breaking anything on
 any other platform, so feel free to.