...

...

...

import sys sys.path ['', '/usr/lib64/pypy-1.4.1/lib_pypy', '/usr/lib64/pypy-1.4.1/lib-python/modified- 2.5.2', '/usr/lib64/pypy-1.4.1/lib-python/2.5.2', '/usr/lib64/pypy-1.4.1/lib-

New submission from Dave Malcolm <dmalcolm@redhat.com>: Short version: I'm working on packaging PyPy in RPM form for Fedora; see: https://bugzilla.redhat.com/show_bug.cgi?id=588941 and ran into an issue with how the pypy binary searches for its libraries. I'm attaching a patch I've written that fixes things, though it's not good enough yet to be applied to hg. Long version: As I understand things, PyPy's prebuilt tarballs have a directory structure generated by pypy/tool/release/package.py. Looking at e.g. http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2 everything is below: pypy-1.4.1-linux64/ with: pypy-1.4.1-linux64/lib_pypy/ pypy-1.4.1-linux64/lib-python/ pypy-1.4.1-linux64/lib-python/2.5.2/*.py (and subdirs) pypy-1.4.1-linux64/lib-python/modified-2.5.2/*.py (and subdirs) pypy-1.4.1-linux64/site-packages/*.py pypy-1.4.1-linux64/bin/pypy (the JIT binary) pypy-1.4.1-linux64/include/ (some header files) (See also pypy/tool/release/test/test_package.py, which verifies this structure) This works for a standalone distribution of PyPy. However, eventually Linux distribution will want to ship "system" builds of PyPy, deployed as .rpmor .deb packages. Those packages ideally ought to follow the layout recommendations of the Filesystem Hierarchy Standard: http://www.pathname.com/fhs/ In particular this suggests separating out programs (to /usr/bin) from libraries (to /usr/lib) and data (/usr/share). Exactly where the pypy libraries should go is debatable; they're all architecture- independent for now, but I hope we'll soon have a rich collection of extension modules that are stable enough to package. So I'm attempting to split the libraries out to /usr/lib (and /usr/lib64 on 64-bit, which is a Fedora-ism, AIUI). My naive attempt to do this leads to a /usr/bin/pypy which emits this message at startup: $ pypy-jit debug: WARNING: library path not found, using compiled-in sys.path 'import site' failed Python 2.5.2 (78826, Dec 15 2010, 01:41:24) [PyPy 1.4.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. debug: OperationError: debug: operror-type: ImportError debug: operror-value: No module named _pypy_interact and has this sys.path: $ pypy -c 'import sys; print sys.path' debug: WARNING: library path not found, using compiled-in sys.path 'import site' failed ['', '/builddir/build/BUILD/pypy-1.4/lib_pypy', '/builddir/build/BUILD/pypy-1.4/lib-python/modified-2.5.2', '/builddir/build/BUILD/pypy-1.4/lib-python/2.5.2', '/builddir/build/BUILD/pypy-1.4/lib-python/2.5.2/plat-linux2'] pypy/translator/goal/app_main.py's startup code calls into sys's state.py, which has: def getinitialpath(prefix): which searches for the dirs, raising OSError if they're not found. If they're all their, it returns them in a list. app_main wraps this as sys.pypy_initial_path, turning OSError into a return of None. I'm attaching a proof-of-concept patch which adds an initial search for the directory structure, this time within LIBRARY_INSTALLATION_PATH. Ideally this would be an optional, configuration-supplied constant, but for now, this is a non-existant variable, which I pre-process within the build, using sed, thus: sed -i \ -e 's|LIBRARY_INSTALLATION_PATH|"%{pypyprefix}"|' \ pypy/translator/goal/app_main.py where %{pypyprefix} is expanded by another tool, giving the string literal "/usr/lib64/pypy-1.4.1" I wanted to prove the concept, and hour-long builds make for tedious experimentation :( Presumably this should be something like a: --library-installation-path=/usr/lib that gets passed to the translate.py Building with this patch (with the above caveat), I obtain a binary that, when copied to /usr/bin/pypy, successfully finds its libraries within our downstream distribution's directory structure (under /usr/lib64/pypy-1.4.1 on this build): $ pypy Python 2.5.2 (, Dec 22 2010, 23:14:02) [PyPy 1.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``PyPy is an exciting technology that lets you to write fast, portable, multi-platform interpreters with less effort'' python/2.5.2/plat-linux2', '/usr/lib64/pypy-1.4.1/site-packages'] and also successfully finds its libraries within the build tree, when run from there in the usual "goal" subdirectory.

...

...

...

import collections collections.__file__ '/usr/lib64/pypy-1.4.1/lib_pypy/collections.pyc'

---------- effort: ??? files: pypy-1.4.1-add-LIBRARY_INSTALLATION_PATH.patch messages: 2031 nosy: dmalcolm, pypy-issue priority: feature release: ??? status: unread title: Packaging PyPy: installation path for PyPy's libraries _______________________________________________________ PyPy development tracker <pypy-dev-issue@codespeak.net> <https://codespeak.net/issue/pypy-dev/issue614> _______________________________________________________