Mailman 3 May 2012 - pypy-dev

how to build pypy with psycopg (links / requirements)
by Gelonida N May 31, 2012

May 31, 2012

Hi, Is there any good document / tutorial showing how to build pypy with the Rpython version of psycopg? Thanks an advance for good links. I'm working on Ubuntu 12.04

4 5

on using the new bitbucket teams feature
by Ronny Pfannschmidt May 30, 2012

May 30, 2012

hi, since bitbucket finally supports teams, i think should look into migrating the pypy user to that feature -- Ronny

1 0

Re: [pypy-dev] change of strategy for the py3k branch?
by Laura Creighton May 30, 2012

May 30, 2012

About 25 years ago (dear god that makes me feel old) when CVS was the new and happening thing in version control systems, we were in the really bad problem area at Human Computing Resources of porting unix to more than 1000, kid you not, of different M68010, M68020, NS16032, NS32032, Zilog-I-forget, MIPS, and I forget what other hardware-based systems, with completely crazy combination of graphical cards, interrupt driven weird hardware and well, trust me, it was a mess. We wanted one kernel … [View More]

1 0

first test with pypy and django
by Gelonida N May 30, 2012

May 30, 2012

Hi, I wanted to make my very first test with pypy by using and django and sqlite3 when running ./manage.py syncdb I'm asked to enter a password and pypy aborts as soon as I press enter. > File "/home/klausf/mypypy/site-packages/django/contrib/auth/management/commands/createsuperuser.py", line 109, in handle > password = getpass.getpass() > File "/usr/lib/pypy/lib-python/2.7/getpass.py", line 74, in unix_getpass > stream.flush() # issue7208 > IOError: [Errno 29]… [View More]

1 0

Re: [pypy-dev] Python FFI
by Laura Creighton May 29, 2012

May 29, 2012

In a message of Tue, 15 May 2012 16:23:14 +0200, Armin Rigo writes: tons of things that I agree with more or less wholesale, though I think that there is a slightly larger market for Cython in the immediate future ... Then comes this: >The simplest FFI we know of for a high-level language is LuaJIT's FFI. > If you are interested, read the four pages starting at >http://luajit.org/ext_ffi.html . A crucial design decision was to >avoid making use of tons of objects representing … [View More]

3 2

test__ffi_call_releases_gil test failing in win32
by Matti Picus May 29, 2012

May 29, 2012

I need some hints how to debug a crash in pypy when running this test test__ffi_call_releases_gil,in pypy.module.pypyjit.test_pypy_c.test__ffi. I fixed the failing test so it can find the Sleep function (in win32) but then pypy crashes, with one of those annoying "report this error" dialog boxes. Here's what I have so far:https://gist.github.com/2821258 It's python code that is run with pypy --jit threshold=150 When run without the --jit, all is OK But with the jit flag pypy (win32, of … [View More]

4 3

test_rpoll.py::test_translate with MinGW32
by bookaa May 28, 2012

May 28, 2012

with set cc=mingw32-gcc this test fail: pytest.py pypy/rlib/test/test_rpoll.py::test_translate this is how to patch: diff -crN pypy-pypy-4a38b43757e3/pypy/rlib/_rsocket_rffi.py pypy-pypy-4a38b43757e3.bookaa/pypy/rlib/_rsocket_rffi.py *** pypy-pypy-4a38b43757e3/pypy/rlib/_rsocket_rffi.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/pypy/rlib/_rsocket_rffi.py Mon May 28 22:00:03 2012 *************** *** 58,69 **** header_lines = [ '#include <WinSock2.h… [View More]>', '#include <WS2tcpip.h>', - '#include <Mstcpip.h>', # winsock2 defines AF_UNIX, but not sockaddr_un '#undef AF_UNIX', ] if _MSVC: header_lines.extend([ # these types do not exist on microsoft compilers 'typedef int ssize_t;', 'typedef unsigned __int16 uint16_t;', --- 58,69 ---- header_lines = [ '#include <WinSock2.h>', '#include <WS2tcpip.h>', # winsock2 defines AF_UNIX, but not sockaddr_un '#undef AF_UNIX', ] if _MSVC: header_lines.extend([ + '#include <Mstcpip.h>', # these types do not exist on microsoft compilers 'typedef int ssize_t;', 'typedef unsigned __int16 uint16_t;', *************** *** 71,77 **** ]) else: # MINGW includes = ('stdint.h',) ! """ header_lines.extend([ '''\ #ifndef _WIN32_WINNT --- 71,77 ---- ]) else: # MINGW includes = ('stdint.h',) ! header_lines.extend([ '''\ #ifndef _WIN32_WINNT *************** *** 89,95 **** u_long keepaliveinterval; };''' ]) ! """ HEADER = '\n'.join(header_lines) COND_HEADER = '' constants = {} --- 89,95 ---- u_long keepaliveinterval; };''' ]) ! HEADER = '\n'.join(header_lines) COND_HEADER = '' constants = {} diff -crN pypy-pypy-4a38b43757e3/pypy/rpython/tool/rfficache.py pypy-pypy-4a38b43757e3.bookaa/pypy/rpython/tool/rfficache.py *** pypy-pypy-4a38b43757e3/pypy/rpython/tool/rfficache.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/pypy/rpython/tool/rfficache.py Mon May 28 21:58:02 2012 *************** *** 13,19 **** from pypy.tool.gcc_cache import build_executable_cache def ask_gcc(question, add_source=""): ! includes = ['stdlib.h', 'stdio.h', 'sys/types.h'] if os.name != 'nt': includes += ['inttypes.h'] include_string = "\n".join(["#include [View Less]

2 1

sys._getframe(1).f_locals is very slow on PyPy 1.8
by Makoto Kuwata May 27, 2012

May 27, 2012

Hi, I found that sys._getframe() (or sys._getframe().f_locals) is very slow in PyPy 1.8. For example, the following takes only 0.0398 sec on PyPy 1.8: def f5(x): return f4(x) def f4(x): return f3(x) def f2(x): return f1(x) def f1(x): reutrn None ## just return None for _ in xrange(1000000): f5(0) But the next one takes 1.3563 sec. def f5(x): return f4(x) def f4(x): return f3(x) def f2(x): return f1(x) def f1(x): … [View More]

6 12

pytest test_rposix.py with mingw32
by bookaa May 27, 2012

May 27, 2012

keywords: _PyVerify_fd __pioinfo _msize PyPy can not pass this test: pytest.py --cc=mingw32-gcc pypy\rlib\test\test_rposix.py::TestPosixUnicode::()::test_is_valid_fd After 2 days of hard work, I finally realise why. Patch file attached. The problem is in _PyVerify_fd: FILE* f = fopen("d:\\55.txt","w"); int no = fileno(f); int flg1 = _PyVerify_fd(no); fclose(f); int flg2 = _PyVerify_fd(no); printf("flg1 %d should be 1, flg2 %d should be 0", flg1, flg2) in file rposix.py, source … [View More]of function _PyVerify_fd is write to a c file, and will be compile to a Windows DLL file. In this function, it use extern __declspec(dllimport) char * __pioinfo[]; __pioinfo as a global variable. I find use __pioinfo in a DLL file is not safe. the __pioinfo and _msize may export from many DLL: msvcrt.dll msvcr90.dll msvcr90d.dll or maybe msvcr70.dll ... If we fopen and fileno in a EXE file, and call _PyVerify_fd in a DLL file, its very danger. The EXE and DLL may reference different __pioinfo and get error. So I think test_rposix.py::TestPosixUnicode::()::test_is_valid_fd can pass in MSVC is only a coincidence. _PyVerify_fd in a DLL and use dllimport __pioinfo is not safe. In my fix, I do not assume any DLL we should use. I try to find the current DLL which export __pioinfo already loaded. Sure this is not very good. But I think if we must place _PyVerify_fd in a DLL, no good solution. After this path, this tests are pass: pytest.py --cc=mingw32-gcc pypy\rlib\test\test_rposix.py pytest.py pypy\rlib\test\test_rposix.py ----------- diff -crN pypy-pypy-4a38b43757e3/_pytest/config.py pypy-pypy-4a38b43757e3.bookaa/_pytest/config.py *** pypy-pypy-4a38b43757e3/_pytest/config.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/_pytest/config.py Sat May 26 20:47:43 2012 *************** *** 157,162 **** --- 157,164 ---- for arg in args + [current]: if hasattr(arg, 'startswith') and arg.startswith("--"): continue + if arg.find('::') != -1: + arg = arg[:arg.find('::')] anchor = current.join(arg, abs=1) if anchor.check(): # we found some file object self._path2confmods[None] = self.getconftestmodules(anchor) diff -crN pypy-pypy-4a38b43757e3/pypy/conftest.py pypy-pypy-4a38b43757e3.bookaa/pypy/conftest.py *** pypy-pypy-4a38b43757e3/pypy/conftest.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/pypy/conftest.py Sat May 26 20:59:51 2012 *************** *** 33,38 **** --- 33,42 ---- raise ValueError("%s not in %s" % (value, PLATFORMS)) set_platform(value, None) + def _set_compiler(opt, opt_str, value, parser): #add by bookaa + from pypy.translator.platform import set_platform + set_platform('host', value) + def pytest_addoption(parser): group = parser.getgroup("pypy options") group.addoption('--view', action="store_true", dest="view", default=False, *************** *** 46,51 **** --- 50,58 ---- group.addoption('-P', '--platform', action="callback", type="string", default="host", callback=_set_platform, help="set up tests to use specified platform as compile/run target") + group.addoption('--cc', action="callback", type="string", #add by bookaa + default="host", callback=_set_compiler, + help="set up tests to use specified compiler") group = parser.getgroup("JIT options") group.addoption('--viewloops', action="store_true", default=False, dest="viewloops", diff -crN pypy-pypy-4a38b43757e3/pypy/rlib/rposix.py pypy-pypy-4a38b43757e3.bookaa/pypy/rlib/rposix.py *** pypy-pypy-4a38b43757e3/pypy/rlib/rposix.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/pypy/rlib/rposix.py Sat May 26 20:51:13 2012 *************** *** 24,34 **** separate_module_sources =[''' /* Lifted completely from CPython 3.3 Modules/posix_module.c */ #include <malloc.h> /* for _msize */ typedef struct { intptr_t osfhnd; char osfile; } my_ioinfo; ! extern __declspec(dllimport) char * __pioinfo[]; #define IOINFO_L2E 5 #define IOINFO_ARRAY_ELTS (1 << IOINFO_L2E) #define IOINFO_ARRAYS 64 --- 24,35 ---- separate_module_sources =[''' /* Lifted completely from CPython 3.3 Modules/posix_module.c */ #include <malloc.h> /* for _msize */ + #include <windows.h> typedef struct { intptr_t osfhnd; char osfile; } my_ioinfo; ! //extern __declspec(dllimport) char * __pioinfo[]; #define IOINFO_L2E 5 #define IOINFO_ARRAY_ELTS (1 << IOINFO_L2E) #define IOINFO_ARRAYS 64 *************** *** 36,41 **** --- 37,58 ---- #define FOPEN 0x01 #define _NO_CONSOLE_FILENO (intptr_t)-2 + char** Get_pioinfo(void** pp) + { + HMODULE hm; + hm = GetModuleHandleA("msvcr90.dll"); + if (hm == 0) + hm = GetModuleHandleA("msvcr90d.dll"); + if (hm == 0) + hm = GetModuleHandleA("msvcrt.dll"); + if (hm) + { + void * p = GetProcAddress(hm, "__pioinfo"); + *pp = GetProcAddress(hm, "_msize"); + return (char**)p; + } + return 0; + } /* This function emulates what the windows CRT does to validate file handles */ int *************** *** 45,56 **** const int i2 = fd & ((1 << IOINFO_L2E) - 1); static size_t sizeof_ioinfo = 0; /* Determine the actual size of the ioinfo structure, * as used by the CRT loaded in memory */ if (sizeof_ioinfo == 0 && __pioinfo[0] != NULL) { ! sizeof_ioinfo = _msize(__pioinfo[0]) / IOINFO_ARRAY_ELTS; } if (sizeof_ioinfo == 0) { /* This should not happen... */ --- 62,77 ---- const int i2 = fd & ((1 << IOINFO_L2E) - 1); static size_t sizeof_ioinfo = 0; + int (*my_msize)(char*) = 0; + char** __pioinfo = Get_pioinfo((void**)&my_msize); + if (__pioinfo == 0) + return 0; /* Determine the actual size of the ioinfo structure, * as used by the CRT loaded in memory */ if (sizeof_ioinfo == 0 && __pioinfo[0] != NULL) { ! sizeof_ioinfo = my_msize(__pioinfo[0]) / IOINFO_ARRAY_ELTS; } if (sizeof_ioinfo == 0) { /* This should not happen... */ *************** *** 75,80 **** --- 96,105 ---- return 0; } ''',] + from pypy.translator.platform import platform + if platform.name == 'mingw32': + separate_module_sources[0] = '''#include <stdint.h> /* for intptr_t */ + ''' + separate_module_sources[0] export_symbols = ['_PyVerify_fd'] else: separate_module_sources = [] diff -crN pypy-pypy-4a38b43757e3/pypy/rpython/tool/rfficache.py pypy-pypy-4a38b43757e3.bookaa/pypy/rpython/tool/rfficache.py *** pypy-pypy-4a38b43757e3/pypy/rpython/tool/rfficache.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/pypy/rpython/tool/rfficache.py Sat May 26 20:47:42 2012 *************** *** 13,19 **** from pypy.tool.gcc_cache import build_executable_cache def ask_gcc(question, add_source=""): ! includes = ['stdlib.h', 'stdio.h', 'sys/types.h'] if os.name != 'nt': includes += ['inttypes.h'] include_string = "\n".join(["#include <%s>" % i for i in includes]) --- 13,23 ---- from pypy.tool.gcc_cache import build_executable_cache def ask_gcc(question, add_source=""): ! from pypy.translator.platform import platform ! if platform.name == 'mingw32': ! includes = ['stdlib.h', 'stdio.h', 'sys/types.h', 'stdint.h'] ! else: ! includes = ['stdlib.h', 'stdio.h', 'sys/types.h'] if os.name != 'nt': includes += ['inttypes.h'] include_string = "\n".join(["#include <%s>" % i for i in includes]) diff -crN pypy-pypy-4a38b43757e3/pypy/translator/platform/posix.py pypy-pypy-4a38b43757e3.bookaa/pypy/translator/platform/posix.py *** pypy-pypy-4a38b43757e3/pypy/translator/platform/posix.py Tue May 22 14:03:20 2012 --- pypy-pypy-4a38b43757e3.bookaa/pypy/translator/platform/posix.py Sat May 26 20:47:42 2012 *************** *** 55,60 **** --- 55,62 ---- if relto: response_file = relto.bestrelpath(response_file) + if self.name == 'mingw32': + return ["-Wl,--export-all-symbols,--version-script=%s" % (response_file,)] return ["-Wl,--export-dynamic,--version-script=%s" % (response_file,)] def _link(self, cc, ofiles, link_args, standalone, exe_name): [View Less]

2 1

add --cc option to pytest
by bookaa May 27, 2012

May 27, 2012

according to http://web.archiveorange.com/archive/v/cZ0K2Jye2cyGARzh9OlA after patch pypy/conftest.py, pytest.py now support --cc option: pytest.py --cc=mingw32-gcc pypy/translator/test/test_exceptiontransform.py but we still can not use --cc option on specify function: pytest.py --cc=mingw32-gcc pypy/translator/test/test_exceptiontransform.py::TestOOType::()::test_simple this is how to add this support: *** pypy-pypy-4a38b43757e3\_pytest\config.py Sat May 26 09:16:32 2012 --- pypy-pypy-… [View More]

3 2