Mailman 3 February 2012 - pypy-dev

crash in cpyext dict iteration
by Stefan Behnel Feb. 25, 2012

Feb. 25, 2012

Hi, I've rewritten Cython's dict iteration code to use normal PyIter_Next() iteration on PyPy, but now I get a crash in one of the tests. Apparently, the iterator fails to raise an exception when the dict is being resized during iteration. The test goes like this: ''' def iterdict_change_size(dict d): """ >>> count, i = 0, -1 >>> d = {1:2, 10:20} >>> for i in d: ... d[i+1] = 5 ... count += 1 ... if count > 5: ... … [View More]

1 0

libgmp
by Timothy Baldridge Feb. 25, 2012

Feb. 25, 2012

For a project I'm working on, I'd like to have support for gmp in pypy. I have a ctypes pypy module, but from what I understand, pypy's ctypes are a bit slow compared to CPython. What's the best way to get access to get access to libgmp from python? Would you be against a pull request that added libgmp as an RPython module? Thanks Timothy -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth)

5 6

Re: [pypy-dev] libgmp
by William ML Leslie Feb. 24, 2012

Feb. 24, 2012

On 24 February 2012 13:05, William ML Leslie <william.leslie.ttg(a)gmail.com> wrote: > On 24 February 2012 12:56, Timothy Baldridge <tbaldridge(a)gmail.com> wrote: >> For a project I'm working on, I'd like to have support for gmp in >> pypy. I have a ctypes pypy module, but from what I understand, pypy's >> ctypes are a bit slow compared to CPython. > > On the contrary, ctypes is particularly fast on pypy for common usage. > Does your benchmark … [View More]

2 1

Invitation to San Francisco Python meetup
by Nam Nguyen Feb. 23, 2012

Feb. 23, 2012

Hi all, I'm helping out the SF Python meetup group organize a pre-PyCon meetup in that same week. I am also soliciting half-hour talks from PyPy developers. So if you happened to stay in San Francisco and the surrounding area, would you attend the meetup and/or give us a talk? I noticed some core PyPy developers would also be attending PyCon 2012. We'd be glad to have you present your talks at the meetup too. That would be very helpful to many people who could not attend your sessions. … [View More]

2 1

Re: [pypy-dev] [pypy-commit] pypy sepcomp2: Functions can be @exported without specifying argument types,
by Antonio Cuni Feb. 23, 2012

Feb. 23, 2012

Hi Amaury, On 02/22/2012 12:34 AM, amauryfa wrote: > Author: Amaury Forgeot d'Arc<amauryfa(a)gmail.com> > Branch: sepcomp2 > Changeset: r52748:ce2d7e8a1b42 > Date: 2012-02-21 23:28 +0100 > http://bitbucket.org/pypy/pypy/changeset/ce2d7e8a1b42/ > + def test_implied_signature(self): > + @export # No explicit signature here. > + def f(x): > + return x + 1.5 > + @export() # This is an explicit signature, with no argument. &… [View More]

2 1

question re: ancient SSL requirement of pypy
by Dale Hubler Feb. 23, 2012

Feb. 23, 2012

Hello, I was requested to install pypy but our computers appear to be too new to run it, having libssl.so.0.9.8e among other newer items. This confuses me because the web page for pypy shows a 2011 date and blog entries from 2012. Can 2005 SSL really be a requirement, I am unable to install such an old item on a cluster where this software might be used. I looked at the pypy site but cannot find any supported platforms, install guide, etc. I am trying this on RedHat EL 5. I tried … [View More]

9 10

ctypes - PyPy 1.8 slower than Python 2.6.5
by Sébastien Volle Feb. 23, 2012

Feb. 23, 2012

Hi all, My team is working on a project of fast packet sniffers and I'm comparing performance between different languages. So, we came up with a simple ARP sniffer that I ported to Python using ctypes. During my investigations, I turned out that using ctypes, PyPy 1.8 is 4x slower than CPython 2.6.5. After looking at the PyPy buglist, it's seems there are couple open issues about ctypes so I figured I would ask you guys first before filing a new bug. I'm pretty new to ctypes and pypy so I'm … [View More]not sure I understand what's going. My program seems to spend a lot of time in ctypes/function.py:_convert_args though, has the following profile trace demonstrates: $ pypy -m cProfile -s time arp.py Packet buffer now at 0x9CCECB2 Capture started elapsed time : 3983.84ms Total packets : 35571 packets/s : 8928.81 7839198 function calls (7838340 primitive calls) in 4.105 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 69876 0.546 0.000 1.584 0.000 function.py:480(_convert_args) 214696 0.256 0.000 0.429 0.000 structure.py:236(_subarray) 1052195 0.206 0.000 0.206 0.000 {isinstance} 632437 0.192 0.000 0.192 0.000 {method 'append' of 'list' objects} 175326 0.187 0.000 0.407 0.000 function.py:350(_call_funcptr) 1 0.173 0.173 4.105 4.105 arp.py:1(<module>) 209628 0.158 0.000 0.587 0.000 primitive.py:272(from_param) 214696 0.149 0.000 0.963 0.000 structure.py:90(__get__) 71144 0.143 0.000 0.198 0.000 structure.py:216(__new__) 106713 0.130 0.000 0.208 0.000 array.py:70(_CData_output) 105450 0.124 0.000 2.281 0.000 function.py:689(__call__) 69876 0.123 0.000 1.943 0.000 function.py:278(__call__) 321412 0.102 0.000 0.102 0.000 {method 'fromaddress' of 'Array' objects} 209628 0.088 0.000 0.811 0.000 function.py:437(_conv_param) 179125 0.083 0.000 0.083 0.000 {method 'fieldaddress' of 'StructureInstance' objects} 69883 0.080 0.000 0.122 0.000 primitive.py:308(__init__) 71142 0.076 0.000 0.320 0.000 structure.py:174(from_address) 105450 0.075 0.000 0.145 0.000 function.py:593(_build_result) 139755 0.072 0.000 0.120 0.000 primitive.py:64(generic_xxx_p_from_param) 107983 0.070 0.000 0.125 0.000 basics.py:60(_CData_output) 209828 0.062 0.000 0.062 0.000 {method 'get' of 'dict' objects} 107986 0.055 0.000 0.055 0.000 {method '__new__' of '_ctypes.primitive.SimpleType' objects} 71142 0.052 0.000 0.372 0.000 pointer.py:77(getcontents) 35578 0.052 0.000 0.125 0.000 pointer.py:62(__init__) 35576 0.050 0.000 0.062 0.000 pointer.py:83(setcontents) 106713 0.047 0.000 0.047 0.000 {method '__new__' of '_ctypes.array.ArrayMeta' objects} 139750 0.043 0.000 0.181 0.000 primitive.py:84(from_param_void_p) 209625 0.043 0.000 0.691 0.000 basics.py:50(get_ffi_param) 283592/283435 0.041 0.000 0.041 0.000 {len} 71144 0.040 0.000 0.040 0.000 {method '__new__' of '_ctypes.structure.StructOrUnionMeta' objects} 105450 0.039 0.000 0.039 0.000 {method 'free_temp_buffers' of '_ffi.FuncPtr' objects} 176683 0.037 0.000 0.037 0.000 {hasattr} 35571 0.037 0.000 0.423 0.000 pcap.py:89(next) Anyway, I attached my full project to this mail, including a sample pcap capture file. It requires libpcap to be installed on your system. Run arp.py to run the sniffer on the included demo.pcap capture file. I realize I should try and narrow down the issue some more, but I can't really afford to spend too much time on this right now so hopefully, this will be a least a bit helpful. Thanks! Regards, Sébastien [View Less]

10 59

STM status
by Armin Rigo Feb. 22, 2012

Feb. 22, 2012

Hi all, An update for STM: today I managed to build a pypy using the new "stm gc". It runs richards.py on tannit: in 1 thread: 2320 ms per iteration in 2 threads: 1410 ms per iteration in 4 threads: 785 ms per iteration in 8 threads: 685 ms per iteration The small gap between 4 and 8 threads is due to tannit having only 4 "real" cpus, each one hyperthreaded. The additional gain is thus smaller than expected. For comparison, a "pypy --jit off" runs at 650ms per iteration. So the single-… [View More]

5 12

Re: [pypy-dev] [pypy-commit] pypy default: issue1059 testing
by Alex Gaynor Feb. 21, 2012

Feb. 21, 2012

Unfortunately this commit has some bad effects. Going through an iterator in popitem() will result in O(N**2) behavior for repeated calls. If you look at the r_dict implementation of popitem() you can see the fix there. Alex On Mon, Feb 20, 2012 at 6:18 AM, cfbolz <noreply(a)buildbot.pypy.org> wrote: > Author: Carl Friedrich Bolz <cfbolz(a)gmx.de> > Branch: > Changeset: r52670:4b254e123047 > Date: 2012-02-20 12:17 +0100 > http://bitbucket.org/pypy/pypy/changeset/… [View More]4b254e123047/ > > Log: issue1059 testing > > make the .__dict__.clear method of builtin types raise an error. Fix > popitem on dict proxies (builtin types raise an error, normal types > work normally). > > diff --git a/pypy/objspace/std/dictmultiobject.py > b/pypy/objspace/std/dictmultiobject.py > --- a/pypy/objspace/std/dictmultiobject.py > +++ b/pypy/objspace/std/dictmultiobject.py > @@ -142,6 +142,13 @@ > else: > return result > > + def popitem(self, w_dict): > + space = self.space > + iterator = self.iter(w_dict) > + w_key, w_value = iterator.next() > + self.delitem(w_dict, w_key) > + return (w_key, w_value) > + > def clear(self, w_dict): > strategy = self.space.fromcache(EmptyDictStrategy) > storage = strategy.get_empty_storage() > diff --git a/pypy/objspace/std/dictproxyobject.py > b/pypy/objspace/std/dictproxyobject.py > --- a/pypy/objspace/std/dictproxyobject.py > +++ b/pypy/objspace/std/dictproxyobject.py > @@ -3,7 +3,7 @@ > from pypy.objspace.std.dictmultiobject import W_DictMultiObject, > IteratorImplementation > from pypy.objspace.std.dictmultiobject import DictStrategy > from pypy.objspace.std.typeobject import unwrap_cell > -from pypy.interpreter.error import OperationError > +from pypy.interpreter.error import OperationError, operationerrfmt > > from pypy.rlib import rerased > > @@ -44,7 +44,8 @@ > raise > if not w_type.is_cpytype(): > raise > - # xxx obscure workaround: allow cpyext to write to > type->tp_dict. > + # xxx obscure workaround: allow cpyext to write to > type->tp_dict > + # xxx even in the case of a builtin type. > # xxx like CPython, we assume that this is only done early > after > # xxx the type is created, and we don't invalidate any cache. > w_type.dict_w[key] = w_value > @@ -86,8 +87,14 @@ > for (key, w_value) in > self.unerase(w_dict.dstorage).dict_w.iteritems()] > > def clear(self, w_dict): > - self.unerase(w_dict.dstorage).dict_w.clear() > - self.unerase(w_dict.dstorage).mutated(None) > + space = self.space > + w_type = self.unerase(w_dict.dstorage) > + if (not space.config.objspace.std.mutable_builtintypes > + and not w_type.is_heaptype()): > + msg = "can't clear dictionary of type '%s'" > + raise operationerrfmt(space.w_TypeError, msg, w_type.name) > + w_type.dict_w.clear() > + w_type.mutated(None) > > class DictProxyIteratorImplementation(IteratorImplementation): > def __init__(self, space, strategy, dictimplementation): > diff --git a/pypy/objspace/std/test/test_dictproxy.py > b/pypy/objspace/std/test/test_dictproxy.py > --- a/pypy/objspace/std/test/test_dictproxy.py > +++ b/pypy/objspace/std/test/test_dictproxy.py > @@ -22,6 +22,9 @@ > assert NotEmpty.string == 1 > raises(TypeError, 'NotEmpty.__dict__.setdefault(15, 1)') > > + key, value = NotEmpty.__dict__.popitem() > + assert (key == 'a' and value == 1) or (key == 'b' and value == 4) > + > def test_dictproxyeq(self): > class a(object): > pass > @@ -43,6 +46,11 @@ > assert s1 == s2 > assert s1.startswith('{') and s1.endswith('}') > > + def test_immutable_dict_on_builtin_type(self): > + raises(TypeError, "int.__dict__['a'] = 1") > + raises(TypeError, int.__dict__.popitem) > + raises(TypeError, int.__dict__.clear) > + > class AppTestUserObjectMethodCache(AppTestUserObject): > def setup_class(cls): > cls.space = gettestobjspace( > diff --git a/pypy/objspace/std/test/test_typeobject.py > b/pypy/objspace/std/test/test_typeobject.py > --- a/pypy/objspace/std/test/test_typeobject.py > +++ b/pypy/objspace/std/test/test_typeobject.py > @@ -993,7 +993,9 @@ > raises(TypeError, setattr, list, 'append', 42) > raises(TypeError, setattr, list, 'foobar', 42) > raises(TypeError, delattr, dict, 'keys') > - > + raises(TypeError, 'int.__dict__["a"] = 1') > + raises(TypeError, 'int.__dict__.clear()') > + > def test_nontype_in_mro(self): > class OldStyle: > pass > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > http://mail.python.org/mailman/listinfo/pypy-commit > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero [View Less]

2 2

Building PyPy with cx_Oracle fails
by Phillip Class Feb. 21, 2012

Feb. 21, 2012

Building PyPy with cx_Oracle fails. I am using Python 2.7 on Ubuntu 64-bit with Oracle 11.2. Build command is: python translate.py -Ojit targetpypystandalone.py --withmod-oracle. Below is the traceback: [Timer] Timings: [Timer] annotate --- 1834.3 s [Timer] rtype_lltype --- 1697.2 s [Timer] pyjitpl_lltype --- 1216.3 s [Timer] =========================================== [Timer] Total: --- 4747.9 s [translation:ERROR]… [View More] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 309, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/translator/driver.py", line 814, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/translator/driver.py", line 287, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/translator/driver.py", line 399, in task_pyjitpl_lltype [translation:ERROR] backend_name=self.config.translation.jit_backend, inline=True) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/jit/metainterp/warmspot.py", line 48, in apply_jit [translation:ERROR] warmrunnerdesc.finish() [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/jit/metainterp/warmspot.py", line 236, in finish [translation:ERROR] self.annhelper.finish() [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/rpython/annlowlevel.py", line 240, in finish [translation:ERROR] self.finish_annotate() [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/rpython/annlowlevel.py", line 259, in finish_annotate [translation:ERROR] ann.complete_helpers(self.policy) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 180, in complete_helpers [translation:ERROR] self.complete() [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 254, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 452, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 512, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 714, in consider_op [translation:ERROR] raise_nicer_exception(op, str(graph)) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 711, in consider_op [translation:ERROR] resultcell = consider_meth(*argcells) [translation:ERROR] File "<4444-codegen /home/pclass/Desktop/pypy/pypy/annotation/annrpython.py:749>", line 3, in consider_op_simple_call [translation:ERROR] return arg.simple_call(*args) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/unaryop.py", line 175, in simple_call [translation:ERROR] return obj.call(getbookkeeper().build_args("simple_call", args_s)) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/unaryop.py", line 706, in call [translation:ERROR] return bookkeeper.pbc_call(pbc, args) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/bookkeeper.py", line 668, in pbc_call [translation:ERROR] results.append(desc.pycall(schedule, args, s_previous_result, op)) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/description.py", line 976, in pycall [translation:ERROR] return self.funcdesc.pycall(schedule, args, s_previous_result, op) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/description.py", line 297, in pycall [translation:ERROR] result = schedule(graph, inputcells) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/bookkeeper.py", line 664, in schedule [translation:ERROR] return self.annotator.recursivecall(graph, whence, inputcells) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 395, in recursivecall [translation:ERROR] position_key) [translation:ERROR] File "/home/pclass/Desktop/pypy/pypy/annotation/annrpython.py", line 235, in addpendingblock [translation:ERROR] assert annmodel.unionof(s_oldarg, s_newarg) == s_oldarg [translation:ERROR] AssertionError': [translation:ERROR] .. v2309 = simple_call(v2301, v2302, v2303, v2304, v2305, v2306, v2307, v2308) [translation:ERROR] .. '(pypy.module.pypyjit.policy:49)PyPyJitIface._compile_hook' [translation:ERROR] Processing block: [translation:ERROR] block@226 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (pypy.module.pypyjit.policy:49)PyPyJitIface._compile_hook [translation:ERROR] containing the following operations: [translation:ERROR] v2309 = simple_call(v2301, v2302, v2303, v2304, v2305, v2306, v2307, v2308) [translation:ERROR] --end-- [translation] start debugger... > /home/pclass/Desktop/pypy/pypy/annotation/annrpython.py(235)addpendingblock() -> assert annmodel.unionof(s_oldarg, s_newarg) == s_oldarg (Pdb+) [View Less]

2 1