Mailman 3 March 2013 - pypy-dev

Experimental GIL
by Marko Tasic March 23, 2013

March 23, 2013

Hi, I have had an idea for some time to re-implement how GIL works, but nothing radical. On purpose I'm saying that I don't want to "remove" GIL, but to modify it, so people who follow your list don't get too excited. Let me describe what I want to accomplish, and what kind of results am looking for, and also hear any possible design or implementation issue I/we may face. I want threads to work in parallel as they already do, with very little waiting times on other threads in case of GIL'ing. Actually, I want to use time reserved for GIL to do something else. To be clear, I don't need JIT, and lets take it out of discussion for some time later. First step: ------------- My solution require wait-free lock-free queue, so many threads can only append/push to it atomically. Every thread created, including main thread, has to have (at least one) such a queue, and lets say queue has pointer to thread where it belongs. Not only queue is required, but also conditional variable (such as pthread_cond_t) per thread. What I imagine is high-level thread object that consists of low-level thread, low-level wait-free lock-free queue, and low-level conditional variable. Imagine that this high-level thread object is local per each OS thread created. From now on I will call high-level thread object just "thread". Every mutable object created has to have pointer to a thread in which its created. As a result, this way we can know origin of a mutable object by thread by inspecting mutable object itself, so as a consequence also know low-level thread, low-level queue and low-level condition variable associated by it. Having this special field is naive approach, so of course we can have allocation areas per thread, and GC can be smarter, but lets just forget about it for now. Second step: ----------------- Every time "ceval" equivalent function observes a mutable object on bytecode interpretation, it checks if local thread is same as object's thread. If they are same, then proceed with interpretation of local ceval operation. If they are different, then: - lets call local thread just T - lets call operation just O - take low-level queue of object's thread, lets call it Q - group O and T into item and push it to Q - set T's condition variable to wait Third step: ------------- Lets imagine that we have old style GIL that counts opcodes executed, and after N opcodes instead of acquiring GIL, it processes queue. This processing is done by popping out operation (O) and thread (T) items from local thread's queue, and serially executing operations. After each executed operation it sends signal to condition variable passed inside thread which was passed in same queue item where is operation. Once, signal is received by awaiting thread, until now awaiting thread proceeds with execution, while thread that was processing queue also continues executing queued operations. Conclusion: --------------- There is ceval per thread. All ceval's run at same time. In the best case, they don't wait on each other because threads do not touch objects created in other threads. In the worst case, there is always only one thread executing operation, and all other are waiting it to finish, and this is close to how GIL already works (almost). Special care has to be taken for threads that are going to finish, and what happens with their objects if they are referenced by other objects that belong to other threads. So I have also some questions: - How flexible is to modify GIL in pypy? - Where should I look? - Do I need to fully translate pypy in order to test it? - I need wait-free look-free queue, so what kind of low-level atomic primitives do RPython has to offer? - I need to have special field for all mutable objects, so how to do so? I do not expect any performance boost, just correctness of approach. Perhaps, I'm wrong. What do you think, is this practical approach? Regards, Marko Tasic

2 1

Re: [pypy-dev] [pypy-commit] pypy default: (fijal, alex) kill 'int' as a multimethod
by Carl Friedrich Bolz March 23, 2013

March 23, 2013

Hi Maciej, Alex, I really don't like this commit. It does two unrelated pretty fundamental changes together, killing small integers and making int a normal method. The first of these is not even explained in the commit message. I understand the reasons for removing small integers (they don't work with the JIT anyway, fixing that would be hard, ...). But it would still be good to have two commits, one that removes them, the other one that does your refactoring. This would give people that want to revive them at a later point a place to start. I don't actually know what should be done now. Backing out your change and doing it in two steps would be the right approach somehow, even if it requires some work. Cheers, Carl Friedrich On 03/23/2013 08:02 PM, fijal wrote: > Author: Maciej Fijalkowski <fijall(a)gmail.com> > Branch: > Changeset: r62700:e258f1470515 > Date: 2013-03-23 12:01 -0700 > http://bitbucket.org/pypy/pypy/changeset/e258f1470515/ > > Log: (fijal, alex) kill 'int' as a multimethod > > diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py > --- a/pypy/config/pypyoption.py > +++ b/pypy/config/pypyoption.py > @@ -189,11 +189,6 @@ > BoolOption("withtproxy", "support transparent proxies", > default=True), > > - BoolOption("withsmallint", "use tagged integers", > - default=False, > - requires=[("objspace.std.withprebuiltint", False), > - ("translation.taggedpointers", True)]), > - > BoolOption("withprebuiltint", "prebuild commonly used int objects", > default=False), > > @@ -204,9 +199,7 @@ > default=100, cmdline="--prebuiltintto"), > > BoolOption("withsmalllong", "use a version of 'long' in a C long long", > - default=False, > - requires=[("objspace.std.withsmallint", False)]), > - # ^^^ because of missing delegate_xx2yy > + default=False), > > BoolOption("withstrbuf", "use strings optimized for addition (ver 2)", > default=False), > diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py > --- a/pypy/interpreter/baseobjspace.py > +++ b/pypy/interpreter/baseobjspace.py > @@ -223,6 +223,22 @@ > raise OperationError(space.w_TypeError, > typed_unwrap_error_msg(space, "integer", self)) > > + def int(self, space): > + w_impl = space.lookup(self, '__int__') > + if w_impl is None: > + typename = space.type(self).getname(space) > + raise operationerrfmt(space.w_TypeError, > + "unsupported operand type for int(): '%s'", > + typename) > + w_result = space.get_and_call_function(w_impl, self) > + > + if (space.isinstance_w(w_result, space.w_int) or > + space.isinstance_w(w_result, space.w_long)): > + return w_result > + typename = space.type(w_result).getname(space) > + msg = "__int__ returned non-int (type '%s')" > + raise operationerrfmt(space.w_TypeError, msg, typename) > + > def __spacebind__(self, space): > return self > > @@ -1306,6 +1322,9 @@ > def int_w(self, w_obj): > return w_obj.int_w(self) > > + def int(self, w_obj): > + return w_obj.int(self) > + > def uint_w(self, w_obj): > return w_obj.uint_w(self) > > diff --git a/pypy/objspace/descroperation.py b/pypy/objspace/descroperation.py > --- a/pypy/objspace/descroperation.py > +++ b/pypy/objspace/descroperation.py > @@ -808,7 +808,6 @@ > # more of the above manually-coded operations as well) > > for targetname, specialname, checkerspec in [ > - ('int', '__int__', ("space.w_int", "space.w_long")), > ('index', '__index__', ("space.w_int", "space.w_long")), > ('long', '__long__', ("space.w_int", "space.w_long")), > ('float', '__float__', ("space.w_float",))]: > diff --git a/pypy/objspace/std/complexobject.py b/pypy/objspace/std/complexobject.py > --- a/pypy/objspace/std/complexobject.py > +++ b/pypy/objspace/std/complexobject.py > @@ -111,6 +111,9 @@ > > return w_result > > + def int(self, space): > + raise OperationError(space.w_TypeError, space.wrap("can't convert complex to int; use int(abs(z))")) > + > registerimplementation(W_ComplexObject) > > w_one = W_ComplexObject(1, 0) > @@ -245,9 +248,6 @@ > def float__Complex(space, w_complex): > raise OperationError(space.w_TypeError, space.wrap("can't convert complex to float; use abs(z)")) > > -def int__Complex(space, w_complex): > - raise OperationError(space.w_TypeError, space.wrap("can't convert complex to int; use int(abs(z))")) > - > def complex_conjugate__Complex(space, w_self): > #w_real = space.call_function(space.w_float,space.wrap(w_self.realval)) > #w_imag = space.call_function(space.w_float,space.wrap(-w_self.imagval)) > diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py > --- a/pypy/objspace/std/floatobject.py > +++ b/pypy/objspace/std/floatobject.py > @@ -58,6 +58,14 @@ > def float_w(self, space): > return self.floatval > > + def int(self, space): > + try: > + value = ovfcheck_float_to_int(self.floatval) > + except OverflowError: > + return space.long(self) > + else: > + return space.newint(value) > + > def __repr__(self): > return "<W_FloatObject(%f)>" % self.floatval > > @@ -89,14 +97,6 @@ > a = w_float1.floatval > return W_FloatObject(a) > > -def int__Float(space, w_value): > - try: > - value = ovfcheck_float_to_int(w_value.floatval) > - except OverflowError: > - return space.long(w_value) > - else: > - return space.newint(value) > - > def long__Float(space, w_floatobj): > try: > return W_LongObject.fromfloat(space, w_floatobj.floatval) > diff --git a/pypy/objspace/std/frame.py b/pypy/objspace/std/frame.py > --- a/pypy/objspace/std/frame.py > +++ b/pypy/objspace/std/frame.py > @@ -6,7 +6,7 @@ > from pypy.interpreter import pyopcode > from pypy.interpreter.pyframe import PyFrame > from pypy.interpreter.error import OperationError > -from pypy.objspace.std import intobject, smallintobject > +from pypy.objspace.std import intobject > from pypy.objspace.std.multimethod import FailedToImplement > from pypy.objspace.std.listobject import W_ListObject > > @@ -23,20 +23,6 @@ > raise AssertionError > > > -def small_int_BINARY_ADD(f, oparg, next_instr): > - w_2 = f.popvalue() > - w_1 = f.popvalue() > - if (type(w_1) is smallintobject.W_SmallIntObject and > - type(w_2) is smallintobject.W_SmallIntObject): > - try: > - w_result = smallintobject.add__SmallInt_SmallInt(f.space, w_1, w_2) > - except FailedToImplement: > - w_result = f.space.add(w_1, w_2) > - else: > - w_result = f.space.add(w_1, w_2) > - f.pushvalue(w_result) > - > - > def int_BINARY_ADD(f, oparg, next_instr): > w_2 = f.popvalue() > w_1 = f.popvalue() > @@ -102,10 +88,7 @@ > class StdObjSpaceFrame(BaseFrame): > pass > if space.config.objspace.std.optimized_int_add: > - if space.config.objspace.std.withsmallint: > - StdObjSpaceFrame.BINARY_ADD = small_int_BINARY_ADD > - else: > - StdObjSpaceFrame.BINARY_ADD = int_BINARY_ADD > + StdObjSpaceFrame.BINARY_ADD = int_BINARY_ADD > if space.config.objspace.std.optimized_list_getitem: > StdObjSpaceFrame.BINARY_SUBSCR = list_BINARY_SUBSCR > if space.config.objspace.opcodes.CALL_METHOD: > diff --git a/pypy/objspace/std/intobject.py b/pypy/objspace/std/intobject.py > --- a/pypy/objspace/std/intobject.py > +++ b/pypy/objspace/std/intobject.py > @@ -1,6 +1,6 @@ > from pypy.interpreter.error import OperationError > from pypy.objspace.std import newformat > -from pypy.objspace.std.inttype import wrapint > +from pypy.objspace.std.inttype import wrapint, W_AbstractIntObject > from pypy.objspace.std.model import registerimplementation, W_Object > from pypy.objspace.std.multimethod import FailedToImplementArgs > from pypy.objspace.std.noneobject import W_NoneObject > @@ -16,25 +16,6 @@ > something CPython does not do anymore. > """ > > -class W_AbstractIntObject(W_Object): > - __slots__ = () > - > - def is_w(self, space, w_other): > - if not isinstance(w_other, W_AbstractIntObject): > - return False > - if self.user_overridden_class or w_other.user_overridden_class: > - return self is w_other > - return space.int_w(self) == space.int_w(w_other) > - > - def immutable_unique_id(self, space): > - if self.user_overridden_class: > - return None > - from pypy.objspace.std.model import IDTAG_INT as tag > - b = space.bigint_w(self) > - b = b.lshift(3).or_(rbigint.fromint(tag)) > - return space.newlong_from_rbigint(b) > - > - > class W_IntObject(W_AbstractIntObject): > __slots__ = 'intval' > _immutable_fields_ = ['intval'] > @@ -67,6 +48,17 @@ > def float_w(self, space): > return float(self.intval) > > + def int(self, space): > + # XXX find a better way to do it > + if (type(self) != W_IntObject and > + space.lookup(self, '__int__') is not > + space.lookup_in_type_where(space.w_int, '__int__')[1]): > + return W_Object.int(self, space) > + if space.is_w(space.type(self), space.w_int): > + return self > + a = self.intval > + return wrapint(space, a) > + > registerimplementation(W_IntObject) > > # NB: This code is shared by smallintobject.py, and thus no other Int > @@ -104,7 +96,7 @@ > # unlike CPython, we don't special-case the value -1 in most of our > # hash functions, so there is not much sense special-casing it here either. > # Make sure this is consistent with the hash of floats and longs. > - return get_integer(space, w_int1) > + return w_int1.int(space) > > # coerce > def coerce__Int_Int(space, w_int1, w_int2): > @@ -251,7 +243,7 @@ > > def abs__Int(space, w_int1): > if w_int1.intval >= 0: > - return get_integer(space, w_int1) > + return w_int1.int(space) > else: > return get_negint(space, w_int1) > > @@ -278,7 +270,7 @@ > space.wrap("negative shift count")) > else: #b >= LONG_BIT > if a == 0: > - return get_integer(space, w_int1) > + return w_int1.int(space) > raise FailedToImplementArgs(space.w_OverflowError, > space.wrap("integer left shift")) > > @@ -291,7 +283,7 @@ > space.wrap("negative shift count")) > else: # b >= LONG_BIT > if a == 0: > - return get_integer(space, w_int1) > + return w_int1.int(space) > if a < 0: > a = -1 > else: > @@ -321,17 +313,13 @@ > # int__Int is supposed to do nothing, unless it has > # a derived integer object, where it should return > # an exact one. > -def int__Int(space, w_int1): > - if space.is_w(space.type(w_int1), space.w_int): > - return w_int1 > - a = w_int1.intval > - return wrapint(space, a) > -get_integer = int__Int > -pos__Int = int__Int > -trunc__Int = int__Int > + > +def pos__Int(self, space): > + return self.int(space) > +trunc__Int = pos__Int > > def index__Int(space, w_int1): > - return get_integer(space, w_int1) > + return w_int1.int(space) > > def float__Int(space, w_int1): > a = w_int1.intval > diff --git a/pypy/objspace/std/inttype.py b/pypy/objspace/std/inttype.py > --- a/pypy/objspace/std/inttype.py > +++ b/pypy/objspace/std/inttype.py > @@ -1,5 +1,6 @@ > from pypy.interpreter import typedef > -from pypy.interpreter.gateway import interp2app, unwrap_spec, WrappedDefault > +from pypy.interpreter.gateway import interp2app, unwrap_spec, WrappedDefault,\ > + interpindirect2app > from pypy.interpreter.error import OperationError, operationerrfmt > from pypy.interpreter.buffer import Buffer > from pypy.objspace.std.register_all import register_all > @@ -7,8 +8,10 @@ > from pypy.objspace.std.strutil import (string_to_int, string_to_bigint, > ParseStringError, > ParseStringOverflowError) > +from pypy.objspace.std.model import W_Object > from rpython.rlib.rarithmetic import r_uint > from rpython.rlib.objectmodel import instantiate > +from rpython.rlib.rbigint import rbigint > > # ____________________________________________________________ > > @@ -36,14 +39,7 @@ > > > def wrapint(space, x): > - if space.config.objspace.std.withsmallint: > - from pypy.objspace.std.smallintobject import W_SmallIntObject > - try: > - return W_SmallIntObject(x) > - except OverflowError: > - from pypy.objspace.std.intobject import W_IntObject > - return W_IntObject(x) > - elif space.config.objspace.std.withprebuiltint: > + if space.config.objspace.std.withprebuiltint: > from pypy.objspace.std.intobject import W_IntObject > lower = space.config.objspace.std.prebuiltintfrom > upper = space.config.objspace.std.prebuiltintto > @@ -185,6 +181,27 @@ > > # ____________________________________________________________ > > +class W_AbstractIntObject(W_Object): > + __slots__ = () > + > + def is_w(self, space, w_other): > + if not isinstance(w_other, W_AbstractIntObject): > + return False > + if self.user_overridden_class or w_other.user_overridden_class: > + return self is w_other > + return space.int_w(self) == space.int_w(w_other) > + > + def immutable_unique_id(self, space): > + if self.user_overridden_class: > + return None > + from pypy.objspace.std.model import IDTAG_INT as tag > + b = space.bigint_w(self) > + b = b.lshift(3).or_(rbigint.fromint(tag)) > + return space.newlong_from_rbigint(b) > + > + def int(self, space): > + raise NotImplementedError > + > int_typedef = StdTypeDef("int", > __doc__ = '''int(x[, base]) -> integer > > @@ -201,5 +218,6 @@ > denominator = typedef.GetSetProperty(descr_get_denominator), > real = typedef.GetSetProperty(descr_get_real), > imag = typedef.GetSetProperty(descr_get_imag), > + __int__ = interpindirect2app(W_AbstractIntObject.int), > ) > int_typedef.registermethods(globals()) > diff --git a/pypy/objspace/std/longobject.py b/pypy/objspace/std/longobject.py > --- a/pypy/objspace/std/longobject.py > +++ b/pypy/objspace/std/longobject.py > @@ -87,6 +87,12 @@ > def float_w(self, space): > return self.num.tofloat() > > + def int(self, space): > + try: > + return space.newint(self.num.toint()) > + except OverflowError: > + return long__Long(space, self) > + > def __repr__(self): > return '<W_LongObject(%d)>' % self.num.tolong() > > @@ -130,12 +136,6 @@ > def long__Int(space, w_intobj): > return space.newlong(w_intobj.intval) > > -def int__Long(space, w_value): > - try: > - return space.newint(w_value.num.toint()) > - except OverflowError: > - return long__Long(space, w_value) > - > def index__Long(space, w_value): > return long__Long(space, w_value) > > diff --git a/pypy/objspace/std/model.py b/pypy/objspace/std/model.py > --- a/pypy/objspace/std/model.py > +++ b/pypy/objspace/std/model.py > @@ -17,7 +17,6 @@ > option_to_typename = { > "withspecialisedtuple" : ["specialisedtupleobject.W_SpecialisedTupleObject"], > "withsmalltuple" : ["smalltupleobject.W_SmallTupleObject"], > - "withsmallint" : ["smallintobject.W_SmallIntObject"], > "withsmalllong" : ["smalllongobject.W_SmallLongObject"], > "withstrbuf" : ["strbufobject.W_StringBufferObject"], > } > @@ -158,18 +157,6 @@ > # when trying to dispatch multimethods. > # XXX build these lists a bit more automatically later > > - if config.objspace.std.withsmallint: > - from pypy.objspace.std import smallintobject > - self.typeorder[boolobject.W_BoolObject] += [ > - (smallintobject.W_SmallIntObject, boolobject.delegate_Bool2SmallInt), > - ] > - self.typeorder[smallintobject.W_SmallIntObject] += [ > - (intobject.W_IntObject, smallintobject.delegate_SmallInt2Int), > - (floatobject.W_FloatObject, smallintobject.delegate_SmallInt2Float), > - (longobject.W_LongObject, smallintobject.delegate_SmallInt2Long), > - (complexobject.W_ComplexObject, smallintobject.delegate_SmallInt2Complex), > - ] > - > if config.objspace.usemodules.micronumpy: > from pypy.module.micronumpy.stdobjspace import register_delegates > register_delegates(self.typeorder) > @@ -424,7 +411,7 @@ > ['delattr', 'delete', 'get', 'id', 'inplace_div', 'inplace_floordiv', > 'inplace_lshift', 'inplace_mod', 'inplace_pow', 'inplace_rshift', > 'inplace_truediv', 'is_', 'set', 'setattr', 'type', 'userdel', > - 'isinstance', 'issubtype']) > + 'isinstance', 'issubtype', 'int']) > # XXX should we just remove those from the method table or we're happy > # with just not having multimethods? > > @@ -442,7 +429,7 @@ > > # add all regular multimethods here > for _name, _symbol, _arity, _specialnames in ObjSpace.MethodTable: > - if _name not in locals() or _name in NOT_MULTIMETHODS: > + if _name not in locals() and _name not in NOT_MULTIMETHODS: > mm = StdObjSpaceMultiMethod(_symbol, _arity, _specialnames) > locals()[_name] = mm > del mm > diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py > --- a/pypy/objspace/std/objspace.py > +++ b/pypy/objspace/std/objspace.py > @@ -26,7 +26,6 @@ > from pypy.objspace.std.iterobject import W_SeqIterObject > from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject > from pypy.objspace.std.sliceobject import W_SliceObject > -from pypy.objspace.std.smallintobject import W_SmallIntObject > from pypy.objspace.std.stringobject import W_StringObject > from pypy.objspace.std.unicodeobject import W_UnicodeObject > from pypy.objspace.std.tupleobject import W_AbstractTupleObject > @@ -580,16 +579,8 @@ > self.setitem(w_obj, self.wrap(key), w_value) > > def getindex_w(self, w_obj, w_exception, objdescr=None): > - # Performance shortcut for the common case of w_obj being an int. > - # If withsmallint is disabled, we check for W_IntObject. > - # If withsmallint is enabled, we only check for W_SmallIntObject - it's > - # probably not useful to have a shortcut for W_IntObject at all then. > - if self.config.objspace.std.withsmallint: > - if type(w_obj) is W_SmallIntObject: > - return w_obj.intval > - else: > - if type(w_obj) is W_IntObject: > - return w_obj.intval > + if type(w_obj) is W_IntObject: > + return w_obj.intval > return ObjSpace.getindex_w(self, w_obj, w_exception, objdescr) > > def call_method(self, w_obj, methname, *arg_w): > diff --git a/pypy/objspace/std/smallintobject.py b/pypy/objspace/std/smallintobject.py > deleted file mode 100644 > --- a/pypy/objspace/std/smallintobject.py > +++ /dev/null > @@ -1,87 +0,0 @@ > -""" > -Implementation of small ints, stored as odd-valued pointers in the > -translated PyPy. To enable them, see inttype.py. > -""" > -from pypy.objspace.std import intobject > -from pypy.objspace.std.model import registerimplementation, W_Object > -from pypy.objspace.std.register_all import register_all > -from pypy.objspace.std.noneobject import W_NoneObject > -from pypy.objspace.std.intobject import W_AbstractIntObject, W_IntObject > -from pypy.interpreter.error import OperationError > -from rpython.rlib.objectmodel import UnboxedValue > -from rpython.rlib.rbigint import rbigint > -from rpython.rlib.rarithmetic import r_uint > -from rpython.tool.sourcetools import func_with_new_name > -from pypy.objspace.std.inttype import wrapint > - > -class W_SmallIntObject(W_AbstractIntObject, UnboxedValue): > - __slots__ = 'intval' > - from pypy.objspace.std.inttype import int_typedef as typedef > - > - def unwrap(w_self, space): > - return int(w_self.intval) > - int_w = unwrap > - > - def uint_w(w_self, space): > - intval = w_self.intval > - if intval < 0: > - raise OperationError(space.w_ValueError, > - space.wrap("cannot convert negative integer to unsigned")) > - else: > - return r_uint(intval) > - > - def bigint_w(w_self, space): > - return rbigint.fromint(w_self.intval) > - > - > -registerimplementation(W_SmallIntObject) > - > - > -def delegate_SmallInt2Int(space, w_small): > - return W_IntObject(w_small.intval) > - > -def delegate_SmallInt2Long(space, w_small): > - return space.newlong(w_small.intval) > - > -def delegate_SmallInt2Float(space, w_small): > - return space.newfloat(float(w_small.intval)) > - > -def delegate_SmallInt2Complex(space, w_small): > - return space.newcomplex(float(w_small.intval), 0.0) > - > -def add__SmallInt_SmallInt(space, w_a, w_b): > - return wrapint(space, w_a.intval + w_b.intval) # cannot overflow > - > -def sub__SmallInt_SmallInt(space, w_a, w_b): > - return wrapint(space, w_a.intval - w_b.intval) # cannot overflow > - > -def floordiv__SmallInt_SmallInt(space, w_a, w_b): > - return wrapint(space, w_a.intval // w_b.intval) # cannot overflow > - > -div__SmallInt_SmallInt = floordiv__SmallInt_SmallInt > - > -def mod__SmallInt_SmallInt(space, w_a, w_b): > - return wrapint(space, w_a.intval % w_b.intval) # cannot overflow > - > -def divmod__SmallInt_SmallInt(space, w_a, w_b): > - w = wrapint(space, w_a.intval // w_b.intval) # cannot overflow > - z = wrapint(space, w_a.intval % w_b.intval) > - return space.newtuple([w, z]) > - > -def copy_multimethods(ns): > - """Copy integer multimethods for small int.""" > - for name, func in intobject.__dict__.iteritems(): > - if "__Int" in name: > - new_name = name.replace("Int", "SmallInt") > - if new_name not in ns: > - # Copy the function, so the annotator specializes it for > - # W_SmallIntObject. > - ns[new_name] = func = func_with_new_name(func, new_name, globals=ns) > - else: > - ns[name] = func > - ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"] > - ns["get_negint"] = ns["neg__SmallInt"] > - > -copy_multimethods(globals()) > - > -register_all(vars()) > diff --git a/pypy/objspace/std/smalllongobject.py b/pypy/objspace/std/smalllongobject.py > --- a/pypy/objspace/std/smalllongobject.py > +++ b/pypy/objspace/std/smalllongobject.py > @@ -69,7 +69,15 @@ > def float_w(self, space): > return float(self.longlong) > > + def int(self, space): > + a = self.longlong > + b = intmask(a) > + if b == a: > + return space.newint(b) > + else: > + return self > > + > registerimplementation(W_SmallLongObject) > > # ____________________________________________________________ > @@ -119,14 +127,6 @@ > def long__SmallLong(space, w_value): > return w_value > > -def int__SmallLong(space, w_value): > - a = w_value.longlong > - b = intmask(a) > - if b == a: > - return space.newint(b) > - else: > - return w_value > - > def index__SmallLong(space, w_value): > return w_value > > diff --git a/pypy/objspace/std/test/test_intobject.py b/pypy/objspace/std/test/test_intobject.py > --- a/pypy/objspace/std/test/test_intobject.py > +++ b/pypy/objspace/std/test/test_intobject.py > @@ -268,7 +268,7 @@ > > def test_int(self): > f1 = iobj.W_IntObject(1) > - result = iobj.int__Int(self.space, f1) > + result = f1.int(self.space) > assert result == f1 > > def test_oct(self): > diff --git a/pypy/objspace/std/test/test_smallintobject.py b/pypy/objspace/std/test/test_smallintobject.py > deleted file mode 100644 > --- a/pypy/objspace/std/test/test_smallintobject.py > +++ /dev/null > @@ -1,229 +0,0 @@ > -import sys, py > - > -#from pypy.objspace.std.model import WITHSMALLINT > -#if not WITHSMALLINT: > -# py.test.skip("WITHSMALLINT is not enabled") > - > -from pypy.objspace.std.inttype import wrapint > -from pypy.objspace.std.multimethod import FailedToImplement > -from rpython.rlib.rarithmetic import r_uint > - > -from pypy.objspace.std.test.test_intobject import AppTestInt > - > -class TestW_IntObject: > - spaceconfig = {"objspace.std.withsmallint": True} > - > - def test_int_w(self): > - assert self.space.int_w(self.space.wrap(42)) == 42 > - > - def test_uint_w(self): > - space = self.space > - assert space.uint_w(space.wrap(42)) == 42 > - assert isinstance(space.uint_w(space.wrap(42)), r_uint) > - space.raises_w(space.w_ValueError, space.uint_w, space.wrap(-1)) > - > - def test_repr(self): > - x = 1 > - f1 = wrapint(self.space, x) > - result = self.space.repr(f1) > - assert self.space.unwrap(result) == repr(x) > - > - def test_str(self): > - x = 12345 > - f1 = wrapint(self.space, x) > - result = self.space.str(f1) > - assert self.space.unwrap(result) == str(x) > - > - def test_hash(self): > - x = 42 > - f1 = wrapint(self.space, x) > - result = self.space.hash(f1) > - assert result.intval == hash(x) > - > - def test_compare(self): > - import operator > - optab = ['lt', 'le', 'eq', 'ne', 'gt', 'ge'] > - for x in (-10, -1, 0, 1, 2, 1000, sys.maxint): > - for y in (-sys.maxint-1, -11, -9, -2, 0, 1, 3, 1111, sys.maxint): > - for op in optab: > - wx = wrapint(self.space, x) > - wy = wrapint(self.space, y) > - res = getattr(operator, op)(x, y) > - method = getattr(self.space, op) > - myres = method(wx, wy) > - assert self.space.unwrap(myres) == res > - > - def test_add(self): > - for x in [1, 100, sys.maxint // 2 - 50, > - sys.maxint // 2, sys.maxint - 1000, sys.maxint]: > - for y in [1, 100, sys.maxint // 2 - 50, > - sys.maxint // 2, sys.maxint - 1000, sys.maxint]: > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - result = self.space.unwrap(self.space.add(f1, f2)) > - assert result == x+y > - > - def test_sub(self): > - for x in [1, 100, sys.maxint // 2 - 50, > - sys.maxint // 2, sys.maxint - 1000, sys.maxint]: > - for y in [1, 100, sys.maxint // 2 - 50, > - sys.maxint // 2, sys.maxint - 1000, sys.maxint]: > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - result = self.space.unwrap(self.space.sub(f1, f2)) > - assert result == x-y > - > - def test_mul(self): > - for x in [0, 1, 100, sys.maxint // 2 - 50, sys.maxint - 1000]: > - for y in [0, 1, 100, sys.maxint // 2 - 50, sys.maxint - 1000]: > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - result = self.space.unwrap(self.space.mul(f1, f2)) > - assert result == x*y > - > - > - def test_div(self): > - for i in range(10): > - res = i//3 > - f1 = wrapint(self.space, i) > - f2 = wrapint(self.space, 3) > - result = self.space.div(f1, f2) > - assert result.intval == res > - > - def test_mod(self): > - x = 1 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.mod(f1, f2) > - assert v.intval == x % y > - # not that mod cannot overflow > - > - def test_divmod(self): > - x = 1 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - ret = self.space.divmod(f1, f2) > - v, w = self.space.unwrap(ret) > - assert (v, w) == divmod(x, y) > - > - def test_pow_iii(self): > - x = 10 > - y = 2 > - z = 13 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - f3 = wrapint(self.space, z) > - v = self.space.pow(f1, f2, f3) > - assert v.intval == pow(x, y, z) > - f1, f2, f3 = [wrapint(self.space, i) for i in (10, -1, 42)] > - self.space.raises_w(self.space.w_TypeError, > - self.space.pow, > - f1, f2, f3) > - f1, f2, f3 = [wrapint(self.space, i) for i in (10, 5, 0)] > - self.space.raises_w(self.space.w_ValueError, > - self.space.pow, > - f1, f2, f3) > - > - def test_pow_iin(self): > - x = 10 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.pow(f1, f2, self.space.w_None) > - assert v.intval == x ** y > - > - def test_neg(self): > - x = 42 > - f1 = wrapint(self.space, x) > - v = self.space.neg(f1) > - assert v.intval == -x > - > - def test_pos(self): > - x = 42 > - f1 = wrapint(self.space, x) > - v = self.space.pos(f1) > - assert v.intval == +x > - x = -42 > - f1 = wrapint(self.space, x) > - v = self.space.pos(f1) > - assert v.intval == +x > - > - def test_abs(self): > - x = 42 > - f1 = wrapint(self.space, x) > - v = self.space.abs(f1) > - assert v.intval == abs(x) > - x = -42 > - f1 = wrapint(self.space, x) > - v = self.space.abs(f1) > - assert v.intval == abs(x) > - > - def test_invert(self): > - x = 42 > - f1 = wrapint(self.space, x) > - v = self.space.invert(f1) > - assert v.intval == ~x > - > - def test_lshift(self): > - x = 12345678 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.lshift(f1, f2) > - assert v.intval == x << y > - > - def test_rshift(self): > - x = 12345678 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.rshift(f1, f2) > - assert v.intval == x >> y > - > - def test_and(self): > - x = 12345678 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.and_(f1, f2) > - assert v.intval == x & y > - > - def test_xor(self): > - x = 12345678 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.xor(f1, f2) > - assert v.intval == x ^ y > - > - def test_or(self): > - x = 12345678 > - y = 2 > - f1 = wrapint(self.space, x) > - f2 = wrapint(self.space, y) > - v = self.space.or_(f1, f2) > - assert v.intval == x | y > - > - def test_int(self): > - f1 = wrapint(self.space, 1) > - result = self.space.int(f1) > - assert result == f1 > - > - def test_oct(self): > - x = 012345 > - f1 = wrapint(self.space, x) > - result = self.space.oct(f1) > - assert self.space.unwrap(result) == oct(x) > - > - def test_hex(self): > - x = 0x12345 > - f1 = wrapint(self.space, x) > - result = self.space.hex(f1) > - assert self.space.unwrap(result) == hex(x) > - > - > -class AppTestSmallInt(AppTestInt): > - spaceconfig = {"objspace.std.optimized_int_add" : True, > - "objspace.std.withsmallint" : True} > diff --git a/pypy/objspace/std/typeobject.py b/pypy/objspace/std/typeobject.py > --- a/pypy/objspace/std/typeobject.py > +++ b/pypy/objspace/std/typeobject.py > @@ -857,12 +857,6 @@ > def _pure_issubtype(w_sub, w_type, version_tag1, version_tag2): > return _issubtype(w_sub, w_type) > > -def issubtype__Type_Type(space, w_type, w_sub): > - return space.newbool(w_sub.issubtype(w_type)) > - > -def isinstance__Type_ANY(space, w_type, w_inst): > - return space.newbool(space.type(w_inst).issubtype(w_type)) > - > def repr__Type(space, w_obj): > w_mod = w_obj.get_module() > if not space.isinstance_w(w_mod, space.w_str): > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > http://mail.python.org/mailman/listinfo/pypy-commit >

2 1

Memory corruption on socket.gethostbyname_ex
by Vasily Evseenko March 22, 2013

March 22, 2013

Hi, There are some memory corruption while calling socket.gethostbyname_ex from the thread while other thread executes JIT'ed code: In such case socket.gethostbyname_ex returns random entries from /etc/hosts instead of DNS lookup. ------ import sys import socket import threading domain = 'google.com' def lookup(x): res = socket.gethostbyname_ex(x) if res[0] != 'google.com': print 'BUG: resolve %s -> %s' % (x, res) sys.stdout.flush() def test(): # do some JIT for i in xrange(1000000000): pass for i in range(100): th = threading.Thread(target=lookup, args=[domain]) th.start() th = threading.Thread(target=test) th.start() print 'startup done' ------ Affected versions: PyPy 1.8, 1.9, 2.0b1 on Linux (Fedora 16, Debian GNU/Linux 6.0) --- Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Mar 19 2013, 07:39:19) [PyPy 2.0.0-beta1 with GCC 4.4.5] on linux2 Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Dec 18 2012, 05:33:54) [PyPy 2.0.0-beta1 with GCC 4.6.3] on linux2 Python 2.7.2 (2346207d99463f299f09f3e151c9d5fa9158f71b, May 15 2012, 11:08:08) [PyPy 1.8.0 with GCC 4.6.3] on linux2 Python 2.7.2 (341e1e3821fff77db3bb5cdb7a4851626298c44e, Oct 23 2012, 11:37:42) [PyPy 1.9.0 with GCC 4.6.3] on linux2 --- Not affected: PyPy 2.0b1 on MacOSX --- Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Mar 19 2013, 07:52:27) [PyPy 2.0.0-beta1] on darwin Python 2.7.3 (7e4f0faa3d51, Nov 21 2012, 15:59:46) [PyPy 2.0.0-beta1 with GCC 4.2.1] on darwin ---

2 1

strange difference between py3k and default on win32 build
by matti picus March 22, 2013

March 22, 2013

I tried kicking the new windows buildbot on py3k branch, and got a linking error about missing zlib in openssl http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/897/steps/tr… but I couldn't find any relevant diff between the two branches that would cause that. Before I commit a random fix to the problem I would like to understand why. Anyone have any ideas? Matti

2 1

why doesn't buildbot master sort (after I told it to)
by matti picus March 22, 2013

March 22, 2013

I know this is not all that important, but... It annoyed me that there is so much stuff on this page http://buildbot.pypy.org/nightly so I changed our buildbot code to sort by filesystem mtime, and put trunk on top. I tried it out by writing tests, and even installing a debug buildbot, and creating some directories in my local ~/nightly It works locally but not on buildbot. Any ideas why? Matti

3 3

frame.f_trace role
by anatoly techtonik March 21, 2013

March 21, 2013

Hi, I am puzzled. The documentation mentions writable .f_trace attribute for a frame. "if not None, is a function called at the start of each source code line" http://docs.python.org/2/reference/datamodel.html#types But there is also a trace function set by sys.settrace with 'line' event http://docs.python.org/2/library/sys.html#sys.settrace What's the difference? Why the f_trace is needed? It is also unclear who sets f_trace, because it should be set at the beginning of each frame before the frame starts executing. Why do I need it? I wanted to make a scroller for a live Python code long ago, and now that I am deep inside Python frames (thanks to xtrace bugreport I got earlier) it sounds like a good time for it. The only concern I have is if 'line' event will be enough to cover all the code that Python executes in settrace set function. -- anatoly t.

2 1

PyCon videos are up
by Joe Hillenbrand March 21, 2013

March 21, 2013

For those on the list who have not seen Armin and Maciej's talk yet: http://www.pyvideo.org/video/1739/pypy-without-the-gil -Joe

1 0

rsdl
by Carl Friedrich Bolz March 20, 2013

March 20, 2013

Hi all, is anybody against moving rsdl back from lang-gameboy to rlib? The Smalltalk implementation is starting to use it too, and it's a bit awkward to have to check out lang-gameboy as well. The disadvantage is that right now the rsdl tests are a bit silly (require human interaction). Cheers, Carl Friedrich

3 5

Re: [pypy-dev] rsdl
by Tim Felgentreff March 20, 2013

March 20, 2013

The tests that require interaction are skipped if no interactive shell is attached to stdin (i.e. unless you pass -s to py.test) -Tim *From:* Ronny Pfannschmidt <Ronny.Pfannschmidt(a)gmx.de> *Sent:* 20 March 2013 11:34 *To:* Carl Friedrich Bolz <cfbolz(a)gmx.de> *CC:* PyPy Developer Mailing List <pypy-dev(a)python.org> *Subject:* Re: [pypy-dev] rsdl Hi Carl, we could mark those tests as needing interaction and skip them if some option is not given -- Ronny On 03/20/2013 10:40 AM, Carl Friedrich Bolz wrote: Hi all, is anybody against moving rsdl back from lang-gameboy to rlib? The Smalltalk implementation is starting to use it too, and it's a bit awkward to have to check out lang-gameboy as well. The disadvantage is that right now the rsdl tests are a bit silly (require human interaction). Cheers, Carl Friedrich ______________________________**_________________ pypy-dev mailing list pypy-dev(a)python.org http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev> ______________________________**_________________ pypy-dev mailing list pypy-dev(a)python.org http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>

1 0

Re: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
by Christian Rafael March 19, 2013

March 19, 2013

Olá Sr.David, Sugiro que eu elabore um documento com todas as especificações e partir dele discutamos os detalhes. Abaixo algumas informações necessárias: 1º - Preciso que o senhor me apresente o plano de negócios do PyPy ou algum outro documento. 2º - Devo enviar os próximos emails para pypy-dev(a)python.org? Atenciosamente, Christian Rafael - Paradix Corporation. christian(a)paradix.com.br www.paradix.com.br > Date: Mon, 18 Mar 2013 17:34:35 -0400 > Subject: Re: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br) > From: dje.gcc(a)gmail.com > To: christian(a)paradix.com.br > CC: andrewsmedina(a)gmail.com > > Ola Sr. Christian, > > Como você sugere que nos discutamos o plano e sua abordagem para a > promoção do PyPy? > > Obrigado, David > > 2013/3/14 Christian Rafael <christian(a)paradix.com.br>: > > Olá Sr.David, > > > > Para mim também é indiferente, não tinha pensado na possibilidade de um > > subdomínio br.pypy.org seria excelente. > > Também tenho disponibilidade para estar fazendo apresentações em Escolas > > Técnicas e Faculdades, mas tudo isso deve ser devidamente formalizado para > > que tudo seja de acordo com o plano de negócios do senhores e não ocorra > > divergências. > > > > > > Grato, > > Christian Rafael - Paradix Corporation. > > christian(a)paradix.com.br > > www.paradix.com.br > > > >> Date: Wed, 13 Mar 2013 20:15:10 -0400 > >> Subject: Re: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br) > >> From: dje.gcc(a)gmail.com > >> To: andrewsmedina(a)gmail.com; christian(a)paradix.com.br > >> CC: pypy-dev(a)python.org > > > >> > >> 2013/3/13 Andrews Medina <andrewsmedina(a)gmail.com>: > >> > Olá David, > >> > > >> > 2013/3/13 David Edelsohn <dje.gcc(a)gmail.com>: > >> >> > >> >> A comunidade PyPy gostaria de sua ajuda com a tradução e para > >> >> encorajar os desenvolvedores latino-americanos para contribuir para o > >> >> PyPy. Um novo domínio exige a aprovação da comunidade PyPy. Você > >> >> acredita que os desenvolvedores latino-americanos preferem interagir > >> >> com um domínio no BR domínio nível superior ao invés de páginas > >> >> portuguesas e espanholas em pypy.org? Ou br.pypy.org? > >> > > >> > Eu sou brasileiro também e acho indiferente/desnecessário um novo > >> > site/dominio. > >> > >> Nós definitivamente queremos incentivar isso - um domínio separado ou > >> um domínio / páginas dentro pypy.org. Estamos discutindo os detalhes. > >> PyCon US está em andamento, por isso todos estão ocupados. Por favor, > >> tenham paciência. > >> > >> Obrigado, > >> David

1 0