Hi Frank, On 20 December 2016 at 01:29, Frank Wang <frankw@mit.edu> wrote:

This also seems to happen in a lot of classes because W_Root is the parent class for many classes. Is there a way to universally turn off this immutable flag (assuming it's okay) in all classes?

You can hack at the place where the error is raised. Assuming you're translating without the JIT, it should not cause too much problems. As an alternative to adding an attribute to every object, you could also use a rpython.rlib.rweakref.RWeakKeyDictionary(W_Root, RbFlags), for some class RbFlags which has got the rb_flags dictionary and/or any other related data. The advantage is that it is a less intrusive change, reduces memory usage if many objects don't need the extra information at all, and avoids the immutable problem. The inconvenient is that going from the object to the rb_flags dictionary is slower. There are also alternatives that are better if you are ok with supporting not *all* objects. For example, you can edit objspace/std/mapdict.py to add a new dynamic attribute that is internally called ``"rbflags", SPECIAL``; see ``"weakref", SPECIAL`` for an existing example. This would work for all objects that use mapdicts, i.e. all instances of user-defined classes, and additionally instances of many built-in types as well---but not instances of the core types like "int" or "str" or "list" or "tuple" or "dict". A bientôt, Armin.

Hi Armin, I've taken your advice, and my W_Root has these additions (in bold): class W_Root(object): _attrs_ = ['__weakref__', *'rb_flags'*] _must_be_light_finalizer_ = True user_overridden_class = False # Riverbed flags * rb_flags = None* * def add_rbflags(self, new_flags):* * if self.rb_flags is None:* * self.rb_flags = {}* * for flag in new_flags:* * self.rb_flags[flag] = None* * def set_rbflags(self, new_flags):* * self.rb_flags = new_flags* * def get_rbflags(self):* * if self.rb_flags is None:* * self.rb_flags = {}* * return self.rb_flags* In pyopcode.py, I've made the following modifications. I've shown them in bold. The high level idea is that for a binary operation, I want to take the union of the rb_flags in w_1 and w_2 and set the union to be the rb_flags in w_result. def binaryoperation(operationname): """NOT_RPYTHON""" def opimpl(self, *ignored): operation = getattr(self.space, operationname) w_2 = self.popvalue() w_1 = self.popvalue() w_result = operation(w_1, w_2) # union flags in w_1 and w_2 and propagate to result * w_1_rbflags = w_1.get_rbflags()* * w_2_rbflags = w_2.get_rbflags()* * w_1_rbflags.update(w_2_rbflags)* * w_result.set_rbflags(w_1_rbflags)* self.pushvalue(w_result) opimpl.binop = operationname return func_with_new_name(opimpl, "opcode_impl_for_%s" % operationname) However, when I do this, I get the following error: [translation:ERROR] UnionError: Offending annotations: SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>) SomeInstance(can_be_None=True, classdef=pypy.interpreter.baseobjspace.W_Root) v945 = setattr(self_268, ('rb_flags'), v944) In <FunctionGraph of (pypy.interpreter.baseobjspace:46)W_Root.get_rbflags at 0x40f26de8>: Happened at file /home/ubuntu/pypy2-v5.3.1-src/pypy/interpreter/baseobjspace.py line 48 ==> self.rb_flags = {} Known variable annotations: self_268 = SomeInstance(can_be_None=False, classdef=pypy.interpreter.baseobjspace.W_Root) v944 = SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>) Processing block: block@15 is a <class 'rpython.flowspace.flowcontext.SpamBlock'> in (pypy.interpreter.baseobjspace:46)W_Root.get_rbflags containing the following operations: v944 = newdict() v945 = setattr(self_268, ('rb_flags'), v944) --end-- Any thoughts on this? Frank On Mon, Dec 19, 2016 at 10:32 PM, Frank Wang <frankw@mit.edu> wrote:

Hi Armin,

It seems that with your suggestions, things have started to work!

Thanks again for the help!

On Mon, Dec 19, 2016 at 5:57 PM Armin Rigo <armin.rigo@gmail.com> wrote:

...

Hi Frank,

...

This also seems to happen in a lot of classes because W_Root is the

On 20 December 2016 at 01:29, Frank Wang <frankw@mit.edu> wrote: parent

...

class for many classes. Is there a way to universally turn off this immutable flag (assuming it's okay) in all classes?

You can hack at the place where the error is raised. Assuming you're translating without the JIT, it should not cause too much problems.

As an alternative to adding an attribute to every object, you could also use a rpython.rlib.rweakref.RWeakKeyDictionary(W_Root, RbFlags), for some class RbFlags which has got the rb_flags dictionary and/or any other related data. The advantage is that it is a less intrusive change, reduces memory usage if many objects don't need the extra information at all, and avoids the immutable problem. The inconvenient is that going from the object to the rb_flags dictionary is slower.

There are also alternatives that are better if you are ok with supporting not *all* objects. For example, you can edit objspace/std/mapdict.py to add a new dynamic attribute that is internally called ``"rbflags", SPECIAL``; see ``"weakref", SPECIAL`` for an existing example. This would work for all objects that use mapdicts, i.e. all instances of user-defined classes, and additionally instances of many built-in types as well---but not instances of the core types like "int" or "str" or "list" or "tuple" or "dict".

A bientôt,

Armin.

Hi William, Good catch! I was just trying to get something to work, but it seems like you found another bug. I am now checking if it is None: # union flags in w_1 and w_2 and propagate to result if isinstance(w_result, W_Root) and w_result is not None: rb_results = {} if isinstance(w_1, W_Root) and w_1 is not None: w_1_rbflags = w_1.get_rbflags() rb_results.update(w_1_rbflags) if isinstance(w_2, W_Root) and w_2 is not None: w_2_rbflags = w_2.get_rbflags() rb_results.update(w_2_rbflags) w_result.set_rbflags(rb_results) I now get this error: [translation:ERROR] UnionError: Offending annotations: SomeInstance(can_be_None=False, classdef=pypy.interpreter.baseobjspace.W_Root) SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>) Occurred processing the following simple_call: <MethodDesc 'set_rbflags' of <ClassDef 'pypy.interpreter.baseobjspace.W_Root'> bound to <ClassDef 'pypy.interpreter.baseobjspace.W_Root'> {}> returning v831 = simple_call(v830, rb_results_0) In <FunctionGraph of (pypy.interpreter.pyopcode:40)opcode_impl_for_div__star_2 at 0x1c190b10>: Happened at file /home/ubuntu/pypy2-v5.3.1-src/pypy/interpreter/pyopcode.py line 55 ==> w_result.set_rbflags(rb_results) Known variable annotations: v830 = SomePBC(can_be_None=False, descriptions={...1...}, knowntype=method, subset_of=None) rb_results_0 = SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>) Processing block: block@203 is a <class 'rpython.flowspace.flowcontext.SpamBlock'> in (pypy.interpreter.pyopcode:40)opcode_impl_for_div__star_2 containing the following operations: v830 = getattr(w_result_0, ('set_rbflags')) v831 = simple_call(v830, rb_results_0) --end-- I assume this happens because w_result can be a "subset" of None. What is the proper way to check if it's None? Frank On Wed, Jan 11, 2017 at 3:00 PM, William ML Leslie < william.leslie.ttg@gmail.com> wrote:

On 12 January 2017 at 09:08, Frank Wang <frankw@mit.edu> wrote:

...

def binaryoperation(operationname): """NOT_RPYTHON""" def opimpl(self, *ignored): operation = getattr(self.space, operationname) w_2 = self.popvalue() w_1 = self.popvalue() w_result = operation(w_1, w_2)

# union flags in w_1 and w_2 and propagate to result w_1_rbflags = w_1.get_rbflags() w_2_rbflags = w_2.get_rbflags() w_1_rbflags.update(w_2_rbflags) w_result.set_rbflags(w_1_rbflags)

self.pushvalue(w_result)

It looks like, if an app-level exception is in progress, w_result may be None. Try checking for that before setting the flags.

Also: did you mean to alter w_1's rbflags? Seems strange that you'd alter w_1 but not w_2.

-- William Leslie

Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.

Re-hi, On 13 January 2017 at 12:34, Armin Rigo <armin.rigo@gmail.com> wrote:

says: you are storing in the rb_flags attribute something that is sometimes a dict, and sometimes an instance of W_Root.

Another potential reason would be that there is one buggy call somewhere that calls .set_rbflags(x) where x is an instance of W_Root instead of a dictionary. A bientôt, Armin.