CPython 2.7: Weakset data changing size during internal iteration

[Terry Reedy]

On 6/1/2012 11:23 AM, Temia Eszteri wrote:
> I've got a bit of a problem - my project uses weak sets in multiple
> areas, the problem case in particular being to indicate what objects
> are using a particular texture, if any, so that its priority in OpenGL
> can be adjusted to match at the same time as it being (de)referenced
> by any explicit calls.
>
> Problem is that for certain high-frequency operations, it seems
> there's too much data going in and out for it to handle - the
> following traceback is given to me (project path changed to protect
> the innocent):
>
> Traceback (most recent call last):
>    File "C:\foo\bar\game.py", line 279, in update
>      self.player.update()
>    File "C:\foo\bar\player.py", line 87, in update
>      PlayerBullet((self.x + 8, self.y + 9), 0, self.parent)
>    File "C:\foo\bar\player.py", line 96, in __init__
>      self.sprite = video.Sprite("testbullet", 0)
>    File "C:\foo\bar\video.py", line 95, in __init__
>      self.opengl_id = reference_texture(self, target)
>    File "C:\foo\bar\video.py", line 310, in reference_texture
>      if not video_handler.textures[target].references:

I gather that the .references attribute is sometimes/always a weakset. 
To determine its boolean value, it computes its length. For regular 
sets, this is sensible as .__len__() returns a pre-computed value.

>    File "C:\Python27\lib\_weakrefset.py", line 66, in __len__
>      return sum(x() is not None for x in self.data)

Given that len(weakset) is defined (sensibly) as the number of currently 
active members, it must count. weakset should really have .__bool__ 
method that uses any() instead of sum(). That might reduce, but not 
necessarily eliminate your problem.

>    File "C:\Python27\lib\_weakrefset.py", line 66, in<genexpr>
>      return sum(x() is not None for x in self.data)
> RuntimeError: Set changed size during iteration

I can think of two reasons:

1. You are using multiple threads and another thread does something to 
change the size of the set during the iteration. Solution? put a lock 
around the if-statement so no other thread can change self.data during 
the iteration.

2. Weakset members remove themselves from the set before returning None. 
(Just a thought, in case you are not using threads).

-- 
Terry Jan Reedy