[Python-ideas] easy thread-safety [was: fork]

[Ron Adam]

On 08/20/2015 12:51 PM, Steven D'Aprano wrote:
> On Thu, Aug 20, 2015 at 12:27:55PM -0400, Ron Adam wrote:
>
>> >When a bytecode to load an object is executed such as LOAD_FAST, it gets
>> >it's reference to the object from the function's list of names in it's
>> >code object.
> Bytes codes are implementation, not semantics: there is no part of the
> Python language that promises that retrieving local variables will use a
> byte code LOAD_FAST. That's not how IronPython or Jython work, and there
> is no stability guarantee that CPython will always use LOAD_FAST.

Yes, but semantics needs a workable implementation at some point.

The semantics is to have a way to make names (for mutable objects) in 
outer scopes not be visible to function defined in inner scopes.

   def foo(x):
       """ Example of hiding a mutable object from inner scopes. """
       mutable items
       items = [1, 2, 3]
       def bar(y):
       """ can't see items here. So can't mutate it."""
           return -y
       return [bar(y)+x for y in items]

So foo is able to protect items from being mutated by functions defined 
in it's scope.   We could use localonly instead of mutable, but in the 
context of threading mutable may be more appropriate. (Way too soon to 
decide what color to paint this bike.)

It may seem like it isn't needed, because you have control over what a 
function has access too... ie... just don't do that.  But when you have 
many programmers working on large projects, things can get messy.  And 
this helps with that, but also helps in the case of threads.


>> >LOAD_FAST 0,  reads __code__.co_varnames[0]
>> >LOAD_FAST 1,  reads __code__.co_varnames[1]
>> >
>> >Adding a co_mutables name list to the __code__ attribute, along with new
>> >bytecodes to access them, it would create a way to keep private local
>> >names without changing how the other bytecodes work.
>> >
>> >LOAD_MUTABLE 0,  would get the first reference in __code__.co_mutables.

> What difference would it make to have a different implemention for
> retrieving and setting the object? How will having this byte code "keep
> private local names"?

If someone wanted to abuse it, they could.  But that is true for many 
other areas of Python.

Just as declaring a name as global or nonlocal changes which co_xxxx 
attribute a name reference is in, declaring it mutable would do the 
same.  And just as the compiler generates different bytecode for global 
and nonlocal, it would generate different bytecode in this case too. 
That bytecode, LOAD_MUTABLE, (and STORE_MUTABLE) would always look in 
the co_mutables list for it's references.  Just as the other bytecodes 
look in certain lists for their references.   So it using an 
implementation consistent with how the other names are referenced.  The 
important point of that is the name *won't* be in co_names, 
co_freenames, or co_cellvars.

While it may be possible to do without using a new name list in 
__code__, that would require keeping track of which names are local 
only, and which one are local but visible in the scope for functions 
defined under that function, in some other way.

As I said it's a partial solution.  Shared & mutable names passed as 
function arguments will still need to be protected   But I thought the 
idea was interesting enough to post.by locks or some other means in 
threads, but they will be easy to identify and other mutable objects 
that are only used locally in a function won't need additional 
protections as they won't be visible outside of that function.  (without 
abusing the code object, or introspecting it.)

To describe this further would probably require me to actually attempt 
to write a patch. I'm not sure I'm up to that on my own right now.

Cheers,
    Ron