[Python-Dev] Tweak to PEP 523 for storing a tuple in co_extra

[Christian Heimes]

On 2016-09-04 00:03, Yury Selivanov wrote:
> 
> 
> On 2016-09-03 12:27 PM, Brett Cannon wrote:
>> Below is the `co_extra` section of PEP 523 with the update saying that
>> users are expected to put a tuple in the field for easier simultaneous
>> use of the field.
>>
>> Since the `co_extra` discussions do not affect CPython itself I'm
>> planning on landing the changes stemming from the PEP probably on Monday.
> 
> Tuples are immutable.  If you have multiple co_extra users then they
> will have to either mutate tuple (which isn't always possible, for
> instance, you can't increase size), or to replace it with another tuple.
> 
> Creating lists is a bit more expensive, but item access speed should be
> in the same ballpark.
> 
> Another question -- sorry if this was discussed before -- why do we want
> a PyObject* there at all?  I.e. why don't we create a dedicated struct
> CoExtraContainer to manage the stuff in co_extra?  My understanding is
> that the users of co_extra are C-level python optimizers and profilers,
> which don't need the overhead of CPython API.
> 
> This way my work to add an extra caching layer (which I'm very much
> willing to continue to work on) wouldn't require another set of extra
> fields for code objects.

Quick idea before I go to bed:

You could adopt a similar API to OpenSSL's CRYPTO_get_ex_new_index()
API,
https://www.openssl.org/docs/manmaster/crypto/CRYPTO_get_ex_new_index.html


static int code_index = 0;

int PyCodeObject_NewIndex() {
    return code_index++;
}

A library like Pyjion has to acquire an index first. In further calls it
uses the index as offset into the new co_extra field. Libraries don't
have to hard-code their offset and two libraries will never conflict.
PyCode_New() can pre-populate co_extra with a PyTuple of size
code_index. This avoids most resizes if you load Pyjion early. For
code_index == 0 leaf the field NULL.

Christian