[Python-Dev] Tweak to PEP 523 for storing a tuple in co_extra

[Yury Selivanov]

On 2016-09-03 4:15 PM, Christian Heimes wrote:
> On 2016-09-04 00:03, Yury Selivanov wrote:
>>
>> On 2016-09-03 12:27 PM, Brett Cannon wrote:
>>> Below is the `co_extra` section of PEP 523 with the update saying that
>>> users are expected to put a tuple in the field for easier simultaneous
>>> use of the field.
>>>
>>> Since the `co_extra` discussions do not affect CPython itself I'm
>>> planning on landing the changes stemming from the PEP probably on Monday.
>> Tuples are immutable.  If you have multiple co_extra users then they
>> will have to either mutate tuple (which isn't always possible, for
>> instance, you can't increase size), or to replace it with another tuple.
>>
>> Creating lists is a bit more expensive, but item access speed should be
>> in the same ballpark.
>>
>> Another question -- sorry if this was discussed before -- why do we want
>> a PyObject* there at all?  I.e. why don't we create a dedicated struct
>> CoExtraContainer to manage the stuff in co_extra?  My understanding is
>> that the users of co_extra are C-level python optimizers and profilers,
>> which don't need the overhead of CPython API.
>>
>> This way my work to add an extra caching layer (which I'm very much
>> willing to continue to work on) wouldn't require another set of extra
>> fields for code objects.
> Quick idea before I go to bed:
>
> You could adopt a similar API to OpenSSL's CRYPTO_get_ex_new_index()
> API,
> https://www.openssl.org/docs/manmaster/crypto/CRYPTO_get_ex_new_index.html
>
>
> static int code_index = 0;
>
> int PyCodeObject_NewIndex() {
>      return code_index++;
> }
>
> A library like Pyjion has to acquire an index first. In further calls it
> uses the index as offset into the new co_extra field. Libraries don't
> have to hard-code their offset and two libraries will never conflict.
> PyCode_New() can pre-populate co_extra with a PyTuple of size
> code_index. This avoids most resizes if you load Pyjion early. For
> code_index == 0 leaf the field NULL.

Sounds like a very good idea!

Yury