<p dir="ltr">Brett, I have not followed everything here but I have no problem with tweaks at this level as long as you are happy with it.</p>
<p dir="ltr">--Guido (mobile)</p>
<div class="gmail_extra"><br><div class="gmail_quote">On Sep 3, 2016 5:39 PM, "Brett Cannon" <<a href="mailto:brett@python.org">brett@python.org</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Sat, 3 Sep 2016 at 17:27 Yury Selivanov <<a href="mailto:yselivanov.ml@gmail.com" target="_blank">yselivanov.ml@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
On 2016-09-03 5:19 PM, Brett Cannon wrote:<br>
><br>
><br>
> On Sat, 3 Sep 2016 at 16:43 Yury Selivanov <<a href="mailto:yselivanov.ml@gmail.com" target="_blank">yselivanov.ml@gmail.com</a><br>
> <mailto:<a href="mailto:yselivanov.ml@gmail.com" target="_blank">yselivanov.ml@gmail.<wbr>com</a>>> wrote:<br>
><br>
><br>
><br>
> On 2016-09-03 4:15 PM, Christian Heimes wrote:<br>
> > On 2016-09-04 00:03, Yury Selivanov wrote:<br>
> >><br>
> >> On 2016-09-03 12:27 PM, Brett Cannon wrote:<br>
> >>> Below is the `co_extra` section of PEP 523 with the update<br>
> saying that<br>
> >>> users are expected to put a tuple in the field for easier<br>
> simultaneous<br>
> >>> use of the field.<br>
> >>><br>
> >>> Since the `co_extra` discussions do not affect CPython itself I'm<br>
> >>> planning on landing the changes stemming from the PEP probably<br>
> on Monday.<br>
> >> Tuples are immutable. If you have multiple co_extra users then<br>
> they<br>
> >> will have to either mutate tuple (which isn't always possible, for<br>
> >> instance, you can't increase size), or to replace it with<br>
> another tuple.<br>
> >><br>
> >> Creating lists is a bit more expensive, but item access speed<br>
> should be<br>
> >> in the same ballpark.<br>
> >><br>
> >> Another question -- sorry if this was discussed before -- why<br>
> do we want<br>
> >> a PyObject* there at all? I.e. why don't we create a dedicated<br>
> struct<br>
> >> CoExtraContainer to manage the stuff in co_extra? My<br>
> understanding is<br>
> >> that the users of co_extra are C-level python optimizers and<br>
> profilers,<br>
> >> which don't need the overhead of CPython API.<br>
><br>
><br>
> As Chris pointed out in another email, the overhead is only in the<br>
> allocation, not the iteration/access if you use the PyTuple macros to<br>
> get the size and index into the tuple the overhead is negligible.<br>
<br>
Yes, my point was that it's as cheap to use a list as a tuple for<br>
co_extra. If we decide to store PyObject in co_extra.<br>
<br>
> >><br>
> >> This way my work to add an extra caching layer (which I'm very much<br>
> >> willing to continue to work on) wouldn't require another set of<br>
> extra<br>
> >> fields for code objects.<br>
> > Quick idea before I go to bed:<br>
> ><br>
> > You could adopt a similar API to OpenSSL's CRYPTO_get_ex_new_index()<br>
> > API,<br>
> ><br>
> <a href="https://www.openssl.org/docs/manmaster/crypto/CRYPTO_get_ex_new_index.html" rel="noreferrer" target="_blank">https://www.openssl.org/docs/<wbr>manmaster/crypto/CRYPTO_get_<wbr>ex_new_index.html</a><br>
> ><br>
> ><br>
> > static int code_index = 0;<br>
> ><br>
> > int PyCodeObject_NewIndex() {<br>
> > return code_index++;<br>
> > }<br>
> ><br>
> > A library like Pyjion has to acquire an index first. In further<br>
> calls it<br>
> > uses the index as offset into the new co_extra field. Libraries<br>
> don't<br>
> > have to hard-code their offset and two libraries will never<br>
> conflict.<br>
> > PyCode_New() can pre-populate co_extra with a PyTuple of size<br>
> > code_index. This avoids most resizes if you load Pyjion early. For<br>
> > code_index == 0 leaf the field NULL.<br>
><br>
> Sounds like a very good idea!<br>
><br>
><br>
> The problem with this is the pre-population. If you don't get your<br>
> index assigned before the very first code object is allocated then you<br>
> still have to manage the size of the tuple in co_extra. So what this<br>
> would do is avoid the iteration but not the allocation overhead.<br>
><br>
> If we open up the can of worms in terms of custom functions for this<br>
> (which I was trying to avoid), then you end up with Py_ssize_t<br>
> _PyCode_ExtraIndex(), PyObject *<br>
> _PyCode_GetExtra(PyCodeObject *code, Py_ssize_t index), and int<br>
> _PyCode_SetExtra(PyCodeObject *code, Py_ssize_t index, PyObject *data)<br>
> which does all the right things for creating or resizing the tuple as<br>
> necessary and which I think matches mostly what Nick had proposed<br>
> earlier. But the pseudo-code for _PyCode_GetExtra() would be::<br>
><br>
> if co_extra is None:<br>
> co_extra = (None,) * _next_extra_index;<br>
> return None<br>
> elif len(co_extra) < index - 1:<br>
> ... pad out tuple<br>
> return None<br>
> else:<br>
> return co_extra[index]<br>
><br>
> Is that going to save us enough to want to have a custom API for this?<br>
<br>
But without that new API (basically what Christian proposed) you'd need<br>
to iterate over the list in order to find the object that belongs to<br>
Pyjion.</blockquote><div><br></div><div>Yes.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> If we manage to implement my opcode caching idea, we'll have at<br>
least two known users of co_extra. Without a way to claim a particular<br>
index in co_extra you will have some overhead to locate your objects.<br></blockquote><div><br></div><div>Two things. One, I would want any new API to start with an underscore so people know we can and will change its semantics as necessary. Two, Guido would have to re-accept the PEP as this is a shift in the use of the field if this is how people want to go.</div></div></div>
<br>______________________________<wbr>_________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-dev" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/guido%40python.org" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/options/python-dev/<wbr>guido%40python.org</a><br>
<br></blockquote></div></div>