Mailman 3 June 2018 - Python-Dev

Re: [Python-Dev] About [].append == [].append
by Jeroen Demeyer June 21, 2018

June 21, 2018

On 2018-06-21 13:33, Ivan Pozdeev via Python-Dev wrote: > First, tell us what problem you're solving. There is no specific problem I want to solve here. I just noticed an inconsistency and I wondered if it would be OK to change the implementation of comparisons of builtin_function_or_method instances. It's a valid question to ask even if it doesn't solve an actual problem.

2 1

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods
by Jeroen Demeyer June 21, 2018

June 21, 2018

On 2018-06-21 11:22, Victor Stinner wrote: > https://www.python.org/dev/peps/pep-0580/#the-c-call-protocol > > CCALL_VARARGS: cc_func(PyObject *self, PyObject *args) > > If we add a new calling convention This is not a *new* calling convention, it's the *existing* calling convention for METH_VARARGS. Obviously, we need to continue to support that.

1 0

Re: [Python-Dev] Can we make METH_FASTCALL public, from Python 3.7? (ref: PEP 579
by Jeroen Demeyer June 21, 2018

June 21, 2018

On 2018-06-20 17:42, INADA Naoki wrote: > I don't have any idea about changing METH_FASTCALL more. > If Victor and Serhiy think so, and PyPy maintainers like it too, I want > to make it public > as soon as possible. There are two different things here: The first is documenting METH_FASTCALL such that everybody can create built-in functions using the METH_FASTCALL signature. I think that the API for METH_FASTCALL (without or with METH_KEYWORDS) is fine, so I support making it public. This is really just a documentation issue, so I see no reason why it couldn't be added to 3.7.0 if we're fast. The API for calling functions using the FASTCALL convention is more of a mess though. There are functions taking keyword arguments as dict and functions taking them as tuple. As I mentioned in PEP 580, I'd like to merge these and simply allow either a dict or a tuple. Since this would require an API change, this won't be for 3.7.0. Jeroen.

2 1

Can we make METH_FASTCALL public, from Python 3.7? (ref: PEP 579
by INADA Naoki June 21, 2018

June 21, 2018

Hi, All. First of all, thank you Jeroen for writing nice PEPs. When I read PEP 579, I think "6. METH_FASTCALL is private and undocumented" should be solved first. I don't have any idea about changing METH_FASTCALL more. If Victor and Serhiy think so, and PyPy maintainers like it too, I want to make it public as soon as possible. _PyObject_FastCall* APIs are private in Python 3.7. But METH_FASTCALL is not completely private (start without underscore, but not documented) Can we call it as public, stable by adding document, if Ned allows? It's used widely in Python internals already. I suppose that making it public doesn't make Python 3.7 unstable much. If we can't at Python 3.7, I think we should do it at 3.8. Regards, -- INADA Naoki <songofacandy(a)gmail.com>

6 8

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods
by Jeroen Demeyer June 20, 2018

June 20, 2018

On 2018-06-20 16:42, Antoine Pitrou wrote: > I'm wondering what amount of code and debugging is needed for, say, > Cython or Numba to implement that protocol as a caller, without going > through the C API's indirections (for performance). The goal is to have a really fast C API without a lot of indirections. If Cython or Numba can implement the protocol faster than CPython, we should just change the CPython implementation to be equally fast.

1 0

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods
by Jeroen Demeyer June 20, 2018

June 20, 2018

On 2018-06-20 16:09, Antoine Pitrou wrote: > But there seems to be some complication on top of that: > > - PyCCall_FastCall() accepts several types for the keywords, even a > dict; That is actually a *simplification* instead of a *complication*. Currently, there is a huge amount of code duplication between _PyMethodDef_RawFastCallKeywords and _PyMethodDef_RawFastCallDict. Folding both of these in one function actually makes things simpler. > does it get forwarded as-is to the `cc_func` or is it first > transformed? Transformed (obviously, otherwise it would be a huge backwards incompatibility problem). > - there's CCALL_OBJCLASS and CCALL_SLICE_SELF which have, well, > non-obvious behaviour (especially the latter), especially as it is > conditioned on the value of other fields or flags It's actually quite obvious when you think of it: both are needed to support existing use cases. Perhaps it's just not explained well enough in the PEP. > I wonder if there's a way to push some of the specificities out of the > protocol and into the C API that mediates between the protocol and > actual callers? Sorry, I have no idea what you mean here. Actually, those flags are handled by the C API. The actual C functions don't need to care about those flags.

2 1

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
by Jeroen Demeyer June 20, 2018

June 20, 2018

On 2018-06-18 16:55, INADA Naoki wrote: > Speeding up most python function and some bultin functions was very > significant. > But I doubt making some 3rd party call 20% faster can make real > applications significant faster. These two sentences are almost contradictory. I find it strange to claim that a given optimization was "very significant" in specific cases while saying that the same optimization won't matter in other cases. People *have* done benchmarks for actual code and this is causing actual slow-downs of around 20% in actual applications. That is the main reason why I am trying to push this PEP (or PEP 575 which solves the same problem in a different way). Jeroen.

4 5

Re: [Python-Dev] C-level calling
by Jeroen Demeyer June 20, 2018

June 20, 2018

On 2018-06-20 08:00, Stefan Behnel wrote: > Just to add another bit of background on top of the current discussion, > there is an idea around, especially in the scipy/big-data community, (and > I'm not giving any guarantees here that it will lead to a PEP + > implementation, as it depends on people's workload) to design a dedicated C > level calling interface for Python. Think of it as similar to the buffer > interface, but for calling arbitrary C functions by bypassing the Python > call interface entirely. Objects that wrap some kind of C function (and > there are tons of them in the CPython world) would gain C signature meta > data, maybe even for overloaded signatures, and C code that wants to call > them could validate that meta data and call them as native C calls. See also https://www.python.org/dev/peps/pep-0579/#allowing-native-c-arguments I specifically designed PEP 580 to be extendable such that it would be possible to add features later.

1 0

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
by Jeroen Demeyer June 20, 2018

June 20, 2018

On 2018-06-18 15:09, Victor Stinner wrote: > There are multiple issues with tp_fastcall: Personally, I think that you are exaggerating these issues. Below, I'm writing the word FASTCALL to refer to tp_fastcall in your patch as well as my C call protocol in the PEP-in-progress. > * ABI issue: it's possible to load a C extension using the old ABI, > without tp_fastcall: it's not possible to write type->tp_fastcall on > such type. This limitation causes different issues. It's not hard to check for FASTCALL support and have a case distinction between using tp_call and FASTCALL. > * If tp_call is modified, tp_fastcall may be outdated. I plan to support FASTCALL only for extension types. Those cannot be changed from Python. If it turns out that FASTCALL might give significant benefits also for heap types, we can deal with those modifications: we already need to deal with such modifications anyway for existing slots like __call__. > * Many public functions of the C API still requires the tuple and dict > to pass positional and keyword arguments, so a compatibility layer is > required to types who only want to implement FASTCALL. Related issue: > what is something calls tp_call with (args: tuple, kwargs: dict)? > Crash or call a compatibility layer converting arguments to FASTCALL > calling convention? You make it sound as if such a "compatibility layer" is a big issue. You just need one C API function to put in the tp_call slot which calls the object instead using FASTCALL.

3 2

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
by Jeroen Demeyer June 19, 2018

June 19, 2018

On 2018-06-18 03:34, INADA Naoki wrote: > Victor had tried to add `tp_fastcall` slot, but he suspended his effort > because > it's benefit is not enough for it's complexity. > https://bugs.python.org/issue29259 I has a quick look at that patch and it's really orthogonal to what I'm proposing. I'm proposing to use the slot *instead* of existing fastcall optimizations. Victor's patch was about adding fastcall support to classes that didn't support it before. Jeroen.

5 4