Mailman 3 - capi-sig

Subtle C API discrepancy: PyEval_GetLocals() vs PyEval_GetGlobals() with no active Python frame
by Nick Coghlan April 29, 2022

April 29, 2022

There's more background in the Discourse thread [1], but the key C API design topic is that PyEval_GetLocals() and PyEval_GetGlobals() currently behave differently when there's no Python frame active in the current thread: * PyEval_GetGlobals(): returns NULL without setting an exception * PyEval_GetLocals(): returns NULL *and* sets an exception PyEval_GetLocals() has been doing the latter since it gained the ability to report memory allocation errors when dealing with optimised frames back in Python 3.4. The "no active Python frame" case isn't really an error (as it's a valid state in applications that embed CPython), so I'm wondering if PyEval_GetLocals() should partially go back to its pre-3.4 behaviour, and not set an exception in the "no active Python frame" case. Cheers, Nick. [1] https://discuss.python.org/t/subtle-c-api-discrepancy-pyeval-getlocals-vs-p… -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1

ANN: expy 0.6.5 released!
by Yingjie Lan April 16, 2022

April 16, 2022

EXPY is an express way to extend Python! EXPY provides a way to extend python in an elegant way. For more information and a tutorial, see: http://expy.sourceforge.net/ What's new: 1. Correct treatment of __init__ method. 2. Give warnings of missing Py_INCREF on appropriate special type methods. 3. Documentation update. Cheers, Yingjie

3 3

Deallocating objects from previous run
by Victor Milovanov March 31, 2022

March 31, 2022

Hi, TL;DR; Is it safe to Py_DecRef an object that was created before Py_Finalize after Py_Initialize was called to restart Python runtime? I am debugging 3.9+ support in https://github.com/pythonnet/pythonnet The issue I originally got is validate_list from gcmodule.c fails at assert(prev == GC_PREV(head)); In debugging this, I sprinkled my code with calls to validate_list, and found that the assertion fails in the following scenario: Py_Initialize(); ... PyObject* myObj = PyObject_Call(simple_class, empty_tuple, NULL); Py_Finalize(); ... Py_Initialize(); ... validate_list(gc.generations[2], collecting_clear_unreachable_clear); // OK Py_DecRef(myObj); validate_list(gc.generations[2], collecting_clear_unreachable_clear); // EXPLOSION Now I have not yet tried to narrow down the steps further, because I realised that I am unsure about the question in TL;DR;. E.g. is this use of myObj supported by embedding API in general, or is this expected, that after reinitializing the runtime Py_DecRef on old objects would break GC? In my scenario simple_class is defined in Python as inheriting from object, and only has two attributes set at runtime. One has value builtins.StopIteration, and the other one is an instance of traceback. Regards, Victor

4 3

PyFrameObject structure has been moved to the internal C API
by Victor Stinner March 21, 2022

March 21, 2022

Hi, tl; dr the C API of PyFrameObject changed. If you have issues, please speak up! I moved the PyFrameObject structure to the internal C API headers. You should now use the public C API to access its members: * ``f_back``: use :c:func:`PyFrame_GetBack`. * ``f_blockstack``: removed. * ``f_builtins``: use ``PyObject_GetAttrString((PyObject*)frame, "f_builtins")``. * ``f_code``: use :c:func:`PyFrame_GetCode`. * ``f_gen``: removed. * ``f_globals``: use ``PyObject_GetAttrString((PyObject*)frame, "f_globals")``. * ``f_iblock``: removed. * ``f_lasti``: use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``. Code using ``f_lasti`` with ``PyCode_Addr2Line()`` must use :c:func:`PyFrame_GetLineNumber` instead. * ``f_lineno``: use :c:func:`PyFrame_GetLineNumber` * ``f_locals``: use ``PyObject_GetAttrString((PyObject*)frame, "f_locals")``. * ``f_stackdepth``: removed. * ``f_state``: no public API (renamed to ``f_frame.f_state``). * ``f_trace``: no public API. * ``f_trace_lines``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_lines")`` (it also be modified). * ``f_trace_opcodes``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_opcodes")`` (it also be modified). * ``f_localsplus``: no public API (renamed to ``f_frame.localsplus``). * ``f_valuestack``: removed. See What's New in Python 3.11 for more details (I completed the doc, but it will only be updated online tomorrow, there is a cron task once per day): https://docs.python.org/dev/whatsnew/3.11.html#id2 You can use the pythoncapi_compat project to get PyFrame_GetBack() and PyFrame_GetCode() on Python 3.8 and older: https://pythoncapi-compat.readthedocs.io/ If there is no public C API to access the PyFrameObject members used by your project, please speak up! See for example my PR to use the public C API in coverage: https://github.com/nedbat/coveragepy/pull/1331 There is an on-going work for 2 years to add new getter functions for PyFrameObject: https://bugs.python.org/issue40421 In Python 3.10, there was no rush to make the structure internal. Things changed quickly in Python 3.11: a heavy rework on Python internals is on-going to optimize Python (especially ceval.c and how frames are created and used). For example, the _PyEval_EvalFrameDefault() function now gets a new InterpreterFrame structure, it no longer takes a PyFrameObject. Python 3.11 now creates Python frame objects on demand. Because of these changes, reading directly PyFrameObject.f_back is now an error since the member is not always filled, but it wasn't a compiler error. You must now call the public PyFrame_GetBack() function rather than accessing directly to the member. Since Python 2.3 (that's quite old ;-)), reading directly PyFrameObject.f_lineno is unsafe since it depends if the Python code is being traced or not: you must use PyFrame_GetLineNumber() instead. See also the issue for the longer rationale: https://bugs.python.org/issue46836 I know that these changes are inconvenient, but I'm here if you need my help for updating your project! Moreover, Python 3.11 is scheduled for October, so there are still a few months to adapt your project, and maybe add new getter functions. The best would be to add all required getter functions before Python 3.11 beta1 scheduled for the beginning of May (2022-05-06). Victor -- Night gathers, and now my watch begins. It shall not end until my death.

3 6

ANNOUNCE: PyBindGen 0.8
by Gustavo Carneiro March 10, 2022

March 10, 2022

The other day I released PyBindGen 0.8. Main news is that it features a new experimental header file scanner based on pygccxml (i.e., it's similar to py++ in scope, if not in maturity, but does not use boost.pythonunderneath). == What == PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write, and does not depend on any library or header files besides Python itself. == Where == https://launchpad.net/pybindgen/ == NEWS == - Support C++ instance attributes through getter/setter methods - Support functions as methods of C++ classes - Support the PyObject* type - Support unsigned int, C strings (char*) (from Mark Lee) - Add basic support for enum types - New experimental automatic module generator based on C/C++ header file scanner and annotations in comments, using pygccxml - Some bug fixes -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert

2 1

C API equivalent to abc.ABCMeta.register?
by Daniel Stutzbach March 2, 2022

March 2, 2022

I'm the author of an extension module (blist) that provides a type that fits the MutableSequence API. Is there a canonical way for me to register the type as a MutableSequence from the C API? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

3 2

Replacing PySequence_Check / PyMapping_Check
by Aviram Hassan Feb. 13, 2022

Feb. 13, 2022

Hi all, Tl;dr - As mentioned in issue https://bugs.python.org/issue5945, `PySequence_Check` and `PyMapping_Check` should be deprecated and discontinued. The reason for this is that they do not provide accurate results and can lead to unexpected behavior. (PyMapping_Check for a list will return True, and PySequence_Check for a mapping that doesn't subclass dict will also return true) This proposal is to have a process to deprecate those and introduce new functions instead. Thanks to the latest changes in the C-API, we now have flags set for classes that are sequences or mapping - those were created for the pattern matching functionality. This lets us have a path for clear and accurate determination of sequence or mapping. The use cases for the change are as follows: - C-API code that wishes to use the PySequence/PyMapping API and to guarantee correct behavior needs to check it is indeed a valid sequence/mapping - Static types casting from ambiguous “PyObject” to “PySequence” & “PyMapping” types as provided by PyO3 ( https://pyo3.rs/v0.15.1/conversions/tables.html#argument-types) The correct way to determine right now if an object is a sequence, is to check its `Py_TPFLAGS_SEQUENCE` flag and check if it’s a string, bytes or bytearray as they are the only sequences not having `TPFLAGS_SEQUENCE` per specification. This is cumbersome for most non-expert developers not aware of the nuances in the C-API. Having a C-API that provides this check out of the box can ease it for most. `PyMapping_Check` and `PySequence_Check` are still widely used and are **part of the stable ABI** even though core developers recommend avoiding those. This is very odd to stabilize and is an API that should be deprecated. Our suggested change is to add two new functions: `PyMapping_CheckNew` & `PySequence_CheckNew`. They will be implemented using the flags check and the str/bytes/bytearray check. Those functions will be 100% accurate and follow the existing documentations. We will do the following process to improve C-API: 1. Add these functions. 2. Add a deprecation notice on `PyMapping_Check` and `PySequence_Check` and remove all built-in usage of it. We will recommend the use of the new functions. 3. Drop the old functions. 4. Alias the new functions to `PyMapping_Check` and `PySequence_Check` and add a deprecation notice of `PyMapping_CheckNew` and `PySequence_CheckNew`. 5. Rename `PyMapping_CheckNew` and `PySequence_CheckNew` to `Mapping_Check` and `PySequence_Check`. The timeline will probably be long as it’s part of stable ABI, but in our opinion, the change is for the better. Caveats: - Sequences and Mappings will match the typing.Sequence and typing.Mapping abcs. Duck typing is further “discouraged”. We do not think of it as an issue since Python already has idioms and static type checkers that discourage this sort of duck typing. - C extensions skipping multiple minor Python versions will use the new code without their knowledge, which is not backwards compatible. This is an issue with any removal of a stable ABI or language function, and we consider it acceptable. Thanks to Bar Harel for helping me with the proposal. This is was initially discussed in this bpo: https://bugs.python.org/issue46376 Best regards, Aviram

6 6

Software Development - Application Development - Game Development
by Gary Jan. 10, 2022

Jan. 10, 2022

You are receiving this email because we wish you to use our cost-effective IT services. We are a China based Custom Software Application Development provider. We offer full cycle custom software programming services, from product idea, software development to support and enhancement. We employ a large pool of software engineers coming from different backgrounds. We are able to balance product development efforts and project duration to your business needs. Core Services: Custom Application Development Custom Software Development iPhone Application Development iPad Application Development Mobile Application Development Android Mobile Application Development Soft Product Development Game Design & Development Game Testing & Quality Assurance Cloud Computing Solutions: CRM Software Application Development ERP Software Application Development Enterprise Portal Solution Sales Force Automation Solution Game Design & Development Knowledge Management Solution Workflow Management Solution SharePoint Development Services Microsoft Online Services Microsoft Azure Services Best regards, Gary ITVIATSA Software Development Contact: ibsoftware(a)yeah.net Pls send address to larryremove123(a)msn.com for remove

2 2

Adding PyType_GetModuleByDef to the limited API
by Petr Viktorin Jan. 6, 2022

Jan. 6, 2022

In bpo-42100, I added the private function _PyType_GetModuleByDef, which allows module state access from slot methods (like tp_init or nb_add), the main thing missing from PEP 573 (Module State Access from C Extension Methods). The method is now used in 10 stdlib modules, without known issues. Since it was has optimized (thanks, Victor!), I haven't seen complaints about performance issues. Sadly, it has a loop so it's much slower than using C statics, but it's OK for many use cases. I haven't found a better way to get per-module state for isolated modules (and the stable ABI). So, I'd like to drop the leading underscore, and add PyType_GetModuleByDef directly to the limited API. The function itself can be implemented using only limited API, though it's a bit tricky to do so correctly (and our implementation uses private speedups), so it's better if extension authors can use it as a pre-made building block. The fact that it can be implemented with only limited API means that adding it should not add big burden for long-term maintenance in CPython, nor for alternate implementations of the C-API. Is anyone against? Does anyone feel this should be discussed more widely than on capi-sig?

5 5

gauravvneherkhamb007@gmail.com
by Gaurav Meherkhamb Jan. 4, 2022

Jan. 4, 2022

1 0