Mailman 3 - Python-Dev

Switching to Discourse
by Petr Viktorin Jan. 26, 2023

Jan. 26, 2023

Hello, Currently development discussions are split between multiple communication channels, for example: - python-dev and discuss.python.org for design discussions, - GitHub Issues and Pull Requests for specific changes, - IRC, Discord and private chats for real-time discussions, - Topic-specific channels like typing-sig. While most of these serve different needs, there is too much overlap between python-dev and discuss.python.org. It seems that for most people, this situation is worse than sticking to either one platform – even if we don't go with that person's favorite. The discuss.python.org experiment has been going on for quite a while, and while the platform is not without its issues, we consider it a success. The Core Development category is busier than python-dev. According to staff, discuss.python.org is much easier to moderate.. If you're following python-dev but not discuss.python.org, you're missing out. The Steering Council would like to switch from python-dev to discuss.python.org. Practically, this means: - Moving the required PEP announcements to discuss.python.org - Moving discuss.python.org up in the devguide communications page (https://devguide.python.org/communication/) - And that's it? I imagine that the mailing list will stay around for continuing past discussion threads and for announcements, eventually switching to auto-reject incoming messages with a pointer to discuss.python.org. To be clear, discuss.python.org allows editing posts, which is frankly handy for typos and clarifications. Editing alone should not be used for adding new info -- we should cultivate a culture of being friendly to mail users & notification watchers. This probably bears repeating in a few places. We're aware not everyone wants to use the discuss.python.org website, but there are some ways to avoid it: - For new PEPs, you can point your RSS client to https://www.python.org/dev/peps/peps.rss – it's not e-mail, but many email clients have RSS support. You can also watch the Steering Council issues on GitHub (https://github.com/python/steering-council/issues/) for important questions and discussions. - You can use discuss.python.org's “mailing list mode” (which subscribes you to all new posts), possibly with filtering and/or categorizing messages locally. However, we would like to know if this will pose an undue burden to anyone, if there are workflows or usage problems that we are not aware of. As mentioned, this is something the Steering Council thinks is a good idea, but we want to make sure we're aware of all the impact when we make the final decision. – Petr, on behalf of the Steering Council

43 114

Requesting a review on a PR
by Wheeler Law Jan. 21, 2023

Jan. 21, 2023

Greetings folks, I opened an issue[1] related to respecting the `http.client.HTTPConnection.debuglevel` value; I opened a subsequent PR[2] that fixes the issue back in November and I was wondering if I could get some feedback on it. Kind regards, Wheeler [1]: https://github.com/python/cpython/issues/99352 [2]: https://github.com/python/cpython/pull/99353

2 1

Exception compatibility with aliens
by Frederick Virchanza Gotham Jan. 19, 2023

Jan. 19, 2023

I have sent this email to two mailing lists for programming language proposals, the one for C++ and the one for Python. If you reply to this email, please make sure you reply to both. You can see the mailing list archives for this month for each list here: C++ : https://lists.isocpp.org/std-proposals/2023/01/date.php Python : https://mail.python.org/archives/list/python-dev@python.org/2023/1/ Both C++ and Python have exception handling, however a C++ program which links with a Python library is unable to handle an exception thrown from Python. Is it conceivable in the future that the C++ Standards Committee and the Python Steering Council could cooperate and furnish each other with an exhaustive list of exceptions thrown from their respective standard libraries, so that both languages can implement compatibility with each other's exceptions? If such a list were to accompany each C++ standard and each Python Language Reference, then for example the C++ standard library could provide classes for handling Python exceptions: namespace std { struct alien_exception /* does not inherit from std::exception */ { enum class language : unsigned int { java=1u, csharp, python }; virtual language lang(void) const noexcept = 0; virtual char const *what(void) const noexcept = 0; }; namespace aliens { namespace java { struct exception : alien_exception { char const *getLocalizedMessage(void) const noexcept; char const *getMessage(void) const noexcept; std::stacktrace getStackTrace(void) const noexcept; char const *what(void) const noexcept override { return this->getMessage(); } language lang(void) const noexcept override { return language::java; } }; } namespace python { struct exception : alien_exception { char const *const *notes(void) const noexcept; /* null terminated */ std::stacktrace traceback() const noexcept; char const *what(void) const noexcept override { return this->notes()[0u]; } language lang(void) const noexcept override { return language::python; } }; } namespace python { struct BaseException : exception {}; struct BaseExceptionGroup : BaseException {}; struct GeneratorExit : BaseException {}; struct KeyboardInterrupt : BaseException {}; struct SystemExit : BaseException {}; struct Exception : BaseException {}; struct ArithmeticError : Exception {}; struct FloatingPointError : ArithmeticError {}; struct OverflowError : ArithmeticError {}; struct ZeroDivisionError : ArithmeticError {}; struct AssertionError : Exception {}; struct AttributeError : Exception {}; struct BufferError : Exception {}; struct EOFError : Exception {}; struct ExceptionGroupBaseExceptionGroup : Exception {}; struct ImportError : Exception {}; struct ModuleNotFoundError : ImportError {}; struct LookupError : Exception {}; struct IndexError : LookupError {}; struct KeyError : LookupError {}; struct MemoryError : Exception {}; struct NameError : Exception {}; struct UnboundLocalError : NameError {}; struct OSError : Exception {}; struct BlockingIOError : OSError {}; struct ChildProcessError : OSError {}; struct ConnectionError : OSError {}; struct BrokenPipeError : ConnectionError {}; struct ConnectionAbortedError : ConnectionError {}; struct ConnectionRefusedError : ConnectionError {}; struct ConnectionResetError : ConnectionError {}; struct FileExistsError : OSError {}; struct FileNotFoundError : OSError {}; struct InterruptedError : OSError {}; struct IsADirectoryError : OSError {}; struct NotADirectoryError : OSError {}; struct PermissionError : OSError {}; struct ProcessLookupError : OSError {}; struct TimeoutError : OSError {}; struct ReferenceError : Exception {}; struct RuntimeError : Exception {}; struct NotImplementedError : RuntimeError {}; struct RecursionError : RuntimeError {}; struct StopAsyncIteration : Exception {}; struct StopIteration : Exception {}; struct SyntaxError : Exception {}; struct IndentationError : SyntaxError {}; struct TabError : IndentationError {}; struct SystemError : Exception {}; struct TypeError : Exception {}; struct ValueError : Exception {}; struct UnicodeError : ValueError {}; struct UnicodeDecodeError : UnicodeError {}; struct UnicodeEncodeError : UnicodeError {}; struct UnicodeTranslateError : UnicodeError {}; struct Warning : Exception {}; struct BytesWarning : Warning {}; struct DeprecationWarning : Warning {}; struct EncodingWarning : Warning {}; struct FutureWarning : Warning {}; struct ImportWarning : Warning {}; struct PendingDeprecationWarning : Warning {}; struct ResourceWarning : Warning {}; struct RuntimeWarning : Warning {}; struct SyntaxWarning : Warning {}; struct UnicodeWarning : Warning {}; struct UserWarning : Warning {}; } // close namespace 'python' } // close namespace 'aliens' class forced_stack_unwind {}; } // close namespace 'std' extern int Some_Func_In_Python_Library(void) { return 7; /* or throws an exception */ } int main(void) { try { Some_Func_In_Python_Library(); } catch ( std::aliens::python::FloatingPointError const & ) { // The specific Python exception we want to handle } catch ( std::aliens::python::exception const & ) { // Any exception thrown from Python } catch ( std::alien_exception const &) { // Maybe the Python library linked with an // Eiffel library that threw an exception } catch ( std::forced_stack_unwind ) { } }

3 2

Re: [std-proposals] Exception compatibility with aliens
by Frederick Virchanza Gotham Jan. 18, 2023

Jan. 18, 2023

On Wed, Jan 18, 2023 at 3:07 PM Jason McKesson wrote: > > Also, this proposal seems to be missing the biggest issue with > cross-language exception handling: the fact that you can't throw > exceptions across languages. The only thing you *can* do is catch > exceptions on the source language end, convert them into some data > packet, and throw a different exception on the destination language > side. Yes this is what I had in mind. Behind the scenes, the C++ compiler would catch the Python exception and then throw something that C++ can deal with "such as std::aliens::python::exception".

3 2

C- Python on Windows
by Guenther Sohler Jan. 18, 2023

Jan. 18, 2023

Hi Group, I have successfully used C-Python (3) in a software project in unix and its great stuff! The environment was cmake using g++ in Linux Now when i want to get my project compiled in windows, whats the easiest development chain ? Is there something like a python.dll which i can link to my project and having an embedded python interpreter ? Maybe the question is too simple, but i could not yet find the right place to read docs. Thank you for your hints!

3 2

Python 3.12.0 alpha 4 released
by Thomas Wouters Jan. 10, 2023

Jan. 10, 2023

I'm pleased to announce the release of Python 3.12 alpha 4. https://www.python.org/downloads/release/python-3120a4/ *This is an early developer preview of Python 3.12*. Major new features of the 3.12 series, compared to 3.11 Python 3.12 is still in development. This release, 3.12.0a4 is the fourth of seven planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be added up until the start of the beta phase (2023-05-08) and, if necessary, may be modified or deleted up until the release candidate phase (2023-07-31). Please keep in mind that this is a preview release and its use is *not *recommended for production environments. Many new features for Python 3.12 are still being planned and written. Among the new major new features and changes so far: - Even more improved error messages. More exceptions potentially caused by typos now make suggestions to the user. - Support for the Linux perf profiler to report Python function names in traces. - The deprecated wstr and wstr_length members of the C implementation of unicode objects were removed, per PEP 623 <https://peps.python.org/pep-0623>. - In the unittest module, a number of long deprecated methods and classes were removed. (They had been deprecated since Python 3.1 or 3.2). - The deprecated smtpd and distutils modules have been removed (see PEP 594 <https://peps.python.org/pep-0594/> and PEP 632 <https://peps.python.org/pep-0632/>). The setuptools package (installed by default in virtualenvs and many other places) continues to provide the distutils module. - A number of other old, broken and deprecated functions, classes and methods have been removed. - (Hey,* fellow core developer*, if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>.) For more details on the changes to Python 3.12, see What's new in Python 3.12 <https://docs.python.org/dev/whatsnew/3.12.html>. The next pre-release of Python 3.12 will be 3.12.0a5, currently scheduled for 2023-02-06. More resources Online Documentation <https://docs.python.org/3.12/>. PEP 693 <https://www.python.org/dev/peps/pep-0693/>, the Python 3.12 Release Schedule. Report bugs via GitHub Issues <https://github.com/python/cpython/issues>. Help fund Python and its community <https://www.python.org/psf/donations/>. And now for something completely different Two haikus apt, as Python's development springs ever forward. I write, erase, rewrite > Erase again, and then > A poppy blooms. Haiku by Katsushika Hokusai. O snail > Climb Mount Fuji, > But slowly, slowly! Haiku by Kobayashi Issa. Enjoy the new releases Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Regards from chilly Amsterdam, Your release team, Thomas Wouters Ned Deily Steve Dower -- Thomas Wouters <thomas(a)python.org>

1 0

Fwd: Python 3.11 performance with frame pointers
by Daan De Meyer Jan. 4, 2023

Jan. 4, 2023

Hi, As part of the proposal to enable frame pointers by default in Fedora (https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer), we did some benchmarking to figure out the expected performance impact. The performance impact was generally minimal, except for the pyperformance benchmark suite where we noticed a more substantial difference between a system built with frame pointers and a system built without frame pointers. The results can be found here: https://github.com/DaanDeMeyer/fpbench (look at the mean difference column for the pyperformance results where the percentage is the slowdown compared to a system built without frame pointers). One of the biggest slowdowns was on the scimark_sparse_mat_mult benchmark which slowed down 9.5% when the system (including python) was built with frame pointers. Note that these benchmarks were run against Python 3.11 on a Fedora 37 x86_64 system (one built with frame pointers, another built without frame pointers). The system used to run the benchmarks was an Amazon EC2 machine. We did look a bit into the reasons behind this slowdown. I'll quote the investigation by Andrii on the Fesco issue thread here (https://pagure.io/fesco/issue/2817): > So I did look a bit at Python with and without frame pointers trying to > understand pyperformance > regressions. > First, perf data suggests that big chunk of CPU is spent in _PyEval_EvalFrameDefault, > so I looked specifically into it (also we had to use DWARF mode for perf for apples-to-apples > comparison, and a bunch of stack traces weren't symbolized properly, which just again > reminds why having frame pointers is important). > perf annotation of _PyEval_EvalFrameDefault didn't show any obvious hot spots, the work > seemed to be distributed pretty similarly with or without frame pointers. Also scrolling through > _PyEval_EvalFrameDefault disassembly also showed that instruction patterns between fp > and no-fp versions are very similar. > But just a few interesting observations. > The size of _PyEval_EvalFrameDefault function specifically (and all the other functions didn't > change much in that regard) increased very significantly from 46104 to 53592 bytes, which is a > considerable 15% increase. Looking deeper, I believe it's all due to more stack spills and > reloads due to one less register available to keep local variables in registers instead of on the stack. > Looking at _PyEval_EvalFrameDefault C code, it is a humongous one function with gigantic switch > statement that implements Python instruction handling logic. So the function itself is big and it has > a lot of local state in different branches, which to me explained why there is so much stack spill/load. > Grepping for instruction of the form mov -0xf0(%rbp),%rcx or mov 0x50(%rsp),%r10 (and their reverse > variants), I see that there is a substantial amount of stack spill/load in _PyEval_EvalFrameDefault > disassembly already in default no frame pointer variant (1870 out of 11181 total instructions in that > function, 16.7%), and it just increases further in frame pointer version (2341 out of 11733 instructions, 20%). > One more interesting observation. With no frame pointers, GCC generates stack accesses using %rsp > with small positive offsets, which results in pretty compact binary instruction representation, e.g.: > 0x00000000001cce40 <+44160>: 4c 8b 54 24 50 mov 0x50(%rsp),%r10 > This uses 5 bytes. But if frame pointers are enabled, GCC switches to using %rbp-relative offsets, > which are all negative. And that seems to result in much bigger instructions, taking now 7 bytes instead of 5: > 0x00000000001d3969 <+53065>: 48 8b 8d 10 ff ff ff mov -0xf0(%rbp),%rcx > I found it pretty interesting. I'd imagine GCC should be capable to keep using %rsp addressing just fine > regardless of %rbp and save on instruction sizes, but apparently it doesn't. Not sure why. But this instruction > increase, coupled with increase of number of spills/reloads, actually explains huge increase in byte size of > _PyEval_EvalFrameDefault: (2341 - 1870) * 7 + 1870 * 2 = 7037 (2 extra bytes for existing 1870 instructions > that were switched from %rsp+positive offset to %rbp + negative offset, plus 7 bytes for each of new 471 instructions). > I'm no compiler expert, but it would be nice for someone from GCC community to check this as well (please CC > relevant folks, if you know them). > In summary, to put it bluntly, there is just more work to do for CPU saving/restoring state to/from stack. But I don't > think _PyEval_EvalFrameDefault example is typical of how application code is written, nor is it, generally speaking, > a good idea to do so much within single gigantic function. So I believe it's more of an outlier than a typical case. We have a few questions: - Is this slowdown when Python is built with frame pointers to be expected? Has the Python community done any of their own experiments with building Python with and without frame pointers? - Is there anything we can do to fix the slowdown when Python is built with frame pointers? - Should we expect any change in benchmark results if we benchmark against Python 3.12? Supposedly there are changes in Python 3.12 related to frame pointers so we're wondering if those changes might affect these results in any way. Cheers, Daan De Meyer

2 1

Re: PEP 701 – Syntactic formalization of f-strings
by Rob Cliffe Dec. 22, 2022

Dec. 22, 2022

Great stuff! 😁 Rob Cliffe On 19/12/2022 17:59, Pablo Galindo Salgado wrote: > > Hi everyone, > > I am very excited to share with you a PEP thatBatuhan Taskaya, > Lysandros Nikolaou and myself have been working on recently:PEP 701 - > PEP 701 – Syntactic formalization of f-strings > <https://peps.python.org/pep-0701/>. > > We believe this will be a great improvement in both the > maintainability of CPython and the usability of f-strings. > > We look forward to hearing what you think about this and to get your > feedback! > > *Here is a TLDR for your convenience:* > > * The PEP proposes a formalized grammar for f-strings in Python by > adding f-strings directly into the Grammar instead of using a > two-pass hand-written parser. > * This would lift some existing restrictions for f-strings that (we > believe) will improve the user experience with f-strings. > * Other benefits include: > o Reduced maintenance costs for f-string parsing code as well as > improved usability for users and library developers. > o Better error messages involving f-strings by leveraging the > PEG parser machinery. > o The proposed changes would improve the overall consistency of > the language and provide a way for alternative implementations > to accurately implement f-strings. > > *** IMPORTANT: direct all discussions to the discussion thread on > discourse: > https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/2… > *** > > Thanks a lot, everyone for your time! > > Regards from rainy London, > Pablo Galindo Salgado > > _______________________________________________ > Python-Dev mailing list --python-dev(a)python.org > To unsubscribe send an email topython-dev-leave(a)python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived athttps://mail.python.org/archives/list/python-dev@python.org/message/IU4O3… > Code of Conduct:http://python.org/psf/codeofconduct/

1 0

Feature Request: Adding Way to Annotate Class Variables Distinct from Instance Variables
by sakaic2003＠gmail.com Dec. 22, 2022

Dec. 22, 2022

Hello folks, I am Chihiro, a.k.a. Frodo821, and this is my first post to this group. I searched for a way to annotate both class variables and instance variables with different types and concluded that there is currently no way to do this. For example, what I want to: ``` class Foo: variable_both_class_and_instance = Field(desc="descriptive string") def __init__(self, value: int): self.variable_both_class_and_instance = value assert isinstance(Foo.variable_both_class_and_instance, Field) assert isinstance(Foo().variable_both_class_and_instance, int) ``` In this example, I want to annotate `Foo.variable_both_class_and_instance` with `Field` when it is accessed as a class variable. On the other hand, I want to annotate `Foo.variable_both_class_and_instance` with `int` when it is accessed as an instance variable. I don't have any smart ideas to accomplish this, but I think `typing.ClassVar` could be extended this like: ``` class Foo: variable_both_class_and_instance: Union[ClassVar[Field], int] = Field(desc="descriptive string") def __init__(self, value: int): self.variable_both_class_and_instance = value assert isinstance(Foo.variable_both_class_and_instance, Field) assert isinstance(Foo().variable_both_class_and_instance, int) ``` Do you have any ideas or comments about this?

4 3

PEP 701 – Syntactic formalization of f-strings
by Pablo Galindo Salgado Dec. 21, 2022

Dec. 21, 2022

Hi everyone, I am very excited to share with you a PEP that Batuhan Taskaya, Lysandros Nikolaou and myself have been working on recently: PEP 701 - PEP 701 – Syntactic formalization of f-strings <https://peps.python.org/pep-0701/>. We believe this will be a great improvement in both the maintainability of CPython and the usability of f-strings. We look forward to hearing what you think about this and to get your feedback! *Here is a TLDR for your convenience:* - The PEP proposes a formalized grammar for f-strings in Python by adding f-strings directly into the Grammar instead of using a two-pass hand-written parser. - This would lift some existing restrictions for f-strings that (we believe) will improve the user experience with f-strings. - Other benefits include: - Reduced maintenance costs for f-string parsing code as well as improved usability for users and library developers. - Better error messages involving f-strings by leveraging the PEG parser machinery. - The proposed changes would improve the overall consistency of the language and provide a way for alternative implementations to accurately implement f-strings. *** IMPORTANT: direct all discussions to the discussion thread on discourse: https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/2… <https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/2…> *** Thanks a lot, everyone for your time! Regards from rainy London, Pablo Galindo Salgado

1 1