Mailman 3 August 2022 - Python-Dev

Proposal: Allow non-default after default arguments
by Sebastian Rittau 17 Oct '23

17 Oct '23

Currently, Python doesn't allow non-default arguments after default arguments: >>> def foo(x=None, y): pass File "<stdin>", line 1 def foo(x=None, y): pass ^ SyntaxError: non-default argument follows default argument I believe that at the time this was introduced, no use cases for this were known and this is is supposed to prevent a source of bugs. I have two use cases for this, one fringe, but valid, the other more important: The fringe use case: Suppose you have a function that takes a 2D coordinate value as separate "x" and "y" arguments. The "x" argument is optional, the "y" argument isn't. Currently there are two ways to do this, none of them particularly great: def foo(y, x): # reverse x and y arguments, confusing ... def foo(x, y=None): # Treat the x argument as y if only one argument is provided if y is None: x, y = y, x ... To me, the "natural" solution looks like this: def foo(x=None, y): ... # Called like this: foo(1, 2) foo(y=2) This could also be useful when evolving APIs. For example there is a function "bar" that takes two required arguments. In a later version, the first argument gains a useful default, the second doesn't. There is no sensible way to evolve the API at the moment. The more important use case involves @overloads. A condensed example of a function where the return type depends on an "encoding" parameter, followed by further parameters that could be called like this: foo(123, "utf-8") # would return bytes foo(encoding="utf-8") foo(123, None) # would return str foo(encoding=None) foo(x=123) # would return str foo() This could ideally be written as: @overload def foo(x: int = ..., encoding: None = ...) -> str: ... @overload def foo(x: int = ..., encoding: str) -> bytes: ... # plus the actual implementation But due to the syntax constraint, this needs to be hacked around with a third overload: @overload def foo(x: int = ... encoding: None = ...) -> str: ... @overload def foo(x: int, encoding: str) -> bytes: ... # for foo(123, "utf-8") @overload def foo(*, encoding: str) -> bytes: ... # for foo(encoding="utf-8") Not only is this hard to read, real examples in typeshed are usually more complex, with many arguments before or after the affected argument or even multiple affected arguments. This often becomes too complex to write or maintain. Here is one example from the wild: https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b740467… Allowing non-default arguments after default arguments would solve both use cases above and eliminates a special case. I'm also not sure what exactly the current SyntaxError really protects us from. Adding a non-default after a default argument can't really lead bugs. - Sebastian

11 14

Python multithreading without the GIL
by Sam Gross 31 Jul '23

31 Jul '23

Hi, I've been working on changes to CPython to allow it to run without the global interpreter lock. I'd like to share a working proof-of-concept that can run without the GIL. The proof-of-concept involves substantial changes to CPython internals, but relatively few changes to the C-API. It is compatible with many C extensions: extensions must be rebuilt, but usually require small or no modifications to source code. I've built compatible versions of packages from the scientific Python ecosystem, and they are installable through the bundled "pip". Source code: https://github.com/colesbury/nogil Design overview: https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlej… My goal with the proof-of-concept is to demonstrate that removing the GIL is feasible and worthwhile, and that the technical ideas of the project could serve as a basis of such an effort. I'd like to start a discussion about these ideas and gauge the community's interest in this approach to removing the GIL. Regards, Sam Gross colesbury(a)gmail.com / sgross(a)fb.com

24 48

Mixed Python/C debugging
by Skip Montanaro 26 Apr '23

26 Apr '23

Having tried comp.lang.python with no response, I turn here... After at least ten years away from Python's run-time interpreter & byte code compiler, I'm getting set to familiarize myself with that again. This will, I think, entail debugging a mixed Python/C environment. I'm an Emacs user and am aware that GDB since 7.0 has support for debugging at the Python code level. Is Emacs+GDB my best bet? Are there any Python IDEs which support C-level breakpoints and debugging? Thanks, Skip

5 6

PEP 448 review
by Guido van Rossum 30 Mar '23

30 Mar '23

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)". I think Georg Brandl nailed it: """ *I like the "sequence and dict flattening" part of the PEP, mostly because itis consistent and should be easy to understand, but the comprehension syntaxenhancements seem to be bad for readability and "comprehending" what the codedoes.The call syntax part is a mixed bag on the one hand it is nice to be consistent with the extended possibilities in literals (flattening), but on the other hand there would be small but annoying inconsistencies anyways (e.g. the duplicate kwarg case above).* """ Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.) For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case. The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me. So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression). The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them: >>> xs = [1, 2, 3] >>> ys = ['abc', 'def'] >>> zs = [99] >>> [*xs, *ys, *zs] [1, 2, 3, 'abc', 'def', 99] >>> But now suppose you have a list of iterables: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xss[0], *xss[1], *xss[2]] [1, 2, 3, 'abc', 'def', 99] >>> Wouldn't it be nice if you could write the latter using a comprehension? >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs for xs in xss] [1, 2, 3, 'abc', 'def', 99] >>> This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs[:2] for xs in xss] [1, 2, 'abc', 'def', 99] >>> On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident). All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2. I would also like code reviewers (Benjamin?) to start reviewing the patch <http://bugs.python.org/issue2292>, taking into account that the comprehension part needs to be removed. -- --Guido van Rossum (python.org/~guido)

7 17

PEP 638: Syntactic macros
by Mark Shannon 06 Feb '23

06 Feb '23

Hi everyone, I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/ All comments and suggestions are welcome. Cheers, Mark

16 25

Switching to Discourse
by Petr Viktorin 26 Jan '23

26 Jan '23

Hello, Currently development discussions are split between multiple communication channels, for example: - python-dev and discuss.python.org for design discussions, - GitHub Issues and Pull Requests for specific changes, - IRC, Discord and private chats for real-time discussions, - Topic-specific channels like typing-sig. While most of these serve different needs, there is too much overlap between python-dev and discuss.python.org. It seems that for most people, this situation is worse than sticking to either one platform – even if we don't go with that person's favorite. The discuss.python.org experiment has been going on for quite a while, and while the platform is not without its issues, we consider it a success. The Core Development category is busier than python-dev. According to staff, discuss.python.org is much easier to moderate.. If you're following python-dev but not discuss.python.org, you're missing out. The Steering Council would like to switch from python-dev to discuss.python.org. Practically, this means: - Moving the required PEP announcements to discuss.python.org - Moving discuss.python.org up in the devguide communications page (https://devguide.python.org/communication/) - And that's it? I imagine that the mailing list will stay around for continuing past discussion threads and for announcements, eventually switching to auto-reject incoming messages with a pointer to discuss.python.org. To be clear, discuss.python.org allows editing posts, which is frankly handy for typos and clarifications. Editing alone should not be used for adding new info -- we should cultivate a culture of being friendly to mail users & notification watchers. This probably bears repeating in a few places. We're aware not everyone wants to use the discuss.python.org website, but there are some ways to avoid it: - For new PEPs, you can point your RSS client to https://www.python.org/dev/peps/peps.rss – it's not e-mail, but many email clients have RSS support. You can also watch the Steering Council issues on GitHub (https://github.com/python/steering-council/issues/) for important questions and discussions. - You can use discuss.python.org's “mailing list mode” (which subscribes you to all new posts), possibly with filtering and/or categorizing messages locally. However, we would like to know if this will pose an undue burden to anyone, if there are workflows or usage problems that we are not aware of. As mentioned, this is something the Steering Council thinks is a good idea, but we want to make sure we're aware of all the impact when we make the final decision. – Petr, on behalf of the Steering Council

43 114

How about using modern C++ in development of CPython ?
by redradist＠gmail.com 19 Oct '22

19 Oct '22

Hi all, What about of using modern C++ in developing CPython ? With C++ we can get the following benefits: 1) Readability - less code, most code is hidden by abstraction without losing performance 2) Maintainability - no code duplication in favor of using reusable classes 3) RAII - Resource Acquisition Is Initialization, predictable allocation and free resources ...

23 51

PEP 505 (None-aware operators) for Python 3.11
by Doug Swarin 21 Sep '22

21 Sep '22

Hello, I've been following PEP 505 (https://www.python.org/dev/peps/pep-0505/) since it was first proposed for Python 3.8. It's been deferred for some time and I'm interested in seeing it in Python 3.11, but I know there were also a number of objections which resulted in it being deferred (including by one of the original authors, Mr. Dower). I did email both Mr. Dower and Mr. Haase and they graciously gave me their permission to bring it up on this list for discussion and hopefully final pronouncement one way or the other. I personally believe that the PEP will result in a significant reduction in boilerplate code, and it is substantially similar to the same operators now found in a number of other languages, especially C# and JavaScript (https://wikipedia.org/wiki/Null_coalescing_operator and https://wikipedia.org/wiki/Safe_navigation_operator). I believe strong and valid arguments can be made about the use of None being a fundamental flaw in some types of coding (and that adding additional support for it to the language will increase the use of None in this way), but I also believe there are many use cases in programming where it is by far the simplest way to express various semantics, and the fact exists that None is already used extensively in large quantities of code, and further that there is already a great deal of code written to constantly test against None and break out of a statement without throwing an error. I also understand the argument that especially the maybe-dot (?.) and maybe-subscript (?[) operators can decrease readability of code and also believe these are valid arguments against it. While I believe the existence and use of these operators in other languages definitely helps the case that these can be used and understood successfully, I think it is entirely valid to either consider other syntax (though I prefer the chosen syntax of PEP 505), or even to reduce PEP 505 to having only the coalesce operator (??) and the maybe-assign operator (??=). Separately, I have implemented a pure-Python solution for PEP505 (which is definitely rather beta) which might help test the waters for a final implementation in CPython (though the CPython implementation would of course be much more efficient). It can be found at https://pypi.org/project/pep505/ Thanks, Doug

22 53

New contribution to Python built-in libraries.
by Kiruthik.M.R 11-B 29 Aug '22

29 Aug '22

Greetings, I'm Kiruthik from India. I've been studying coding for the past two years. I know that strings are immutable in Python. But, most of the people can't change the elements in the string as default. So, I came up with a solution by creating a function called *strangelt. *This function has three parameters in it and works faster. It has the ability to get a string from the user and make changes to the element of the string using index values. I found that it would be a pretty good solution for people's problems. I hereby kindly request you to include this function in Python built-in libraries from the release of the next version. Expansion of strangelt: *str*ing+ch*ang*e+*el*ement *Syntax:* *string_variable=strangelt(string,index,new value to be replaced)* Thank You, Kiruthik.

2 1

Adding new escapes to regex module
by MRAB 18 Aug '22

18 Aug '22

Other regex implementations have escape sequences for horizontal whitespace (`\h` and `\H`) and vertical whitespace (`\v` and `\V`). The regex module already supports `\h`, but I can't use `\v` because it represents `\0x0b', as it does in the re module. Now that someone has asked for it, I'm trying to find a nice way of adding it, and I'm currently thinking that maybe I could use `\y` and `\Y` instead as they look a little like `\v` and `\V`, and, also, vertical whitespace is sort-of in the y-direction. As far as I can tell, only ProgressSQL uses them, and, even then, it's for what everyone else writes as `\b` and `\B`. I want the regex module to remain compatible with the re module, in case they get added there sometime in the future. Opinions?

4 7