Mailman 3 August 2020 - Python-ideas

Idea: Extend "for ... else ..." to allow "for ... if break ..." else
by Jonathan Fine Aug. 7, 2020

Aug. 7, 2020

There's been a discussion of the "for ... else ..." construction in Python. Here's a suggestion as to how we might improve matters, and perhaps usefully extend the language. I hope it might benefit some, without harming anyone else. Aside: I've not read the whole of the previous discussion, and it may be that some or all of what I say has already been expressed there. So apologies for any missing credits to others. The errors, I'm responsible for. Here's what happens at present. for i in items: LOOP_BODY else: print('exit via StopIteration') I find the following clearer (where 'if break' is a new language 'compound keyword'). for i in items: LOOP_BODY if break: pass else: print('exit via StopIteration') This would now allow us to write for i in items: LOOP_BODY if break: print('exit via break') else: print('exit via StopIteration') Sometimes a loop has multiple break commands, which have different purposes. So a further extension might be for i in items: LOOP_BODY if break left: print('exit via break left') elif break right: print('exit via break right') elif break: print('exit via some other break') else: print('exit via StopIteration') To allow this to work, we extend the language to allow commands such as break left break right as well as the usual break Here the identifiers 'left' and 'right' look like variable names, but in fact they are labels. The compiler can, and I think should, produce a hard-coded jump from command break right to the code block under elif break right: This is the basic idea, keeping the syntax close to Python as it now is (but of course adding to the semantics). Some programmers, and I think I'm one, might want a clearer distinction between for i in items: LOOP_BODY if break: do_something and for i in items: LOOP_BODY if broke: do_something For this reason I tentatively suggest instead for i in items: LOOP_BODY case: break left: print('exit via break left') break right: print('exit via break right') break: print('exit via some other break') else: print('exit via StopIteration') Some people have asked a built-in means to detect, at the end of the iteration, whether theitems was empty to begin with. The syntax could easily provide this, for example via case: zero: print('no items to process') However, if we adopt this perhaps we'd also like to retain the simpler for ... if break ... else ... construct. I hope this helps. -- Jonathan

7 19

Improve handling of Unicode quotes and hyphens
by Steve Barnes Aug. 6, 2020

Aug. 6, 2020

Hi All, Apologies if this has already been discussed to death. Python 3 allows Unicode characters in strings and identifiers but the actual quotation marks are only accepted in plain ASCII, i.e. the following all successfully initialise strings: ``` S1 = "Double Quoted" # Opened and closed with chr(34)0x22 S2 = 'Single Quoted' # Opened and closed with chr(39)0x27 ``` But the following all result in an error - "SyntaxError: invalid character in identifier": ``` S1 = "Double Quoted" # Opened with \u201c and closed with \u201d S2 = 'Single Quoted' # Opened with \u2018 and closed with \u2019 ``` To the experienced eye, and depending on the character font used, it is "obvious" what the problem is! The wrong quotation marks were used. The big problem, especially for beginners, is that the same keys were typed, just in the "wrong" editor or even the wrong editor mode or context I have found that in Outlook if the font is FixSys or I am replying to a plain text email it is fine but otherwise it is "helpful" - unfortunately, especially on Windows, "wrong" editors abound and include, but are not limited to, MS-Outlook, MS-Word, some online editing environments such as Quora. On top of that is the helpful substitution of a m-hyphen for minus when you press space a word later so: A = 3 - 2 # With a space syntax error due to \u2013 A = 3 - 2 # No Space or CR after I last typed it is OK as 0x2d Use cases that catch people out: 1. Sending a code snipped by email using Outlook 2. User manuals written in MS-Word - (many peoples work environment) 3. Articles published on Quora - people expect to be able to copy and paste the code for some reason. I am sure that many us have encountered these issues or similar. What can be done? 1. Persuade Microsoft, and others, to stop being so helpful by default - good luck with that! 2. Tell all users that they need to use a "proper" editor or IDE - This seems like adding an additional barrier to new & casual users. 3. Better yet tell them to use a "proper" OS like .... - At the very least many of us have to use Windows at work. 4. Start accepting hyphens as minus & Unicode quotation marks - this would be the ideal answer for pasted code but has a lot of possible things to iron out such as do we require that the quotes match and are in the typographically correct order. It is also quite a big & complex change to the python interpreter. 5. Normalise the input to the python interpreter (at least for these characters and possibly a few others) so that entering or reading from a file S1 = "Double Quoted" becomes S1 = "Double Quoted", etc. - this should be a easier change to the interpreter but, from a purist point of view, could be said to make us as bad as the others because we are not honouring what the user entered. 6. Change the error message "SyntaxError: invalid character in identifier" to include which character and it's Unicode value so that it becomes "SyntaxError: invalid character 0x201c " in identifier" - this is almost certainly the easiest change and fits well with explicit is better than implicit but still leaves it to the user to correct the erroneous input (which could be argued is both good and bad). I would like to suggest that an incremental approach might be the best - clarifying the existing error message being the thing that should not break anything and either substituting for problem characters or processing them "properly" as a later enhancement. Steve Barnes

16 46

New monotonic clock that tracks across system suspend
by Rishav Kundu Aug. 6, 2020

Aug. 6, 2020

Consider the monotonic clock defined in pytime.c. Whether or not it tracks time across system suspend is implementation-defined (and not documented either). PEP 418 [1] has details along with the raw system APIs used. We could introduce two new monotonic clocks that distinguish between whether they track across system suspend or not. These clocks exist on at least Windows, macOS, and Linux. [1] https://www.python.org/dev/peps/pep-0418/#id56 — Rishav P.S.: This is my first time submitting to the Python mailing lists. Suggestions for improvement are appreciated :)

2 1

Introduce a boundary parameter for typing.get_type_hints when used with a class object
by Dominik Vilsmeier Aug. 5, 2020

Aug. 5, 2020

In one of my projects I'm reusing class-level type annotations to identify relevant attributes for serialization, e.g. similar to the following: attrs = {name: getattr(obj, name) for name in get_type_hints(type(obj))} This is convenient because it merges the type annotations from the various stages in the class hierarchy, e.g. class Base: a: int class Derived(Base): b: str results in `attrs == dict(a=..., b=...)`. However it becomes inconvenient if external base classes are involved that define their own, unrelated type annotations, e.g. class External: # from some other distribution unrelated: float class Base(External): a: int It would be helpful if `get_type_hints` had a `boundary` parameter that, when used with a class object, determines the upper boundary for the MRO. So it could be used in the following way: get_type_hints(type(derived_obj), boundary=Base) to exclude any type annotations further up the class hierarchy (including the ones from `Base`). Regarding the implementation this would effectively skip over base classes in the reverse MRO until it reaches the `boundary`. What do you think?

3 4

Re: Idea: Extend "for ... else ..." to allow "for ... if break ..." else
by Jonathan Fine Aug. 5, 2020

Aug. 5, 2020

Hi Rob You wrote: So: You're asking that the bytecode generated for the for-loop depends > on something that happens (or not) after the end of the for-loop body > (which could be arbitrarily long). > > I speak from ignorance, but I suspect that even with the new parser, > which I am reliably informed can make the tea and implement world peace, > that would be asking a lot. > I'm also speaking from ignorance. However, here I'm the optimist. Ignorance is bliss. You're the pessimist. The devil is in the details. Until we have knowledge and wisdom on this, I think we need both points of view. So I'm grateful for your https://en.wikipedia.org/wiki/Grain_of_salt (American English, pinch of salt in British English). I hope it makes my dish taste better! -- Jonathan

4 4

How to prevent shared memory from being corrupted ?
by Vinay Sharma Aug. 4, 2020

Aug. 4, 2020

Problem: Currently, let’s say I create a shared_memory segment using mulitprocessing.shared_memory.SharedMemory <https://docs.python.org/3.10/library/multiprocessing.shared_memory.html> in Process 1 and open the same in Process 2. Then, I try to write some data to the shared memory segment using both the processes, so for me to prevent any race condition (data corruption), either these operations must be atomic, or I should be able to lock / unlock shared memory segment, which I cannot at the moment. I earlier posted a solution <https://mail.python.org/archives/list/python-ideas@python.org/thread/X4AKFF…> to this problem, which received positive response, but there weren’t many responses to it, despite the fact this problem makes shared_memory practically unusable if there are simultaneous writes. So, the purpose of this post is to have discussion about the solution of the same. Some Solutions: 1. Support for shared semaphores across unrelated processes to lock/unlock shared memory segment. --> More Details <https://mail.python.org/archives/list/python-ideas@python.org/thread/X4AKFF…> 2. Let the first bit in the shared memory segment be the synchronisation bit used for locking/unlocking. --> A process can only lock the shared memory segment if this bit is unset, and will set this bit after acquiring lock, and similarly unset this bit after unlocking. Although set/unset operations must be atomic. Therefore, the following tools can be used: type __sync_add_and_fetch (type *ptr, type value, ...) type __sync_sub_and_fetch (type *ptr, type value, ...) type __sync_or_and_fetch (type *ptr, type value, ...) Documentation of these can be found at below links: https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html#g_t_005f_00… <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html#g_t_005f_00…> https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html#g_t_005f_… <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html#g_t_005f_…> Any other ideas/solutions are very welcome.

6 20

Default behavior for random.sample when no k
by Ram Rachum Aug. 4, 2020

Aug. 4, 2020

When writing some code now, I needed to produce a shuffled version of `range(10, 10 ** 5)`. This is one way to do it: shuffled_numbers = list(range(10, 10 ** 5)) random.shuffle(shuffled_numbers) I don't like it because (1) it's too imperative and (2) I'm calling the list "shuffled" even before it's shuffled. Another solution is this: shuffled_numbers = random.sample(range(10, 10 ** 5), k=len(range(10, 10 ** 5))) This is better because it solves the 2 points above. However, it is quite cumbersome. I notice that the `random.sample` function doesn't have a default behavior set when you don't specify `k`. This is fortunate, because we could make that behavior just automatically take the length of the first argument. So we could do this: shuffled_numbers = random.sample(range(10, 10 ** 5)) What do you think? Thanks, Ram.

8 14

Filtered wildcard imports?
by Sebastian M. Ernst Aug. 3, 2020

Aug. 3, 2020

Hi all, yet another (possibly bad?) idea from day-to-day work ... I occasionally need to import a lot of "stuff" from certain modules. The "stuff" is usually following a pattern. E.g. I have modules that (mostly) collect special exception classes and I usually need them all in one push - but nothing else from those modules. So a general wildcard import is a bad idea in a lot of ways: ```python from somemodule import * ``` What I usually do instead is something along the following lines: ```python import somemodule as _somemodule _globals = globals() for _attr in dir(_somemodule): # looking for "stuff" if not _attr.endswith('Error'): # the filter - this is not "stuff" continue _globals[_attr] = getattr(_somemodule, _attr) # the "import" del _globals, _attr, _somemodule # some cleanup ``` The above selects and "imports" everything ending on "Error" into the global namespace. What I would love to do instead is something like a "filtered wildcard import", maybe through regular expressions or shell-like matching: ```python from somemodule import *Error ``` Best regards, Sebastian

5 4