Mailman 3 April 2019 - Python-ideas

Using rightarrow "->" for typing annotation of functions
by Vaibhav Karve 26 Apr '19

26 Apr '19

(Note: This idea is about a particular static typecheking (typing?) annotation syntax). The idea is that currently the use of the "->" (right arrow) is restricted to only function definition annotation. Can we extent it to declaration of type for functions even outside their definitions? Example: Currently we write: f: Callable[[int, Dict[str, int]], str] # declaring the type of some fake function This would be much cleaner if we could write: f: int -> Dict[str, int] -> str # One of the possibilities or even: f: int, Dict[str, int] -> str # Another possibility I have no idea how this will affect the existing syntax (and if this will have any bad repercussions/notational misuse). I just thought it would be nicer to: a) Not have to spell out Callable b) Not have to use all those square brackets c) Have the same notation work for both the function annotation as well as for declaring the type. This is my first time posting an idea to python-ideas. So apologies if i am not following some conventions that i might not be aware of. Vaibhav Karve

7 6

Add a `dir_fd` parameter to `os.truncate`?
by Sebastian M. Ernst 25 Apr '19

25 Apr '19

Hi everyone, many methods in `os` have a `dir_fd` parameter, for instance `unlink` [1]: ```python os.unlink(path, *, dir_fd=None) ``` The `dir_fd` parameter [2] allows it to describe paths relative to directory descriptors. Otherwise, relative paths are relative to the current working directory. The implementation of `truncate` in `os` does not have this parameter [3]: ```python os.truncate(path, length) ``` On POSIX-like systems for instance, the `os` module actually imports [4] this function from the `posix module`. There, one can see that it [5] just calls the `truncate` system call [6]. This kind of implicitly explains why there is no `dir_fd` parameter: There is no such thing like a `truncateat` system call, which would be required for handling the `dir_fd` parameter [7, 8]. However, it is possible to work around this limitation: ```python def truncate(path, length, dir_fd = None): if dir_fd is None: return os.truncate(path, length) else: fd = os.open(path, flags = os.O_WRONLY, dir_fd = dir_fd) ret = os.ftruncate(fd, length) os.close(fd) return ret ``` Why not add a convenience function or wrapper like above to the `os` module, which closes this gap and is more consistent with other methods? Best regards, Sebastian [1] https://docs.python.org/3/library/os.html#os.unlink [2] https://docs.python.org/3/library/os.html#dir-fd [3] https://docs.python.org/3/library/os.html#os.truncate [4] https://github.com/python/cpython/blob/3.7/Lib/os.py#L135 [5] https://github.com/python/cpython/blob/3.7/Modules/posixmodule.c#L9042 [6] https://github.com/python/cpython/blob/3.7/Modules/posixmodule.c#L9079 [7] https://stackoverflow.com/q/52871892/1672565 [8] https://stackoverflow.com/q/55765181/1672565

2 1

Re: [Python-ideas] What are the strong use cases for str.rindex()? (John Lin)
by Thautwarm Zhao 25 Apr '19

25 Apr '19

IMO, there're lots of use cases in parsing related stuffs, which requires rindex a lot, say, when you have generated a tokenizer which might across multiple lines: line 8: X """ line 9: line 10: """ In this case, we need to extract 2 tokens X and , a multiline whitespace string. After getting each token we're to compute/update the current column and line number. If the line number gets advanced then we use rindex('\n') to help with updating the new column number, otherwise, col_offset += len(current_token) . However, the reason why we don't need list.rindex but do for str.rindex is simple I'd say: str is immutable and has no O(1) reverse method. On the other hand, when it comes to list, you can use list.index after list.reverse, and after a bunch of operations you can resume the state by invoking list.reverse again. On Wed, Apr 24, 2019, 12:11 AM <python-ideas-request(a)python.org wrote: > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > Today's Topics: > > 1. What are the strong use cases for str.rindex()? (???) > > > > ---------- Forwarded message ---------- > From: "林自均" <johnlinp(a)gmail.com> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Tue, 23 Apr 2019 14:44:25 +0800 > Subject: [Python-ideas] What are the strong use cases for str.rindex()? > Hi all, > > I found that there are str.index() and str.rindex(), but there is only > list.index() and no list.rindex(). So I filed the issue > https://bugs.python.org/issue36639 to provide list.rindex(). However, the > issue was rejected and closed with the comment: > > > There were known, strong use cases for str.rindex(). The list.rindex() > method was intentionally omitted. AFAICT no compelling use cases have > arisen, so we should continue to leave it out. In general, we don't grow > the core APIs unnecessarily. > > However, I am not sure what the known, strong use cases for str.rindex() > are. Why doesn't the strong use cases apply on list.rindex()? Could anyone > give me some examples? Thanks. > > Best, > John Lin > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas >

2 1

CPython Bytecode Assembler
by Batuhan Osman Taşkaya 24 Apr '19

24 Apr '19

Hello, Currently it is hard to assemble cpython bytecode without help of 3rd party libraries (like: vstinner/bytecode). I'm proposing an assembler to standard library and an API to cpython's peephole optimizer. Also an interface like `ast.NodeVisitor` and `ast.NodeTransformer` for bytecode objects will may be handy. It would help if you are doing; - Runtime patching - Specific optimizations at bytecode level

4 4

contains_any_in and contains_all_in
by João Matos 24 Apr '19

24 Apr '19

Hello, If we want to check if a string contains any/all of several other strings we have to use several or/and conditions or any/all. For any: if ('string1' in master_string or 'string2' in master_string or 'string3' in master_string): or if any(item in master_string for item in ['string1', 'string2', 'string3']): For all: if ('string1' in master_string and 'string2' in master_string and'string3' in master_string): or if all(item in master_string for item in ['string1', 'string2', 'string3']): I suggest adding some "sugar" to make it more readable by adding contains_any_in and contains_all_in to look like this For any: if master_string contains_any_in ['string1', 'string2', 'string3']: For all: if master_string contains_all_in ['string1', 'string2', 'string3]: What do you think? Thanks, JM

4 7

Catching the return value of a generator at the end of a for loop
by Stefano Borini 17 Apr '19

17 Apr '19

given the following code def g(): yield 2 yield 3 return 6 for x in g(): print(x) The output is obviously 2 3 As far as I know, there is currently no way to capture the StopIteration value when the generator is used in a for loop. Is it true? If not, would a syntax like: for x in g() return v: print(x) print(v) # prints 6 be useful? It would be syntactic sugar for (corner cases omitted) def g(): yield 2 yield 3 return 6 it = iter(g()) while True: try: x = next(it) except StopIteration as exc: v = exc.value break else: print(x) print(v) -- Kind regards, Stefano Borini

5 6

Logical tracebacks
by Antoine Pitrou 16 Apr '19

16 Apr '19

Hello, I apologize because I'm only going to throw a very vague idea and I don't currently have time or motivation to explore it myself. But I think it may prove interesting for other people and perhaps spur some concrete actionable proposal. With the growing complexity of Python software stacks, the length of tracebacks is continuously growing and is frequently making debugging errors and issues more tedious than it should be. This is a language-agnostic problem. Java software is often mocked for its ridiculously long tracebacks, but Python might come close in the future. Especially since Python is often the a language of choice for non computer science professionals, including but not only as a teaching language, this would be a problem worth solving. We already recognized the issue some years ago, and even implemented a focussed fix for one specific context: the elision of importlib frames when an import error occurs: https://bugs.python.org/issue15110 However, there are many contexts where implementation details would benefit from being hidden from tracebacks (the classical example being the internals of framework or middleware code, such as Django, Dask, etc.). We would therefore have to define some kind of protocol by which tracebacks can be enumerated, not only as frames, but as logical execution blocks, comprised of one or several frames each, whose boundaries would reflect the boundaries of the various logical execution layers (again: framework, middleware...) involved in the frame stack. We would probably also need some flag(s) to disable the feature in cases where the full stack frame is wanted (I imagine elaborate UIs could also allow switching back and forth from both representations). This would need a lot more thinking, and perhaps exploring what kind of hacks already exist in the wild to achieve similar functionality. Again, I'm just throwing this around for others to play with. Regards Antoine.

6 5

Syntax for allowing extra keys when unpacking a dict as keyword arguments
by Viktor Roytman 14 Apr '19

14 Apr '19

Currently, unpacking a dict in order to pass its items as keyword arguments to a function will fail if there are keys present in the dict that are invalid keyword arguments: >>> def func(*, a): ... pass ... >>> func(**{'a': 1, 'b': 2}) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: func() got an unexpected keyword argument 'b' The standard approach I have encountered in this scenario is to pass in the keyword arguments explicitly like so func( a=kwargs_dict["a"], b=kwargs_dict["b"], c=kwargs_dict["c"], ) But this grows more cumbersome as the number of keyword arguments grows. There are a number of other workarounds, such as using a dict comprehension to select only the required keys, but I think it would be more convenient to have this be a feature of the language. I don't know what a nice syntax for this would be, or even how feasible it is.

11 20

Re: [Python-ideas] Syntax for allowing extra keys when unpacking a dict as keyword arguments
by Viktor Roytman 12 Apr '19

12 Apr '19

> > That seems to me to be quite different issue. Just throwing invalid stuff > on the ground in this scenario will avoid a crash but lose data. This seems > much worse to me than the crash. > Throwing it away does seem extreme. Maybe something that indicates what's left over? In other words: result, leftover_kwargs = func(<some syntax for this>kwargs) or result = func(<some syntax for this>kwargs) assert kwargs == {'whatever is': 'left over'}

1 0

Re: [Python-ideas] Syntax for allowing extra keys when unpacking a dict as keyword arguments
by Lucas Bourneuf 12 Apr '19

12 Apr '19

Hello ! I made *fief*, a small python package allowing just that using a decorator: from fief import filter_effective_parameters as fief @fief def func(a, b): # some implementation # and then, use it as you want to: func(**MY_BIG_CONFIG_DICT_WITH_MANY_WEIRD_KEYS) The code is quite simple. You may want to use it, with modifications (i didn't touch the code for months, maybe years ; it could probably be improved now). Link: https://github.com/aluriak/fief The code: def filter_effective_parameters(func): """Decorator that filter out parameters in kwargs that are not related to any formal parameter of the given function. """ @wraps(func) def wrapper(*args, **kwargs): formal_parameters = frozenset(signature(func).parameters.keys()) return func(*args, **{ arg: value for arg, value in kwargs.items() if arg in formal_parameters }) return wrapper Best regards, --lucas ----- Mail original ----- > De: "Viktor Roytman" <viktor.roytman(a)gmail.com> > À: "python-ideas" <python-ideas(a)googlegroups.com> > Envoyé: Vendredi 12 Avril 2019 18:01:43 > Objet: Re: [Python-ideas] Syntax for allowing extra keys when unpacking a dict as keyword arguments > I could see this being an option, but to someone unfamiliar with it, it > might seem strange that * unpacks iterables, ** unpacks dicts, and *** is a > special thing only for keyword arguments that mostly behaves like **. > > On Friday, April 12, 2019 at 11:26:37 AM UTC-4, Bruce Leban wrote: >> >> >> On Fri, Apr 12, 2019, 8:12 AM Viktor Roytman <viktor...(a)gmail.com >> <javascript:> wrote: >> >>> >>> >>> func(**{'a': 1, 'b': 2}) >>> Traceback (most recent call last): >>> File "<stdin>", line 1, in <module> >>> TypeError: func() got an unexpected keyword argument 'b' >>> >> >> Perhaps func(***kws)? >> >> I think this is a real problem given the frequent convention that you can >> freely add fields to json objects with the additional fields to be ignored. >> > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/

3 3