Mailman 3 August 2020 - Python-ideas

Reliable Ordering Of Sets
by Cade Brown Aug. 25, 2020

Aug. 25, 2020

Hello all, I have a suggestion/idea for the Python standard (and/or a CPython implementation detail, considering its memory impact): having sets behave similar to dictionaries in that they preserve first-appearance order (i.e. dictionaries are ordered by key which was first inserted). As is said in the Python standard, a 'set' is by definition unordered. Yet, internally they must be stored in some order, and when iterated upon, converted to string, etc, some ordering must be used. Right now, it is unspecified, which can lead to interesting results I noticed: >>> print ({1, 2, 3}, {3, 2, 1}) {1, 2, 3} {1, 2, 3} >>> print ({1, 5, 9}, {9, 5, 1}) {1, 5, 9} {9, 5, 1} While, obviously, things like `A == B` are not affected by this, many algorithms may affect their output if sets are treated as iterables (again, the argument could be made, as it was for 'dict' as well, that they should not treat the order as relevant at all). Further, I believe that having a specification regarding the order and some guarantees could be useful and important to cohesiveness of Python as a whole. Despite the obvious non-ordered nature of sets, I still think it is worth making them behave like `dict` objects (which are also, in some sense, 'unordered' by nature, and yet Python still specifies the order for 3.7+). I would like to suggests that `set` objects are ordered by insertion, so that: * Sets have a defined, repeatable order for maximum reproducibility (assuming the code generates the set in a stable way) * Tests which are outside of Python can do string comparisons and expect the same output each time (as with `dict` objects generated in a predictable way). * On a high-level note, Python specifies more exact behavior. It is my belief (and I'm sure many others share this as well) that unspecified/implementation-dependent/otherwise-undependable features should be eliminated and replaced with exact semantics that do not suprise users. Thoughts? Thanks, ~ Cade Brown Working on MAGMA <https://icl.cs.utk.edu/magma/> http://cade.site http://chemicaldevelopment.us

5 7

Re: Adding additionnal common request handlers to http.server
by Simon Aug. 24, 2020

Aug. 24, 2020

> Fair enough. I guess the real question is, how much advantage is > RESTRequestHandler over directly subclassing BaseHTTPRequestHandler? > Maybe it'd be worth it just to simplify that case. > > ChrisA Well, maybe an example would be more telling. Below is a basic usage of the handler I wrote : @RESTRequestHandler.route('/get/<obj_id:int>', methods=['GET']) def get_obj(request, obj_id): objs = [ {'bar': 'baz'}, {'lorem': 'ipsum'}, ] if obj_id > len(objs): raise HTTPStatusException(HTTPStatus.NOT_FOUND) return HTTPStatus.OK, objs[obj_id] @RESTRequestHandler.errorhandler('*') def handle_errors(r, exc): return exc.status_code, {'code': exc.status_code, 'message' : exc.message} with HTTPServer(('localhost', 8080), RESTRequestHandler) as httpd: httpd.serve_forever() This example demonstrates how such a handler would be used, using a syntax that flask users should be pretty familiar with. The handler integrates both a route and an errorhandler decorator, which can be used to add new routes to the API that's being built. It's not something that's in the standard library either. By the way, RESTRequestHandler subclasses SimpleHTTPRequestHandler. Overall the advantage is pretty clear, it provides a much simpler API for devs who wish to spin up an API, which would be the ultimate goal of this PEP. Not to mention, the handler I wrote is 184 lines of code, having to reimplement for every API is cumbersome.

2 1

Re: Adding additionnal common request handlers to http.server
by Simon Aug. 24, 2020

Aug. 24, 2020

> > > The standard library module http.server already has 2 request handlers, > with SimpleHTTPRequestHandler and CGIHTTPRequestHandler, the first serves > files from a directory, and the second executes CGI scripts. > > > > Two new handlers could be included in the standard library : > > > > - WebhookRequestHandler : Would spin up an HTTP server that one could > connect webhooks to. Webhooks are really useful when dealing with APIs. So > far, the third-party package, requests, is used when interacting with Web > APIs, but there is nothing in the standard library that implements the > webhook standard. > > > > - APIRequestHandler : This can be more controversial, especially with > third-party packages like Flask, starlette, and django that will be much > more powerful and secure than this, but this handler would be used to spin > up a server that would make it easy to serve a simple restful API. > > > > Not really in favour of "API" as a name. If you want a simple RESTful > API framework, I'd call it something with REST in the name. But I'm > not sure that that's needed, since you'd quickly outgrow a simple > class and need to go for Flask/Django/etc. > > Webhooks are very tempting. There's a lot to be said for properly > implementing this protocol in the standard library. It might be a bit > harder, though, since it basically wants asynchronous or threaded I/O. > Maybe this would be a good addition to asyncio? > > ChrisA My idea behind the APIRequestHandler (by the way, RESTRequestHandler might actually be better) was to provide devs with a simple interface that doesn't require any setup in order to spin up RESTful APIs for personal projects. You're ultimately right, though, in that to scale up, a more robust framework is needed (as I previously mentionned, flask, Django and starlette are much more powerful than a simple class), but they come with their own drawbacks : they are not in the standard library, and they are very heavy. For a dev that just wants to spin up a very basic API, in order to expose their IoT devices to their network for instance, even flask is overkill. And because of the size of these modules, they are unsuitable for embedded development (on a raspberry pi compute module/zero, or with microPython). In my opinion, REST is the best protocol to make two pieces of software communicate with each other, because of its simplicity, yet, the standard library lacks in that regard in my opinion.

3 2

Enhancement: Adding support for private and name mangled type hints in dataclasses module
by zachb1996＠yahoo.com Aug. 24, 2020

Aug. 24, 2020

I have a proposal for an addition to the dataclasses module that I think would make it easier to use private and name mangled variables. One of the benefits of the dataclass decorator is that it helps lessen the amount of code when you just need a simple constructor. A common pattern in python that isn't addressed by the current implementation is the pattern ```python class MyClass: def __init__(self, a: str, b: str): self._a = a self.__b = b ``` right now the only straightforward way of doing this with a dataclass is ```python @dataclass class MyClass: a: InitVar[str] b: InitVar[str] def __post_init__(self, a, b): self._a = a self.__b = b ``` My proposal would be to add two types to the dataclasses module, one called PrivateVar and one called MangledVar. These would add The above pattern (without the post_init ) using the syntax, ```python @dataclass class MyClass: a: PrivateVar[str] b: MangledVar[str] ``` I've already looked into what would need to be added to dataclasses to implement these and it's just a few lines of code since the class implementation would be very similar to InitVar. Is this something that other see the possibility of being added to python? (Also, I know name mangling isn't super common but I think it may as well be added alongside a private variable implementation))

2 2

Adding additionnal common request handlers to http.server
by Simon Aug. 24, 2020

Aug. 24, 2020

The standard library module http.server already has 2 request handlers, with SimpleHTTPRequestHandler and CGIHTTPRequestHandler, the first serves files from a directory, and the second executes CGI scripts. Two new handlers could be included in the standard library : - WebhookRequestHandler : Would spin up an HTTP server that one could connect webhooks to. Webhooks are really useful when dealing with APIs. So far, the third-party package, requests, is used when interacting with Web APIs, but there is nothing in the standard library that implements the webhook standard. - APIRequestHandler : This can be more controversial, especially with third-party packages like Flask, starlette, and django that will be much more powerful and secure than this, but this handler would be used to spin up a server that would make it easy to serve a simple restful API. I have already implemented both handlers in pure python (without third-party packages obviously), I'm just wondering if it's a good idea to make it a PEP, and implement these handlers as part of the python standard library.

2 1

raise TypeError when giving wrong type of argument in functions
by rawmin.rx＠gmail.com Aug. 24, 2020

Aug. 24, 2020

I wanted to know if there could be TypeError when giving wrong type for arguments in functions (This usually happens when using other module's function) e.g: def sum(nom1: int, nom2: int): nom = nom1 + nom2 return nom print(sum('hello',2)) if you run this code you will get TypeError for line 2 because 'you can only concatenate str (not "int") to str' But what am i saying is can it raise TypeError for line 4 because I gave 'hello' as nom1 and nom1 should be int Now if I want to check arguments types I should use: def sum(nom1: int, nom2: int): if isinstance(nom1, int) and isinstance(nom1, int): nom = nom1 + nom2 return nom else: raise TypeError('nom1 and nom2 should be int') print(sum('hello',2)) But if they can add what am I am i saying it can decrease lines of this function by 50% and also function author should not worry about checking types anymore! (I know I could use 'assert' but I just wanted to write it as simple as possible)

3 3

Improve error message for trying to import itself
by Gustav O Aug. 24, 2020

Aug. 24, 2020

If a file named "tkinter.py" is created and tkinter is imported and used within the file, the following exception is raised: "AttributeError: partially initialized module 'tkinter' has no attribute 'Tk' (most likely due to a circular import)" I've spoken to multiple beginners who got stuck on this and just couldn't figure out what was happening. Just like the error message tells us, it's due to a "circular import". In other words, we're trying to import the very file that is doing the import. To make the message easier to understand, I propose that we change the message to something that mentions that the program is trying to import itself. I think the following would be an option: "ImportError: partially initialized module 'tkinter' can't import itself" I can't think of any reason to import itself in a way like this, but even if there is a valid reason, the error message could probably be improved. Of course, this applies to more than just tkinter.

8 26

basic matrix object
by Stefano Borini Aug. 24, 2020

Aug. 24, 2020

Excuse me if I am out of the loop and this is already available, but I haven't seen it and googling is not exactly easy as numpy introduces considerable noise. With the introduction of the statistics module, the standard library provides basic statistical functions that can be useful in many scenarios, including teaching. The math module has plenty of mathematical functions that are very interesting, but no Matrix object. When newcomers want to have a matrix object, they end up implementing a list of lists (as suggested by the documentation, e.g. see https://docs.python.org/3/tutorial/datastructures.html?highlight=matrix#nes… and https://docs.python.org/3/faq/programming.html?highlight=matrix#how-do-i-cr…) but it's ambiguous (is each entry a row or a column?), and does not enforce types or equal length of each entry. Generally, when a Matrix object is needed, numpy is the point of reference, but numpy requires understanding pip, installing modules, maybe creating a virtual environment and finally understanding numpy. I would propose a simple, straightforward, low performance object to perform trivial operations such as matrix multiplication, transposition, addition, linear problem solving, determinant. The absolute bare minimum for an introductory linear algebra course. The syntax should mimic numpy to train newcomers to the numpy syntax whenever possible, and of course should implement the @ operator. There is already a similar entity in the "array" module, which is a simple version of a numpy array, except that is only one-dimensional. -- Kind regards, Stefano Borini

13 28

Add parse_duration to datetime - a golang like fucntion to parse duration
by Oz Aug. 23, 2020

Aug. 23, 2020

Hi Everyone, I really like how go parses durations: ``` hours, _ := time.ParseDuration("10h") complex, _ := time.ParseDuration("1h10m10s") micro, _ := time.ParseDuration("1µs") // The package also accepts the incorrect but common prefix u for micro. micro2, _ := time.ParseDuration("1us") ``` Consider the example in https://docs.python.org/3/library/datetime.html#timedelta-objects: ``` >>> from datetime import timedelta >>> delta = timedelta( ... days=50, ... seconds=27, ... microseconds=10, ... milliseconds=29000, ... minutes=5, ... hours=8, ... weeks=2 ... ) ``` With a go like parsing it would be: ``` >>> datetime.parse_duration("2w50d8h5m27s10ms2000us") ``` Go lang's implementation only supports "ns", "us" (or "µs"), "ms", "s", "m", "h", but that does not mean Python has to restirct itself to these units. There a few similar pypi packages and SO answers with similar implementations, so there a basis to start from: https://github.com/wroberts/pytimeparse https://github.com/oleiade/durations But the code can be as simple as this: from datetime import timedelta import re ``` regex = re.compile( r'((?P<weeks>[\.\d]+?)w)?' r'((?P<days>[\.\d]+?)d)?' r'((?P<hours>[\.\d]+?)h)?' r'((?P<minutes>[\.\d]+?)m)?' r'((?P<seconds>[\.\d]+?)s)?' r'((?P<microseconds>[\.\d]+?)ms)?' r'((?P<milliseconds>[\.\d]+?)us)?$' ) def parse_time(time_str): """ Parse a time string e.g. (2h13m) into a timedelta object. Modified from virhilo's answer at https://stackoverflow.com/a/4628148/851699 :param time_str: A string identifying a duration. (eg. 2h13m) :return datetime.timedelta: A datetime.timedelta object """ parts = regex.match(time_str) assert parts is not None, "Could not parse any time information from '{}'. Examples of valid strings: '8h', '2d8h5m20s', '2m4s'".format(time_str) time_params = {name: float(param) for name, param in parts.groupdict().items() if param} return timedelta(**time_params) print(repr(parse_time("2w50d8h5m27s10ms2000us"))) ``` This is an extended version of: https://stackoverflow.com/a/51916936/492620\ Is there someone willing to sponsor a PR for adding this to the STL? I'm willing to work on the code as well as the tests and documentation (I contributed small changes to docs.python.org and the `calendar` module in the past). Best regards, Oz

7 11

PEP 472 - regarding d[x=1, y=2] and similar
by Jonathan Fine Aug. 22, 2020

Aug. 22, 2020

I'd like to sound out consensus regarding mapping access, where none of the keys are positional. In particular, I suggest that PEP 472 allow syntax and semantics such as >>> d[x=1, y=2] = 42 >>> d[x=1, y=2] 42 and ask whether the class >>> X = type(d) should be part of standard Python. NO ARGUMENTS At present, item access requires an argument, as a matter of syntax. >>> d[] SyntaxError: invalid syntax Compare this to >>> fn() NameError: name 'fn' is not defined I'd like d[] to become valid syntax. SEMANTICS OF NO ARGUMENTS I can see two basic ways of allowing no arguments. One is for the interpreter to construct an object that is the argument passed to __getitem__ and so forth. The other is to not pass an argument at all. I see this as a secondary question. NO POSITIONAL ARGUMENTS I'd like >>> d[x=1, y=2] to be valid syntax. It's not clear to me that all agree with this. Even if there are no objections, I'd like positive confirmation. CONSEQUENCE Suppose >>> d[x=1, y=2] is valid syntax. If so, then there is I think consensus that >>> d[x=1, y=2] = 42 >>> d[x=1, y=2] 42 can be implemented, where d is an instance of a suitable class. Otherwise, what's the point? QUESTION Suppose we have >>> d[x=1, y=2] = 42 >>> d[x=1, y=2] 42 where d is an instance of a suitable class X that has no special knowledge of keywords. In other words, we also have for example >>> d[a='alpha', g='gamma', z=12] = 'cheese' >>> d[a='alpha', g='gamma', z=12] 'cheese' My question is this: Should such a class >>> X = type(d) be part of standard Python, as part of PEP 472? (My answer is: Yes, it should be in standard Python.) At this time, I'm interested in canvassing opinions. Discussion of different opinions perhaps can take place later, or elsewhere. My main concern is to know if there is at present a rough consensus regarding the above. -- Jonathan

16 83