Mailman 3 April 2020 - Python-ideas

zip should return partial results in StopIteration
by Soni L. 23 Apr '20

23 Apr '20

I feel like zip could return partial results: try: next(zip([0], [])) except StopIteration as exc: assert StopIteration.args == (0,) how much would this break?

10 24

list.append(x) could return x
by J. Pic 20 Apr '20

20 Apr '20

Hi all, Currently, list.append(x) mutates the list and returns None. It would be a little syntactic sugar to return x, for example: something = mylist.append(Something()) What do you think ? Thanks in advance for your replies -- ∞

7 10

collections.UpdateDict, collections.InsertDict, collections.InsertOrIgnoreDict
by Soni L. 20 Apr '20

20 Apr '20

currently dicts have insert-or-update semantics, e.g.: >>> dict([((), 1), ((), 2)]) {(): 2} this is fine. however, in many cases insert or insert-or-ignore are more useful: (hypothetical examples) >>> InsertOrIgnoreDict([((), 1), ((), 2)]) {(): 1} >>> InsertDict([((), 1), ((), 2)]) Traceback: [...] ValueError: key exists these are useful when dealing with custom config file formats, config overrides, among other things. additionally we could also get an UpdateDict, a *view* into a dict that only allows updating existing entries: >>> a = {'a': 1} >>> b = {'a': 2, 'b': 3} >>> UpdateDict(a).update(b) >>> a {'a': 2} but I don't see an obvious usefulness to this one. it'd just be for consistency with other SQL-esque dict wrappers (like the above InsertOrIgnoreDict and InsertDict).

5 8

TLS session resumption
by Ander Juaristi 18 Apr '20

18 Apr '20

Hi list, I haven't seen this being discussed, but TL;DR I just want to propose to extend the ssl module so that TLS session state can be serialized to disk. I see a similar issue was being discussed here [0], but cannot tell why it (apparently) didn't get into. The link to the patch is broken. TLS session resumption is currently supported, but only within the same process. To the best of my knowledge, there is no way to save the TLS session to a file to resume the TLS session later on. Please tell me how this is done if I'm wrong. # # open a TLS connection # with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock: with context.wrap_socket(sock, server_hostname=hostname) \ as ssock: ssock.connect((hostname, 443)) ssl_session = ssock.session # get TLS session # # later on, open a new connection resuming session # with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock: with context.wrap_socket(sock, server_hostname=hostname, session=ssl_session) as ssock: ssock.connect((hostname, 443)) print("-- TLS session %s resumed --" % ("WAS" if ssock.session_reused else "WAS NOT") However TLS sessions live for hours, if not days, and hence it may make sense to persist them to disk for later retrieval. I think this is a real world use case though I have not any numbers to back this up. The values currently exposed by SSLSession (id, time, timeout) are not enough to resume a TLS session. There are a number of parameters that need to be stored to resume a TLS session at a later time, some of which are not currently retrievable from python's ssl module. For instance, the master secret. I'm not a fan of exposing the master secret directly with a getter as that could be and will definitely be misused by someone in spite of the bright red warnings, leading to disaster. What OpenSSL currently does, as do other implementations, is to allow the SSL_SESSION object to be serialized, with i2d_SSL_SESSION(). This function will return a DER-encoded ASN.1 object containing all the needed state to resume the TLS session. I do believe exposing this object from the ssl module (as an opaque bytes object) would be a useful compromise with very little chance of misuse. One possible incarnation of this idea would be add two new methods to SSLSession, to_der(), and from_der(): class SSLSession: def to_der(): """ Returns OpenSSL's DER-encoded SSL_SESSION object """ @staticmethod def from_der(der_bytes): """ Creates a new SSLSession object from the given DER-encoded SSL_SESSION object """ If this has already been discussed (likely), what are the reasons it was rejected? Thanks! [0] https://bugs.python.org/issue19500

5 6

Re: Keyword arguments self-assignment
by Steven D'Aprano 18 Apr '20

18 Apr '20

On Fri, Apr 17, 2020 at 03:33:56PM -0400, David Mertz wrote: > I think I'm the first person to mention Lisp symbols. Maybe not > though. I know it's not a competition, but I think Stephen Turnbull beat you by five hours :-) (At least if I can do timezone calculations correctly.) -- Steven

2 1

Proposed class for collections: dynamicdict
by Steele Farnsworth 17 Apr '20

17 Apr '20

I have implemented a class in the C code of the collections module which has similar behavior to the defaultdict class. This class, dynamicdict, supplies values for keys that have not yet been added using a default factory callable, but unlike defaultdict, the missing key itself is passed to the callable. This code can be seen here: https://github.com/swfarnsworth/cpython/blob/3.8/Modules/_collectionsmodule… While this does introduce a lot of redundant code, I'm not sure how it could be done without copying the implementation of the defaultdict class and adjusting how the default factory is called. For example, I don't believe that it's possible to support both behaviors within the defaultdict class without breaking backwards compatibility or adding another parameter to the constructor for the defaultdict class. I don't know if a PEP is needed for this to be added to the standard library, or if it's something that anyone other than myself wants to see added, but I would be happy to further explain the concept, implementation, potential use cases, or anything else that might work towards the adoption of this feature. Regards, Steele

6 13

Re: Should dataclass init call super?
by Christopher Barker 16 Apr '20

16 Apr '20

On Mon, Apr 13, 2020 at 9:22 PM Neil Girdhar <mistersheik(a)gmail.com> wrote: > Cool, thanks for doing the relevant research. > For my part, I'd like to see an aeefort to move dataclasses forward. Now that they are in the standard library, they do need to remain pretty stable, but there's still room for extending them. But it's a bit hard when ideas and PRs are mingled in with everything else Python. Maybe a gitHub repo just for dataclasses? @Eric V. Smith <eric(a)trueblade.com>: what do you think? IS there a way to keep them moving forward? > I'm just going to swap dataclasses for actual classes whenever I need > inheritance. It seems like a pity though. > For my part, I've gotten around it (for a different reason...) with an extra inheritance dance: @dataclass MyClassBase: .... MyRealClass(MyClassBase, Some_other_baseclass): def __init__(self., args, kwargs): dc_args, dc_kwargs = some_magic_with_self.__dataclass_fields__ MyClassBase.__init__(dc_args, dc_kwargs) super().__init__(self, args, kwargs) and you could put that __init__ in a mixin to re-use it. Or, frankly, just give your dataclass some extra fields that are needed by the superclass you want to use. -CHB Best, > > Neil > > On Tue, Apr 14, 2020 at 12:07 AM Christopher Barker <pythonchb(a)gmail.com> > wrote: > >> I feel like it would be nice to be able to use dataclasses more often >>> without worrying that you cannot use dataclasses in cooperative >>> inheritance. Perhaps, dataclasses could call super with unused args and >>> kwargs? >>> >> >> There is a PR on gitHub where a user has requested that dataclasses >> (optionally) except any extra kwargs along. I think it saw some support, >> but has stalled out. Im pretty sure in that case, the extra kwargs would be >> ignored. >> >> https://github.com/python/cpython/pull/19206 >> https://bugs.python.org/issue33493 >> >> -CHB >> >> On Sun, Apr 12, 2020 at 12:49 AM Neil Girdhar <mistersheik(a)gmail.com> >> wrote: >> >>> class X: >>> def __init__(self, x, **kwargs): >>> super().__init__(**kwargs) >>> print(x, kwargs) >>> >>> >>> >>> @dataclass >>> class Y(X): >>> y: int >>> >>> >>> Y(1) # What should happen? >>> Y(1, 2) # What should happen? >>> >>> >>> I feel like it would be nice to be able to use dataclasses more often >>> without worrying that you cannot use dataclasses in cooperative >>> inheritance. Perhaps, dataclasses could call super with unused args and >>> kwargs? >>> >>> Best, >>> >>> Neil >>> _______________________________________________ >>> Python-ideas mailing list -- python-ideas(a)python.org >>> To unsubscribe send an email to python-ideas-leave(a)python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/6YMRI… >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> >> >> -- >> Christopher Barker, PhD >> >> Python Language Consulting >> - Teaching >> - Scientific Software Development >> - Desktop GUI and Web Development >> - wxPython, numpy, scipy, Cython >> -- >> Christopher Barker, PhD >> >> Python Language Consulting >> - Teaching >> - Scientific Software Development >> - Desktop GUI and Web Development >> - wxPython, numpy, scipy, Cython >> > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

8 19

Exception spaces
by Soni L. 12 Apr '20

12 Apr '20

Sometimes, you have an API: @abc.abstractmethod def get_property_value(self, prop): """Returns the value associated with the given property. Args: prop (DataProperty): The property. Returns: The value associated with the given property. Raises: PropertyError: If the property is not supported by this config source. LookupError: If the property is supported, but isn't available. ValueError: If the property doesn't have exactly one value. """ raise PropertyError and you don't want your API to mask bugs. How would it mask bugs? For example, if API consumers do an: try: x = foo.get_property_value(prop) except ValueError: # handle it then the following implementation: def get_property_value(self, prop): iterator = self.get_property_values(prop) try: # note: unpacking ret, = iterator except LookupError as exc: raise RuntimeError from exc # don't accidentally swallow bugs in the iterator return ret def get_property_values(self, prop): try: factory = self.get_supported_properties()[prop] except KeyError as exc: raise PropertyError from exc iterator = factory(self._obj) try: first = next(iterator) except StopIteration: return (x for x in ()) except abdl.exceptions.ValidationError as exc: raise LookupError from exc except LookupError as exc: raise RuntimeError from exc # don't accidentally swallow bugs in the iterator return itertools.chain([first], iterator) could cause you to accidentally swallow unrelated ValueErrors. so instead this needs to be rewritten. you can't just unpack things from the iterator, you need to wrap the iterator into another iterator, that then converts ValueError into RuntimeError. but if that ValueError is part of *your* API requirements... it's impossible to use! so my proposal is that we get "exception spaces". they'd be used through the 'in' keyword, as in "except in" and "raise in". for example: X = object() Y = object() def foo(): raise LookupError in X def bar(): try: foo() except LookupError in Y: print("bar: caught LookupError in Y, ignoring") def baz(): try: bar() except LookupError in X: print("baz: caught LookupError in X, re-raising in Y") raise in Y def qux(): try: baz() except LookupError in Y: print("qux: caught LookupError in Y, ignoring") qux() # would print: # --- # baz: caught LookupError in X, re-raising in Y # qux: caught LookupError in Y, ignoring # --- # note the lack of "bar" (or perhaps "raise in X LookupError" and "except in Y LookupError" etc) and then you can adjust the above implementations accordingly: (btw, anyone knows how to tell apart a ValueError from a specific unpacking and a ValueError from an iterator being used in that unpacking?) def get_property_value(self, prop, espace=None): iterator = self.get_property_values(prop, espace=espace) try: # note: unpacking ret, = iterator except ValueError: raise in espace # except LookupError as exc: raise RuntimeError from exc # no longer needed return ret def get_property_values(self, prop, espace=None): try: factory = self.get_supported_properties()[prop] except KeyError as exc: raise PropertyError in espace from exc iterator = factory(self._obj) try: first = next(iterator) except StopIteration: return (x for x in ()) except abdl.exceptions.ValidationError as exc: raise LookupError in espace from exc # except LookupError as exc: raise RuntimeError from exc # no longer needed return itertools.chain([first], iterator) as well as the caller: espace = object() try: x = foo.get_property_value(prop, espace=espace) except ValueError in espace: # handle it I feel like this feature would significantly reduce bugs in python code, as well as significantly improve the error messages related to bugs. This would be even better than what we did with StopIteration! This would be comparable to Rust's Result type, where you can have Result<Result<YourValue, YourError>, TheirError> and the like (except slightly/a lot more powerful).

11 42

Should dataclass init call super?
by Neil Girdhar 12 Apr '20

12 Apr '20

Dear Pythoners, class X: def __init__(self, x, **kwargs): super().__init__(**kwargs) print(x, kwargs) @dataclass class Y(X): y: int Y(1) # What should happen? Y(1, 2) # What should happen? I feel like it would be nice to be able to use dataclasses more often without worrying that you cannot use dataclasses in cooperative inheritance. Perhaps, dataclasses could call super with unused args and kwargs? Best, Neil

1 0

Range of Indexes
by Chamoun Saoma 12 Apr '20

12 Apr '20

I have an idea to improve the python language through the creation of a new and very readable function. ################# def ranlen(x): #range of length of x return range(len(x)) ################# This will be a simple to use and very readable way of creating ranges of indexes. I hope for this to become a new pythogenic way of writing code, especially because using range(leng(x)) is so essential when implementing various algorithms.

5 5