Mailman 3 November 2018 - Python-ideas

Re: [Python-ideas] dict.setdefault_call(), or API variations thereupon
by Alex Shafer 06 Nov '18

06 Nov '18

I had actually managed to miss collections.defaultdict! I'd like to instead propose that a reference to that be added to the dict.setdefault docs. I can't imagine I'm the only one that has missed this. Date: Fri, 2 Nov 2018 12:12:45 +1100 > From: Chris Angelico <rosuav(a)gmail.com> > To: python-ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] dict.setdefault_call(), or API variations > thereupon > Message-ID: > < > CAPTjJmqg_qtK3OfR+4VAaaNa7JXjHjHLpnx6EfEZX5n4tttqCQ(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > On Fri, Nov 2, 2018 at 12:07 PM Alex Shafer <ashafer01(a)gmail.com> wrote: > > Other APIs I've considered for this are a new keyword argument to the > existing `setdefault()`, or perhaps more radically for Python, a new > keyword argument to the `dict()` constructor that would get called as an > implicit default for `setdefault()` and perhaps used in other scenarios > (essentially defining a type for dict values). > > > > The time machine has been put to good use here. Are you aware of > __missing__ and collections.defaultdict? You just create a defaultdict > with a callable (very common to use a class like "list"), and any time > you try to use something that's missing, it'll call that to generate a > value. > > from collections import defaultdict > d = defaultdict(list) > for category, item in some_stuff: > d[category].append(item) > > Easy way to group things into their categories. > > ChrisA >

7 12

Proposal to add a key field to the bisect library
by Tiberiu Chibici 04 Nov '18

04 Nov '18

Hi, I would like to propose an improvement to the functions in the bisect library, to add a 'key' parameter, similar to 'sorted' or other system functions. -- Chibici Tiberiu

3 4

Re: [Python-ideas] Are we supposed to be able to have our own class dictionary in python 3?
by Amit Green 04 Nov '18

04 Nov '18

Thanks Daniel, I found my answer here (using your link): https://docs.python.org/3/reference/datamodel.html#preparing-the-class-name… """ When a new class is created by type.__new__, the object provided as the namespace parameter is copied to a new ordered mapping and the original object is discarded. """ Therefore the answer seems to be that https://www.python.org/dev/peps/pep-3115/ needs to be updated & fixed. Replace the following: """ def __new__(cls, name, bases, classdict): # Note that we replace the classdict with a regular # dict before passing it to the superclass, so that we # don't continue to record member names after the class # has been created. result = type.__new__(cls, name, bases, dict(classdict)) result.member_names = classdict.member_names return result """ With: """ def __new__(cls, name, bases, classdict): result = type.__new__(cls, name, bases, classdict) result.member_names = classdict.member_names return result """ Removing the incorrect comments & the copying of `classdict` I will go file a bug report to that effect. Thanks, Joy Diamond. On Sat, Nov 3, 2018 at 7:55 PM Daniel Moisset <dfmoisset(a)gmail.com> wrote: > If I understood correctly what you want, it's possible with a metaclass. > Check the __prepare__ method at > https://docs.python.org/3/reference/datamodel.html#preparing-the-class-name… > and Pep 3115 > > On Sat, 3 Nov 2018, 20:55 Joy Diamond <python.gem(a)gmail.com wrote: > >> Team, >> >> Are we supposed to be able to have our own class dictionary in python 3? >> >> If we currently cannot -- do we want to be able to? >> >> That we can have out own class dictionary in python 3 is strongly implied >> in the following at https://www.python.org/dev/peps/pep-3115/ where it >> says: >> >> """ >> >> # The metaclass invocation >> def __new__(cls, name, bases, classdict): >> # Note that we replace the classdict with a regular >> # dict before passing it to the superclass, so that we >> # don't continue to record member names after the class >> # has been created. >> result = type.__new__(cls, name, bases, dict(classdict)) >> result.member_names = classdict.member_names >> return result >> >> """ >> >> I don't understand this. As far as I can tell, no matter what class >> dictionary you pass into `type.__new__` it creates a copy of it. >> >> Am I missing something? Is this supposed to work? Is the documentation >> wrong? >> >> Thanks, >> >> Joy Diamond. >> >> Program that shows that the class dictionary created is not what we pass >> in --- Shows the actual symbol table is `dict` not `SymbolTable` >> >> class SymbolTable(dict): >> pass >> >> members = SymbolTable(a = 1) >> >> X = type('X', ((object,)), members) >> >> members['b'] = 2 >> >> print('X.a: {}'.format(X.a)) >> >> try: >> print('X.b: {}'.format(X.b)) >> except AttributeError as e: >> print('X.b: does not exist') >> >> # >> # Get the actual symbol table of `X`, bypassing the mapping proxy. >> # >> X__symbol_table = __import__('gc').get_referents(X.__dict__)[0] >> >> print('The type of the actual symbol table of X is: {} with keys: >> {}'.format( >> type(X__symbol_table), >> X__symbol_table.keys())) >> >> >> # Prints out >> # X.a: 1 >> # X.b: does not exist >> # The type of the actual symbol table of X is: <class 'dict'> with keys: >> dict_keys(['a', '__module__', '__dict__', '__weakref__', '__doc__']) >> >> >> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. >> www.avast.com >> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> >> <#m_3863327966708087084_m_7370047320667964513_m_603818574152564677_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >

2 1

Re: [Python-ideas] Are we supposed to be able to have our own class dictionary in python 3?
by Daniel Moisset 03 Nov '18

03 Nov '18

Sorry, should have replied to the list too On Sat, 3 Nov 2018, 23:55 Daniel Moisset <dfmoisset(a)gmail.com wrote: > If I understood correctly what you want, it's possible with a metaclass. > Check the __prepare__ method at > https://docs.python.org/3/reference/datamodel.html#preparing-the-class-name… > and Pep 3115 > > On Sat, 3 Nov 2018, 20:55 Joy Diamond <python.gem(a)gmail.com wrote: > >> Team, >> >> Are we supposed to be able to have our own class dictionary in python 3? >> >> If we currently cannot -- do we want to be able to? >> >> That we can have out own class dictionary in python 3 is strongly implied >> in the following at https://www.python.org/dev/peps/pep-3115/ where it >> says: >> >> """ >> >> # The metaclass invocation >> def __new__(cls, name, bases, classdict): >> # Note that we replace the classdict with a regular >> # dict before passing it to the superclass, so that we >> # don't continue to record member names after the class >> # has been created. >> result = type.__new__(cls, name, bases, dict(classdict)) >> result.member_names = classdict.member_names >> return result >> >> """ >> >> I don't understand this. As far as I can tell, no matter what class >> dictionary you pass into `type.__new__` it creates a copy of it. >> >> Am I missing something? Is this supposed to work? Is the documentation >> wrong? >> >> Thanks, >> >> Joy Diamond. >> >> Program that shows that the class dictionary created is not what we pass >> in --- Shows the actual symbol table is `dict` not `SymbolTable` >> >> class SymbolTable(dict): >> pass >> >> members = SymbolTable(a = 1) >> >> X = type('X', ((object,)), members) >> >> members['b'] = 2 >> >> print('X.a: {}'.format(X.a)) >> >> try: >> print('X.b: {}'.format(X.b)) >> except AttributeError as e: >> print('X.b: does not exist') >> >> # >> # Get the actual symbol table of `X`, bypassing the mapping proxy. >> # >> X__symbol_table = __import__('gc').get_referents(X.__dict__)[0] >> >> print('The type of the actual symbol table of X is: {} with keys: >> {}'.format( >> type(X__symbol_table), >> X__symbol_table.keys())) >> >> >> # Prints out >> # X.a: 1 >> # X.b: does not exist >> # The type of the actual symbol table of X is: <class 'dict'> with keys: >> dict_keys(['a', '__module__', '__dict__', '__weakref__', '__doc__']) >> >> >> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. >> www.avast.com >> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> >> <#m_8752709194315235090_m_603818574152564677_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >

1 0

Are we supposed to be able to have our own class dictionary in python 3?
by Joy Diamond 03 Nov '18

03 Nov '18

Team, Are we supposed to be able to have our own class dictionary in python 3? If we currently cannot -- do we want to be able to? That we can have out own class dictionary in python 3 is strongly implied in the following at https://www.python.org/dev/peps/pep-3115/ where it says: """ # The metaclass invocation def __new__(cls, name, bases, classdict): # Note that we replace the classdict with a regular # dict before passing it to the superclass, so that we # don't continue to record member names after the class # has been created. result = type.__new__(cls, name, bases, dict(classdict)) result.member_names = classdict.member_names return result """ I don't understand this. As far as I can tell, no matter what class dictionary you pass into `type.__new__` it creates a copy of it. Am I missing something? Is this supposed to work? Is the documentation wrong? Thanks, Joy Diamond. Program that shows that the class dictionary created is not what we pass in --- Shows the actual symbol table is `dict` not `SymbolTable` class SymbolTable(dict): pass members = SymbolTable(a = 1) X = type('X', ((object,)), members) members['b'] = 2 print('X.a: {}'.format(X.a)) try: print('X.b: {}'.format(X.b)) except AttributeError as e: print('X.b: does not exist') # # Get the actual symbol table of `X`, bypassing the mapping proxy. # X__symbol_table = __import__('gc').get_referents(X.__dict__)[0] print('The type of the actual symbol table of X is: {} with keys: {}'.format( type(X__symbol_table), X__symbol_table.keys())) # Prints out # X.a: 1 # X.b: does not exist # The type of the actual symbol table of X is: <class 'dict'> with keys: dict_keys(['a', '__module__', '__dict__', '__weakref__', '__doc__']) <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

1 0

dict.setdefault_call(), or API variations thereupon
by Alex Shafer 02 Nov '18

02 Nov '18

I'd like to propose an addition to `dict` but I'm not necessarily proposing what's written here as the API. When I initially saw the need for this myself, I hastily wrote it as: def setdefault_call(a_dict, key, default_func): try: return a_dict[key] except KeyError: default = default_func() a_dict[key] = default return default If its not clear, the purpose is to eliminate the overhead of creating an empty list or similar in situations like this: d = {} for i in range(1000000): # some large loop l = d.setdefault(somekey, []) l.append(somevalue) # instead... for i in range(1000000): l = d.setdefault_call(somekey, list) l.append(somevalue) One potential drawback I see to the concept is that I think there will be a need to explicitly say "no arguments can get passed into this call". Otherwise users may defeat the purpose with constructions like this: d.setdefault_call("foo", list, ["default value"]) I'd mainly like feedback on this concept overall, and if its liked, perhaps an API discussion to follow. Thanks! PS Other APIs I've considered for this are a new keyword argument to the existing `setdefault()`, or perhaps more radically for Python, a new keyword argument to the `dict()` constructor that would get called as an implicit default for `setdefault()` and perhaps used in other scenarios (essentially defining a type for dict values).

3 2

Allow Context Managers to Support Suspended Execution
by David Allemang 01 Nov '18

01 Nov '18

I do not think there is currently a good way for Context Managers to support suspended execution, as in await or yield. Both of these instructions cause the interpreter to leave the with block, yet no indication of this (temporary) exit or subsequent re-entrance is given to the context manager. If the intent of a Context Manager is to say "no matter how this block is entered or exited, the context will be correctly maintained", then this needs to be possible. I would propose magic methods __suspend__ and __resume__ as companions to the existing __enter__ and __exit__ methods (and their async variants). __suspend__, if present, would be called upon suspending execution on an await or yield statement, and __resume__, if present, would be called when execution is resumed. If __suspend__ or __resume__ are not present then nothing should be done, so that the behavior of existing context managers is preserved. Here is an example demonstrating the issue with await: https://gist.github.com/allemangD/bba8dc2d059310623f752ebf65bb6cdc and one with yield: https://gist.github.com/allemangD/f2534f16d3a0c642c2cdc02c544e854f The context manager used is clearly not thread-safe, and I'm not actually sure how to approach a thread-safe implementation with the proposed __suspend__ and __resume__ - but I don't believe that introducing these new methods would create any issues that aren't already present with __enter__ and __exit__. It's worth noting that the context manager used in those examples is, essentially, identical contextlib's redirect_stdout and decimal's localcontext managers. Any context manager such as these which modify global state or the behavior of global functions would benefit from this. It may also make sense to, for example, have the __suspend__ method on file objects flush buffers without closing the file, similar to their current __exit__ behavior, but I'm unsure what impact this would have on performance. It is important, though, that yield and await not use __enter__ or __exit__, as not all context-managers are reusable. I'm unsure what the best term would be to describe this type of context, as the documentation for contextlib already gives a different definition for "reentrant" - I would then call them "suspendable" contexts. It would make sense to have an @suspendable decorator, probably in contextlib, to indicate that a context manager can use __enter__ and __exit__ methods rather than __suspend__ and __resume__. All it would need to do is define __suspend__ to call __enter__() and __resume__ to call __exit__(None, None, None). It is also important, since __suspend__ and __resume__ would be called after a context is entered but before it is exited, that __suspend__ not accept any parameters and that __resume__ not use its return value. __suspend__ could not be triggered by an exception, only by a yield or await, and __resume__ could not have its return value named with as. Thanks, David

5 5

Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae
by julien tayon 01 Nov '18

01 Nov '18

Hello :) the idea is described here:http://jul.github.io/cv/pres.html#printable Summary of the idea : Take a linear algebrae book, and implements all the rules as a TDD.https://github.com/jul/archery/blob/master/consistent_algebrae.py make it works based on abstract base class and sets of Mixins.https://archery.readthedocs.io/en/latest/ And see if we can make cos/__abs__/dot and if it gives naively the intended results ? (spoiler: yes) Making it work with dict, and "other" dictionary like counter by using ineritancehttps://archery.readthedocs.io/en/latest/#advanced-usage My idea is : wouldn't it be nice if we introduced geometries as sets of mixins for objects ? (Hilbertian algebrae could be nice too, and we could make MutableMapping behave like bra/kets). So I was proposing a soft discussion on : could we agree that it would be nice to consider operation overloading as a whole set of behaviours that could profit from being consistent in a categorized way ? (like the + of [] could be the + of "RecordAlgebrae") Meaning we could define sets of "expected behaviour consistent interaction between operators" as we defined the abc and call them algebrae? I offer the LinearAlgebrae Mixins as a POC, and was thinking of creating a unittest to qualify if an object is following the rules of linear algebrae. What are your opinions ? I don't actually see a lot of use case except it was funny to build. But maybe it can be of use. Cordialement -- Julien

8 12