Is there any further thoughts on including this in 3.6? Similar to the recent discussion on OrderedDict namespaces for metaclasses, this would simplify / enable a number of type factory use cases where proper metaclasses are overkill. This feature would also be quite nice in say pandas where the (currently unspecified) field order used in the definition of frames is preserved in user-visible displays.
From: zreed@fastmail.com Subject: [Python-Dev] PEP 468
Is there any further thoughts on including this in 3.6? Similar to the recent discussion on OrderedDict namespaces for metaclasses, this would simplify / enable a number of type factory use cases where proper metaclasses are overkill. This feature would also be quite nice in say pandas where the (currently unspecified) field order used in the definition of frames is preserved in user-visible displays.
As stated by Guido (and pointed out in the PEP): Making **kwds ordered is still open, but requires careful design and implementation to avoid slowing down function calls that don't benefit. The PEP has not been updated in a while, though. Python 3.5 has been released, and with it a C implementation of OrderedDict. Eric, are you still interested in this? IIRC that PEP was one of the motivating use cases for implementing OrderedDict in C. Maybe it's time for a second round of discussion on Python-ideas? -Emanuel
On Thu, Jun 9, 2016 at 1:10 PM, Émanuel Barry <vgr255@live.ca> wrote:
As stated by Guido (and pointed out in the PEP):
Making **kwds ordered is still open, but requires careful design and implementation to avoid slowing down function calls that don't benefit.
The PEP has not been updated in a while, though. Python 3.5 has been released, and with it a C implementation of OrderedDict.
Eric, are you still interested in this?
Yes, but wasn't planning on dusting it off yet (i.e. in time for 3.6). I'm certainly not opposed to someone picking up the banner. <wink-wink>
IIRC that PEP was one of the motivating use cases for implementing OrderedDict in C.
Correct, though I'm not sure OrderedDict needs to be involved any more.
Maybe it's time for a second round of discussion on Python-ideas?
Fine with me, though I won't have a lot of time in the 3.6 timeframe to handle a high-volume discussion or push through an implementation. -eric
I would be super excited for this feature, so if there's a reasonable chance of it being picked up I don't mind doing the implementation work. On Fri, Jun 10, 2016, at 11:54 AM, Eric Snow wrote:
On Thu, Jun 9, 2016 at 1:10 PM, Émanuel Barry <vgr255@live.ca> wrote:
As stated by Guido (and pointed out in the PEP):
Making **kwds ordered is still open, but requires careful design and implementation to avoid slowing down function calls that don't benefit.
The PEP has not been updated in a while, though. Python 3.5 has been released, and with it a C implementation of OrderedDict.
Eric, are you still interested in this?
Yes, but wasn't planning on dusting it off yet (i.e. in time for 3.6). I'm certainly not opposed to someone picking up the banner. <wink-wink>
IIRC that PEP was one of the motivating use cases for implementing OrderedDict in C.
Correct, though I'm not sure OrderedDict needs to be involved any more.
Maybe it's time for a second round of discussion on Python-ideas?
Fine with me, though I won't have a lot of time in the 3.6 timeframe to handle a high-volume discussion or push through an implementation.
-eric
Eric, have you any work in progress on compact dicts? On Fri, Jun 10, 2016 at 12:54 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Thu, Jun 9, 2016 at 1:10 PM, Émanuel Barry <vgr255@live.ca> wrote:
As stated by Guido (and pointed out in the PEP):
Making **kwds ordered is still open, but requires careful design and implementation to avoid slowing down function calls that don't benefit.
The PEP has not been updated in a while, though. Python 3.5 has been released, and with it a C implementation of OrderedDict.
Eric, are you still interested in this?
Yes, but wasn't planning on dusting it off yet (i.e. in time for 3.6). I'm certainly not opposed to someone picking up the banner. <wink-wink>
IIRC that PEP was one of the motivating use cases for implementing OrderedDict in C.
Correct, though I'm not sure OrderedDict needs to be involved any more.
Maybe it's time for a second round of discussion on Python-ideas?
Fine with me, though I won't have a lot of time in the 3.6 timeframe to handle a high-volume discussion or push through an implementation.
-eric _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/leewangzhong%2Bpython%40g...
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it. P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard. [1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered. On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com <javascript:;>> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
On 06/10/2016 02:13 PM, Franklin? Lee wrote:
P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition.
It would be great for kwargs, but not for class definition: del's can happen there, so we need PEP 520 with OrderedDict so the definition order is not lost when an item is deleted during class creation. -- ~Ethan~
On 2016-06-13 17:34, Ethan Furman wrote:
On 06/10/2016 02:13 PM, Franklin? Lee wrote:
P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition.
It would be great for kwargs, but not for class definition: del's can happen there, so we need PEP 520 with OrderedDict so the definition order is not lost when an item is deleted during class creation.
The order can be lost when an item is deleted because it moves the last item into the 'hole' left by the deleted item. This could be avoided by expanding the items to include the index of the 'previous' and 'next' item, so that they could be handled like a doubly-linked list. The disadvantage would be that it would use more memory.
On 06/13/2016 05:05 PM, MRAB wrote:
This could be avoided by expanding the items to include the index of the 'previous' and 'next' item, so that they could be handled like a doubly-linked list.
The disadvantage would be that it would use more memory.
Another, easier technique: don't fill holes. Same disadvantage (increased memory use), but easier to write and maintain. //arry/
On 2016-06-14 01:47, Larry Hastings wrote:
On 06/13/2016 05:05 PM, MRAB wrote:
This could be avoided by expanding the items to include the index of the 'previous' and 'next' item, so that they could be handled like a doubly-linked list.
The disadvantage would be that it would use more memory.
Another, easier technique: don't fill holes. Same disadvantage (increased memory use), but easier to write and maintain.
When iterating over the dict, you'd need to skip over the holes, so it would be a good idea to compact it a some point, when there are too many holes.
On Jun 13, 2016 6:16 PM, "MRAB" <python@mrabarnett.plus.com> wrote:
On 2016-06-14 01:47, Larry Hastings wrote:
On 06/13/2016 05:05 PM, MRAB wrote:
This could be avoided by expanding the items to include the index of the 'previous' and 'next' item, so that they could be handled like a doubly-linked list.
The disadvantage would be that it would use more memory.
Another, easier technique: don't fill holes. Same disadvantage (increased memory use), but easier to write and maintain.
When iterating over the dict, you'd need to skip over the holes, so it
would be a good idea to compact it a some point, when there are too many holes. Right -- but if you wait for some ratio of holes to filled space before compacting, you can amortize the cost down, and have a good big-O complexity for both del and iteration simultaneously. Same basic principle as using proportional overallocation when appending to a list, just in reverse. I believe this is what pypy's implementation actually does. -n
On 06/13/2016 05:47 PM, Larry Hastings wrote:
On 06/13/2016 05:05 PM, MRAB wrote:
This could be avoided by expanding the items to include the index of the 'previous' and 'next' item, so that they could be handled like a doubly-linked list.
The disadvantage would be that it would use more memory.
Another, easier technique: don't fill holes. Same disadvantage (increased memory use), but easier to write and maintain.
I hope this is just an academic discussion: suddenly having Python's dicts grow continuously is going to have nasty consequences somewhere. -- ~Ethan~
Compact OrderedDicts can leave gaps, and once in a while compactify. For example, whenever the entry table is full, it can decide whether to resize (and only copy non-gaps), or just compactactify Compact regular dicts can swap from the back and have no gaps. I don't see the point of discussing these details. Isn't it enough to say that these are solvable problems, which we can worry about if/when someone actually decides to sit down and implement compact dicts? P.S.: Sorry about the repeated emails. I think it was the iOS Gmail app. On Jun 13, 2016 10:23 PM, "Ethan Furman" <ethan@stoneleaf.us> wrote:
On 06/13/2016 05:47 PM, Larry Hastings wrote:
On 06/13/2016 05:05 PM, MRAB wrote:
This could be avoided by expanding the items to include the index of the 'previous' and 'next' item, so that they could be handled like a doubly-linked list.
The disadvantage would be that it would use more memory.
Another, easier technique: don't fill holes. Same disadvantage (increased memory use), but easier to write and maintain.
I hope this is just an academic discussion: suddenly having Python's
dicts grow continuously is going to have nasty consequences somewhere.
-- ~Ethan~
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it. P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard. [1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered. On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com <javascript:;>> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it. P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard. [1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered. On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com <javascript:;>> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it. P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard. [1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered. On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com <javascript:;>> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it. P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard. [1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered. On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com <javascript:;>> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
Can someone block Franklin until his mailer stops resending this message? --Guido (mobile) On Jun 13, 2016 2:26 PM, "Franklin? Lee" <leewangzhong+python@gmail.com> wrote:
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it.
P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard.
[1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered.
On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
I've set him to moderation for now. Beyond that we'd have to unsubscribe him altogether and ask him to resubscribe later. TJG On 13/06/2016 22:34, Guido van Rossum wrote:
Can someone block Franklin until his mailer stops resending this message?
--Guido (mobile)
On Jun 13, 2016 2:26 PM, "Franklin? Lee" <leewangzhong+python@gmail.com <mailto:leewangzhong%2Bpython@gmail.com>> wrote:
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it.
P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard.
[1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered.
On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com <mailto:ericsnowcurrently@gmail.com>> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote: > Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
_______________________________________________ Python-Dev mailing list Python-Dev@python.org <mailto:Python-Dev@python.org> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mail%40timgolden.me.uk
I am. I was just wondering if there was an in-progress effort I should be looking at, because I am interested in extensions to it. P.S.: If anyone is missing the relevance, Raymond Hettinger's compact dicts are inherently ordered until a delitem happens.[1] That could be "good enough" for many purposes, including kwargs and class definition. If CPython implements efficient compact dicts, it would be easier to propose order-preserving (or initially-order-preserving) dicts in some places in the standard. [1] Whether delitem preserves order depends on whether you want to allow gaps in your compact entry table. PyPy implemented compact dicts and chose(?) to make dicts ordered. On Saturday, June 11, 2016, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee <leewangzhong+python@gmail.com <javascript:;>> wrote:
Eric, have you any work in progress on compact dicts?
Nope. I presume you are talking the proposal Raymond made a while back.
-eric
On Thu, Jun 9, 2016 at 12:41 PM, <zreed@fastmail.com> wrote:
Is there any further thoughts on including this in 3.6?
I don't have any plans and I don't know of anyone willing to champion the PEP for 3.6. Note that the implementation itself shouldn't take very long.
Similar to the recent discussion on OrderedDict namespaces for metaclasses, this would simplify / enable a number of type factory use cases where proper metaclasses are overkill. This feature would also be quite nice in say pandas where the (currently unspecified) field order used in the definition of frames is preserved in user-visible displays.
Good point. One weakness of the PEP has been sufficient justification. The greater number of compelling use cases, the better. So thanks! :) -eric
participants (10)
-
Eric Snow -
Ethan Furman -
Franklin? Lee -
Guido van Rossum -
Larry Hastings -
MRAB -
Nathaniel Smith -
Tim Golden -
zreed＠fastmail.com -
Émanuel Barry