[Python-ideas] Dict literal use for custom dict classes

[Jim Baker]

For Jython, ordered dict semantics for the dict type *could* possibly work.

Currently, dict objects are backed by
java.util.concurrent.ConcurrentHashMap, to get correct semantics with
respect to possible del when iterating over the dict; and to provide
volatile memory semantics, to match CPython's memory model, informal as it
may be. Using CHM also likely helps with Jython's threading story. Note
that we can not simply use a java.util.LinkedHashMap that has been wrapped
with java.util.collections.synchronizedMap; at the very least we would get
this behavior:

The iterators returned by the iterator method of the collections returned
> by all of this class's collection view methods are fail-fast: if the map is
> structurally modified at any time after the iterator is created, in any way
> except through the iterator's own remove method, the iterator will throw a
> ConcurrentModificationException. Thus, in the face of concurrent
> modification, the iterator fails quickly and cleanly, rather than risking
> arbitrary, non-deterministic behavior at an undetermined time in the future.

(http://docs.oracle.com/javase/7/docs/api/java/util/LinkedHashMap.html)

But there is an alternative: Caffeine is an interesting project that has
matured significantly since when I took a look at it last year (
https://github.com/ben-manes/caffeine). Caffeine implements a generally
useful concurrent linked hash map which provides the necessary weak
iteration semantics we need for Python compatibility; and it looks like
Caffeine may have performance comparable to, or better than CHM (but not
clear if that extends to map construction, currently a pretty heavy cost
for Jython). Caffeine also builds on the implementation experience of
Google Guava, which Jython currently uses extensively for internal runtime
caches.

So it's certainly worth exploring if this possible change for Python gets
further interest - we will want to benchmark and really understand because
dict/__dict__ support is one of the most critical aspects of good Python
performance.

- Jim

On Wed, Dec 16, 2015 at 4:17 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Wed, Dec 16, 2015 at 10:47 AM, Mike Miller <python-ideas at mgmiller.net>
> wrote:
> >
> > On 2015-12-15 11:02, Paul Moore wrote:
> >>
> >> while it's a bit too special case on its own, one
> >> possibility would be to have
> >>
> >>      callable{k1: v1, k2: v2, ...}
> >>
> >> be syntactic sugar for
> >>
> >>      callable([(k1, k1), (k2, v2), ...])
> >
> >
> > Very interesting... I've faced the issue several times over the years
> when
> > I've wanted to unpack values into a function call in an ordered manner
> (but
> > it hasn't been available).  Perhaps::
> >
> >     callable{**odict}
> >
> > In fact with callables I'd even go so far as wish that ordered unpacking
> was
> > the default somehow, though I guess that probably isn't possible due to
> > history.
>
> That's not so clear, actually! It turns out that PyPy was able to make
> their regular 'dict' implementation ordered, while at the same time
> making it faster and more memory-efficient compared to their previous
> (CPython-like) implementation:
>
>
> http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
>
> So in PyPy all these issues are automatically solved for free. The
> $1e6-question these other proposals have to answer is, why not do what
> PyPy did? Maybe there is a good reason not to, but it seems like it'll
> be difficult to get consensus on moving forward on any of these other
> more complicated proposals until someone has first made a serious
> attempt at porting PyPy's dict to CPython and is able to clearly
> describe why it didn't work.
>
> (3.5 does have a faster C implementation of OrderedDict, thanks to
> tireless efforts by Eric Snow -- https://bugs.python.org/issue16991 --
> but this implementation uses a very different and less cache-efficient
> strategy than PyPy.)
>
> -n
>
> --
> Nathaniel J. Smith -- http://vorpus.org
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151216/7e5d1825/attachment.html>