[Python-ideas] Let's be more orderly!

[Don Spaulding]

On Wed, May 15, 2013 at 1:01 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> From: Don Spaulding <donspauldingii at gmail.com>
> Sent: Tuesday, May 14, 2013 6:57 PM
> >In the interest of moving the discussion forward, I've had a few use
> cases along these lines.  Let's say I want to create simple HTML elements
> by hand:
>
> >
> >    def create_element(tag, text='', **attributes):
> >        attrs = ['{}="{}"'.format(k,v) for k, v in attributes.items()]
> >        return "<{0} {1}>{2}</{0}>".format(tag, ' '.join(attrs), text)
> >
> >    print(create_element('img', alt="Some cool stuff.",
> src="coolstuff.jpg"))
> >    <img src="coolstuff.jpg" alt="Some cool stuff."></img>
>
> Well, HTML explicitly assigns no meaning to the order of attributes. And I
> think this is a symptom of a larger problem. Every month, half a dozen
> people come to StackOverflow asking how to get an ordered dictionary. Most
> of them are asking because they want to preserve the order of JSON
> objects—which, again, is explicitly defined as unordered. If code relies on
> the order of HTML attributes, or JSON object members, it's wrong, and it's
> going to break, and it's better to find that out early.
>

Yes, I'm aware that HTML and JSON are explicit about the fact that order
should not matter to parsers.  But just because I know that, and you know
that, doesn't mean that the person in charge of developing the XML-based or
JSON-based web service I'm trying to write a wrapper for knows that.  Twice
now I've encountered poorly-written web services that have choked on
something like:

<request>
  <action>modifyStuff</action>
  <user>user_123456</user>
</request>

 ...with an error to the effect of "Cannot modifyStuff without specifying
user credentials".  So someone else has built a system around an XML parser
that doesn't know that sibling elements aren't guaranteed to appear in any
particular order.  Obviously the best fix is for them to use a different
parser, but my point is that there's no fix available to my function short
of writing all calling locations into a list-of-tuples format.  My function
doesn't care about the order per se, it just doesn't want to *change the
order* of the input while it's generating the output.

As another example, the most recent instance where I've wanted to upgrade a
regular dict to an odict, was when discovering a bug in a coworker's lookup
table.  It was a table that mapped an employee_type string to a Django
queryset to be searched for a particular user_id.  Consider this lookup
function:

EMPLOYEE_TYPES = {
    'agent': Agent.objects.all(),
    'staff': Staff.objects.all(),
    'associate': Associate.objects.all()
}

def get_employee_type(user_id):
   for typ, queryset in EMP_TYPES.items():
        if queryset.filter(user_id=user_id).exists():
            return typ

The bug we hit was because we cared about checking each queryset in the
order they were specified.  My coworker knows that dicts are unordered, but
it slipped his mind while writing this code.  Regardless, once you find a
bug like this, what are your options for fixing this to process the lookups
in a specific order?

The first thing you think of is, "Oh, I just need to use an OrderedDict.".
Well, technically yes, except there's no convenient way to instantiate an
OrderedDict with more than one element at a time.  So now you're back to
rewriting calling sites into order-preserving lists-of-tuples again.  Which
is why I think the OrderedDict.__init__ case is in and of itself
compelling.   ;-)


>It's not that it's impossible to do, it's that dict-based API's would
> benefit from the function being able to decide on its own whether or not it
> cared about the order of arguments.  Having to express a kwargs-based or
> plain-old-dict-based function as a list-of-2-tuples function is... uncool.
> ;-)
>
>
> This is an interesting idea. If there were a way for the function to
> decide what type is used for creating its kwargs, you could do all kinds of
> cool things—have that switch you could turn on or off I just mentioned for
> different kinds of testing, or preserve order in "debug mode" but leave it
> arbitrary and as fast as possible in "production mode", or take a
> blist.sorteddict if you're intending to stash it and use it as the starting
> point for a blist.sorteddict anyway, or whatever. And it wouldn't affect
> the 99% of functions that don't care.
>
> The syntax seems pretty obvious:
>
>     def kwargs(mapping_constructor):
>         def deco(fn):
>             fn.kwargs_mapping_constructor = mapping_constructor
>             return fn
>         return deco
>
>     @kwargs(OrderedDict)
>     def foo(a, b, *args, **kwargs):
>         pass
>


That's an interesting concept.  It would certainly address the most common
need I see for better OrderedDict support in the language.


> Handling this at the calling site is a bit harder, but still not that hard.
>
>
I don't see how this would require changes to the calling site.  Can you
elaborate?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130515/fe99746b/attachment-0001.html>