[Python-ideas] new format spec for iterable types

[Andrew Barnert]

On Sep 10, 2015, at 01:58, Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de> wrote:
> 
>> On 10.09.2015 02:03, Andrew Barnert via Python-ideas wrote:
>>> On Sep 9, 2015, at 16:24, random832 at fastmail.us wrote:
>>> 
>>>> On Wed, Sep 9, 2015, at 18:39, Andrew Barnert via Python-ideas wrote:
>>>> I believe he posted a more detailed version of the idea on one of the
>>>> other spinoff threads from the f-string thread, but I don't have a link.
>>>> But there are lots of possibilities, and if you want to start
>>>> bikeshedding, it doesn't matter that much what his original color was.
>>>> For example, here's a complete proposal:
>>>> 
>>>>    class MyJoiner:
>>>>        def __init__(self, value):
>>>>            self.value = value
>>>>        def __format__(self, spec):
>>>>            return spec.join(map(str, self.value))
>>>>    string.register_converter('join', MyJoiner)
>>> 
>>> Er, I wanted it to be something more like
>>> 
>>> def __format__(self, spec):
>>>   sep, fmt = # 'somehow' break up spec into two parts
>> 
>> I covered later in the same message how this simple version could be extended to a smarter version that does that, or even more, without requiring any further changes to str.format. I just wanted to show the simplest version first, and then show that designing for that doesn't lose any flexibility.
> 
> Ok, I think I got the idea.
> One question though: how would you prevent this from getting competely out of hand?

Same way we keep types with weird __format__ methods, nested or multi-clause comprehensions, import hooks, operator overloads like using __ror__ to partial functions, metaclasses, subclass hooks, multiple inheritance, dynamic method lookup, descriptors, etc. from getting completely out of hand: trust users to have some taste, and don't write bad documentation that would convince them to abuse it. :)

>> And meanwhile, the alternative seems to be having something similar, but not exposing it publicly, and just baking in a handful of hardcoded converters for join, html, re-escape, etc., and I don't see why str should know about all of those things, or why extending that set when we realize that we forgot about shlex should require a patch to str and a new Python version.
>> 
>>> The Joiner class wouldn't have to exist as a builtin, it could be
>>> private to the format function.
>> 
>> If it's custom-registerable, it can be on PyPI, or in the middle of your app, although of course there could be some converters, maybe including your Joiner, somewhere in the stdlib, or even private to format, as well.
> 
> The strength of this idea - flexibility - could also be called its biggest weakness and that is scaring me. Essentially, such converters would be completely free to do anything they want: change their input at will, return something completely unrelated, have side-effects. All of that hidden behind a simple !token in a replacement field.
> While the idea is really cool and certainly powerful if used responsibly, it could also create completely unreadable code.

There aren't any obvious reasons for anyone to write such unreadable code, so I don't see it being a real attractive nuisance.

> Just adding one single hardcoded converter for joining iterables looks like a much more reasonable and realistic idea and now that I understand the concept I have to say I really like it.
> 
> Just paraphrasing once more to see if a understood things correctly this time:
> The !j converter converts the iterable to an instance of a Joiner class just like !s, !r and !a convert to a str instance. After that conversion the __format__ method of the new object gets called with the format_spec string (which specifies the separator and the inner format spec) as argument and that method produces the joint string.
> 
> So everything follows the existing logic of a converter and no really new replacement field syntax is required. Great and +1!

Yep, and I'm +1 on it as well.

But in also at least +0.5 on the custom converter idea, because joining is the fourth idea people have come up with for converters in the past few weeks, and I'd bet there are another few widely-usable ideas, plus some good uses for specific applications (different web frameworks, scientific computing, etc.). When I get a chance, I'll hack something up to play with it and see if it's as useful as I'm expecting.