[Python-ideas] new format spec for iterable types

[Andrew Barnert]

On Sep 9, 2015, at 15:03, Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de> wrote:
> 
>> On 09.09.2015 23:28, Andrew Barnert via Python-ideas wrote:
>>> On Sep 9, 2015, at 06:41, Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de> wrote:
>>> 
>>> 3)
>>> Finally, the alternative idea of having the new functionality handled by a new !converter, like:
>>> 
>>> "List: {0!j:,}".format([1.2, 3.4, 5.6])
>>> 
>>> I considered this idea before posting the original proposal, but, in addition to requiring a change to str.format (which would need to recognize the new token), this approach would need either:
>>> 
>>> - a new special method (e.g., __join__) to be implemented for every type that should support it, which is worse than for my original proposal or
>>> 
>>> - the str.format method must react directly to the converter flag, which is then no different to the above solution just that it uses !j instead of *. Personally, I find the * syntax more readable, plus, the !j syntax would then suggest that this is a regular converter (calling a special method of the object) when, in fact, it is not.
>>> Please correct me, if I misunderstood something about this alternative proposal.
>> 
>> But the format method already _does_ react directly to the conversion flag. As the docs say, the "type coercion" (call to str, repr, or ascii) happens before formatting, and then the __format__ method is called on the result. A new !j would be a "regular converter"; it just calls a new join function (which returns something whose __format__ method then does the right thing) instead of the str, repr, or ascii functions.
> 
> Ah, I see! Thanks for correcting me here. Somehow, I had the mental picture that the format converters would call the object's __str__ and __repr__ methods directly (and so you'd need an additional __join__ method for the new converter), but that's not the case then.
> 
>> And random's custom converter idea would work similarly, except that presumably his !join would specify a function registered to handle the "join" conversion in some way rather than being hardcoded to a builtin.
> 
> How would such a registration work (sorry, I haven't had the time to search the list for his previous mention of this idea)? A new builtin certainly won't fly.

I believe he posted a more detailed version of the idea on one of the other spinoff threads from the f-string thread, but I don't have a link. But there are lots of possibilities, and if you want to start bikeshedding, it doesn't matter that much what his original color was. For example, here's a complete proposal:

    class MyJoiner:
        def __init__(self, value):
            self.value = value
        def __format__(self, spec):
            return spec.join(map(str, self.value))
    string.register_converter('join', MyJoiner)

That last line adds it to some global table (maybe string._converters, or maybe it's not exposed at the Python level at all; whatever).

In str.format, instead of reading a single character after a !, it reads until colon or end of field; if that's more than a single character, it looks it up in the global table and calls the registered callable. So, in this case, "{spam!join:-}"
would call MyJoiner(spam).__format__('-').

Any more complexity can be added to MyJoiner pretty easily, so this small extension to str.format seems sufficient for anything you might want. For example, if you want a three-part format spec that includes the join string, a format spec to pass to each element, and a format spec to apply to the whole thing:

    def __format__(self, spec):
        joinstr, _, spec = spec.partition(':')
        espec, _, jspec = spec.partition(':')
        bits = (format(e, espec) for e in self.value)
        joined = joinstr.join(bits)
        return format(joined, jspec)

Or maybe it would be better to have a standard way to do multi-part format specs--maybe even passing arguments to a converter rather than cramming them in the spec--but this seems simple and flexible enough.

It might also be worth having multiple converters called in a chain, but I can't think of a use case for that, so I'll ignore it.

Most converters will be classes that just store the constructor argument and use it in __format__, so it seems tedious to repeat that boilerplate for 90% of them, but that's easy to fix with a decorator:

    def simple_converter(func):
        class Converter:
            def __init__(self, value):
                self.value = value
            def __format__(self, spec):
                return func(self.value, spec)

Meanwhile, maybe you want the register function to be a decorator:

    def register_converter(name):
        def decorator(func):
            _global_converter_table[name] = func
            return func
        return decorator

So now, the original example becomes:

    @string.register_converter('join')
    @string.simple_converter
    def my_joiner(values, joinstr):
        return joinstr.join(map(str, values))