Mailman 3 Add a "partial with placeholders" function to the functools module - Python-ideas

27 Jul 2019

      Hello,

currently, regarding positional arguments, `partial` gives us the option to partialize functions from the left. There's been some interest about partializing functions from the right instead (e.g. [SO post, 9k views, 39 upvotes](https://stackoverflow.com/q/7811247/3767239)), especially w.r.t. the various `str` methods.

I propose adding a function to `functools` that works with placeholders and thus offers even greater flexibility. The Ellipsis literal `...` seems a intuitive choice for that task. When eventually calling such a "partial placeholder" object, it would fill in placeholders from the left and add remaining `args` to the right. In terms of implementation this can be realized as a subclass of `partial` itself.

## Implementation

    from functools import partial
    from reprlib import recursive_repr

    class partial_placehold(partial):
        placeholder = Ellipsis

        def __call__(self, /, *args, **keywords):
            args = iter(args)
            try:
                old_args = [x if x is not self.placeholder else next(args) for x in self.args]
            except StopIteration:
                raise TypeError('too few arguments were supplied') from None
            keywords = {**self.keywords, **keywords}
            return self.func(*old_args, *args, **keywords)

        @recursive_repr()
        def __repr__(self):
            qualname = type(self).__qualname__
            args = [repr(self.func)]
            args.extend(repr(x) if x is not self.placeholder else '...' for x in self.args)  # Only this line deviates from `partial.__repr__`; could also factor that out into a separate method.
            args.extend(f"{k}={v!r}" for (k, v) in self.keywords.items())
            if type(self).__module__ == "functools":
                return f"functools.{qualname}({', '.join(args)})"
            return f"{qualname}({', '.join(args)})"

    # Would need to add something for compatibility with `partial`, i.e. for partializing a placeholder function.

## Example

This allows for example the following usage:

    replace_dots_with_underscore = partial_placehold(str.replace, ..., '.', '_')
    replace_dots_with_underscore('foo.bar.baz')

## Relevance

Sure we could also use a `lambda` instead ([as discussed here](https://mail.python.org/archives/list/python-ideas@python.org/message/YD5OQE...)) but there was a reason `partial` was introduced and I think the same arguments apply here too. Though most functions allow partializing via keyword arguments and this is undoubtedly a cleaner way, some might not and for example built-ins' methods won't allow it. Especially Python 3.8's introduction of positional-only parameters (PEP 570) might give rise to cases where `partial` is not sufficient.
In case inspection is desired a `lambda` does not provide much information (sure you could always dig deeper with `inspect` for example but that's not the point). Consider the following example of a pre-defined sequence of default postprocessing steps and the user might add their own or remove existing ones, as appropriate:

    postprocessing_steps = [
        lambda s: s.replace('foo', 'bar'),
    ]
    print(postprocessing_steps[0])  # 

This doesn't give a lot of information about what the lambda actually does (and thus whether the user should remove it or not). Using the `partial_placehold` instead, it's clear what is happening:

    postprocessing_steps = [
        partial_placehold(str.replace, ..., 'foo', 'bar'),
    ]
    print(postprocessing_steps[0])  # partial_placehold(, ..., 'foo', 'bar')

## Compatibility

The proposed solution works with the current syntax and the usage of Ellipsis as a placeholder object is likely not to collide with actually used values (in any case the user might still reassign the `.placeholder` attribute).
Because the direction of partializing is unchanged (still left to right) this doesn't introduce ambiguities which might come with a "right partial" function. Creating a placeholder function from a `partial` object is possible without any changes, the opposite way requires an additional check to result in a placeholder object again.

## Possible confusion

Regarding the usage of Ellipsis right now, in `numpy` or `typing` for example, it always represents a placeholder for multiple "things", not a single one:

    array[..., None]  # All the dimensions of `array` plus a new one.
    typing.Tuple[str, ...]  # Any number of str objects.

So the expectations might be biased in that sense. For example:

    def foo(a, b, c, d):
        pass

    p_foo = partial_placehold(foo, ..., 1, 2)
    p_foo(3, 4)

Someone else reviewing the code might now assume that the `...` means to act as a placeholder for all arguments except the last two (and hence `p_foo(3, 4)` would be equivalent to `foo(3, 4, 1, 2)` while it actually is equivalent to `foo(3, 1, 2, 4)`). But this would be again some kind of "right partial" function and also the function name implies something else; documentation might clarify as well, of course.

## Conclusion

Adding a "partial with placeholders" function to `functools` allows for covering use cases where the standard `partial` is not sufficient. No new syntax is required and the implementation is fairly straightforward given the inheritance from `partial`. Ellipsis `...` seems an intuitive choice for acting as a placeholder (concerning both, conflicts with actual partial values and code readability). There are uses cases where such a function would provide a clean solution and there is an interest in the community (https://stackoverflow.com/q/7811247/3767239, https://stackoverflow.com/q/19701775/3767239 for example). Especially with the introduction of positional-only parameters new use cases are likely to arise.

-----

**Related threads:**

* https://mail.python.org/archives/list/python-ideas@python.org/message/TVNCM7... - Mentions essentially a similar idea.

The original [PEP 309 -- Partial Function Application](https://www.python.org/dev/peps/pep-0309/) also mentions:
...
Partially applying arguments from the right, or inserting arguments at arbitrary positions creates its own problems, but pending discovery of a good implementation and non-confusing semantics, I don't think it should be ruled out.

Add a "partial with placeholders" function to the functools module

Dominik Vilsmeier

Anders Hovmöller

Andrew Barnert

Anders Hovmöller

Andrew Barnert

MRAB

Guido van Rossum

Dominik Vilsmeier

Steven D'Aprano

David Mertz

Dominik Vilsmeier

tags

participants (7)