
Moving a discussion from the PEP309 SF tracker (Patch #941881) to here, since it's gone beyond the initial PEP 309 concept (and the SF tracker is a lousy place to have a design discussion, anyway). The discussion started when Steven Bethard pointed out that partial objects can't be used as instance methods (using new.instancemethod means that the automatically supplied 'self' argument ends up in the wrong place, after the originally supplied arguments). This problem is mentioned only indirectly in the PEP (pointing out that a version which prepended later arguments, rather than appending them might be useful). Such a class wouldn't solve the problem anyway, as it is only the *first* argument we want to give special treatment. Keyword arguments won't help us, since the 'self' argument is always positional. The initial suggestion was to provide a __get__ method on partial objects, which forces the insertion of the reference to self at the beginning of the argument list instead of at the end: def __get__(self, obj, type=None): if obj is None: return self return partial(self.fn, obj, *self.args, **self.kw) However, this breaks down for nested partial functions - the call to the nested partial again moves the 'self' argument to after the originally supplied argument list. This can be addressed by automatically 'unfolding' nested partials (which should also give a speed benefit when supplying arguments piecemeal, since building incrementally or all at once will get you to the same place): def __init__(*args, **kw): self = args[0] try: func = args[1] except IndexError: raise TypeError("Expected at least 2 arguments, got %s" % len(args)) if isinstance(func, partial): self.fn = func.fn self.args = func.args + args[2:] d = func.kw.copy() d.update(kw) self.kw = d else: self.fn, self.args, self.kw = (func, args[2:], kw) At this point, the one thing you can't do is use a partial function as a *class* method, as the classmethod implementation doesn't give descriptors any special treatment. So, instead of the above, I propose the inclusion of a callable 'partialmethod' descriptor in the functional module that takes the first positional argument supplied at call time and prepends it in the actual function call (this still requires automatic 'unfolding'in order to work correctly with nested partial functions): class partialmethod(partial): def __call__(self, *args, **kw): if kw and self.kw: d = self.kw.copy() d.update(kw) else: d = kw or self.kw if args: first = args[:1] rest = args[1:] else: first = rest = () return self.fn(*(first + self.args + rest), **d) def __get__(self, obj, type=None): if obj is None: return self return partial(self.fn, obj, *self.args, **self.kw) Using a function that simply prints its arguments: Py> class C: ... a = functional.partialmethod(f, 'a') ... b = classmethod(functional.partialmethod(f, 'b')) ... c = staticmethod(functional.partial(f, 'c')) ... d = functional.partial(functional.partialmethod(f, 'd', 1), 2) ... Py> C.e = new.instancemethod(functional.partialmethod(f, 'e'), None, C) Py> C().a(0) ((<__main__.C instance at 0x00A95710>, 'a'), {}, 0) Py> C().b(0) (<class __main__.C at 0x00A93FC0>, 'b', 0) Py> C().c(0) ('c', 0) Py> C().d(0) ('d', 1, 2, 0) Py> C().e(0) (<__main__.C instance at 0x00A95710>, 'e', 0) Notice that you *don't* want to use partialmethod when creating a static method. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

On Sat, 26 Feb 2005 16:50:06 +1000, Nick Coghlan <ncoghlan@iinet.net.au> wrote:
While I see a theoretical need for this functionality, I don't know of a practical use case. PEP 309 is marked as accepted in its current form, and has an implementation in C (which appears to be justified on speed grounds - see the SF tracker). The new code is in Python - if it's to be included, then at *some* stage it needs translating to C. Personally, I'd rather see partial as it stands, with its current limitations, included. The alternative seems to be a potentially long discussion, petering out without conclusion, and the whole thing missing Python 2.5. (I know that's a long way off, but this already happened with 2.4...)
No problem with this, as long as: 1. An implementation (which needs to work alongside the existing C code) is provided 2. PEP 309 is updated to include an explanation of the issue and a justification for this solution. 3. The documentation is updated to make it clear why there are two callables, and which to use where. (That could be difficult to explain clearly - I understand some of the issues here, but just barely - the docs would need to be much clearer). Personally, I feel this classifies as YAGNI (whereas, I'd use the current partial a *lot*). OTOH, I think the fact that the existing partial can't be used as an instance method does deserve mention in the documentation, regardless. For that your explanation (quoted above, and repeated here) would suffice: """ partial objects can't be used as instance methods (using new.instancemethod means that the automatically supplied 'self' argument ends up in the wrong place, after the originally supplied arguments) """ Paul.

Paul Moore wrote:
Yes - I certainly don't think this is a reason to hold off checking in what we already have. The only question is whether or not it gets tweaked before the first 2.5 alpha.
That's the plan - the Python code is just the easiest way to make the intended semantics clear for the discussion.
2. PEP 309 is updated to include an explanation of the issue and a justification for this solution.
Seems like the most sensible place to record it for posterity (assuming we actually end up doing anything about it)
The basic concept is that if you're building a partial function in general, use 'partial'. If you're building an instance method or class method (i.e. the two cases where Python automatically provides the first argument), use 'partialmethod', because the standard 'partial' generally won't do the right thing with the first argument.
Personally, I feel this classifies as YAGNI (whereas, I'd use the current partial a *lot*).
I can certainly appreciate that point of view. It's a rather large conceptual hole though, so if it can be plugged elegantly, I'd like to see that happen before 2.5 goes live.
The distinction is actually finer than that - it can work in some cases, provided the predefined arguments are all given as keyword arguments rather than positional arguments. The real issue is that there may be situations where you don't have control over how the function you want to turn into an instance method was created, and if someone has used partial with positional arguments to create the function, you may have no practical way out. 'partialmethod' fixes that - it allows creating a partial function which expects the next positional argument to be the first argument to the underlying function, while remaining positional arguments are appended as usual. Regards, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Samuele Pedroni wrote:
It certainly *is* a possible definition, but you can get essentially that behaviour using new.instancemethod, so a new descriptor isn't necessary for that case. The suggested descriptor was to get an alternate behaviour which injected the automatically supplied self argument at the start of the list of positional arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

On Sat, 26 Feb 2005 16:50:06 +1000, Nick Coghlan <ncoghlan@iinet.net.au> wrote:
While I see a theoretical need for this functionality, I don't know of a practical use case. PEP 309 is marked as accepted in its current form, and has an implementation in C (which appears to be justified on speed grounds - see the SF tracker). The new code is in Python - if it's to be included, then at *some* stage it needs translating to C. Personally, I'd rather see partial as it stands, with its current limitations, included. The alternative seems to be a potentially long discussion, petering out without conclusion, and the whole thing missing Python 2.5. (I know that's a long way off, but this already happened with 2.4...)
No problem with this, as long as: 1. An implementation (which needs to work alongside the existing C code) is provided 2. PEP 309 is updated to include an explanation of the issue and a justification for this solution. 3. The documentation is updated to make it clear why there are two callables, and which to use where. (That could be difficult to explain clearly - I understand some of the issues here, but just barely - the docs would need to be much clearer). Personally, I feel this classifies as YAGNI (whereas, I'd use the current partial a *lot*). OTOH, I think the fact that the existing partial can't be used as an instance method does deserve mention in the documentation, regardless. For that your explanation (quoted above, and repeated here) would suffice: """ partial objects can't be used as instance methods (using new.instancemethod means that the automatically supplied 'self' argument ends up in the wrong place, after the originally supplied arguments) """ Paul.

Paul Moore wrote:
Yes - I certainly don't think this is a reason to hold off checking in what we already have. The only question is whether or not it gets tweaked before the first 2.5 alpha.
That's the plan - the Python code is just the easiest way to make the intended semantics clear for the discussion.
2. PEP 309 is updated to include an explanation of the issue and a justification for this solution.
Seems like the most sensible place to record it for posterity (assuming we actually end up doing anything about it)
The basic concept is that if you're building a partial function in general, use 'partial'. If you're building an instance method or class method (i.e. the two cases where Python automatically provides the first argument), use 'partialmethod', because the standard 'partial' generally won't do the right thing with the first argument.
Personally, I feel this classifies as YAGNI (whereas, I'd use the current partial a *lot*).
I can certainly appreciate that point of view. It's a rather large conceptual hole though, so if it can be plugged elegantly, I'd like to see that happen before 2.5 goes live.
The distinction is actually finer than that - it can work in some cases, provided the predefined arguments are all given as keyword arguments rather than positional arguments. The real issue is that there may be situations where you don't have control over how the function you want to turn into an instance method was created, and if someone has used partial with positional arguments to create the function, you may have no practical way out. 'partialmethod' fixes that - it allows creating a partial function which expects the next positional argument to be the first argument to the underlying function, while remaining positional arguments are appended as usual. Regards, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

Samuele Pedroni wrote:
It certainly *is* a possible definition, but you can get essentially that behaviour using new.instancemethod, so a new descriptor isn't necessary for that case. The suggested descriptor was to get an alternate behaviour which injected the automatically supplied self argument at the start of the list of positional arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net
participants (3)
-
Nick Coghlan
-
Paul Moore
-
Samuele Pedroni