On Sat, 26 Feb 2005 19:26:11 -0500, Raymond Hettinger wrote:

Are you sure about that? Contriving examples is easy, but download a few modules, scan them for use cases, and you may find, as I did, that partial() rarely applies. The argument order tends to be problematic.

Grepping through the standard library yields no favorable examples.

I also didn't find many the last time I looked through: http://mail.python.org/pipermail/python-list/2004-December/257990.html

In inspect.py, you could replace "formatvarkw=lambda name: '**' + name" with "partial(operator.add, '**') but that would not be an improvement.

Yeah, I remember thinking that the nicer way to write this was probably formatvarkw='**%s'.__mod__ Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy

On Sun, 27 Feb 2005 09:31:26 +0100, "Martin v. Löwis" wrote:

Raymond Hettinger wrote:

...

Are you sure about that? Contriving examples is easy, but download a few modules, scan them for use cases, and you may find, as I did, that partial() rarely applies. The argument order tends to be problematic.

So would you like to see the decision to accept PEP 309 reverted?

Either revert the decision, or apply the patch. I don't feel comfortable advocating that the decision be reverted - according to the CVS log, PEP 309 was accepted by Guido on 31 March 2004, so I don't think it's for me to argue against that decision... (I've already stated my position - I don't have any problem with the function as it stands, and speed is not crucial to me so I have no preference between the C and Python implementations. But it's not the end of the world whatever happens...). Paul.

Raymond Hettinger wrote:

I would like for the principal advocates to reach a consensus that the proposed implementation is a winner.

That I cannot understand. Do you want the advocates to verify that the implementation conforms to the specification? or that the implementation of the PEP is faster than any other existing implementation of the PEP? These two hold, I believe.

Ideally, that decision should be informed by trying it out on their own, real code and seeing whether it offers genuine improvements.

Performance-wise, or usability-wise? Because usability-wise, all implementations of the PEP are identical, so all implementations of the PEP should offer the precisely same improvements.

Along the way, they should assess whether it is as applicable as expected, whether the existing limitations are problematic, and whether performance is an issue.

Ah, so you question the specification, not the implementation of it.

My concern is that with Guido having approved the idea in abstract form, the actual implementation has escaped scrutiny. Also, if the API is different from the PEP, acceptance should not be automatic.

AFAICT, the proposed patch implements the behaviour of the PEP exactly.

If functional.partial() isn't a clear winner, it may be a reasonable to ask that it be released in the wild and evolve before being solidified in the standard library. My sense is that that the current implementation is far from its highest state of evolution.

Again, this I cannot understand. I do believe that there is no better way to implement the PEP. The PEP very explicitly defines what precisely functional.partial is, and the implementation follows that specification very closely. Regards, Martin

Paul Moore wrote:

While I'm not saying that it's too late to attempt to persuade Guido to reverse himself, it does seem to me to be a lot of fuss over a fairly small function - and no-one said anything like this at the time.

I would probably fuss much less if it would not simultaneously introduce a new module as well.

When I put up 5 reviews to get Martin to look at this, I honestly believed that it was a simple case of an accepted PEP with a complete implementation (admittedly scattered over a couple of SF patches), and would simply be a matter of committing it.

That was a fair assumption. However, it turned out that a) people still have doubts about the proposed functionality of the PEP. For some, it does too much, for others, too little. Changing the PEP now would be much cheaper than first committing the changes, and then redoing the PEP again, as we might need to deprecate the functional.partial first. So as part of the review, I need to confirm that there still is no opposition to the PEP (which now appears to be the case) b) it is not obvious that the patch is complete. It probably is, but I would have committed a single patch much quicker than collecting bits and pieces from multiple patches, only to find out that they won't integrate properly. c) it appears that the implementation of the PEP is incorrect (as Raymond just discovered). Again, it is better to require a perfect implementation before committing the changes, instead of pushing the contributor afterwards to add the missing changes.

IMHO, the burden is on those who want the "Accepted" status revoking to persuade Guido to pronounce to that effect.

Most certainly. So far, nobody stepped forward and requested that this status is revoked, so no persuading is necessary. However, as part of the review process, it *is* necessary to check again whether somebody would have preferred that the PEP is revoked - atleast when the acceptance of the PEP is many months old.

Otherwise, based on the standard PEP workflow process, it's time to move on, and ensure that the patches provide a complete implementation, and assuming they do to commit them.

Correct. I would have done so more readily if I knew how the "Accepted" status got into the document. I could have researched that (going through old email archives), or I could just ask whether people agree that the status is indeed "Accepted".

(But I don't want to put myself up as a big "champion" of PEP 309 - I like it, and I'd like to get the "accepted and there's a patch, but not yet implemented" status resolved, but that's all. I'm not going to switch to Perl if the patch isn't accepted :-))

It seems to me that the patch will be committed shortly, assuming somebody corrects the remaining flaws in the implementation. I could do that, but I would prefer if somebody contributed an updated patch. Regards, Martin

...

Along the way, they should assess whether it is as applicable as expected, whether the existing limitations are problematic, and whether performance is an issue.

Ah, so you question the specification, not the implementation of it.

I do believe that there is no better way to implement the PEP. The PEP very explicitly defines what

My only issue with the PEP is that it seemed much more promising when reading it than when looking for real code that could benefit from it. I liked it much better until I tried to use it. My hope is that the advocates will try it for themselves before pushing this one in on faith. precisely

functional.partial is, and the implementation follows that specification very closely.

My reading of the PEP did not include making the structure members public. This complicates and slows the implementation. The notion of introducing mutable state to the PFA is at odds with the driving forces behind functional programming (i.e. statelessness). If necessary for introspection, the structure members can be made read-only. Also, there may be room to improve the implementation by building on the passed-in dictionary rather than creating a copy of the one in the partial object. The current choice may be the correct one because it has the outer call override the defaults in the event of a keyword conflict -- if so, that should be documented. The test for callability is redundant and can be removed. The traverse() function can be simplified with the PyVISIT macro. Overall, I have no major objections to the PEP or the patch. Before it goes in on auto-pilot, it would be darned nice if the proponents said that they've found it helpful in real code and that they are satisfied with the timings. partial(str.__add__, 'Ray')('mond')

I did a quick experiment:

...

python -m timeit -s "from operator import itemgetter; l=range(8)" "itemgetter(1)(l)" 1000000 loops, best of 3: 0.548 usec per loop

...

python -m timeit -s "l=range(8)" "(lambda x:x[1])(l)" 1000000 loops, best of 3: 0.597 usec per loop

That's far less of a difference than I expected from itemgetter!

You've timed how long it takes to both construct and apply the retrieval function. The relevant part is only the application: C:\pydev>python -m timeit -r9 -s "from operator import itemgetter; s=range(8); f=itemgetter(1)" "f(s)" 1000000 loops, best of 9: 0.806 usec per loop C:\pydev>python -m timeit -r9 -s "s=range(8); f=lambda x:x[1]" "f(s)" 100000 loops, best of 9: 1.18 usec per loop So the savings is about 30% which is neither astronomical, nor negligible. Raymond

Nick Coghlan wrote:

Raymond Hettinger wrote:

...

* The instance method limitation never came up for me. However, it bites to have a tool working in a way that doesn't match your mental model. We have to document the limitations, keep them in mind while programming, and hope to remember them as possible causes if bugs ever arise. It would be great if these limitations could get ironed out.

The 'best' idea I've come up with so far is to make partial a class factory instead of a straight class, taking an argument that states how many positional arguments to prepend at call time. A negative value would result in the addition of (len(callargs)+1) to the position at call time.

Other dynamic languages, like Lisp, are in the same boat in this respect--real currying (and things that look like it) doesn't work too well because the API wasn't designed with it in mind (curried languages have functions that take less-frequently-changing parameters first, but most other languages take them last). Scheme has a nice solution for this in SRFI 26 (http://srfi.schemers.org/srfi-26/). It looks like this: (cut vector-set! x <> 0) That produces a function that takes one argument. The <> is an argument slot; for every <> in the cut form, the resultant callable takes another argument. (This explanation is incomplete, and there are some other features; read the SRFI for details.) I've been using something similar in Python for a while, and I really like it. It doesn't look as good because the slot has to be a real object and not punctuation, but it works just as well. For example: cut(islice, cutslot, 0, 2) That's pretty readable to me. My version also allows the resultant callable to take any number of parameters after the slots have been satisfied, so partial is just the special case of no explicit slots. Perhaps a full example will make it clearer: >>> def test(a, b, c): ... print 'a', a, 'b', b, 'c', c ... >>> f = cut(test, cutslot, 'bravo') >>> f('alpha', 'charlie') a alpha b bravo c charlie Here, b is specialized at cut time, a is passed through the slot, and c is passed through the implicit slots at the end. The only thing this can't do is a generic right-"curry"--where we don't know how many parameters come before the one we want to specialize. If someone wants to do that, they're probably better off using keyword arguments. So far, my most common use for this is to specialize the first argument to map, zip, or reduce. Very few cases actually need an explicit cutslot, but those that do (like the islice example above) look pretty good with it. My reasons for using cut instead of lambda are usually cosmetic--the cut form is shorter and reads better when what I'm doing would be a curry in a language designed for that. Throw in a compose function and I almost never need to use lambda in a decorator <ducks and runs from the anti-lambda crowd>.