Partial function application 'from the right'
Hi, I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. E.g., to create a function which splits a string on commas, you can't say # Won't work when called: split_comma = partial(str.split, sep = ',') and to create a 'log to base 10' function, you can't say # Won't work when called: log_10 = partial(math.log, base = 10.0) because str.split and math.log don't take keyword arguments. PEP-309, which introduces functools.partial, mentions For completeness, another object that appends partial arguments after those supplied in the function call (maybe called rightcurry) has been suggested. 'Completeness' by itself doesn't seem to have been a compelling reason to introduce this feature, but the above cases show that it would be of real use. I've created a patch which adds a 'partial_right' function. The two examples above:
import functools, math
split_comma = functools.partial_right(str.split, ',') split_comma('a,b,c') ['a', 'b', 'c']
log_10 = functools.partial_right(math.log, 10.0) log_10(100.0) 2.0
and a general illustrative one:
def all_args(*args): return args ... functools.partial_right(all_args, 1, 2)(3, 4) (3, 4, 1, 2)
I was prompted to look at this by a recent message on python-dev: Xavier Morel <catch-all@masklinn.net>, Thu, 22 Jan 2009 14:44:41 +0100:
[...] drop(iterable, n) has to be written islice(iterable, n, None) (and of course the naming isn't ideal), and you can't really use functools.partial since the iterator is the first argument (unless there's a way to partially apply only the tail args without kwargs).
Xavier's case becomes:
import functools, itertools drop = functools.partial_right(itertools.islice, None, None) list(drop(range(10), 5)) [5, 6, 7, 8, 9]
The patch adds a 'from_right' member to partial objects, which can be True for the new from-the-right behaviour, or False for the existing from-the-left behaviour. It's quite small, only c.40 lines, plus a c.70-line change to test_functools.py. I imagine a documention patch would be c.20 lines. Would there be any interest in this? Ben.
Ben North wrote:
I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. ... Would there be any interest in this?
I think so. Post your patch to the tracker. Even if eventually rejected, it will be there for people to use.
Hello, Ben North <ben <at> redfrontdoor.org> writes:
I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. E.g., to create a function which splits a string on commas, you can't say
# Won't work when called: split_comma = partial(str.split, sep = ',')
In py3k, we could also use "..." (the Ellipsis object) to denote places where an argument is missing, so that: split_comma = partial(str.split, ..., ',') would do what you want. Regards Antoine.
Antoine Pitrou wrote:
... In py3k, we could also use "..." (the Ellipsis object) to denote places where an argument is missing, so that: split_comma = partial(str.split, ..., ',') would do what you want.
Thus preventing any use of partial when an argument could be an the Ellipsis instance. --Scott David Daniels Scott.Daniels@Acm.Org
Scott David Daniels <Scott.Daniels <at> Acm.Org> writes:
Antoine Pitrou wrote:
... In py3k, we could also use "..." (the Ellipsis object) to denote places where an argument is missing, so that: split_comma = partial(str.split, ..., ',') would do what you want.
Thus preventing any use of partial when an argument could be an the Ellipsis instance.
Obviously, it is the drawback :) But Ellipsis is hardly used anywhere, and it reads good in this very use case.
This discussion probably belongs to python-ideas, but since we already have this thread, I'll reply here instead of opening a new thread there. Ellipsis was introduced into python to serve needs of the numeric python community. If you think of numpy multiarrays as functions taking ndim number of arguments, then ellipsis is used to denote any number of missing arguments and : is used to denote a single missing argument. By this analogy, partial(f, ..., *args) is right_partial with '...' standing for any number of missing arguments. I you want to specify exactly one missing argument, you would want to write partial(f, :, *args), which is not a valid syntax even in Py3. If one is willing to use [] instead of () with partial, it is possible to implement partial[f, ..., *args] and partial[f, x, :, z] already in Py2, but I would rather see : allowed in the argument lists or some other short syntax for missing arguments. If such syntax is introduced, the need for partial may even go away with partial(str.split, :, ',') spelled simply as str.split(:, ','). On Thu, Jan 29, 2009 at 3:04 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Scott David Daniels <Scott.Daniels <at> Acm.Org> writes:
Antoine Pitrou wrote:
... In py3k, we could also use "..." (the Ellipsis object) to denote places where an argument is missing, so that: split_comma = partial(str.split, ..., ',') would do what you want.
Thus preventing any use of partial when an argument could be an the Ellipsis instance.
Obviously, it is the drawback :) But Ellipsis is hardly used anywhere, and it reads good in this very use case.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gma...
Alexander Belopolsky <alexander.belopolsky <at> gmail.com> writes:
By this analogy, partial(f, ..., *args) is right_partial with '...' standing for any number of missing arguments. I you want to specify exactly one missing argument, you would want to write partial(f, :, *args), which is not a valid syntax even in Py3.
Yes, of course, but... the meaning which numpy attributes to Ellipsis does not have to be the same in other libraries. Otherwise this meaning would have been embedded in the interpreter itself, while it hasn't. The point of using Ellipsis in this case is not to be numpy-friendly, but rather to exploit the fact that it is a very rarely used object, and that it has an alternate spelling which suits very well (visually speaking) the purpose being discussed. Regards Antoine.
On Thu, Jan 29, 2009 at 4:04 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Alexander Belopolsky <alexander.belopolsky <at> gmail.com> writes:
By this analogy, partial(f, ..., *args) is right_partial with '...' standing for any number of missing arguments. I you want to specify exactly one missing argument, you would want to write partial(f, :, *args), which is not a valid syntax even in Py3.
Yes, of course, but... the meaning which numpy attributes to Ellipsis does not have to be the same in other libraries. Otherwise this meaning would have been embedded in the interpreter itself, while it hasn't.
The meaning which numpy attributes to Ellipsis is also the meaning that mathematical notation has attached to Ellipsis for a very long time. See: http://en.wikipedia.org/wiki/Ellipsis#In_mathematical_notation -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
On 29-Jan-09, at 3:21 PM, Daniel Stutzbach wrote:
On Thu, Jan 29, 2009 at 4:04 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: Alexander Belopolsky <alexander.belopolsky <at> gmail.com> writes:
By this analogy, partial(f, ..., *args) is right_partial with '...' standing for any number of missing arguments. I you want to specify exactly one missing argument, you would want to write partial(f, :, *args), which is not a valid syntax even in Py3.
Yes, of course, but... the meaning which numpy attributes to Ellipsis does not have to be the same in other libraries. Otherwise this meaning would have been embedded in the interpreter itself, while it hasn't.
The meaning which numpy attributes to Ellipsis is also the meaning that mathematical notation has attached to Ellipsis for a very long time.
And yet, python isn't confined to mathematical notation. *, ** are both overloaded for use in argument lists to no-one's peril, AFAICT. -Mike
On Thu, Jan 29, 2009 at 5:24 PM, Mike Klaas <mike.klaas@gmail.com> wrote:
And yet, python isn't confined to mathematical notation. *, ** are both overloaded for use in argument lists to no-one's peril, AFAICT.
Certainly, but there is no danger of confusion them for multiplication in context, whereas: split_comma = partial(str.split, ..., ',') to me looks like "make ',' the last argument" rather than "make ',' the second argument". -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
On 29-Jan-09, at 3:38 PM, Daniel Stutzbach wrote:
On Thu, Jan 29, 2009 at 5:24 PM, Mike Klaas <mike.klaas@gmail.com> wrote:
And yet, python isn't confined to mathematical notation. *, ** are both overloaded for use in argument lists to no-one's peril, AFAICT.
Certainly, but there is no danger of confusion them for multiplication in context, whereas:
split_comma = partial(str.split, ..., ',')
to me looks like "make ',' the last argument" rather than "make ',' the second argument".
Yes, I agree. I mistakenly thought that that was the proposal under discussion (that partial(f, ..., 2) == right_curry(f, 2)) -Mike
I am just replying to the end of this thread to throw in a reminder about my partial.skip patch, which allows the following usage: split_one = partial(str.split, partial.skip, 1) Not looking to say "mine is better", but if the idea is being given merit, I like the skipping arguments method better than just the "right partial", which I think is confusing combined with keyword and optional arguments. And, this patch already exists. Could it be re-evaluated? On Fri, Jan 30, 2009 at 4:20 PM, Mike Klaas <mike.klaas@gmail.com> wrote:
On 29-Jan-09, at 3:38 PM, Daniel Stutzbach wrote:
On Thu, Jan 29, 2009 at 5:24 PM, Mike Klaas <mike.klaas@gmail.com> wrote:
And yet, python isn't confined to mathematical notation. *, ** are both overloaded for use in argument lists to no-one's peril, AFAICT.
Certainly, but there is no danger of confusion them for multiplication in context, whereas:
split_comma = partial(str.split, ..., ',')
to me looks like "make ',' the last argument" rather than "make ',' the second argument".
Yes, I agree. I mistakenly thought that that was the proposal under discussion (that partial(f, ..., 2) == right_curry(f, 2))
-Mike _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Calvin Spealman <ironfroggy <at> gmail.com> writes:
I am just replying to the end of this thread to throw in a reminder about my partial.skip patch, which allows the following usage:
split_one = partial(str.split, partial.skip, 1)
Not looking to say "mine is better", but if the idea is being given merit, I like the skipping arguments method better than just the "right partial", which I think is confusing combined with keyword and optional arguments. And, this patch already exists. Could it be re-evaluated?
Sorry, where is the patch? If one writes X = partial.skip, it looks quite nice: split_one = partial(str.split, X, 1) Regards Antoine.
On Fri, Jan 30, 2009 at 7:38 PM, Calvin Spealman <ironfroggy@gmail.com> wrote:
I am just replying to the end of this thread to throw in a reminder about my partial.skip patch, which allows the following usage:
split_one = partial(str.split, partial.skip, 1)
Not looking to say "mine is better", but if the idea is being given merit, I like the skipping arguments method better than just the "right partial", which I think is confusing combined with keyword and optional arguments. And, this patch already exists. Could it be re-evaluated?
+1 but I don't know where the patch is. -- Cheers, Leif
http://bugs.python.org/issue1706256 Took me a couple days to catch up on this thread so here is the link for any interested. Could it be possible to reevaluate this? On Sat, Jan 31, 2009 at 2:40 PM, Leif Walsh <leif.walsh@gmail.com> wrote:
On Fri, Jan 30, 2009 at 7:38 PM, Calvin Spealman <ironfroggy@gmail.com> wrote:
I am just replying to the end of this thread to throw in a reminder about my partial.skip patch, which allows the following usage:
split_one = partial(str.split, partial.skip, 1)
Not looking to say "mine is better", but if the idea is being given merit, I like the skipping arguments method better than just the "right partial", which I think is confusing combined with keyword and optional arguments. And, this patch already exists. Could it be re-evaluated?
+1 but I don't know where the patch is.
-- Cheers, Leif
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Mike Klaas wrote:
On 29-Jan-09, at 3:21 PM, Daniel Stutzbach wrote: [...]
The meaning which numpy attributes to Ellipsis is also the meaning that mathematical notation has attached to Ellipsis for a very long time.
And yet, python isn't confined to mathematical notation. *, ** are both overloaded for use in argument lists to no-one's peril, AFAICT.
With the crucial difference that * and ** are purely syntax, but Ellipsis is an object. -Andrew.
On Thu, Jan 29, 2009 at 9:12 AM, Ben North <ben@redfrontdoor.org> wrote:
I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. E.g., to create a function which splits a string on commas, you can't say
First of all, many functions like this are easy to handle yourself. Example:
def split_comma(s): return str.split(s, ',')
That said, it seems to me that if we're going to add to functools.partial, we should go all the way and allow keyword arguments (or a dict of them, if it's otherwise too hard to implement). Otherwise, in another few {days, weeks, months} we'll see another thread like this clamoring for a keyword-sensitive functools.partial. Come to think of it, I would imagine the next iteration would ask for a way to curry arbitrary positional arguments, and I can't come up with a simple and beautiful way to do that off the top of my head. Maybe this is an argument for keeping functools.partial the way it is and forcing developers to write their own currying functions. -- Cheers, Leif
Leif Walsh wrote:
That said, it seems to me that if we're going to add to functools.partial, we should go all the way and allow keyword arguments (or a dict of them, if it's otherwise too hard to implement). Otherwise, in another few {days, weeks, months} we'll see another thread like this clamoring for a keyword-sensitive functools.partial.
functools.partial *does* support keyword arguments - it's just that some functions and methods written in C (such as string methods) *don't*, so partial's keyword support doesn't help. A functools.rpartial would go some way towards addressing that. Using the standalone Ellipsis to denote missing arguments would probably start to miss the whole point of functools.partial: the only reason for its existence is that it is *faster than the equivalent Python function*. If partial starts messing about looking for missing arguments and then slotting them in, then it is likely to slow down to the point where you would be better off skipping it and writing a dedicated function that adds the extra arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Nick Coghlan <ncoghlan <at> gmail.com> writes:
If partial starts messing about looking for missing arguments and then slotting them in, then it is likely to slow down to the point where you would be better off skipping it and writing a dedicated function that adds the extra arguments.
Looking for missing arguments is very cheap, just raw pointer compares (Ellipsis is a singleton). In comparison, the cost of executing a dedicated Python function would be overwhelming.
On Thu, Jan 29, 2009 at 6:12 AM, Ben North <ben@redfrontdoor.org> wrote:
Hi,
I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. E.g., to create a function which splits a string on commas, you can't say
# Won't work when called: split_comma = partial(str.split, sep = ',') [snip] I've created a patch which adds a 'partial_right' function. The two examples above:
import functools, math
split_comma = functools.partial_right(str.split, ',') split_comma('a,b,c') ['a', 'b', 'c']
log_10 = functools.partial_right(math.log, 10.0) log_10(100.0) 2.0
Can you point to real code that this makes more readable? Collin
participants (12)
-
Alexander Belopolsky -
Andrew Bennetts -
Antoine Pitrou -
Ben North -
Calvin Spealman -
Collin Winter -
Daniel Stutzbach -
Leif Walsh -
Mike Klaas -
Nick Coghlan -
Scott David Daniels -
Terry Reedy