Parametrized any() and all() ?
Hello any() and all() are very useful small functions, and I am wondering if it could be interesting to have them work with different operators, by using a callable. e.g. something like: import operator def any(iterable, filter=operator.truth): for element in iterable: if filter(element): return True return False For instance I could then us any() to find out if there's a None in the sequence: if any(iterable, op=lambda x: x is None): raise SomeError("There's a none in that list") Granted, it's easy to do it myself in a small util function - but since any() and all() are in Python... Cheers Tarek -- Tarek Ziadé · http://ziade.org · @tarek_ziade
Hey Tarek, I would write that as any(x is None for x in it) -- the example you gave doesn't really strike me as an improvement over that, although I could see how many there are cases where it's nicer... On Wed, Jan 16, 2013 at 11:30 AM, Tarek Ziadé <tarek@ziade.org> wrote:
-- cheers lvh
On 1/16/13 11:33 AM, Laurens Van Houtven wrote:
Hey Tarek,
I would write that as any(x is None for x in it)
But here you're building yet another iterable to adapt it to any(), which seems to me overkill if we can just parametrized the loop in any() Cheers Tarek -- Tarek Ziadé · http://ziade.org · @tarek_ziade
On 2013-01-16, at 11:44 , Tarek Ziadé wrote:
It's just a generator, and will be terminated early if possible. I'm pretty sure adding a key function to any and all has already been submitted several times, and from what I remember it was struck down every time because the use case is covered by Laurens's suggestion: key functions are necessary when you'd otherwise need DSU (because the result is the original input, not the key function's output) but it's not the case for any() and all() Here's the previous/latest instance: http://mail.python.org/pipermail/python-ideas/2012-July/015837.html
On 16/01/13 22:10, Nick Coghlan wrote:
For all we know, adding a filter function will be a pessimization, not an optimization, using more memory and/or being slower than using a generator expression. It certainly isn't clear to me that creating a generator expression like (x is None for x in it) is more expensive than creating a filter function like (lambda x: x is None). -1 on adding a filter function. -- Steven
On 1/16/13 3:10 PM, Steven D'Aprano wrote:
I abandoned the idea, but I'd be curious to understand how creating several iterables with one that has an 'if', can be more efficient than having a single iterable with an 'if'...
-- Tarek Ziadé · http://ziade.org · @tarek_ziade
On 16 Jan, 2013, at 15:52, Tarek Ziadé <tarek@ziade.org> wrote:
Have you any reason to assume that "any(x is None for x in it)" is slow? I wouldn't be surprised if a key argument for any/all would have a higher overhead than the generator expression (if there is any difference). The key function would have to be called after all, with the overhead of normal function calls. Ronald
On 1/16/13 7:47 PM, Antoine Pitrou wrote:
You know, discussing performance without posting benchmark numbers is generally pointless.
Sure, yes, so I tried to implement it by adapting the current any() : http://tarek.pastebin.mozilla.org/2068630 but it is 20% slower in my benchmark. However, I have no idea if my implementation is the right way to do things. Cheers Tarek -- Tarek Ziadé · http://ziade.org · @tarek_ziade
On Fri, Jan 18, 2013 at 10:30 PM, Tarek Ziadé <tarek@ziade.org> wrote:
Resuming an existing frame (i.e. using a generator expression) is almost always going to be faster than going through the argument passing machinery and initialising a *new* frame. Chaining C level iterators together (e.g. map, itertools) is even faster. DSU is great for cases where you need it, but a transformation pipeline is otherwise likely to be faster (or at least not substantially slower). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Jan 18, 2013 at 5:52 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
It took me a sec. :) DSU == "Decorate-Sort-Undecorate". [1] -eric [1] http://en.wikipedia.org/wiki/Decorate-sort-undecorate
On Sat, Jan 19, 2013 at 06:02:32PM -0800, alex23 <wuwei23@gmail.com> wrote:
"In 1989, a random of the journalistic persuasion asked hacker Paul Boutin "What do you think will be the biggest problem in computing in the 90s?" Paul's straight-faced response: "There are only 17,000 three-letter acronyms." (To be exact, there are 26^3 = 17,576.)" Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
Hey Tarek, I would write that as any(x is None for x in it) -- the example you gave doesn't really strike me as an improvement over that, although I could see how many there are cases where it's nicer... On Wed, Jan 16, 2013 at 11:30 AM, Tarek Ziadé <tarek@ziade.org> wrote:
-- cheers lvh
On 1/16/13 11:33 AM, Laurens Van Houtven wrote:
Hey Tarek,
I would write that as any(x is None for x in it)
But here you're building yet another iterable to adapt it to any(), which seems to me overkill if we can just parametrized the loop in any() Cheers Tarek -- Tarek Ziadé · http://ziade.org · @tarek_ziade
On 2013-01-16, at 11:44 , Tarek Ziadé wrote:
It's just a generator, and will be terminated early if possible. I'm pretty sure adding a key function to any and all has already been submitted several times, and from what I remember it was struck down every time because the use case is covered by Laurens's suggestion: key functions are necessary when you'd otherwise need DSU (because the result is the original input, not the key function's output) but it's not the case for any() and all() Here's the previous/latest instance: http://mail.python.org/pipermail/python-ideas/2012-July/015837.html
On 16/01/13 22:10, Nick Coghlan wrote:
For all we know, adding a filter function will be a pessimization, not an optimization, using more memory and/or being slower than using a generator expression. It certainly isn't clear to me that creating a generator expression like (x is None for x in it) is more expensive than creating a filter function like (lambda x: x is None). -1 on adding a filter function. -- Steven
On 1/16/13 3:10 PM, Steven D'Aprano wrote:
I abandoned the idea, but I'd be curious to understand how creating several iterables with one that has an 'if', can be more efficient than having a single iterable with an 'if'...
-- Tarek Ziadé · http://ziade.org · @tarek_ziade
On 16 Jan, 2013, at 15:52, Tarek Ziadé <tarek@ziade.org> wrote:
Have you any reason to assume that "any(x is None for x in it)" is slow? I wouldn't be surprised if a key argument for any/all would have a higher overhead than the generator expression (if there is any difference). The key function would have to be called after all, with the overhead of normal function calls. Ronald
On 1/16/13 7:47 PM, Antoine Pitrou wrote:
You know, discussing performance without posting benchmark numbers is generally pointless.
Sure, yes, so I tried to implement it by adapting the current any() : http://tarek.pastebin.mozilla.org/2068630 but it is 20% slower in my benchmark. However, I have no idea if my implementation is the right way to do things. Cheers Tarek -- Tarek Ziadé · http://ziade.org · @tarek_ziade
On Fri, Jan 18, 2013 at 10:30 PM, Tarek Ziadé <tarek@ziade.org> wrote:
Resuming an existing frame (i.e. using a generator expression) is almost always going to be faster than going through the argument passing machinery and initialising a *new* frame. Chaining C level iterators together (e.g. map, itertools) is even faster. DSU is great for cases where you need it, but a transformation pipeline is otherwise likely to be faster (or at least not substantially slower). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Jan 18, 2013 at 5:52 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
It took me a sec. :) DSU == "Decorate-Sort-Undecorate". [1] -eric [1] http://en.wikipedia.org/wiki/Decorate-sort-undecorate
On Sat, Jan 19, 2013 at 06:02:32PM -0800, alex23 <wuwei23@gmail.com> wrote:
"In 1989, a random of the journalistic persuasion asked hacker Paul Boutin "What do you think will be the biggest problem in computing in the 90s?" Paul's straight-faced response: "There are only 17,000 three-letter acronyms." (To be exact, there are 26^3 = 17,576.)" Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
participants (13)
-
alex23 -
Antoine Pitrou -
Eric Snow -
INADA Naoki -
Laurens Van Houtven -
Masklinn -
Nick Coghlan -
Oleg Broytman -
Oscar Benjamin -
Ronald Oussoren -
Steven D'Aprano -
Tarek Ziadé -
Terry Reedy