Mailman 3 Predicate Sets - Python-ideas

newer
Re: [Python-ideas] Add `n_threads`...

Predicate Sets

Daniel da Silva

20 Jan 2014 20 Jan '14

5:11 a.m.

Below is a description of a very simple but immensely useful class called a "predicate set". In combination with the set and list comprehensions they would allow another natural layer of reasoning with mathematical set logic in Python. In my opinion, a concept like this would be best located in the functools module. *Overview:* Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion. A predicate set would be a set-like class that is instantiated with a predicate function that is called to determine ``a in the_predicate_set''.

...

...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False 'abc' in myset True len(myself) Traceback (most recent call last): [...] TypeError

*Example Uses:* # Dynamic excludes in searching foo_files = search_files('foo', exclude=set(['a.out', 'Makefile'])) bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~'))) # exclude *~ # Use in place of a set with an ORM validusernames = predicateset(lambda s: re.match(s, '[a-zA-Z0-9]+')) class Users(db.Model): username = db.StringProperty(choices=validusernames) password = db.StringProperty()

Attachments:

attachment.htm (text/html — 1.7 KB)

Show replies by date

Ian Foote

20 Jan 20 Jan

5:34 a.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 19/01/14 23:41, Daniel da Silva wrote:

...

Below is a description of a very simple but immensely useful class called a "predicate set". In combination with the set and list comprehensions they would allow another natural layer of reasoning with mathematical set logic in Python.

In my opinion, a concept like this would be best located in the functools module.

*Overview:* Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion. A predicate set would be a set-like class that is instantiated with a predicate function that is called to determine ``a in the_predicate_set''.

...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False 'abc' in myset True len(myself) Traceback (most recent call last): [...] TypeError * * *Example Uses:* # Dynamic excludes in searching foo_files = search_files('foo', exclude=set(['a.out', 'Makefile'])) bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~'))) # exclude *~

# Use in place of a set with an ORM validusernames = predicateset(lambda s: re.match(s, '[a-zA-Z0-9]+'))

class Users(db.Model): username = db.StringProperty(choices=validusernames) password = db.StringProperty()

Hi Daniel, That's an interesting idea. I'm not sure it would be used enough to include in the standard library though. Have you considered releasing an implementation on PyPI? That has the advantage that people can start using it earlier than would be possible if it was added to the standard library. Regards, Ian -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJS3GgrAAoJEODsV4MF7PWzI1EH/0FKiJYKZgRd6iW04Ic9NPXw QL+EKQU0UdRjCvP9IWrBSdGYnmB06YHdwyeLPpk0+amGSzXpsMGNRHtAXhxjba00 1Q9UKHnVcIj3kgjfYg+LKezMVJHQF4vE+umrbMQFeWBt7FEKfqseCbyDRIZAm9I8 G/dOzP3dxC4lktlCtLv6sfVD8D648A9wMNX5879SoUKjX+Qs0ySZ9CVxhBbyFVgP kXLG1/9NlmkyJmWsL6hHwWYI9WwnJ433Ts74bqmwOaTDlGdmmZNHfQT5kIHzRK8V g8XXZWxct8EVvTjyL+//n+DuSsFEDxhXTX0gGXMs0xDXunbDBHWNggs9G2B+GI0= =WsXD -----END PGP SIGNATURE-----

Terry Reedy

6:04 a.m.

On 1/19/2014 6:41 PM, Daniel da Silva wrote:

...

Below is a description of a very simple but immensely useful class called a "predicate set". In combination with the set and list comprehensions they would allow another natural layer of reasoning with mathematical set logic in Python.

Sets defined by predicates are usually infinite and mathematical set logic works fine with such.

...

*Overview:* Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion. A predicate set would be a set-like class that is instantiated with a predicate function that is called to determine ``a in the_predicate_set''.

...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False 'abc' in myset True len(myself) Traceback (most recent call last): [...] TypeError

This illustrates the problem with the idea. Only containment is really straightforward. (I am aware that some operations could be implemented by defining new predicates. To combines sets with predicatesets, the sets would have to be represented by predicates, as done below.)

...

*Example Uses:* # Dynamic excludes in searching foo_files = search_files('foo', exclude=set(['a.out', 'Makefile'])) bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~'))) # exclude *~

# Use in place of a set with an ORM validusernames = predicateset(lambda s: re.match(s, '[a-zA-Z0-9]+'))

I think these examples are backwards. The APIs should accept functions either in addition to or instead of collections. It is trivial to turn a collection into a predicate

...

...
...
p = {'a', 'b', 'c'}.__contains__ p('a') True p('d') False

You need realistic examples that use other operations (but not len ;-). -- Terry Jan Reedy

Devin Jeanpierre

6:47 a.m.

On Sun, Jan 19, 2014 at 3:41 PM, Daniel da Silva wrote:

...

Below is a description of a very simple but immensely useful class called a "predicate set". In combination with the set and list comprehensions they would allow another natural layer of reasoning with mathematical set logic in Python.

Efficiently implementing the set operators (intersection, union, etc.) requires using ROBDDs (reduced ordered binary decision diagrams), which are complex enough to deserve their _own_ library. It's not a simple task, and shouldn't be written from scratch. That said, if you implemented it, and did it efficiently, I'd find it hugely helpful. I ended up implementing it on my own in a bit of a brute force fashion once (I used truth tables instead of BDDs): https://bitbucket.org/devin.jeanpierre/replay/src/4ca3e412e511a9af87c335303c... (I make no claims to this being good or correct code) -- Devin

Andrew Barnert

1:26 p.m.

From: Daniel da Silva Sent: Sunday, January 19, 2014 3:41 PM

...

Overview: Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion.

The whole point of modern set theory is that sets cannot be defined by a predicate alone; only by a predicate _and a set to apply it over_. Which we already have in set comprehensions. And your suggestion has the exact same problem that naive set theory had:

...

...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False

...

...
...
russellset = predicateset(lambda s: s not in s) russellset in russelset

Presumably this should cause the computer to scream "DOES NOT COMPUTE!" and blow up, which I think would be hard to implement in CPython. Still, this could be useful despite not being mathematically consistent. Python functions don't have to be mathematical functions, and you could easily just state that using a predicateset that turns out to be a proper class as undefined behavior, so it's perfectly acceptable if an implementation wants to hang forever or fail with a recursion error or whatever. Anyway, the way you've designed this, as far as I can tell, there's nothing stopping it from being a module on PyPI that you can come back and propose for inclusion in the stdlib if a lot of people start using it. So I'd say go for it. (And you can even propose syntax, a comprehension with no for clause: {x if expression(x)}, if it's popular enough that seems warranted.) Also, this isn't a Set in Python terms—or an Iterable or a Sized; it's just a Container. Which is perfectly reasonable, and means len(s) and iter(s) failing is exactly what you should expect. But the name could lead people to expect it to be a Set. Then again, "predicatecontainer" sounds horrible, so maybe the small potential for confusion is fine. You still need to work out the details. Most of them seem easy, but there are some interesting questions. * It's presumably immutable, and therefore Hashable. (It can fail if its predicate isn't—which most callables are, but that's not guaranteed—but I believe that's fine for Hashables.) * Is the predicate callable accessible through a public name, or do you have to access it through __contains__? * Presumably intersection, union, difference, and symmetric_difference with another predicateset do the obvious thing (or/and/and not/xor the predicates). Or is there something more efficient you could do? There are some modules on PyPI that deal with boolean combinations of predicates; maybe just borrow the design or even import the implementation from one of them? * intersection with a set or other Iterable can return a set, equivalent to {x for x in s if x in ps}. And __rand__ allows it to work in the wrong direction when using the operator. But set.intersection(predicateset) will raise a TypeError, and there's not much you can do about that. (And the same goes for the other methods.) * union, difference, and symmetric difference with an Iterable presumably turns the other argument into a predicateset(x in s) and then operates on that? Or is there a better way to do it? * isdisjoint with a set or other iterable is easy, but what about with another predicateset? An error? * issubset and issuperset don't seem implementable, except in the special case that one predicate is made by intersection or union from the other; do they just not exist? * Do you want other operations from naive set theory that don't make sense for Python sets, like the unconstrained complement? They could all be implemented with the existing operations and a set of all things (e.g., self.complement() is just predicateset(lambda x: True)).difference(self)), so maybe not. But they might be convenient. (Again, tying in with the boolean-predicates libraries, most of them have a "not" type operation.) The big problem is coming up with a compelling use case. This one doesn't sell me: bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~'))) It seems like it make more sense to have exclude take a function, so you could just write: bar_files = search_files('bar', exclude=lambda fname: not fname.endswith('~')) In general, calling a function is just as easy, natural, and readable as testing membership; calling filter or using a comprehension would generally be simpler than creating a predicateset just to use intersection; etc. And in cases where sometimes a container is useful, but sometimes a function is better… well, look at re.sub or BeautifulSoup.find. I've seen people who didn't know that you could pass a function to re.sub, but nobody who, on seeing it, had any trouble understanding what it did. Maybe there's a use for "legacy" APIs that were designed around containers and would be hard to change. For example, many file-picker dialogs let you specify the acceptable extensions, but not a filter function. But in most cases, that's because they're ultimately calling some underlying C/ObjC/.NET/whatever function that needs an array, and a predicateset won't help there anyway. (Or, put another way, they're not designed around containers, they're designed around iterables.)

Chris Angelico

1:39 p.m.

On Mon, Jan 20, 2014 at 6:56 PM, Andrew Barnert wrote:

...

Also, this isn't a Set in Python terms—or an Iterable or a Sized; it's just a Container. Which is perfectly reasonable, and means len(s) and iter(s) failing is exactly what you should expect. But the name could lead people to expect it to be a Set. Then again, "predicatecontainer" sounds horrible, so maybe the small potential for confusion is fine.

If I might be permitted to bikeshed the name a little: My first thought (from the subject line) was that this was a set *of* predicates, not a set *defined by a* predicate. But a frozenset isn't a set of frozens either, so this might be less confusing than I thought. ChrisA

Devin Jeanpierre

3:56 p.m.

On Sun, Jan 19, 2014 at 11:56 PM, Andrew Barnert wrote:

...

From: Daniel da Silva

...
Overview: Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion.

The whole point of modern set theory is that sets cannot be defined by a predicate alone; only by a predicate _and a set to apply it over_. Which we already have in set comprehensions.

And your suggestion has the exact same problem that naive set theory had:

...
...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False

...
...
...
russellset = predicateset(lambda s: s not in s) russellset in russelset

Presumably this should cause the computer to scream "DOES NOT COMPUTE!" and blow up, which I think would be hard to implement in CPython.

Still, this could be useful despite not being mathematically consistent.

No; what you have shown is that a predicateset can't both accept the function you specified, and also have its containment method always return a value (as opposed to raising an exception or not halting). You have not shown that the idea of a predicateset is inherently contradictory, unless that idea includes both of those facts -- and that would indeed be silly, since, as you've shown, that is an idea with self-contradicting requirements. In contrast, naive set theory thought all of those things: a set can be defined in that way, and a set either contains something or not, but not neither and not both. And Russell proved that this is impossible. There is not any kind of fundamental problem with the idea of a Python set-like object defined by Python predicates. Python sets aren't mathematical sets, and Python predicates aren't mathematical predicates. Things can be different from how they are described in mathematics, without being internally inconsistent, and without being useless. [...]

...

The big problem is coming up with a compelling use case. [...] In general, calling a function is just as easy, natural, and readable as testing membership; calling filter or using a comprehension would generally be simpler than creating a predicateset just to use intersection; etc.

Yes. If a predicate set is just a thin wrapper around predicates, it is pointless. IMO the only utility of specially wrapping predicates is allowing them to be combined efficiently, but the bulk of the work there is just in manipulating sets of bitvectors (best done with ROBDDs as far as I know). Arguably the work after that is trivial. -- Devin

Andrew Barnert

21 Jan 21 Jan

4:22 a.m.

On Jan 20, 2014, at 2:26, Devin Jeanpierre wrote:

...

There is not any kind of fundamental problem with the idea of a Python set-like object defined by Python predicates. Python sets aren't mathematical sets, and Python predicates aren't mathematical predicates. Things can be different from how they are described in mathematics, without being internally inconsistent, and without being useless.

I made the exact same point in the rest of the paragraph that you cut off, except I said that python functions aren't mathematical functions instead of saying predicates. The original post was suggesting that Python should have predicateset because that's how mathematicians define sets. That is wrong--and, more importantly, irrelevant. Whether a predicateset class is useful or not has to do with its usefulness in writing and reading Python programs, and nothing else. Maybe I should have made the point about it being irrelevant first, and just mentioned the fact that it's wrong as a parenthetical comment. But I'm just too fond of the idea of being able to write a program that Captain Kirk or Zoe Heriot can use to blow up the computer after it takes over the world, which sadly Python does not yet have. (If I remember right, the computer Zoe did it to was programmed in Algol.)

Georg Brandl

12:35 a.m.

Am 20.01.2014 08:56, schrieb Andrew Barnert:

...

From: Daniel da Silva Sent: Sunday, January 19, 2014 3:41 PM

...
Overview: Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion.

The whole point of modern set theory is that sets cannot be defined by a predicate alone; only by a predicate _and a set to apply it over_. Which we already have in set comprehensions.

And your suggestion has the exact same problem that naive set theory had:

...
...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False

...
...
...
russellset = predicateset(lambda s: s not in s) russellset in russelset

Presumably this should cause the computer to scream "DOES NOT COMPUTE!" and blow up...

I think it will just raise a NameError... SCNR, Georg

David Mertz

1:40 a.m.

Although a cute point, I'm not too concerned about the Russell's Paradox issue. The obvious implementation will get a "RuntimeError: maximum recursion depth exceeded" in that case. But then, no predicate is guaranteed to halt, so that's not really special to the russellset. On the other hand, even though I think the idea of a 'predicateset' is cute mathematically, I'm not really sure what it actually gets you, even in readability. I am perfectly happy spelling this: mypset = predicateset(somefunc) if x in mypset: ... As: if somefunc(x): ... Even for the set operators, set comprehensions seem pretty much equally elegant: such_that = {1, 2, 3} & mypset # Looks nice, I agree But then, this looks pretty nice also: such_that = {x for x in {1, 2, 3} if somefunc(x)} OK, sure the predicateset version might save a few characters, but not all that many. If you want to combine predicate sets that's really just like combining predicates. It *does* sort of remind me that I'd like some standard HOFs as builtins or in the standard library (probably in functools). But still, where you might write: in_both_sets = mypset & mypset2 It's not bad to write a small support module: # combinators.py def allP(*fns): return lambda x: all(f(x) for f in fns) def anyP(*fns): return lambda x: any(f(x) for f in fns) Then express the intersection as: in_both_pred = allP(somefunc, somefunc2)

...

From there, you can just use the predicate 'in_both_pred' as above. Similarly for union, define:

in_either_pred = anyP(somefunc, somefunc2) On Mon, Jan 20, 2014 at 11:05 AM, Georg Brandl wrote:

...

Am 20.01.2014 08:56, schrieb Andrew Barnert:

...
From: Daniel da Silva Sent: Sunday, January 19, 2014 3:41 PM

...
Overview: Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion.

The whole point of modern set theory is that sets cannot be defined by a predicate alone; only by a predicate _and a set to apply it over_. Which we already have in set comprehensions.

And your suggestion has the exact same problem that naive set theory had:

...
...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False

...
...
...
russellset = predicateset(lambda s: s not in s) russellset in russelset

Presumably this should cause the computer to scream "DOES NOT COMPUTE!" and blow up...

I think it will just raise a NameError...

SCNR, Georg

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

Steven D'Aprano

6:37 a.m.

On Sun, Jan 19, 2014 at 11:56:49PM -0800, Andrew Barnert wrote:

...

And your suggestion has the exact same problem that naive set theory had:

...

...
...
...
russellset = predicateset(lambda s: s not in s) russellset in russelset

Presumably this should cause the computer to scream "DOES NOT COMPUTE!" and blow up, which I think would be hard to implement in CPython.

It should just raise an exception. I leave implementation as an exercise for the reader :-) This sort of thing is a staple of bad old science fiction, where the Hero would save the world by getting the super-intelligent Artificial Intelligence Doomsday Computer to calculate some variation of the above. But of course, a *truely* intelligent computer would merely say "I see what you did there. Good try, feeble meatbag, but not good enough" and launch the missiles.

...

The big problem is coming up with a compelling use case. This one doesn't sell me:

bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~')))

If it's a project on PyPI, the only use-case necessary is the author thinks it's cool.

...

It seems like it make more sense to have exclude take a function, so you could just write:

bar_files = search_files('bar', exclude=lambda fname: not fname.endswith('~'))

What if you want to filter according to multiple conditions? A tuple of functions makes sense. Add a helper function that tests against those multiple functions, and you're halfway to this PredicateSet. Adding set-like methods seems like overkill. -- Steven

Haoyi Li

7:21 a.m.

...

What if you want to filter according to multiple conditions?

What's wrong with lambda fname: func1(fname) and func2(fname) and func3(fname) ? On Mon, Jan 20, 2014 at 5:07 PM, Steven D'Aprano wrote:

...

On Sun, Jan 19, 2014 at 11:56:49PM -0800, Andrew Barnert wrote:

...
And your suggestion has the exact same problem that naive set theory had:

...
...
...
...
russellset = predicateset(lambda s: s not in s) russellset in russelset

Presumably this should cause the computer to scream "DOES NOT COMPUTE!" and blow up, which I think would be hard to implement in CPython.

It should just raise an exception. I leave implementation as an exercise for the reader :-)

This sort of thing is a staple of bad old science fiction, where the Hero would save the world by getting the super-intelligent Artificial Intelligence Doomsday Computer to calculate some variation of the above. But of course, a *truely* intelligent computer would merely say "I see what you did there. Good try, feeble meatbag, but not good enough" and launch the missiles.

...
The big problem is coming up with a compelling use case. This one doesn't sell me:

bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~')))

If it's a project on PyPI, the only use-case necessary is the author thinks it's cool.

...
It seems like it make more sense to have exclude take a function, so you could just write:

bar_files = search_files('bar', exclude=lambda fname: not fname.endswith('~'))

What if you want to filter according to multiple conditions? A tuple of functions makes sense. Add a helper function that tests against those multiple functions, and you're halfway to this PredicateSet. Adding set-like methods seems like overkill.

-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Steven D'Aprano

8 a.m.

On Mon, Jan 20, 2014 at 05:51:36PM -0800, Haoyi Li wrote:

...

...
What if you want to filter according to multiple conditions?

What's wrong with

lambda fname: func1(fname) and func2(fname) and func3(fname)

That is a single compound condition, not multiple conditions. Think about a GUI application with a file selection dialog box, or a search utility. You might offer a rich set of filters, all optional, all selectable by the user at runtime: [x] Hidden dot files .foo [ ] Backup files foo~ [x] File extensions: [ ] Images [x] Text files [ ] Java code [x] Custom: [ zip,tar,foo,bar,baz ] [x] File owner: [ steve ] [ ] Group: [ ] [ ] Modified date between: [ ] and [ ] etc. It's not practical to create one single giant filter function that looks like this: def filter(name): head, ext = os.path.splitext(name) return ( (show_hidden_dot_files and name.startswith('.')) and (show_backup_tilde_files and name.endswith('~')) and (show_images and ext in list_of_image_extensions) and ... ) It would be a pain to maintain and extend, and testing would be horrible. Better to have each setting provide a single filter function, then combine the active filters into a list: def filter(name, list_of_filters): for f in list_of_filters: if not f(name): return False return True One might even use a class to represent the list of filters, and give it "all" and "any" methods, and allow multiple lists to combine so you can say things like: "show the file if *all* of these conditions are true, or if *any* of these different conditions are true, but not if *any* of these conditions are true" which of course is terribly overkill for a simple file selection dialog box, but might be useful for a more complex search engine. None of this should be read as supporting the original request to add PredicateSet into the standard library. But I encourage the OP to write his own library and put it on PyPI. -- Steven

Haoyi Li

2:20 p.m.

Doesn't *all(map(list_of_filters, value))* do what we want here, then? Maybe *imap* if you want the early bailout behavior. I'm all for having *all*, *map*, *any*, etc. be methods rather than top-level-functions (yay, less namespace pollution!), but if we're talking about a list of functions, it seems we can do exactly what we want very concisely using normal list- and function- operations. On Mon, Jan 20, 2014 at 6:30 PM, Steven D'Aprano wrote:

...

On Mon, Jan 20, 2014 at 05:51:36PM -0800, Haoyi Li wrote:

...
...
What if you want to filter according to multiple conditions?

What's wrong with

lambda fname: func1(fname) and func2(fname) and func3(fname)

That is a single compound condition, not multiple conditions.

Think about a GUI application with a file selection dialog box, or a search utility. You might offer a rich set of filters, all optional, all selectable by the user at runtime:

[x] Hidden dot files .foo [ ] Backup files foo~ [x] File extensions: [ ] Images [x] Text files [ ] Java code [x] Custom: [ zip,tar,foo,bar,baz ] [x] File owner: [ steve ] [ ] Group: [ ] [ ] Modified date between: [ ] and [ ]

etc. It's not practical to create one single giant filter function that looks like this:

def filter(name): head, ext = os.path.splitext(name) return ( (show_hidden_dot_files and name.startswith('.')) and (show_backup_tilde_files and name.endswith('~')) and (show_images and ext in list_of_image_extensions) and ... )

It would be a pain to maintain and extend, and testing would be horrible. Better to have each setting provide a single filter function, then combine the active filters into a list:

def filter(name, list_of_filters): for f in list_of_filters: if not f(name): return False return True

One might even use a class to represent the list of filters, and give it "all" and "any" methods, and allow multiple lists to combine so you can say things like:

"show the file if *all* of these conditions are true, or if *any* of these different conditions are true, but not if *any* of these conditions are true"

which of course is terribly overkill for a simple file selection dialog box, but might be useful for a more complex search engine.

None of this should be read as supporting the original request to add PredicateSet into the standard library. But I encourage the OP to write his own library and put it on PyPI.

-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Haoyi Li

2:28 p.m.

...

*all(map(list_of_filters, value))*

Scratch that, what I actually want is *all(map(lambda f: f(value), list_of_filters))* I always mix up the order of things going into *map* =( On Tue, Jan 21, 2014 at 12:50 AM, Haoyi Li wrote:

...

Doesn't

*all(map(list_of_filters, value))*

do what we want here, then? Maybe *imap* if you want the early bailout behavior.

I'm all for having *all*, *map*, *any*, etc. be methods rather than top-level-functions (yay, less namespace pollution!), but if we're talking about a list of functions, it seems we can do exactly what we want very concisely using normal list- and function- operations.

On Mon, Jan 20, 2014 at 6:30 PM, Steven D'Aprano wrote:

...
On Mon, Jan 20, 2014 at 05:51:36PM -0800, Haoyi Li wrote:

...
...
What if you want to filter according to multiple conditions?

What's wrong with

lambda fname: func1(fname) and func2(fname) and func3(fname)

That is a single compound condition, not multiple conditions.

Think about a GUI application with a file selection dialog box, or a search utility. You might offer a rich set of filters, all optional, all selectable by the user at runtime:

[x] Hidden dot files .foo [ ] Backup files foo~ [x] File extensions: [ ] Images [x] Text files [ ] Java code [x] Custom: [ zip,tar,foo,bar,baz ] [x] File owner: [ steve ] [ ] Group: [ ] [ ] Modified date between: [ ] and [ ]

etc. It's not practical to create one single giant filter function that looks like this:

def filter(name): head, ext = os.path.splitext(name) return ( (show_hidden_dot_files and name.startswith('.')) and (show_backup_tilde_files and name.endswith('~')) and (show_images and ext in list_of_image_extensions) and ... )

It would be a pain to maintain and extend, and testing would be horrible. Better to have each setting provide a single filter function, then combine the active filters into a list:

def filter(name, list_of_filters): for f in list_of_filters: if not f(name): return False return True

One might even use a class to represent the list of filters, and give it "all" and "any" methods, and allow multiple lists to combine so you can say things like:

"show the file if *all* of these conditions are true, or if *any* of these different conditions are true, but not if *any* of these conditions are true"

which of course is terribly overkill for a simple file selection dialog box, but might be useful for a more complex search engine.

None of this should be read as supporting the original request to add PredicateSet into the standard library. But I encourage the OP to write his own library and put it on PyPI.

-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Serhiy Storchaka

2:39 p.m.

21.01.14 10:58, Haoyi Li написав(ла):

...

...
*all(map(list_of_filters, value))*

Scratch that, what I actually want is

*all(map(lambda f: f(value), list_of_filters))* * * I always mix up the order of things going into *map* =(* *

all(f(value) for f in list_of_filters) looks cleaner to me. Perhaps slightly more efficient (but much less readable) form: all(map(operator.methodcaller('__call__', value), list_of_filters)

David Mertz

22 Jan 22 Jan

12:09 a.m.

Isn't that exactly what I suggested up-thread with my suggested small library of combinators? E.g.: def allP(*fns): return lambda x: all(f(x) for f in fns) I like encapsulating it better since it encourages naming such combined functions, e.g.: this_and_that = allP((this, that)) I feel like that encourages reuse and readability when one later wants to write: set_with_predicate = {x for x in baseset if this_and_that(x)} Or: if this_and_that(x): ... On Tue, Jan 21, 2014 at 1:09 AM, Serhiy Storchaka wrote:

...

21.01.14 10:58, Haoyi Li написав(ла):

...
...
*all(map(list_of_filters, value))*

Scratch that, what I actually want is

*all(map(lambda f: f(value), list_of_filters))* * * I always mix up the order of things going into *map* =(* *

all(f(value) for f in list_of_filters)

looks cleaner to me.

Perhaps slightly more efficient (but much less readable) form:

all(map(operator.methodcaller('__call__', value), list_of_filters)

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

spir

21 Jan 21 Jan

3:23 a.m.

On 01/20/2014 12:41 AM, Daniel da Silva wrote:

...

Below is a description of a very simple but immensely useful class called a "predicate set". In combination with the set and list comprehensions they would allow another natural layer of reasoning with mathematical set logic in Python.

In my opinion, a concept like this would be best located in the functools module.

*Overview:* Sets in mathematics can be defined by a list of elements without repetitions, and alternatively by a predicate (function) that determines inclusion. A predicate set would be a set-like class that is instantiated with a predicate function that is called to determine ``a in the_predicate_set''.

...
...
myset = predicateset(lambda s: s.startswith('a')) 'xyz' in myset False 'abc' in myset True len(myself) Traceback (most recent call last): [...] TypeError

*Example Uses:* # Dynamic excludes in searching foo_files = search_files('foo', exclude=set(['a.out', 'Makefile'])) bar_files = search_files('bar', exclude=predicateset(lambda fname: not fname.endswith('~'))) # exclude *~

# Use in place of a set with an ORM validusernames = predicateset(lambda s: re.match(s, '[a-zA-Z0-9]+'))

class Users(db.Model): username = db.StringProperty(choices=validusernames) password = db.StringProperty()

While the theoretical interest is clear, I don't see the actual point. A predicate set without any actual set (in the ordinary prog sense) is just a criterion function (the predicate) returning a logical true/false, right? (Note: any logical func, any logical expression on a variable, does define a predicate set, doesn't it?) So, we already have this builtin ;-).

...

...
...
crit = lambda s: s.startswith('a') crit("xyz") False crit("abc") True

One could make a trivial class to build such constructs as objects and implement the 'in' operator for them. class PredSet: def __init__ (self, crit): self.crit = crit def __contains__ (self, x): return self.crit(x) crit = lambda s: s.startswith('a') s = PredSet(crit) print("xyz" in s, "abc" in s) But I don't see any advantage in terms of clarity: crit(x) is as clear, isn't it. One also could add an actual set to such objects, which would automagically put items inside, eg whenever they are checked via the criterion func. (Somewhat like string pools.) class PredSet: def __init__ (self, crit): self.crit = crit self.items = set() def __contains__ (self, x): if self.crit(x): self.items.add(x) return True return False s = PredSet(crit) print("xyz" in s, "abc" in s, "ablah" in s) print(s.items) Would certainly be nice, but I cannot see any usage. All in all, I guess I'm missing the actual point. Denis

3742

Age (days ago)

3744

Last active (days ago)

List overview

Download

17 comments

12 participants

participants (12)

Andrew Barnert
Chris Angelico
Daniel da Silva
David Mertz
Devin Jeanpierre
Georg Brandl
Haoyi Li
Ian Foote
Serhiy Storchaka
spir
Steven D'Aprano
Terry Reedy

Predicate Sets

Daniel da Silva

Haoyi Li

Haoyi Li

Haoyi Li

spir

tags

participants (12)