Mailman 3 Non-boolean return from __contains__ - Python-ideas

Non-boolean return from contains

older
PEP 380 alternative: A yielding...

Alex Gaynor

25 Jul 2010 25 Jul '10

11:45 p.m.

Recently I've been wondering why __contains__ casts all of it's returns to be boolean values. Specifically I'd like to propose that __contains__'s return values be passed directly back as the result of the `in` operation. As a result I'd further propose the introduction of __not_contains__, which is the `not in` operator. The primary usecase for this is something like an expression recorder. For example in SQLAlchemy one can do: User.id == 3, but not User.id in SQLList([1, 2, 3]), because it returns a bool always. __not_contains__ is needed to be the analog of this, as it cannot be merely be a negation of __contains__ when it's returning a non-bool result. There should be no backwards compatibility issues to making __contains__ return non-bools, unless there is code like: x = y in foo assert type(x) is bool However, for __not_contains__ I'd propose the default implementation be: def __not_contains__(self, val): x = val in self if type(x) is not bool: raise TypeError("%s returned a non-boolean value from __contains__ and didn't provide an implementation of __not_contains__") return not x This is not perfect (and it's at odds with the fact that __ne__ doesn't return not self == other), but it seems to allow both the desired flexibility and backwards compatibility. I'm not sure if this is something that'd be covered by the language moratorium, but if not I can try putting together a patch for this. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me

Show replies by date

Raymond Hettinger

26 Jul 26 Jul

12:18 a.m.

On Jul 25, 2010, at 11:15 AM, Alex Gaynor wrote:

...

Recently I've been wondering why __contains__ casts all of it's returns to be boolean values. Specifically I'd like to propose that __contains__'s return values be passed directly back as the result of the `in` operation.

x = y in z # where x is a non boolean. Yuck. One of the beautiful aspects of __contains__ is that its simply signature allows it to be used polymorphically throughout the whole language. It would be ashamed to throw-away this virtue so that you can have a operator version of something that should really be a method (like find() for example). -1 on the proposal because it makes the language harder to grok while conferring only a dubious benefit (replacing well named methods with a non-descriptive use of an operator). There is no "natural" interpretation of an in-operator returning a non-boolean. If the above snippet assigns "foo" to x, what does that mean? If it assigns -10, what does that mean? Language design is about associating meanings (semantics) with syntax. ISTM, this would be poor design. Raymond

Michael Foord

3:02 a.m.

On 25 July 2010 19:48, Raymond Hettinger wrote:

...

On Jul 25, 2010, at 11:15 AM, Alex Gaynor wrote:

...
Recently I've been wondering why __contains__ casts all of it's returns to be boolean values. Specifically I'd like to propose that __contains__'s return values be passed directly back as the result of the `in` operation.

x = y in z # where x is a non boolean.

Yuck.

How is it any worse than: x = y > z # where x is a non boolean And all the other operators that already do this? Michael

...

One of the beautiful aspects of __contains__ is that its simply signature allows it to be used polymorphically throughout the whole language. It would be ashamed to throw-away this virtue so that you can have a operator version of something that should really be a method (like find() for example).

-1 on the proposal because it makes the language harder to grok while conferring only a dubious benefit (replacing well named methods with a non-descriptive use of an operator).

There is no "natural" interpretation of an in-operator returning a non-boolean. If the above snippet assigns "foo" to x, what does that mean? If it assigns -10, what does that mean? Language design is about associating meanings (semantics) with syntax. ISTM, this would be poor design.

Raymond _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- http://www.voidspace.org.uk

Raymond Hettinger

5:31 a.m.

On Jul 25, 2010, at 2:32 PM, Michael Foord wrote:

...

x = y in z # where x is a non boolean.

Yuck.

How is it any worse than:

x = y > z # where x is a non boolean

And all the other operators that already do this?

Terrible sales technique: "how is this any worse than ..." ;-) Other operations such as rich comparisons have complicated our lives but had sufficient offsetting benefits than made them more bearable. Rich comparisons cause no end of trouble but at least they allow the numeric folks to implement some well studied behaviors than have proven benefits in their domain. In contrast, this proposal offers almost zero benefit to offset the pain it will cause. The OP didn't even offer a compelling use case or a single piece of code that wouldn't be better written with a normal method. No existing code expects "in" to return a non-boolean. A lot of code for containers or that uses containers implicitly expects simple invariants to hold: for x in container: assert x in container Raymond P.S. With rich comparisons, we've lost basics assumptions like equality operations being reflexsive, symmetric, and transitive. We should be cautioned by that experience and only go down that path again if there is a darned good reason.

Alex Gaynor

7:50 a.m.

On Sun, Jul 25, 2010 at 7:01 PM, Raymond Hettinger wrote:

...

On Jul 25, 2010, at 2:32 PM, Michael Foord wrote:

...
x = y in z # where x is a non boolean.

Yuck.

How is it any worse than:

x = y > z # where x is a non boolean

And all the other operators that already do this?

Terrible sales technique: "how is this any worse than ..." ;-) Other operations such as rich comparisons have complicated our lives but had sufficient offsetting benefits than made them more bearable. Rich comparisons cause no end of trouble but at least they allow the numeric folks to implement some well studied behaviors than have proven benefits in their domain. In contrast, this proposal offers almost zero benefit to offset the pain it will cause. The OP didn't even offer a compelling use case or a single piece of code that wouldn't be better written with a normal method. No existing code expects "in" to return a non-boolean. A lot of code for containers or that uses containers implicitly expects simple invariants to hold: for x in container: assert x in container

Raymond

P.S. With rich comparisons, we've lost basics assumptions like equality operations being reflexsive, symmetric, and transitive. We should be cautioned by that experience and only go down that path again if there is a darned good reason.

Fundamentally the argument in favor of it is the same as for the other comparison operators: you want to do symbolic manipulation using the "normal" syntax, as a DSL. My example is that of a SQL expression builder: SQLAlchemy uses User.id == 3 to create a clause where the ID is 3, but for "id in [1, 2, 3]" it has: User.id.in_([1, 2, 3]), which is rather unseamly IMO (at least as much as having User.id.eq(3) would be). Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me

Mathias Panzenböck

27 Jul 27 Jul

7:59 p.m.

On 07/26/2010 04:20 AM, Alex Gaynor wrote:

...

Fundamentally the argument in favor of it is the same as for the other comparison operators: you want to do symbolic manipulation using the "normal" syntax, as a DSL. My example is that of a SQL expression builder: SQLAlchemy uses User.id == 3 to create a clause where the ID is 3, but for "id in [1, 2, 3]" it has: User.id.in_([1, 2, 3]), which is rather unseamly IMO (at least as much as having User.id.eq(3) would be).

This is a bad example for your wish because this code:

...

...
...
id in [1, 2, 3]

translates into:

...

...
...
[1, 2, 3].__contains__(id)

So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes. -panzi

Guido van Rossum

9:29 p.m.

On Tue, Jul 27, 2010 at 3:29 PM, Mathias Panzenböck wrote:

...

On 07/26/2010 04:20 AM, Alex Gaynor wrote:

...
Fundamentally the argument in favor of it is the same as for the other comparison operators: you want to do symbolic manipulation using the "normal" syntax, as a DSL. My example is that of a SQL expression builder: SQLAlchemy uses User.id == 3 to create a clause where the ID is 3, but for "id in [1, 2, 3]" it has: User.id.in_([1, 2, 3]), which is rather unseamly IMO (at least as much as having User.id.eq(3) would be).

This is a bad example for your wish because this code:

...
...
...
id in [1, 2, 3]

translates into:

...
...
...
[1, 2, 3].__contains__(id)

So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-( -- --Guido van Rossum (python.org/~guido)

Alex Gaynor

9:32 p.m.

On Tue, Jul 27, 2010 at 10:59 AM, Guido van Rossum wrote:

...

On Tue, Jul 27, 2010 at 3:29 PM, Mathias Panzenböck wrote:

...
On 07/26/2010 04:20 AM, Alex Gaynor wrote:

...
Fundamentally the argument in favor of it is the same as for the other comparison operators: you want to do symbolic manipulation using the "normal" syntax, as a DSL. My example is that of a SQL expression builder: SQLAlchemy uses User.id == 3 to create a clause where the ID is 3, but for "id in [1, 2, 3]" it has: User.id.in_([1, 2, 3]), which is rather unseamly IMO (at least as much as having User.id.eq(3) would be).

This is a bad example for your wish because this code:

...
...
...
id in [1, 2, 3]

translates into:

...
...
...
[1, 2, 3].__contains__(id)

So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

-- --Guido van Rossum (python.org/~guido)

Well, in my original example I wrapped the list with a SQLList() container class. I thought of the issue before, but it hardly seems like a blocker, the numpy stuff is unaffected for example: they're not using a builtin container, and for myself I'm willing to wrap my lists to get the pretty syntax. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me

Guido van Rossum

9:37 p.m.

On Tue, Jul 27, 2010 at 9:02 AM, Alex Gaynor wrote:

...

On Tue, Jul 27, 2010 at 10:59 AM, Guido van Rossum wrote:

...
On Tue, Jul 27, 2010 at 3:29 PM, Mathias Panzenböck wrote:

...
On 07/26/2010 04:20 AM, Alex Gaynor wrote:

...
Fundamentally the argument in favor of it is the same as for the other comparison operators: you want to do symbolic manipulation using the "normal" syntax, as a DSL. My example is that of a SQL expression builder: SQLAlchemy uses User.id == 3 to create a clause where the ID is 3, but for "id in [1, 2, 3]" it has: User.id.in_([1, 2, 3]), which is rather unseamly IMO (at least as much as having User.id.eq(3) would be).

This is a bad example for your wish because this code:

...
...
...
id in [1, 2, 3]

translates into:

...
...
...
[1, 2, 3].__contains__(id)

So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

-- --Guido van Rossum (python.org/~guido)

Well, in my original example I wrapped the list with a SQLList() container class. I thought of the issue before, but it hardly seems like a blocker, the numpy stuff is unaffected for example: they're not using a builtin container, and for myself I'm willing to wrap my lists to get the pretty syntax.

Well, writing "x in wrapper(y)" is hardly prettier than "contains(y, x)", if you compare it to "x in y". And it is certainly another thing that can go wrong in a non-obvious way. -- --Guido van Rossum (python.org/~guido)

Alexander Belopolsky

10:12 p.m.

On Tue, Jul 27, 2010 at 11:59 AM, Guido van Rossum wrote: ..

...

...
So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

It may kill a use case or two, but not the proposal. In the libraries like numpy where all python containers get replaced, this is not an issue. Also this problem invites __rcontains__ solution, but the proposal is not very attractive to begin with. IMO, operators that are not symbols such as +, - or &, but words such as 'in', 'not' or 'and' don't offer much advantage over function calls.

Michael Foord

10:19 p.m.

On 27 July 2010 17:42, Alexander Belopolsky wrote:

...

On Tue, Jul 27, 2010 at 11:59 AM, Guido van Rossum wrote: ..

...
...
So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

It may kill a use case or two, but not the proposal. In the libraries like numpy where all python containers get replaced, this is not an issue. Also this problem invites __rcontains__ solution,

Wasn't the lack of an __rcontains__ a problem for the web-sig guys trying to work out the bytes / strings issue? For what it's worth I think that guido is correct that a better solution for the expression -> query problem is to introduce an expression tree, as is done for LINQ (which has been enormously popular amongst .NET developers). All the best, Michael Foord

...

but the proposal is not very attractive to begin with. IMO, operators that are not symbols such as +, - or &, but words such as 'in', 'not' or 'and' don't offer much advantage over function calls. _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- http://www.voidspace.org.uk

Ian Bicking

28 Jul 28 Jul

12:59 a.m.

On Tue, Jul 27, 2010 at 11:49 AM, Michael Foord wrote:

...

On 27 July 2010 17:42, Alexander Belopolsky < alexander.belopolsky@gmail.com> wrote:

...
On Tue, Jul 27, 2010 at 11:59 AM, Guido van Rossum wrote: ..

...
...
So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

It may kill a use case or two, but not the proposal. In the libraries like numpy where all python containers get replaced, this is not an issue. Also this problem invites __rcontains__ solution,

Wasn't the lack of an __rcontains__ a problem for the web-sig guys trying to work out the bytes / strings issue?

I think PJE wanted to implement a string type that was bytes+encoding (as opposed to using Python's native strings). You can overload __add__ etc so everything works, but you couldn't make this work: encodedbytes(b'1234', 'utf8') in '12345' because '12345'.__contains__ would reject the encodedbytes type outright. __rcontains__ would work because here '12345' would know that it didn't understand encodedbytes. It wouldn't work for lists though, as [].__contains__ can handle *any* type, as it just tests for equality across all of its members. So it's not like __radd__ because the original object can't know that it should defer to the other argument. -- Ian Bicking | http://blog.ianbicking.org

Cameron Simpson

3:40 a.m.

On 27Jul2010 16:59, Guido van Rossum wrote: | On Tue, Jul 27, 2010 at 3:29 PM, Mathias Panzenböck | wrote: | > On 07/26/2010 04:20 AM, Alex Gaynor wrote: | >> | >> Fundamentally the argument in favor of it is the same as for the other | >> comparison operators: you want to do symbolic manipulation using the | >> "normal" syntax, as a DSL. My example is that of a SQL expression | >> builder: SQLAlchemy uses User.id == 3 to create a clause where the ID | >> is 3, but for "id in [1, 2, 3]" it has: User.id.in_([1, 2, 3]), which | >> is rather unseamly IMO (at least as much as having User.id.eq(3) would | >> be). | >> | > | > This is a bad example for your wish because this code: | >>>> id in [1, 2, 3] | > | > translates into: | >>>> [1, 2, 3].__contains__(id) | > | > So it doesn't help that 'in' may return something else than a bool | > because the method is called on the wrong object for your purposes. | | Well that pretty much kills the proposal. I can't believe nobody | (including myself) figured this out earlier in the thread. :-( That's a real shame. ".__rcontains__", anyone? For the record (since I just said +0.5 to +1), I'm down to +0 on the proposal; I think the idea's good and removes an (to my mind) arbitrary constraint on __contains__, but now I haven't got a use case:-( Alex's "id in wrapper([1,2,3])" doesn't seem better than the existing "column.in_([1,2,3])" that already exists, alas. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ "Are we alpinists, or are we tourists" followed by "tourists! tourists!" - Kobus Barnard in rec.climbing, on things he's heard firsthand

Greg Ewing

6:33 a.m.

Guido van Rossum wrote:

...

On Tue, Jul 27, 2010 at 3:29 PM, Mathias Panzenböck

...
So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

Alternatively, it could be taken as a sign that there is a special method missing -- there should be an __in__ method that is tried on the first operand before trying __contains__ on the second. (And if we'd thought of this at the beginning, __contains__ would have been called __rin__. :-) -- Greg

Raymond Hettinger

6:39 a.m.

On Jul 27, 2010, at 6:03 PM, Greg Ewing wrote:

...

Guido van Rossum wrote:

...
On Tue, Jul 27, 2010 at 3:29 PM, Mathias Panzenböck

...
So it doesn't help that 'in' may return something else than a bool because the method is called on the wrong object for your purposes.

Well that pretty much kills the proposal. I can't believe nobody (including myself) figured this out earlier in the thread. :-(

Alternatively, it could be taken as a sign that there is a special method missing -- there should be an __in__ method that is tried on the first operand before trying __contains__ on the second. (And if we'd thought of this at the beginning, __contains__ would have been called __rin__. :-)

Don't forget __not_in__ and __not_rin__ ;-) Raymond

Ian Bicking

26 Jul 26 Jul

8:31 a.m.

On Sun, Jul 25, 2010 at 7:01 PM, Raymond Hettinger < raymond.hettinger@gmail.com> wrote:

...

P.S. With rich comparisons, we've lost basics assumptions like equality operations being reflexsive, symmetric, and transitive. We should be cautioned by that experience and only go down that path again if there is a darned good reason.

Well, it's done, and this would simply complete that. I don't see how holding the line at __contains__ help anything; we've gone most of the way down the path, and this just gets us a little further along (and, or, and not still being an issue). Also I don't think this affects reflexive/symmetric/transitive aspects of equality, since any overloading allows that to happen, right? Rich comparisons affect other things. I assume this would fall under the moratorium though? -- Ian Bicking | http://blog.ianbicking.org

Guido van Rossum

8:38 a.m.

On Sun, Jul 25, 2010 at 8:01 PM, Ian Bicking wrote:

...

I assume this would fall under the moratorium though?

Since it's not adding new syntax, that's debatable. -- --Guido van Rossum (python.org/~guido)

Guido van Rossum

8:37 a.m.

On Sun, Jul 25, 2010 at 5:01 PM, Raymond Hettinger wrote:

...

On Jul 25, 2010, at 2:32 PM, Michael Foord wrote:

...
x = y in z # where x is a non boolean.

Yuck.

How is it any worse than:

x = y > z # where x is a non boolean

And all the other operators that already do this?

Terrible sales technique: "how is this any worse than ..." ;-)

Whoa. Reformulate as a consistency argument and I totally buy it.

...

Other operations such as rich comparisons have complicated our lives but had sufficient offsetting benefits than made them more bearable. Rich comparisons cause no end of trouble but at least they allow the numeric folks to implement some well studied behaviors than have proven benefits in their domain.

True. The argument for rich comparisons is that "A = B <= C" where B and C are matrices of the same shape could return a matrix of bools of the same shape, like a generalization for "A = [b <= c for b, c in zip(B, C)]".

...

In contrast, this proposal offers almost zero benefit to offset the pain it will cause. The OP didn't even offer a compelling use case or a single piece of code that wouldn't be better written with a normal method.

OTOH, there is a similar use case: "A = B in C" could be defined by the author of a matrix type as (the similar generalization of) "A = [b in c for b, c in zip(B, C)]". This is still somewhat less compelling than for rich comparisons because the elements of C corresponding to those in would have to be sequences. But it is not totally uncompelling. BTW Alex *did* mention a use case: expression recoding like SQLAlchemy.

...

No existing code expects "in" to return a non-boolean.

Most existing code also doesn't care, and all predefined implementations of __contains__ will still return bools. It's only folks like NumPy who would care. We should ask them though -- they had a chance to ask for this when rich comparisons were introduced, but apparently didn't.

...

A lot of code for containers or that uses containers implicitly expects simple invariants to hold: for x in container: assert x in container

Yeah, a lot of code using comparisons also breaks when comparisons don't return bools. It's a specialized use, but I don't see it as anathema. OTOH the real solution would be something like LINQ in C# (http://msdn.microsoft.com/en-us/netframework/aa904594.aspx, http://en.wikipedia.org/wiki/Language_Integrated_Query).

...

Raymond

P.S. With rich comparisons, we've lost basics assumptions like equality operations being reflexive, symmetric, and transitive. We should be cautioned by that experience and only go down that path again if there is a darned good reason.

So where's the pain? I don't recall ever seeing a report from someone who was bitten by this. -- --Guido van Rossum (python.org/~guido)

Masklinn

11:20 a.m.

On 2010-07-26, at 05:07 , Guido van Rossum wrote:

...

...
A lot of code for containers or that uses containers implicitly expects simple invariants to hold: for x in container: assert x in container

Yeah, a lot of code using comparisons also breaks when comparisons don't return bools. It's a specialized use, but I don't see it as anathema.

OTOH the real solution would be something like LINQ in C# (http://msdn.microsoft.com/en-us/netframework/aa904594.aspx, http://en.wikipedia.org/wiki/Language_Integrated_Query).

Most of LINQ itself (the LINQ library, as opposed to the query syntaxes which are solely syntactic sugar and statically compiled into LINQ method calls) can already be implemented in Python. The things that might be missing are *some* LINQ-supporting features. Likely expression trees[0], maybe (but probably not) less limited and terser lambdas. [0] http://msdn.microsoft.com/en-us/library/bb397951.aspx

Guido van Rossum

7:58 p.m.

On Sun, Jul 25, 2010 at 10:50 PM, Masklinn wrote:

...

On 2010-07-26, at 05:07 , Guido van Rossum wrote:

...
...
A lot of code for containers or that uses containers implicitly expects simple invariants to hold: for x in container: assert x in container

Yeah, a lot of code using comparisons also breaks when comparisons don't return bools. It's a specialized use, but I don't see it as anathema.

OTOH the real solution would be something like LINQ in C# (http://msdn.microsoft.com/en-us/netframework/aa904594.aspx, http://en.wikipedia.org/wiki/Language_Integrated_Query).

Most of LINQ itself (the LINQ library, as opposed to the query syntaxes which are solely syntactic sugar and statically compiled into LINQ method calls) can already be implemented in Python.

Well, the point of allowing more general __contains__ overloading is exactly to improve upon the query syntax -- you may call it syntactic sugar (often a derogatory term), but you currently cannot translate an 'in' operator into a parse tree like you can for '<' or '+'. (The other odd ducks are 'and' and 'or', though in a pinch one can use '&' and '|' for those. I forget in which camp 'not' falls.

...

The things that might be missing are *some* LINQ-supporting features. Likely expression trees[0], maybe (but probably not) less limited and terser lambdas.

[0] http://msdn.microsoft.com/en-us/library/bb397951.aspx

That's exactly the point I am driving at here. :-) -- --Guido van Rossum (python.org/~guido)

Masklinn

8:18 p.m.

On 2010-07-26, at 16:28 , Guido van Rossum wrote:

...

On Sun, Jul 25, 2010 at 10:50 PM, Masklinn wrote:

...
On 2010-07-26, at 05:07 , Guido van Rossum wrote:

...
...
A lot of code for containers or that uses containers implicitly expects simple invariants to hold: for x in container: assert x in container

Yeah, a lot of code using comparisons also breaks when comparisons don't return bools. It's a specialized use, but I don't see it as anathema.

OTOH the real solution would be something like LINQ in C# (http://msdn.microsoft.com/en-us/netframework/aa904594.aspx, http://en.wikipedia.org/wiki/Language_Integrated_Query).

Most of LINQ itself (the LINQ library, as opposed to the query syntaxes which are solely syntactic sugar and statically compiled into LINQ method calls) can already be implemented in Python.

Well, the point of allowing more general __contains__ overloading is exactly to improve upon the query syntax -- you may call it syntactic sugar (often a derogatory term) I didn't intend it as such, I just meant that there is nothing the LINQ query syntax allows which isn't available (usually more clearly as far as I'm concerned) via the library part of the same.

...

but you currently cannot translate an 'in' operator into a parse tree like you can for '<' or '+'. Why not? How would it be different from + or bool no?

...

...
The things that might be missing are *some* LINQ-supporting features. Likely expression trees[0], maybe (but probably not) less limited and terser lambdas.

[0] http://msdn.microsoft.com/en-us/library/bb397951.aspx

That's exactly the point I am driving at here. :-) Oh well, I probably missed the hints then :(

Oleg Broytman

8:23 p.m.

On Mon, Jul 26, 2010 at 07:28:30AM -0700, Guido van Rossum wrote:

...

other odd ducks are 'and' and 'or', though in a pinch one can use '&' and '|' for those. I forget in which camp 'not' falls.

'not' is like 'and' and 'or'. '~' is like '&' and '|'; its magic method is __invert__. I maintain a similar all-magic-methods-overridden class, just for a different ORM (SQLObject), so I am greatly interested in the discussion. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

Ian Bicking

9:39 p.m.

On Mon, Jul 26, 2010 at 9:53 AM, Oleg Broytman wrote:

...

On Mon, Jul 26, 2010 at 07:28:30AM -0700, Guido van Rossum wrote:

...
other odd ducks are 'and' and 'or', though in a pinch one can use '&' and '|' for those. I forget in which camp 'not' falls.

'not' is like 'and' and 'or'. '~' is like '&' and '|'; its magic method is __invert__.

~ actually works okay (except for it reading poorly -- it's a rather obscure operator), but & and | have precedence that makes them very error-prone in my experience, e.g., (query.column == 'x' | query.column == 'y') becomes the SQL "(column == ('x' OR column)) == 'y'". I know overriding "and" and "or" has been discussed some time ago, though I'm not sure what the exact reason is that it never went anywhere. I remember it as one of those epic-and-boring comp.lang.python threads ;) One obvious complication is that they are short-circuit operations. That is, "a and b" must evaluate a, figure out if it is false, and if so then it returns a. But "a or b" must evaluate a, figure out if it is TRUE, and if so then returns a. So if there was anything like __and__ and __or__ it would have to simply disable any short-circuiting. -- Ian Bicking | http://blog.ianbicking.org

Alexander Belopolsky

9:53 p.m.

On Mon, Jul 26, 2010 at 12:09 PM, Ian Bicking wrote: ..

...

I know overriding "and" and "or" has been discussed some time ago, though I'm not sure what the exact reason is that it never went anywhere.

Indeed. See PEP 335. http://www.python.org/dev/peps/pep-0335/.

...

I remember it as one of those epic-and-boring comp.lang.python threads ;) One obvious complication is that they are short-circuit operations. That is, "a and b" must evaluate a, figure out if it is false, and if so then it returns a. But "a or b" must evaluate a, figure out if it is TRUE, and if so then returns a. So if there was anything like __and__ and __or__ it would have to simply disable any short-circuiting.

The PEP deals with short-circuiting AFAIK.

Greg Ewing

27 Jul 27 Jul

12:58 p.m.

Ian Bicking wrote:

...

So if there was anything like __and__ and __or__ it would have to simply disable any short-circuiting.

Not true -- in PEP 335 I showed how it can be done without giving up any short-circuiting possibilities. -- Greg

Alexander Belopolsky

26 Jul 26 Jul

8:24 p.m.

On Mon, Jul 26, 2010 at 10:28 AM, Guido van Rossum wrote: ..

...

Well, the point of allowing more general __contains__ overloading is exactly to improve upon the query syntax -- you may call it syntactic sugar (often a derogatory term), but you currently cannot translate an 'in' operator into a parse tree like you can for '<' or '+'.

FWIW, I am +0 on a more general __contains__ and query or expression building are the areas where I could use it. Since we are at it, can we have a decision on whether mp_length and sq_length will change their signatures to return PyObject* rather than Py_ssize_t. In other words, are we going to allow virtual sequences like py3k range to have length greater than sys.maxsize? There is an old open issue that turns on this decision: http://bugs.python.org/issue2690 Here I am -1 on making the change but +1 on the range patch in the issue.

Greg Ewing

27 Jul 27 Jul

12:03 p.m.

Guido van Rossum wrote:

...

The other odd ducks are 'and' and 'or',

Well, I tried to do something about that with the Overloaded Boolean Operators proposal, but it seems to have met with a not-very-enthusiastic response. Have you had any more thoughts about it? Do you think it's a problem worth solving, or are '&' and '|' good enough? Or would you rather see some completely different mechanism introduced for getting parse trees from expressions, a la LINQ?

...

I forget in which camp 'not' falls.

If I remember correctly, it's not currently overridable independently from __bool__, but there would be no difficulty in making it so, because there is no control flow involved. -- Greg

Guido van Rossum

7:32 p.m.

On Mon, Jul 26, 2010 at 11:33 PM, Greg Ewing wrote:

...

Guido van Rossum wrote:

...
The other odd ducks are 'and' and 'or',

Well, I tried to do something about that with the Overloaded Boolean Operators proposal, but it seems to have met with a not-very-enthusiastic response.

Have you had any more thoughts about it? Do you think it's a problem worth solving, or are '&' and '|' good enough? Or would you rather see some completely different mechanism introduced for getting parse trees from expressions, a la LINQ?

I think that the approach of overloading tons of operators will always cause a somewhat cramped style, even if we fix some individual operators. There just are too many things that can go wrong, and they will be hard to debug for the *users* of the libraries that provide this stuff. (One reason is that the easy generalization from 1-2 simple examples often fails for non-obvious reasons.) Therefore I think the LINQ approach, which (IIUC) converts an expression into a parse tree when certain syntax is encountered, and calls a built-in method with that parse tree, would be a fresh breath of air. No need deriding it just because Microsoft came up with it first. -- --Guido van Rossum (python.org/~guido)

Robert Kern

9:55 p.m.

On 7/27/10 9:02 AM, Guido van Rossum wrote:

...

Therefore I think the LINQ approach, which (IIUC) converts an expression into a parse tree when certain syntax is encountered, and calls a built-in method with that parse tree, would be a fresh breath of air. No need deriding it just because Microsoft came up with it first.

I've occasionally wished that we could repurpose backticks for expression literals: expr = `x + y*z` assert isinstance(expr, ast.Expression) And triple backticks for blocks of statements: block = ``` try: frobnicate() except FrobError: print("Not on my watch!") ``` assert isinstance(block, ast.Module) Too bad backticks look like grit on Tim's monitor! -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

Masklinn

10:12 p.m.

On 2010-07-27, at 18:25 , Robert Kern wrote:

...

On 7/27/10 9:02 AM, Guido van Rossum wrote:

...
Therefore I think the LINQ approach, which (IIUC) converts an expression into a parse tree when certain syntax is encountered, and calls a built-in method with that parse tree, would be a fresh breath of air. No need deriding it just because Microsoft came up with it first.

I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

And triple backticks for blocks of statements:

block = ``` try: frobnicate() except FrobError: print("Not on my watch!") ``` assert isinstance(block, ast.Module)

Too bad backticks look like grit on Tim's monitor!

What about french quotes expr = «x + y * z» block = ««« try: frobnicate() except FrobError: print("Oh no you di'n't") »»» ? Or maybe some question marks? expr = ¿x + y * z?

Antoine Pitrou

10:35 p.m.

On Tue, 27 Jul 2010 18:42:32 +0200 Masklinn wrote:

...

...
Too bad backticks look like grit on Tim's monitor!

What about french quotes

expr = «x + y * z»

You should require non-breaking spaces (U+00A0 or U+202F) between them and the enclosed expression: expr = « x + y * z » (with U+00A0) expr = « x + y * z » (with U+202F) (I hope my editor doesn't fool me here)

...

block = ««« try: frobnicate() except FrobError: print("Oh no you di'n't") »»»

This would be quite a derogatory use of French quotes.

Bruce Leban

10:55 p.m.

The idea of LINQ is that you write the expression directly in the language and it translates into a query expression. It's going to be operating on an expression parse tree, right? Rather than trying to change the allowable expressions maybe the question is to figure out how to translate what we have and find what we can't express with what we have (and that's an orthogonal question and has nothing to do with __xxx__ functions). On Tue, Jul 27, 2010 at 9:42 AM, Masklinn wrote:

...

What about french quotes

expr = «x + y * z»

Isn't there are already a syntax for this? expr = lambda: x + y * z Maybe you want some conversion of that lambda into a different form: expr = @ast lambda: x + y + z --- Bruce http://www.vroospeak.com http://google-gruyere.appspot.com

MRAB

28 Jul 28 Jul

12:54 a.m.

Bruce Leban wrote:

...

The idea of LINQ is that you write the expression directly in the language and it translates into a query expression. It's going to be operating on an expression parse tree, right? Rather than trying to change the allowable expressions maybe the question is to figure out how to translate what we have and find what we can't express with what we have (and that's an orthogonal question and has nothing to do with __xxx__ functions).

On Tue, Jul 27, 2010 at 9:42 AM, Masklinn mailto:masklinn@masklinn.net> wrote:

What about french quotes

expr = «x + y * z»

Isn't there are already a syntax for this?

expr = lambda: x + y * z

Maybe you want some conversion of that lambda into a different form:

expr = @ast lambda: x + y + z

Or: results = db => sql_expression where the parse tree for "sql_expression" is passed to db.__linq__. The parse tree is compiled to SQL and cached for possible future use.

Ian Bicking

12:54 a.m.

On Tue, Jul 27, 2010 at 12:25 PM, Bruce Leban wrote:

...

The idea of LINQ is that you write the expression directly in the language and it translates into a query expression. It's going to be operating on an expression parse tree, right? Rather than trying to change the allowable expressions maybe the question is to figure out how to translate what we have and find what we can't express with what we have (and that's an orthogonal question and has nothing to do with __xxx__ functions).

On Tue, Jul 27, 2010 at 9:42 AM, Masklinn wrote:

...
What about french quotes

expr = «x + y * z»

Isn't there are already a syntax for this?

expr = lambda: x + y * z

Maybe you want some conversion of that lambda into a different form:

expr = @ast lambda: x + y + z

There's also an unexecuted expression in generator expressions, which is prettier than lambda. There's two places where I've seen people doing this in Python (not counting the operator overloading, of which there are many examples). The first is DejaVu ( http://www.aminus.net/dejavu/chrome/common/doc/trunk/managing.html#Querying) which decompiles lambdas. Then it does partial translation to SQL, and I think will actually execute things in Python when they can't be translated (e.g., if you are using a Python function on a database-derived result). But it can easily break between Python versions, and only works with CPython. It also seems to have some fairly complex rules about partial evaluation. The other place is peak.rules (http://pypi.python.org/pypi/PEAK-Rules) which uses a strings for conditions. My understanding is that the string is compiled to an AST and then analyzed, so partial expressions shared by many conditions can be efficiently evaluated together. Also it changes scopes (the expression is defined outside the function, but evaluated in the context of specific function arguments). Maybe it'd be helpful to consider actual examples in the context of SQL... def users_over_age(minimum_age=timedelta(years=18)): return User.select("datetime.now() - user.birth_date > minimum_age") # or... return User.select(datetime.now() - User.birth_date > minimum_age) def users_with_addresses(): return User.select("sql_exists(Address.select('address.user_id == user.id'))") # or ... return User.select(sql_exists(Address.select(Address.user_id == User.user_id)) def users_in_list(list_of_users_or_ids): list_of_ids = [item.id if isinstance(item, User) else item for item in list_of_users_or_ids] return User.select("user.id in list_of_ids") # or ... return User.select(sql_in(User.id, list_of_ids)) Well, I'm not seeing any advantage. You could do things like: def valid_email(email): # Obviously write it better... return re.match(r'[a-z0-9]+@[a-z0-9.-]+', email) def users_with_valid_email(): return User.select("valid_email(user.email)") and then have it detect (ala DejaVu) that valid_email() cannot be translated to SQL, so select everything then filter it with that function. This looks clever, but usually this kind of transparency will only bite; as in this example, what looks like it might be a normal kind of query is actually an extremely expensive query that might take a very long time to complete. (My impression is that LINQ is clever like this too, allowing expressions that are evaluated in part in different locations?) I was worried about binding arguments, but potentially it can work nicely. E.g., all these only take a single variable from the outer scope, but imagine something like: def expired_users(time=timedelta(days=30): return User.select("user.last_login < (datetime.now() - time)") if you were clever you could detect that "datetime.now() - time" can be statically computed. If you weren't clever you might send the expression to the database (which actually isn't terrible). But maybe consider a case: def users_with_ip(ip): return User.select("user.last_login_ip == encode_ip(ip)") where encode_ip does something like turn dotted numbers into an integer. If the mapper is clever it might tell that there are no SQL expressions in the arguments to encode_ip, and it can evaluate it early. Except... what if the function does something like return a random number? Then you've changed things by evaluating it once instead of for every user. So maybe you can't do that optimization, and so the only way to make this work is to create a local variable to make explicit that you only want to evaluate the argument once. As such, the status quo is better (User.select(User.last_login_ip == encode_ip(ip))) because the way it is evaluated is more obvious, and the constraints are clearer. This is managed because "magic" stuff is very specific (those column objects, which have all the operator overloading), and everything else is plain Python. -- Ian Bicking | http://blog.ianbicking.org

Greg Ewing

6:57 a.m.

Bruce Leban wrote:

...

Isn't there are already a syntax for this?

expr = lambda: x + y * z

Maybe you want some conversion of that lambda into a different form:

expr = @ast lambda: x + y + z

If you need new syntax for this, then it's a sign that there *isn't* already a syntax for what we want. Given that we need new syntax anyway, there doesn't seem to be any point in bothering with the lambda: expr = @ast: x + y + z or any other suitable syntax. -- Greg

Guido van Rossum

27 Jul 27 Jul

11:34 p.m.

On Tue, Jul 27, 2010 at 5:25 PM, Robert Kern wrote:

...

I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree: expr = parse("x + y*z") assert isinstance(expr, ast.Expression) The advantage of this approach is that you can define a different language too... -- --Guido van Rossum (python.org/~guido)

Alex Gaynor

11:44 p.m.

On Tue, Jul 27, 2010 at 1:04 PM, Guido van Rossum wrote:

...

On Tue, Jul 27, 2010 at 5:25 PM, Robert Kern wrote:

...
I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree:

expr = parse("x + y*z") assert isinstance(expr, ast.Expression)

The advantage of this approach is that you can define a different language too...

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

As an interesting (but perhaps not relevant) data point, the documentation is rather nebulous as to whether the bool cast exists (it says things like "should return true if", but never explicitly that it takes the boolean value of the return from __contains__), further it doesn't seem to be tested at all (to the point where I only noticed today that PyPy's behavior is different, since this apparently breaks no tests). Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me

Raymond Hettinger

28 Jul 28 Jul

12:08 a.m.

On Jul 27, 2010, at 11:04 AM, Guido van Rossum wrote:

...

On Tue, Jul 27, 2010 at 5:25 PM, Robert Kern wrote:

...
I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree:

expr = parse("x + y*z") assert isinstance(expr, ast.Expression)

The advantage of this approach is that you can define a different language too...

Starting with string literals and a parse function seems like a great design decision. It already works and doesn't require new syntax. And, as Guido pointed out, it future proofs the design by freeing the domain specific language from the constraints of Python itself. Raymond

Antoine Pitrou

12:36 a.m.

On Tue, 27 Jul 2010 11:38:15 -0700 Raymond Hettinger wrote:

...

On Jul 27, 2010, at 11:04 AM, Guido van Rossum wrote:

...
On Tue, Jul 27, 2010 at 5:25 PM, Robert Kern wrote:

...
I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree:

expr = parse("x + y*z") assert isinstance(expr, ast.Expression)

The advantage of this approach is that you can define a different language too...

Starting with string literals and a parse function seems like a great design decision. It already works and doesn't require new syntax.

Yes, you only have to write a dedicated full-fledged parser, your code isn't highlighted properly in text editors, and you can't easily access Python objects declared in the enclosing scope. I guess it explains that none of the common ORMs seem to have adopted such a “great design decision” :-) Regards Antoine.

Oleg Broytman

12:52 a.m.

On Tue, Jul 27, 2010 at 09:06:42PM +0200, Antoine Pitrou wrote:

...

Yes, you only have to write a dedicated full-fledged parser, your code isn't highlighted properly in text editors, and you can't easily access Python objects declared in the enclosing scope.

I guess it explains that none of the common ORMs seem to have adopted such a ???great design decision??? :-)

Python ORMs are about mapping between *Python* and SQL - we don't need no stinking DSLs! ;) Oleg. -- Oleg Broytman http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

Masklinn

12:30 a.m.

On 2010-07-27, at 20:04 , Guido van Rossum wrote:

...

On Tue, Jul 27, 2010 at 5:25 PM, Robert Kern wrote:

...
I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree:

expr = parse("x + y*z") assert isinstance(expr, ast.Expression)

The advantage of this approach is that you can define a different language too…

The nice thing about having it be special-sauce syntax is that it can be parsed along with the rest of the script, failing early, and it can be stored in bytecode. Whereas the string itself will only be parsed when the function is actually executed.

Alexander Belopolsky

12:35 a.m.

On Tue, Jul 27, 2010 at 3:00 PM, Masklinn wrote: ..

...

The nice thing about having it be special-sauce syntax is that it can be parsed along with the rest of the script, failing early, and it can be stored in bytecode. Whereas the string itself will only be parsed when the function is actually executed.

Not enough to justify new syntax IMO. Just create your parse trees at the module or class level. chances are that's where they belong anyways. I would be very interested to see the parse() function. It does not exist (yet), right?

Greg Ewing

7:20 a.m.

Alexander Belopolsky wrote:

...

Not enough to justify new syntax IMO. Just create your parse trees at the module or class level. chances are that's where they belong anyways.

Except when they don't, and would be clearer written in-line at the point of use. -- Greg

Terry Reedy

29 Jul 29 Jul

10:48 p.m.

On 7/27/2010 3:05 PM, Alexander Belopolsky wrote:

...

anyways. I would be very interested to see the parse() function. It does not exist (yet), right?

...

...
...
def expr(s): return ast.parse(s, mode='eval')

...

...
...
e = expr('a+b') e <_ast.Expression object at 0x00F8DCF0>

-- Terry Jan Reedy

Greg Ewing

28 Jul 28 Jul

7:13 a.m.

Guido van Rossum wrote:

...

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree:

expr = parse("x + y*z") assert isinstance(expr, ast.Expression)

The advantage of this approach is that you can define a different language too...

This is more or less what we have now when we pass SQL queries as strings to database engines. There are numerous problems with this. One of them is the fact that editors have no idea that the code inside the string is code, so they can't help you out with any syntax highlighting, formatting, etc. Another is that it makes passing parameters to the embedded code very awkward. One of the things I would like to get from a code-as-ast feature is a natural way of embedding sub-expressions that *do* get evaluated according to the normal Python rules. For example, one should be able to write something like cust = "SMITH" date = today() sales = select(transactions, @ast: customer_code == cust and transaction_date == date) and have it possible for the implementation of select() to easily and safely evaluate 'cust' and 'date' in the calling environment. -- Greg

Chris Rebert

7:26 a.m.

On Tue, Jul 27, 2010 at 6:43 PM, Greg Ewing wrote: <snip>

...

One of the things I would like to get from a code-as-ast feature is a natural way of embedding sub-expressions that *do* get evaluated according to the normal Python rules. For example, one should be able to write something like

cust = "SMITH" date = today() sales = select(transactions, @ast: customer_code == cust and transaction_date == date)

and have it possible for the implementation of select() to easily and safely evaluate 'cust' and 'date' in the calling environment.

In other words, you want (possibly an implicit form of) the comma operator from Scheme's quasiquote.[1] Maybe Paul Graham /was/ onto something. Cheers, Chris -- [1] http://www.cs.hut.fi/Studies/T-93.210/schemetutorial/node7.html

Greg Ewing

7:40 a.m.

Chris Rebert wrote:

...

In other words, you want (possibly an implicit form of) the comma operator from Scheme's quasiquote.

I thought about that, but I'd rather avoid having to expicitly mark the sub-expressions if possible. The way I envisage it, each of the AST nodes would have an eval() method which would evaluate it in the calling environment. It would be up to the consumer of the AST to decide when to call it. -- Greg

Terry Reedy

29 Jul 29 Jul

10:48 p.m.

On 7/27/2010 2:04 PM, Guido van Rossum wrote:

...

On Tue, Jul 27, 2010 at 5:25 PM, Robert Kern wrote:

...
I've occasionally wished that we could repurpose backticks for expression literals:

expr = `x + y*z` assert isinstance(expr, ast.Expression)

Maybe you could just as well make it a plain string literal and call a function that parses it into a parse tree:

expr = parse("x + y*z") assert isinstance(expr, ast.Expression)

The advantage of this approach is that you can define a different language too...

and that is already exists, and is more visible than backticks

...

...
...
def expr(s): return ast.parse(s, mode='eval') # defaults is 'exec'

...

...
...
e = expr('a+b') e <_ast.Expression object at 0x00F8DCF0>

-- Terry Jan Reedy

Bruce Frederiksen

28 Jul 28 Jul

5:47 a.m.

For the LINQ approach, I'd rather see an open ended hook for allowing any required syntax, rather than only SQL-like syntax. OTOH, re.compile and db_cursor.execute are two examples where no new mechanism is needed. And you'd have to quote the new syntax somehow anyway since the Python parser wouldn't understand it... Which makes me wonder if it really makes sense to try to overload these operators in order to generate some kind of mini-language, vs just using your own syntax like re.compile or db_cursor.execute does. The one catch is that it is often nice to be able to refer to Python variables (at least; or perhaps full expressions) within the mini-language. The db_cursor.execute is an example, and putting the placeholders in the SQL syntax with arguments later works, but gets tedious. To be able to include Python expressions directly within the mini-language, the library implementing the new syntax would have to be able to translate the new syntax into Python and have it spliced into the code where it was used. Something like an intelligent macro expansion. This means that the library's translation code has to be called from the Python compiler. I'm not familiar enough with the compiler to know how crazy this is. Mython tries to do something similar. -Bruce On Tue, Jul 27, 2010 at 10:02 AM, Guido van Rossum wrote:

...

Therefore I think the LINQ approach, which (IIUC) converts an expression into a parse tree when certain syntax is encountered, and calls a built-in method with that parse tree, would be a fresh breath of air. No need deriding it just because Microsoft came up with it first.

Ian Bicking

8:12 a.m.

On Tue, Jul 27, 2010 at 7:17 PM, Bruce Frederiksen wrote:

...

The one catch is that it is often nice to be able to refer to Python variables (at least; or perhaps full expressions) within the mini-language. The db_cursor.execute is an example, and putting the placeholders in the SQL syntax with arguments later works, but gets tedious. To be able to include Python expressions directly within the mini-language, the library implementing the new syntax would have to be able to translate the new syntax into Python and have it spliced into the code where it was used. Something like an intelligent macro expansion. This means that the library's translation code has to be called from the Python compiler.

I'm not familiar enough with the compiler to know how crazy this is. Mython tries to do something similar.

Basically templating languages do this, and a templating language could be used for exactly this sort of purpose (but *not* simply a generic templating language, then you get SQL or whatever-else injection problems). I put together a small example that might work for SQL: http://svn.colorstudy.com/home/ianb/recipes/sqltemplate.py Unfortunately templating languages in Python aren't nearly as easy to implement as they should be, and the results are not as elegant as they could be. I've been using templating snippets a lot more in my code lately, and find it more handy than I would have originally expected. It doesn't seem unreasonable in this case either. Well... unless you want to introspect the expression, which would mean parsing the resulting SQL (and is then hard). -- Ian Bicking | http://blog.ianbicking.org

Cameron Simpson

3:25 a.m.

On 25Jul2010 11:48, Raymond Hettinger wrote: | On Jul 25, 2010, at 11:15 AM, Alex Gaynor wrote: | > Recently I've been wondering why __contains__ casts all of it's | > returns to be boolean values. Specifically I'd like to propose that | > __contains__'s return values be passed directly back as the result of | > the `in` operation. | | x = y in z # where x is a non boolean. | | Yuck. | | One of the beautiful aspects of __contains__ is that its simply signature | allows it to be used polymorphically throughout the whole language. Didn't we have the dual of this argument a week or so ago, where rantingrick was complaining that ints could be used as booleans, and that it was simply appalling? That Python should immediately make 0 also behave as True because he didn't feel it was "empty". His argument was widely opposed, and IMHO rightly so. Personally, I'm +0.5 on the proposal: - because Python already allows pretty much anything to be used in a Boolean context, this means that anything can be "used polymorphically throughout the whole language", to use your term above; I do not think it breaks anything - do any of the other comparison methods enforce Booleanness? ==/__eq__ doesn't and I didn't think the others did. All that is required for functionality is sane choice of return value by the implementors. - have you used SQLAlchemy? Its SQL constrction by writing: .select([...columns...], table.c.COL1 == 3) is extremely programmer friendly, and works directly off overloading the column object's .__eq__() method to return something that gets made into a robust SQL query later. I'm going to snip two of your paragraphs here and proceed to: | There is no "natural" interpretation of an in-operator returning | a non-boolean. There is in the SQLAlchemy example above; "in" with an SQLA column object would return a hook to make a "value in (a,b,c,...)" SQL expression. It is all about context, and in Python the .__* methods let objects provide the context for evaluation of expressions - that's what polymorphism does for us. The proposal changes nothing for pre-existing uses. It in no way causes: False in [False] to return False, because it doesn't change bool.__contains__. The proposal it to not coerce the result of __contains__ to bool(), allowing _new_ objects to return a more nuanced result for __contains__ for their own purposes. As long as that make sense in the use context, I believe this is a plus and not a minus. We can all write nonsensical code by implementing __eq__ with gibberish. So what? | If the above snippet assigns "foo" to x, what | does that mean? If it assigns -10, what does that mean? In current Python, it means "true". | Language design is about associating meanings (semantics) | with syntax. ISTM, this would be poor design. We already allow programmers to do that all over the place with the special methods. This proposal removes an apparently arbitrary restriction on __contains__ that doesn't seem to be applied to the other comparators. +0.5, verging on +1. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ It is in those arenas that he previously extinguished himself. - Chuck Rogers

5013

Age (days ago)

5017

Last active (days ago)

List overview

Download

50 comments

19 participants

participants (19)

Alex Gaynor
Alexander Belopolsky
Antoine Pitrou
Bruce Frederiksen
Bruce Leban
Cameron Simpson
Chris Rebert
Greg Ewing
Guido van Rossum
Ian Bicking
Masklinn
Mathias Panzenböck
Michael Foord
Michael Foord
MRAB
Oleg Broytman
Raymond Hettinger
Robert Kern
Terry Reedy

Non-boolean return from __contains__

Mathias Panzenböck

Cameron Simpson

Oleg Broytman

Oleg Broytman

Cameron Simpson

tags

participants (19)

Non-boolean return from contains