<div dir="ltr"><div class="markdown-here-wrapper" style=""><p style="margin:0px 0px 1.2em!important">On Fri, Aug 3, 2018 at 1:47 PM Todd <a href="http://mailto:toddrjen@gmail.com">toddrjen@gmail.com</a> wrote:</p>
<p style="margin:0px 0px 1.2em!important"></p><div class="markdown-here-exclude"><p></p><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>The operators would be:</div><div><br></div><div>bNOT - boolean "not"<br></div><div>bAND - boolean "and"</div><div>bOR - boolean "or"</div><div>bXOR - boolean "xor"</div></div></blockquote><p></p></div><p style="margin:0px 0px 1.2em!important"></p>
<p style="margin:0px 0px 1.2em!important">These look pretty ugly to me. But that could just be a matter of familiarity.</p>
<p style="margin:0px 0px 1.2em!important">For what it’s worth, the Apache Spark project offers a popular DataFrame API for querying tabular data, similar to Pandas. The project overloaded the bitwise operators <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">&</code>, <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">|</code>, and <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">~</code> since they could not override the boolean operators <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">and</code>, <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">or</code>, and <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">not</code>.</p>
<p style="margin:0px 0px 1.2em!important">For example:</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code class="hljs language-python" style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline;white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important;display:block;overflow-x:auto;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,248)">non_python_rhode_islanders = (
    person
    .where(~person[<span class="hljs-string" style="color:rgb(221,17,68)">'is_python_programmer'</span>])
    .where(person[<span class="hljs-string" style="color:rgb(221,17,68)">'state'</span>] == <span class="hljs-string" style="color:rgb(221,17,68)">'RI'</span> & person[<span class="hljs-string" style="color:rgb(221,17,68)">'age'</span>] > <span class="hljs-number" style="color:rgb(0,128,128)">18</span>)
    .select(<span class="hljs-string" style="color:rgb(221,17,68)">'first_name'</span>, <span class="hljs-string" style="color:rgb(221,17,68)">'last_name'</span>)
)
non_python_rhode_islanders.show(<span class="hljs-number" style="color:rgb(0,128,128)">20</span>)
</code></pre>
<p style="margin:0px 0px 1.2em!important">This did lead to <a href="https://issues.apache.org/jira/browse/SPARK-8568">confusion among users</a> since people (myself included) would initially try the boolean operators and wonder why they weren’t working. So the Spark devs <a href="https://github.com/apache/spark/pull/6961/files">added a warning</a> to catch when users were making this mistake. But now it seems quite OK to me to use <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">&</code>, <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">|</code>, and <code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px;display:inline">~</code> in the context of Spark DataFrames, even though their use doesn’t match their designed meaning. It’s unfortunate, but I think the Spark devs made a practical choice that works well enough for their users.</p>
<p style="margin:0px 0px 1.2em!important">PEP 335 would have addressed this issue by letting developers overload the common boolean operators directly, but from what I gather of <a href="https://mail.python.org/pipermail/python-dev/2012-March/117510.html">Guido’s rejection</a>, the biggest problem was that it would have had an undue performance impact on non-users of boolean operator overloading. (Not sure if I interpreted his email correctly.)</p>
<div title="MDH:PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiPk9uIEZyaSwgQXVnIDMsIDIw
MTggYXQgMTo0NyBQTSBUb2RkICZsdDt0b2RkcmplbkBnbWFpbC5jb20mZ3Q7IHdyb3RlOjxicj48
L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46IDBweCAw
cHggMHB4IDAuOGV4OyBib3JkZXItbGVmdDogMXB4IHNvbGlkIHJnYigyMDQsIDIwNCwgMjA0KTsg
cGFkZGluZy1sZWZ0OiAxZXg7Ij48ZGl2IGRpcj0ibHRyIj48ZGl2PlRoZSBvcGVyYXRvcnMgd291
bGQgYmU6PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5iTk9UIC0gYm9vbGVhbiAibm90Ijxicj48
L2Rpdj48ZGl2PmJBTkQgLSBib29sZWFuICJhbmQiPC9kaXY+PGRpdj5iT1IgLSBib29sZWFuICJv
ciI8L2Rpdj48ZGl2PmJYT1IgLSBib29sZWFuICJ4b3IiPC9kaXY+PC9kaXY+PC9ibG9ja3F1b3Rl
PjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhlc2UgbG9vayBwcmV0dHkgdWdseSB0byBtZS4gQnV0IHRo
YXQgY291bGQganVzdCBiZSBhIG1hdHRlciBvZiBmYW1pbGlhcml0eS48L2Rpdj48ZGl2Pjxicj48
L2Rpdj48ZGl2PkZvciB3aGF0IGl0J3Mgd29ydGgsIHRoZSBBcGFjaGUgU3BhcmsgcHJvamVjdCBv
ZmZlcnMgYSBwb3B1bGFyIERhdGFGcmFtZSBBUEkgZm9yIHF1ZXJ5aW5nIHRhYnVsYXIgZGF0YSwg
c2ltaWxhciB0byBQYW5kYXMuIFRoZSBwcm9qZWN0IG92ZXJsb2FkZWQgdGhlIGJpdHdpc2Ugb3Bl
cmF0b3JzIGAmYW1wO2AsIGB8YCwgYW5kIGB+YCBzaW5jZSB0aGV5IGNvdWxkIG5vdCBvdmVycmlk
ZSB0aGUgYm9vbGVhbiBvcGVyYXRvcnMgYGFuZGAsIGBvcmAsIGFuZCBgbm90YC48L2Rpdj48ZGl2
Pjxicj48L2Rpdj48ZGl2PkZvciBleGFtcGxlOjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+YGBg
cHl0aG9uPC9kaXY+PGRpdj5ub25fcHl0aG9uX3Job2RlX2lzbGFuZGVycyA9ICg8L2Rpdj48ZGl2
PiZuYnNwOyAmbmJzcDsgcGVyc29uPC9kaXY+PGRpdj4mbmJzcDsgJm5ic3A7IC53aGVyZSh+cGVy
c29uWydpc19weXRob25fcHJvZ3JhbW1lciddKTwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAud2hl
cmUocGVyc29uWydzdGF0ZSddID09ICdSSScgJmFtcDsgcGVyc29uWydhZ2UnXSAmZ3Q7IDE4KTwv
ZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAuc2VsZWN0KCdmaXJzdF9uYW1lJywgJ2xhc3RfbmFtZScp
PC9kaXY+PGRpdj4pPC9kaXY+PGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOiBzbWFsbDsgYmFj
a2dyb3VuZC1jb2xvcjogcmdiKDI1NSwgMjU1LCAyNTUpOyB0ZXh0LWRlY29yYXRpb24tc3R5bGU6
IGluaXRpYWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsgZmxvYXQ6IG5vbmU7IGRp
c3BsYXk6IGlubGluZTsiPjxzcGFuIHN0eWxlPSJ0ZXh0LWRlY29yYXRpb24tc3R5bGU6IGluaXRp
YWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsgZmxvYXQ6IG5vbmU7IGRpc3BsYXk6
IGlubGluZTsiPm5vbl9weXRob25fcmhvZGVfaXNsYW5kZXJzPC9zcGFuPi5zaG93KDIwKTwvc3Bh
bj48YnI+PC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgZGlkIGxl
YWQgdG8gW2NvbmZ1c2lvbiBhbW9uZyB1c2Vyc10oaHR0cHM6Ly9pc3N1ZXMuYXBhY2hlLm9yZy9q
aXJhL2Jyb3dzZS9TUEFSSy04NTY4KSBzaW5jZSBwZW9wbGUgKG15c2VsZiBpbmNsdWRlZCkgd291
bGQgaW5pdGlhbGx5IHRyeSB0aGUgYm9vbGVhbiBvcGVyYXRvcnMgYW5kIHdvbmRlciB3aHkgdGhl
eSB3ZXJlbid0IHdvcmtpbmcuIFNvIHRoZSBTcGFyayBkZXZzIFthZGRlZCBhIHdhcm5pbmddKGh0
dHBzOi8vZ2l0aHViLmNvbS9hcGFjaGUvc3BhcmsvcHVsbC82OTYxL2ZpbGVzKSB0byBjYXRjaCB3
aGVuIHVzZXJzIHdlcmUgbWFraW5nIHRoaXMgbWlzdGFrZS4gQnV0IG5vdyBpdCBzZWVtcyBxdWl0
ZSBPSyB0byBtZSB0byB1c2UgYCZhbXA7YCwgYHxgLCBhbmQgYH5gIGluIHRoZSBjb250ZXh0IG9m
IFNwYXJrIERhdGFGcmFtZXMsIGV2ZW4gdGhvdWdoIHRoZWlyIHVzZSBkb2Vzbid0IG1hdGNoIHRo
ZWlyIGRlc2lnbmVkIG1lYW5pbmcuIEl0J3MgdW5mb3J0dW5hdGUsIGJ1dCBJIHRoaW5rIHRoZSBT
cGFyayBkZXZzIG1hZGUgYSBwcmFjdGljYWwgY2hvaWNlIHRoYXQgd29ya3Mgd2VsbCBlbm91Z2gg
Zm9yIHRoZWlyIHVzZXJzLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+UEVQIDMzNSB3b3VsZCBo
YXZlIGFkZHJlc3NlZCB0aGlzIGlzc3VlIGJ5IGxldHRpbmcgZGV2ZWxvcGVycyBvdmVybG9hZCB0
aGUgY29tbW9uIGJvb2xlYW4gb3BlcmF0b3JzIGRpcmVjdGx5LCBidXQgZnJvbSB3aGF0IEkgZ2F0
aGVyIG9mIFtHdWlkbydzIHJlamVjdGlvbl0oaHR0cHM6Ly9tYWlsLnB5dGhvbi5vcmcvcGlwZXJt
YWlsL3B5dGhvbi1kZXYvMjAxMi1NYXJjaC8xMTc1MTAuaHRtbCksIHRoZSBiaWdnZXN0IHByb2Js
ZW0gd2FzIHRoYXQgaXQgd291bGQgaGF2ZSBoYWQgYW4gdW5kdWUgcGVyZm9ybWFuY2UgaW1wYWN0
IG9uIG5vbi11c2VycyBvZiBib29sZWFuIG9wZXJhdG9yIG92ZXJsb2FkaW5nLiAoTm90IHN1cmUg
aWYgSSBpbnRlcnByZXRlZCBoaXMgZW1haWwgY29ycmVjdGx5Lik8YnI+PC9kaXY+PC9kaXY+" style="height:0;width:0;max-height:0;max-width:0;overflow:hidden;font-size:0em;padding:0;margin:0">​</div></div></div>