<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">def filter(rule, whatever):<br>
if rule.x in whatever.x:<br>
return True<br>
<br>
rules = get_rules()<br>
whatevers = get_whatevers()<br>
for rule in rules:<br>
for whatever in whatevers:<br>
if filter(rule, whatever):<br>
cnt = cnt + 1<br>
<br>
return cnt<br></blockquote><div><br></div><div>This code seems almost certainly broken as written. Not just suboptimal, but just plain wrong. As a start, it has a return statement outside a function or method body, so it's not even syntactical (also, `cnt` is never initialized). But assuming we just print out `cnt` it still gives an answer we probably don't want.</div><div><br></div><div>For example, I wrote get_rules() and get_whatevers() functions that produce a list of "things" that have an x attribute. Each thing holds a (distinct) word from a Project Gutenberg book (<span style="color:rgb(0,0,0);white-space:pre-wrap">Title: Animal Locomotion: </span><span style="color:rgb(0,0,0);white-space:pre-wrap">Or walking, swimming, and flying, with a dissertation on aëronautics; </span><span style="color:rgb(0,0,0);white-space:pre-wrap">Author: J. Bell Pettigrew). Whatevers omit a few of the words, since the code suggests there should be more rules. In particular:</span></div><div><span style="white-space:pre-wrap;color:rgb(0,0,0)"><br></span></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace"><span style="white-space:pre-wrap;color:rgb(0,0,0)">In [1]: </span><font color="#000000"><span style="white-space:pre-wrap">len(get_rules()), len(get_whatevers())</span></font></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;color:rgb(0,0,0);border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline"><font face="monospace, monospace">Out [1]: (12306, 12301)</font></pre></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><span style="color:rgb(0,0,0);white-space:pre-wrap"><br></span></div><div><span style="color:rgb(0,0,0);white-space:pre-wrap">Running the probably wrong code is indeed slow:</span></div><div><span style="color:rgb(0,0,0);white-space:pre-wrap"><br></span></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">In [2]: </span></font></div><div><span style="white-space:pre-wrap;color:rgb(0,0,0);font-family:monospace,monospace">%%time</span></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">def filter(rule, whatever):</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"> if rule.x in whatever.x:</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"> return True</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"> </span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">rules = get_rules()</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">whatevers = get_whatevers()</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">cnt = 0</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">for rule in rules:</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"> for whatever in whatevers:</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"> if filter(rule, whatever):</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"> cnt = cnt + 1</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">
</span></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">print(cnt)</span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><br></span></font></div></div></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap">Out [2]: </span></font></div><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;color:rgb(0,0,0);border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline"><font face="monospace, monospace">110134
CPU times: user 53.1 s, sys: 190 ms, total: 53.3 s
Wall time: 53.6 s</font></pre></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>It's hard for me to imagine why this is the question one would want answered. It seems much more likely you'd want to know:</div><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">In [3]:</font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">%%time</font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">len({thing.x for thing in get_rules()} - {thing.x for thing in get_whatevers()}) </font></div><div><font face="monospace, monospace"><br></font></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">Out [3]:</font></div><div><span style="background-color:transparent;font-size:inherit;white-space:pre-wrap;color:rgb(0,0,0)"><font face="monospace, monospace">CPU times: user 104 ms, sys: 4.89 ms, total: 109 ms Wall time: 112 ms</font></span></div><div><span style="background-color:transparent;font-size:inherit;white-space:pre-wrap;color:rgb(0,0,0)"><font face="monospace, monospace">5</font></span></div><div><span style="background-color:transparent;font-size:inherit;white-space:pre-wrap;color:rgb(0,0,0)"><font face="monospace, monospace"><br></font></span></div></div></div></blockquote><font color="#000000" face="arial, helvetica, sans-serif"><span style="white-space:pre-wrap">So that's 500 times faster, more Pythonic, and seems to actually answer the question one would want answered.</span></font><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="white-space:pre-wrap"><br></span></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="white-space:pre-wrap">However, let's suppose there really is a reason to answer the question in the original code. Using more sensible basic datatypes, we only get about a 3x speedup:</span></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="white-space:pre-wrap"><br></span></font></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>In [4]:</div></span></font></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>%%time</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>rules = {thing.x for thing in get_rules()}</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>whatevers = {thing.x for thing in get_whatevers()}</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>cnt = 0</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>for rule in rules:</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div> for whatever in whatevers:</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div> if rule in whatever:</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div> cnt += 1</div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div> </div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>print(cnt)</div><div><br></div></span></font></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div>Out [4]:</div></span></font></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">110134</pre></div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">CPU times: user 18.3 s, sys: 96.9 ms, total: 18.4 s</pre></div></span></font></div><div><font color="#000000" face="monospace, monospace"><span style="white-space:pre-wrap"><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">Wall time: 18.5 s</pre><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline"><br></pre></div></span></font></div></blockquote><font color="#000000" face="arial, helvetica, sans-serif"><span style="font-size:14px;white-space:pre-wrap">I'm sure there is room for speedup if the actual problem being solved was better described. Maybe something involving </span></font><font face="monospace, monospace">itertools.product()</font><font face="arial, helvetica, sans-serif">, but it's hard to say without knowing what the correct behavior actually is.</font><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Overall this is similar to saying you could implement bogosort in Python, and it would be much slower than calling Timsort with `</font><font face="monospace, monospace">sorted(my_stuff)</font><font face="arial, helvetica, sans-serif">`</font></div></div>