<div dir="ltr">Okay, I implemented it. Might be getting something wrong because I've never worked with the internals of this module before. See attached file for a demonstration, and here's the code for just the method:<div><br></div><div><div> def filter(self, fn, iterable, timeout=None):</div><div> </div><div> if timeout is not None:</div><div> end_time = timeout + time.time()</div><div><br></div><div> items_and_futures = [</div><div> (item, self.submit(fn, item)) for item in iterable</div><div> ]</div><div><br></div><div> # Yield must be hidden in closure so that the futures are submitted</div><div> # before the first iterator value is required.</div><div> def result_iterator():</div><div> try:</div><div> for item, future in items_and_futures:</div><div> if timeout is None:</div><div> result = future.result()</div><div> else:</div><div> result = future.result(end_time - time.time())</div><div> if result:</div><div> yield item</div><div> finally:</div><div> for _, future in items_and_futures:</div><div> future.cancel()</div><div> return result_iterator()</div></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, May 2, 2015 at 1:39 AM, Andrew Barnert <span dir="ltr"><<a href="mailto:abarnert@yahoo.com" target="_blank">abarnert@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><span class=""><div>On May 1, 2015, at 08:13, Ram Rachum <<a href="mailto:ram@rachum.com" target="_blank">ram@rachum.com</a>> wrote:</div><div><br></div><blockquote type="cite"><div><div dir="ltr">I envisioned it being implemented directly on `Executor`, so it'll automatically apply to all executor types. (I'll be happy to write the implementation if we have a general feeling that this is a desired feature.)</div></div></blockquote><div><br></div></span><div>I'd say just write it if you want it. If it turns out to be so trivial everyone decides it's unnecessary to add, you've only wasted 10 minutes. If it turns out to be tricky enough to take more time, that in itself will be a great argument that it should be added so users don't screw it up themselves. </div><div><br></div><div>Plus, of course, even if it gets rejected, you'll have the code you want for your own project. :)</div><div><div class="h5"><br><blockquote type="cite"><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 1, 2015 at 6:08 PM, Guido van Rossum <span dir="ltr"><<a href="mailto:guido@python.org" target="_blank">guido@python.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sounds like should be an easy patch. Of course, needs to work for ProcessPoolExecutor too.<br></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Fri, May 1, 2015 at 1:12 AM, Ram Rachum <span dir="ltr"><<a href="mailto:ram@rachum.com" target="_blank">ram@rachum.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hi,<div><br></div><div>What do you think about adding a method: `Executor.filter`?</div><div><br></div><div>I was using something like this: </div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">my_things = [thing for thing in things if some_condition(thing)]<br></blockquote><br></div><div>But the problem was that `some_condition` took a long time to run waiting on I/O, which is a great candidate for parallelizing with ThreadPoolExecutor. I made it work using `Executor.map` and some improvizing, but it would be nicer if I could do:</div><div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>with concurrent.futures.ThreadPoolExecutor(100) as executor:</div><div> my_things = executor.filter(some_condition, things)</div><div><br></div></blockquote>And have the condition run in parallel on all the threads.</div><div><br></div><div>What do you think? </div><div><br></div><div><br></div><div>Thanks,</div><div>Ram.</div></div>
<br></div></div>_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" target="_blank">http://python.org/psf/codeofconduct/</a><span><font color="#888888"><br></font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><br>-- <br><div>--Guido van Rossum (<a href="http://python.org/~guido" target="_blank">python.org/~guido</a>)</div>
</font></span></div>
</blockquote></div><br></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Python-ideas mailing list</span><br><span><a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a></span><br><span><a href="https://mail.python.org/mailman/listinfo/python-ideas" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a></span><br><span>Code of Conduct: <a href="http://python.org/psf/codeofconduct/" target="_blank">http://python.org/psf/codeofconduct/</a></span></div></blockquote></div></div></div></blockquote></div><br></div>