On Thu, 3 Mar 2022 at 01:28, Ricky Teachey <ricky@teachey.org> wrote:
I think I'm -0.5 but I have a question for the people on here smarter than me (pretty much all):
Is there some opportunity for some kind of compiler magic when the iterable of a for loop is fully contained in a place easily findable by the compiler, and not spread over multiple if and for statements?
Yes, there absolutely is, and you can see some of that by disassembling:
def f1(): ... for x in [1,2,3]: print(x) ... def f2(): ... for x in (1,2,3): print(x) ...
In theory, the first one means "build a list with three values, then iterate over it", and the second one means "iterate over the constant tuple (1,2,3)", but CPython implements them both the same way. Try "dis.dis(f1)" and "dis.dis(f2)" to see this similarity.
I am imagining that something like this could be magically "looked into" and made more efficient in some way, maybe by JIT complier or something:
JIT compilation is orthogonal to this sort of optimization; what you're thinking of here is a form of constant folding.
for x for y in range(11, 100, 3) if (y % 10) for x in range(y): frob(x)
compared to this:
for y in range(11, 100, 3): if (y % 10); for x in range(y): frob(x)
Am I instilling too much faith in the power of the complier on this Ash Wednesday morning? ;)
I think you are, and here's why: It's far too easy for something to have disrupted the optimization (for instance, shadowing the name "range"). Where a use-case is common, the compiler can be taught to optimize it; where it's uncommon, the effort isn't worth it. Constant folding is an incredibly useful technique, but it's not quite able to do this :) JIT compilation is also incredibly powerful. I don't know whether it'd be able to optimize what you're thinking of, but if it did, what you'd end up with is a much more generic optimization that applies to all loops, or maybe to all calls to range(). There are some other possibilities. It wouldn't surprise me if PyPy has a special check to optimize "for VAR in range(...):" that consists of (a) verifying that range hasn't been shadowed, and then (b) iterating over the numbers directly, without constructing a range object. On the other hand, it also wouldn't greatly surprise me if it doesn't, on the basis that this isn't beneficial enough to have special code, and the generic handling is able to cover it. ChrisA