Optimize out unused variables
I doubted whether or not to write, but since Guido has already touched a similar topic (see "Live variable analysis -> earlier release"), so will I write. It is common to assign some values which are never used to variables because the syntax demands this. For example: head, _, rest = path.partition('/') first, second, *_ = line.split() for _ in range(10): ... [k for k, _ in pairs] What if make such assignments no-op? It will only work with some limitations: 1. The variable is local. Not global, not nonlocal, not cell. 2. The variable name must start with '_'. Otherwise there would be larger risk of breaking a legal code which uses locals() with str.format() or like. 3. "del" is considered a use of the variable. So explicitly deleted variables will not be optimized out. 4. Star-assignment still consumes the iterator. It just might not to keep all values in memory. STORE_FAST will be replaced with POP_TOP in these cases. I wrote some code few weeks ago, and it is not too complex. My doubts are only that the benefit of the optimization with the above limitations is very restricted.
On Apr 8, 2020, at 10:59, Serhiy Storchaka <storchaka@gmail.com> wrote:
Could you go so far as to remove the variable from the locals if its only assignment(s) are optimized out? I’m not sure how much benefit that would provide. (Surely it would sometimes mean an f_locals array fits into one cache line instead of two, or that a whole code object stays around in L2 cache for the next time it’s called instead of being ejected, but often enough to make a difference? Maybe not…)
I wrote some code few weeks ago, and it is not too complex. My doubts are only that the benefit of the optimization with the above limitations is very restricted.
Like Guido’s idea, this seems like something that should definitely be safe enough as an opt-in decorator or whatever, and implementable that way. And that also seems like the best way to answer those doubts. Write or find some code that you think should benefit, add the decorator, benchmark, and see. Also, with an opt-in mechanism, you could relax the restrictions. For example, by default @killunused only kills unused assignments that meet your restrictions, but if I know it’s safe I can @killunused("_”, “dummy”) and it kills unused assignments to those names even if it wouldn’t normally do so. Then you could see if there are any cases where it’s useful, but only with the restrictions relaxed, and maybe use that as a guide to whether it’s worth finding a way to aim for looser restrictions in the first place or not.
I like the idea of formalizing "unused variables". How about having a syntax for it? Allowing a "." instead of an identifier to signify this behaviour [reusing Serhiy's examples]: head, ., rest = path.partition('/') first, second, *. = line.split() for . in range(10): ... [k for k, . in pairs] Potentially for unpacking one could use nothing, e.g. first, second, * = line.split() I don't think "." is necessarily a good symbol for it, but I think the grammar would cope. On Wed, 8 Apr 2020 at 22:56, Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:
09.04.20 00:54, Andrew Barnert via Python-ideas пише:
Could you go so far as to remove the variable from the locals if its only assignment(s) are optimized out? I’m not sure how much benefit that would provide. (Surely it would sometimes mean an f_locals array fits into one cache line instead of two, or that a whole code object stays around in L2 cache for the next time it’s called instead of being ejected, but often enough to make a difference? Maybe not…)
I did no do this specially, it was a natural consequence of optimizing out assignments. So yes, they are removed from f_locals.
Like Guido’s idea, this seems like something that should definitely be safe enough as an opt-in decorator or whatever, and implementable that way. And that also seems like the best way to answer those doubts. Write or find some code that you think should benefit, add the decorator, benchmark, and see.
Also, with an opt-in mechanism, you could relax the restrictions. For example, by default @killunused only kills unused assignments that meet your restrictions, but if I know it’s safe I can @killunused("_”, “dummy”) and it kills unused assignments to those names even if it wouldn’t normally do so. Then you could see if there are any cases where it’s useful, but only with the restrictions relaxed, and maybe use that as a guide to whether it’s worth finding a way to aim for looser restrictions in the first place or not.
It would be much more complex. It is just 30 lines of simple code added in symtable.c and compile.c, but with the decorator you would need to write complex code in Python which parses bytecode, analyzes dataflow, patches bytecode, removes names from locals and recalculates all local's indices. I am not interesting in doing this.
On Apr 8, 2020, at 10:59, Serhiy Storchaka <storchaka@gmail.com> wrote:
Could you go so far as to remove the variable from the locals if its only assignment(s) are optimized out? I’m not sure how much benefit that would provide. (Surely it would sometimes mean an f_locals array fits into one cache line instead of two, or that a whole code object stays around in L2 cache for the next time it’s called instead of being ejected, but often enough to make a difference? Maybe not…)
I wrote some code few weeks ago, and it is not too complex. My doubts are only that the benefit of the optimization with the above limitations is very restricted.
Like Guido’s idea, this seems like something that should definitely be safe enough as an opt-in decorator or whatever, and implementable that way. And that also seems like the best way to answer those doubts. Write or find some code that you think should benefit, add the decorator, benchmark, and see. Also, with an opt-in mechanism, you could relax the restrictions. For example, by default @killunused only kills unused assignments that meet your restrictions, but if I know it’s safe I can @killunused("_”, “dummy”) and it kills unused assignments to those names even if it wouldn’t normally do so. Then you could see if there are any cases where it’s useful, but only with the restrictions relaxed, and maybe use that as a guide to whether it’s worth finding a way to aim for looser restrictions in the first place or not.
I like the idea of formalizing "unused variables". How about having a syntax for it? Allowing a "." instead of an identifier to signify this behaviour [reusing Serhiy's examples]: head, ., rest = path.partition('/') first, second, *. = line.split() for . in range(10): ... [k for k, . in pairs] Potentially for unpacking one could use nothing, e.g. first, second, * = line.split() I don't think "." is necessarily a good symbol for it, but I think the grammar would cope. On Wed, 8 Apr 2020 at 22:56, Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:
09.04.20 00:54, Andrew Barnert via Python-ideas пише:
Could you go so far as to remove the variable from the locals if its only assignment(s) are optimized out? I’m not sure how much benefit that would provide. (Surely it would sometimes mean an f_locals array fits into one cache line instead of two, or that a whole code object stays around in L2 cache for the next time it’s called instead of being ejected, but often enough to make a difference? Maybe not…)
I did no do this specially, it was a natural consequence of optimizing out assignments. So yes, they are removed from f_locals.
Like Guido’s idea, this seems like something that should definitely be safe enough as an opt-in decorator or whatever, and implementable that way. And that also seems like the best way to answer those doubts. Write or find some code that you think should benefit, add the decorator, benchmark, and see.
Also, with an opt-in mechanism, you could relax the restrictions. For example, by default @killunused only kills unused assignments that meet your restrictions, but if I know it’s safe I can @killunused("_”, “dummy”) and it kills unused assignments to those names even if it wouldn’t normally do so. Then you could see if there are any cases where it’s useful, but only with the restrictions relaxed, and maybe use that as a guide to whether it’s worth finding a way to aim for looser restrictions in the first place or not.
It would be much more complex. It is just 30 lines of simple code added in symtable.c and compile.c, but with the decorator you would need to write complex code in Python which parses bytecode, analyzes dataflow, patches bytecode, removes names from locals and recalculates all local's indices. I am not interesting in doing this.
participants (3)
-
Andrew Barnert -
Henk-Jaap Wagenaar -
Serhiy Storchaka