Simple question about how the optimizer works

[Andrew Dalke]

James J. Besemer:
>So I was suggesting an optimizer that when it saw a form like:
>
>    res = []
>    for x in list:
>        if x:
>            res.append( x )
>
>it would substitute a call to something
>
>    filter( lambda x : x is not None, list )

First, a minor point, this should be (if I understand the current
Boolean work)

    filter( bool, list )

("if x:" corresponds to (old-style) "if operator.truth(x) == 1:" or
(new-style) "if bool(x):" and is not the same as "x not None" because
you can override __nonzero__ or __len__, which may be used by the
truth test.)

There's also fiendish possibilities like

>>> class Fiendish:
...   def __nonzero__(self):
...     return len(sys._getframe().f_back.f_locals.get("res", [])) < 3
...
>>> list = [Fiendish(), Fiendish(), Fiendish(), Fiendish(), Fiendish(),
Fiendish()
, Fiendish(), Fiendish()]
>>> res = []
>>> for x in list:
...     if x:
...             res.append(x)
...
>>> res
[<__main__.Fiendish instance at 0x120288ef8>, <__main__.Fiendish instance at
0x120
28d428>, <__main__.Fiendish instance at 0x12028fbe8>]
>>> len(res)
3
>>> res = []
>>> import operator
>>> res = filter(operator.truth, list)
>>> res
[<__main__.Fiendish instance at 0x120288ef8>, <__main__.Fiendish instance at
0x120
28d428>, <__main__.Fiendish instance at 0x12028fbe8>, <__main__.Fiendish
instance
at 0x12028e098>, <__main__.Fiendish instance at 0x12028a7a8>,
<__main__.Fiendish i
nstance at 0x12028ded8>, <__main__.Fiendish instance at 0x1202904b8>,
<__main__.Fi
endish instance at 0x12028da28>]
>>> len(res)
8
>>>

Unless the compiler could look (somehow!) into the definition of each
__nonzero__, or disable optimizations if _getframe(), sys.exc_info(), etc.
are used *anywhere* in the Python code, then there's no way it can build
that optimization.

                    Andrew
                    dalke at dalkescientific.com