[ python-Bugs-1048870 ] call arg of lambda not updating

[SourceForge.net]

Bugs item #1048870, was opened at 2004-10-17 16:55
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1048870&group_id=5470

>Category: Parser/Compiler
Group: Python 2.3
Status: Open
>Resolution: None
>Priority: 6
Submitted By: Kevin Quick (kquick)
>Assigned to: Jeremy Hylton (jhylton)
Summary: call arg of lambda not updating

Initial Comment:
The attachment contains a script intended to do performance testing
on various types of dictionary keys.  Lambda functions are used to
"generate" a sequence of keys with bound arguments to arrays that are
modified during execution.

Script fails with pop() on empty list error, but examination shows that 
the lambda function from the *previous* call is being used for the 
current call; if proper lambda was used, list would not be empty and
operation should succeed.

Script is large and has been "grown", so it's not very elegant, but either 
I'm missing somewhere where I'm making a stupid mistake or else the
lambda argument to the function call isn't being set properly.  Don't be 
put off by the size of the script; the problem is easily demonstrated and 
fairly clearly understood in just a few minutes of looking at the 
generated stack trace relative to the code.

As it exists, script will fail immediately due to the bug.  If it worked as 
expected, at least 3-4 lines of output should be generated before 
encountering new code that hasn't yet been fully debugged.  New code 
could be removed, but it was left in because some of the complexity of 
core code is due to this latter code, and it doesn't obstruct the primary 
bug, so it can be ignored unless of interest.

python
Python 2.3.3 (#1, May 27 2004, 20:44:16) 
[GCC 3.3.2 20031218 (Gentoo Linux 3.3.2-r5, propolice-3.3-7)] on 
linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Thanks for looking at this!


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2004-10-18 00:41

Message:
Logged In: YES 
user_id=80475

Confirmed.  Also, the behavior persists into Py2.4b1.

The error disappears when the lambda is replaced by an
equivalent one line def statement:

  def nextkey2(N=keynums2,K=key_src): return K[N.pop()]

The opcodes are the same for both; however, the
co_firstlineno attribute is incorrect for the lambda version.

The pure python compiler module does not make the same error.



----------------------------------------------------------------------

Comment By: Kevin Quick (kquick)
Date: 2004-10-17 23:48

Message:
Logged In: YES 
user_id=6133

Reopened: Problem not addressed.  Please *read* my previous comment.

Yes, the script has a bug in it.  That's not what's being reported here.

The bug is in the backtrace produced by the Python interpreter.  See below.  
Line number reported for lambda function raising exception is line 14 below.  
Actual line number for lambda generating exception is line 18.


$ python lambda_bug.py
Traceback (most recent call last):
  File "lambda_bug.py", line 23, in ?
    seqtest()
  File "lambda_bug.py", line 19, in seqtest
    seq_func1(nextkey2, num_elem+2)
  File "lambda_bug.py", line 5, in seq_func1
    key_genfunc()
  File "lambda_bug.py", line 14, in <lambda>
    nextkey1 = lambda N=keynums1,K=key_src: K[N.pop()]
IndexError: pop from empty list


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-10-17 23:34

Message:
Logged In: YES 
user_id=80475

Closing as invalid -- this is a bug in the OP's script, not
in Python itself.

The error is obious when the code is simplified further by
in-lining the function.  Also, it helps to explicitly set
the initial values of variables between the two sections:

#!/usr/bin/env python
num_elem = 100

key_src = range(num_elem+1)
keynums1 = key_src[:]

assert key_src[-1] == 100
for _ in xrange(num_elem):
    keynums1[key_src.pop()]

keynums2 = [0]
for _ in xrange(102):
    key_src[keynums2.pop()]


I think your coding error was in assuming that keynum2 held
a different value before the second half was run.  If so,
that could have been found by using pdb or by inserting
print statements to reveal what is going on.


----------------------------------------------------------------------

Comment By: Kevin Quick (kquick)
Date: 2004-10-17 23:02

Message:
Logged In: YES 
user_id=6133

OK, example script minimized.

Lambda arguments are not the problem; thanks for the pointer, but I'm not 
having trouble with those.  

I have determined that proper lambda is being used, but not indicated as shown 
by attached script, which forces an exception to occur in the lambda function 
but backtrace shows wrong lambda function.  Reducing 2nd argument of 2nd 
call to seq_func1 to 'num_args' instead of 'num_args+2' will not encounter the 
error, but the nature of the implementation requires that it is using the correct 
lambda, so it's the backtrace attribution that is apparently wrong.

Mea culpa: my original script version did request wrong 'num_args' causing 
actual bug, but incorrect backtrace led me down an alternate path... I suspected 
that I had some incorrect code, but I couldn't resolve that against the error 
report.  Still can't.  :)


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-10-17 18:04

Message:
Logged In: YES 
user_id=80475

Please resubmit after boiling this down to the simplest
possible thing that doesn't work according to your
expectations. 

Submitting a whole application and playing the "find the
bug" game is not good use of developer time.  Often, when a
submitter starts taking away the unnecessary pieces and
isolates the issue, they find that there was a bug in their
own code or a bug in their own understanding of how the
language works.

When it comes to lambdas, the most frequent misunderstanding
is knowing that default arguments bind only once as the time
of the lambda declaration and that free variables bind
whenever the resulting function is invoked.  Here's an
example of code that looks similar but behaves differently:
 
def once(x): return x
def twice(x): return 2*x
def thrice(x): return 3*x
funcs = [once, twice, thrice]

flim = [lambda x:funcs[0](x), lambda x:funcs[1](x), lambda
x:funcs[2](x)]
flam = [lambda x:f(x) for f in funcs]

print flim[0](1), flim[1](1), flim[2](1)
print flam[0](1), flam[1](1), flam[2](1)

The difference in output is because nested scopes bind
names, while argument binding binds values.

Hope this helps.  If not, please resubmit with the smallest
case that demonstrates the problem.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1048870&group_id=5470