what is lambda used for in real code?

Steven Bethard steven.bethard at gmail.com
Fri Dec 31 07:53:55 CET 2004


I thought it might be useful to put the recent lambda threads into 
perspective a bit.  I was wondering what lambda gets used for in "real" 
code, so I grepped my Python Lib directory.  Here are some of the ones I 
looked, classified by how I would rewrite them (if I could):


* Rewritable as def statements (<name> = lambda <args>: <expr> usage)
These are lambdas used when a lambda wasn't needed -- an anonymous 
function was created with lambda and then immediately bound to a name. 
Since this is essentially what def does, using lambdas here is (IMHO) silly.

pickletools.py: getpos = lambda: None
     def getpos(): return None
tarfile.py:     normpath = lambda path:
                     os.path.normpath(path).replace(os.sep, "/")
     def normpath(path): os.path.normpath(path).replace(os.sep, "/")
urllib2.py:     H = lambda x: md5.new(x).hexdigest()
     def H(x): md5.new(x).hexdigest()
urllib2.py:     H = lambda x: sha.new(x).hexdigest()
     def H(x): sha.new(x).hexdigest()


* Rewritable with existing functions
Mainly these are examples of code that can benefit from using the 
functions available in the operator module, especially 
operator.itemgetter and operator.attrgetter (available in 2.4)

cgi.py:                 return map(lambda v: v.value, value)
     return map(operator.attrgetter('value'), value)
CGIHTTPServer.py:  nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()))
     nobody = 1 + max(map(operator.itemgetter(2), pwd.getpwall()))
SimpleXMLRPCServer.py:  server.register_function(lambda x,y: x+y, 'add')
     server.register_function(operator.add, 'add')
SimpleXMLRPCServer.py:  server.register_function(lambda x,y: x+y, 'add')
     server.register_function(operator.add, 'add')
sre_constants.py:        items.sort(key=lambda a: a[1])
     items.sort(key=operator.itemgetter(1))
tarfile.py:              return map(lambda m: m.name, self.infolist())
     return map(operator.attrgetter('name'), self.infolist())


* Rewritable with list comprehensions/generator expressions
Lambdas in map or filter expressions can often be replaced by an 
appropriate list comprehension or generator expression (in Python 2.3/2.4)

cgi.py:            plist = map(lambda x: x.strip(), line.split(';'))
     plist = [x.strip() for x in line.split(';')
cgi.py:            return map(lambda v: v.value, value)
     return [v.value for v in value]
CGIHTTPServer.py:  nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()))
     nobody = 1 + max(x[2] for x in pwd.getpwall())
glob.py:           names=filter(lambda x: x[0]!='.',names)
     names=[x for x in names if x[0] != '.']
hmac.py:           return "".join(map(lambda x, y: chr(ord(x) ^ ord(y)),
                                       s1, s2))
     return "".join(chr(ord(x) ^ ord(y)) for x, y in zip(s1, s2))
imaplib.py:        l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and
                            '" "'.join(x[1]) or ''), l)
     l = ['%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or '')
          for x in l]
inspect.py:        suffixes = map(lambda (suffix, mode, mtype):
                                     (-len(suffix), suffix, mode, mtype),
                                   imp.get_suffixes())
     suffixes = [(-len(suffix), suffix, mode, mtype)
                 for suffix, mode, mtype in imp.get_suffixes()
inspect.py:        return join(map(lambda o, c=convert, j=join:
                                    strseq(o, c, j), object))
     return join([strseq(o, convert, join) for o in object])
mailcap.py:        entries = filter(lambda e,key=key: key in e, entries)
     entries = [e for e in entries if key in e]
poplib.py:         digest = ''.join(map(lambda x:'%02x'%ord(x), digest))
     digest = ''.join('%02x' % ord(x) for x in digest)
pstats.py:         if line and not filter(lambda x,a=abbrevs:
                                               x not in a,line.split()):
     if line and not [x for x in line.split() if x not in abbrevs]:
tabnanny.py:       firsts = map(lambda tup: str(tup[0]), w)
     firsts = [str(tup[0]) for tup in w]
tarfile.py:        return map(lambda m: m.name, self.infolist())
     return [m.name for m in self.infolist()]
tarfile.py:        return filter(lambda m: m.type in REGULAR_TYPES,
                                  self.tarfile.getmembers())
     return [m for m in self.tarfile.getmembers()
             if m.type in REGULAR_TYPES]
urllib2.py:        return map(lambda x: x.strip(), list)
     return [x.strip() for x in list]
webbrowser.py:     _tryorder = filter(lambda x: x.lower() in _browsers
                                       or x.find("%s") > -1, _tryorder
     _tryorder = [x for x in _tryorder
                  if x.lower() in _browsers or x.find("%s") > -1]


* Functions I don't know how to rewrite
Some functions I looked at, I couldn't figure out a way to rewrite them 
without introducing a new name or adding new statements.  (Warning: I 
have trouble following code that uses 'reduce', so I only glossed over 
lambdas in reduce calls.)

calendar.py:    _months.insert(0, lambda x: "")
cgitb.py: inspect.formatargvalues(args, varargs, varkw, locals,
                  formatvalue=lambda value: '=' + pydoc.html.repr(value))
cgitb.py: inspect.formatargvalues(args, varargs, varkw, locals,
                  formatvalue=lambda value: '=' + pydoc.text.repr(value))
csv.py:       quotechar = reduce(lambda a, b, quotes = quotes:
                       (quotes[a] > quotes[b]) and a or b, quotes.keys())
csv.py:            delim = reduce(lambda a, b, delims = delims:
                       (delims[a] > delims[b]) and a or b, delims.keys())
difflib.py:       matches = reduce(lambda sum, triple: sum + triple[-1],
                                    self.get_matching_blocks(), 0)
gettext.py:    return eval('lambda n: int(%s)' % plural)
gettext.py:        self.plural = lambda n: int(n != 1)
inspect.py:    classes.sort(key=lambda c: (c.__module__, c.__name__))
inspect.py: def formatargspec(args, varargs=None, varkw=None,
                   ...
                   formatvarargs=lambda name: '*' + name,
                   formatvarkw=lambda name: '**' + name,
                   formatvalue=lambda value: '=' + repr(value),
inspect.py: def formatargvalues(args, varargs, varkw, locals,
                     ...
                     formatvarargs=lambda name: '*' + name,
                     formatvarkw=lambda name: '**' + name,
                     formatvalue=lambda value: '=' + repr(value),
pyclbr.py:    objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
                                          getattr(b, 'lineno', 0)))
SimpleHTTPServer.py:        list.sort(key=lambda a: a.lower())
subprocess.py:      p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
symtable.py:       self.__params = self.__idents_matching(lambda x:
                                                           x & DEF_PARAM)
symtable.py:       self.__locals = self.__idents_matching(lambda x:
                                                           x & DEF_BOUND)
symtable.py:       self.__globals = self.__idents_matching(lambda x:
                                                            x & glob)
urllib2.py:setattr(self, '%s_open' % type,
                    lambda r, proxy=url, type=type, meth=self.proxy_open:
                    meth(r, proxy, type))
xdrlib.py: unpacktest = [
         (up.unpack_uint,   (), lambda x: x == 9),
         (up.unpack_bool,   (), lambda x: not x),
         (up.unpack_bool,   (), lambda x: x),
         (up.unpack_uhyper, (), lambda x: x == 45L),
         (up.unpack_float,  (), lambda x: 1.89 < x < 1.91),
         (up.unpack_double, (), lambda x: 1.89 < x < 1.91),
         (up.unpack_string, (), lambda x: x == 'hello world'),
         (up.unpack_list,   (up.unpack_uint,), lambda x: x == range(5)),
         (up.unpack_array,  (up.unpack_string,),
          lambda x: x == ['what', 'is', 'hapnin', 'doctor']),
         ]



Of the functions that I don't know how to rewrite, I think there are a 
few interesting cases:

(1) lambda x: ""
This is the kind of parameter adaptation that I think Jeff Shannon was 
talking about in another lambda thread.  Using the ignoreargs function I 
suggested there[1], you could rewrite this as:
     ignoreargs(str, 1)


(2) lambda a: a.lower()
My first thought here was to use str.lower instead of the lambda, but of 
course that doesn't work if 'a' is a unicode object:

py> str.lower(u'a')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
TypeError: descriptor 'lower' requires a 'str' object but received a 
'unicode'

It's too bad I can't do something like:
     basestring.lower


(3)  self.plural = lambda n: int(n != 1)
Note that this is *almost* writable with def syntax.  If only we could do:
     def self.plural(n):
         int(n != 1)


(4) objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
                                getattr(b, 'lineno', 0)))
My first intuition here was to try something like:
     objs.sort(key=operator.attrgetter('lineno'))
but this doesn't work because then we don't get the default value of 0 
if the attribute doesn't exist.  I wonder if operator.attrgetter should 
take an optional "default" parameter like getattr does:
     Help on built-in function getattr in module __builtin__:

     getattr(...)
         getattr(object, name[, default]) -> value


(5) lambda x: x & DEF_PARAM
This could probably be written as:
    functional.partial(operator.and_, DEF_PARAM)
if PEP 309[2] was accepted, thought I'm not claiming that's any clearer...



So, those are my thoughts on how lambdas are "really" used.  If others 
out there have real-life code that uses lambdas in interesting ways, 
feel free to share them here!

Steve

[1]http://mail.python.org/pipermail/python-list/2004-December/257982.html
[2]http://python.fyxm.net/peps/pep-0309.html



More information about the Python-list mailing list