[Tutor] Lamdas and locality
Michael Meier
meiermic at ee.ethz.ch
Sat Mar 3 20:01:52 CET 2007
Hello
I ran the following code in python:
>>> ls = [(lambda op: op + i) for i in [1,2,3]]
>>> ls
[<function <lambda> at 0xb7de4cdc>, <function <lambda> at 0xb7de4d14>,
<function <lambda> at 0xb7de4d4c>]
>>> for l in ls:
... print l(5)
...
8
8
8
>>>
I am quite surprised of the result. I'm generating three lamdas. What I
want to do is that the first lamda adds 1 to the operand and returns it,
the second lamda return 2 plus the operand and so on.
However all the three lamdas, despite being not the same object in
memory, are all adding 3, so they've got to have a reference to and not
to 1,2,3 respectively.
Why are the lamdas having a reference to the same integer? What am I
getting wrong here? :P
On Sat, 2007-03-03 at 17:24 +0100, tutor-request at python.org wrote:
> Send Tutor mailing list submissions to
> tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
> tutor-request at python.org
>
> You can reach the person managing the list at
> tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
> 1. Re: geeks like us and the rest of THEM (Andrei)
> 2. Re: dictionaries and memory handling (Andrei)
> 3. Re: Yet another list comprehension question (Kent Johnson)
> 4. trouble understanding the python environment in OSX
> (Clay Wiedemann)
> 5. Re: trouble understanding the python environment in OSX
> (Alan Gauld)
> 6. Re: trouble understanding the python environment in OSX
> (Kent Johnson)
> 7. Re: Yet another list comprehension question (Andrei)
> 8. Re: Yet another list comprehension question (Kent Johnson)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 03 Mar 2007 13:40:05 +0100
> From: Andrei <project5 at redrival.net>
> Subject: Re: [Tutor] geeks like us and the rest of THEM
> To: tutor at python.org
> Message-ID: <esbqal$qpu$1 at sea.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> <snip>
> > That's part of it. a good installer is another, and I am loo]king at a
> > couple. A solid and drool-proof server is the remaining leg of the
> > tripod. So far, none of the servers I have seen are satisfactory on all
> > scores- simplicity to install, reliability, economy of cost, (free or
> > unlimited license to me for a modest fee) and ease to configure if the
> > use\r so wishes. the closest I have yet seen SMALL HTTP SERVER. And it
> > is very good.
> > http://smallsrv.com/
>
> You could have a look at Aprelium's Abyss server (X1, which is freely
> distributable in unmodified form IIRC, but closed source). It has a good
> looking webbased configuration console and is cross-platform if you ever
> need that. However, you'll probably end up digging in its config
> programmatically to set up paths and such for the user - shouldn't be
> too hard using some webscraping if the files are not readily modifiable.
> Its setup is about 350 kB.
>
> In terms of installers, NSIS (with the package to make it look modern,
> instead of that awful default look - forgot its name) or InnoSetup are
> the usual suspects. NSIS has a smaller overhead, but an uglier language.
>
> --
> Yours,
>
> Andrei
>
> =====
> Mail address in header catches spam. Real contact info:
> ''.join([''.join(s) for s in zip(
> "poet at aao.l pmfe!Pes ontuei ulcpss edtels,s hr' one oC.",
> "rjc5wndon.Sa-re laed o s npbi ot.Ira h it oteesn edt C")])
>
>
>
> ------------------------------
>
> Message: 2
> Date: Sat, 03 Mar 2007 14:18:24 +0100
> From: Andrei <project5 at redrival.net>
> Subject: Re: [Tutor] dictionaries and memory handling
> To: tutor at python.org
> Message-ID: <esbsif$6ec$1 at sea.gmane.org>
> Content-Type: text/plain; charset=windows-1252; format=flowed
>
> > But most of my dictionaries are nested, and since both keys and values
> > in the dbm 'dictionaries' have to be strings, I can't immediately see
> > how I could get it to work.
> >
> > A bit more detail: I deal with conditional probabilities, with up to 4
> > parameters. These parameters are numbers or words and determine the
> > value (which is always a number). E.g. I have a dictionary
> > {p1:{p2:{p3:{p4:value}}}}, where the p's are different parameters. I
> > sometimes need to sum over one or more of the parameters ? for now I
> > have managed to structure the dictionaries so that I only need to sum
> > over the innermost parameter, although this has been a bit cumbersome.
>
> Depends a bit on how many keys each of the dictionaries is going to have
> and in what order they're filled. You can pickle/cPickle an arbitrary
> amount of data as one value, so the whole {p2:{p3:{p4:value}}} story
> could be a value of the p1 key in the bsddb. However, you may not like
> this if you need to retrieve p1 and add new stuff to it all the time,
> because the pickling and unpickling cycles may not be beneficial to the
> performance.
>
> If you find this is a problem, you could opt to e.g. keep the first
> layer of the dictionary in memory, but map each value to a separate
> bsddb, so that you'd need to do less pickling/unpickling.
>
> Alternatively you could choose to store them as a sort of path in the
> bsddb, like this (obviously wasteful in terms of space):
> 'p1/p2/p3/p4': '20'
> 'p1/p2/p3/p5': '45'
>
> Combinations of the approaches above are also possible and I have no
> idea which would be best.
>
> >> - Will the dictionaries take up less memory if I use numbers rather
> >> than words as keys (i.e. will {3:45, 6:77, 9:33} consume less memory
> >> than {"eloquent":45, "helpless":77, "samaritan":33} )? And if so:
> >> Slightly less, or substantially less memory?
>
> Here's a simple test:
>
> >>> x = [str(i) for i in range(100000, 1000000)]
> vs.
> >>> x = [i for i in range(100000, 1000000)]
>
> The interpreter takes about 50 MB for the str() version and about 20 for
> the non-str() version - eyeballed in the Task manager. So it does make a
> significant difference, though not an order-of-magnitude difference.
> That may be enough for now, but if this script stays in use, you're
> bound at some point to need even more.
>
> >> - What are common methods to monitor the memory usage of a script?
> >> Can I add a snippet to the code that prints out how many MBs of
> >> memory a certain dictionary takes up at that particular time?
>
> Not as such. In your case, I think the task manager would be enough. You
> only have this one demanding data structure I assume, so in a rough
> approximation you can pretend that whatever the task manager reports
> (have a look at VM and peak memory usage columns, not just memory usage)
> is caused by the contents of the dictionary.
>
> --
> Yours,
>
> Andrei
>
> =====
> Mail address in header catches spam. Real contact info:
> ''.join([''.join(s) for s in zip(
> "poet at aao.l pmfe!Pes ontuei ulcpss edtels,s hr' one oC.",
> "rjc5wndon.Sa-re laed o s npbi ot.Ira h it oteesn edt C")])
>
>
>
> ------------------------------
>
> Message: 3
> Date: Sat, 03 Mar 2007 08:47:55 -0500
> From: Kent Johnson <kent37 at tds.net>
> Subject: Re: [Tutor] Yet another list comprehension question
> To: Alan Gauld <alan.gauld at btinternet.com>
> Cc: tutor at python.org
> Message-ID: <45E97C8B.1060205 at tds.net>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Alan Gauld wrote:
> > "Smith, Jeff" <jsmith at medplus.com> wrote
> >
> >> In other words, applying somefun to the results of the iterator
> >> return
> >> duplicates but I want the constructed list to contain none.
> >
> >> l = [somefun(i) for i some-iterator if somefun(i) not in l]
> >>
> >> doesn't work (not that I expected it to).
> >
> > Why not use a Set?
> >
> > s = Set([somefun(i) for i in some-iterator])
> >
> > Might be slow for big lists though...
>
> This is a popular question. It comes up frequently on comp.lang.python
> and there are many recipes in the online cookbook. Here is a FAQ:
> http://www.effbot.org/pyfaq/how-do-you-remove-duplicates-from-a-list.htm
>
> I think using a set is the fastest solution if the list items are
> hashable and you don't care about order.
>
> If the list items are hashable and you do care about order then there is
> this mild hack:
> s = set()
> [ i for i in lst if i not in s and not s.add(i) ]
>
> but with the requirement of calling somefunc(i) my guess is that an
> explicit loop will be faster.
>
> The cookbook is down right now but there is a link in the above FAQ.
>
> Kent
>
>
> ------------------------------
>
> Message: 4
> Date: Sat, 3 Mar 2007 08:50:09 -0500
> From: "Clay Wiedemann" <clay.wiedemann at gmail.com>
> Subject: [Tutor] trouble understanding the python environment in OSX
> To: tutor at python.org
> Message-ID:
> <781164cc0703030550i509e79e5waa2f2c3a4950faad at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hi,
>
> I am not only new to Python, but new to programming in an environment
> like this, where knowing a little of how the underpinning work is
> essential.*
>
> I am running the latest OSX (10.4.8) and know a little about moving
> about in the terminal. But the world of paths, etc. has me a little
> confused. And now it is getting in my way. I was wondering if I could
> get a little help with some things.
>
> - Can anyone point me to a well-regarded and pithy tutorial regarding
> the UNIX underpinnings of OSX, especially anything that would help
> with Python. (e.g., I can navigate to Python.framework and versions
> but see 2.5 and Current, which confuses me since 2.5 is the current
> version.) I would love to know what the structure of the Python
> installation means.
>
> - I am having trouble installing the BeautifulSoup package. The
> installer stumbles on a unicode issue (that the Soup developer says is
> a problem with my Python installation) -- so what is the best way to
> manually install this? And does the 2.5 / Current divide matter?
>
>
>
> Thanks in advance for any help with these basic issues.
>
> Good morning from Brooklyn,
> Clay
>
>
> * Any experience with coding webpages is not helping here
>
>
>
> - - - - - - -
>
> Clay S. Wiedemann
>
> aim: khlav
> wii: 3905 4571 6175 2469
> twitter: seastokes
>
>
> ------------------------------
>
> Message: 5
> Date: Sat, 3 Mar 2007 14:22:11 -0000
> From: "Alan Gauld" <alan.gauld at btinternet.com>
> Subject: Re: [Tutor] trouble understanding the python environment in
> OSX
> To: tutor at python.org
> Message-ID: <esc06r$g49$1 at sea.gmane.org>
> Content-Type: text/plain; format=flowed; charset="iso-8859-1";
> reply-type=original
>
>
> "Clay Wiedemann" <clay.wiedemann at gmail.com> wrote
>
> > I am not only new to Python, but new to programming in an
> > environment
> > like this, where knowing a little of how the underpinning work is
> > essential.*
>
> Actually I very rarely need to do anything with the environment.
> Most of my iBook Python programming is done in the
> src/python folder of my home directory. I just import
> stuff and it works...
>
> > I am running the latest OSX (10.4.8) and know a little about moving
> > about in the terminal. But the world of paths, etc. has me a little
> > confused.
>
> If you have specific situations where paths are tripping you
> up we can probably offer help. In general you should only need
> 1) The path to Python set up in your login shell script PATH
> 2) The path to Pythons modules in PYTHONPATH
> ie. in your file: ~/.bash_profile.
> (I assume you use bash since think thats the new default
> in 10.4 - I am still on 10.2 which uses tcsh as shell)
>
> > - Can anyone point me to a well-regarded and pithy tutorial
> > regarding
> > the UNIX underpinnings of OSX, especially anything that would help
> > with Python.
>
> Apple have lots of guidance, this is a good jump off page:
>
> http://developer.apple.com/referencelibrary/GettingStarted/GS_MacOSXServer/index.html
>
> Or more usefully there is Pogue's Missing Manual book.
> This is probably best for traditional Mac users stepping into Unix
> for the first time.
> For more depth I found MacOS X Unleashed good, but longgggg
> And finally MacOSX Hacks is good but not really a tutorial.
>
> HTH,
>
>
> --
> Alan Gauld
> Author of the Learn to Program web site
> http://www.freenetpages.co.uk/hp/alan.gauld
>
>
>
>
> ------------------------------
>
> Message: 6
> Date: Sat, 03 Mar 2007 09:20:43 -0500
> From: Kent Johnson <kent37 at tds.net>
> Subject: Re: [Tutor] trouble understanding the python environment in
> OSX
> To: Clay Wiedemann <clay.wiedemann at gmail.com>
> Cc: tutor at python.org
> Message-ID: <45E9843B.3030703 at tds.net>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Clay Wiedemann wrote:
>
> > I am running the latest OSX (10.4.8) and know a little about moving
> > about in the terminal. But the world of paths, etc. has me a little
> > confused. And now it is getting in my way. I was wondering if I could
> > get a little help with some things.
> >
> > - Can anyone point me to a well-regarded and pithy tutorial regarding
> > the UNIX underpinnings of OSX, especially anything that would help
> > with Python. (e.g., I can navigate to Python.framework and versions
> > but see 2.5 and Current, which confuses me since 2.5 is the current
> > version.) I would love to know what the structure of the Python
> > installation means.
>
> Current is an alias for 2.5, which is the current install.
> >
> > - I am having trouble installing the BeautifulSoup package. The
> > installer stumbles on a unicode issue (that the Soup developer says is
> > a problem with my Python installation) -- so what is the best way to
> > manually install this? And does the 2.5 / Current divide matter?
>
> What is the error you get in the installer?
>
> The installer is not really needed in this case. You should be able to
> just download the single file and put it in
> .../Current/lib/python2.5/site-packages
>
> Kent
> >
> >
> >
> > Thanks in advance for any help with these basic issues.
> >
> > Good morning from Brooklyn,
> > Clay
> >
> >
> > * Any experience with coding webpages is not helping here
> >
> >
> >
> > - - - - - - -
> >
> > Clay S. Wiedemann
> >
> > aim: khlav
> > wii: 3905 4571 6175 2469
> > twitter: seastokes
> > _______________________________________________
> > Tutor maillist - Tutor at python.org
> > http://mail.python.org/mailman/listinfo/tutor
> >
>
>
>
> ------------------------------
>
> Message: 7
> Date: Sat, 03 Mar 2007 16:43:00 +0100
> From: Andrei <project5 at redrival.net>
> Subject: Re: [Tutor] Yet another list comprehension question
> To: tutor at python.org
> Message-ID: <esc51k$t8k$1 at sea.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> >> Alternatively, you could put the results as keys in a dictionary,
> >> then request
> >> mydict.keys() to get a list of unique outcomes.
> >
> > I thought of that too, but couldn't think how to do it in a list
> > comprehension. It seemed like it should be possible but I
> > couldn't think of how - and didn't have a python interpreter
> > handy...
>
> I wouldn't do it in a list comprehension - I took a bit of liberty with
> the topic and looked more at the actual problem :). Of course the loop
> can be turned into a list comprehension, but it serves no purpose
> whatsoever:
>
> >>> d = {}
> >>> [d.__setitem__(s, '') for s in myiterator]
>
> By the way, the fromkeys method of the dict type can turn this into a
> oneliner:
>
> >>> mylist = [1,2,3,1,2,4] # may also be some iterator
> >>> print dict.fromkeys(mylist).keys()
> ... [1,2,3,4]
>
> The set solution is the Most Obvious Way to do it, but the dict one
> doesn't require an understanding of list comprehensions.
>
> --
> Yours,
>
> Andrei
>
> =====
> Mail address in header catches spam. Real contact info:
> ''.join([''.join(s) for s in zip(
> "poet at aao.l pmfe!Pes ontuei ulcpss edtels,s hr' one oC.",
> "rjc5wndon.Sa-re laed o s npbi ot.Ira h it oteesn edt C")])
>
>
>
> ------------------------------
>
> Message: 8
> Date: Sat, 03 Mar 2007 11:24:35 -0500
> From: Kent Johnson <kent37 at tds.net>
> Subject: Re: [Tutor] Yet another list comprehension question
> To: Andrei <project5 at redrival.net>
> Cc: tutor at python.org
> Message-ID: <45E9A143.1040707 at tds.net>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Andrei wrote:
> >>> Alternatively, you could put the results as keys in a dictionary,
> >>> then request
> >>> mydict.keys() to get a list of unique outcomes.
> >> I thought of that too, but couldn't think how to do it in a list
> >> comprehension. It seemed like it should be possible but I
> >> couldn't think of how - and didn't have a python interpreter
> >> handy...
> >
> > I wouldn't do it in a list comprehension - I took a bit of liberty with
> > the topic and looked more at the actual problem :). Of course the loop
> > can be turned into a list comprehension, but it serves no purpose
> > whatsoever:
> >
> > >>> d = {}
> > >>> [d.__setitem__(s, '') for s in myiterator]
> >
> > By the way, the fromkeys method of the dict type can turn this into a
> > oneliner:
> >
> > >>> mylist = [1,2,3,1,2,4] # may also be some iterator
> > >>> print dict.fromkeys(mylist).keys()
> > ... [1,2,3,4]
> >
> > The set solution is the Most Obvious Way to do it, but the dict one
> > doesn't require an understanding of list comprehensions.
>
> The list comp in Alan's solution is just needed to introduce somefunc().
> You have to do the same in yours to match the original problem. The set
> version of your solution doesn't need a list comp either, it is just
> list(set(mylist))
>
> There is really no reason in modern Python to use dicts as sets.
>
> Kent
>
>
> ------------------------------
>
> _______________________________________________
> Tutor maillist - Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
> End of Tutor Digest, Vol 37, Issue 10
> *************************************
More information about the Tutor
mailing list