Jim Fulton wrote:

Gordon McMillan wrote:

...

Then how do I get the old behavior?

Just pass 0 or 1 argument.

...

(I've been known to do odd things - like change behavior based on the number of args - in extension modules, but this ain't an extension).

It *is* a built-in method. It will be handled just like dictionaries handle the second argument to get.

d.get(nonexistantkey) does not throw an exception, it returns None. If list.pop() does not throw an exception when list is empty, it's new behavior. Which are you asking for: breaking code that expects IndexError Violating Pythonic expectations by, in effect, creating 2 methods list.pop(void) list.pop(default_return) - Gordon

Jim Fulton wrote:

Gordon McMillan wrote:

...

...

Violating Pythonic expectations by, in effect, creating 2 methods list.pop(void) list.pop(default_return)

Yes, except that I disagree that this is non-pythonic.

I'll leave the final determination to Mr. Python, but I disagree. Offhand I can't think of a built-in that can't be expressed in normal Python notation, where "optional" args are really defaulted args. Which would lead us to either a new list method, or redefining pop: def pop(usedefault=0, default=None) and making you use 2 args. But maybe I've missed a precedent because I'm so used to it. (Hmm, I guess string.split is a sort-of precedent, because the first default arg behaves differently than anything you could pass in). - Gordon

BTW, a good precedent for what I want is getattr. getattr(None,'spam') raises an error, but: getattr(None,'spam',1) returns 1 Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

"Barry A. Warsaw" wrote:

...

...

...

...

...

"JF" == Jim Fulton writes:

JF> BTW, a good precedent for what I want JF> is getattr.

JF> getattr(None,'spam')

JF> raises an error, but:

JF> getattr(None,'spam',1)

JF> returns 1

Okay, how did this one sneak in, huh?

I don't know. Someone told me about it. I find it wildly useful.

I didn't even realize this had been added to getattr()! CVS reveals it was added b/w 1.5.1 and 1.5.2a1, so maybe I just missed the checkin message.

Fred, the built-in-funcs doc needs updating:

http://www.python.org/doc/current/lib/built-in-funcs.html

FWIW, the CVS log message says this feature is experimental too. :)

Eek! I want it to stay! I also really like list.pop. :) Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

Jim Fulton wrote:

[getattr(obj,name[,default])]

...

Okay, how did this one sneak in, huh?

I don't know. Someone told me about it. I find it wildly useful.

Me too... ;-)

...

I didn't even realize this had been added to getattr()! CVS reveals it was added b/w 1.5.1 and 1.5.2a1, so maybe I just missed the checkin message.

http://www.deja.com/getdoc.xp?AN=366635977 -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 162 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

"Barry A. Warsaw" wrote:

...

...

...

...

...

"M" == M writes:

M> http://www.deja.com/getdoc.xp?AN=366635977

Ah, thanks! Your rationale was exactly the reason why I added dict.get(). I'm still not 100% sure about list.pop() though, since it's not exactly equivalent -- list.pop() modifies the list as a side-effect :) Makes me think you might want an alternative spelling for list[s], call it list.get() and put the optional default on that method. Then again, maybe list.pop() with an optional default is good enough.

list.get and list.pop are different, since get wouldn't modify the list and pop would. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

"Barry A. Warsaw" wrote:

...

...

...

...

...

"JF" == Jim Fulton writes:

JF> list.get and list.pop are different, since get wouldn't modify JF> the list and pop would.

Right. Would we need them both?

Sure. Since a sequence is sort of a special kind of mapping, get makes sense. I definately, want pop. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

[Barry]

FWIW, the CVS log message says this feature [3-arg getattr] is experimental too. :)

[Jim]

Eek! I want it to stay!

I also really like list.pop. :)

Don't panic: Guido has never removed a feature explicitly called "experimental"; he's only removed non-experimental ones. that's-why-we-call-stackless-python-"an-experiment"<wink>-ly y'rs - tim

Jim Fulton wrote:

...

Violating Pythonic expectations by, in effect, creating 2 methods list.pop(void) list.pop(default_return)

Yes, except that I disagree that this is non-pythonic.

Wouldn't a generic builtin for these kinds of things be better, e.g. a function returning a default value in case an exception occurs... something like: tryexcept(list.pop(), IndexError, default) which returns default in case an IndexError occurs. Don't think this would be much faster that the explicit try:...except: though... -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 162 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

[M.-A. Lemburg]

Wouldn't a generic builtin for these kinds of things be better, e.g. a function returning a default value in case an exception occurs... something like:

tryexcept(list.pop(), IndexError, default)

which returns default in case an IndexError occurs. Don't think this would be much faster that the explicit try:...except: though...

As a function (builtin or not), tryexcept will never get called if list.pop() raises an exception. tryexcept would need to be a new statement type, and the compiler would have to generate code akin to try: whatever = list.pop() except IndexError: whatever = default If you want to do it in a C function instead to avoid the Python-level exception overhead, the compiler would have to wrap list.pop() in a lambda in order to delay evaluation until the C code got control; and then you've got worse overhead <wink>. generalization-is-the-devil's-playground-ly y'rs - tim

"M.-A. Lemburg" wrote: ...

Also, has anybody considered writing list.pop(..,default) this way:

if list: obj = list.pop() else: obj = default

No exceptions, no changes, fast as hell :-)

Yes, that's the best way to go, I think. But wasn't the primary question directed on an atomic function which is thread-safe? I'm not sure, this thread has grown too fast :-) -- Christian Tismer :^) mailto:tismer@appliedbiometrics.com Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net 10553 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

... Hmm, so maybe we're not after lists after all: maybe what we need is access to the global interpreter lock in Python, so that we can write:

sys.lock.acquire() if list: obj = list.pop() else: obj = default sys.lock.release()

The thread attempting the sys.lock.acquire() necessarily already owns the global lock, so the attempt to acquire it is a guaranteed deadlock -- arguably not helpful <wink>.

Or maybe we need some general lock in the thread module for these purposes... don't know. It's been some time since I used threads.

Jim could easily allocate a list lock for this purpose if that's what he wanted; and wrap it in a class with a nice interface too. He'd eventually end up with the std Queue.py module, though. But if he doesn't want the overhead of an exception when the queue is empty, he sure doesn't want the comparatively huge overhead of a (any flavor of) lock either (which may drag the OS into the picture). There's nothing wrong with wanting a fast thread-safe queue! I just don't like the idea of adding an otherwise-ugly new gimmick to core lists for it; also have to wonder about Jim's larger picture if he's writing stuff in Python that's *so* time-critical that the overhead of an ordinary exception from time to time is a genuine problem. The verbosity of the alternative can be hidden in a lock-free class or function, if it's the clumsiness instead of the time that's grating.

M.-A. Lemburg writes:

True, sys.lock.acquire() would have to set a flag *not* to release the lock until the next call to sys.lock.release(), which then clears this flag again. Sort of a lock for the unlocking the lock ;-)

Could this work, or am I having a mind twister somewhere in there again ?

Sounds like a critical section to me. On Windows, those are lightweight and very handy. You can build one with Python thread primitives, but unfortunately, they come out on the heavy side. Locks come in 4 types, categorized by whether they can be released only by the owning thread, and whether they can be acquired recursively. The interpreter lock is in the opposite quadrant from a critical section, so "sys.lock.freeze()" and "sys.lock.thaw()" have little chance of having an efficient implementation on any platform. A shame. That would be pretty cool. - Gordon

Actually, I think all that's needed is another global like the interpreter_lock in ceval.c. Since this lock is only accessed via abstract functions, I presume the unlock flag could easily be added.

Well, my personal opinion is that this is really quite wrong. The most obvious thing to me is that we are exposing an implementation detail we all would dearly like to see removed one day - the global interpreter lock. But even if we ignore that, it seems to me that you are describing an application abstraction, not a language abstraction. This thread started with Jim wanting a thread-safe, atomic list operation. This is not an unusual requirement (ie, a thread-safe, atomic operation), so languages give you access to primitives that let you build this. To my mind, you are asking for the equivilent of a C function that says "suspend all threads except me, cos Im doing something _really_ important". C does not provide that, and I have never thought it should. As Gordon said, Win32 has critical sections, but these are really just lightweight locks. I really dont see how Python is different - it gives you all the tools you need to build these abstractions. I really dont see what you are after that can not be done with a lock. If the performance is a problem, then to paraphrase the Timbot, it may be questionable if you are using Python appropriately in this case. Mark.

[The previous mail got truncated due to insufficient disk space; here is a summary...] Mark Hammond wrote:

...

Actually, I think all that's needed is another global like the interpreter_lock in ceval.c. Since this lock is only accessed via abstract functions, I presume the unlock flag could easily be added.

Well, my personal opinion is that this is really quite wrong. The most obvious thing to me is that we are exposing an implementation detail we all would dearly like to see removed one day - the global interpreter lock.

But even if we ignore that, it seems to me that you are describing an application abstraction, not a language abstraction. This thread started with Jim wanting a thread-safe, atomic list operation. This is not an unusual requirement (ie, a thread-safe, atomic operation), so languages give you access to primitives that let you build this.

To my mind, you are asking for the equivilent of a C function that says "suspend all threads except me, cos Im doing something _really_ important". C does not provide that, and I have never thought it should. As Gordon said, Win32 has critical sections, but these are really just lightweight locks. I really dont see how Python is different - it gives you all the tools you need to build these abstractions.

I really dont see what you are after that can not be done with a lock. If the performance is a problem, then to paraphrase the Timbot, it may be questionable if you are using Python appropriately in this case.

The locked section may not be leading in the right direction, but it surely helps in situations where you cannot otherwise enforce useage of an object specific lock, e.g. for builtin file objects (some APIs insist on getting the real thing, not a thread safe wrapper). Here is a hack that let's you do much the same with an unpatched Python interpreter: sys.setcheckinterval(sys.maxint) # *) # >=10 Python OPs to flush the ticker counter and have the new # check interavl setting take effect: 0==0; 0==0; 0==0; 0==0 try: ...lock section... finally: sys.setcheckinterval(10) *) sys.setcheckinterval should really return the previous value so that we can reset the value to the original one afterwards. Note that the lock section may not call code which uses the Py_*_ALLOW_THREADS macros. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 157 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Christian Tismer wrote:

"M.-A. Lemburg" wrote: ...

...

Also, has anybody considered writing list.pop(..,default) this way:

if list: obj = list.pop() else: obj = default

No exceptions, no changes, fast as hell :-)

Yes, that's the best way to go, I think. But wasn't the primary question directed on an atomic function which is thread-safe?

Right. And the above code doesn't solve this problem. Tim's code *does* solve the problem. It's the code we were using. It is a bit verbose though.

I'm not sure, this thread has grown too fast :-)

Don't they all? Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

Fred> Giving up already? Wouldn't you just love this as an expression Fred> operator (which could work)? Fred> How about: Fred> top = list.pop() excepting IndexError, default Why not go all the way to Perl with top = list.pop() unless IndexError ??? ;-) Skip

Fred> Skip Montanaro writes: >> Why not go all the way to Perl with >> >> top = list.pop() unless IndexError Fred> Trying to kill me, Skip? ;-) Nope, just a flesh wound. I'll wait for the resulting infection to really do you in. ;-) Fred> Actually, the semantics are different. If we interpret that using Fred> the Perl semantics for "unless", don't we have the same thing as: Yes, but the flavor is the same. Reading Perl code that uses the unless keyword always seemed counterintuitive to me. Something like x = y unless foo; always reads to me like, "Assign y to x. No, wait a minute. I forgot something. Only do that if foo isn't true." What was so bad about if (!foo) { x = y; } That was my initial reaction to the use of the trailing except. We argue a lot in the Python community about whether or not a proposed language feature increases the expressive power of the language or not (which is a good idea in my opinion). The Perl community has apparently never been afflicted with that disease. smiles all 'round... Skip

...

...

...

...

"Gordo" == Gordon McMillan writes:

Gordo> Which are you asking for: breaking code that expects Gordo> IndexError Violating Pythonic expectations by, in effect, Gordo> creating 2 methods Gordo> list.pop(void) Gordo> list.pop(default_return) The docs /do/ say that list.pop() is experimental, so that probably gives Guido all the out he'd need to change the semantics :). I myself have yet to use list.pop() so I don't know how disasterous the change in semantics would be to existing code. -Barry