Re: Python-checkins digest, Vol 1 #370 - 8 msgs

Not sure about remove(), index() and count(), but the change to .append() will break *lots* of code ! -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

MAL wrote:
Not sure about remove(), index() and count(), but the change to .append() will break *lots* of code !
ouch. missed this checkin. I thought all the append fixes in the standard library were made to make it clearer that this use was deprecated. some random grepping through our code repositories makes me think this should be left for Py3K. </F>

Agreed. But 1.6 is as good a point to break it as any -- what's the point in putting this off? This isn't big enough to wait for 2.0. I could've done it in the 1.5.x series, but decided not to. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Hmm, I'd say it doesn't hurt leaving .append() as it is until 2.0. This small change will cause lots of trouble because it's hard to find (just like the x = 2L; print x[:-1] thingie, btw)... even though it's easy to fix. Note: the CVS cgi.py uses list.append(x,y) too. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Sigh. It will be just as hard to fix later... I'd really like to take a hard line stance on this one. After all the docs are quite clear.
Note: the CVS cgi.py uses list.append(x,y) too.
Already fixed in CVS. --Guido van Rossum (home page: http://www.python.org/~guido/)

MAL:
Hmm, I'd say it doesn't hurt leaving .append() as it is until 2.0.
fwiw, I definitely agree. I've spotted too many places where this change cause a program to silently misbehave, rather than blow up (PIL's JPEG plugin is just one example...). I suggest adding an explicit warning in the 1.6 docs, and removing it in the first post-1.6 release. appendnanny.py, anyone? </F>

/F:
Sigh... This smells of a too-inclusive except clause... Otherwise the program should have raised a clear exception. I suggest to fix it rather than whine... Am I responsible for everybody else's bad coding style? If it's not in the docs, where does everybody get the idea that this is legal? (The few cases in the std library are unlikely to be the only source; they were in pretty obscure places.) --Guido van Rossum (home page: http://www.python.org/~guido/)

I think you're responsible for a little bit of the confusion. Most people read the docs at the beginning to learn the basics, and then experiment with the interpreter. The fact that it worked is naturally interpreted by users as meaning that it should work. Very few of Python's semantics are accidental, so people "believe" the interpreter. Teaching people when ()'s are necessary in Python and when they're not is not a trivial task if you're talking to someone who'se never heard the difference between a parse tree and a pear tree. In my courses I typically say "()'s are sometimes optional" and leave it at that -- I expect the students' experience to guide them. That experience will be interaction-driven, not doc-driven. max/min() is another example, btw. What's the "right" way? To call them with N arguments where N > 1, or with a sequence argument? If I look it up in the doc, I can tell (it's the latter, folks) -- but it seems arbitrary. After all, the max/min calls/macros typically used in other languages require 2 arguments, so extending that to N arguments is conceptually at least as easy as shrinking it to a sequence argument. --david

It's a fair cop.
Yes -- and this is one reason why I want to fix append(). I should've fixed it years ago.
This is different. Maybe the docs are wrong; I always intended for both max(a, b, ...) and max(seq) to be valid. (BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 28 Feb 2000, Guido van Rossum wrote:
This is different. Maybe the docs are wrong; I always intended for both max(a, b, ...) and max(seq) to be valid.
I suppose in this case it's clear what you mean just from the number of arguments. But there is a potential surprise if someone who expects to have to say max(a, b, ...) then writes apply(max, tuple) and tuple turns out to only have one element. (I don't think i've ever realized that we could use min() or max() on a sequence.)
(BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.)
Indeed -- but then who do you trust? The first element of the sequence? Is it acceptable for max(a, b, c, d) to read as "a, please tell me which is the maximum among yourself, b, c, and d" ? Does 'a' then have to take care of the type-comparison logic for consistency with everything else? What if 'a' happens to be a built-in type but 'c' is a user-defined instance? -- ?!ng

(BTW, I was wrong about the docs. The docs explain quite clearly that max(a) is different from max(a, b, ...). Learning Python and Python Essential Reference also document both forms.)
Yes, but there simply isn't any need to do this, so it won't occur in practice.
No, that's not what I meant. __min__ and __max__ would be methods of sequences. The min() and max() functions would catch the special case of multiple arguments and translate to min((a, b, ...)) etc. before looking for __min__. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 28 Feb 2000, Ka-Ping Yee wrote:
Ping, I hardly think Python is going to ever have multi-methods... Of course, __max__ (just like __add__ now) should (in some theoretical definition of should) be implemented with multi-dispatch. See MAL's suggestion for coercion for pragmatic (though not theoretic) possible workarounds. -- Moshe Zadka <mzadka@geocities.com>. INTERNET: Learn what you know. Share what you don't.

From: Guido van Rossum <guido@python.org>
(BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.)
I suppose so, although I think the uses of a smart __contains__ are much more frequent than the uses of a smart __max__. On the other hand, I do think that it might be nice to have that sort of hook in the rich array world... On the topic of rich comparisons, I think I have a complete game plan in my head, if not in code. I had to do some figuring out of the mods to the compilation phase to allow short-circuiting with minimal performance impact, as you and Jim H. discussed on the list way back when. But, as you can guess, I'm a bit short on time. [For those of you who don't know, I have a 4-day old daughter at home, and, more relevantly, she has an older brother =)]. I would really like a bit more discussion and decision on coercions before finalizing the rich comparison patches, as I think a coherent coercion strategy will help simplify the patches. Marc-Andre is short on time due to the Unicode stuff, and he posted a teaser to spark some discussion, which got no response at all. I'm not surprised, it's an ugly problem. Did anyone have thoughts that they'd want to share on the topic? --david

That's probably a reflection of the fact that min/max are less frequently used than 'in'. (Which is reflected in making min/max "mere" functions while 'in' is a built-in operator.) I was thinking of any sequence representation that keeps its items sorted (like the old ABC "lists"). Of course, if you're using a hash table, 'in' is trivially answered, but min/max aren't.
On the other hand, I do think that it might be nice to have that sort of hook in the rich array world...
Really? The min/max functions already do all their looping in C.
[I guess you get to worry about the older brother while your wife takes care of the newborn? :-)]
I have no children [yet], but Python is my baby -- and I'm way overcommitted to other Python projects. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Right, but it might make sense to define min() on an array to mean something different than what min-the-builtin could guess from whatever __lt__ returns -- In fact, IIRC, min() on an array which implemented rich comparisons would raise an exception. I don't want to specify the semantics now (arrays are weird sequences), I just appreciate the hook. The semantics would probably depend on the flavor of array one wanted to use.
[I guess you get to worry about the older brother while your wife takes care of the newborn? :-)]
Got it it one. --david

[DA]
On the other hand, I do think that it might be nice to have that sort of hook in the rich array world...
[me]
Really? The min/max functions already do all their looping in C.
[DA]
You're right. I bet Moshe is already coding... (Tip: you could reuse the same flag bit and simply rename it a bit, rather than having separate flag bits. The binary compatibility requirement is only between released versions, not between CVS versions.) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 28 Feb 2000, Guido van Rossum wrote:
My company has an algorithm for knowing when something is risky to implement and use: when I say "hey, that's cool". If you want to follow their wisdom, you shouldn't add __max__ and __min__... My first though was "What use could that be?" and my second was "Yey! We can implement lattices *right* in Python! Cool!", but I don't see any place it's needed <0.9 wink> -- Moshe Zadka <mzadka@geocities.com>. INTERNET: Learn what you know. Share what you don't.

[Guido]
[David Ascher]
I think you're responsible for a little bit of the confusion. <about append working being documentation enough>
David, you're taking Guido out of context: he did not say the "append" was bad style, but that the "too inclusive excpetion" is bad style. -- Moshe Zadka <mzadka@geocities.com>. INTERNET: Learn what you know. Share what you don't.

Moshe Zadka wrote:
yeah, but it wasn't, in this case -- the problem here is that you've added TypeError to the list of exceptions that append may raise, and that it does so on code that works perfectly fine under 1.5.2. ... David points out that Python lets you leave out the parens around tuples in lots of places. consider: "a = 1, 2, 3" vs. "a = (1, 2, 3)" "a, b, c = a" vs. "(a, b, c) = a" "a[a, b, c]" vs. "a[(a, b, c)]" IOW, expecting 'append' to do what it did before is perfectly reasonable (especially given 'extend'), also if you consider what the library reference says: s.append(x) -> same as s[len(s):len(s)] = [x] s.extend(x) -> same as s[len(s):len(s)] = x (raise exception if x is not a list) a[k] -> the item of a with key k a[k] = x -> set a[k] to x ... but now that we have appendnanny, I don't really mind... </F>

[/F]
... but now that we have appendnanny, I don't really mind...
Hmm -- I named *my* thing "checkappend". appendnanny would have been inappropriate: the tabnanny is a merciless harpy nagging about things a grownup should never have to worry about to begin with; i.e., a particularly unpleasant type of nanny. I'm afraid lots of grownups are going to have a short-term problem with the .append change, though. So they don't need a nanny -- they need a tool unimaginatively named to reflect its oh-so-serious purpose. iow-the-tabnanny-holds-its-users-in-contempt-but-checkappend- doesn't<wink>-ly y'rs - tim

[Moshe]
David, you're taking Guido out of context: he did not say the "append" was bad style, but that the "too inclusive excpetion" is bad style.
While that paragraph was placed ambiguously, I *did* mean it to apply to multi-arg append(). It also applies to "too inclusive exception" :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I think you've got a wrong impression here: this is not so much a design question, it's a timing problem: two months are not enough to get all our software ready for 1.6 ... -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

+1 on breaking it now, rather than deferring it Yet Again. IMO, there has been plenty of warning, and there is plenty of time to correct the software. I'm +0 on adding a warning architecture to Python to support issuing a warning/error when .append is called with multiple arguments. -g On Mon, 28 Feb 2000, Guido van Rossum wrote:
-- Greg Stein, http://www.lyra.org/

Greg Stein wrote:
Well, the (bad) effect of this patch is that you cannot run PythonWin any longer unless Mark either supplies an updated distribution, or one corrects the two barfing Scintilla support scripts by hand. Bad for me, since I'm building Stackless Python against 1.5.2+, and that means the users will see PythonWin barf when installing SLP. Adding a warning instead of raising an exception would be nice IMHO, since the warning could probably contain the file name and line number to change, and I would leave my users with this easy task. cheers - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

Christian> Well, the (bad) effect of this patch is that you cannot run Christian> PythonWin any longer unless Mark either supplies an updated Christian> distribution, or one corrects the two barfing Scintilla Christian> support scripts by hand. Zope has the multi-arg append problem as well. It wasn't clear to me where all the problems resided. I've been thinking mostly about the problem in Python code. Thankfully Tim released checkappend last night! Even after getting rid of all the cases it found I was still getting some flaky tracebacks which suggest to me that some C code makes the same sort of assumptions. I've never labored under the assumption that append would take multiple arguments. I was mildly surprised to find an instance of this crop up in my own code... Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/

On Tue, 29 Feb 2000, Christian Tismer wrote:
Yes, but there is no reason to assume this won't happen. Why don't we simply move forward with the assumption that PythonWin and Scintilla will be updated? If we stand around pointing at all the uses of append that are incorrect and claim that is why we can't move forward, then we won't get anywhere. Instead, let's just *MOVE* and see that software authors update accordingly. It isn't like it is a difficult change to make. Heck, PythonWin and Scintilla could be updated within the week and re-released. *WAY* ahead of the 1.6 release.
Bad for me, since I'm building Stackless Python against 1.5.2+, and that means the users will see PythonWin barf when installing SLP.
If you're building a system using an interim release of Python, then I think you need to take responsibility for that. If you don't want those people to have problems, then you can back out the list.append change. Or you can release patches to PythonWin. I don't think the Python world at large should be hampered because somebody is using an unstable/interim version of Python. Again: we couldn't move forward.
Yes, this would be nice. But somebody has to take the time to code it up. The warning won't appear out of nowhere... Cheers, -g -- Greg Stein, http://www.lyra.org/

Why don't we simply move forward with the assumption that PythonWin and Scintilla will be updated?
Done :-) However, I think dropping it now _is_ a little heavy handed. I decided to do a wider search and found a few in, eg, Sam Rushings calldll based ODBC package. Personally, I would much prefer a warning now, and drop it later. _Then_ we can say we have made enough noise about it. It would only be 2 years ago that I became aware that this "feature" of append was not a feature at all - up until then I used it purposely, and habits are sometimes hard to change :-) MArk.

On Wed, 1 Mar 2000, Mark Hammond wrote:
hehe...
What's the difference between a warning and an error? If you're running a program and it suddenly spits out a warning about a misuse of list.append, I'd certainly see that as "the program did something unexpected; that is an error." But this is all moot. Guido has already said that we would be amenable to a warning/error infrastructure which list.append could use. His description used some awkward sentences, so I'm not sure (without spending some brain cycles to parse the email) exactly what his desired defaults and behavior are. But hey... the possibility is there, and is just waiting for somebody to code it. IMO, Guido has left an out for people that are upset with the current hard-line approach. One of those people just needs to spend a bit of time coming up with a patch :-) And yes, Guido is also the Benevolent Dictator and can certainly have his mind changed, so people can definitely continue pestering him to back away from the hard-line approach... Cheers, -g -- Greg Stein, http://www.lyra.org/

On Tue, 29 Feb 2000, Greg Stein wrote:
A big, big difference. Perhaps to one of us, it's the minor inconvenience of reading the error message and inserting a couple of parentheses in the appropriate file -- but to the end user, it's the difference between the program working (albeit noisily) and *not* working. When the program throws an exception and stops, it is safe to say most users will declare it broken and give up. We can't assume that they're going to be able to figure out what to edit (or be brave enough to try) just by reading the error message... or even what interpreter flag to give, if errors (rather than warnings) are the default behaviour. -- ?!ng

On Wed, 1 Mar 2000, Mark Hammond wrote:
I agree with mark. Why the sudden rush?? It seems to me to be unfair to make such a change - one that will break peoples code - without advanced warning, which typically is handled by a deprecation period. There *are* going to be people who won't be informed of the change in the short span of less than a single release. Just because it won't cause you pain isn't a good reason to disregard the pain of those that will suffer, particularly when you can do something relatively low-cost to avoid it. Ken klm@digicool.com

On Tue, 29 Feb 2000, Ken Manheimer wrote:
Sudden rush?!? Mark said he knew about it for a couple years. Same here. It was a long while ago that .append()'s semantics were specified to "no longer" accept multiple arguments. I see in the HISTORY file, that changes were made to Python 1.4 (October, 1996) to avoid calling append() with multiple arguments. So, that is over three years that append() has had multiple-args deprecated. There was probably discussion even before that, but I can't seem to find something to quote. Seems like plenty of time -- far from rushed. Cheers, -g -- Greg Stein, http://www.lyra.org/

On Tue, 29 Feb 2000, Greg Stein wrote:
None the less, for those practicing it, the incorrectness of it will be fresh news. I would be less sympathetic with them if there was recent warning, eg, the schedule for changing it in the next release was part of the current release. But if you tell somebody you're going to change something, and then don't for a few years, you probably need to renew the warning before you make the change. Don't you think so? Why not? Ken klm@digicool.com

Software configuration management is HARD. Every sudden backwards incompatible change (warranted or not) makes it harder. Mutli-arg append is not hurting anyone as much as a sudden change to it would. It would be better to leave append() alone and publicize its near-term removal rather than cause random, part-time supported modules to stop working because their programmers may be too busy to update them right now. So no, I'm not stepping up to do it. But I'm also saying that the better "lazy" option is to put something in a prominent place in the documentation and otherwise leave it alone. <aside> As far as I am concerned, a formal warning-based deprecation mechanism is necessary for Python's continued evolution. Perhaps we can even expose the deprecation flag to the programmer so we can say: if deprecation: print "This module isn't supported anymore." if deprecation: print "Use method FooEx instead." If we had a deprecation mechanism, maybe introducing new keywords would not be quite so painful. Version x deprecates, version y adds the keyword. Mayhap we should also deprecate implicit truncating integral division while we are at it... </aside> -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml

I'm tired of this rhetoric. It's not like I'm changing existing Python installations retroactively. I'm planning to release a new version of Python which no longer supports certain long-obsolete and undocumented behavior. If you maintain a non-core Python module, you should test it against the new release and fix anything that comes up. This is why we have an alpha and beta test cycle and even before that the CVS version. If you are a Python user who depends on a 3rd party module, you need to find out whether the new version is compatible with the 3rd party code you are using, or whether there's a newer version available that solves the incompatibility. There are people who still run Python 1.4 (really!) because they haven't upgraded. I don't have a problem with that -- they don't get much support, but it's their choice, and they may not need the new features introduced since then. I expect that lots of people won't upgrade their Python 1.5.2 to 1.6 right away -- they'll wait until the other modules/packages they need are compatible with 1.6. Multi-arg append probably won't be the only reason why e.g. Digital Creations may need to release an update to Zope for Python 1.6. Zope comes with its own version of Python anyway, so they have control over when they make the switch. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, 29 Feb 2000, Guido van Rossum wrote:
I wholeheartedly support his approach. Just ask Mark Hammond :-) how many times I've said "let's change the code to make it Right; people aren't required to upgrade [and break their code]." Of course, his counter is that people need to upgrade to fix other, unrelated problems. So I relax and try again later :-). But I still maintain that they can independently grab the specific fixes and leave the other changes we make. Maybe it is grey, but I think this change is quite fine. Especially given Tim's tool. Cheers, -g -- Greg Stein, http://www.lyra.org/

[Greg Stein]
What the heck does Tim's one-eyed trouser snake have to do with this? I know *it* likes to think it's the measure of all things, but, frankly, my tool barely affects the world at all a mere two feet beyond its base <wink>. tim-and-his-tool-think-the-change-is-a-mixed-thing-but-on-balance- the-best-thing-ly y'rs - tim

On Wed, 1 Mar 2000, Tim Peters wrote:
Heh. Now how is one supposed to respond to *that* ??! All right. Fine. +3 cool points go to Tim. :-) -- Greg Stein, http://www.lyra.org/

Guido van Rossum wrote:
My concernc is when I want to build an application with a module that only works with Python 1.5.2 and another one that only works with Python 1.6. If we can avoid that situation by making 1.6 compatible with 1.5.2. we should. By the time 1.7 comes around I will accept that everyone has had enough time to update their modules. Remember that many module authors are just part time volunteers. They may only use Python every few months when they get a spare weekend! I really hope that Andrew is wrong when he predicts that there may be lots of different places where Python 1.6 breaks code! I'm in favor of being a total jerk when it comes to Py3K but Python has been pretty conservative thus far. Could someone remind in one sentence what the downside is for treating this as a warning condition as Java does with its deprecated features? Then the CP4E people don't get into bad habits and those same CP4E people trying to use older modules don't run into frustrating runtime errors. Do it for the CP4E people! (how's that for rhetoric) -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "We still do not know why mathematics is true and whether it is certain. But we know what we do not know in an immeasurably richer way than we did. And learning this has been a remarkable achievement, among the greatest and least known of the modern era." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml

[Paul Prescod]
Simply the lack of anything to build on: Python has no sort of runtime warning system now, and nobody has volunteered to create one. If you do <wink>, remember that stdout & stderr may go to the bit bucket in a GUI app. The bit about dropping the "L" suffix on longs seems unwarnable-about in any case (short of warning every time anyone uses long()). remember-that-you-asked-for-the-problems-not-for-solutions<wink>-ly y'rs - tim

On Tue, 29 Feb 2000, Ken Manheimer wrote:
I agree. Note that Guido posted a note to c.l.py on Monday. I believe that meets your notification criteria. Cheers, -g -- Greg Stein, http://www.lyra.org/

Greg Stein wrote:
Note that Guido posted a note to c.l.py on Monday. I believe that meets your notification criteria.
ahem. do you seriously believe that everyone in the Python universe reads comp.lang.python? afaik, most Python programmers don't. ... so as far as I'm concerned, this was officially deprecated with Guido's post. afaik, no official python documentation has explicitly mentioned this (and the fact that it doesn't explicitly allow it doesn't really matter, since the docs don't explicitly allow the x[a, b, c] syntax either. both work in 1.5.2). has anyone checked the recent crop of Python books, btw? the eff-bot guide uses old syntax in two examples out of 320. how about the others? ... sigh. running checkappend over a 50k LOC application, I just realized that it doesn't catch a very common append pydiom. how fun. even though 99% of all append calls are "legal", this "minor" change will break every single application and library we have :-( oh, wait. xmlrpclib isn't affected. always something! </F>

On Wed, 1 Mar 2000, Fredrik Lundh wrote:
Now you're simply taking my comments out of context. Not a proper thing to do. Ken said that he wanted notification along certain guidelines. I said that I believed Guido's post did just that. Period. Personally, I think it is fine. I also think that a CHANGES file that arrives with 1.6 that points out the incompatibility is also fine.
And which is that? Care to help out? Maybe just a little bit? Or do you just want to talk about how bad this change is? :-( Cheers, -g -- Greg Stein, http://www.lyra.org/

Greg Stein <gstein@lyra.org> wrote:
my point was that most Python programmers won't see that notification. when these people download 1.6 final and find that all theirs apps just broke, they probably won't be happy with a pointer to dejanews.
And which is that? Care to help out? Maybe just a little bit?
this rather common pydiom: append = list.append for x in something: append(...) it's used a lot where performance matters.
Or do you just want to talk about how bad this change is? :-(
yes, I think it's bad. I've been using Python since 1.2, and no other change has had the same consequences (wrt. time/money required to fix it) call me a crappy programmer if you want, but I'm sure there are others out there who are nearly as bad. and lots of them won't be aware of this change until some- one upgrades the python interpreter on their server. </F>

Fredrik Lundh wrote:
Dito. Anyone remember the str(2L) == '2' change, BTW ? That one will cost lots of money in case someone implemented an eShop using the common str(2L)[:-1] idiom... There will need to be a big warning sign somewhere that people see *before* finding the download link. (IMHO, anyways.)
Same here. checkappend.py doesn't find these (a great tool BTW, thanks Tim; I noticed that it leaks memory badly though).
-- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

[/F]
[M.-A. Lemburg]
Same here. checkappend.py doesn't find these
As detailed in a c.l.py posting, I have yet to find a single instance of this actually called with multiple arguments. Pointing out that it's *possible* isn't the same as demonstrating it's an actual problem. I'm quite willing to believe that it is, but haven't yet seen evidence of it. For whatever reason, people seem much (and, in my experience so far, infinitely <wink>) more prone to make the list.append(1, 2, 3) error than the maybethisisanappend(1, 2, 3) error.
(a great tool BTW, thanks Tim; I noticed that it leaks memory badly though).
Which Python? Which OS? How do you know? What were you running it over? Using 1.5.2 under Win95, according to wintop, & over the whole CVS tree, the total (code + data) virtual memory allocated to it peaked at about 2Mb a few seconds into the run, and actually decreased as time went on. So, akin to the bound method multi-argument append problem, the "checkappend leak problem" is something I simply have no reason to believe <wink>. Check your claim again? checkappend.py itself obviously creates no cycles or holds on to any state across files, so if you're seeing a leak it must be a bug in some other part of the version of Python + std libraries you're using. Maybe a new 1.6 bug? Something you did while adding Unicode? Etc. Tell us what you were running. Has anyone else seen a leak?

Tim Peters wrote:
Haven't had time to check this yet, but I'm pretty sure there are some instances of this idiom in my code. Note that I did in fact code like this on purpose: it saves a tuple construction for every append, which can make a difference in tight loops...
Of course... still there are hidden instances of the problem which are yet to be revealed. For my own code the siutation is even worse, since I sometimes did: add = list.append for x in y: add(x,1,2)
That's Python 1.5 on Linux2. I let the script run over a large lib directory and my projects directory. In the projects directory the script consumed as much as 240MB of process size.
I'll try the same thing again using Python1.5.2 and the CVS version. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

"M.-A. Lemburg" wrote:
Using the Unicode patched CVS version there's no leak anymore. Couldn't find a 1.5.2 version on my machine... I'll build one later. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Fredrik Lundh writes:
There are more things in 1.6 that might require fixing existing code: str(2L) returning '2', the int/long changes, the Unicode changes, and if it gets added, garbage collection -- and bugs caused by those changes might not be catchable by a nanny. IMHO it's too early to point at the .append() change as breaking too much existing code; there may be changes that break a lot more. I'd wait and see what happens once the 1.6 alphas become available; if c.l.p is filled with shrieks and groans, GvR might decide to back the offending change out. (Or he might not...) -- A.M. Kuchling http://starship.python.net/crew/amk/ I have no skills with machines. I fear them, and because I cannot help attributing human qualities to them, I suspect that they hate me and will kill me if they can. -- Robertson Davies, "Reading"

[/F]
The "Subscriptions" section of the Reference Manual explicitly allows for dict[a, b, c] and explicitly does not allow for sequence[a, b, c] The "Mapping Types" section of the Library Ref does not explicitly allow for it, though, and if you read it as implicitly allowing for it (based on the Reference Manual's clarification of "key" syntax), you would also have to read the Library Ref as allowing for dict.has_key(a, b, c) Which 1.5.2 does allow, but which Guido very recently patched to treat as a syntax error.
[And, later, after prodding by GregS]
This limitation was pointed out in checkappend's module docstring. Doesn't make it any easier for you to swallow, but I needed to point out that you didn't *have* to stumble into this the hard way <wink>.
What would you like to do, then? The code will be at least as broken a year from now, and probably more so -- unless you fix it. So this sounds like an indirect argument for never changing Python's behavior here. Frankly, I expect you could fix the 50K LOC in less time than it took me to write this naggy response <0.50K wink>. embrace-change-ly y'rs - tim

Tim Peters wrote:
I'd thought we'd agreed that nobody reads the reference manual ;-)
What would you like to do, then?
more time to fix it, perhaps? it's surely a minor code change, but fixing it can be harder than you think (just witness Gerrit's bogus patches) after all, python might be free, but more and more people are investing lots of money in using it [1].
The code will be at least as broken a year from now, and probably more so -- unless you fix it.
sure. we've already started. but it's a lot of work, and it's quite likely that it will take a while until we can be 100% confident that all the changes are pro- perly done. (not all software have a 100% complete test suite that simply says "yes, this works" or "no, it doesn't") </F> 1) fwiw, some poor soul over here posted a short note to the pythonworks mailing, mentioning that we've now fixed the price. a major flamewar erupted, and my mail- box is now full of mail from unknowns telling me that I must be a complete moron that doesn't understand that Python is just a toy system, which everyone uses just be- cause they cannot afford anything better...

On Tue, 29 Feb 2000, Greg Stein wrote:
Actually, by "part of the current release", i meant having the deprecation/impending-deletion warning in the release notes for the release before the one where the deletion happens - saying it's being deprecated now, will be deleted next time around. Ken klm@digicool.com I mean, you tell one guy it's blue. He tells his guy it's brown, and it lands on the page sorta purple. Wavy Gravy/Hugh Romney

Fredrik Lundh writes ( about .append(x,y) ):
I suggest adding an explicit warning in the 1.6 docs, and removing it in the first post-1.6 release.
What about adding a command-line switch for enabling warnings, as has been suggested long ago? The .append() change could then print a warning in 1.6alphas (and betas?), but still run, and be turned into an error later. -- A.M. Kuchling http://starship.python.net/crew/amk/ Well, that's a little thing -- the specification. -- Guido van Rossum at IPC7

That's better. I propose that the warnings are normally on, and that there are flags to turn them off or thrn them into errors. Alternatively, the default should be warnings == errors, with flags to turn them into warnings or turn them off. Turning them off by default seems to defeat the purpose -- only the most zealous will use them. The warning messages should show a source file and line number and display the source code line if available, just like in a traceback; but no traceback info should be printed. Also, a warning should only be printed once per message/file/lineno combination, per Python invocation. A dictionary could take care of this. Anybody care to further specify and then code up such a mechanism? --Guido van Rossum (home page: http://www.python.org/~guido/)

[/F]
appendnanny.py, anyone?
Yes, assuming nobody else has done so (there are still 200 new inbox msgs I haven't gotten to yet!), I intend to write that tonight, and post it to the patches list. grep is miserably inadequate for this task (I know -- I've it on more than one occasion for this purpose). nanny-nanny-ly y'rs - tim

Somebody feel like whipping up a patch to fix the rest of the .append errors in the CVS tree? I'm out of time! .\Tools\scripts\mailerdaemon.py(221): list.append(errordict[e], errorfirst[e], errorlast[e], e) .\Demo\tkinter\www\fmt.py(224): self.para.words.append(self.nextfont, text, \ .\Demo\tkinter\www\fmt.py(343): para.words.append('r', '', 0, 0, 0, 0) # temporary, deleted at end .\Demo\tkinter\www\sgmllib.py(199): attrs.append(string.lower(attrname), attrvalue) .\Demo\sgi\video\VcrIndex.py(84): sorted.append(self.movies[name]['-ALL-']['START'], name) .\Demo\sgi\video\VcrIndex.py(100): sorted.append(scenedict[name], name) .\Demo\sgi\video\VcrIndex.py(116): sorted.append(scenedict[name]['START'], name) .\Demo\sgi\gl\glstdwin\glstdwin.py(314): G.queue.append(WE_DEACTIVATE, G.focus, None) .\Demo\sgi\gl\glstdwin\glstdwin.py(317): G.queue.append(WE_ACTIVATE, G.focus, None) .\Demo\ibrowse\ibrowse.py(492): win.last.append(lastnode, win.textobj.getfocus())

Done.
This whole directory should be nixed -- it's 5 years old and contains mostly misinformation, like ancient versions of standard library modules...
This hardware probably doesn't even exist any more... I'm for nixing it too.
All these are stdwin-related. Stdwin will also go out of service per 1.6. (Conclusion: most multi-arg append() calls are *very* old, or contributed by others. Sigh. I must've given bad examples long ago...) --Guido van Rossum (home page: http://www.python.org/~guido/)

[Tim, runs checkappend.py over the entire CVS tree, comes up with surprisingly many remaining problems, and surprisingly few false hits] [Guido fixes mailerdaemon.py, and argues for nuking Demo\tkinter\www\ (the whole directory) Demo\sgi\video\VcrIndex.py (unclear whether the dir or just the file) Demo\sgi\gl\glstdwin\glstdwin.py (stdwin-related) Demo\ibrowse\ibrowse.py (stdwin-related)
Then the sooner someone nukes them from the CVS tree, the sooner my automated hourly checkappend complaint generator will stop pestering Python-Dev about them <wink>.
(Conclusion: most multi-arg append() calls are *very* old,
But part of that is because we went thru this exercise a couple years ago too, and you repaired all the ones in the less obscure parts of the distribution then.
or contributed by others. Sigh. I must've given bad examples long ago...)
Na, I doubt that. Most people will not read a language defn, at least not until "something doesn't work". If the compiler accepts a thing, they simply *assume* it's correct. It's pretty easy (at least for me!) to make this particular mistake as a careless typo, so I assume that's the "source origin" for many of these too. As soon you *notice* you've done it, and that nothing bad happened, the natural tendencies are to (a) believe it's OK, and (b) save 4 keystrokes (incl. the SHIFTs) over & over again in the glorious indefinite future <wink>. Reminds me of a c.l.py thread a while back, wherein someone did stuff like None, x, y, None = function_returning_a_4_tuple to mean that they didn't care what the 1st & 4th values were. It happened to work, so they did it more & more. Eventually a function containing this mistake needed to reference None after that line, and "suddenly for no reason at all Python stopped working". To the extent that you're serious about CP4E, you're begging for more of this, not less <wink>. newbies-even-keep-on-doing-things-that-*don't*-work!-ly y'rs - tim

To the extent that you're serious about CP4E, you're begging for more of this, not less <wink>.
Which is exactly why I am breaking multi-arg append now -- this is my last chance. --Guido van Rossum (home page: http://www.python.org/~guido/)

MAL wrote:
Not sure about remove(), index() and count(), but the change to .append() will break *lots* of code !
ouch. missed this checkin. I thought all the append fixes in the standard library were made to make it clearer that this use was deprecated. some random grepping through our code repositories makes me think this should be left for Py3K. </F>

Agreed. But 1.6 is as good a point to break it as any -- what's the point in putting this off? This isn't big enough to wait for 2.0. I could've done it in the 1.5.x series, but decided not to. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Hmm, I'd say it doesn't hurt leaving .append() as it is until 2.0. This small change will cause lots of trouble because it's hard to find (just like the x = 2L; print x[:-1] thingie, btw)... even though it's easy to fix. Note: the CVS cgi.py uses list.append(x,y) too. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Sigh. It will be just as hard to fix later... I'd really like to take a hard line stance on this one. After all the docs are quite clear.
Note: the CVS cgi.py uses list.append(x,y) too.
Already fixed in CVS. --Guido van Rossum (home page: http://www.python.org/~guido/)

MAL:
Hmm, I'd say it doesn't hurt leaving .append() as it is until 2.0.
fwiw, I definitely agree. I've spotted too many places where this change cause a program to silently misbehave, rather than blow up (PIL's JPEG plugin is just one example...). I suggest adding an explicit warning in the 1.6 docs, and removing it in the first post-1.6 release. appendnanny.py, anyone? </F>

/F:
Sigh... This smells of a too-inclusive except clause... Otherwise the program should have raised a clear exception. I suggest to fix it rather than whine... Am I responsible for everybody else's bad coding style? If it's not in the docs, where does everybody get the idea that this is legal? (The few cases in the std library are unlikely to be the only source; they were in pretty obscure places.) --Guido van Rossum (home page: http://www.python.org/~guido/)

I think you're responsible for a little bit of the confusion. Most people read the docs at the beginning to learn the basics, and then experiment with the interpreter. The fact that it worked is naturally interpreted by users as meaning that it should work. Very few of Python's semantics are accidental, so people "believe" the interpreter. Teaching people when ()'s are necessary in Python and when they're not is not a trivial task if you're talking to someone who'se never heard the difference between a parse tree and a pear tree. In my courses I typically say "()'s are sometimes optional" and leave it at that -- I expect the students' experience to guide them. That experience will be interaction-driven, not doc-driven. max/min() is another example, btw. What's the "right" way? To call them with N arguments where N > 1, or with a sequence argument? If I look it up in the doc, I can tell (it's the latter, folks) -- but it seems arbitrary. After all, the max/min calls/macros typically used in other languages require 2 arguments, so extending that to N arguments is conceptually at least as easy as shrinking it to a sequence argument. --david

It's a fair cop.
Yes -- and this is one reason why I want to fix append(). I should've fixed it years ago.
This is different. Maybe the docs are wrong; I always intended for both max(a, b, ...) and max(seq) to be valid. (BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 28 Feb 2000, Guido van Rossum wrote:
This is different. Maybe the docs are wrong; I always intended for both max(a, b, ...) and max(seq) to be valid.
I suppose in this case it's clear what you mean just from the number of arguments. But there is a potential surprise if someone who expects to have to say max(a, b, ...) then writes apply(max, tuple) and tuple turns out to only have one element. (I don't think i've ever realized that we could use min() or max() on a sequence.)
(BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.)
Indeed -- but then who do you trust? The first element of the sequence? Is it acceptable for max(a, b, c, d) to read as "a, please tell me which is the maximum among yourself, b, c, and d" ? Does 'a' then have to take care of the type-comparison logic for consistency with everything else? What if 'a' happens to be a built-in type but 'c' is a user-defined instance? -- ?!ng

(BTW, I was wrong about the docs. The docs explain quite clearly that max(a) is different from max(a, b, ...). Learning Python and Python Essential Reference also document both forms.)
Yes, but there simply isn't any need to do this, so it won't occur in practice.
No, that's not what I meant. __min__ and __max__ would be methods of sequences. The min() and max() functions would catch the special case of multiple arguments and translate to min((a, b, ...)) etc. before looking for __min__. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 28 Feb 2000, Ka-Ping Yee wrote:
Ping, I hardly think Python is going to ever have multi-methods... Of course, __max__ (just like __add__ now) should (in some theoretical definition of should) be implemented with multi-dispatch. See MAL's suggestion for coercion for pragmatic (though not theoretic) possible workarounds. -- Moshe Zadka <mzadka@geocities.com>. INTERNET: Learn what you know. Share what you don't.

From: Guido van Rossum <guido@python.org>
(BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.)
I suppose so, although I think the uses of a smart __contains__ are much more frequent than the uses of a smart __max__. On the other hand, I do think that it might be nice to have that sort of hook in the rich array world... On the topic of rich comparisons, I think I have a complete game plan in my head, if not in code. I had to do some figuring out of the mods to the compilation phase to allow short-circuiting with minimal performance impact, as you and Jim H. discussed on the list way back when. But, as you can guess, I'm a bit short on time. [For those of you who don't know, I have a 4-day old daughter at home, and, more relevantly, she has an older brother =)]. I would really like a bit more discussion and decision on coercions before finalizing the rich comparison patches, as I think a coherent coercion strategy will help simplify the patches. Marc-Andre is short on time due to the Unicode stuff, and he posted a teaser to spark some discussion, which got no response at all. I'm not surprised, it's an ugly problem. Did anyone have thoughts that they'd want to share on the topic? --david

That's probably a reflection of the fact that min/max are less frequently used than 'in'. (Which is reflected in making min/max "mere" functions while 'in' is a built-in operator.) I was thinking of any sequence representation that keeps its items sorted (like the old ABC "lists"). Of course, if you're using a hash table, 'in' is trivially answered, but min/max aren't.
On the other hand, I do think that it might be nice to have that sort of hook in the rich array world...
Really? The min/max functions already do all their looping in C.
[I guess you get to worry about the older brother while your wife takes care of the newborn? :-)]
I have no children [yet], but Python is my baby -- and I'm way overcommitted to other Python projects. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Right, but it might make sense to define min() on an array to mean something different than what min-the-builtin could guess from whatever __lt__ returns -- In fact, IIRC, min() on an array which implemented rich comparisons would raise an exception. I don't want to specify the semantics now (arrays are weird sequences), I just appreciate the hook. The semantics would probably depend on the flavor of array one wanted to use.
[I guess you get to worry about the older brother while your wife takes care of the newborn? :-)]
Got it it one. --david

[DA]
On the other hand, I do think that it might be nice to have that sort of hook in the rich array world...
[me]
Really? The min/max functions already do all their looping in C.
[DA]
You're right. I bet Moshe is already coding... (Tip: you could reuse the same flag bit and simply rename it a bit, rather than having separate flag bits. The binary compatibility requirement is only between released versions, not between CVS versions.) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, 28 Feb 2000, Guido van Rossum wrote:
My company has an algorithm for knowing when something is risky to implement and use: when I say "hey, that's cool". If you want to follow their wisdom, you shouldn't add __max__ and __min__... My first though was "What use could that be?" and my second was "Yey! We can implement lattices *right* in Python! Cool!", but I don't see any place it's needed <0.9 wink> -- Moshe Zadka <mzadka@geocities.com>. INTERNET: Learn what you know. Share what you don't.

[Guido]
[David Ascher]
I think you're responsible for a little bit of the confusion. <about append working being documentation enough>
David, you're taking Guido out of context: he did not say the "append" was bad style, but that the "too inclusive excpetion" is bad style. -- Moshe Zadka <mzadka@geocities.com>. INTERNET: Learn what you know. Share what you don't.

Moshe Zadka wrote:
yeah, but it wasn't, in this case -- the problem here is that you've added TypeError to the list of exceptions that append may raise, and that it does so on code that works perfectly fine under 1.5.2. ... David points out that Python lets you leave out the parens around tuples in lots of places. consider: "a = 1, 2, 3" vs. "a = (1, 2, 3)" "a, b, c = a" vs. "(a, b, c) = a" "a[a, b, c]" vs. "a[(a, b, c)]" IOW, expecting 'append' to do what it did before is perfectly reasonable (especially given 'extend'), also if you consider what the library reference says: s.append(x) -> same as s[len(s):len(s)] = [x] s.extend(x) -> same as s[len(s):len(s)] = x (raise exception if x is not a list) a[k] -> the item of a with key k a[k] = x -> set a[k] to x ... but now that we have appendnanny, I don't really mind... </F>

[/F]
... but now that we have appendnanny, I don't really mind...
Hmm -- I named *my* thing "checkappend". appendnanny would have been inappropriate: the tabnanny is a merciless harpy nagging about things a grownup should never have to worry about to begin with; i.e., a particularly unpleasant type of nanny. I'm afraid lots of grownups are going to have a short-term problem with the .append change, though. So they don't need a nanny -- they need a tool unimaginatively named to reflect its oh-so-serious purpose. iow-the-tabnanny-holds-its-users-in-contempt-but-checkappend- doesn't<wink>-ly y'rs - tim

[Moshe]
David, you're taking Guido out of context: he did not say the "append" was bad style, but that the "too inclusive excpetion" is bad style.
While that paragraph was placed ambiguously, I *did* mean it to apply to multi-arg append(). It also applies to "too inclusive exception" :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I think you've got a wrong impression here: this is not so much a design question, it's a timing problem: two months are not enough to get all our software ready for 1.6 ... -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

+1 on breaking it now, rather than deferring it Yet Again. IMO, there has been plenty of warning, and there is plenty of time to correct the software. I'm +0 on adding a warning architecture to Python to support issuing a warning/error when .append is called with multiple arguments. -g On Mon, 28 Feb 2000, Guido van Rossum wrote:
-- Greg Stein, http://www.lyra.org/

Greg Stein wrote:
Well, the (bad) effect of this patch is that you cannot run PythonWin any longer unless Mark either supplies an updated distribution, or one corrects the two barfing Scintilla support scripts by hand. Bad for me, since I'm building Stackless Python against 1.5.2+, and that means the users will see PythonWin barf when installing SLP. Adding a warning instead of raising an exception would be nice IMHO, since the warning could probably contain the file name and line number to change, and I would leave my users with this easy task. cheers - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

Christian> Well, the (bad) effect of this patch is that you cannot run Christian> PythonWin any longer unless Mark either supplies an updated Christian> distribution, or one corrects the two barfing Scintilla Christian> support scripts by hand. Zope has the multi-arg append problem as well. It wasn't clear to me where all the problems resided. I've been thinking mostly about the problem in Python code. Thankfully Tim released checkappend last night! Even after getting rid of all the cases it found I was still getting some flaky tracebacks which suggest to me that some C code makes the same sort of assumptions. I've never labored under the assumption that append would take multiple arguments. I was mildly surprised to find an instance of this crop up in my own code... Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/

On Tue, 29 Feb 2000, Christian Tismer wrote:
Yes, but there is no reason to assume this won't happen. Why don't we simply move forward with the assumption that PythonWin and Scintilla will be updated? If we stand around pointing at all the uses of append that are incorrect and claim that is why we can't move forward, then we won't get anywhere. Instead, let's just *MOVE* and see that software authors update accordingly. It isn't like it is a difficult change to make. Heck, PythonWin and Scintilla could be updated within the week and re-released. *WAY* ahead of the 1.6 release.
Bad for me, since I'm building Stackless Python against 1.5.2+, and that means the users will see PythonWin barf when installing SLP.
If you're building a system using an interim release of Python, then I think you need to take responsibility for that. If you don't want those people to have problems, then you can back out the list.append change. Or you can release patches to PythonWin. I don't think the Python world at large should be hampered because somebody is using an unstable/interim version of Python. Again: we couldn't move forward.
Yes, this would be nice. But somebody has to take the time to code it up. The warning won't appear out of nowhere... Cheers, -g -- Greg Stein, http://www.lyra.org/

Why don't we simply move forward with the assumption that PythonWin and Scintilla will be updated?
Done :-) However, I think dropping it now _is_ a little heavy handed. I decided to do a wider search and found a few in, eg, Sam Rushings calldll based ODBC package. Personally, I would much prefer a warning now, and drop it later. _Then_ we can say we have made enough noise about it. It would only be 2 years ago that I became aware that this "feature" of append was not a feature at all - up until then I used it purposely, and habits are sometimes hard to change :-) MArk.

On Wed, 1 Mar 2000, Mark Hammond wrote:
hehe...
What's the difference between a warning and an error? If you're running a program and it suddenly spits out a warning about a misuse of list.append, I'd certainly see that as "the program did something unexpected; that is an error." But this is all moot. Guido has already said that we would be amenable to a warning/error infrastructure which list.append could use. His description used some awkward sentences, so I'm not sure (without spending some brain cycles to parse the email) exactly what his desired defaults and behavior are. But hey... the possibility is there, and is just waiting for somebody to code it. IMO, Guido has left an out for people that are upset with the current hard-line approach. One of those people just needs to spend a bit of time coming up with a patch :-) And yes, Guido is also the Benevolent Dictator and can certainly have his mind changed, so people can definitely continue pestering him to back away from the hard-line approach... Cheers, -g -- Greg Stein, http://www.lyra.org/

On Tue, 29 Feb 2000, Greg Stein wrote:
A big, big difference. Perhaps to one of us, it's the minor inconvenience of reading the error message and inserting a couple of parentheses in the appropriate file -- but to the end user, it's the difference between the program working (albeit noisily) and *not* working. When the program throws an exception and stops, it is safe to say most users will declare it broken and give up. We can't assume that they're going to be able to figure out what to edit (or be brave enough to try) just by reading the error message... or even what interpreter flag to give, if errors (rather than warnings) are the default behaviour. -- ?!ng

On Wed, 1 Mar 2000, Mark Hammond wrote:
I agree with mark. Why the sudden rush?? It seems to me to be unfair to make such a change - one that will break peoples code - without advanced warning, which typically is handled by a deprecation period. There *are* going to be people who won't be informed of the change in the short span of less than a single release. Just because it won't cause you pain isn't a good reason to disregard the pain of those that will suffer, particularly when you can do something relatively low-cost to avoid it. Ken klm@digicool.com

On Tue, 29 Feb 2000, Ken Manheimer wrote:
Sudden rush?!? Mark said he knew about it for a couple years. Same here. It was a long while ago that .append()'s semantics were specified to "no longer" accept multiple arguments. I see in the HISTORY file, that changes were made to Python 1.4 (October, 1996) to avoid calling append() with multiple arguments. So, that is over three years that append() has had multiple-args deprecated. There was probably discussion even before that, but I can't seem to find something to quote. Seems like plenty of time -- far from rushed. Cheers, -g -- Greg Stein, http://www.lyra.org/

On Tue, 29 Feb 2000, Greg Stein wrote:
None the less, for those practicing it, the incorrectness of it will be fresh news. I would be less sympathetic with them if there was recent warning, eg, the schedule for changing it in the next release was part of the current release. But if you tell somebody you're going to change something, and then don't for a few years, you probably need to renew the warning before you make the change. Don't you think so? Why not? Ken klm@digicool.com

Software configuration management is HARD. Every sudden backwards incompatible change (warranted or not) makes it harder. Mutli-arg append is not hurting anyone as much as a sudden change to it would. It would be better to leave append() alone and publicize its near-term removal rather than cause random, part-time supported modules to stop working because their programmers may be too busy to update them right now. So no, I'm not stepping up to do it. But I'm also saying that the better "lazy" option is to put something in a prominent place in the documentation and otherwise leave it alone. <aside> As far as I am concerned, a formal warning-based deprecation mechanism is necessary for Python's continued evolution. Perhaps we can even expose the deprecation flag to the programmer so we can say: if deprecation: print "This module isn't supported anymore." if deprecation: print "Use method FooEx instead." If we had a deprecation mechanism, maybe introducing new keywords would not be quite so painful. Version x deprecates, version y adds the keyword. Mayhap we should also deprecate implicit truncating integral division while we are at it... </aside> -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml

I'm tired of this rhetoric. It's not like I'm changing existing Python installations retroactively. I'm planning to release a new version of Python which no longer supports certain long-obsolete and undocumented behavior. If you maintain a non-core Python module, you should test it against the new release and fix anything that comes up. This is why we have an alpha and beta test cycle and even before that the CVS version. If you are a Python user who depends on a 3rd party module, you need to find out whether the new version is compatible with the 3rd party code you are using, or whether there's a newer version available that solves the incompatibility. There are people who still run Python 1.4 (really!) because they haven't upgraded. I don't have a problem with that -- they don't get much support, but it's their choice, and they may not need the new features introduced since then. I expect that lots of people won't upgrade their Python 1.5.2 to 1.6 right away -- they'll wait until the other modules/packages they need are compatible with 1.6. Multi-arg append probably won't be the only reason why e.g. Digital Creations may need to release an update to Zope for Python 1.6. Zope comes with its own version of Python anyway, so they have control over when they make the switch. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, 29 Feb 2000, Guido van Rossum wrote:
I wholeheartedly support his approach. Just ask Mark Hammond :-) how many times I've said "let's change the code to make it Right; people aren't required to upgrade [and break their code]." Of course, his counter is that people need to upgrade to fix other, unrelated problems. So I relax and try again later :-). But I still maintain that they can independently grab the specific fixes and leave the other changes we make. Maybe it is grey, but I think this change is quite fine. Especially given Tim's tool. Cheers, -g -- Greg Stein, http://www.lyra.org/

[Greg Stein]
What the heck does Tim's one-eyed trouser snake have to do with this? I know *it* likes to think it's the measure of all things, but, frankly, my tool barely affects the world at all a mere two feet beyond its base <wink>. tim-and-his-tool-think-the-change-is-a-mixed-thing-but-on-balance- the-best-thing-ly y'rs - tim

On Wed, 1 Mar 2000, Tim Peters wrote:
Heh. Now how is one supposed to respond to *that* ??! All right. Fine. +3 cool points go to Tim. :-) -- Greg Stein, http://www.lyra.org/

Guido van Rossum wrote:
My concernc is when I want to build an application with a module that only works with Python 1.5.2 and another one that only works with Python 1.6. If we can avoid that situation by making 1.6 compatible with 1.5.2. we should. By the time 1.7 comes around I will accept that everyone has had enough time to update their modules. Remember that many module authors are just part time volunteers. They may only use Python every few months when they get a spare weekend! I really hope that Andrew is wrong when he predicts that there may be lots of different places where Python 1.6 breaks code! I'm in favor of being a total jerk when it comes to Py3K but Python has been pretty conservative thus far. Could someone remind in one sentence what the downside is for treating this as a warning condition as Java does with its deprecated features? Then the CP4E people don't get into bad habits and those same CP4E people trying to use older modules don't run into frustrating runtime errors. Do it for the CP4E people! (how's that for rhetoric) -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "We still do not know why mathematics is true and whether it is certain. But we know what we do not know in an immeasurably richer way than we did. And learning this has been a remarkable achievement, among the greatest and least known of the modern era." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml

[Paul Prescod]
Simply the lack of anything to build on: Python has no sort of runtime warning system now, and nobody has volunteered to create one. If you do <wink>, remember that stdout & stderr may go to the bit bucket in a GUI app. The bit about dropping the "L" suffix on longs seems unwarnable-about in any case (short of warning every time anyone uses long()). remember-that-you-asked-for-the-problems-not-for-solutions<wink>-ly y'rs - tim

On Tue, 29 Feb 2000, Ken Manheimer wrote:
I agree. Note that Guido posted a note to c.l.py on Monday. I believe that meets your notification criteria. Cheers, -g -- Greg Stein, http://www.lyra.org/

Greg Stein wrote:
Note that Guido posted a note to c.l.py on Monday. I believe that meets your notification criteria.
ahem. do you seriously believe that everyone in the Python universe reads comp.lang.python? afaik, most Python programmers don't. ... so as far as I'm concerned, this was officially deprecated with Guido's post. afaik, no official python documentation has explicitly mentioned this (and the fact that it doesn't explicitly allow it doesn't really matter, since the docs don't explicitly allow the x[a, b, c] syntax either. both work in 1.5.2). has anyone checked the recent crop of Python books, btw? the eff-bot guide uses old syntax in two examples out of 320. how about the others? ... sigh. running checkappend over a 50k LOC application, I just realized that it doesn't catch a very common append pydiom. how fun. even though 99% of all append calls are "legal", this "minor" change will break every single application and library we have :-( oh, wait. xmlrpclib isn't affected. always something! </F>

On Wed, 1 Mar 2000, Fredrik Lundh wrote:
Now you're simply taking my comments out of context. Not a proper thing to do. Ken said that he wanted notification along certain guidelines. I said that I believed Guido's post did just that. Period. Personally, I think it is fine. I also think that a CHANGES file that arrives with 1.6 that points out the incompatibility is also fine.
And which is that? Care to help out? Maybe just a little bit? Or do you just want to talk about how bad this change is? :-( Cheers, -g -- Greg Stein, http://www.lyra.org/

Greg Stein <gstein@lyra.org> wrote:
my point was that most Python programmers won't see that notification. when these people download 1.6 final and find that all theirs apps just broke, they probably won't be happy with a pointer to dejanews.
And which is that? Care to help out? Maybe just a little bit?
this rather common pydiom: append = list.append for x in something: append(...) it's used a lot where performance matters.
Or do you just want to talk about how bad this change is? :-(
yes, I think it's bad. I've been using Python since 1.2, and no other change has had the same consequences (wrt. time/money required to fix it) call me a crappy programmer if you want, but I'm sure there are others out there who are nearly as bad. and lots of them won't be aware of this change until some- one upgrades the python interpreter on their server. </F>
participants (15)
-
Andrew M. Kuchling
-
Christian Tismer
-
David Ascher
-
Fredrik Lundh
-
Fredrik Lundh
-
Greg Stein
-
Guido van Rossum
-
Ka-Ping Yee
-
Ken Manheimer
-
M.-A. Lemburg
-
Mark Hammond
-
Moshe Zadka
-
Paul Prescod
-
Skip Montanaro
-
Tim Peters