PEP 303: Extend divmod() for Multiple Divisors

I humbly submit this PEP for your dissection. As I am not subscribed to python-dev, please make sure that your comments are CC:ed to <bellman+pep-divmod@lysator.liu.se>, so I can see them. I have also posted this to comp.lang.python. I have written an implementation also, but I need to check it some more to see if I've got the reference counting correct before I dare post it. :-) PEP: 303 Title: Extend divmod() for Multiple Divisors Version: $Revision: 1.2 $ Last-Modified: $Date: 2002/12/31 16:02:49 $ Author: Thomas Bellman <bellman+pep-divmod@lysator.liu.se> Status: Draft Type: Standards Track Content-Type: text/plain Created: 31-Dec-2002 Python-Version: 2.3 Post-History: Abstract This PEP describes an extension to the built-in divmod() function, allowing it to take multiple divisors, chaining several calls to divmod() into one. Specification The built-in divmod() function would be changed to accept multiple divisors, changing its signature from divmod(dividend, divisor) to divmod(dividend, *divisors). The dividend is divided by the last divisor, giving a quotient and a remainder. The quotient is then divided by the second to last divisor, giving a new quotient and remainder. This is repeated until all divisors have been used, and divmod() then returns a tuple consisting of the quotient from the last step, and the remainders from all the steps. A Python implementation of the new divmod() behaviour could look like: def divmod(dividend, *divisors): modulos = () q = dividend while divisors: q,r = q.__divmod__(divisors[-1]) modulos = (r,) + modulos divisors = divisors[:-1] return (q,) + modulos Motivation Occasionally one wants to perform a chain of divmod() operations, calling divmod() on the quotient from the previous step, with varying divisors. The most common case is probably converting a number of seconds into weeks, days, hours, minutes and seconds. This would today be written as: def secs_to_wdhms(seconds): m,s = divmod(seconds, 60) h,m = divmod(m, 60) d,h = divmod(h, 24) w,d = divmod(d, 7) return (w,d,h,m,s) This is tedious and easy to get wrong each time you need it. If instead the divmod() built-in is changed according the proposal, the code for converting seconds to weeks, days, hours, minutes and seconds then become def secs_to_wdhms(seconds): w,d,h,m,s = divmod(seconds, 7, 24, 60, 60) return (w,d,h,m,s) which is easier to type, easier to type correctly, and easier to read. Other applications are: - Astronomical angles (declination is measured in degrees, minutes and seconds, right ascension is measured in hours, minutes and seconds). - Old British currency (1 pound = 20 shilling, 1 shilling = 12 pence) - Anglo-Saxon length units: 1 mile = 1760 yards, 1 yard = 3 feet, 1 foot = 12 inches. - Anglo-Saxon weight units: 1 long ton = 160 stone, 1 stone = 14 pounds, 1 pound = 16 ounce, 1 ounce = 16 dram - British volumes: 1 gallon = 4 quart, 1 quart = 2 pint, 1 pint = 20 fluid ounces Rationale The idea comes from APL, which has an operator that does this. (I don't remember what the operator looks like, and it would probably be impossible to render in ASCII anyway.) The APL operator takes a list as its second operand, while this PEP proposes that each divisor should be a separate argument to the divmod() function. This is mainly because it is expected that the most common uses will have the divisors as constants right in the call (as the 7, 24, 60, 60 above), and adding a set of parentheses or brackets would just clutter the call. Requiring an explicit sequence as the second argument to divmod() would seriously break backwards compatibility. Making divmod() check its second argument for being a sequence is deemed to be too ugly to contemplate. And in the case where one *does* have a sequence that is computed other-where, it is easy enough to write divmod(x, *divs) instead. Requiring at least one divisor, i.e rejecting divmod(x), has been considered, but no good reason to do so has come to mind, and is thus allowed in the name of generality. Calling divmod() with no divisors should still return a tuple (of one element). Code that calls divmod() with a varying number of divisors, and thus gets a return value with an "unknown" number of elements, would otherwise have to special case that case. Code that *knows* it is calling divmod() with no divisors is considered to be too silly to warrant a special case. Processing the divisors in the other direction, i.e dividing with the first divisor first, instead of dividing with the last divisor first, has been considered. However, the result comes with the most significant part first and the least significant part last (think of the chained divmod as a way of splitting a number into "digits", with varying weights), and it is reasonable to specify the divisors (weights) in the same order as the result. The inverse operation: def inverse_divmod(seq, *factors): product = seq[0] for x,y in zip(factors, seq[1:]): product = product * x + y return product could also be useful. However, writing seconds = (((((w * 7) + d) * 24 + h) * 60 + m) * 60 + s) is less cumbersome both to write and to read than the chained divmods. It is therefore deemed to be less important, and its introduction can be deferred to its own PEP. Also, such a function needs a good name, and the PEP author has not managed to come up with one yet. Calling divmod("spam") does not raise an error, despite strings supporting neither division nor modulo. However, unless we know the other object too, we can't determine whether divmod() would work or not, and thus it seems silly to forbid it. Backwards Compatibility Any module that replaces the divmod() function in the __builtin__ module, may cause other modules using the new syntax to break. It is expected that this is very uncommon. Code that expects a TypeError exception when calling divmod() with anything but two arguments will break. This is also expected to be very uncommon. No other issues regarding backwards compatibility are known. Reference Implementation Not finished yet, but it seems a rather straightforward new implementation of the function builtin_divmod() in Python/bltinmodule.c Copyright This document has been placed in the public domain. -- Thomas Bellman, Lysator Computer Club, Linköping University, Sweden "Adde parvum parvo magnus acervus erit" ! bellman @ lysator.liu.se (From The Mythical Man-Month) ! Make Love -- Nicht Wahr!

On the one hand I find this a nice backwards-compatible extension of divmod(). On the other hand I've coded this use case with separate divmod() calls and never had any trouble getting it right. So I think this doesn't address a real need, and it conflicts with the need to avoid unnecessary growth of the language. Explaining it (especially why it works backward) takes some effort, and I don't want to do that to all the book authors etc. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I totally agree. This nice extension to divmod() turns the function into something more polynomial algebra related, and divmod is no longer the proper name of the function. Also, the specialized use of it makes it more suitable to be put into a general algebra module for Python, for sure not into the builtins. Having that said, let's have a closer look at divmod. It appears in 36 .py files out of about 1500 .py files in the Python 2.2 source distribution. This is little more than 2 percent. The effort to correctly remember the order of the arguments and the results tuple does not suffice to justify the existance of this function at all. Furthermore, divmod does not provide any functionality that isn't easily obtained by a simple div and mod. Finally, and for me this is the killer argument for divmod: The standard use of divmod is splitting numbers over small integer bases. I can only see an advantage for divisions which come at high computational cost, which is factorizing polyomials or large numbers. But for the majority of applications, a quick time measurement of the function call overhead against doing an inline div and mod, I found that divmod compared to / and % is at least 50 percent *slower* to compute, due to the fact that a function call, followed by building and unpacking a result tuple is more expensive than the optimized interpreter code. The real computation is neglectible for small numbers, compared to the overhead of the engine. Therefore, I consider divmod a waste of code and a function to be deprecated ASAP (and since years). Save brain cells, save computation time, and save paper and ink of the book writers by dropping divmod! Vice versa, if at all, I suggest a built in /% operator as a faster equivalen to single / and %, but I believe this code can be used more efficiently. divmod should be a special case of an algebraic extension module and removed from the builtins. Python has got a lot of extensions, modelled towards more user-friendlyness. I do think that this goal can be achieved not only by extending, but also by deprecating functions which have little or no benefits over built-in operators. The existence of divmod gives the wrong feeling of optimizing one's code, which is a lie to the user. divmod does not save computation time, does not reduce programming time, does not simplify programs, eats paper and code space. divmod is also even not found in common compiled languages any longer. Even there, division isn't expensive enough to justify yet another function to remember and to implement. Please drop divmod. We will not miss it. from-__past__-import-divmod - ly y'rs -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Therefore, I consider divmod a waste of code and a function to be deprecated ASAP (and since years).
Maybe it wasn't such a good idea.
Save brain cells, save computation time, and save paper and ink of the book writers by dropping divmod!
At this point, any change causes waste, so unless it's truly broken (which I don't believe) I'm not for changing it.
That would have to be //% to be in line with the // operator, of course. But I'd rather spend the effort towards making the compiler smart enough to recognize that divmod is a built-in so it can generate more efficient code (and I *am* prepared to make the necessary -- slight -- changes to the language so that the compiler can make this deduction safely).
I find mm, ss = divmod(ss, 60) easier to read than mm, ss = ss//60, ss%60 --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote: ...
I stand corrected. And yes, catching certain builtins sounds like an optimzation path that is still open. Hey, len would have an incredible boost!!!
Sure it is. Until Silvester night, I even had no idea that divmod is so drastically slower. Hee hee. Which gave me a nice chance to start a whole rant towards shrinking the language, which I found an attractive new way to shake you up. Don't take it too serious :-)) I-wish-you-a-divmod-ly-new-2003-and-all-the-best -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Guido van Rossum wrote: ...
Before I retract completely, I have a small addition. mm = ss//60; ss %= 60 is very readable to me, very efficient, and doesn't use extra tuples. It is also 2 characters shorter than the divmod version (both with the usual spacing, of course). On the other hand, divmod clearly says what is going to happen, and in fact it is a higher level approach. On dropping features, I came to this in the morning: You said you might optimize certain builtins by the compiler, instead of removing divmod. How about this: If divmod is supported by the compiler, there isn't any reason to keep it as a compiled C module. Instead, divmod could be a piece of Python code, which is injected into builtins somewhere at startup time. Since he compiler supports it, it just needs to be there at all, but does not need to waste space in the Python executable. I'm not saying this just to fight divmod out of the language. This is ridiculous. But in the long term, there might be quite a lot of redundancy introduced by enhanced versions of the compiler. Instead of letting the code grow on and on, I'd like to propose some shrinking attempt like in this case. If the compiler eats 95 percent of calls to some code already, I believe it makes sense to replace that piece of code by Python code and drop the C version. The smaller the C code base, the better for Python. Less static code also improves the effect of compilers like Psyco. Let's make a smaller and better kernel. Get rid of ballast and have less code to maintain. This is a real wish and *no* Silvester joke :-) cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

A good optimizing compiler could also get rid of extra tuples.
It is also 2 characters shorter than the divmod version (both with the usual spacing, of course).
Shame on you. Would you rather write Perl? :-)
On the other hand, divmod clearly says what is going to happen, and in fact it is a higher level approach.
Yes.
But what about long division, where divmod can (in extreme cases) really save time by doing the division only once?
Let's first make some steps towards the better compiler I alluded to. Then we can start cutting.
This is a real wish and *no* Silvester joke :-)
If only I knew what Silvester was. Is it a German holiday celebrating the invention of root beer? :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote: ...
Yes. It could also figure out how to save one division if it sees the above line instead of divmod. But do you have it? ...
[proposal to replace divmod by Python code]
But what about long division, where divmod can (in extreme cases) really save time by doing the division only once?
Would this case be special-cased in the interpreter code? If not, the long divmod version would probably not be replaced. ...
Let's first make some steps towards the better compiler I alluded to. Then we can start cutting.
If this is a promise, I'm happy. Enhancing the compiler *and* cutting code really improves the ovrall quality.
Silvester is New Year's Eve; Hogmanay in Scotland. We have lots of celebration, drinks, and many fire works at midnight, to bomb the bad ghosts away for the new year. We also define new goals, things to do better. One of my goals is to enhance Python while reducing its C code base. Another one is to get the portable part of Stackless 3.0 into the Python core. cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

[Christian Tismer]
Therefore, I consider divmod a waste of code and a function to be deprecated ASAP (and since years).
[Guido]
Maybe it wasn't such a good idea.
You can remove divmod() when I'm dead. Before then, it stays. I'm sure all will agree that's a reasonable compromise.
On my box, divmod(x, y) is already 40% faster than "x//y; x%y" when x is a 63-bit int and y is 137354. In the hundreds of places I've used it in programs slinging multi-thousand bit integers, it's essentially twice as fast. But I don't care so much about that as that divmod() is the right conceptual spelling for the operation it performs. If we have to drop a builtin, I never liked reduce <wink>, although Jeremy pointed out that its most common use case no longer requires writing a lambda, or importing operator.add:
reduce(int.__add__, range(11)) 55
This is a little suprising if you toss a mix of types into it, though, since int.__add__ isn't operator.add:
reduce(int.__add__, [1, 2, 3.0]) NotImplemented
OTOH,
It's a good way to reverse-engineer the internals <wink>. "on-topic"-is-my-middle-name-ly y'rs - tim

"TP" == Tim Peters <tim.one@comcast.net> writes:
TP> If we have to drop a builtin, I never liked reduce <wink>, TP> although Jeremy pointed out that its most common use case no TP> longer requires writing a lambda, or importing operator.add: Hey, if we'll killing off builtins, I vote for apply(). -Barry

Raymond Hettinger wrote:
Hey, this is the positive response I was hoping for. You got the point. We do have functions which might perhaps not be dropped, but removed from C code. There is also a side effect for apply(), for instance: Controversely to divmod, apply is still found in 166 .py files of the 2.2 source dist. This is still more than 10 %, although apply() can be completely replaced by the new calling patterns. I believe that in the code base of the average user, this will look even worse, since nobody cares about changing working code. If we now quasi-deprecate apply by making it *slower*, simply by replacing the C code by a Python function that itself uses the new calling style, then we have less C code, still compatibility, *and* a good argument for everybody to replace apply by the new way to go. ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Raymond Hettinger wrote:
Hey, this is (or was) January 1st, not April 1st. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

M.-A. Lemburg wrote:
Right. On April 1st, the code size of Python will be fixed to a certain amount, and every additional C code must come with an equivalent amount of code to be dropped. Until then, starting with January 1st, you can get bonus code by proposing code obsoletion in advance. Later code reductions will be laid out in a PEP. The Python C code should be shrunk to 50% by end of 2004. The goal is to get it down to the pure bootstrap code of a JIT until 2007. This should be doable within at most 8 KB of binary code. -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Hey, if we'll killing off builtins, I vote for apply().
Agreed, it's redundant. You can help by checking in documentation that marks it as deprecated and code that adds a PendingDeprecationWarning to it (unfortunately it's so common that I wouldn't want to risk a plain DeprecationWarning). BTW, I recently find myself longing for an extension of the *args, **kw syntax, as follows: foo(1, 2, 3, *args, 5, 6, 7, **kw) ^^^^^^^ This part is currently not allowed. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido wrote:
Hey, if we'll killing off builtins, I vote for apply().
You've lost me here. I've recently written a piece of code that uses a lookup table on the name of a file to find the right function to apply to it; if I don't use apply for this, what should I use? An explicit case statement cannot be dynamically modified; using eval() requires a conversion to string (and is arguably even uglier than apply). Paul Hughett

Paul Hughett wrote:
Since a function is a first class callable object, you just pick it out of your lookup table func = look[key] and call it with the args and kwds which you got, using the new asterisk syntax: ret = func(*args, **kwds) ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

I'm +0 on this (reasons below), but since Guido already said no: I find this function pretty useful, as it serves to collapse ugly code, and I hate ugly python code. However, the name sucks. With your changes, 'divmod' doesn't describe it anymore. So I suggest you come up with a better name and offer it as a contribution to, say, the math module. In that case I would prefer to have the reverse function too. (The fact that using it with 2 args is equal to calling divmod is at most a funny fact. As Guido says, growing the library = good; growing the language = questionable) []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo@laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/

Lalo> I'm +0 on this (reasons below), but since Guido already said no: Lalo> I find this function pretty useful, as it serves to collapse Lalo> ugly code, and I hate ugly python code. However, the name Lalo> sucks. With your changes, 'divmod' doesn't describe it anymore. Lalo> So I suggest you come up with a better name and offer it as a Lalo> contribution to, say, the math module. In that case I would Lalo> prefer to have the reverse function too. If it helps, APL uses the names "base" and "represent" for these two functions. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark

Andrew Koenig wrote:
I thought it used "radix" and not "base"? Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff

On the one hand I find this a nice backwards-compatible extension of divmod(). On the other hand I've coded this use case with separate divmod() calls and never had any trouble getting it right. So I think this doesn't address a real need, and it conflicts with the need to avoid unnecessary growth of the language. Explaining it (especially why it works backward) takes some effort, and I don't want to do that to all the book authors etc. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I totally agree. This nice extension to divmod() turns the function into something more polynomial algebra related, and divmod is no longer the proper name of the function. Also, the specialized use of it makes it more suitable to be put into a general algebra module for Python, for sure not into the builtins. Having that said, let's have a closer look at divmod. It appears in 36 .py files out of about 1500 .py files in the Python 2.2 source distribution. This is little more than 2 percent. The effort to correctly remember the order of the arguments and the results tuple does not suffice to justify the existance of this function at all. Furthermore, divmod does not provide any functionality that isn't easily obtained by a simple div and mod. Finally, and for me this is the killer argument for divmod: The standard use of divmod is splitting numbers over small integer bases. I can only see an advantage for divisions which come at high computational cost, which is factorizing polyomials or large numbers. But for the majority of applications, a quick time measurement of the function call overhead against doing an inline div and mod, I found that divmod compared to / and % is at least 50 percent *slower* to compute, due to the fact that a function call, followed by building and unpacking a result tuple is more expensive than the optimized interpreter code. The real computation is neglectible for small numbers, compared to the overhead of the engine. Therefore, I consider divmod a waste of code and a function to be deprecated ASAP (and since years). Save brain cells, save computation time, and save paper and ink of the book writers by dropping divmod! Vice versa, if at all, I suggest a built in /% operator as a faster equivalen to single / and %, but I believe this code can be used more efficiently. divmod should be a special case of an algebraic extension module and removed from the builtins. Python has got a lot of extensions, modelled towards more user-friendlyness. I do think that this goal can be achieved not only by extending, but also by deprecating functions which have little or no benefits over built-in operators. The existence of divmod gives the wrong feeling of optimizing one's code, which is a lie to the user. divmod does not save computation time, does not reduce programming time, does not simplify programs, eats paper and code space. divmod is also even not found in common compiled languages any longer. Even there, division isn't expensive enough to justify yet another function to remember and to implement. Please drop divmod. We will not miss it. from-__past__-import-divmod - ly y'rs -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Therefore, I consider divmod a waste of code and a function to be deprecated ASAP (and since years).
Maybe it wasn't such a good idea.
Save brain cells, save computation time, and save paper and ink of the book writers by dropping divmod!
At this point, any change causes waste, so unless it's truly broken (which I don't believe) I'm not for changing it.
That would have to be //% to be in line with the // operator, of course. But I'd rather spend the effort towards making the compiler smart enough to recognize that divmod is a built-in so it can generate more efficient code (and I *am* prepared to make the necessary -- slight -- changes to the language so that the compiler can make this deduction safely).
I find mm, ss = divmod(ss, 60) easier to read than mm, ss = ss//60, ss%60 --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote: ...
I stand corrected. And yes, catching certain builtins sounds like an optimzation path that is still open. Hey, len would have an incredible boost!!!
Sure it is. Until Silvester night, I even had no idea that divmod is so drastically slower. Hee hee. Which gave me a nice chance to start a whole rant towards shrinking the language, which I found an attractive new way to shake you up. Don't take it too serious :-)) I-wish-you-a-divmod-ly-new-2003-and-all-the-best -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Guido van Rossum wrote: ...
Before I retract completely, I have a small addition. mm = ss//60; ss %= 60 is very readable to me, very efficient, and doesn't use extra tuples. It is also 2 characters shorter than the divmod version (both with the usual spacing, of course). On the other hand, divmod clearly says what is going to happen, and in fact it is a higher level approach. On dropping features, I came to this in the morning: You said you might optimize certain builtins by the compiler, instead of removing divmod. How about this: If divmod is supported by the compiler, there isn't any reason to keep it as a compiled C module. Instead, divmod could be a piece of Python code, which is injected into builtins somewhere at startup time. Since he compiler supports it, it just needs to be there at all, but does not need to waste space in the Python executable. I'm not saying this just to fight divmod out of the language. This is ridiculous. But in the long term, there might be quite a lot of redundancy introduced by enhanced versions of the compiler. Instead of letting the code grow on and on, I'd like to propose some shrinking attempt like in this case. If the compiler eats 95 percent of calls to some code already, I believe it makes sense to replace that piece of code by Python code and drop the C version. The smaller the C code base, the better for Python. Less static code also improves the effect of compilers like Psyco. Let's make a smaller and better kernel. Get rid of ballast and have less code to maintain. This is a real wish and *no* Silvester joke :-) cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

A good optimizing compiler could also get rid of extra tuples.
It is also 2 characters shorter than the divmod version (both with the usual spacing, of course).
Shame on you. Would you rather write Perl? :-)
On the other hand, divmod clearly says what is going to happen, and in fact it is a higher level approach.
Yes.
But what about long division, where divmod can (in extreme cases) really save time by doing the division only once?
Let's first make some steps towards the better compiler I alluded to. Then we can start cutting.
This is a real wish and *no* Silvester joke :-)
If only I knew what Silvester was. Is it a German holiday celebrating the invention of root beer? :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote: ...
Yes. It could also figure out how to save one division if it sees the above line instead of divmod. But do you have it? ...
[proposal to replace divmod by Python code]
But what about long division, where divmod can (in extreme cases) really save time by doing the division only once?
Would this case be special-cased in the interpreter code? If not, the long divmod version would probably not be replaced. ...
Let's first make some steps towards the better compiler I alluded to. Then we can start cutting.
If this is a promise, I'm happy. Enhancing the compiler *and* cutting code really improves the ovrall quality.
Silvester is New Year's Eve; Hogmanay in Scotland. We have lots of celebration, drinks, and many fire works at midnight, to bomb the bad ghosts away for the new year. We also define new goals, things to do better. One of my goals is to enhance Python while reducing its C code base. Another one is to get the portable part of Stackless 3.0 into the Python core. cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

[Christian Tismer]
Therefore, I consider divmod a waste of code and a function to be deprecated ASAP (and since years).
[Guido]
Maybe it wasn't such a good idea.
You can remove divmod() when I'm dead. Before then, it stays. I'm sure all will agree that's a reasonable compromise.
On my box, divmod(x, y) is already 40% faster than "x//y; x%y" when x is a 63-bit int and y is 137354. In the hundreds of places I've used it in programs slinging multi-thousand bit integers, it's essentially twice as fast. But I don't care so much about that as that divmod() is the right conceptual spelling for the operation it performs. If we have to drop a builtin, I never liked reduce <wink>, although Jeremy pointed out that its most common use case no longer requires writing a lambda, or importing operator.add:
reduce(int.__add__, range(11)) 55
This is a little suprising if you toss a mix of types into it, though, since int.__add__ isn't operator.add:
reduce(int.__add__, [1, 2, 3.0]) NotImplemented
OTOH,
It's a good way to reverse-engineer the internals <wink>. "on-topic"-is-my-middle-name-ly y'rs - tim

"TP" == Tim Peters <tim.one@comcast.net> writes:
TP> If we have to drop a builtin, I never liked reduce <wink>, TP> although Jeremy pointed out that its most common use case no TP> longer requires writing a lambda, or importing operator.add: Hey, if we'll killing off builtins, I vote for apply(). -Barry

Raymond Hettinger wrote:
Hey, this is the positive response I was hoping for. You got the point. We do have functions which might perhaps not be dropped, but removed from C code. There is also a side effect for apply(), for instance: Controversely to divmod, apply is still found in 166 .py files of the 2.2 source dist. This is still more than 10 %, although apply() can be completely replaced by the new calling patterns. I believe that in the code base of the average user, this will look even worse, since nobody cares about changing working code. If we now quasi-deprecate apply by making it *slower*, simply by replacing the C code by a Python function that itself uses the new calling style, then we have less C code, still compatibility, *and* a good argument for everybody to replace apply by the new way to go. ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Raymond Hettinger wrote:
Hey, this is (or was) January 1st, not April 1st. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

M.-A. Lemburg wrote:
Right. On April 1st, the code size of Python will be fixed to a certain amount, and every additional C code must come with an equivalent amount of code to be dropped. Until then, starting with January 1st, you can get bonus code by proposing code obsoletion in advance. Later code reductions will be laid out in a PEP. The Python C code should be shrunk to 50% by end of 2004. The goal is to get it down to the pure bootstrap code of a JIT until 2007. This should be doable within at most 8 KB of binary code. -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

Hey, if we'll killing off builtins, I vote for apply().
Agreed, it's redundant. You can help by checking in documentation that marks it as deprecated and code that adds a PendingDeprecationWarning to it (unfortunately it's so common that I wouldn't want to risk a plain DeprecationWarning). BTW, I recently find myself longing for an extension of the *args, **kw syntax, as follows: foo(1, 2, 3, *args, 5, 6, 7, **kw) ^^^^^^^ This part is currently not allowed. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido wrote:
Hey, if we'll killing off builtins, I vote for apply().
You've lost me here. I've recently written a piece of code that uses a lookup table on the name of a file to find the right function to apply to it; if I don't use apply for this, what should I use? An explicit case statement cannot be dynamically modified; using eval() requires a conversion to string (and is arguably even uglier than apply). Paul Hughett

Paul Hughett wrote:
Since a function is a first class callable object, you just pick it out of your lookup table func = look[key] and call it with the args and kwds which you got, using the new asterisk syntax: ret = func(*args, **kwds) ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

I'm +0 on this (reasons below), but since Guido already said no: I find this function pretty useful, as it serves to collapse ugly code, and I hate ugly python code. However, the name sucks. With your changes, 'divmod' doesn't describe it anymore. So I suggest you come up with a better name and offer it as a contribution to, say, the math module. In that case I would prefer to have the reverse function too. (The fact that using it with 2 args is equal to calling divmod is at most a funny fact. As Guido says, growing the library = good; growing the language = questionable) []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo@laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/

Lalo> I'm +0 on this (reasons below), but since Guido already said no: Lalo> I find this function pretty useful, as it serves to collapse Lalo> ugly code, and I hate ugly python code. However, the name Lalo> sucks. With your changes, 'divmod' doesn't describe it anymore. Lalo> So I suggest you come up with a better name and offer it as a Lalo> contribution to, say, the math module. In that case I would Lalo> prefer to have the reverse function too. If it helps, APL uses the names "base" and "represent" for these two functions. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark

Andrew Koenig wrote:
I thought it used "radix" and not "base"? Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff
participants (12)
-
Alex Martelli
-
Andrew Koenig
-
barry@python.org
-
Ben Laurie
-
Christian Tismer
-
Guido van Rossum
-
Lalo Martins
-
M.-A. Lemburg
-
Paul Hughett
-
Raymond Hettinger
-
Thomas Bellman
-
Tim Peters