Mailman 3 July 2001 - Python-Dev

Ready to merge descr-branch back into the trunk?
by Guido van Rossum July 28, 2001

July 28, 2001

Is it time to merge the descr-branch (from which 2.2a1 was built) back into the trunk? In the fray over PEP 238 I haven't seen too much feedback on the alpha release, but there have been plenty of downloads. Telling from the bug reports, a few people have clearly been kicking the tires quite a bit. I don't think we'll have to withdraw the type/class unification, and I'd like to fire Tim from his branch-merge duties. :) I'll post a qury about this on c.l.py too. --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

ActiveCobolScript?
by Paul Prescod July 28, 2001

July 28, 2001

http://www.cobolscript.com/ Too bad this up and coming language already has a corporate benefactor. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook

1 1

Re: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
by Michael Hudson July 28, 2001

July 28, 2001

Not directly relavent to the PEP, but... Guido van Rossum <guido(a)zope.com> writes: > Q. What about code compiled by the codeop module? > > A. Alas, this will always use the default semantics (set by the -D > command line option). This is a general problem with the > future statement; PEP 236[4] lists it as an unresolved > problem. You could have your own clone of codeop.py that > includes a future division statement, but that's not a general > solution. Did you look at my Nasty Hack(tm) to bodge around this? It's at http://starship.python.net/crew/mwh/hacks/codeop-hack.diff if you haven't. I'm not sure it will work with what you're planning for division, but it works for generators (and worked for nested scopes when that was relavent). There are a host of saner ways round this, of course - like adding an optional "flags" argument to compile, for instance. Cheers, M. -- ARTHUR: Why should a rock hum? FORD: Maybe it feels good about being a rock. -- The Hitch-Hikers Guide to the Galaxy, Episode 8

2 1

[OT] Expat 1.95.2 released
by Fred L. Drake, Jr. July 28, 2001

July 28, 2001

Slightly off-topic, but this may be interesting to a few of you... In case anyone is interested, Expat 1.95.2 has been released, with both a source archive for Unix users and a handy installer for Windows victims (thanks to Tim Peters for getting me started!). This release fixes some small bugs and improves the portability of the build process (and there is one for Windows this time). You can pick up the 1.95.2 release at: http://sourceforge.net/projects/expat/ -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation _______________________________________________ XML-SIG maillist - XML-SIG(a)python.org http://mail.python.org/mailman/listinfo/xml-sig

1 0

Changing the Division Operator -- PEP 238, rev 1.12
by Guido van Rossum July 27, 2001

July 27, 2001

Here's a new revision of PEP 238. I've incorporated clarifications of issues that were brought up during the discussion of rev 1.10 -- from typos via rewording of ambiguous phrasing to the addition of new open issues. I've decided not to go for the "quotient and ratio" terminology -- my rationale is in the PEP. I'm posting this also to c.l.py and c.l.py.a, to make sure enough people see it. Feel free to discuss it either in c.l.py or here in python-dev, but please don't change the subject. --Guido van Rossum (home page: http://www.python.org/~guido/) PEP: 238 Title: Changing the Division Operator Version: $Revision: 1.12 $ Author: pep(a)zadka.site.co.il (Moshe Zadka), guido(a)python.org (Guido van Rossum) Status: Draft Type: Standards Track Created: 11-Mar-2001 Python-Version: 2.2 Post-History: 16-Mar-2001, 26-Jul-2001, 27-Jul-2001 Abstract The current division (/) operator has an ambiguous meaning for numerical arguments: it returns the floor of the mathematical result of division if the arguments are ints or longs, but it returns a reasonable approximation of the division result if the arguments are floats or complex. This makes expressions expecting float or complex results error-prone when integers are not expected but possible as inputs. We propose to fix this by introducing different operators for different operations: x/y to return a reasonable approximation of the mathematical result of the division ("true division"), x//y to return the floor ("floor division"). We call the current, mixed meaning of x/y "classic division". Because of severe backwards compatibility issues, not to mention a major flamewar on c.l.py, we propose the following transitional measures (starting with Python 2.2): - Classic division will remain the default in the Python 2.x series; true division will be standard in Python 3.0. - The // operator will be available to request floor division unambiguously. - The future division statement, spelled "from __future__ import division", will change the / operator to mean true division throughout the module. - A command line option will enable run-time warnings for classic division applied to int or long arguments; another command line option will make true division the default. - The standard library will use the future division statement and the // operator when appropriate, so as to completely avoid classic division. Motivation The classic division operator makes it hard to write numerical expressions that are supposed to give correct results from arbitrary numerical inputs. For all other operators, one can write down a formula such as x*y**2 + z, and the calculated result will be close to the mathematical result (within the limits of numerical accuracy, of course) for any numerical input type (int, long, float, or complex). But division poses a problem: if the expressions for both arguments happen to have an integral type, it implements floor division rather than true division. The problem is unique to dynamically typed languages: in a statically typed language like C, the inputs, typically function arguments, would be declared as double or float, and when a call passes an integer argument, it is converted to double or float at the time of the call. Python doesn't have argument type declarations, so integer arguments can easily find their way into an expression. The problem is particularly pernicious since ints are perfect substitutes for floats in all other circumstances: math.sqrt(2) returns the same value as math.sqrt(2.0), 3.14*100 and 3.14*100.0 return the same value, and so on. Thus, the author of a numerical routine may only use floating point numbers to test his code, and believe that it works correctly, and a user may accidentally pass in an integer input value and get incorrect results. Another way to look at this is that classic division makes it difficult to write polymorphic functions that work well with either float or int arguments; all other operators already do the right thing. No algorithm that works for both ints and floats has a need for truncating division in one case and true division in the other. The correct work-around is subtle: casting an argument to float() is wrong if it could be a complex number; adding 0.0 to an argument doesn't preserve the sign of the argument if it was minus zero. The only solution without either downside is multiplying an argument (typically the first) by 1.0. This leaves the value and sign unchanged for float and complex, and turns int and long into a float with the corresponding value. It is the opinion of the authors that this is a real design bug in Python, and that it should be fixed sooner rather than later. Assuming Python usage will continue to grow, the cost of leaving this bug in the language will eventually outweigh the cost of fixing old code -- there is an upper bound to the amount of code to be fixed, but the amount of code that might be affected by the bug in the future is unbounded. Another reason for this change is the desire to ultimately unify Python's numeric model. This is the subject of PEP 228[0] (which is currently incomplete). A unified numeric model removes most of the user's need to be aware of different numerical types. This is good for beginners, but also takes away concerns about different numeric behavior for advanced programmers. (Of course, it won't remove concerns about numerical stability and accuracy.) In a unified numeric model, the different types (int, long, float, complex, and possibly others, such as a new rational type) serve mostly as storage optimizations, and to some extent to indicate orthogonal properties such as inexactness or complexity. In a unified model, the integer 1 should be indistinguishable from the floating point number 1.0 (except for its inexactness), and both should behave the same in all numeric contexts. Clearly, in a unified numeric model, if a==b and c==d, a/c should equal b/d (taking some liberties due to rounding for inexact numbers), and since everybody agrees that 1.0/2.0 equals 0.5, 1/2 should also equal 0.5. Likewise, since 1//2 equals zero, 1.0//2.0 should also equal zero. Variations Aesthetically, x//y doesn't please everyone, and hence several variations have been proposed: x div y, or div(x, y), sometimes in combination with x mod y or mod(x, y) as an alternative spelling for x%y. We consider these solutions inferior, on the following grounds. - Using x div y would introduce a new keyword. Since div is a popular identifier, this would break a fair amount of existing code, unless the new keyword was only recognized under a future division statement. Since it is expected that the majority of code that needs to be converted is dividing integers, this would greatly increase the need for the future division statement. Even with a future statement, the general sentiment against adding new keywords unless absolutely necessary argues against this. - Using div(x, y) makes the conversion of old code much harder. Replacing x/y with x//y or x div y can be done with a simple query replace; in most cases the programmer can easily verify that a particular module only works with integers so all occurrences of x/y can be replaced. (The query replace is still needed to weed out slashes occurring in comments or string literals.) Replacing x/y with div(x, y) would require a much more intelligent tool, since the extent of the expressions to the left and right of the / must be analyzed before the placement of the "div(" and ")" part can be decided. Alternatives In order to reduce the amount of old code that needs to be converted, several alternative proposals have been put forth. Here is a brief discussion of each proposal (or category of proposals). If you know of an alternative that was discussed on c.l.py that isn't mentioned here, please mail the second author. - Let / keep its classic semantics; introduce // for true division. This still leaves a broken operator in the language, and invites to use the broken behavior. It also shuts off the road to a unified numeric model a la PEP 228[0]. - Let int division return a special "portmanteau" type that behaves as an integer in integer context, but like a float in a float context. The problem with this is that after a few operations, the int and the float value could be miles apart, it's unclear which value should be used in comparisons, and of course many contexts (like conversion to string) don't have a clear integer or float context. - Use a directive to use specific division semantics in a module, rather than a future statement. This retains classic division as a permanent wart in the language, requiring future generations of Python programmers to be aware of the problem and the remedies. - Use "from __past__ import division" to use classic division semantics in a module. This also retains the classic division as a permanent wart, or at least for a long time (eventually the past division statement could raise an ImportError). - Use a directive (or some other way) to specify the Python version for which a specific piece of code was developed. This requires future Python interpreters to be able to emulate *exactly* several previous versions of Python, and moreover to do so for multiple versions within the same interpreter. This is way too much work. A much simpler solution is to keep multiple interpreters installed. API Changes During the transitional phase, we have to support *three* division operators within the same program: classic division (for / in modules without a future division statement), true division (for / in modules with a future division statement), and floor division (for //). Each operator comes in two flavors: regular, and as an augmented assignment operator (/= or //=). The names associated with these variations are: - Overloaded operator methods: __div__(), __floordiv__(), __truediv__(); __idiv__(), __ifloordiv__(), __itruediv__(). - Abstract API C functions: PyNumber_Divide(), PyNumber_FloorDivide(), PyNumber_TrueDivide(); PyNumber_InPlaceDivide(), PyNumber_InPlaceFloorDivide(), PyNumber_InPlaceTrueDivide(). - Byte code opcodes: BINARY_DIVIDE, BINARY_FLOOR_DIVIDE, BINARY_TRUE_DIVIDE; INPLACE_DIVIDE, INPLACE_FLOOR_DIVIDE, INPLACE_TRUE_DIVIDE. - PyNumberMethod slots: nb_divide, nb_floor_divide, nb_true_divide, nb_inplace_divide, nb_inplace_floor_divide, nb_inplace_true_divide. The added PyNumberMethod slots require an additional flag in tp_flags; this flag will be named Py_TPFLAGS_HAVE_NEWDIVIDE and will be included in Py_TPFLAGS_DEFAULT. The true and floor division APIs will look for the corresponding slots and call that; when that slot is NULL, they will raise an exception. There is no fallback to the classic divide slot. In Python 3.0, the classic division semantics will be removed; the classic division APIs will become synonymous with true division. Command Line Option The -D command line option takes a string argument that can take three values: "old", "warn", or "new". The default is "old" in Python 2.2 but will change to "warn" in later 2.x versions. The "old" value means the classic division operator acts as described. The "warn" value means the classic division operator issues a warning (a DeprecationWarning using the standard warning framework) when applied to ints or longs. The "new" value changes the default globally so that the / operator is always interpreted as true division. The "new" option is only intended for use in certain educational environments, where true division is required, but asking the students to include the future division statement in all their code would be a problem. This option will not be supported in Python 3.0; Python 3.0 will always interpret / as true division. (Other names have been proposed, like -Dclassic, -Dclassic-warn, -Dtrue, or -Dold_division etc.; these seem more verbose to me without much advantage. After all the term classic division is not used in the language at all (only in the PEP), and the term true division is rarely used in the language -- only in __truediv__.) Semantics of Floor Division Floor division will be implemented in all the Python numeric types, and will have the semantics of a // b == floor(a/b) except that the result type will be the common type into which a and b are coerced before the operation. Specifically, if a and b are of the same type, a//b will be of that type too. If the inputs are of different types, they are first coerced to a common type using the same rules used for all other arithmetic operators. In particular, if a and b are both ints or longs, the result has the same type and value as for classic division on these types (including the case of mixed input types; int//long and long//int will both return a long). For floating point inputs, the result is a float. For example: 3.5//2.0 == 1.0 For complex numbers, // raises an exception, since float() of a complex number is not allowed. For user-defined classes and extension types, all semantics are up to the implementation of the class or type. Semantics of True Division True division for ints and longs will convert the arguments to float and then apply a float division. That is, even 2/1 will return a float (2.0), not an int. For floats and complex, it will be the same as classic division. Note that for long arguments, true division may lose information; this is in the nature of true division (as long as rationals are not in the language). Algorithms that consciously use longs should consider using //. If and when a rational type is added to Python (see PEP 239[2]), true division for ints and longs should probably return a rational. This avoids the problem with true division of longs losing information. But until then, for consistency, float is the only choice for true division. The Future Division Statement If "from __future__ import division" is present in a module, or if -Dnew is used, the / and /= operators are translated to true division opcodes; otherwise they are translated to classic division (until Python 3.0 comes along, where they are always translated to true division). The future division statement has no effect on the recognition or translation of // and //=. See PEP 236[4] for the general rules for future statements. (It has been proposed to use a longer phrase, like "true_division" or "modern_division". These don't seem to add much information.) Open Issues - It has been proposed to call // the quotient operator, and the / operator the ratio operator. I'm not sure about this -- for some people quotient is just a synonym for division, and ratio suggests rational numbers, which is wrong. I prefer the terminology to be slightly awkward if that avoids unambiguity. Also, for some folks "quotient" suggests truncation towards zero, not towards infinity as "floor division" says explicitly. - It has been argued that a command line option to change the default is evil. It can certainly be dangerous in the wrong hands: for example, it would be impossible to combine a 3rd party library package that requires -Dnew with another one that requires -Dold. But I believe that the VPython folks need a way to enable true division by default, and other educators might need the same. These usually have enough control over the library packages available in their environment. - For very large long integers, the definition of true division as returning a float causes problems, since the range of Python longs is much larger than that of Python floats. This problem will disappear if and when rational numbers are supported. In the interim, maybe the long-to-float conversion could be made to raise OverflowError if the long is out of range. FAQ Q. Why isn't true division called float division? A. Because I want to keep the door open to *possibly* introducing rationals and making 1/2 return a rational rather than a float. See PEP 239[2]. Q. Why is there a need for __truediv__ and __itruediv__? A. We don't want to make user-defined classes second-class citizens. Certainly not with the type/class unification going on. Q. How do I write code that works under the classic rules as well as under the new rules without using // or a future division statement? A. Use x*1.0/y for true division, divmod(x, y)[0] for int division. Especially the latter is best hidden inside a function. You may also write float(x)/y for true division if you are sure that you don't expect complex numbers. If you know your integers are never negative, you can use int(x/y) -- while the documentation of int() says that int() can round or truncate depending on the C implementation, we know of no C implementation that doesn't truncate, and we're going to change the spec for int() to promise truncation. Note that for negative ints, classic division (and floor division) round towards negative infinity, while int() rounds towards zero. Q. How do I specify the division semantics for input(), compile(), execfile(), eval() and exec? A. They inherit the choice from the invoking module. PEP 236[4] lists this as a partially resolved problem. Q. What about code compiled by the codeop module? A. Alas, this will always use the default semantics (set by the -D command line option). This is a general problem with the future statement; PEP 236[4] lists it as an unresolved problem. You could have your own clone of codeop.py that includes a future division statement, but that's not a general solution. Q. Will there be conversion tools or aids? A. Certainly, but these are outside the scope of the PEP. Q. Why is my question not answered here? A. Because we weren't aware of it. If it's been discussed on c.l.py and you believe the answer is of general interest, please notify the second author. (We don't have the time or inclination to answer every question sent in private email, hence the requirement that it be discussed on c.l.py first.) Implementation A very early implementation (not yet following the above spec, but supporting // and the future division statement) is available from the SourceForge patch manager[5]. References [0] PEP 228, Reworking Python's Numeric Model http://www.python.org/peps/pep-0228.html [1] PEP 237, Unifying Long Integers and Integers, Zadka, http://www.python.org/peps/pep-0237.html [2] PEP 239, Adding a Rational Type to Python, Zadka, http://www.python.org/peps/pep-0239.html [3] PEP 240, Adding a Rational Literal to Python, Zadka, http://www.python.org/peps/pep-0240.html [4] PEP 236, Back to the __future__, Peters, http://www.python.org/peps/pep-0236.html [5] Patch 443474, from __future__ import division http://sourceforge.net/tracker/index.php?func=detail&aid=443474&group_id=54… Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End:

2 1

dpython interaction with imaginary types
by Michael McLay July 27, 2001

July 27, 2001

Paul F. Dubois writes: > In dpython, what is 2.0j? Is the "standard" way of writing complex numbers, > 3.0 + 2.0j, valid? If this expression is placed in a module with a '.py' extension it will work exactly as it does today. An exception will be raised if the expression is in a '.dp' module because the 3.0 would be a decimal number and the complex type is expecting a binary number for the real portiion.

1 0

Re: [Python-Dev] PEP for adding a decimal type to Python
by Michael McLay July 27, 2001

July 27, 2001

On Friday 27 July 2001 12:57 pm, Guido van Rossum wrote: > Michael, > > Your PEP doesn't spell out what happens when a binary and a decimal > number are the input for a numerical operator. I believe you said > that this would be an unconditional error. > > But I foresee serious problems. Most standard library modules use > numbers. Most of the modules using numbers occasionally use a literal > (e.g. 0 or 1). According to your PEP, literals in module files ending > with .py default to binary. This means that almost any use of a > standard library module from your "dpython" will fail as soon as a > literal is used. No, because the '.py' file will generate bytecodes for a number literals as binary number when the module is compiled. If a '.dp' file imports the contents of a '.py' file the binary numbers will be imported as binary numbers. If the '.dp' file will need to use the binary number in a calculation with a decimal number the binary number will have to be cast it to a decimal number. --------------------- #gui.py BLUE = 155 x_axis = 1024 y_axis = 768 -------------------- #calculator.dp import gui ytd_interest = 0.04 # ytd_interest is now a decimal number win = gui.open_window(gui.bg, x_size=gui.x_axis, y_size=gui.y_axis) app = win.dialog("Bank Balance", bankbalance_callback) bb = app.get_bankbalance() # bb now contains a string newbalance = decimal(bb) *ytd_interest # now update the display app.set_bankbalance(str(newbalance)) ------------------- In the example the gui module was used in the calculator module, but they were alway handled as binary numbers. The parser did not convert them to decimal numbers because they had been parsed into a gui.pyc file prior to being loaded into calculator.dp. > I can't believe that this will work satisfactorily. I think it will. There will be some cases where it might be necessary to add modules of convenience functions to make it easier to to use applications that cross boundaries, but I think these cases will be rare. Immediately following the introduction of the decimal number types all binary modules will work as the work today. There will be no additional pain to continue using those module. There will be no decimal modules, so there is no problem with making them work with the binary modules. As decimal module users start developing applications they will develop techniques for working with the binary modules. Initially it may require a significant effort, but eventually bondaries will be created and they two domains will coexists. > Another example of the kind of problem your approach runs into: what > should the type of len("abc") be? 3d or 3b? Should it depend on the > default mode? That is an interesting question. With my current proposal the following would be required: stlen = decimal(len("abc")) A dlen() function could be added, or perhaps allowing the automatic promotion of int to a decimal would be a reasonable exception. That is one case were there is no chance of data loss. I'm not apposed to automatic conversions if there is no danger of errors being introduced. > I suppose sequence indexing has to accept decimal as well as binary > integers as indexes -- certainly in a decimal program you will want to > be able to use decimal integers for indexes. That is how I would expect it to work.

1 0

Re: [Python-Dev] PEP for adding a decimal type to Python
by Michael McLay July 27, 2001

July 27, 2001

On Thursday 26 July 2001 04:52 pm, Tim Peters wrote: > > [1] ANSI standard X3.274-1996. > > (See http://www2.hursley.ibm.com/decimal/deccode.html) > > Michael, this is merely a standard for *encoding* decimal numbers; it > doesn't say anything about semantics, or exceptions, or anything else > visible to users. This was a proposal for a mechanism for mingling types safely. It was not intended as a definition of how decimal numbers should be implemented. My implementation tests the interaction of the current number types with the decimal type and I only completed enought of the decimal type implementation to support this testing. I was not expecting to discuss how decimal types should work. That has been discussed already. I was primarily interested in testing the effects of adding a new number type as I described in the PEP. What did you think of the idea of adding a new command and file format? > Are you aware that Aahz is implementing "the real" spec for Python, a level > up at > > http://www2.hursley.ibm.com/decimal/ > > under "Base specification"? There are so few people working on the decimal > idea that I hate to see it fragmented already. Yes I have played with the Decimal.py module. I developed decimalobject.c so I could test the inpact of introducing an additional command and file format to Python. I expect this code to be replaced. As I said in the PEP I also think the decimal number implementation will evolve into a type that supports inheritance.

5 6

Re: [Python-Dev] PEP for adding a decimal type to Python
by Paul F. Dubois July 27, 2001

July 27, 2001

In dpython, what is 2.0j? Is the "standard" way of writing complex numbers, 3.0 + 2.0j, valid?

1 0

Future division patch available (PEP 238)
by Guido van Rossum July 27, 2001

July 27, 2001

For those interested in the future of the division operator a la PEP 238, I've produced a reasonably complete patch (relative to the CVS trunk, but it probably also works for the descr-branch or the 2.2a1 release). Get it here: http://sourceforge.net/tracker/index.php?func=detail&aid=443474&group_id=54… It works as follows: - unconditionally, there's a new operator // that will always do int division (and an in-place companion //=). - by default, / is unchanged (and so is /=). - after "from __future__ import division", / is changed to return a float result from int or long operands (and so is /=). Read the patch description for more details; the implementation of int and float division are semi-lame. There's no warning yet for int division returning a truncated result; I'm not sure if I want such a warning to be part of 2.2 (maybe if it's off by default). I'm cc'ing Bruce Sherwood and Davin Scherer, because they asked for this and used a similar implementation in VPython. When this patch (or something not entirely unlike it) is accepted into Python 2.2, they will no longer have to maintain their own hacked Python. (We've already added 10**-15 returning a float to 2.2a1, also specifically for them; that was easier because it used to be an error, so no backwards compatibility code or future statement is necessary there.) I thought again about the merits of the '//' operator vs. 'div' (either as a function or as a keyword binary operator), and figured that '//' is the best choice: it doesn't introduce a new keyword (which would cause more pain), and it works as an augmented assignment (//=) as well. --Guido van Rossum (home page: http://www.python.org/~guido/)

4 3