Mailman 3 October 2000 - Python-Dev

Python 2.0b2 note for Windows developers
by Tim Peters 15 Oct '00

15 Oct '00

Since most Python users on Windows don't have any use for them, I trimmed the Python 2.0b2 installer by leaving out the debug-build .lib, .pyd, .exe and .dll files. If you want them, they're available in a separate zip archive; read the Windows Users notes at http://www.pythonlabs.com/products/python2.0/download_python2.0b2.html for info and a download link. If you don't already know *why* you might want them, trust me: you don't want them <wink>. they-don't-even-make-good-paperweights-ly y'rs - tim

3 5

Simple xml.sax example?
by Skip Montanaro 15 Oct '00

15 Oct '00

Is there a simple example available that demonstrates how to use elements of the new xml.sax package? I'd prefer something that doesn't require me to install anything other than what comes with the Python 2.0 distribution? Thanks, Skip

3 3

distutils extra sources
by Martin v. Loewis 15 Oct '00

15 Oct '00

> Can you recommend how to include this in the source distribution for > the final release? Even though I was not asked, I still answer :-) I think having the source code for the installer is essential, just so that people won't get frightened by the potential for a Trojan Horse in front of the gates (or, rather, the windows?-) I'd put the installer (i.e. all of distutils/misc except for get_metadata.py) into Tools/wininst. (I'd actually preferred to see the wininst sources in distutils in a different directory as well). In case you need some descripting text as well, here's my proposal. In Tools/README, maybe it would be wininst Sources for Windows installer of distutils.command.bdist_wininst. For Tools/wininst/README, see the text below. One complication is that invoking bdist_wininst as a program will regenerate the module, but it expects the installer binary in ../../misc/wininst.exe; I'm not sure whether it is worth the effort to change that for the Python distribution. Regards, Martin THE WINDOWS INSTALLER ===================== The distutils support the command bdist_wininst, which will produce a windows auto-installing executable using the information in the setup function. The executable consists of a user interface program, concatenated with a zip archive of the distributed files. This directory contains the sources of the user interface program. The bdist_wininst command itself does not need the sources, instead, it has an embedded copy of the installer binary file. If the installer program is modified, bdist_wininst.py must be updated by invoking it as a program (i.e. "python bdist_wininst.py") Building the installer ---------------------- To build the installer, you need a copy of Microsoft Visual C++ version 6; the project file is wininst.dsp. You may also use the executable compressor UPX, which is available from http://upx.tsx.org. UPX must be installed into c:\util. Using UPX is not strictly required - it just reduces the size of the installation program itself. Using the bdist_wininst command ------------------------------- To create an binary distribution of your package for Windows, simply invoke "python setup.py bdist_wininst" in your package directory. If your package does not contain C modules, you may invoke this command on any system; if it does contain C modules, bdist_wininst will need the C compiler used to build Python (typically MSVC++). It will not need MSVC++ or UPX to build the setup program. To build the ZIP file contained in the installer, bdist_wininst will first try an external zip program (zip.exe) that understands the -rq option, such as Info-ZIP (http://www.info-zip.org/pub/infozip/Zip.html). If no such program is found, it will next try the zipfile module, which is available if Python was build with zlib support.

1 0

Mandrake vs glibc
by Martin v. Loewis 15 Oct '00

15 Oct '00

> So, if David's report is correct, and best I understand it you were > all using 7.1 too with at least some level of optimization, it's A > Mystery why CVS test_math.py succeeds on that system (its new > math.sqrt(-1) test would probably fail; its new math.exp(+huge) test > would almost certainly fail). I'm anal enough about this stuff that > I'd try to figure out why if I had a Mandrake system, but in the > cosmic scheme of things I doubt it's important enough for anyone > else to burn time on. Well, I don't know about Mandrake, but I have glibc 2.1.94 here, and I get Python 2.0 (#104, Oct 15 2000, 15:47:19) [GCC 2.95.2 19991024 (release)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import math >>> math.sqrt(-1) Traceback (most recent call last): File "<stdin>", line 1, in ? OverflowError: math range error >>> math.exp(800) inf >>> math.exp(-800) 0.0 It turns that Tcl8.3 and Tk8.3, on my system, link with -lieee. That is implemented as _LIB_VERSION_TYPE _LIB_VERSION = _IEEE_; in this version of glibc, which in turn changes the behaviour of double __exp(double x) /* wrapper exp */ { double z; z = __ieee754_exp(x); if(_LIB_VERSION == _IEEE_) return z; if(__finite(x)) { if(x>o_threshold) return __kernel_standard(x,x,6); /* exp overflow */ else if(x<u_threshold) return __kernel_standard(x,x,7); /* exp underflow */ } return z; } As a rationale for linking with -lieee, the Tcl/Tk configure.in offers # Also, Linux requires the "ieee" library for math to # work right (and it must appear before "-lm"). Without testing, it seems that this would give some interesting consequences on installations where Tkinter is a shared module: the outcome of these functions could depend on whether Tkinter is imported or not. I can't produce such a scenario with the BeOpen binaries on a RedHat systems; the Tcl 8.0 RPMs of RedHat include the tcltk-8.0-ieee.patch, which removes the line AC_CHECK_LIB(ieee, main, [MATH_LIBS="-lieee $MATH_LIBS"]) from both Tcl's and Tk's configure.in. Of course, the problem is not restricted to Tcl: any extension module that links with a shared library that includes libieee.a would change the behaviour of Python's math functions. Regards, Martin

1 0

RE: [Python-checkins] CVS: distutils/distutils util.py,1.55,1.56
by Jeremy Hylton 14 Oct '00

14 Oct '00

Greg, Do you intend these most recent changes to be included in 2.0 final on Monday? Just giving you a heads up in case you didn't realize what the release schedule is. The only changes we should be making are critical bugs fixes for mainline code (non-demo, tools, etc.). Also, there is some code in the top-level of the CVS repository that hasn't been put in the previous source tarballs; at least the C code for the Windows installer, but maybe other things as well. Can you recommend how to include this in the source distribution for the final release? We'd like to include *all* the source code for distutils. Jeremy

1 0

can't build with threads on OSF/1
by Charles G Waldman 14 Oct '00

14 Oct '00

Sorry, I don't have time to investigate this fully right now. Maybe this weekend. For now this is just a heads-up. Hopefully, something is just misconfigured on this system. The machine in question is running OSF1 V4.0 878 alpha I can only build Python from current CVS sources on OSF/1 if I specify --without-thread. If I don't specify this I get the following errors at link time: Unresolved: __pthread_mutex_init __pthread_mutex_destroy __pthread_mutex_lock __pthread_mutex_unlock __pthread_cond_init __pthread_cond_destroy __pthread_cond_signal __pthread_cond_wait __pthread_create __pthread_detach I tried sending a message with the output of ./configure and make attached, but it was over the 40K limit. If anybody wants to see these, email me.

4 10

RE: Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)
by Mark Favas 14 Oct '00

14 Oct '00

[Huaiyu Zhu, replying to Tim...] >It is interesting to know that someone actually relies on >OverflowError to detect underflow. But why not let them continue that >as well, given that all of these would be available in the future >anyway? I think I'm the someone referred to - but I _don't_ rely on this at all. I was reporting that, on my platforms, the behaviour of exp(-666) did _not_ change between 1.5.2 and 2.0, both of which raised an OverflowError (for underflow). I prefer Tim's change so that exp(-huge) will produce 0, and that exp(huge) produces an OverflowError, even though this changes the behaviour when moving from 1.5.2 to 2.x... [Huaiyu Zhu again] >Before that happens, why consistency accross platform is deemed more >important than consistency accross releases? My naive thinking is >that people port more from 1.5.2 to 2.0 than from one platform to >another, isn't it? :-) In spite of the smiley, I'd have to say (politely) "Rubbish!". One of the many attractions of Python to me _is_ cross-platform consistency - more so than up-version consistency. In my previous life as a computational chemist I spent a lot of effort in writing code that was portable - indeed, we developed a pre-processor language (which spat out FORTRAN) to try to more easily handle OS/compiler/word-size/FP-characteristic/phase-of-moon issues. I first came up against IEEE stuff in the early 80's, and I agree with Tim - as a user, I hated the non-stop defaults. Mollycoddled by CDC machines, I was used to my code stopping when I divided by zero - it usually indicated a coding or algorithmic error. When (if) Python supports access to masking/unmasking 754 exceptions, that'll be good - in the meantime, from my POV, the steps taken by Tim/Guido are in the right direction. -- Mark Favas - m.favas(a)per.dem.csiro.au CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA

1 0

RE: Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)
by Tim Peters 13 Oct '00

13 Oct '00

[Tim] > ... > We cannot consider this to be a bug since Python has had no *intended* > behavior whatsoever wrt 754 gimmicks. We can and do consider gripes > about these accidents to be feature requests. [Huaiyu Zhu] > Yes it varies widely in Python. But isn't that precisely because Python > does not support IEEE? Yes, but that's darned near tautological <0.5 wink>. > If Python does not actively undermine IEEE on platforms that have it, > would it still vary widely across these platforms? Yes. Your own -lieee example showed that math.sqrt(-1) computes not only a non-754 result, but a senseless result, on your platform under 1.5.2. If I show you a platform, how are you going to decide whether it "has" IEEE? 754 is a very involved std, and it's still rare to find a platform C that supports it correctly. On platform C's that do support it correctly, I know of none that do so without requiring platform-specific tricks. Python has *nothing* portable it can rely on, and its mass of utterly non-754-aware code building on haphazard C implementations adds up to the random x-platform crap we have today. ... [T] > The patch to stop setting -lieee was contributed by a Python user who > claimed it fixed bugs on *their* platform. That's "the reason". > We don't want to screw them either. [H] > Q: What bug? Which platform? Maybe there is another way that has less > impact on others? How to find out? I didn't check it in, and you can search the CVS archives as easily as I can (I'm not inclined to, since I don't think it can affect the outcome of this). Jeremy Hylton may, or may not, remember more about it (IIRC, he said he checked it in). > ... > If there is a way to fix the bug on that other platform while still keep > -lieee on Linux, would that be acceptable? No: Guido *wants* overflow to raise OverflowError by default, and *wants* sqrt(-1.0) to raise ValueError by default. Python's x-platform fp numerics today are an utter mess, and the best we can do for 2.0 is to make them as consistent as we have time to make them, despite that we know it will break some of your code (but you can still link with -lieee if you feel you must), and that Mark Favas has confirmed it will break code under both his Solaris and Tru64 Unix systems (but for a reason opposite of yours: he's been getting OverflowError on underflow since 1.5.2, and we're taking that away from him, and he'll have *no* way to go back). At this point we favor forcing consistency over maintaining platform accidents, no matter how attached people have gotten to them; else we'll be stuck with them forever. [oh some of Kahan's papers, at http://http.cs.berkeley.edu/~wkahan/] > Interesting! I've glanced through some of Prof Kahan's writings. He is an entertaining writer, although his rhetoric is often more extreme than either of ours <wink>. > It appears that he favors raising flags that can be checked optionally, and > he is against mandatary raising exceptions. Have you read the 754 std? He's in favor of the 754 std. [quotes snipped, except for my favorite, which I've paraphrased many times over the years on c.l.py] > ... > They are called "Exceptions" because to any policy for handling them, > imposed in advance upon all programmers by the computer system, some > programmers will have good reason to take exception. > ... > From these quotes and others I understand that he wants f.p. exceptions to > be merely messanges, not errors that force programmers to take extra > actions, unless the programmer choose to. So what aspect of Java did he > think is deficient? It is the "Trap" style exception that mandate a users > action. His paper "How JAVA's Floating-Point Hurts Everyone Everywhere" lists his 5 main gripes about Java in bullet points at the top of page 3. The paper runs on to 80 pages. You somehow managed to miss 79.89 of them <0.4 wink>, reducing it to a bogus claim about Java that Kahan didn't make (Java has neither "Trap" nor "Flag" style fp exception signaling). > He told "A cautionary Tale of the Ariane 5", Which has nothing to do with Java -- the software in question was written in Ada, which (unlike Java) has "Trap" style. Why he put it in a paper about Java is a bit of a mystery; I write it off to rhetorical excess. > the European rocket that mulfunctioned after lift off. It turned out > that a piece of software produced an unimportant exception that is > not caught by a handler and caused debugging data to be dumped to a > critical memory location. This would have been avoided if the exception > merely raised a flag and returned a NaN, and the program continued. Actually, that was impossible to do, and he didn't suggest that. What he wrote is Had overflow merely obeyed the IEEE 754 default policy, the recalibration software would have raised a flag and delivered an invalid result both to be ignored by the motor guidance programs, and the Ariane 5 would have pursued its intended trajectory. Note that he didn't say NaN. He couldn't, because this was overflow in a conversion from Float to Integer. Despite his cheery optimism here, 754 is approximately useless in this case, since there is no generally accepted "invalid result" in integer formats (and 754 doesn't define one either -- the result of overflow in converting floats to ints is ill-defined in 754; 754 doesn't even require that it set the overflow flag). > So do we want every Python program to bomb whenever there is a floating > point value outside bound and is not trapped by try/except? By default, yes, Guido does, but for overflow, not underflow. If you read the following section of Kahan's paper, you should have noticed that he is *just* as critical of just returning Infs and NaNs (which is all Java does, and which is the accidental Python behavior you're arguing we keep in glibc) as he is of just trapping! His conclusion: "Without Traps nor Flags, Java s floating-point is Dangerous": Without flags, detecting rare creations of Inf and NaN before they disappear requires programmed tests and branches that, besides duplicating tests already performed by the hardware, slow down the program and impel a programmer to make decisions prematurely in many cases. Worse, a plethora of tests and branches undermines a programs modularity, clarity and concurrency. I agree with him there wholeheartedly: without the 754-mandated flags, *and* without raising exceptions, the subset of 754 remaining sucks. >> ironically, 754 is hardest to sell to those who could benefit from >> it the most. > So far the only concern from number-crunchers voiced here is that it might > allow sqrt(-1)==0, which is not the case. Gimme a break! In the very msg Paul Dubois said that, he also said "Some people use Inf but most people want the code to STOP so they can find out where the INFS started. Otherwise, two hours later you have big arrays of Infs and no idea how it happened.". I'm starting to drop my assumption of good faith on your part: what you remember and how you paraphrase (whether it's Paul or Kahan) is extremely selective, beyond normal bounds of zeal-inspired distortion. > I was a fortran programmer before. I switched to using IEEE ten years > ago as soon as it was available on the machine I was using. So there > are different opinions. Certainly, and good 754 support will cater to most of them. I don't believe your opinion (or mine, for that matter) reflects anything even close to the majority view here, though. As I said in my reply to Paul, my observations (based on 15 years on the vendor side of the fp biz) matched his exactly (based on his experience with hundreds of numeric code authors): a vast majority still want divide-by-0, invalid operation, and overflow to raise an exception the vast majority of the time. > ... > Obviously we agree on the benefit of having a choice. But we > have different perspective on what choice means. Your want a choice > to force user to take explicit action, I want a choice to ignore these > events. No. I want exactly the choices 754 lays out: user-settable sticky flags and user-maskable exceptions. We're not getting that for 2.0. And we're *never* getting it if 2.0 doesn't do *something* to start disabusing people of the notion that their historical numeric platform accidents in Python will be faithfully preserved forever more. > The ideal solution might be to have a sys.fp_exception class. Each > exception would set a flag like fp_exception.overflow so that user can > check them. There could also be a variable like fp_exception.force_trap, > which when set, could make each f.p. exception to produce a Python > exception. This would confirm to IEEE 754 and give both of us a choice > we want. Kinda, except that each of the 5 754-defined traps must be individually maskable. 754 spells out what's needed. A PEP will be required to spell out the Python- and C-level APIs. I've corresponded (just -- no time now) a little with Kevin Jacobs about that offline. > Now that we are in feature freeze for 2.0, we can't get both choices. > Can we adopt the one that does not break behaviors on previous official > release, even if that's only by accident? Not as far as Guido or I are concerned, no. You can link with -lieee yourself if you must, although I can't recommend perpetuating the x-platform confusion here. We want to say that, e.g., math.exp works substantially the same way on *all* platforms under 2.0, not repeat the same decade-old "umm, don't know, and no way to guess -- try it on your platform and see what it does" yet again. I don't expect 2.0 will actually achieve that on all platforms, but it should on the primary ones. > ... > So now I know why people reported that NaN and Inf were broken on HPUX > and Windows and some others. If Python had simply ignored the flags ... Then we would have broken 754 subsets in slightly subtler ways, and still wildly varying across platforms. > ... > Could you please point a URL to "presubstitution"? In the paragraphs I > quoted the only mention of presubstitute is with NaNs and Inf, etc. Sorry, I don't save URLs, and don't recall whether this was in a paper or on David Hough's "Numeric Interest" mailing list in the early 90's. A Google search on "presubstitution Kahan" turns up 4 promising-looking hits I'll leave you to pursue. The basic idea is that the programmer gets to specify, via library calls (whatever), which particular fp value each 754 exceptional case returns. So, e.g., if it made sense for some part of the app, the programmer could say they wanted +- pi returned instead +- Inf, if and whenever overflow happened. Then set it back to +- Inf (or anything else they needed) later. This would be best if supported in HW directly, of course. > ... > You are not confirming to 754 until you allow all the exceptions to go > through with well defined values as default. I know that, but don't care. As I said, enabling overflow + invalid + div0 by default is the only scheme that can be *sold*. If you can find two NumPy users in favor of disabling them all by default, I'll be mildly astonished. 754 did itself enormous harm by insisting on that trivial point (it's trivial because, with proper support, anyone can change the defaults to anything they like with one line of code): I worked in the supercomputer biz at the time 754 was adopted, and for a decade after, and the "non stop" defaults were *universally* opposed by customers. Scared the hell out of language stds committees too, which goes a lot farther than the 754 folks would like to admit toward explaining why language committees stayed away from 754 bindings in droves. C99 is 15 years(!) after the fact. > Raising Python exceptions on some of them does not confirm to 754, as > the default IEEE 754 behavior on f.p. exception is not a Python exception, > but just a raised flag. See above. > Before that happens, what is the motive to make IEEE less usable for those > who have it (pretty much every one nowadays), even if just by accident? I think I've repeated the reasons to death already, and you've repeated to death that you'll never agree. So no more from me on that. > ... > But just to clarify some wordings you quoted in another post: I meant that > sqrt(-1)==0 is insane, whether silent or not. I agree. > Raising an exception is sane, Not according to 754's default rules, which you wanted me to take as sacred just a few paragraphs ago <wink>. > although OverflowError is not so reasonable. I'd say it's insane. > Raising a ValueError is reasonable. I expect we'll have to make InvalidOperationError a subclass of ValueError for that reason, when 754 support materializes. > But just returning NaN without a fuss is even better. You're going to find that's a very lonely position. > My favorite is returning 1j, as MatPy does. Then you should use cmath.sqrt instead of math.sqrt. Even $3 business calculators have a sqrt key these days, and you do not want to be on the other end of email when trying to explain to a businessperson why their crappy std deviation program returned a complex number <0.1 wink>. > if-it-ain't-broke-don't-fix-it-ly y'rs it-it-weren't-broke-we-wouldn't-ly y'rs - tim

2 1

warnings --with-pydebug
by Jeremy Hylton 13 Oct '00

13 Oct '00

If you configure using '--with-pydebug' you get a lot of compile-time warnings. ../../Python/ceval.c: In function `eval_code2': ../../Python/ceval.c:672: warning: value computed is not used ../../Python/ceval.c:675: warning: value computed is not used ../../Python/ceval.c:678: warning: value computed is not used ../../Python/ceval.c:680: warning: value computed is not used ../../Python/ceval.c:681: warning: value computed is not used [etc.] I assume these are caused by the definition of Py_INCREF #define Py_INCREF(op) (_Py_RefTotal++, (op)->ob_refcnt++) Is there some way to change this macro so that the warnings are silenced? Jeremy

2 1

RE: [Python-Dev] RE: Possible bug (was Re: numpy, overflow, inf, ieee, and rich comparison)
by Charles G Waldman 13 Oct '00

13 Oct '00

Tim Peters wrote: > I'm very keen to know whether the current CVS version of Python > (2.0c1) still raises OverflowError for math.exp(-746) on your > Solaris and/or Tru64 Unix boxes. I just repeated this test, on the following 4 platforms: 1) SunOS fsui02 5.6 Generic_105181-21 sun4u sparc SUNW,Ultra-Enterprise cc: WorkShop Compilers 4.2 30 Oct 1996 C 4.2 2) OSF1 V4.0 878 alpha DEC C V5.6-079 on Digital UNIX V4.0 (Rev. 878) 3) IRIX64 6.5 07151432 IP27 MIPSpro Compilers: Version 7.2.1 4) Linux 2.2.16 #31 SMP Thu Jun 29 14:51:26 CDT 2000 i686 gcc version 2.95.2 19991024 (release) On all of the above systems, math.exp(-999) returns 0.0, math.sqrt(-1) raises an exception, and test_math.py completes successfully. Testing-ly (not to be confused with testily) yr's, cgw

4 6