Mailman 3 November 2001 - Python-Dev

Re: [Python-Dev] Re: Future division detection
by Michael Hudson 05 Nov '01

05 Nov '01

com-nospam(a)ccraig.org (Christopher A. Craig) writes: > "Tim Peters" <tim.one(a)home.com> writes: > > > [Christopher A. Craig, playing with modifying the meaning of division] > > > ... > > > While doing this I was thinking that I would change true_division on > > > ints and floats to return a rational and change the rational code to > > > return a long if the denominator is 1. This works great, except that > > > if future division is off then rationals can suddenly become longs and > > > do not automatically cast back. This makes it virtually impossible to > > > guarantee a correct result to nearly any rational computation that > > > involves a division. > > > > > > So I wanted to know if there is some way to detect, at the object > > > level, if the CO_FUTURE_DIVISION feature is active. > > > > I'm unclear on what you're asking. In case it helps, note this section in > > __future__.py: > > > > Hmm, I thought I was being too verbose, I guess I was being too > unclear. I have a rational module (in C) which I am trying to patch > to be an object satisfying PEP-239. The problem is that I would like > to have an rational cast back to a integer iff (1) the denominator is 1 > and (2) future division is active. > > If I don't check (2) then I get the situation that if future division > is inactive then `(rational('1/3')*3)/5` would yield 0 instead of 1/5. > This makes rationals pretty much useless unless future division is > active. > > What I would like to do is have a function that can check to see if > the CO_FUTURE_DIVISION flag is set and if it is and the denominator is > 1 then return the numerator, else return the rational. > > I have a strong suspicion that this can't be done, in which case I > just won't do the cast back automatically. Does this: int is_future_div(void) { PyCompilerFlags cf; PyEval_MergeCompilerFlags(&cf); return cf.cf_flags & CO_FUTURE_DIVISION; } work? You'll need to change this when future division becomes the default, but I think it'll work today. This is a murky dark corner of the interpreter, though -- so don't blame me when it breaks! Cheers, M. -- >> REVIEW OF THE YEAR, 2000 << It was shit. Give us another one. -- NTK Know, 2000-12-29, http://www.ntk.net/

2 1

PEP 252 comment
by Michael McLay 05 Nov '01

05 Nov '01

The potential for optimizing the new class/type sytem in Python can be improved by adding an additional constraint to the name lookup rules[1]. This new constrain only apply to the new name lookup rules for types that include __slots__ definitions so the change should not break backward compatibility. - Any name defined by __slot__ is not allowed as a key in the type dict. This rule would eliminate special type attribute names, like __dict__ and __class__, from the names allowed in __slots__. The constraint would also make it illegal to injected a name into the type dictionary from outside the class definition if that name is one of the names in _slots__. In the following example the costraint would allow the assignment to B.d, but not the assignment to B.b. The assignment of 'c=5' would trigger a TypeError because the c is defined in __slots__. The '__dict__' in __slots__ would also trigger a type error because it is a special attribute name that is defined for all types. >>> class B(object): a = 3 c = 5 __slots__ = ['b', '__dict__' , 'c'] def __init__(self): self.b = 4 >>> B.d = 'fish' >>> b = B() >>> b.b 4 >>> b.d 'fish' >>> B.b = 3.2 >>> b.b 3.2000000000000002 >>> b.b = 55 Traceback (most recent call last): File "<pyshell#29>", line 1, in ? b.b = 55 AttributeError: 'B' object attribute 'b' is read-only >>> The rule eliminates the need to look in the type dictionary if the name is defined in __slots__. The cryptic AttributeError message in the example would never occur. A slightly more restrictive rule would extend the constraint to eliminate any addition of names to the type dict from outside of the class definition. This additional constraint would make the assignment to B.d illegal in the previous example. This rule would make it possible to know the complete list of all names that could be referenced by an object of the given type. [1] From http://python.sourceforge.net/peps/pep-0252.html In the more complicated case, there's a conflict between names stored in the instance dict and names stored in the type dict. If both dicts have an entry with the same key, which one should we return? Looking at classic Python for guidance, I find conflicting rules: for class instances, the instance dict overrides the class dict, *except* for the special attributes (like __dict__ and __class__), which have priority over the instance dict. - I resolved this with the following set of rules, implemented in PyObject_GenericGetAttr(): 1. Look in the type dict. If you find a *data* descriptor, use its get() method to produce the result. This takes care of special attributes like __dict__ and __class__. 2. Look in the instance dict. If you find anything, that's it. (This takes care of the requirement that normally the instance dict overrides the class dict.) 3. Look in the type dict again (in reality this uses the saved result from step 1, of course). If you find a descriptor, use its get() method; if you find something else, that's it; if it's not there, raise AttributeError.

1 0

Patch submitted for cross-platform newline support
by Jack Jansen 05 Nov '01

05 Nov '01

Hi folks, I've just submitted a patch that allows Python to do cross-platform newline handling, #476814. This sort-of happened after the promised PEP didn't materialise, and there was discussion about this among myself, Guido and the pythonmac-sig, and at some point Guido said "Ok, code it!". The patch has been tested on SGI Irix and in MacPython only. I'd like people to have a look at this. There's one thing I don't like, and that's the changes I had to make to the parser sourcefiles. They're needed, but I'm open to suggestions as to a better way to do this. And of course I'm open to any suggestions. I'm also interested in discussing whether a patch like this is appropriate while we're in beta. On the one hand I would say it is, because the feature is disabled by default. On the other hand there are changes (albeit mainly cosmetic ones) in a large number of places. Another argument for allowing this even while in beta is that I really really want it for Mac OS X (but this might not be a very strong argument, I guess:-). Here's the summary of the patch: This patch enables Python to interpret all known newline conventions, CR, LF or CRLF, on all platforms. This support is enabled by configuring with --with-universal-newlines (so by default it is off, and everything should behave as usual). With universal newline support enabled two things happen: - When importing or otherwise parsing .py files any newline convention is accepted. - Python code can pass a new "t" mode parameter to open() which reads files with any newline convention. "t" cannot be combined with any other mode flags like "w" or "+", for obvious reasons. File objects have a new attribute "newlines" which contains the type of newlines encountered in the file (or None when no newline has been seen, or "mixed" if there were various types of newlines). Also included is a test script which tests both file I/O and parsing. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++

6 6

OS/2 VAC++ patches
by Andrew MacIntyre 05 Nov '01

05 Nov '01

I have looked at Michael Muller's VAC++ patches, and added comments as followups. In summary: 473749 - has several (minor IMO) stylistic issues (addition of OS/2 #ifdefs). The intent of the changes looks OK, as do most of the actual changes (those in OS/2 VAC specific files or existing OS/2 #ifdefs). 474169 - looks good to go (changes inside existing OS/2 #ifdef) 474500 - looks good to go (isolated to OS/2 specific file) Comments on the stylistic issues? -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac(a)bullseye.apana.org.au | Snail: PO Box 370 andymac(a)pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia

2 2

Memory leaks and uninitialized memory
by Neal Norwitz 05 Nov '01

05 Nov '01

I just submitted a bunch of patches to SF to correct memory leaks, possible memory leaks, and uninitialized memory reads (UMR). The problems were found by using purify on the regression tests. There are still some problems that I have not be able to correct. Here is a list of the biggest problems, the details are at the end of this mail: test_long_future - leaks 43124 bytes (future division on longs) test_asynchat (and others) - leaks 2k+ socketmodule.c:633 test___all__ (and others) - UMR marshal.c:438 test_parser - leaks 5k - parsermodule.c:721 recurses to line 716 There are other smaller problems, but this mail is already quite long. Once the bigger problems are resolved, I can work more on the small ones. Let me know if there's more info necessary to fix these problems. Also, if anyone wants to see the complete list of regression tests run and the results, let me know. What is the best format to post this info? Would there be a better way to deal with these kinds or problems? Neal -- test_long_future: there are memory leak problems with binary operations which use longs and future division: the following code leaks 32 bytes: >>> from __future__ import division >>> 5L / 3L here's the stack trace, using current CVS snapshot (from Sat): malloc [rtlib.o] muladd1 [longobject.c:51] PyLong_FromString [longobject.c:1027] parsenumber [compile.c:1096] com_atom [compile.c:1461] com_power [compile.c:1853] com_factor [compile.c:1982] com_term [compile.c:1994] com_arith_expr [compile.c:2027] com_shift_expr [compile.c:2053] com_and_expr [compile.c:2079] com_xor_expr [compile.c:2101] com_expr [compile.c:2123] com_comparison [compile.c:2177] com_and_test [compile.c:2252] com_test [compile.c:2353] and: muladd1 [longobject.c:51] PyLong_FromString [longobject.c:1027] parsenumber [compile.c:1096] com_atom [compile.c:1461] com_power [compile.c:1853] com_factor [compile.c:1982] com_term [compile.c:1992] com_arith_expr [compile.c:2027] com_shift_expr [compile.c:2053] com_and_expr [compile.c:2079] com_xor_expr [compile.c:2101] com_expr [compile.c:2123] com_comparison [compile.c:2177] com_and_test [compile.c:2252] com_test [compile.c:2353] here's a different stack trace if it helps: x_add [longobject.c:51] long_add [longobject.c:1408] binary_op1 [abstract.c:343] PyNumber_Add [abstract.c:578] eval_frame [ceval.c:960] PyEval_EvalCodeEx [ceval.c:2549] fast_function [ceval.c:3116] eval_frame [ceval.c:1996] PyEval_EvalCodeEx [ceval.c:2549] PyEval_EvalCode [ceval.c:483] PyImport_ExecCodeModuleEx [import.c:494] load_source_module [import.c:764] load_module [import.c:1348] import_submodule [import.c:1887] load_next [import.c:1743] test_asynchat: memory is allocated from socketmodule.c:633 (getaddr) it appears the memory is deallocated, but purify is still complaining here's the stack trace: malloc [rtlib.o] __IPv6_alloc [getipnodeby.c] getipnodebyname [getipnodeby.c] get_addr [getaddrinfo.c] getaddrinfo [libsocket.so.1] setipaddr [socketmodule.c:633] getsockaddrarg [socketmodule.c:821] PySocketSock_bind [socketmodule.c:1186] fast_cfunction [ceval.c:3086] eval_frame [ceval.c:1979] PyEval_EvalCodeEx [ceval.c:2549] PyEval_EvalCode [ceval.c:483] PyImport_ExecCodeModuleEx [import.c:494] load_source_module [import.c:764] load_module [import.c:1348] import_submodule [import.c:1887] test___all__: I'm not sure if this is a problem in Python or not. It's possible that the problem is atof()/strtod() reads the string in 4 byte increments. This is occurring while in: __big_float_times_power [libc.so.1] __decimal_to_binary_integer [libc.so.1] __decimal_to_unpacked [libc.so.1] decimal_to_double [libc.so.1] strtod [libc.so.1] r_object [marshal.c:438] r_object [marshal.c:524] r_object [marshal.c:595] PyMarshal_ReadObjectFromString [marshal.c:741] PyMarshal_ReadLastObjectFromFile [marshal.c:700] load_source_module [import.c:582] load_module [import.c:1348] import_submodule [import.c:1887] load_next [import.c:1743] import_module_ex [import.c:1594] PyImport_ImportModuleEx [import.c:1635] Reading 2 bytes from 0xffbe9034 on the stack. Address 0xffbe9034 is 452 bytes below frame pointer in function __decimal_to_unpacked. test_parser: This memory was allocated from: malloc [rtlib.o] PyNode_AddChild [node.c:35] build_node_children [parsermodule.c:716] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] build_node_children [parsermodule.c:721] Block of 20 bytes (69 times); last block at 0x4fc238

3 6

2.1.2: patching Modules/
by Martin von Loewis 03 Nov '01

03 Nov '01

[\n\ -> \n" conversion] > Did these actually fix anything? Looking at the check-in message to mathmodule.c 2.61, we see author: tim_one; # Mechanical fiddling to make this easier to work with in my editor. So I guess the answer is: "yes, it makes Tim's editor happy" :-) Apart from that, it apparently didn't fix any further problems. > many files: config.h -> pyconfig.h: Don't see a point - if it's broken in > 2.1.1, it'll still be broken in 2.1.2. Indeed. Furthermore, applications (including distutils) may rely on knowing the name of the config.h; those would break with the change. > Modules/zlibmodule.c: allow threads in zlib, misc other fixes - diff > is nearly the entire file. The fix for patch #403753 would be good, > but is an incompatibility. Hm. I don't think this qualifies as a "bug fix". It fixes a problem, yes, but applications have to be changed to make use of the feature. > gcmodule: the lock to stop multiple calls to collect() - pretty sure > I've got a working patch for this, but it's hard to test. Why can't you use the patch as-is? It would apply without problems, if generation0 hadn't been renamed to _PyGC_generation0. > _curses* If "didn't build before, does now" also is a candidate for inclusion, I think you could more or less use _cursesmodule.c as-is. If you only want fixes for systems on which it already worked, it will be more difficult. Regards, Martin

2 1

Re: [Python-checkins] CVS: python/dist/src/Modules shamodule.c,2.15,2.16
by Andrew Kuchling 02 Nov '01

02 Nov '01

On Fri, Nov 02, 2001 at 02:04:19PM -0800, Fred L. Drake wrote: >Simplfy the insint() macro to use PyModule_AddIntConstant(). Should this clean-up be done more generally? ute Modules>grep -l insint *.c dlmodule.c pcremodule.c selectmodule.c shamodule.c socketmodule.c ute Modules> --amk

2 1

Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstatcache.tex,1.3,1.4
by Neil Schemenauer 02 Nov '01

02 Nov '01

Fred L. Drake wrote: > Add deprecation notice to statcache. Should a warning be added to the module as well? Neil

2 1

who's maintaining tools/idle?
by Fredrik Lundh 02 Nov '01

02 Nov '01

just noticed that CVS is merging changes into icons/minusnode.gif every time I do a cvs update. doesn't sound right. turns out that minusnode.gif and plusnode.gif has been checked in as text files (no -kb option). how can I fix this? (I should know, but I only use perforce these days, and my memory isn't what it used to be...) </F>

3 3

Future of SSL
by Gerhard Häring 02 Nov '01

02 Nov '01

First I'd like to say hello. I'm new to this list :-) I've some concerns with the latest development wrt SSL development in Python. The SSL support in the Python core seems to have been added with Python 2.0, and Guido told me in a SF patch discussion, that Pythonlabs doesn't know the code very well. Meanwhile it seems like Jeremy has tried fixing bugs. I've also submitted several patches to the SSL code. As you might know, there are several third-party SSL implementatations for Python (m2crypto, pyOpenSSL, POW). I've contacted the maintainers of these packages and asked them if they'd like to review my patches. The two that answered told me that the reason they implemented their own packages was that the SSL support in Python itself was very incomplete and, to sum it up, Python's SSL is just plain broken. The current state is that there is a single ssl() method in the socket module that can be used to instantiate client-side SSL connections. There's also a patch on Sourceforge that adds server-side SSL functionality. Several points why the current API is broken: - no support for specifying the SSL protocol (SSL v2/SSL v3/SSL v2|3) - no proper certicate checking - SSL objects are neither compatible with sockets nor are they file-like objects - no advanced features like SSL callbacks (to allow the user to accept a cerficicate, for example) What I'd suggest for Python 2.2 is to *not* add any new features, like server-side SSL but only accept bugfixes for the current client-side code. As the current implementation is broken and there is probably little SSL knowledge in the Python core team, I propose to "outsource" the problem: It should be possible to define a "Python SSL interface" that describes an API for SSL. The various modules in Python that use SSL (urllib, smtp, ...) would then be rewritten to use the new API. The socketmodule.c would be rewritten to use the new API instead. Then, wrappers could be written for the various SSL modules that wrap them into the new "Python SSL interface" API. As for me, I'm not an expert in SSL, but I'd be willing to try coordinate the efforts, write a PEP, talk with the module maintainers and such. I'd be grateful to hear your opinions about this newbie proposal :-) Gerhard -- mail: gerhard <at> bigfoot <dot> de registered Linux user #64239 web: http://www.cs.fhm.edu/~ifw00065/ OpenPGP public key id 86AB43C0 public key fingerprint: DEC1 1D02 5743 1159 CD20 A4B6 7B22 6575 86AB 43C0 reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b')))

9 23