Mailman 3 June 2006 - Python-Dev

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (1)
by Tim Peters June 7, 2006

June 7, 2006

[moving to python-dev] [Tim, gets different results across whole runs of python_d ../Lib/test/regrtest.py -R 2:40: test_filecmp test_exceptions ] >>> Does that make any sense? Not to me -- I don't know of a clear reason >>> other than wild loads/stores for why such runs should ever differ. [Neal] >> The problem generally has to do with modules cacheing things. [Martin] > Then the numbers should repeat on each run. Right. The point wasn't that I saw a … [View More]variety of different integers in the "leak vector" in a single run, it was that I got different results _across_ runs: no leaks in either test, a leak in one but not the other, or (very rarely) leaks in both. > So wild loads/stores are a more compelling explanation. Of course, it > *should* even be repeatable with wild loads/stores, unless the OS > shuffles the address space on each run (which at least Linux, > unfortunately, does). Well, I just tried this (illegal) C program under VC 7.1: #include <stdio.h> #include <stdlib.h> int main() { int *p; int i, sum; p = (int *)malloc(sizeof(int)); printf("%p %d ...", p, sum); for (sum = 0, i = -12; i < 29; ++i) sum += p[i]; printf("%d\n", sum); return 0; } Here are several runs. Note that the address malloc() returns is always the same, but adding the junk "around" that address often gives a different result, and the stack trash `sum` contains at first also varies. Put those together, and they show that wild loads from stack trash _and_ heap trash can vary across runs: C:\Code>cl boogle.c Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. boogle.c c:\code\boogle.c(9) : warning C4700: local variable 'sum' used without having been initialized Microsoft (R) Incremental Linker Version 7.10.3077 Copyright (C) Microsoft Corporation. All rights reserved. /out:boogle.exe boogle.obj C:\Code>for %i in (1 2 3 4 5 6 7 8 9) do boogle.exe C:\Code>boogle.exe 00322EA8 315894624 ...75845519 C:\Code>boogle.exe 00322EA8 316050874 ...125913836 C:\Code>boogle.exe 00322EA8 316050874 ...125913836 C:\Code>boogle.exe 00322EA8 316207124 ...8930763 C:\Code>boogle.exe 00322EA8 316207124 ...8930763 C:\Code>boogle.exe 00322EA8 316207124 ...8930763 C:\Code>boogle.exe 00322EA8 316363374 ...42224689 C:\Code>boogle.exe 00322EA8 316363374 ...42224689 C:\Code>boogle.exe 00322EA8 316519624 ...92948238 How did I pick -12 and 29 for i's bounds? Easy: I started with much larger bounds, and reduced them haphazardly until the program stopped segfaulting :-) Now I hate to think this is "the cause" for regrtest -R varying across identical runs, but I don't have many other suspects in mind. For example, I tried forcing random.seed() at the start of regrtest.main(), but that didn't make any visible difference. [View Less]

3 4

How to fix the buffer object's broken char buffer support
by Brett Cannon June 7, 2006

June 7, 2006

If you run ``import array; int(buffer(array.array('c')))`` the interpreter will segfault. While investigating this I discovered that buffer objects, for their tp_as_buffer->bf_getcharbuffer, return the result by calling the wrapped object bf_getreadbuffer or bf_getwritebuffer. This is wrong since it is essentially redirecting the expected call to the wrong tp_as_buffer slot for the wrapped object. Plus it doesn't have Py_TPFLAGS_HAVE_GETCHARBUFFER defined. I see two options here. One is … [View More]

2 2

Re: [Python-Dev] Stdlib Logging questions (PEP 337 SoC)
by Phillip J. Eby June 7, 2006

June 7, 2006

At 09:04 PM 6/5/2006 -0400, Jim Jewett wrote: >On 6/4/06, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > can we please delay the import until it's actually needed? i.e., > > until after some logging option is enabled? > >I have asked her to make this change. > >I don't like the extra conditional dance it causes, but I agree that >not wanting to log is a valid use case. > >On the other hand, the one-time import cost is pretty low for a >long-… [View More]

6 7

Stdlib Logging questions (PEP 337 SoC)
by Jim Jewett June 6, 2006

June 6, 2006

Jackilyn is adding logging to several stdlib modules for the Google Summer of Code (PEP 337), and asked me to review her first few changes. There were a few comments that I felt I should double-check with Python-dev first, in case my own intuition is wrong. For reference, she is adding the following prologue to several modules: import logging _log = logging.getLogger('py.NAME') where NAME is the module name (1) Should we ask Vinay Sajip to add a convenience function (for getting … [View More]the stdlib logger) to the logging module itself? It seems like it might be overkill, amd it wouldn't save even a single line. On the other hand, I wonder if we will eventually end up wanting a common choke point for hooks. PEP 337 did not recommend any such function, and recommended setting the handlers and level for py.* as sufficient control. So I'm inclined to say "no changes to the logging package itself". (2) Should NAME be module.__name__? Note that this means the log messages would be different when the module is run as a script (and the name is therefore __main__). I'm not thrilled with repeating the module name as a string, but I think that may be the lesser of evils. (3) Should she put out a message when a (now logged) module is loaded? If so, at what level? I'm leaning towards _log.debug("Stdlib NAME loaded as ", __name__) (4) Should she add (other) debugging information that is not already present? I think that "if __debug__: print ..." or "if debuglevel>0: sys.stdout.write..." should be changed, and things like set_debuglevel should be translated to calls to change the level of the appropriate logger. What about print lines that are currently commented out? Should these be displayed at debug level? (I think so.) What about useful information that isn't currently displayed? For instance, the asyncore module is careful to reuse a global mapping if it already exists (presumably from a reload), but that seems rare enough that it should be worth at least an info message. What about normal config info, such as the typical case of creating an empty mapping? In Java, I would expect a CONFIG message either way. In python, that seems like too much of a change, but ... at debug level ... maybe not doing it is just a bad habit. Unless convinced otherwise, I'll ask her to use debug for commented out lines that were still important enough to leave in the code, info message for unexpected choices, and not to bother with logging "yup, did the normal thing" (5) Should she clean up other issues when touching a module? In general, stdlib code isn't updated just for style reasons, unless there is a deprecation -- but in these cases, the module is already changing slightly. My inclination would be that she doesn't have to fix everything, but should generally do so when either it is "easy" or the old code is arguably a bug (such as bare excepts). (And "easy" might change as the summer wears on.) -jJ [View Less]

4 4

Stdlib Logging questions (PEP 337 SoC)
by Jim Jewett June 6, 2006

June 6, 2006

>> py.asyncore.dispatcher.hits >> py.asyncore.dispatcher.messages > These handler names seem really specific, though. Why have > 'dispatcher' in there? The existing logging that she is replacing is done through methods of the dispatcher class. The dispatcher class is only a portion of the whole module. > Part of Jackilyn's task should be to refine and improve the PEP. Agreed. > Logging is probably irrelevant for many modules, but which ones are > those? … [View More]

4 3

ssize_t: ints in header files
by Jim Jewett June 6, 2006

June 6, 2006

(Neal Norwitz asked about changing some additional ints and longs to ssize_t) Martin v. Löwis replied: > ... column numbers shouldn't exceed 16-bits, and line #s > shouldn't exceed 31 bits. Why these particular numbers? As nearly as I can tell, 8 bits is already too many columns for human readability. If python is being used as an intermediate language (code is automatically generated, and not read by humans), then I don't see any justification for any particular limits, except as an … [View More]

2 1

Include/structmember.h, Py_ssize_t
by Thomas Heller June 6, 2006

June 6, 2006

In Include/structmember.h, there is no T_... constant for Py_ssize_t member fields. Should there be one? Thomas

3 2

Re: [Python-Dev] Stdlib Logging questions (PEP 337 SoC)
by Phillip J. Eby June 6, 2006

June 6, 2006

At 10:13 AM 6/6/2006 -0400, Jim Jewett wrote: >On 6/5/06, Phillip J. Eby <pje(a)telecommunity.com> wrote: > >>I notice you've completely avoided the question of whether this should be >>being done at all. > >>As far as I can tell, this PEP hasn't actually been discussed. Please >>don't waste time changing modules for which there is no consensus that this >>*should* be done. > >Under a specific PEP number, no. The concept of adding logging to … [View More]

1 0

DC Python sprint on July 29th
by A.M. Kuchling June 6, 2006

June 6, 2006

The Arlington sprint this past Saturday went well, though the number of Python developers was small and people mostly worked on other projects. The CanDo group, the largest at the sprint with about 10 people, will be having a three-day sprint July 28-30 (Fri-Sun) at the same location. We should take advantage of the opportunity to have another Python sprint. Let's schedule it for Saturday July 29th (the day after OSCON ends in Oregon). --amk

1 0

feature request: inspect.isgenerator
by Michele Simionato June 6, 2006

June 6, 2006

Is there still time for new feature requests in Python 2.5? I am missing a isgenerator function in the inspect module. Right now I am using def isgenerator(func): return func.func_code.co_flags == 99 but it is rather ugly (horrible indeed). Michele Simionato

6 17