Mailman 3 June 2006 - Python-Dev

Request for patch review
by Georg Brandl June 4, 2006

June 4, 2006

I've worked on two patches for NeedForSpeed, and would like someone familiar with the areas they touch to review them before I check them in, breaking all the buildbots which aren't broken yet ;) They are: http://python.org/sf/1346214 Better dead code elimination for the AST compiler http://python.org/sf/921466 Reduce number of open calls on startup GB Thanks! Georg

2 2

Re: [Python-Dev] Python Benchmarks
by john.m.camara＠comcast.net June 4, 2006

June 4, 2006

Here are my suggestions: - While running bench marks don't listen to music, watch videos, use the keyboard/mouse, or run anything other than the bench mark code. Seams like common sense to me. - I would average the timings of runs instead of taking the minimum value as sometimes bench marks could be running code that is not deterministic in its calculations (could be using random numbers that effect convergence). - Before calculating the average number I would throw out samples outside 3 … [View More]sigmas (the outliers). This would eliminate the samples that are out of wack due to events that are out of our control. To use this approach it would be necessary to run some minimum number of times. I believe 30-40 samples would be necessary but I'm no expert in statistics. I base this on my recollection of a study on this I did some time in the late 90s. I use to have a better feel for the number of samples that is required based on the number of sigmas that is used to determine the outliers but I have to confess that I just normally use a minimum of 100 samples to play it safe. I'm sure with a little experimentation with bench marks the proper number of samples could be determined. Here is a passage I found at http://www.statsoft.com/textbook/stbasic.html#Correlationsf that is related. '''Quantitative Approach to Outliers. Some researchers use quantitative methods to exclude outliers. For example, they exclude observations that are outside the range of �2 standard deviations (or even �1.5 sd's) around the group or design cell mean. In some areas of research, such "cleaning" of the data is absolutely necessary. For example, in cognitive psychology research on reaction times, even if almost all scores in an experiment are in the range of 300-700 milliseconds, just a few "distracted reactions" of 10-15 seconds will completely change the overall picture. Unfortunately, defining an outlier is subjective (as it should be), and the decisions concerning how to identify them must be made on an individual basis (taking into account specific experimental paradigms and/or "accepted practice" and general research experience in the respective area). It should also be noted that in some rare cases, the relative frequency of outliers across a number of groups or cells of a d esign can be subjected to analysis and provide interpretable results. For example, outliers could be indicative of the occurrence of a phenomenon that is qualitatively different than the typical pattern observed or expected in the sample, thus the relative frequency of outliers could provide evidence of a relative frequency of departure from the process or phenomenon that is typical for the majority of cases in a group.''' Now I personally feel that using 1.5 or 2 sigma approach is rather loose for the case of bench marks and the suggestion I gave of 3 might be too tight. From experimentation we might find that 2.5 is more appropriate. I usually use this approach while reviewing data obtained by fairly accurate sensors so being being conservative using 3 sigmas works well for these cases. The last statement in the passage is worthy to note as a high ratio of outliers could be used as an indication that the bench mark results for a particular run are invalid. This could be used to throw out bad results due to some one starting to listen to music while the bench marks are running, anti virus software starts to run, etc. - Another improvement to bench marks can be obtained when both the old and new code is available to be benched mark together. By running the bench marks of both codes together we could eliminate effects of noise if we assume noise at a given point of time would be applied to both sets of code. Here is a modified version of the code that Andrew wrote previously to show this clearer than my words. def compute_old(): x = 0 for i in range(1000): for j in range(1000): x = x + 1 def compute_new(): x = 0 for i in range(1000): for j in range(1000): x += 1 def bench(): t1 = time.clock() compute_old() t2 = time.clock() compute_new() t3 = time.clock() return t2-t1, t3-t2 times_old = [] times_new = [] for i in range(1000): time_old, time_new = bench() times_old.append(time_old) times_new.append(time_new) John [View Less]

2 1

ssize_t question: longs in header files
by Neal Norwitz June 3, 2006

June 3, 2006

$ grep long */*.h minus comments, etc yields several questions about whether some values should use Py_ssize_t rather than C longs. In particular: * hash values Include/abstract.h: long PyObject_Hash(PyObject *o); // also in object.h Include/object.h:typedef long (*hashfunc)(PyObject *); * ints: Include/intobject.h: long ob_ival; * thread values (probably no need to change since platform dependent) Attached is the whole list. This is just for C longs. The int list is much … [View More]

6 12

test_ctypes failures on ppc64 debian
by Thomas Heller June 3, 2006

June 3, 2006

test_ctypes fails on the ppc64 machine. I don't have access to such a machine myself, so I would have to do some trial and error, or try to print some diagnostic information. This should not be done in the trunk, so the question is: can the buildbots build branches? I assume I just have to enter a revision number and press the force-build button, is this correct? Or would someone consider this abuse? Thomas

3 3

Removing Mac OS 9 cruft
by Neal Norwitz June 3, 2006

June 3, 2006

I was about to remove Mac/IDE scripts, but it looks like there might be more stuff that is OS 9 related and should be removed. Other possibilities look like (everything under Mac/): Demo/* this is a bit more speculative IDE scripts/* MPW/* Tools/IDE/* this references IDE scripts, so presumably it should be toast? Tools/macfreeze/* Unsupported/mactcp/dnrglue.c Wastemods/* I'm going mostly based on what has been modified somewhat recently. Can someone confirm/reject these? I'… [View More]

4 3

Re: [Python-Dev] SF patch #1473257: "Add a gi_code attr to generators"
by Phillip J. Eby June 2, 2006

June 2, 2006

At 09:53 PM 5/31/2006 +0200, Collin Winter wrote: >Hi Phillip, > >Do you have any opinion on this patch (http://python.org/sf/1473257), >which is assigned to you? I didn't know it was assigned to me. I guess SF doesn't send any notifications, and neither did Georg, so your email is the very first time that I've heard of it. I don't have any opinion, but perhaps Python-Dev does?

4 3

test_unicode failure on MIPS
by Neal Norwitz June 2, 2006

June 2, 2006

Any ideas? http://www.python.org/dev/buildbot/all/MIPS%20Debian%20trunk/builds/176/ste… ====================================================================== FAIL: test_count (test.test_unicode.UnicodeTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/test/test_unicode.py", line 97, in test_count string_tests.CommonTest.test_count(self) File "/home/pybot/… [View More]

2 2

valgrind report
by Neal Norwitz June 2, 2006

June 2, 2006

Looks pretty good, except for 1 cjk problem: test_codecencodings_jp Invalid read of size 1 at 0x110AEBC3: shift_jis_2004_decode (_codecs_jp.c:642) by 0xBFCBDB7: mbidecoder_decode (multibytecodec.c:839) Address 0xAEC376B is 0 bytes after a block of size 3 alloc'd at 0x4A19B7E: malloc (vg_replace_malloc.c:149) by 0xBFCBF54: mbidecoder_decode (multibytecodec.c:1023) n

1 0

Re: [Python-Dev] feature request: inspect.isgenerator
by Michael Chermside June 2, 2006

June 2, 2006

Phillip J. Eby writes: > Yes, I think the whole concept of inspecting for this is broken. > *Any* function can return a generator-iterator. A generator > function is just a function that happens to always return one. Just following up on Phillip's comments, consider the following functions: def foo(x): while still_going(x): yield some_func(x) def bar(x): while still_going(x): yield other_func(x) def foo_or_bar(x): … [View More]

1 0

S/390 buildbot URLs problematic
by Georg Brandl June 1, 2006

June 1, 2006

The S/390 buildbot should be renamed. While the URLs buildbot generates in its email messages work, the ones that are on the overview page don't. Georg

1 0