Mailman 3 May 2001 - Python-Dev

type/class unification and ExtensionClass
by skip＠pobox.com May 22, 2001

May 22, 2001

I know Guido has recently been working on some of the type/class unification issues (PEPs 252 and 253). Will this affect ExtensionClass? In particular, will it go away or have to be reworked significantly for Python 2.2 or 2.3? The new PyGtk wrappers use the ExtensionClass module. I'm curious about how hard it would be to move away from ExtensionClass for these wrappers. My reading of PEP 253 suggests this shouldn't be too difficult. I'd ask Guido directly, but I figure other people on … [View More]

2 1

Re: [Python-checkins] CVS: python/dist/src/Objects tupleobject.c,2.48,2.49
by Guido van Rossum May 21, 2001

May 21, 2001

Tim wrote: > BUG ALERT: The tuple (and list) richcmp algorithm is arguably wrong, > because it won't believe there's any difference unless Py_EQ returns false > for some corresponding elements: > > >>> class C: > ... def __lt__(x, y): return 1 > ... __eq__ = __lt__ > ... > >>> C() < C() > 1 > >>> (C(),) < (C(),) > 0 > >>> > > That doesn't make sense -- provided you believe the defn. of C makes … [View More]

2 6

Re: [Python-Dev] Classes and Metaclasses in Smalltalk
by Guido van Rossum May 21, 2001

May 21, 2001

Hi Alan, Thanks a lot for your input. I am cc'ing this reply to python-dev because I think my reply will be interesting for others. (Python-dev'ers: Alan expressed concern that introducing Smalltalk metaclasses would make Python unnecessarily complicated.) The way my thinking is currently going, it's not likely that Python will get a metaclass system similar to Smalltalk. However, unifying types and classes is useful for other reasons: please go to http://python.sourceforge.net/peps/ to … [View More]

1 0

test_mailbox now fails on Windows
by Tim Peters May 21, 2001

May 21, 2001

Appears to be because new code uses os.link, which doesn't exist on Windows. BTW, test_urllib2.py is still failing on Windows (and has been for a couple of weeks).

1 0

explanations for more pybench slowdowns
by Jeremy Hylton May 21, 2001

May 21, 2001

I did some profiles of more of the pybench slowdowns this afternoon and found a few causes for several problem benchmarks. I just made a couple of small changes for BuiltinFunctionCalls. The problem here is that PyCFunction calls were optimized for flags == 0 and not flags == METH_VARARGS, which is more common. The scary thing about BuiltinFunctinoCalls is that the profiler shows it spending almost 30% of its time in PyArg_ParseTuple(). It certainly is a shame that we have this complicated, … [View More]slow run-time parsing mechanism to deal with a static property of the code, namely how many arguments it takes and whether their types are. A few of the other tests, SimpleComplexArithmetic and CreateStringsWithConcat, are slower because of the new coercion logic. I didn't spend much time on SimpleComplexArithmetic, but I did look at CreateStringsWithConcat in some detail. The basic problem is that "ab" + "cd" gets compiled to BINARY_ADD, which in turn calls PyNumber_Add("ab", "cd"). This function tries all sorts of different ways to coerce the strings into addable numbers before giving up and trying sequence concat. It looks like the new coercion rules have optimized number ops at the expense of string ops. If you're writing programs with lots of numbers, you probably think that's peachy. If you're parsing HTML, perhaps you don't :-). I looked at the test suite to see how often it is called with non-number arguments. The answer is 77% of the time, but almost all of those calls are from test_unicodedata. If that one test is excluded, the majority of the calls (~90%) are with numbers. But the majority of those calls just come from a few tests -- test_pow, test_long, test_mutants, test_strftime. If I were to do something about the coercions, I would see if there was a way to quickly determine that PyNumber_Add() ain't gonna have any luck. Then we could bail to things like string_concat more quickly. I also looked at SmallLists. It seems that the only significant change since 1.5.2 is the garbage collection. This tests spends a lot more time deallocating lists than it used to, and the only change I see in the code is the GC. I assume, but haven't checked, that the story is similar for SmallTuples. So the primary things that have slowed down since 1.5.2 seem to be: comparisons, coercion, and memory management for containers. These also seem to be the things that have improved the most in terms of features, completeness, etc. Looks like we need to revisit them and sort out the performance issues. Jeremy [View Less]

4 4

[development doc updates]
by fdrake＠cj42289-a.reston1.va.home.com May 21, 2001

May 21, 2001

The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Incremental updates to the Python 2.2 documentation.

1 0

RE: [Python-checkins] CVS: python/dist/src/Doc/tut tut.tex,1.133.2.1,1.133.2.2
by Tim Peters May 21, 2001

May 21, 2001

[Fred checkin] > > *************** > > *** 2610,2617 **** > > \begin{verbatim} > > >>> x = 10 * 3.14 > > ! >>> y = 200*200 > > >>> s = 'The value of x is ' + `x` + ', and y is ' + `y` + '...' > > >>> print s > > ! The value of x is 31.4, and y is 40000... > > >>> # Reverse quotes work on other types besides numbers: > > ... p = [x, y] > > --- 2610,2617 ---- > > \… [View More]begin{verbatim} > > >>> x = 10 * 3.14 > > ! >>> y = 200 * 200 > > >>> s = 'The value of x is ' + `x` + ', and y is ' + `y` + '...' > > >>> print s > > ! The value of x is 31.400000000000002, and y is 40000... > > >>> # Reverse quotes work on other types besides numbers: > > ... p = [x, y] [Guido] > Hmm... The tutorial now contains at least one example of floating > point imprecision. Does it also contain text to explain this? (I'm > sure Tim would be happy to provide some if there isn't any. :-) [Fred] > It contains others, and I don't think there's an explanation. Some > text from Tim to explain this would be greatly apprectiated! Actually, 31.400000000000002 wasn't a true improvement over the earlier 31.4: so long as we rely on the platform C to format floats, the output isn't well-defined (the last digit or so can and will vary across boxes). I can certainly explain that this is so, and even why, but unsure the tutorial is the right place for it. In any case the tutorial shouldn't be giving examples whose output is platform-dependent. For example, don't use 10 * 3.14, use 10 * 3.25. Want me to scour the tutorial for all such cases? Or we could put the attached function at the start of the tutorial and use it to format floats: >>> f2ds(10 * 3.14) '31400000000000002131628207280300557613372802734375e-48' >>> I'm sure newbies would feel assured by that <wink>. def f2ds(x): """Return float x as exact decimal string. The string is of the form: "-", if and only if x is < 0. One or more decimal digits. The last digit is not 0 unless x is 0. "e" The exponent, a (possibly signed) integer """ import math # XXX ignoring infinities and NaNs for now. if x == 0: return "0e0" sign = "" if x < 0: sign = "-" x = -x f, e = math.frexp(x) assert 0.5 <= f < 1.0 # x = f * 2**e exactly # Suck up CHUNK bits at a time; 28 is enough so that we suck # up all bits in 2 iterations for all known binary double- # precision formats, and small enough to fit in an int. CHUNK = 28 top = 0L # invariant: x = (top + f) * 2**e exactly while f: f = math.ldexp(f, CHUNK) digit = int(f) assert digit >> CHUNK == 0 top = (top << CHUNK) | digit f -= digit assert 0.0 <= f < 1.0 e -= CHUNK assert top > 0 # Now x = top * 2**e exactly. Get rid of trailing 0 bits if e < 0 # (purely to increase efficiency a little later -- this loop can # be removed without changing the result). while e < 0 and top & 1 == 0: top >>= 1 e += 1 # Transform this into an equal value top' * 10**e'. if e > 0: top <<= e e = 0 elif e < 0: # Exact is top/2**-e. Multiply top and bottom by 5**-e to # get top*5**-e/10**-e = top*5**-e * 10**e top *= 5L**-e # Nuke trailing (decimal) zeroes. while 1: assert top > 0 newtop, rem = divmod(top, 10L) if rem: break top = newtop e += 1 return "%s%de%d" % (sign, top, e) [View Less]

3 5

mmap module
by Paul Barrett May 21, 2001

May 21, 2001

In the CVS log of the mmapmodule.c, Tim Peters says: "The code really needs to be rethought from scratch (not by me, though ...)." Well, I might be the person to do the rethinking, but I'd first like to know what Tim has in mind. I've been playing around with this module lately and tend to agree that some enhancements could be made, particularly to prevent "bus errors" and "segmentation faults". The ability to have offsets into a file that are not multiples of the system pagesize would also … [View More]

2 2

Performance compares
by M.-A. Lemburg May 20, 2001

May 20, 2001

After having read a little into the comparison thread, I tried some performance compares on my own: the one between the current CVS version and Python 1.5.2. Both versions were compiled on the same Linux machine, using the same GCC compiler and optimization settings. Here are the results from pybench 0.9 and pystone; some of the figures show quite dramatic slow-downs. I'm not sure where they result from, but they do concern me a bit, since the upgrade path from 1.5.2 is probably the most … [View More]common one to be expected in user-land. Since it is possible that these figures result from my specific machine setup, I'd like to know what other people see on their machines. Thanks. -- Python 1.5.2: Pystone(1.1) time for 10000 passes = 3.26 This machine benchmarks at 3067.48 pystones/second Python CVS: Pystone(1.1) time for 10000 passes = 4.43 This machine benchmarks at 2257.34 pystones/second -- PYBENCH 0.9 Benchmark: /home/lemburg/tmp/pybench-cvs-O.pyb (rounds=10, warp=20) Tests: per run per oper. diff *) ------------------------------------------------------------------------ BuiltinFunctionCalls: 1152.60 ms 9.04 us +64.70% BuiltinMethodLookup: 903.90 ms 1.72 us CompareFloats: 908.30 ms 2.02 us +40.94% CompareFloatsIntegers: 1276.25 ms 2.84 us +37.15% CompareIntegers: 1075.50 ms 1.19 us +21.09% CompareLongs: 989.40 ms 2.20 us +47.12% CompareStrings: 844.80 ms 2.25 us +33.99% CompareUnicode: 1018.65 ms 2.72 us n/a ConcatStrings: 1226.30 ms 8.18 us +92.56% ConcatUnicode: 1575.40 ms 10.50 us n/a CreateInstances: 2094.05 ms 49.86 us +101.86% CreateStringsWithConcat: 1515.75 ms 7.58 us +111.67% CreateUnicodeWithConcat: 1833.85 ms 9.17 us n/a DictCreation: 2795.30 ms 18.64 us +203.34% DictWithFloatKeys: 2285.70 ms 3.81 us +18.73% DictWithIntegerKeys: 1444.65 ms 2.41 us +58.53% DictWithStringKeys: 1262.60 ms 2.10 us +52.83% ForLoops: 989.95 ms 99.00 us -10.01% IfThenElse: 1232.45 ms 1.83 us +23.25% ListSlicing: 621.40 ms 177.54 us NestedForLoops: 986.60 ms 2.82 us +52.09% NormalClassAttribute: 1231.15 ms 2.05 us +36.70% NormalInstanceAttribute: 1114.15 ms 1.86 us +27.11% PythonFunctionCalls: 1251.25 ms 7.58 us +46.09% PythonMethodCalls: 1034.35 ms 13.79 us +42.19% Recursion: 922.15 ms 73.77 us +36.76% SecondImport: 1055.45 ms 42.22 us +100.47% SecondPackageImport: 1061.35 ms 42.45 us +96.31% SecondSubmoduleImport: 1292.35 ms 51.69 us +77.89% SimpleComplexArithmetic: 1748.00 ms 7.95 us +120.97% SimpleDictManipulation: 1172.85 ms 3.91 us +47.85% SimpleFloatArithmetic: 881.25 ms 1.60 us +12.30% SimpleIntFloatArithmetic: 833.80 ms 1.26 us SimpleIntegerArithmetic: 839.00 ms 1.27 us SimpleListManipulation: 1252.60 ms 4.64 us +69.37% SimpleLongArithmetic: 1360.65 ms 8.25 us +100.43% SmallLists: 2380.05 ms 9.33 us +116.72% SmallTuples: 1793.80 ms 7.47 us +101.52% SpecialClassAttribute: 1257.35 ms 2.10 us +37.91% SpecialInstanceAttribute: 1340.25 ms 2.23 us +21.13% StringMappings: 1601.50 ms 12.71 us n/a StringPredicates: 1059.70 ms 3.78 us n/a StringSlicing: 1235.90 ms 7.06 us +98.32% TryExcept: 1272.55 ms 0.85 us +28.39% TryRaiseExcept: 1383.45 ms 92.23 us +77.48% TupleSlicing: 1163.05 ms 11.08 us +75.29% UnicodeMappings: 1232.80 ms 68.49 us n/a UnicodePredicates: 1294.95 ms 5.76 us n/a UnicodeProperties: 1410.45 ms 7.05 us n/a UnicodeSlicing: 1296.80 ms 7.41 us n/a ------------------------------------------------------------------------ Average round time: 73388.00 ms n/a *) measured against: /home/lemburg/tmp/pybench-1.5.2-O.pyb (rounds=10, warp=20) (The compares not shown are below noise level (+-10%)) -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ [View Less]

3 7

Re: [Python-Dev] Comparison speed
by Michael Hudson May 20, 2001

May 20, 2001

"Tim Peters" <tim.one(a)home.com> writes: > Ah! Of course. string_compare is hardwired into lookdict_string. > This case may be important enough to merit a distinct > _PyString_Equal function, with just the stuff lookdict_string needs Or just inlining it all into lookdict_string, something like: Index: Objects/dictobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/dictobject.c,v retrieving revision 2.… [View More]90 diff -c -r2.90 dictobject.c *** Objects/dictobject.c 2001/05/19 07:04:38 2.90 --- Objects/dictobject.c 2001/05/20 11:51:28 *************** *** 279,286 **** register unsigned int mask = mp->ma_size-1; dictentry *ep0 = mp->ma_table; register dictentry *ep; - cmpfunc compare = PyString_Type.tp_compare; /* make sure this function doesn't have to handle non-string keys */ if (!PyString_Check(key)) { #ifdef SHOW_CONVERSION_COUNTS --- 279,287 ---- register unsigned int mask = mp->ma_size-1; dictentry *ep0 = mp->ma_table; register dictentry *ep; + #define S(s) ((PyStringObject*)(s)) + /* make sure this function doesn't have to handle non-string keys */ if (!PyString_Check(key)) { #ifdef SHOW_CONVERSION_COUNTS *************** *** 299,305 **** freeslot = ep; else { if (ep->me_hash == hash ! && compare(ep->me_key, key) == 0) { return ep; } freeslot = NULL; --- 300,308 ---- freeslot = ep; else { if (ep->me_hash == hash ! && S(ep->me_key)->ob_size == S(key)->ob_size ! && memcmp(S(ep->me_key)->ob_sval, ! S(key)->ob_sval,S(key)->ob_size) == 0) { return ep; } freeslot = NULL; *************** *** 318,324 **** if (ep->me_key == key || (ep->me_hash == hash && ep->me_key != dummy ! && compare(ep->me_key, key) == 0)) return ep; else if (ep->me_key == dummy && freeslot == NULL) freeslot = ep; --- 321,329 ---- if (ep->me_key == key || (ep->me_hash == hash && ep->me_key != dummy ! && S(ep->me_key)->ob_size == S(key)->ob_size ! && memcmp(S(ep->me_key)->ob_sval, ! S(key)->ob_sval,S(key)->ob_size) == 0)) return ep; else if (ep->me_key == dummy && freeslot == NULL) freeslot = ep; *************** *** 327,332 **** --- 332,339 ---- if (incr > mask) incr ^= mp->ma_poly; /* clears the highest bit */ } + + #undef S } /* (apologies for the use of the preprocessor...). I'll leave it to someone else to work out if this is a win or not... -- >> REVIEW OF THE YEAR, 2000 << It was shit. Give us another one. -- NTK Know, 2000-12-29, http://www.ntk.net/ [View Less]

2 1