Mailman 3 November 2002 - Python-Dev

Currently baking idea for dict.sequpdate(iterable, value=True)
by Raymond Hettinger 26 Nov '02

26 Nov '02

Here's a write-up for a proposed dictionary update method. Comments are invited while I work on a patch, docs, tests, etc. Raymond Hettinger ------------------------------------------------------------------------ METHOD SPECIFICATION def sequpdate(self, iterable, value=True): for elem in iterable: self[elem] = value return self USE CASES # Fast Membership testing termwords = dict('End Quit Stop Abort'.split()) . . . if lexeme in termwords: sys.exit(0) # Removing duplicates seq = dict().sequpdate(seq).keys() # Initializing or resetting mapped accumulators absences = dict('Tom Dick Harry'.split(), 0) for name, date in classlog: absences[name] += 1 # Intentionally raises KeyError for invalid names RATIONALE Examining code in the library, the cookbook, and the vaults of parnassus, the most common practice for membership testing is a sequential search of a list, tuple, or string. Where the writers were sophisticated and cared about performance, they used in-line versions of the python fragment shown above. I have not found a single case where people took the time to "roll their own" pure python function as shown above. Even if they had, the construction time would take twice as long as with a C coded method. IOW, practices are tending toward slow or verbose approaches in the absence of a quick, convenient, standardized tool for moving sequences into dictionaries. The problem of removing duplicates from a sequence is most frequently solved with inline use of a code fragment similar to the one shown above. Again, you almost never see people taking the time to build their own uniq function, factoring all the effort into a single, tested, reusable code fragment. The exception is Tim's wonderful, general purpose recipe for uniquifying anything -- unfortunately, I don't think you ever see his code used in practice. Also, since all containers support __iter__, they can be fed to list(); however, in the absence of the above method, they cannot be fed to dict() without shenanigans for building item lists. The proposed method makes todict() as doable as tolist() and encourages switching to appropriate data structures. OBJECTIONS TO THE PRIOR IDEA OF EXPANDING THE DICT CONSTRUCTOR (SF 575224) 1. The constructor was already overloaded. 2. Weird syntax was needed to fit with other constructor syntax. 3. The sets module was going to meet all needs. The first two comments led to this revision in method form. The Sets module met several needs centering around set mathematics; however, for membership testing, it is so slow that it is almost always preferable to use dictionaries instead (even without this proposed method). The slowness is intrinsic because of the time to search for the __contains__ method in the class and the time to setup a try/except to handle mutable elements. Another reason to prefer dictionaries is that there is one less thing to import and expect readers to understand. My experiences applying the Sets module indicates that it will *never* replace dictionaries for membership testing and will have only infrequent use for uniquification.

8 14

Dictionary evaluation order
by Gustavo Niemeyer 26 Nov '02

26 Nov '02

I was just looking at the bug [#448679] Left to right It mentions that code like that {f1():f2(), f3():f4()} Will call these functions in the order f2, f1, f4, f3. What should we do about it? Tim mentions that "When [Tim] asked Guido about that some years ago, he agreed it was a bug.". Is it too late to fix it, or is it still a desirable fix? The fix should be as easy as that: diff -u -r2.264 compile.c --- Python/compile.c 3 Oct 2002 09:50:47 -0000 2.264 +++ Python/compile.c 26 Nov 2002 04:02:27 -0000 @@ -1527,9 +1527,9 @@ It wants the stack to look like (value) (dict) (key) */ com_addbyte(c, DUP_TOP); com_push(c, 1); - com_node(c, CHILD(n, i+2)); /* value */ - com_addbyte(c, ROT_TWO); com_node(c, CHILD(n, i)); /* key */ + com_node(c, CHILD(n, i+2)); /* value */ + com_addbyte(c, ROT_THREE); com_addbyte(c, STORE_SUBSCR); com_pop(c, 3); } (compiler module should be fixed as well, as it mimicks that behavior) So I belive it just a matter of deciding what should be done. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

5 6

Removing support for little used platforms
by martin＠v.loewis.de 26 Nov '02

26 Nov '02

I'm going to add error messages for all platforms that become unsupported in Python 2.3 really soon. If you have any corrections to the list in PEP 11, please let me know. Regards, Martin

4 3

Re: [Python-checkins] python/dist/src/Misc NEWS,1.542,1.543
by Andrew Kuchling 26 Nov '02

26 Nov '02

On Tue, Nov 26, 2002 at 01:28:07AM -0800, loewis(a)users.sourceforge.net wrote: >+ - _tkinter now returns Tcl objects, instead of strings. Objects which >+ have Python equivalents are converted to Python objects, other objects >+ are wrapped. This can be configured through the wantobjects method, >+ or Tkinter.want_objects. Will it be confusing to have the method be named wantobjects but the module variable named want_objects? (Glancing at the C code, it seems to use only want_objects as a variable name, so the method name is the only inconsistent name.) --amk (www.amk.ca) EDGAR: The worst is not, so long as we can say "This is the worst." -- _King Lear_, IV, i

2 2

Sanity-check wanted: bug #641685
by Andrew Kuchling 26 Nov '02

26 Nov '02

I have a patch (setup.patch) attached to bug #641685 that removes some code duplication in Python's setup.py, which has its own find_library_file() function instead of using the CCompiler.find_library_file() method offered by Distutils. Can someone please review the patch before I check it in? --amk (www.amk.ca) LEAR: Vengeance! plague! death! confusion! -- _King Lear_, II, iv

1 0

Re: [Python-Dev] Re: release22-maint branch broken
by Michael Hudson 26 Nov '02

26 Nov '02

Tim Rice <tim(a)multitalents.net> writes: > On Fri, 22 Nov 2002, A.M. Kuchling wrote: > > > Tim Rice wrote: > > > > > raise DistutilsPlatformError(my_msg) > > > distutils.errors.DistutilsPlatformError: invalid Python installation: > > > unable to open /usr/local/lib/python2.2/config/Makefile (No such file > > > or directory) > > > gmake: *** [sharedmods] Error 1 > > > > The revised version of sysconfig.py figures out if it's in the build > > directory by looking for a landmark file; the landmark is Modules/Setup. > > Does that file exist? > > Yes it does. > > I put some prints in > ... > argv0_path = os.path.dirname(os.path.abspath(sys.executable)) > print argv0_path > landmark = os.path.join(argv0_path, "Modules", "Setup") > print landmark > if not os.path.isfile(landmark): > python_build = 0 > print "python_build = 0" > elif os.path.isfile(os.path.join(argv0_path, "Lib", "os.py")): > python_build = 1 > print "python_build = 1" > else: > python_build = os.path.isfile(os.path.join(os.path.dirname(argv0_path), > "Lib", "os.py")) > print "else" > print python_build > del argv0_path, landmark > ... > > And get > ... > /usr/local/src/utils/Python-2 > /usr/local/src/utils/Python-2/Modules/Setup > else > 0 > running build > ... > > Could this breaking because I build outside of the source tree? I guess so. Where are you building? Why is sys.executable /usr/local/src/utils/Python-2/python? At least it looks like that's what's happened. Can you try (in your build directory) $ ./python ... >>> print sys.executable and see if that looks reasonable? Cheers, M. -- Monte Carlo sampling is no way to understand code. -- Gordon McMillan, comp.lang.python

2 3

Re: [Python-checkins] python/dist/src/Lib/test test_descr.py,1.161,1.162
by Michael Hudson 26 Nov '02

26 Nov '02

gvanrossum(a)users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Lib/test > In directory sc8-pr-cvs1:/tmp/cvs-serv18947 > > Modified Files: > test_descr.py > Log Message: > A tweaked version of Jeremy's patch #642489, to produce better error > messages about MRO conflicts. (Tweaks here: don't print the message, > but compare it with an expected string.) [...] > + raises(TypeError, "MRO conflict among bases B, A", > + type, "X", (A, B), {}) Unfortunately, the order of the bases in the error message depends on the order they come out of a dict. As we all know, this is not a recipe for happiness... Not sure what to do about this; I'm just going to comment this test out in my checkout so I can get on with my stuff. Shout at me if I check the test in like that! Cheers, M. -- There are two kinds of large software systems: those that evolved from small systems and those that don't work. -- Seen on slashdot.org, then quoted by amk

2 1

Some bugs in Python? Is this reportable?
by John D. 26 Nov '02

26 Nov '02

This Python "feature" has been here for so long, I assumed it was supposed to work this way... Problem is when a control-c is typed, it does a CORE DUMP! OpenBSD 3.2, 3.1, 3.0, 2.9, and beyond. i386 build. Standard PC. Python 2.2.2, 2.2.1, 2.1, and beyond. Standard build. Python 2.2.2 (#1, Nov 20 2002, 02:13:00) [GCC 2.95.3 20010125 (prerelease)] on openbsd3 Type "help", "copyright", "credits" or "license" for more information. >>> >>> pid 1234: Fatal error '_pq_insert_tail: Already in priority queue' at line 196 in file /usr/src/lib/libc_r/uthread/uthread_priority_queue.c (errno = 4) Abort (core dumped) John

2 1

Windows builders?
by Mark Hammond 25 Nov '02

25 Nov '02

I'd like some help from a Windows builder. I have a patch to add SSL support, and while I have completed the patch, I would like some help and feedback on how well I did ;) Basically I will just ask you to try my patches as they stand, and ensure you get a nice clean failure building SSL. Then install SSL itself, and make sure my patch then finds and builds it before finally building the .pyd. Mail me if you can help. It shouldn't take too long. Thanks, Mark.

1 0

urllib performance issue on FreeBSD 4.x
by Andrew MacIntyre 25 Nov '02

25 Nov '02

I've been following up a thread on python-list about lousy performance of urllib.urlopen(...).read() on FreeBSD 4.x comparted to using wget to retrieve the same file. I've determined that the following patch (against 2.2.2) makes an enormous difference in throughput: -----8<-----8<-----8<----- *** Lib/httplib.py.orig Mon Oct 7 11:18:17 2002 --- Lib/httplib.py Sun Nov 24 14:44:16 2002 *************** *** 210,216 **** # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details. def __init__(self, sock, debuglevel=0, strict=0): ! self.fp = sock.makefile('rb', 0) self.debuglevel = debuglevel self.strict = strict --- 210,216 ---- # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details. def __init__(self, sock, debuglevel=0, strict=0): ! self.fp = sock.makefile('rb', -1) self.debuglevel = debuglevel self.strict = strict -----8<-----8<-----8<----- Without this patch, d/l a 4MB file from localhost gets a bit over 110kB/s, with the patch gets 4-5.5MB/s on the same system (FBSD 4.4 SMP, 2xC300A, 128MB RAM, ATA66 HD). My question: - why is the socket.fp being set to unbuffered? I can't check the FBSD library source at the moment (and can't get to the RFC's mentioned above either at the moment for that matter), and can only speculate that fread() is resorting to reading from the socket a character at a time. So I'm not sure whether this should be treated as a FreeBSD issue or/and a Python issue. Another poster in the same thread mentions seeing somewhat similar performance problems on Win2k, although not nearly as bad. FWIW, my test script is -----8<-----8<-----8<----- import time import urllib t1 = time.time() u = urllib.urlopen("http://localhost/big_file").read() t2 = time.time() print 'throughput: %f kB/s' % (len(u) / (t2 - t1)) -----8<-----8<-----8<----- Reactions? -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac(a)bullseye.apana.org.au | Snail: PO Box 370 andymac(a)pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia

4 4