Mailman 3 December 2003 - Python-Dev

Python-dev
by Skanigsvv 28 Dec '03

28 Dec '03

1 0

Re: [Python-checkins] python/dist/src/Objects listobject.c,2.172,2.173
by Jeremy Hylton 27 Dec '03

27 Dec '03

On Fri, 2003-12-26 at 14:05, jhylton(a)users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Objects > In directory sc8-pr-cvs1:/tmp/cvs-serv32521 > > Modified Files: > listobject.c > Log Message: > Revert previous two checkins to repair test failure. > > The special-case code that was removed could return a value indicating > success but leave an exception set. test_fileinput failed in a debug > build as a result. If there's some value to the optimization I just removed, let's discuss it on python-dev. I could have fixed the new code, but it seems a lot of hairy new code with magic constants that only covered a few corner cases. The checkin message pointed to a python-list discussion that didn't have much real evidence in it. Jeremy

2 1

2.3.3 Compilation warnings...
by Troels Therkelsen 27 Dec '03

27 Dec '03

Hey all, I don't know what the policy on compilation warnings are with the development of Python, but I'd imagine that it's "no warnings". I get the following warnings: gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes - I. -I./Include -DPy_BUILD_CORE -o Python/errors.o Python/errors.c Python/errors.c:602: warning: function declaration isn't a prototype gcc -pthread -Xlinker -export-dynamic -o python \ Modules/python.o \ libpython2.3.a -lpthread -ldl -lutil -lm libpython2.3.a(posixmodule.o): In function `posix_tmpnam': /usr/local/src/Python-2.3.3/./Modules/posixmodule.c:5833: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.3.a(posixmodule.o): In function `posix_tempnam': /usr/local/src/Python-2.3.3/./Modules/posixmodule.c:5788: the use of `tempnam' is dangerous, better use `mkstemp' building 'readline' extension gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -fno-strict-aliasin g -I. -I/usr/local/src/Python-2.3.3/./Include -I/usr/local/include -I/usr/local/ src/Python-2.3.3/Include -I/usr/local/src/Python-2.3.3 -c /usr/local/src/Python-2.3.3/Modules/readline.c -o build/temp.linux-i686-2.3/readline.o /usr/local/src/Python-2.3.3/Modules/readline.c: In function `flex_complete': /usr/local/src/Python-2.3.3/Modules/readline.c:583: warning: implicit declaration of function `completion_matches' /usr/local/src/Python-2.3.3/Modules/readline.c:583: warning: return makes pointer from integer without a cast I guess the warnings in posixmodule has always been there, I just haven't been paying much attention to it and I thought I might just as well report all warnings. System specs: $ uname -a Linux gateway 2.2.19 #22 Wed Jun 20 18:12:16 PDT 2001 i686 unknown $ gcc -v Reading specs from /usr/lib/gcc-lib/i386-slackware-linux/2.95.3/specs gcc version 2.95.3 20010315 (release) Regards, Troels Therkelsen

2 1

2.3.3: test_import wreaks havoc when it runs out of memory
by Jack Jansen 27 Dec '03

27 Dec '03

During testing 2.3.3 for MacOS9 I think I came across a bug somewhere in import, but I don't have the time to track it down (and there's a workaround), and it's probably only ever going to happen on OS9. Should I put it in the bugs database anyway? If test_import runs out of memory (probably in the test_module_with_large_stack test) it fails with a MemoryError (as it should). However, from that point on there is something fishy in the state of the interpreter: any later test that should simply fail with an ImportError gets a SystemError: NULL return without exception in stead. After a couple of these the interpreter either hangs completely or crashes (I think around the threading tests, but I'm not sure). Note that it's not a question of simply all memory being tied up: it seems test_import does release most of the memory used, and many tests also pass after test_import. -- Jack Jansen, <Jack.Jansen(a)cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

1 0

status of 2.3 branch for maintenance checkins
by Andrew MacIntyre 26 Dec '03

26 Dec '03

I don't recall any advice as to whether the 2.3 branch is again open for checkins post 2.3 - Is Jack still sorting out the Mac release? Regards, Andrew. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac(a)bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac(a)pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia

3 2

Another Strategy for Relative Import
by Chris and Ann Reedy 23 Dec '03

23 Dec '03

Ken Manheimer wrote: > > Even using the leading '.' syntax, it would be good to have a specific > identifier for the containing package, realized in each module and > respected as a special identifier by the package machinery. I like > '__pkg__': > > import __pkg__.sibling # == 'import .sibling' > > and > > import __pkg__.__pkg__.uncle # == 'import ..uncle' > import __pkg__.__pkg__.uncle.cousin # == 'import ..uncle.cousin' > > I think the leading '.' dot forms are a lot nicer, but the presence of > a module named '__pkg__' would be useful for programmatically mucking > navigating the package structure. Delurking: Knowing that the statement: import foo.bar has the effect of setting the variable foo to reference package foo, even if the variable foo already references something else, I was wondering if it would make sense to changes the semantics of import so that if the variable foo is already assigned and is a package (or maybe a module in some cases), then importing foo.bar would have the effect of importing the module or package bar within foo, using foo.__path__ to search for the module bar and foo.__name__ + '.bar' as the name of the module. It seems to me that such an approach might be used in a number of cases: 1. If there is a standard name for the parent of a package, e.g. __pkg__, then import__pkg__.bar just works. 2. This approach allows the following kind of thing to work: if <something>: import xyz as util else: import xyz_old as util import util.bar 3. This could be used to clean up the problem with os.path. The idea here is that since initialization of os creates the attribute os.path which references a module, then: import os.path would have the effect of importing os and then noting that os.path exists and is a module, so no further work is required. I haven't (yet?) attempted to work out whether this could be used to handle all the use cases for relative import. Chris P.S. I've been lurking on this list for a about six months. I just moved between coasts (US) and things are still a mess. I would like to get more involved with Python. I'll send a message in the near future introducing myself.

1 0

Christmas Wishlist
by Raymond Hettinger 23 Dec '03

23 Dec '03

I. eval() to accept custom mapping arguments for globals and locals. This makes it possible to write a smart __getitem__ method for applications like spreadsheets or case-insensitive evaluation. class LowerCaseDict(dict): def __getitem__(self, key): return dict.__getitem__(self, key.lower()) >>> print eval(raw_input('Okay kids, type in an expression:', LowerCaseDict(globals())) Okay kids, type in an expression: HeX(20) 0x14 class SpreadSheet: _cells = {} def __setitem__(self, key, formula): self._cells[key] = formula def __getitem__(self, key): return eval(self._cells[key], self) ss = SpreadSheet() ss['a1'] = '5' ss['a2'] = 'a1*5' ss['a2'] While this seems like a minor feature request, it presents amazing opportunities to avoid writing lexers, parsers, interpreters by transferring the work to the python interpreter. Writing little languages like the spreadsheet class becomes almost trivial. II. Timer.timeit to take an optional context argument. >>> from timeit import Timer >>> a = 20 >>> def f(x): ... return x + 20 >>> Timer('f(10)', env=globals()).timeit() 1.234 The idea is to make it *much* easier to time functions or parts of existing modules. Having to write-out the definition in a long setup string is cumbersome and interferes with syntax highlighting. To instrument modules with timeit, the timer setup string currently has to re-import the whole module -- passing in an execution environment is much more straight-forward. And, as the example shows, passing in globals() makes it easier to experiment with timings using the interpreter. III. enumerate to take an optional argument: enumerate([firstnumber=0,] iterable) The idea is to make enumerate() a complete solution to the loop counter problem: for lineno, line in enumerate(1, file('hist.log')): print lineno, line IV. list.sorted() to become just sorted(). All of the common use cases read better without the "list." prefix: >>> topscores = sorted(team.score for team in teams)[:10] >>> for k, v in sorted(mydict.iteritems()): . . . >>> byAge = operator.itemgetter('age') >>> for age, students in groupby(sorted(studentbody, key=byAge), key=byAge): ... print "Age group:", age ... for student in students: ... print '...', student.name, student.address Also, this Christmas wish cures the weirdness that comes from the classmethod approach: >>> [3,2,1].sorted('cbs') ['b', 'c', 's']. While I worry about the total number of builtins, opposing them on principle is a recipe for putting tools where they don't belong. IMO, making list.sorted() a class method was a clumsy way of avoiding the obvious solution and left it smelling a bit hackish. Raymond Hettinger

35 110

getting rid of default object.__hash__ (SF 660098)
by Guido van Rossum 23 Dec '03

23 Dec '03

New-style classes inherit a default __hash__ from object, but this is incorrect if the inheriting object defines __eq__ or __cmp__. This has been reported several times; the roll-up SF entry is 660098, which has the proposed patch attached as newpatch.txt. I've long pondered this, and it seems the best solution is to simply not define __hash__ on the base object class. After all, you don't inherit __eq__ or __cmp__ either; the default comparison behavior is supplied by intrinsic behavior of comparisons, and the default hash() behavior is also supplied by intrinsic behavior of hash(). Does anybody have any reason why I shouldn't check in this change in 2.4? There's no unit test that fails if I do this. It *is* a slight incompatibility though: if someone implemented their __hash__ in terms of object.__hash__, they would get a somewhat puzzling error message after the change, because object.__hash__ will refer to the type.__hash__ function bound to object, and refuse to take an argument: >>> object.__hash__() 135328576 >>> object.__hash__(42) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: expected 0 arguments, got 1 >>> Since the default hash simply takes id() of the object, it's easy to fix such code once the failure is understood though. --Guido van Rossum (home page: http://www.python.org/~guido/)

3 4

Re: [Python-Dev] OpenBSD anyone?
by kbk＠shore.net 23 Dec '03

23 Dec '03

Guido van Rossum <guido(a)python.org> writes: > Very interesting! Had you done a previous non-debug build in the same > directory? (Even if you did a "make clobber" before restarting -- you > never know what leave-behind could cause this.) Yes. I did do a make clobber && ./configure before building the debug version, but it's not a clean checkout. OK, just did a clean checkout, no patch, and got the same result. > The next thing I'd try would be to start the python executable that >was build under gdb and play around in it, like this: > > $ gdb ./python > [...] > (gdb) run > Starting program: /home/guido/projects/python/dist/src/debug/python > [New Thread 1074895104 (LWP 30404)] > Python 2.4a0 (#2, Dec 22 2003, 11:02:19) > [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> print 2+2 >>>> # try various things that don't need external modules except sys >>>> # define a function, etc... >>>> # if that doesn't segfault, try: >>>> from test import autotest > [...] > > Eventually I expect you'd get a segfault; at that point you can use > the gdb 'bt' command to get a stack trace. Hopefully it'll point to > some innocent C code that gets mistreated by the non-debug compiler... It doesn't get as far as the banner: Script started on Mon Dec 22 14:42:19 2003 hydra /home/kbk/proj/sandbox/python_clean$ gdb ./python GNU gdb 4.16.1 Copyright 1996 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-openbsd3.3"... (gdb) run Starting program: /home/kbk/proj/sandbox/python_clean/./python Program received signal SIGSEGV, Segmentation fault. 0x401900a0 in strchr () (gdb) quit The program is running. Quit anyway (and kill it)? (y or n) y hydra /home/kbk/proj/sandbox/python_clean$ Script done on Mon Dec 22 14:42:38 2003 Investigating. -- KBK

4 7

bug in python arm-linux?: start_new_thread fails after popen
by Michael Lauer 23 Dec '03

23 Dec '03

Hi, I have a python program which works fine on x86 but doesn't work on any of my arm-linux devices (Zaurus, Ipaq, SIMpad) - all of them are running Python 2.3.2 on top of glibc 2.3.2+linuxthreads on top of kernel 2.4.18-rmk6-pxa3 respectively kernel 2.4.19-rmk7. After a long week of debugging I now can reproduce the behaviour with the following minimal case: ---------------------------------------------------------------------- import thread from time import sleep import sys import os def createPipe(): print os.popen( "ls -l" ).read() def threadMain( name ): while True: sys.stderr.write( name+": i'm still running" ) sys.stderr.flush() sleep( 1 ) if __name__ == '__main__': createPipe() print "BEFORE start_new_thread" thread.start_new_thread( threadMain, ( "1", ) ) print "AFTER start_new_thread" createPipe() print "BEFORE start_new_thread" thread.start_new_thread( threadMain, ( "2", ) ) print "AFTER start_new_thread" sleep( 5 ) ----------------------------------------------------------------------- This program - as is - just hangs in the first start_new_thread() and never comes back. If you outcomment the first call to createPipe() out, then it works fine. In the first - hanging - case, an strace shows: ------------------------------------------------------------------------- write(1, "BEFORE start_new_thread\n", 24BEFORE start_new_thread ) = 24 rt_sigprocmask(SIG_BLOCK, ~[33], [RTMIN], 8) = 0 pipe([3, 4]) = 0 clone(child_stack=0x14e080, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND) = 949 write(4, "\314V\1@\5\0\0\0\0\0\0\0\0\0\0\0\34\213\37@\10c\1@\0\0"..., 148) = 148 rt_sigprocmask(SIG_SETMASK, NULL, ~[KILL STOP 33], 8) = 0 write(4, "\0\21\3@\0\0\0\0\0\0\0\0hO\n\0\2503\23\0\377\376\373\377"..., 148) = 148 rt_sigprocmask(SIG_SETMASK, NULL, ~[KILL STOP 33], 8) = 0 rt_sigsuspend(~[KILL STOP RTMIN 33] -------------------------------------------------------------------------- The program must be kill -9'ed at this point. Can anyone explain this behaviour to me - did I find a bug? Sincerely, -- :M: -------------------------------------------------------------------------- Dipl.-Inf. Michael 'Mickey' Lauer mickey(a)tm.informatik.uni-frankfurt.de Raum 10b - ++49 69 798 28358 Fachbereich Informatik und Biologie --------------------------------------------------------------------------

7 22