Python-Dev October 2004

python-dev@python.org

105 participants
98 discussions

by Neil Schemenauer

Perhaps this is more approprate for python-list but I looks like a bug to me. Example code: class A: def __str__(self): return u'\u1234' '%s' % u'\u1234' # this works '%s' % A() # this doesn't work It will work if 'A' subclasses from 'unicode' but should not be necessary, IMHO. Any reason why this shouldn't be fixed? Neil

16 years, 1 month

RE: [Python-Dev] Those import related syntax errors again...

by Samuele Pedroni

Hi. [Mark Hammond] > The point isn't about my suffering as such. The point is more that > python-dev owns a tiny amount of the code out there, and I don't believe we > should put Python's users through this. > > Sure - I would be happy to "upgrade" all the win32all code, no problem. I > am also happy to live in the bleeding edge and take some pain that will > cause. > > The issue is simply the user base, and giving Python a reputation of not > being able to painlessly upgrade even dot revisions. I agree with all this. [As I imagined explicit syntax did not catch up and would require lot of discussions.] [GvR] > > Another way is to use special rules > > (similar to those for class defs), e.g. having > > > > <frag> > > y=3 > > def f(): > > exec "y=2" > > def g(): > > return y > > return g() > > > > print f() > > </frag> > > > > # print 3. > > > > Is that confusing for users? maybe they will more naturally expect 2 > > as outcome (given nested scopes). > > This seems the best compromise to me. It will lead to the least > broken code, because this is the behavior that we had before nested > scopes! It is also quite easy to implement given the current > implementation, I believe. > > Maybe we could introduce a warning rather than an error for this > situation though, because even if this behavior is clearly documented, > it will still be confusing to some, so it is better if we outlaw it in > some future version. > Yes this can be easy to implement but more confusing situations can arise: <frag> y=3 def f(): y=9 exec "y=2" def g(): return y return y,g() print f() </frag> What should this print? the situation leads not to a canonical solution as class def scopes. or <frag> def f(): from foo import * def g(): return y return g() print f() </frag> [Mark Hammond] > > This probably won't be a very popular suggestion, but how about pulling > > nested scopes (I assume they are at the root of the problem) > > until this can be solved cleanly? > > Agreed. While I think nested scopes are kinda cool, I have lived without > them, and really without missing them, for years. At the moment the cure > appears worse then the symptoms in at least a few cases. If nothing else, > it compromises the elegant simplicity of Python that drew me here in the > first place! > > Assuming that people really _do_ want this feature, IMO the bar should be > raised so there are _zero_ backward compatibility issues. I don't say anything about pulling nested scopes (I don't think my opinion can change things in this respect) but I should insist that without explicit syntax IMO raising the bar has a too high impl cost (both performance and complexity) or creates confusion. [Andrew Kuchling] > >Assuming that people really _do_ want this feature, IMO the bar should be > >raised so there are _zero_ backward compatibility issues. > > Even at the cost of additional implementation complexity? At the cost > of having to learn "scopes are nested, unless you do these two things > in which case they're not"? > > Let's not waffle. If nested scopes are worth doing, they're worth > breaking code. Either leave exec and from..import illegal, or back > out nested scopes, or think of some better solution, but let's not > introduce complicated backward compatibility hacks. IMO breaking code would be ok if we issue warnings today and implement nested scopes issuing errors tomorrow. But this is simply a statement about principles and raised impression. IMO import * in an inner scope should end up being an error, not sure about 'exec's. We will need a final BDFL statement. regards, Samuele Pedroni.

16 years, 1 month

mimetypes and _winreg

by Mike Brown

I thought it would be nice to try to improve the mimetypes module by having it, on Windows, query the Registry to get the mapping of filename extensions to media types, since the mimetypes code currently just blindly checks posix-specific paths for httpd-style mapping files. However, it seems that the way to get mappings from the Windows registry is excessively slow in Python. I'm told that the reason has to do with the limited subset of APIs that are exposed in the _winreg module. I think it is that EnumKey(key, index) is querying for the entire list of subkeys for the given key every time you call it. Or something. Whatever the situation is, the code I tried below is way slower than I think it ought to be. Does anyone have any suggestions (besides "write it in C")? Could _winreg possibly be improved to provide an iterator or better interface to get the subkeys? (or certain ones? There are a lot of keys under HKEY_CLASSES_ROOT, and I only need the ones that start with a period). Should I file this as a feature request? Thanks -Mike from _winreg import HKEY_CLASSES_ROOT, OpenKey, EnumKey, QueryValueEx i = 0 typemap = {} try: while 1: subkeyname = EnumKey(HKEY_CLASSES_ROOT, i) try: subkey = OpenKey(HKEY_CLASSES_ROOT, subkeyname) if subkeyname[:1] == '.': data = QueryValueEx(subkey, 'Content Type')[0] print subkeyname, '=', data typemap[subkeyname] = data # data will be unicode except EnvironmentError, WindowsError: pass i += 1 except WindowsError: pass

16 years, 2 months

Minor cookielib changes for 2.4b1

by John J Lee

Would anyone care to check this cookielib.py patch in before 2.4b1? http://python.org/sf/1028908 Quote from above: The patch contains uncontroversial changes to cookielib and associated modules, documentation and tests. Would be unfortunate not to have these in 2.4.0. [...SNIP...] John

16 years, 5 months

Bug [ 959379 ] Implicit close() should check for errors

by Peter Astrand

I'd like to resolve bug 959379. Here's a naive fix. Comments? diff -u -r2.192 fileobject.c --- Objects/fileobject.c 11 Jun 2004 04:49:03 -0000 2.192 +++ Objects/fileobject.c 24 Oct 2004 20:58:49 -0000 @@ -304,7 +304,8 @@ PyObject_ClearWeakRefs((PyObject *) f); if (f->f_fp != NULL && f->f_close != NULL) { Py_BEGIN_ALLOW_THREADS - (*f->f_close)(f->f_fp); + if ((*f->f_close)(f->f_fp) != 0) + perror("close failed"); Py_END_ALLOW_THREADS } PyMem_Free(f->f_setbuf); /Peter Åstrand <astrand(a)lysator.liu.se>

16 years, 5 months

Tests broken on Windows

by Tim Peters

Current test_shutil is apparently trying to do something that can't work on Windows, so it fails: test_shutil test test_shutil failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test_shutil.py", line 28, in test_on_error shutil.rmtree(TESTFN, onerror=self.check_args_to_onerror) File "C:\Code\python\lib\shutil.py", line 168, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "C:\Code\python\lib\test\test_shutil.py", line 36, in check_args_to_onerror self.assertEqual(func, os.remove) AssertionError: <built-in function rmdir> != <built-in function remove> 1 test failed: test_shutil After that, it leaves behind temp directories that cause other tests to fail. Sorry, I've got other problems on my stack, and can't make time for this one.

16 years, 5 months

Windows: PyBanner048.gif damage repaired

by Tim Peters

Lib\email\test\data\PyBanner048.gif wasn't marked binary in CVS. It is now. If you're on Windows, delete your copy, then cvs up to refetch an unmangled copy.

16 years, 5 months

Pickle breakage with reduce of recursive structures (was: AssertionError in pickle's memoize function)

by Dima Dorfman

[Followups to python-dev, please.] [Michael Hohn] > > > under python 2.2, the pickle/unpickle sequence incorrectly restores > > > a larger data structure I have. > > > > > > Under Python 2.3, these structures now give an explicit exception from > > > Pickle.memoize(): > > > assert id(obj) not in self.memo > > > [Tim Peters] > > Assertions are never expected to fail, so "something impossible > > happened" when they do fail. [Michael Hohn] > Here is a code sample that shows the problem I ran into: Summary for the OP: This is a bug in Python. Using cPickle won't help, but if you don't subclass builtin container types others than list, dict, and tuple, using pickle protocol 2 should work. The rest of this message is for python-dev. The simplest breaking case is: t = type('t', (list,), {}) obj = t() obj.append(obj) pickle.dumps(obj) [infinite recursion] The subclass causes save_reduce to be used instead of save_list. For proto < 2, copy_reg._reduce_ex returns (_reconstructor, list(a)), and the args--list(a)--cycle back through obj. Initially it looks like this should be okay, but the args are saved before obj is memoized, and obj can't be memoized until REDUCE can be executed with the args--and there's the cycle. It is even more obviously impossible from the unpickler's perspective because it has to call _reconstructor([obj]) to create obj! There are two separate problems: 1. Any __reduce__ implementation that returns args that cycle back through the object it tried to reduce hasn't done its job. As described above, _reduce_ex is one such implementation. reduce_2 avoids this by using the listitems and dictitems parameters. Since that's a pickler-side feature it can be used in _reduce_ex too. The basetype(obj) hook (documented in PEP 307) would remain for immutable bases; it doesn't work for containers, but user-defined containers already have to implement their own reduce functions. POC patch: http://www.trit.org/~dima/home/reduce_ex.diff At least the set and deque types also have this problem. 2. The pickle implementations don't detect reduction cycles. Pickling an instance of this obviously broken class causes an infinite recursion: class evil(object): def __reduce__(self): return evil, (self,) It's easy to detect this case. POC patch for the pickle module: http://www.trit.org/~dima/home/redcycle.diff BTW, the failed assert the OP is seeing happens when the cycle goes through another object: t = type('t', (list,), {}) obj = t() d = {'obj': obj} obj.append(d) pickle.dumps(obj) [AssertionError] cPickle has the same problem, but it lacks the assert, so it writes garbage instead: new = cPickle.loads(cPickle.dumps(obj)) new[0]['obj'] is new -> False # wrong obj[0]['obj'] is obj -> True # right This makes the reduction cycle check (#2 above) more than just cosmetic since if cPickle had that assert (it should) it would've been a crash. Right now it's garbage output instead, which is arguably worse. Formally complete versions of the above patches will be on SF tomorrow unless someone suggests better alternatives. Dima.

16 years, 5 months

Weekly Python Patch/Bug Summary

by Kurt B. Kaiser

Patch / Bug Summary ___________________ Patches : 243 open ( +3) / 2678 closed (+17) / 2921 total (+20) Bugs : 789 open (+17) / 4545 closed (+17) / 5334 total (+34) RFE : 156 open ( +1) / 131 closed ( +0) / 287 total ( +1) New / Reopened Patches ______________________ Fix various x86_64 build issues (2004-10-19) http://python.org/sf/1050475 opened by Bob Halley make distutils.core.run_setup re-entrant (2004-10-20) http://python.org/sf/1051216 opened by Bob Ippolito xml.dom missing API docs (bugs 1010196, 1013525) (2004-10-21) http://python.org/sf/1051321 opened by Mike Brown locale.getdefaultlocale does not return tuple in some OS (2004-10-21) http://python.org/sf/1051395 opened by Jiwon Seo Syntax error in Misc/NEWS (2004-10-22) CLOSED http://python.org/sf/1051866 opened by Felix Wiemann plistlib: add plst resource functionality, fix bugs (2004-10-22) CLOSED http://python.org/sf/1052399 opened by Bob Ippolito urllib2: better import ftplib and gopherlib etc late (2004-10-24) http://python.org/sf/1053150 opened by Robert Kiendl nntplib: add support for NNTP over SSL (2004-10-24) http://python.org/sf/1053365 opened by Reinhard Speyerer ftplib: add support for MDTM command (2004-10-24) http://python.org/sf/1053369 opened by Reinhard Speyerer Decimal speed up (2004-10-24) CLOSED http://python.org/sf/1053375 opened by Facundo Batista mingw compile (2004-10-25) http://python.org/sf/1053879 opened by Matthias Kramm fix inplace string concat hash error (2004-10-25) CLOSED http://python.org/sf/1054197 opened by Bob Halley automatic XMLRPC boxcarring via multicall for xmlrpclib (2004-10-26) http://python.org/sf/1054434 opened by Eric Ries Fix for Unix macro expansion in citetitle (2004-10-26) http://python.org/sf/1054715 opened by Richard Brodie bdist_deb - Debian packager (2004-10-26) http://python.org/sf/1054967 opened by Geoffrey T. Dairiki CGIHTTPServer can't redirect (2004-10-27) http://python.org/sf/1055159 opened by Thomas Guettler cgi.py does not correctly handle fields with ';' (2004-10-27) http://python.org/sf/1055234 opened by Vova typo in comment (unicodeobject.h) (2004-10-28) http://python.org/sf/1056231 opened by Baris Metin python / pythonw replacement in C (2004-10-29) http://python.org/sf/1056561 opened by Michiel de Hoon No ValueError for safe_substitute() (2004-10-29) http://python.org/sf/1056967 opened by Barry A. Warsaw Patches Closed ______________ __main__ for whichdb.py (2004-10-01) http://python.org/sf/1038388 closed by rhettinger Bug fixes and cleanup for decimal.py (2004-10-17) http://python.org/sf/1048728 closed by rhettinger tarfile.py: patch for bug #1017553 (2004-10-10) http://python.org/sf/1043972 closed by akuchling replace() method in unicode objects (2003-02-05) http://python.org/sf/680863 closed by nnorwitz Remove __file__ after running $PYTHONSTARTUP (2003-04-11) http://python.org/sf/719777 closed by nnorwitz sre fixes for lastindex and minimizing repeats+assertions (2003-03-31) http://python.org/sf/712900 closed by nnorwitz Some bug fixes for regular ex code. (2003-04-14) http://python.org/sf/720991 closed by nnorwitz optimize eval_frame (2003-12-21) http://python.org/sf/864059 closed by nnorwitz Implementation for PEP 318 ([as classmethod] version) (2004-04-08) http://python.org/sf/932100 closed by nnorwitz Add FreeBSD* to configure.in for --enable-shared (2004-10-11) http://python.org/sf/1044395 closed by perky New patch for Python 2.3 spec file. (2004-07-23) http://python.org/sf/996316 closed by jafo Syntax error in Misc/NEWS (2004-10-21) http://python.org/sf/1051866 closed by bcannon plistlib: add plst resource functionality, fix bugs (2004-10-22) http://python.org/sf/1052399 closed by jvr Decimal speed up (2004-10-24) http://python.org/sf/1053375 closed by rhettinger fix inplace string concat hash error (2004-10-25) http://python.org/sf/1054197 closed by rhettinger traceback module caches sources invalid (2003-05-14) http://python.org/sf/737473 closed by perky do not add directory of sys.argv[0] into sys.path (2004-05-02) http://python.org/sf/946373 closed by wrobell New / Reopened Bugs ___________________ 2.4b1 Win32 installer eliminates 2.3 assoc (2004-10-20) http://python.org/sf/1050486 opened by Mike C. Fletcher reindent.py strips execute permission on Unix (2004-10-20) http://python.org/sf/1050828 opened by David Ripton incorrect traceback filename from pyc (2004-10-21) http://python.org/sf/1051638 opened by Kevin Quick HTMLParser doesn't treat endtags in <script> tags as CDATA (2004-10-21) http://python.org/sf/1051840 opened by Luke Bradley Appending non-empty element to a non-empty list yields None (2004-10-22) CLOSED http://python.org/sf/1052036 opened by Peer Janssen Seting defaultencoding through an environment variable (2004-10-22) http://python.org/sf/1052098 opened by Diedrich Vorberg page on python api down (2004-10-22) CLOSED http://python.org/sf/1052274 opened by mperpick pdb runcall should accept keyword arguments (2004-10-22) CLOSED http://python.org/sf/1052503 opened by Leonardo Rochael Almeida filelist.findall should use os.lstat (2004-10-23) http://python.org/sf/1052827 opened by Geoffrey T. Dairiki time module returns wrong timezone under windows (2004-10-25) http://python.org/sf/1053539 reopened by panjunyong time module returns wrong timezone under windows (2004-10-25) http://python.org/sf/1053539 opened by panjunyong Unjustified SyntaxWarning (2004-10-25) CLOSED http://python.org/sf/1053604 opened by Markus Gritsch Why is this expression = False? (2004-10-25) CLOSED http://python.org/sf/1053629 opened by Alexander Rødseth PyOS_InputHook not called in subprocess (2004-10-25) http://python.org/sf/1053687 opened by Michiel de Hoon Segfault in tuple_of_constants (2004-10-25) CLOSED http://python.org/sf/1053819 opened by Duncan Grisby help() nor in tutorial index (2004-10-25) CLOSED http://python.org/sf/1053956 opened by Guido van Rossum Python doesn't exit with proper resultcode on SIGINT (2004-10-25) http://python.org/sf/1054041 opened by James Y Knight serious string hashing error in 2.4b1 (2004-10-25) CLOSED http://python.org/sf/1054139 opened by Bob Halley RE '*.?' cores if len of found string exceeds 10000 (2004-10-26) http://python.org/sf/1054564 opened by Rob New failure in debug build (2004-10-26) http://python.org/sf/1054615 opened by Tim Peters difflib HtmlDiff() extra space on inserted 1 character lines (2004-10-26) http://python.org/sf/1054821 opened by Dan Gass Python may contain NFC/NFKC bug per Unicode PRI #29 (2004-10-26) http://python.org/sf/1054943 opened by Rick McGowan pdb.set_trace should go up one more frame (2004-10-27) http://python.org/sf/1055168 opened by Johannes Gijsbers bdist_wininst broken for pure Python distributions (2004-10-27) CLOSED http://python.org/sf/1055540 opened by Thomas Heller weakref callback vs gc vs threads (2004-10-27) http://python.org/sf/1055820 opened by Tim Peters HTMLParser not compliant to XHTML spec (2004-10-27) http://python.org/sf/1055864 opened by Luke Bradley dir() should only return strings (2004-10-28) http://python.org/sf/1056293 opened by Skip Montanaro subprocess fails for args="...", executable="..." (2004-10-28) http://python.org/sf/1056441 opened by Russell Owen Lambda in list comprehension (2004-10-28) CLOSED http://python.org/sf/1056483 opened by Doug Blank minor error in os.access (2004-10-29) http://python.org/sf/1056498 opened by edward ambiguity in os.tmpfile (2004-10-29) CLOSED http://python.org/sf/1056502 opened by edward ambiguity in os.tmpfile (2004-10-29) http://python.org/sf/1056515 opened by edward subprocess works poorly on Windows with Python 2.3 (2004-10-29) http://python.org/sf/1057048 opened by Russell Owen subprocess on Windows 2: unexpected failure (2004-10-29) http://python.org/sf/1057052 opened by Russell Owen subprocess on Windows: extra windows (2004-10-29) http://python.org/sf/1057061 opened by Russell Owen Bugs Closed ___________ relative on-disk SYSTEM id handling is incorrect (2004-02-22) http://python.org/sf/902037 closed by rhettinger filemode() in tarfile.py makes wrong file mode strings (2004-08-27) http://python.org/sf/1017553 closed by akuchling test_subprocess 2.4b1 fails on FreeBSD 5.2 (2004-10-17) http://python.org/sf/1048808 closed by astrand No command line args when script run without python.exe (2004-09-16) http://python.org/sf/1029047 closed by facundobatista Appending non-empty element to a non-empty list yields None (2004-10-22) http://python.org/sf/1052036 closed by peerjanssen page on python api down (2004-10-22) http://python.org/sf/1052274 closed by loewis pdb runcall should accept keyword arguments (2004-10-22) http://python.org/sf/1052503 closed by rhettinger call arg of lambda not updating (2004-10-17) http://python.org/sf/1048870 closed by rhettinger time module returns wrong timezone under windows (2004-10-25) http://python.org/sf/1053539 closed by tim_one Unjustified SyntaxWarning (2004-10-25) http://python.org/sf/1053604 closed by rhettinger Why is this expression = False? (2004-10-25) http://python.org/sf/1053629 closed by doerwalter Segfault in tuple_of_constants (2004-10-25) http://python.org/sf/1053819 closed by rhettinger help() nor in tutorial index (2004-10-25) http://python.org/sf/1053956 closed by rhettinger serious string hashing error in 2.4b1 (2004-10-25) http://python.org/sf/1054139 closed by rhettinger strptime doesn't work with %U (2004-10-12) http://python.org/sf/1045381 closed by bcannon bdist_wininst broken for pure Python distributions (2004-10-28) http://python.org/sf/1055540 closed by mhammond Lambda in list comprehension (2004-10-28) http://python.org/sf/1056483 closed by rhettinger ambiguity in os.tmpfile (2004-10-28) http://python.org/sf/1056502 closed by facundobatista New / Reopened RFE __________________ Attempt to run all atexit handlers (2004-10-22) http://python.org/sf/1052242 opened by Greg Chapman Seting defaultencoding through an environment variable (2004-10-22) http://python.org/sf/1052098 reopened by loewis RFE Closed __________ Seting defaultencoding through an environment variable (2004-10-22) http://python.org/sf/1052098 closed by loewis

16 years, 5 months

Re: [Python-Dev] bsddb todo for someone - auto-upgrade db format

by Gregory P. Smith

On Thu, Oct 21, 2004 at 12:49:19PM +0400, Oleg Broytmann wrote: > > BTW, just installing is not enough, even when it is come with Python > distribution. Installing a newer version of BerkeleyDB breaks older > databases due to incompatible file formats. fwiw its possible to fix this in the bsddb code. someone willing to do the exception catching database backing-up + auto db_upgrade() work should submit a patch. (i added backing-up as its rude to auto-convert someones database for them to a version their old software can no longer read) -g

16 years, 5 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev October 2004