Author: georg.brandl Date: Fri Nov 26 09:52:36 2010 New Revision: 86784 Log: Merged revisions 85728,85731,85735,85766-85771,85773,85777 via svnmerge from svn+ssh://svn.python.org/python/branches/py3k ........ r85728 | georg.brandl | 2010-10-19 20:54:25 +0200 (Di, 19 Okt 2010) | 1 line #10092: Properly reset locale in Locale*Calendar classes. The context manager was buggy because setlocale() returns the *new* locale, not the old. Also add a test for this. ........ r85731 | georg.brandl | 2010-10-19 23:07:16 +0200 (Di, 19 Okt 2010) | 1 line Be consistent in the spelling of thread-safe(ty). ........ r85735 | georg.brandl | 2010-10-20 08:50:19 +0200 (Mi, 20 Okt 2010) | 1 line Fix r85728: use "" to mean the system default locale, which should work on more systems. ........ r85766 | georg.brandl | 2010-10-21 09:40:03 +0200 (Do, 21 Okt 2010) | 1 line #10159: sort completion matches before comparing to dir() result. ........ r85767 | georg.brandl | 2010-10-21 14:49:28 +0200 (Do, 21 Okt 2010) | 1 line #9095, #8912, #8999: add support in patchcheck for Mercurial checkouts, C file reindenting, and docs whitespace fixing. ........ r85768 | georg.brandl | 2010-10-21 14:59:14 +0200 (Do, 21 Okt 2010) | 1 line #9919: fix off-by-one error in lineno command in Misc/gdbinit; also add newline to its output. ........ r85769 | georg.brandl | 2010-10-21 15:01:23 +0200 (Do, 21 Okt 2010) | 1 line Fix missing import. ........ r85770 | georg.brandl | 2010-10-21 15:29:10 +0200 (Do, 21 Okt 2010) | 1 line #3077: fix h2py substitution of character literals. ........ r85771 | georg.brandl | 2010-10-21 15:34:51 +0200 (Do, 21 Okt 2010) | 1 line #1203650: allow larger list of files in windows makefile for freeze. ........ r85773 | georg.brandl | 2010-10-21 15:45:52 +0200 (Do, 21 Okt 2010) | 1 line #4829: better error message for invalid file mode ........ r85777 | georg.brandl | 2010-10-21 17:44:51 +0200 (Do, 21 Okt 2010) | 1 line Add .hgeol file for the Mercurial EOL extension. ........ Added: python/branches/release31-maint/.hgeol - copied unchanged from r85777, /python/branches/py3k/.hgeol Modified: python/branches/release31-maint/ (props changed) python/branches/release31-maint/Doc/c-api/init.rst python/branches/release31-maint/Doc/library/calendar.rst python/branches/release31-maint/Doc/library/locale.rst python/branches/release31-maint/Doc/library/multiprocessing.rst python/branches/release31-maint/Doc/library/threading.rst python/branches/release31-maint/Lib/calendar.py python/branches/release31-maint/Lib/test/test_calendar.py python/branches/release31-maint/Lib/test/test_rlcompleter.py python/branches/release31-maint/Misc/NEWS python/branches/release31-maint/Misc/gdbinit python/branches/release31-maint/Modules/_io/fileio.c python/branches/release31-maint/Tools/freeze/makeconfig.py python/branches/release31-maint/Tools/freeze/winmakemakefile.py python/branches/release31-maint/Tools/scripts/h2py.py python/branches/release31-maint/Tools/scripts/patchcheck.py python/branches/release31-maint/Tools/scripts/reindent-rst.py python/branches/release31-maint/Tools/scripts/untabify.py Modified: python/branches/release31-maint/Doc/c-api/init.rst ============================================================================== --- python/branches/release31-maint/Doc/c-api/init.rst (original) +++ python/branches/release31-maint/Doc/c-api/init.rst Fri Nov 26 09:52:36 2010 @@ -416,7 +416,7 @@ single: interpreter lock single: lock, interpreter -The Python interpreter is not fully thread safe. In order to support +The Python interpreter is not fully thread-safe. In order to support multi-threaded Python programs, there's a global lock, called the :dfn:`global interpreter lock` or :dfn:`GIL`, that must be held by the current thread before it can safely access Python objects. Without the lock, even the simplest Modified: python/branches/release31-maint/Doc/library/calendar.rst ============================================================================== --- python/branches/release31-maint/Doc/library/calendar.rst (original) +++ python/branches/release31-maint/Doc/library/calendar.rst Fri Nov 26 09:52:36 2010 @@ -170,9 +170,9 @@ .. class:: LocaleTextCalendar(firstweekday=0, locale=None) This subclass of :class:`TextCalendar` can be passed a locale name in the - constructor and will return month and weekday names in the specified - locale. If this locale includes an encoding all strings containing month and - weekday names will be returned as unicode. + constructor and will return month and weekday names in the specified locale. + If this locale includes an encoding all strings containing month and weekday + names will be returned as unicode. .. class:: LocaleHTMLCalendar(firstweekday=0, locale=None) @@ -182,6 +182,12 @@ locale. If this locale includes an encoding all strings containing month and weekday names will be returned as unicode. +.. note:: + + The :meth:`formatweekday` and :meth:`formatmonthname` methods of these two + classes temporarily change the current locale to the given *locale*. Because + the current locale is a process-wide setting, they are not thread-safe. + For simple text calendars this module provides the following functions. Modified: python/branches/release31-maint/Doc/library/locale.rst ============================================================================== --- python/branches/release31-maint/Doc/library/locale.rst (original) +++ python/branches/release31-maint/Doc/library/locale.rst Fri Nov 26 09:52:36 2010 @@ -39,7 +39,7 @@ If *locale* is omitted or ``None``, the current setting for *category* is returned. - :func:`setlocale` is not thread safe on most systems. Applications typically + :func:`setlocale` is not thread-safe on most systems. Applications typically start with a call of :: import locale Modified: python/branches/release31-maint/Doc/library/multiprocessing.rst ============================================================================== --- python/branches/release31-maint/Doc/library/multiprocessing.rst (original) +++ python/branches/release31-maint/Doc/library/multiprocessing.rst Fri Nov 26 09:52:36 2010 @@ -214,7 +214,7 @@ The ``'d'`` and ``'i'`` arguments used when creating ``num`` and ``arr`` are typecodes of the kind used by the :mod:`array` module: ``'d'`` indicates a double precision float and ``'i'`` indicates a signed integer. These shared - objects will be process and thread safe. + objects will be process and thread-safe. For more flexibility in using shared memory one can use the :mod:`multiprocessing.sharedctypes` module which supports the creation of Modified: python/branches/release31-maint/Doc/library/threading.rst ============================================================================== --- python/branches/release31-maint/Doc/library/threading.rst (original) +++ python/branches/release31-maint/Doc/library/threading.rst Fri Nov 26 09:52:36 2010 @@ -753,9 +753,9 @@ Importing in threaded code -------------------------- -While the import machinery is thread safe, there are two key -restrictions on threaded imports due to inherent limitations in the way -that thread safety is provided: +While the import machinery is thread-safe, there are two key restrictions on +threaded imports due to inherent limitations in the way that thread-safety is +provided: * Firstly, other than in the main module, an import should not have the side effect of spawning a new thread and then waiting for that thread in Modified: python/branches/release31-maint/Lib/calendar.py ============================================================================== --- python/branches/release31-maint/Lib/calendar.py (original) +++ python/branches/release31-maint/Lib/calendar.py Fri Nov 26 09:52:36 2010 @@ -486,8 +486,8 @@ self.locale = locale def __enter__(self): - self.oldlocale = _locale.setlocale(_locale.LC_TIME, self.locale) - #return _locale.getlocale(_locale.LC_TIME)[1] + self.oldlocale = _locale.getlocale(_locale.LC_TIME) + _locale.setlocale(_locale.LC_TIME, self.locale) def __exit__(self, *args): _locale.setlocale(_locale.LC_TIME, self.oldlocale) Modified: python/branches/release31-maint/Lib/test/test_calendar.py ============================================================================== --- python/branches/release31-maint/Lib/test/test_calendar.py (original) +++ python/branches/release31-maint/Lib/test/test_calendar.py Fri Nov 26 09:52:36 2010 @@ -2,6 +2,7 @@ import unittest from test import support +import locale result_2004_text = """ @@ -250,6 +251,19 @@ # verify it "acts like a sequence" in two forms of iteration self.assertEqual(value[::-1], list(reversed(value))) + def test_localecalendars(self): + # ensure that Locale{Text,HTML}Calendar resets the locale properly + # (it is still not thread-safe though) + old_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) + try: + calendar.LocaleTextCalendar(locale='').formatmonthname(2010, 10, 10) + except locale.Error: + # cannot set the system default locale -- skip rest of test + return + calendar.LocaleHTMLCalendar(locale='').formatmonthname(2010, 10) + new_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) + self.assertEquals(old_october, new_october) + class MonthCalendarTestCase(unittest.TestCase): def setUp(self): Modified: python/branches/release31-maint/Lib/test/test_rlcompleter.py ============================================================================== --- python/branches/release31-maint/Lib/test/test_rlcompleter.py (original) +++ python/branches/release31-maint/Lib/test/test_rlcompleter.py Fri Nov 26 09:52:36 2010 @@ -31,9 +31,9 @@ def test_global_matches(self): # test with builtins namespace - self.assertEqual(self.stdcompleter.global_matches('di'), + self.assertEqual(sorted(self.stdcompleter.global_matches('di')), [x+'(' for x in dir(builtins) if x.startswith('di')]) - self.assertEqual(self.stdcompleter.global_matches('st'), + self.assertEqual(sorted(self.stdcompleter.global_matches('st')), [x+'(' for x in dir(builtins) if x.startswith('st')]) self.assertEqual(self.stdcompleter.global_matches('akaksajadhak'), []) Modified: python/branches/release31-maint/Misc/NEWS ============================================================================== --- python/branches/release31-maint/Misc/NEWS (original) +++ python/branches/release31-maint/Misc/NEWS Fri Nov 26 09:52:36 2010 @@ -18,6 +18,8 @@ - Issue #10459: Update CJK character names to Unicode 5.1. +- Issue #10092: Properly reset locale in calendar.Locale*Calendar classes. + - Issue #6098: Don't claim DOM level 3 conformance in minidom. - Issue #5762: Fix AttributeError raised by ``xml.dom.minidom`` when an empty Modified: python/branches/release31-maint/Misc/gdbinit ============================================================================== --- python/branches/release31-maint/Misc/gdbinit (original) +++ python/branches/release31-maint/Misc/gdbinit Fri Nov 26 09:52:36 2010 @@ -62,11 +62,12 @@ set $__p = $__p + 1 if ($__ad > $__lasti) set $__continue = 0 + else + set $__li = $__li + *$__p + set $__p = $__p + 1 end - set $__li = $__li + *$__p - set $__p = $__p + 1 end - printf "%d", $__li + printf "%d\n", $__li end # print the current frame - verbose Modified: python/branches/release31-maint/Modules/_io/fileio.c ============================================================================== --- python/branches/release31-maint/Modules/_io/fileio.c (original) +++ python/branches/release31-maint/Modules/_io/fileio.c Fri Nov 26 09:52:36 2010 @@ -265,7 +265,8 @@ if (rwa) { bad_mode: PyErr_SetString(PyExc_ValueError, - "Must have exactly one of read/write/append mode"); + "Must have exactly one of read/write/append " + "mode and at most one plus"); goto error; } rwa = 1; Modified: python/branches/release31-maint/Tools/freeze/makeconfig.py ============================================================================== --- python/branches/release31-maint/Tools/freeze/makeconfig.py (original) +++ python/branches/release31-maint/Tools/freeze/makeconfig.py Fri Nov 26 09:52:36 2010 @@ -1,5 +1,5 @@ import re - +import sys # Write the config.c file @@ -38,7 +38,6 @@ # Test program. def test(): - import sys if not sys.argv[3:]: print('usage: python makeconfig.py config.c.in outputfile', end=' ') print('modulename ...') Modified: python/branches/release31-maint/Tools/freeze/winmakemakefile.py ============================================================================== --- python/branches/release31-maint/Tools/freeze/winmakemakefile.py (original) +++ python/branches/release31-maint/Tools/freeze/winmakemakefile.py Fri Nov 26 09:52:36 2010 @@ -134,12 +134,14 @@ print() ; print() print("$(target)$(debug_suffix)%s: $(temp_dir) $(OBJS)" % (target_ext)) - print("\tlink -out:$(target)$(debug_suffix)%s %s" % (target_ext, target_link_flags), end=' ') - print("\t$(OBJS) \\") - print("\t$(LIBS) \\") - print("\t$(ADDN_LINK_FILES) \\") - print("\t$(pythonlib) $(lcustom) $(l_debug)\\") + print("\tlink -out:$(target)$(debug_suffix)%s %s" % + (target_ext, target_link_flags), "@<<") + print("\t$(OBJS)") + print("\t$(LIBS)") + print("\t$(ADDN_LINK_FILES)") + print("\t$(pythonlib) $(lcustom) $(l_debug)") print("\t$(resources)") + print("<<") print() print("clean:") print("\t-rm -f *.obj") Modified: python/branches/release31-maint/Tools/scripts/h2py.py ============================================================================== --- python/branches/release31-maint/Tools/scripts/h2py.py (original) +++ python/branches/release31-maint/Tools/scripts/h2py.py Fri Nov 26 09:52:36 2010 @@ -93,7 +93,7 @@ for p in ignores: body = p.sub(' ', body) # replace char literals by ord(...) - body = p_char.sub('ord(\\0)', body) + body = p_char.sub("ord('\\1')", body) # Compute negative hexadecimal constants start = 0 UMAX = 2*(sys.maxsize+1) Modified: python/branches/release31-maint/Tools/scripts/patchcheck.py ============================================================================== --- python/branches/release31-maint/Tools/scripts/patchcheck.py (original) +++ python/branches/release31-maint/Tools/scripts/patchcheck.py Fri Nov 26 09:52:36 2010 @@ -1,8 +1,16 @@ +import re +import sys +import shutil import os.path import subprocess -import sys import reindent +import untabify + + +def n_files_str(count): + """Return 'N file(s)' with the proper plurality on 'file'.""" + return "{} file{}".format(count, "s" if count != 1 else "") def status(message, modal=False, info=None): @@ -17,53 +25,105 @@ elif info: print(info(result)) else: - if result: - print("yes") - else: - print("NO") + print("yes" if result else "NO") return result return call_fxn return decorated_fxn + @status("Getting the list of files that have been added/changed", - info=lambda x: "%s files" % len(x)) + info=lambda x: n_files_str(len(x))) def changed_files(): - """Run ``svn status`` and return a set of files that have been - changed/added.""" - cmd = 'svn status --quiet --non-interactive --ignore-externals' - svn_st = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE) - svn_st.wait() - output = [x.decode().rstrip() for x in svn_st.stdout.readlines()] - files = set() - for line in output: - if not line[0] in ('A', 'M'): - continue - line_parts = line.split() - path = line_parts[-1] - if os.path.isfile(path): - files.add(path) - return files + """Get the list of changed or added files from the VCS.""" + if os.path.isdir('.hg'): + vcs = 'hg' + cmd = 'hg status --added --modified --no-status' + elif os.path.isdir('.svn'): + vcs = 'svn' + cmd = 'svn status --quiet --non-interactive --ignore-externals' + else: + sys.exit('need a checkout to get modified files') + + st = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE) + try: + st.wait() + if vcs == 'hg': + return [x.decode().rstrip() for x in st.stdout] + else: + output = (x.decode().rstrip().rsplit(None, 1)[-1] + for x in st.stdout if x[0] in b'AM') + return set(path for path in output if os.path.isfile(path)) + finally: + st.stdout.close() + + +def report_modified_files(file_paths): + count = len(file_paths) + if count == 0: + return n_files_str(count) + else: + lines = ["{}:".format(n_files_str(count))] + for path in file_paths: + lines.append(" {}".format(path)) + return "\n".join(lines) + -@status("Fixing whitespace", info=lambda x: "%s files" % x) +@status("Fixing whitespace", info=report_modified_files) def normalize_whitespace(file_paths): """Make sure that the whitespace for .py files have been normalized.""" reindent.makebackup = False # No need to create backups. - result = list(map(reindent.check, (x for x in file_paths if x.endswith('.py')))) - return sum(result) + fixed = [] + for path in (x for x in file_paths if x.endswith('.py')): + if reindent.check(path): + fixed.append(path) + return fixed + + +@status("Fixing C file whitespace", info=report_modified_files) +def normalize_c_whitespace(file_paths): + """Report if any C files """ + fixed = [] + for path in file_paths: + with open(path, 'r') as f: + if '\t' not in f.read(): + continue + untabify.process(path, 8, verbose=False) + fixed.append(path) + return fixed + + +ws_re = re.compile(br'\s+(\r?\n)$') + +@status("Fixing docs whitespace", info=report_modified_files) +def normalize_docs_whitespace(file_paths): + fixed = [] + for path in file_paths: + try: + with open(path, 'rb') as f: + lines = f.readlines() + new_lines = [ws_re.sub(br'\1', line) for line in lines] + if new_lines != lines: + shutil.copyfile(path, path + '.bak') + with open(path, 'wb') as f: + f.writelines(new_lines) + fixed.append(path) + except Exception as err: + print('Cannot fix %s: %s' % (path, err)) + return fixed + @status("Docs modified", modal=True) def docs_modified(file_paths): - """Report if any files in the Docs directory.""" - for path in file_paths: - if path.startswith("Doc"): - return True - return False + """Report if any file in the Doc directory has been changed.""" + return bool(file_paths) + @status("Misc/ACKS updated", modal=True) def credit_given(file_paths): """Check if Misc/ACKS has been changed.""" return 'Misc/ACKS' in file_paths + @status("Misc/NEWS updated", modal=True) def reported_news(file_paths): """Check if Misc/NEWS has been changed.""" @@ -72,14 +132,22 @@ def main(): file_paths = changed_files() - # PEP 7/8 verification. - normalize_whitespace(file_paths) + python_files = [fn for fn in file_paths if fn.endswith('.py')] + c_files = [fn for fn in file_paths if fn.endswith(('.c', '.h'))] + doc_files = [fn for fn in file_paths if fn.startswith('Doc')] + special_files = {'Misc/ACKS', 'Misc/NEWS'} & set(file_paths) + # PEP 8 whitespace rules enforcement. + normalize_whitespace(python_files) + # C rules enforcement. + normalize_c_whitespace(c_files) + # Doc whitespace enforcement. + normalize_docs_whitespace(doc_files) # Docs updated. - docs_modified(file_paths) + docs_modified(doc_files) # Misc/ACKS changed. - credit_given(file_paths) + credit_given(special_files) # Misc/NEWS changed. - reported_news(file_paths) + reported_news(special_files) # Test suite run and passed. print() Modified: python/branches/release31-maint/Tools/scripts/reindent-rst.py ============================================================================== --- python/branches/release31-maint/Tools/scripts/reindent-rst.py (original) +++ python/branches/release31-maint/Tools/scripts/reindent-rst.py Fri Nov 26 09:52:36 2010 @@ -3,27 +3,12 @@ # Make a reST file compliant to our pre-commit hook. # Currently just remove trailing whitespace. +import sys -import sys, re, shutil - -ws_re = re.compile(r'\s+(\r?\n)$') +import patchcheck def main(argv=sys.argv): - rv = 0 - for filename in argv[1:]: - try: - with open(filename, 'rb') as f: - lines = f.readlines() - new_lines = [ws_re.sub(r'\1', line) for line in lines] - if new_lines != lines: - print('Fixing %s...' % filename) - shutil.copyfile(filename, filename + '.bak') - with open(filename, 'wb') as f: - f.writelines(new_lines) - except Exception as err: - print('Cannot fix %s: %s' % (filename, err)) - rv = 1 - return rv + patchcheck.normalize_docs_whitespace(argv[1:]) if __name__ == '__main__': sys.exit(main()) Modified: python/branches/release31-maint/Tools/scripts/untabify.py ============================================================================== --- python/branches/release31-maint/Tools/scripts/untabify.py (original) +++ python/branches/release31-maint/Tools/scripts/untabify.py Fri Nov 26 09:52:36 2010 @@ -23,7 +23,7 @@ for filename in args: process(filename, tabsize) -def process(filename, tabsize): +def process(filename, tabsize, verbose=True): try: f = open(filename) text = f.read() @@ -46,7 +46,8 @@ f = open(filename, "w") f.write(newtext) f.close() - print(filename) + if verbose: + print(filename) if __name__ == '__main__': main()