Mailman 3 November 2011 - Python-Dev

urllib.request.urlopen struggling in Windows 7
by Thom Ives Nov. 14, 2011

Nov. 14, 2011

Previously, in python 2.6, I had made a lot of use of urllib.urlopen to capture web page content and then post process the data from the site I was downloading. Now, those routines, and the new routines I am trying to use for python 3.2 are running into what seems to be a windows only (maybe even windows 7 only problem). Using the following code with python 3.2.2 (64) on windows 7 ... import urllib.request fp = urllib.request.urlopen(URL_string_that_I_use) string = fp.read() fp.close() print(string.decode("utf8")) I get the following message: Traceback (most recent call last): File "TATest.py", line 5, in <module> string = fp.read() File "d:\python32\lib\http\client.py", line 489, in read return self._read_chunked(amt) File "d:\python32\lib\http\client.py", line 553, in _read_chunked self._safe_read(2) # toss the CRLF at the end of the chunk File "d:\python32\lib\http\client.py", line 592, in _safe_read raise IncompleteRead(b''.join(s), amt) http.client.IncompleteRead: IncompleteRead(0 bytes read, 2 more expected) Using the following code instead ... import urllib.request fp = urllib.request.urlopen(URL_string_that_I_use) for Line in fp: print(Line.decode("utf8").rstrip('\n')) fp.close() I get a fair amount of the web page's content, but then the rest of the capture is thwarted by ... Traceback (most recent call last): File "TATest.py", line 9, in <module> for Line in fp: File "d:\python32\lib\http\client.py", line 489, in read return self._read_chunked(amt) File "d:\python32\lib\http\client.py", line 545, in _read_chunked self._safe_read(2) # toss the CRLF at the end of the chunk File "d:\python32\lib\http\client.py", line 592, in _safe_read raise IncompleteRead(b''.join(s), amt) http.client.IncompleteRead: IncompleteRead(0 bytes read, 2 more expected) Trying to read another page yields ... Traceback (most recent call last): File "TATest.py", line 11, in <module> print(Line.decode("utf8").rstrip('\n')) File "d:\python32\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\x92' in position 21: character maps to <undefined> I do believe this is a windows issue, but can python be made more robust to deal with what is causing it? When trying similar code on Linux, we do not encounter the problem.

2 1

Re: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Fix memory leak with FLUFL-related syntax errors (!)
by Éric Araujo Nov. 14, 2011

Nov. 14, 2011

> changeset: 0feb5a5dbaeb > user: Antoine Pitrou <solipsis(a)pitrou.net> > date: Sun Nov 13 01:02:02 2011 +0100 > summary: > Fix memory leak with FLUFL-related syntax errors (!) I don’t think it is allowed to criticize FLUFL-related code. As the FLUFL is flawless, so are flufly things. I hear the PSU has been notified; a messenger will come t

1 0

Delivery reports about your e-mail
by gh＠ghaering.de Nov. 14, 2011

Nov. 14, 2011

The original message was received at Mon, 14 Nov 2011 12:56:42 +0200 from ghaering.de [173.1.122.19] ----- The following addresses had permanent fatal errors ----- <python-dev(a)python.org> ----- Transcript of session follows ----- while talking to python.org.: >>> MAIL From:gh@ghaering.de <<< 501 gh(a)ghaering.de... Refused

1 0

Re: [Python-Dev] peps: And now for something completely different.
by Antoine Pitrou Nov. 14, 2011

Nov. 14, 2011

On Sun, 13 Nov 2011 22:33:28 +0100 barry.warsaw <python-checkins(a)python.org> wrote: > > +And Now For Something Completely Different > +========================================== So, is the release manager a man with two noses? > +Strings and bytes > +----------------- > + > +Python 2's basic original string type are called 8-bit strings, and > +they play a dual role in Python 2 as both ASCII text and as byte > +arrays. While Python 2 also has a unicode string type, the > +fundamental ambiguity of the core string type, coupled with Python 2's > +default behavior of supporting automatic coercion from 8-bit strings > +to unicodes when the two are combined, often leads to `UnicodeError`s. > +Python 3's standard string type is a unicode, and Python 3 adds a > +bytes type, but critically, no automatic coercion between bytes and > +unicodes is provided. Thus, the core interpreter, its I/O libraries, > +module names, etc. are clear in their distinction between unicode > +strings and bytes. This clarity is often a source of difficulty in > +transitioning existing code to Python 3, because many third party > +libraries and applications are themselves ambiguous in this > +distinction. Once migrated though, most `UnicodeError`s can be > +eliminated. First class unicode (*) support also makes Python much friendlier to non-ASCII natives when it comes to things like filesystem access or error reporting. (*) even though Tom Christiansen would disagree, but perhaps we can settle on first and a half > +Imports > +------- > + > +In Python 3, star imports (e.g. ``from x import *``) are only > +premitted in module level code. permitted Regards Antoine.

2 1

Re: [Python-Dev] [Python-checkins] cpython (2.7): Normalize the keyword arguments documentation notation in re.rst. Closes issue
by Terry Reedy Nov. 14, 2011

Nov. 14, 2011

On 11/13/2011 5:52 PM, eli.bendersky wrote: > http://hg.python.org/cpython/rev/87ecfd5cd5d1 > changeset: 73541:87ecfd5cd5d1 > branch: 2.7 > parent: 73529:c3b063c82ae5 > user: Eli Bendersky<eliben(a)gmail.com> > date: Mon Nov 14 01:02:20 2011 +0200 > summary: > Normalize the keyword arguments documentation notation in re.rst. Closes issue #12875 > -.. function:: compile(pattern[, flags=0]) > +.. function:: compile(pattern, flags=0) ... This issue and the patch are about parameters with *default* arguments, which makes a corresponding argument in a call *optional*. For Python functions, both required and optional arguments can be passed by position (unless disabled) or keyword. Which is to say, for Python functions, any argument can be a keyword argument. I suspect I am not the only person somewhat confused when people use 'keyword' to mean 'optional' or 'default'. tjr

2 1

Hashable memoryviews
by Antoine Pitrou Nov. 13, 2011

Nov. 13, 2011

Hello everyone and Benjamin, Currently, memoryview objects are unhashable: >>> hash(memoryview(b"")) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'memoryview' Compare with Python 2.7: >>> hash(buffer("")) 0 memoryviews already support equality comparison: >>> b"" == memoryview(b"") True If the original object providing the buffer is hashable, then it seems to make sense for the memoryview object to be hashable. This came while porting Twisted to Python 3. What do you think? Regards Antoine.

6 15

order of Misc/ACKS
by Eli Bendersky Nov. 13, 2011

Nov. 13, 2011

The PS: at the top of Misc/ACKS says: PS: In the standard Python distribution, this file is encoded in UTF-8 and the list is in rough alphabetical order by last names. However, the last 3 names in the list don't appear to be part of that alphabetical order. Is this somehow intentional, or just a mistake? Eli

8 9

documenting the Hg commit message hooks in the devguide
by Eli Bendersky Nov. 12, 2011

Nov. 12, 2011

Hi, Our Hg repo has some useful hooks on commit messages that allow to specify which issue to notify for commits, and which issue to close. AFAIU, it's currently documented only in the code of the hook (http://hg.python.org/hooks/file/tip/hgroundup.py). I think adding a short description into the devguide would be a good idea, probably here: http://docs.python.org/devguide/committing.html#commit-messages-and-news-en… Any objections/alternative ideas? Eli

3 3

Summary of Python tracker Issues
by Python tracker Nov. 11, 2011

Nov. 11, 2011

ACTIVITY SUMMARY (2011-11-04 - 2011-11-11) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3110 ( -8) closed 22056 (+50) total 25166 (+42) Open issues with patches: 1325 Issues opened (19) ================== #13340: list.index does not accept None as start or stop http://bugs.python.org/issue13340 reopened by rhettinger #13344: closed sockets don't raise EBADF anymore http://bugs.python.org/issue13344 opened by pitrou #13346: re.split() should behave like string.split() for maxsplit=0 an http://bugs.python.org/issue13346 opened by acg #13348: test_unicode_file fails: shutil.copy2 says "same file" http://bugs.python.org/issue13348 opened by flox #13349: Uninformal error message in index() and remove() functions http://bugs.python.org/issue13349 opened by petri.lehtinen #13354: tcpserver should document non-threaded long-living connections http://bugs.python.org/issue13354 opened by shevek #13355: random.triangular error when low = mode http://bugs.python.org/issue13355 opened by mark108 #13357: HTMLParser parses attributes incorrectly. http://bugs.python.org/issue13357 opened by Michael.Brooks #13358: HTMLParser incorrectly handles cdata elements. http://bugs.python.org/issue13358 opened by Michael.Brooks #13359: urllib2 doesn't escape spaces in http requests http://bugs.python.org/issue13359 opened by davide.rizzo #13368: Possible problem in documentation of module subprocess, method http://bugs.python.org/issue13368 opened by eli.bendersky #13369: timeout with exit code 0 while re-running failed tests http://bugs.python.org/issue13369 opened by flox #13370: test_ctypes fails on osx 10.7 http://bugs.python.org/issue13370 opened by ronaldoussoren #13371: Some Carbon extensions don't build on OSX 10.7 http://bugs.python.org/issue13371 opened by ronaldoussoren #13372: handle_close called twice in poll2 http://bugs.python.org/issue13372 opened by xdegaye #13374: Deprecate usage of the Windows ANSI API in the nt module http://bugs.python.org/issue13374 opened by haypo #13376: readline: pre_input_hook not getting called http://bugs.python.org/issue13376 opened by scates #13378: Change the variable "nsmap" from global to instance (xml.etree http://bugs.python.org/issue13378 opened by Nekmo #13380: ctypes: add an internal function for reseting the ctypes cache http://bugs.python.org/issue13380 opened by meador.inge Most recent 15 issues with no replies (15) ========================================== #13380: ctypes: add an internal function for reseting the ctypes cache http://bugs.python.org/issue13380 #13372: handle_close called twice in poll2 http://bugs.python.org/issue13372 #13369: timeout with exit code 0 while re-running failed tests http://bugs.python.org/issue13369 #13354: tcpserver should document non-threaded long-living connections http://bugs.python.org/issue13354 #13336: packaging.command.Command.copy_file doesn't implement preserve http://bugs.python.org/issue13336 #13330: Attempt full test coverage of LocaleTextCalendar.formatweekday http://bugs.python.org/issue13330 #13325: no address in the representation of asyncore dispatcher after http://bugs.python.org/issue13325 #13294: http.server - HEAD request when no resource is defined. http://bugs.python.org/issue13294 #13282: the table of contents in epub file is too long http://bugs.python.org/issue13282 #13277: tzinfo subclasses information http://bugs.python.org/issue13277 #13276: distutils bdist_wininst created installer does not run the pos http://bugs.python.org/issue13276 #13272: 2to3 fix_renames doesn't rename string.lowercase/uppercase/let http://bugs.python.org/issue13272 #13231: sys.settrace - document 'some other code blocks' for 'call' ev http://bugs.python.org/issue13231 #13217: Missing header dependencies in Makefile http://bugs.python.org/issue13217 #13213: generator.throw() behavior http://bugs.python.org/issue13213 Most recent 15 issues waiting for review (15) ============================================= #13380: ctypes: add an internal function for reseting the ctypes cache http://bugs.python.org/issue13380 #13378: Change the variable "nsmap" from global to instance (xml.etree http://bugs.python.org/issue13378 #13374: Deprecate usage of the Windows ANSI API in the nt module http://bugs.python.org/issue13374 #13372: handle_close called twice in poll2 http://bugs.python.org/issue13372 #13371: Some Carbon extensions don't build on OSX 10.7 http://bugs.python.org/issue13371 #13359: urllib2 doesn't escape spaces in http requests http://bugs.python.org/issue13359 #13338: Not all enumerations used in _Py_ANNOTATE_MEMORY_ORDER http://bugs.python.org/issue13338 #13330: Attempt full test coverage of LocaleTextCalendar.formatweekday http://bugs.python.org/issue13330 #13328: pdb shows code from wrong module http://bugs.python.org/issue13328 #13325: no address in the representation of asyncore dispatcher after http://bugs.python.org/issue13325 #13323: urllib2 does not correctly handle multiple www-authenticate he http://bugs.python.org/issue13323 #13322: buffered read() and write() does not raise BlockingIOError http://bugs.python.org/issue13322 #13305: datetime.strftime("%Y") not consistent for years < 1000 http://bugs.python.org/issue13305 #13301: the script Tools/i18n/msgfmt.py allows arbitrary code executio http://bugs.python.org/issue13301 #13297: xmlrpc.client could accept bytes for input and output http://bugs.python.org/issue13297 Top 10 most discussed issues (10) ================================= #11812: transient socket failure to connect to 'localhost' http://bugs.python.org/issue11812 19 msgs #13193: test_packaging and test_distutils failures http://bugs.python.org/issue13193 17 msgs #13340: list.index does not accept None as start or stop http://bugs.python.org/issue13340 14 msgs #6397: Implementing Solaris "/dev/poll" in the "select" module http://bugs.python.org/issue6397 13 msgs #13322: buffered read() and write() does not raise BlockingIOError http://bugs.python.org/issue13322 10 msgs #13229: Improve tools for iterating over filesystem directories http://bugs.python.org/issue13229 9 msgs #13374: Deprecate usage of the Windows ANSI API in the nt module http://bugs.python.org/issue13374 8 msgs #13211: urllib2.HTTPError does not have 'reason' attribute. http://bugs.python.org/issue13211 7 msgs #13309: test_time fails: time data 'LMT' does not match format '%Z' http://bugs.python.org/issue13309 7 msgs #4489: shutil.rmtree is vulnerable to a symlink attack http://bugs.python.org/issue4489 5 msgs Issues closed (46) ================== #7777: Support needed for AF_RDS family http://bugs.python.org/issue7777 closed by neologix #8025: TypeError: string argument expected, got 'str' http://bugs.python.org/issue8025 closed by ezio.melotti #9896: Introspectable range objects http://bugs.python.org/issue9896 closed by python-dev #11937: Interix support http://bugs.python.org/issue11937 closed by jcea #12163: str.count http://bugs.python.org/issue12163 closed by petri.lehtinen #12260: Make install default to user site-packages http://bugs.python.org/issue12260 closed by eric.araujo #13149: optimization for append-only StringIO http://bugs.python.org/issue13149 closed by pitrou #13161: problems with help() documentation of __i*__ operators http://bugs.python.org/issue13161 closed by eli.bendersky #13191: Typo in argparse documentation http://bugs.python.org/issue13191 closed by eli.bendersky #13200: Add start, stop and step attributes to range objects http://bugs.python.org/issue13200 closed by eric.araujo #13237: subprocess docs should emphasise convenience functions http://bugs.python.org/issue13237 closed by ncoghlan #13254: maildir.items() broken http://bugs.python.org/issue13254 closed by python-dev #13284: email.utils.formatdate function does not handle timezones corr http://bugs.python.org/issue13284 closed by r.david.murray #13292: missing versionadded for bytearray http://bugs.python.org/issue13292 closed by flox #13300: IDLE 3.3 Restart Shell command fails http://bugs.python.org/issue13300 closed by ned.deily #13311: asyncore handle_read should call recv http://bugs.python.org/issue13311 closed by neologix #13321: fstat doesn't accept an object with "fileno" method http://bugs.python.org/issue13321 closed by petri.lehtinen #13326: make clean failed on OpenBSD http://bugs.python.org/issue13326 closed by python-dev #13327: Update utime API to not require explicit None argument http://bugs.python.org/issue13327 closed by brian.curtin #13335: Service application hang in python25.dll http://bugs.python.org/issue13335 closed by terry.reedy #13342: input() builtin always uses "strict" error handler http://bugs.python.org/issue13342 closed by pitrou #13343: Lambda keyword-only argument not updating co_freevars http://bugs.python.org/issue13343 closed by amaury.forgeotdarc #13345: Invisible Files in Windows 7 http://bugs.python.org/issue13345 closed by loewis #13347: .py extension not auto added http://bugs.python.org/issue13347 closed by ned.deily #13350: Use PyUnicode_FromFomat instead of PyUnicode_Format for fixed http://bugs.python.org/issue13350 closed by amaury.forgeotdarc #13351: Strange time complexity when creating nested lists http://bugs.python.org/issue13351 closed by quakes #13352: tutorial section 9.3.3 documentation problem http://bugs.python.org/issue13352 closed by eli.bendersky #13353: documentation problem in logging.handlers.TimedRotatingFileHan http://bugs.python.org/issue13353 closed by vinay.sajip #13356: test_logging warning on 2.7 http://bugs.python.org/issue13356 closed by python-dev #13360: UnicodeWarning raised on sequence and set comparisons http://bugs.python.org/issue13360 closed by flox #13361: getLogger does not check its argument http://bugs.python.org/issue13361 closed by python-dev #13362: Many PEP 8 errors http://bugs.python.org/issue13362 closed by benjamin.peterson #13363: Many usages of dict.keys(), dict.values(), dict.items() when t http://bugs.python.org/issue13363 closed by ned.deily #13364: Duplicated code in decimal module http://bugs.python.org/issue13364 closed by mark.dickinson #13365: str.expandtabs documentation is wrong http://bugs.python.org/issue13365 closed by eli.bendersky #13366: test_pep277 failures under WIndows http://bugs.python.org/issue13366 closed by python-dev #13367: PyCapsule_New's argument *must* not a NULL. http://bugs.python.org/issue13367 closed by benjamin.peterson #13373: Unexpected blocking call to multiprocessing.Queue.get with a t http://bugs.python.org/issue13373 closed by pitrou #13375: Provide a namedtuple style interface for os.walk values http://bugs.python.org/issue13375 closed by ncoghlan #13377: test_codecs "Segmentation fault" on Windows http://bugs.python.org/issue13377 closed by haypo #13379: Wrong Unicode version in unicodedata docstring http://bugs.python.org/issue13379 closed by ezio.melotti #13381: compile fails to compile a ast module object giving a incompre http://bugs.python.org/issue13381 closed by benjamin.peterson #13382: IDLE menu scroll bar does not scroll with OS X 10.4 Apple Tcl/ http://bugs.python.org/issue13382 closed by ned.deily #13383: UnicodeDecodeError in distutils.core.setup when version is uni http://bugs.python.org/issue13383 closed by ezio.melotti #13384: Unnecessary __future__ import in random module http://bugs.python.org/issue13384 closed by brian.curtin #1200313: HTMLParser fails to handle charref in attribute value http://bugs.python.org/issue1200313 closed by ezio.melotti

1 0

unicode_internal codec and the PEP 393
by Victor Stinner Nov. 11, 2011

Nov. 11, 2011

Hi, The unicode_internal decoder doesn't decode surrogate pairs and so test_unicode.UnicodeTest.test_codecs() is failing on Windows (16-bit wchar_t). I don't know if this codec is still revelant with the PEP 393 because the internal representation is now depending on the maximum character (Py_UCS1*, Py_UCS2* or Py_UCS4*), whereas it was a fixed size with Python <= 3.2 (Py_UNICODE*). Should we: * Drop this codec (public and documented, but I don't know if it is used) * Use wchar_t* (Py_UNICODE*) to provide a result similar to Python 3.2, and so fix the decoder to handle surrogate pairs * Use the real representation (Py_UCS1*, Py_UCS2 or Py_UCS4* string) ? The failure on Windows: FAIL: test_codecs (test.test_unicode.UnicodeTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\Buildslave\3.x.moore-windows\build\lib\test\test_unicode.py", line 1408, in test_codecs self.assertEqual(str(u.encode(encoding),encoding), u) AssertionError: '\ud800\udc01\ud840\udc02\ud880\udc03\ud8c0\udc04\ud900\udc05' != '\U00030003\U00040004\U00050005' Victor

2 4