From report at bugs.python.org Sun Feb 1 13:43:32 2015 From: report at bugs.python.org (Martin Panter) Date: Sun, 01 Feb 2015 12:43:32 +0000 Subject: [New-bugs-announce] [issue23360] Content-Type when sending data with urlopen() Message-ID: <1422794612.5.0.505084736818.issue23360@psf.upfronthosting.co.za> New submission from Martin Panter: Currently the documentation gives the impression that the ?data? parameter to Request() has to be in the application/x-www-form-urlencoded format. However I suspect that you can override the type by supplying a Content-Type header, and I would like to document this; see uploaded patch. I noticed that test_urllib2.HandlerTests.test_http() already seems to test the default Content-Type and a custom Content-Type with a Request() object, although I did not see a test for the default Content-Type when supplying ?data? directly to urlopen(). Also I understand the ?charset? parameter on application/x-www-form-urlencoded is not standardized. Would it correspond to the encoding of the %XX codes from urlencode(), which is typically UTF-8, not Latin-1? Or would it correspond to the subsequent string-to-bytes encoding stage, which could just be ASCII since non-ASCII characters are already encoded? Maybe it would be best to drop the advice to set a ?charset? parameter. It was added for Issue 11082. ---------- assignee: docs at python components: Documentation files: non-urlencoded.patch keywords: patch messages: 235166 nosy: docs at python, vadmium priority: normal severity: normal status: open title: Content-Type when sending data with urlopen() versions: Python 3.4 Added file: http://bugs.python.org/file37959/non-urlencoded.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:51:43 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:51:43 +0000 Subject: [New-bugs-announce] [issue23361] integer overflow in winapi_createprocess Message-ID: <1422798703.49.0.874007534993.issue23361@psf.upfronthosting.co.za> New submission from paul: winapi_createprocess takes env_mapping dictionary as a parameter, mapping variables to their env. values. Dictionary with pathologically large values will cause an integer overflow during computation of total space required to store all key-value pairs File: Modules\_winapi.c static PyObject* getenvironment(PyObject* environment) { Py_ssize_t i, envsize, totalsize; ... envsize = PyMapping_Length(environment); keys = PyMapping_Keys(environment); values = PyMapping_Values(environment); if (!keys || !values) goto error; totalsize = 1; /* trailing null character */ for (i = 0; i < envsize; i++) { PyObject* key = PyList_GET_ITEM(keys, i); PyObject* value = PyList_GET_ITEM(values, i); if (! PyUnicode_Check(key) || ! PyUnicode_Check(value)) { PyErr_SetString(PyExc_TypeError, "environment can only contain strings"); goto error; } totalsize += PyUnicode_GET_LENGTH(key) + 1; /* +1 for '=' */ 1 totalsize += PyUnicode_GET_LENGTH(value) + 1; /* +1 for '\0' */ } 2 buffer = PyMem_Malloc(totalsize * sizeof(Py_UCS4)); if (! buffer) goto error; p = buffer; 3 end = buffer + totalsize; 4 for (i = 0; i < envsize; i++) { PyObject* key = PyList_GET_ITEM(keys, i); PyObject* value = PyList_GET_ITEM(values, i); X if (!PyUnicode_AsUCS4(key, p, end - p, 0)) goto error; p += PyUnicode_GET_LENGTH(key); X *p++ = '='; X if (!PyUnicode_AsUCS4(value, p, end - p, 0)) goto error; p += PyUnicode_GET_LENGTH(value); X *p++ = '\0'; } 1. no overflow checks. We can set totalsize to 2^30, with a crafted dictionary. 2. totalsize*4 == 0, so buffer is 0-bytes long 3. end = buffer+2^30 4. envsize == len(env_mapping). We can make this variable as large as we like. X. write past the buffer's end. Note size checks in PyUnicode_AsUCS4 are inefficient, because the size variable (end-p) is very large. ---------- messages: 235168 nosy: pkt priority: normal severity: normal status: open title: integer overflow in winapi_createprocess type: crash versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:53:20 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:53:20 +0000 Subject: [New-bugs-announce] [issue23362] integer overflow in string translate Message-ID: <1422798800.96.0.0931437199134.issue23362@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # PyObject * # _PyUnicode_TranslateCharmap(PyObject *input, # PyObject *mapping, # const char *errors) # { # ... # size = PyUnicode_GET_LENGTH(input); # ... # osize = size; # 1 output = PyMem_Malloc(osize * sizeof(Py_UCS4)); # # 1. Input size = 2^30, so osize*sizeof(Py_UCS4)=2^32==0 (modulo 2^32) and malloc # allocates a 0 byte buffer # # Crash # ----- # # Breakpoint 2, _PyUnicode_TranslateCharmap ( # input='aa...', mapping={97: 'b'}, errors=0x828c82b "ignore") at Objects/unicodeobject.c:8597 # 8597 { # ... # 8636 output = PyMem_Malloc(osize * sizeof(Py_UCS4)); # (gdb) print osize # $1 = 1073741824 # (gdb) print osize*4 # $2 = 0 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x0814aed2 in charmaptranslate_output ( # input='aa...', ipos=51302, mapping={97: 'b'}, output=0xbfc40860, osize=0xbfc40864, opos=0xbfc40868, # res=0xbfc40874) at Objects/unicodeobject.c:8574 # 8574 (*output)[(*opos)++] = PyUnicode_READ_CHAR(*res, 0); # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # s="a"*(2**30) s.translate({ord('a'): 'b'}) ---------- files: poc_translate.py messages: 235169 nosy: pkt priority: normal severity: normal status: open title: integer overflow in string translate type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37961/poc_translate.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:54:13 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:54:13 +0000 Subject: [New-bugs-announce] [issue23363] integer overflow in itertools.permutations Message-ID: <1422798853.28.0.166749700091.issue23363@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # static PyObject * # permutations_new(PyTypeObject *type, PyObject *args, PyObject *kwds) # { # ... # 1 cycles = PyMem_Malloc(r * sizeof(Py_ssize_t)); # ... # for (i=0 ; i0, so we write well beyond the buffer's end # # Crash # ----- # # Breakpoint 1, permutations_new (type=0x83394e0 , args=('A', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:3012 # ... # 3044 indices = PyMem_Malloc(n * sizeof(Py_ssize_t)); # (gdb) print r # $2 = 1073741824 # (gdb) print r*4 # $3 = 0 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x08230900 in permutations_new (type=0x83394e0 , args=('A', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:3054 # 3054 cycles[i] = n - i; # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # import itertools as it it.permutations("A", 2**30) ---------- files: poc_permutations.py messages: 235170 nosy: pkt priority: normal severity: normal status: open title: integer overflow in itertools.permutations type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37962/poc_permutations.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:55:04 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:55:04 +0000 Subject: [New-bugs-announce] [issue23364] integer overflow in itertools.product Message-ID: <1422798904.11.0.859140376922.issue23364@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # static PyObject * # product_new(PyTypeObject *type, PyObject *args, PyObject *kwds) # { # ... # 1 nargs = (repeat == 0) ? 0 : PyTuple_GET_SIZE(args); # 2 npools = nargs * repeat; # # 3 indices = PyMem_Malloc(npools * sizeof(Py_ssize_t)); # ... # # 4 for (i=0; i < nargs ; ++i) { # ... # indices[i] = 0; # } # # 1. nargs is the number of functions arguments (not counting the keyword arg). # We set this value to 2^16 using argument unpacking (*args). # 2. We set the 'repeat' keyword argument to 2^16, so npools=2^32==0 (modulo 2^32) # 3. npools*4=0, so malloc allocates a 0 byte buffer # 4. nargs=2^16, so the loop writes well beyond the buffer's end # # Breakpoint 1, product_new (type=0x8338c80 , # args=('a', ...(truncated), kwds={'repeat': 65536}) # at ./Modules/itertoolsmodule.c:1998 # ... # 2021 nargs = (repeat == 0) ? 0 : PyTuple_GET_SIZE(args); # (gdb) n # 2022 npools = nargs * repeat; # (gdb) print nargs # $14 = 65536 # (gdb) print repeat # $15 = 65536 # (gdb) n # 2024 indices = PyMem_Malloc(npools * sizeof(Py_ssize_t)); # (gdb) print npools # $16 = 0 # (gdb) c # Continuing. # # Crash # ----- # # We crash in a different place, because there was sufficient allocated memory # after the "indices" buffer. # # Program received signal SIGSEGV, Segmentation fault. # 0x08313940 in PyTuple_Type () # (gdb) bt # #0 0x08313940 in PyTuple_Type () # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # #1 0x080f27c7 in PyObject_Hash (v=) at Objects/object.c:747 # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # #2 0x080e132f in PyDict_GetItem (op=, key=) at Objects/dictobject.c:1070 # #2 0x080e132f in PyDict_GetItem (op=, key=) at Objects/dictobject.c:1070 # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # #3 0x080e5261 in _PyDict_GetItemId (dp=, key=0x832bd20 ) at Objects/dictobject.c:2729 # #4 0x0806f0e8 in _PySys_GetObjectId (key=0x832bd20 ) at ./Python/sysmodule.c:57 # #5 0x081bb52a in PyEval_EvalFrameEx (f=Frame 0x404ea1ac, for file , line 1, in (), throwflag=0) at Python/ceval.c:1848 # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # #6 0x081c8574 in PyEval_EvalCodeEx (_co=, globals=, locals=, args=0x0, argcount=0, kws=0x0, kwcount=0, # defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3578 # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # #7 0x081b51ef in PyEval_EvalCode (co=, globals=, locals=) at Python/ceval.c:773 # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # Python Exception 'utf8' codec can't decode byte 0xc8 in position 1: invalid continuation byte: # #8 0x08065e89 in run_mod (mod=0x9ea5758, filename='', globals=, locals=, flags=0xbf85fbc0, arena=0x9e64220) # at Python/pythonrun.c:2180 # #9 0x080637fd in PyRun_InteractiveOneObject (fp=0x40231ac0 <_IO_2_1_stdin_>, filename='', flags=0xbf85fbc0) # at Python/pythonrun.c:1445 # #10 0x08063243 in PyRun_InteractiveLoopFlags (fp=0x40231ac0 <_IO_2_1_stdin_>, filename_str=0x826bc06 "", flags=0xbf85fbc0) # at Python/pythonrun.c:1324 # #11 0x0806305f in PyRun_AnyFileExFlags (fp=0x40231ac0 <_IO_2_1_stdin_>, filename=0x826bc06 "", closeit=0, flags=0xbf85fbc0) # at Python/pythonrun.c:1286 # #12 0x08079e8a in run_file (fp=0x40231ac0 <_IO_2_1_stdin_>, filename=0x0, p_cf=0xbf85fbc0) at Modules/main.c:319 # #13 0x0807a988 in Py_Main (argc=1, argv=0x9e45010) at Modules/main.c:751 # #14 0x0805dc34 in main (argc=1, argv=0xbf85fd04) at ./Modules/python.c:69 # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # import itertools as it args=["a"]*(2**16) it.product(*args, repeat=2**16) ---------- files: poc_product.py messages: 235172 nosy: pkt priority: normal severity: normal status: open title: integer overflow in itertools.product type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37963/poc_product.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:55:43 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:55:43 +0000 Subject: [New-bugs-announce] [issue23365] integer overflow in itertools.combinations_with_replacement Message-ID: <1422798943.75.0.0941782409051.issue23365@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # static PyObject * # cwr_new(PyTypeObject *type, PyObject *args, PyObject *kwds) # { # ... # 1 indices = PyMem_Malloc(r * sizeof(Py_ssize_t)); # ... # for (i=0 ; i0, so we write well beyond the buffer's end # # Crash # ----- # # Breakpoint 1, cwr_new (type=0x83392a0 , args=('AA', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:2684 # 2684 PyObject *pool = NULL; # ... # 2703 indices = PyMem_Malloc(r * sizeof(Py_ssize_t)); # (gdb) print r # $1 = 1073741824 # (gdb) print r*4 # $2 = 0 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x0822fdcd in cwr_new (type=0x83392a0 , args=('AA', 1073741824), kwds=0x0) at ./Modules/itertoolsmodule.c:2710 # 2710 indices[i] = 0; # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # import itertools as it it.combinations_with_replacement("AA", 2**30) ---------- files: poc_cwr.py messages: 235173 nosy: pkt priority: normal severity: normal status: open title: integer overflow in itertools.combinations_with_replacement type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37964/poc_cwr.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:56:22 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:56:22 +0000 Subject: [New-bugs-announce] [issue23366] integer overflow in itertools.combinations Message-ID: <1422798982.69.0.884454194506.issue23366@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # static PyObject * # combinations_new(PyTypeObject *type, PyObject *args, PyObject *kwds) # { # ... # # 1 indices = PyMem_Malloc(r * sizeof(Py_ssize_t)); # ... # # for (i=0 ; i0, so we write well beyond the buffer's end # # Crash # ----- # # Breakpoint 1, combinations_new (type=0x83390c0 , args=('AA', 1073741824), kwds=0x0) # at ./Modules/itertoolsmodule.c:2343 # 2343 PyObject *pool = NULL; # ... # (gdb) n # 2362 indices = PyMem_Malloc(r * sizeof(Py_ssize_t)); # (gdb) print r # $1 = 1073741824 # (gdb) print r*4 # $2 = 0 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x0822f359 in combinations_new (type=0x83390c0 , args=('AA', 1073741824), kwds=0x0) # at ./Modules/itertoolsmodule.c:2369 # 2369 indices[i] = i; # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # import itertools as it it.combinations("AA", 2**30) ---------- files: poc_combinations.py messages: 235174 nosy: pkt priority: normal severity: normal status: open title: integer overflow in itertools.combinations type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37965/poc_combinations.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:57:15 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:57:15 +0000 Subject: [New-bugs-announce] [issue23367] integer overflow in unicodedata.normalize Message-ID: <1422799035.27.0.978256662941.issue23367@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # static PyObject* # unicodedata_normalize(PyObject *self, PyObject *args) # { # ... # if (strcmp(form, "NFKC") == 0) { # if (is_normalized(self, input, 1, 1)) { # Py_INCREF(input); # return input; # } # return nfc_nfkc(self, input, 1); # # We need to pass the is_normalized() check (repeated \xa0 char takes care of # that). nfc_nfkc calls: # # static PyObject* # nfd_nfkd(PyObject *self, PyObject *input, int k) # { # ... # Py_ssize_t space, isize; # ... # isize = PyUnicode_GET_LENGTH(input); # /* Overallocate at most 10 characters. */ # space = (isize > 10 ? 10 : isize) + isize; # osize = space; # 1 output = PyMem_Malloc(space * sizeof(Py_UCS4)); # # 1. if isize=2^30, then space=2^30+10, so space*sizeof(Py_UCS4)=(2^30+10)*4 == # 40 (modulo 2^32), so PyMem_Malloc allocates buffer too small to hold the # result. # # Crash # ----- # # nfd_nfkd (self=, input='...', k=1) at /home/p/Python-3.4.1/Modules/unicodedata.c:552 # 552 stackptr = 0; # (gdb) n # 553 isize = PyUnicode_GET_LENGTH(input); # (gdb) n # 555 space = (isize > 10 ? 10 : isize) + isize; # (gdb) n # 556 osize = space; # (gdb) n # 557 output = PyMem_Malloc(space * sizeof(Py_UCS4)); # (gdb) print space # $9 = 1073741834 # (gdb) print space*4 # $10 = 40 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x40579cbb in nfd_nfkd (self=, input='', k=1) at /home/p/Python-3.4.1/Modules/unicodedata.c:614 # 614 output[o++] = code; # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux import unicodedata as ud s="\xa0"*(2**30) ud.normalize("NFKC", s) ---------- files: poc_unidata_normalize.py messages: 235175 nosy: pkt priority: normal severity: normal status: open title: integer overflow in unicodedata.normalize type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37966/poc_unidata_normalize.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:58:39 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:58:39 +0000 Subject: [New-bugs-announce] [issue23368] integer overflow in _PyUnicode_AsKind Message-ID: <1422799119.61.0.313717828667.issue23368@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # void* # _PyUnicode_AsKind(PyObject *s, unsigned int kind) # { # Py_ssize_t len; # ... # len = PyUnicode_GET_LENGTH(s); # ... # switch (kind) { # ... # case PyUnicode_4BYTE_KIND: # 1 result = PyMem_Malloc(len * sizeof(Py_UCS4)); # ... # else { # assert(skind == PyUnicode_1BYTE_KIND); # 2 _PyUnicode_CONVERT_BYTES( # Py_UCS1, Py_UCS4, # PyUnicode_1BYTE_DATA(s), # PyUnicode_1BYTE_DATA(s) + len, # result); # } # # 1. len equals 2^30, so len*sizeof(Py_UCS4)=2^30*2^2=2^32, which gets casted # down to 0, since PyMem_Malloc takes size_t as the parameter. Resulting buffer # is 0 bytes big. # 2. chars from the source string s (which are 1 byte long) are expanded to 4 # bytes and copied to the 'result' buffer, which is too small to hold them all # # Stack trace # ----------- # # Breakpoint 2, _PyUnicode_AsKind ( # s='a...', kind=4) at Objects/unicodeobject.c:2176 # 2176 if (PyUnicode_READY(s) == -1) # (gdb) n # 2179 len = PyUnicode_GET_LENGTH(s); # (gdb) n # 2180 skind = PyUnicode_KIND(s); # (gdb) n # 2181 if (skind >= kind) { # (gdb) n # 2185 switch (kind) { # (gdb) n # 2198 result = PyMem_Malloc(len * sizeof(Py_UCS4)); # (gdb) print len # $10 = 1073741824 # (gdb) print skind # $11 = 1 # (gdb) print kind # $12 = 4 # (gdb) print len*4 # $13 = 0 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x08130b56 in _PyUnicode_AsKind ( # s='a...', kind=4) at Objects/unicodeobject.c:2210 # 2210 _PyUnicode_CONVERT_BYTES( # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # # POC # --- txt=b"\x0a\x0a\x0a\x00" uni=txt.decode("utf-32") sub="a"*(2**30) uni.count(sub) ---------- files: poc_askind.py messages: 235176 nosy: pkt priority: normal severity: normal status: open title: integer overflow in _PyUnicode_AsKind type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37967/poc_askind.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 14:59:35 2015 From: report at bugs.python.org (paul) Date: Sun, 01 Feb 2015 13:59:35 +0000 Subject: [New-bugs-announce] [issue23369] integer overflow in _json.encode_basestring_ascii Message-ID: <1422799175.9.0.580347379364.issue23369@psf.upfronthosting.co.za> New submission from paul: # static PyObject * # ascii_escape_unicode(PyObject *pystr) # { # ... # # input_chars = PyUnicode_GET_LENGTH(pystr); # input = PyUnicode_DATA(pystr); # kind = PyUnicode_KIND(pystr); # # /* Compute the output size */ # for (i = 0, output_size = 2; i < input_chars; i++) { # Py_UCS4 c = PyUnicode_READ(kind, input, i); # if (S_CHAR(c)) # output_size++; # else { # switch(c) { # ... # default: # 1 output_size += c >= 0x10000 ? 12 : 6; # ... # # 2 rval = PyUnicode_New(output_size, 127); # # 1. if c is \uFFFF then output_size += 6. There are no overflow checks on this # variable, so we can overflow it with a sufficiently long (2**32/6+1 chars) # string # 2. rval buffer is too small to hold the result # # Crash: # ------ # # Breakpoint 3, ascii_escape_unicode (pystr='...') at /home/p/Python-3.4.1/Modules/_json.c:198 # 198 rval = PyUnicode_New(output_size, 127); # (gdb) print output_size # $9 = 4 # (gdb) c # Continuing. # # Program received signal SIGSEGV, Segmentation fault. # 0x4057888f in ascii_escape_unichar (c=65535, # output=0x40572358 "...", # chars=19624) at /home/p/Python-3.4.1/Modules/_json.c:155 # 155 output[chars++] = Py_hexdigits[(c >> 8) & 0xf]; # # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux # from _json import encode_basestring_ascii as enc s="\uffff"*int((2**32)/6+1) enc(s) ---------- files: poc_ascii_escape.py messages: 235177 nosy: pkt priority: normal severity: normal status: open title: integer overflow in _json.encode_basestring_ascii type: crash versions: Python 3.4 Added file: http://bugs.python.org/file37968/poc_ascii_escape.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 16:05:51 2015 From: report at bugs.python.org (Stefan Krah) Date: Sun, 01 Feb 2015 15:05:51 +0000 Subject: [New-bugs-announce] [issue23370] PyBuffer_FromContiguous() off-by-one error for non-contiguous buffers Message-ID: <1422803151.79.0.434214643845.issue23370@psf.upfronthosting.co.za> New submission from Stefan Krah: Same as #23349. ---------- messages: 235178 nosy: skrah priority: normal severity: normal stage: needs patch status: open title: PyBuffer_FromContiguous() off-by-one error for non-contiguous buffers type: behavior versions: Python 2.7, Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 16:58:40 2015 From: report at bugs.python.org (=?utf-8?q?St=C3=A9phane_Lenclud?=) Date: Sun, 01 Feb 2015 15:58:40 +0000 Subject: [New-bugs-announce] [issue23371] mimetypes initialization fails on Windows because of TypeError Message-ID: <1422806320.68.0.990065034592.issue23371@psf.upfronthosting.co.za> New submission from St?phane Lenclud: On my Windows 7 installation mimetypes.py in read_windows_registry _winreg.OpenKey can throw a TypeError exception which is not handled and interrupts the execution. To fix it I added: except TypeError: continue To reproduce it I just need to run te following: c:\python27\python -c "import mimetypes; mimetypes.init()" This error is obviously due to the content of my registry. The subkeyname causing issues have names like: {hexid-hexid-hexid-hexid-hexid} ---------- components: Windows messages: 235179 nosy: Slion, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: mimetypes initialization fails on Windows because of TypeError type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 17:19:44 2015 From: report at bugs.python.org (Alec Nikolas Reiter) Date: Sun, 01 Feb 2015 16:19:44 +0000 Subject: [New-bugs-announce] [issue23372] defaultdict.fromkeys should accept a callable factory Message-ID: <1422807584.64.0.426042157217.issue23372@psf.upfronthosting.co.za> New submission from Alec Nikolas Reiter: Not something I've noticed until today, but defaultdict.fromkeys only accepts an iterable of keys and an initial value. To set the default_factory for the newly created collection, you need to prod at the attribute itself. This isn't a huge issue for me, however adding an optional default_factory to fromkeys would be a nice convenience. ---------- messages: 235180 nosy: justanr priority: normal severity: normal status: open title: defaultdict.fromkeys should accept a callable factory type: behavior versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 18:03:21 2015 From: report at bugs.python.org (Alex Martelli) Date: Sun, 01 Feb 2015 17:03:21 +0000 Subject: [New-bugs-announce] [issue23373] curses.textpad crashes in insert mode on wide window Message-ID: <1422810201.83.0.833050049088.issue23373@psf.upfronthosting.co.za> New submission from Alex Martelli: http://stackoverflow.com/questions/28264353/stack-overflow-in-pythons-curses-is-it-bug-in-the-module/28264823#28264823 for details. a curses.textpad on a wide-enough window, in insert mode, causes a crash by recursion limit exceeded (in _insert_printable_char) upon edit. Workaround is to keep the window underlying the textpad sufficiently narrow. ---------- components: Library (Lib) messages: 235182 nosy: Alex.Martelli priority: normal severity: normal status: open title: curses.textpad crashes in insert mode on wide window type: crash versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 1 20:24:18 2015 From: report at bugs.python.org (Skip Montanaro) Date: Sun, 01 Feb 2015 19:24:18 +0000 Subject: [New-bugs-announce] [issue23374] pydoc 3.x raises UnicodeEncodeError on sqlite3 package Message-ID: <1422818658.77.0.913299562674.issue23374@psf.upfronthosting.co.za> New submission from Skip Montanaro: I'm probably doing something wrong, but I've tried everything I can think of without any success. In Python 2.7, the pydoc command successfully displays help for the sqlite3 package, though it muffs the output of Gerhard H?ring's name, spitting out the original Latin-1 spelling. In Python 3.x, I get a UnicodeEncodeError for my trouble, and it hoses my tty settings to boot, requiring a LF reset LF sequence to put right unless I set PAGER to "cat". Here's a sample run: % PAGER=cat pydoc3.5 sqlite3 Traceback (most recent call last): File "/Users/skip/local/bin/pydoc3.5", line 5, in pydoc.cli() File "/Users/skip/local/lib/python3.5/pydoc.py", line 2591, in cli help.help(arg) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1874, in help elif request: doc(request, 'Help on %s:', output=self._output) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1612, in doc pager(render_doc(thing, title, forceload)) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1412, in pager pager(text) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1428, in return lambda text: pipepager(text, os.environ['PAGER']) File "/Users/skip/local/lib/python3.5/pydoc.py", line 1455, in pipepager pipe.write(text) UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 600: ordinal not in range(128) I understand the error, but I see no way to convince it to use any codec other than "ascii". Stuff I tried: * setting PYTHONIOENCODING to "UTF-8" (suggested by Peter Otten on c.l.py) * setting LANG to "en_US.utf8" This is on a Mac running Yosemite with pydoc invoked in Apple's Terminal app. Display is fine in my browser when I run pydoc as a web server. The source it is attempting to display has a coding cookie, so it should know that the code is encoded using Latin-1. The problem seems to all be about generating output. ---------- components: Library (Lib) messages: 235200 nosy: skip.montanaro priority: normal severity: normal status: open title: pydoc 3.x raises UnicodeEncodeError on sqlite3 package type: crash versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 2 08:21:29 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 02 Feb 2015 07:21:29 +0000 Subject: [New-bugs-announce] [issue23375] test_py3kwarn fails on Windows Message-ID: <1422861689.06.0.83900503025.issue23375@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: http://buildbot.python.org/all/builders/x86%20XP-4%202.7/builds/3092/steps/test/logs/stdio ====================================================================== FAIL: test_optional_module_removals (test.test_py3kwarn.TestStdlibRemovals) ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\test\test_py3kwarn.py", line 395, in test_optional_module_removals self.check_removal(module_name, optional=True) File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows\build\lib\test\test_py3kwarn.py", line 379, in check_removal .format(module_name)) AssertionError: DeprecationWarning not raised for imageop ---------------------------------------------------------------------- ---------- components: Windows messages: 235228 nosy: serhiy.storchaka, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: test_py3kwarn fails on Windows type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 2 13:10:22 2015 From: report at bugs.python.org (Stefan Krah) Date: Mon, 02 Feb 2015 12:10:22 +0000 Subject: [New-bugs-announce] [issue23376] getargs.c: redundant C-contiguity check Message-ID: <1422879022.25.0.491994888207.issue23376@psf.upfronthosting.co.za> Changes by Stefan Krah : ---------- nosy: skrah priority: normal severity: normal status: open title: getargs.c: redundant C-contiguity check type: performance versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 2 13:56:16 2015 From: report at bugs.python.org (Martin Panter) Date: Mon, 02 Feb 2015 12:56:16 +0000 Subject: [New-bugs-announce] [issue23377] HTTPResponse may drop buffer holding next response Message-ID: <1422881776.12.0.998068930599.issue23377@psf.upfronthosting.co.za> New submission from Martin Panter: This is the same issue raised at . Currently, every time a new response is to be received, HTTPConnection passes its raw socket object to HTTPResponse, which calls sock.makefile("rb") and creates a BufferedReader. The BufferedReader is used to parse the header section and read the response body. The problem is that the BufferedReader is closed at the end of reading the response, potentially losing buffered data read from a subsequent response. Normally no data is lost, because most users would read the full response before triggering a new request, and the server would wait for a request before sending a response. But if a user pipelined a second request without reading all of the first response, and the server happened to send the end of the first response and the start of the second response in the same packet, it could trigger the problem. I have added a test called test_httplib.ServerTest.testDoubleResponse() which emulates this scenario. The problem also makes it hard to detect misbehaving servers, or use HTTPConnection to test that a server is behaving correctly. I am adding a patch which creates the BufferedReader once for each connection. This involves changing the API of the HTTPResponse constructor. I think this should be okay because even though it is documented, it says ?Not instantiated directly by user?. It did require changing the tests that call the HTTPResponse constructor though. If absolutely necessary, it may be possible to maintain backwards compatibility if we added a new constructor parameter, and carefully juggled how the close() calls work. ---------- components: Library (Lib) files: http-buffer.patch keywords: patch messages: 235251 nosy: vadmium priority: normal severity: normal status: open title: HTTPResponse may drop buffer holding next response type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file37977/http-buffer.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 2 15:23:10 2015 From: report at bugs.python.org (the mulhern) Date: Mon, 02 Feb 2015 14:23:10 +0000 Subject: [New-bugs-announce] [issue23378] argparse.add_argument action parameter should allow value extend Message-ID: <1422886990.52.0.318776752581.issue23378@psf.upfronthosting.co.za> New submission from the mulhern: As well as the append action it would be convenient for there to be an extend action. This is kind of useful to allow something like: parser.add_argument("--foo", action="extend", nargs="+", type=str) given parser.parse_args("--foo f1 --foo f2 f3 f4".split()) to result in ["f1", "f2", "f3", "f4"]. The action "append" results in [["f1"], ["f2", "f3", "f4"]] And action store in ["f2", "f3", "f4"]. It is easy to write a custom action, but it feels like a fairly common requirement. Probably it would make sense to extend the default, similarly to how append behaves. ---------- components: Library (Lib) messages: 235260 nosy: the.mulhern priority: normal severity: normal status: open title: argparse.add_argument action parameter should allow value extend type: enhancement versions: Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 2 19:06:08 2015 From: report at bugs.python.org (Jonathan Sharpe) Date: Mon, 02 Feb 2015 18:06:08 +0000 Subject: [New-bugs-announce] [issue23379] Incorrect links within PEPs Message-ID: <1422900368.04.0.67258841473.issue23379@psf.upfronthosting.co.za> New submission from Jonathan Sharpe: For example, the link to PEP-340 in PEP-343 points to https://www.python.org/dev/peps/pep-0343/pep-0340.html rather than https://www.python.org/dev/peps/pep-0340/ and the link to PEP-288 from PEP-340 points to https://www.python.org/dev/peps/pep-0340/pep-0288.html rather than https://www.python.org/dev/peps/pep-0288/ ---------- assignee: docs at python components: Documentation messages: 235278 nosy: docs at python, jonrsharpe priority: normal severity: normal status: open title: Incorrect links within PEPs _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 2 23:14:47 2015 From: report at bugs.python.org (Vinson Lee) Date: Mon, 02 Feb 2015 22:14:47 +0000 Subject: [New-bugs-announce] [issue23380] Python 2.7.9 test_gdb fails on Fedora 21 Message-ID: <1422915287.22.0.702973088229.issue23380@psf.upfronthosting.co.za> New submission from Vinson Lee: Python 2.7.9 test_gdb fails on Fedora 21. $ ./python Lib/test/regrtest.py -v test_gdb == CPython 2.7.9 (default, Feb 2 2015, 13:43:56) [GCC 4.9.2 20141101 (Red Hat 4.9.2-1)] == Linux-3.18.3-201.fc21.x86_64-x86_64-with-fedora-21-Twenty_One little-endian == Python-2.7.9/build/test_python_18852 Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0) [1/1] test_gdb test_NULL_instance_dict (test.test_gdb.PrettyPrintTests) Ensure that a PyInstanceObject with with a NULL in_dict is handled ... FAIL test_NULL_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with NULL ob_type is handled gracefully ... FAIL test_NULL_ptr (test.test_gdb.PrettyPrintTests) Ensure that a NULL PyObject* is handled gracefully ... FAIL test_builtin_function (test.test_gdb.PrettyPrintTests) ... FAIL test_builtin_method (test.test_gdb.PrettyPrintTests) ... FAIL test_builtins_help (test.test_gdb.PrettyPrintTests) Ensure that the new-style class _Helper in site.py can be handled ... FAIL test_classic_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of classic class instances ... FAIL test_corrupt_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a corrupt ob_type is handled gracefully ... FAIL test_corrupt_tp_flags (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_flags is handled ... FAIL test_corrupt_tp_name (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_name is handled ... FAIL test_dicts (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of dictionaries ... FAIL test_exceptions (test.test_gdb.PrettyPrintTests) ... FAIL test_frames (test.test_gdb.PrettyPrintTests) ... ok test_frozensets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of frozensets ... FAIL test_getting_backtrace (test.test_gdb.PrettyPrintTests) ... ok test_int (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "int" values ... FAIL test_lists (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of lists ... FAIL test_long (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "long" values ... FAIL test_modern_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of new-style class instances ... FAIL test_selfreferential_dict (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a dict doesn't lead proxyval ... FAIL test_selfreferential_list (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a list doesn't lead proxyval ... FAIL test_selfreferential_new_style_instance (test.test_gdb.PrettyPrintTests) ... FAIL test_selfreferential_old_style_instance (test.test_gdb.PrettyPrintTests) ... FAIL test_sets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of sets ... FAIL test_singletons (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of True, False and None ... FAIL test_strings (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of strings ... FAIL test_subclassing_list (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a list subclass ... FAIL test_subclassing_tuple (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a tuple subclass ... FAIL test_truncation (test.test_gdb.PrettyPrintTests) Verify that very long output is truncated ... FAIL test_tuples (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of tuples ... FAIL test_unicode (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of unicode values ... FAIL test_basic_command (test.test_gdb.PyListTests) Verify that the "py-list" command works ... skipped 'Python was compiled with optimizations' test_one_abs_arg (test.test_gdb.PyListTests) Verify the "py-list" command with one absolute argument ... skipped 'Python was compiled with optimizations' test_two_abs_args (test.test_gdb.PyListTests) Verify the "py-list" command with two absolute arguments ... skipped 'Python was compiled with optimizations' test_down_at_bottom (test.test_gdb.StackNavigationTests) Verify handling of "py-down" at the bottom of the stack ... ok test_pyup_command (test.test_gdb.StackNavigationTests) Verify that the "py-up" command works ... skipped 'Python was compiled with optimizations' test_up_at_top (test.test_gdb.StackNavigationTests) Verify handling of "py-up" at the top of the stack ... ok test_up_then_down (test.test_gdb.StackNavigationTests) Verify "py-up" followed by "py-down" ... skipped 'Python was compiled with optimizations' test_basic_command (test.test_gdb.PyBtTests) Verify that the "py-bt" command works ... skipped 'Python was compiled with optimizations' test_basic_command (test.test_gdb.PyPrintTests) Verify that the "py-print" command works ... skipped 'Python was compiled with optimizations' test_print_after_up (test.test_gdb.PyPrintTests) ... skipped 'Python was compiled with optimizations' test_printing_builtin (test.test_gdb.PyPrintTests) ... skipped 'Python was compiled with optimizations' test_printing_global (test.test_gdb.PyPrintTests) ... skipped 'Python was compiled with optimizations' test_basic_command (test.test_gdb.PyLocalsTests) ... skipped 'Python was compiled with optimizations' test_locals_after_up (test.test_gdb.PyLocalsTests) ... skipped 'Python was compiled with optimizations' ====================================================================== FAIL: test_NULL_instance_dict (test.test_gdb.PrettyPrintTests) Ensure that a PyInstanceObject with with a NULL in_dict is handled ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 476, in test_NULL_instance_dict exptype='Foo') File "Python-2.7.9/Lib/test/test_gdb.py", line 432, in assertSane (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f17050, for file , line 6, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8bb0, globals=globals at entry={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, locals=locals at entry={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8bb0, globals=globals at entry={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, locals=locals at entry={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f4710, flags=0x7fffffffe130, locals={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, globals={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, filename=0x544076 "", mod=0x833030) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe130, locals={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, globals={'foo': , '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, start=257, str=0x7ed010 "\nclass Foo:\n pass\nfoo = Foo()\nfoo.an_int = 42\nprint foo\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "\nclass Foo:\n pass\nfoo = Foo()\nfoo.an_int = 42\nprint foo\n", flags=flags at entry=0x7fffffffe130) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with NULL ob_type is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 447, in test_NULL_ob_type 'set op->ob_type=0') File "Python-2.7.9/Lib/test/test_gdb.py", line 432, in assertSane (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=42, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f861f8, for file , line 1, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f46e0, flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, filename=0x544076 "", mod=0x832da8) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, start=257, str=0x7ed010 "print 42\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "print 42\n", flags=flags at entry=0x7fffffffe160) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_NULL_ptr (test.test_gdb.PrettyPrintTests) Ensure that a NULL PyObject* is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 442, in test_NULL_ptr self.assertEqual(gdb_repr, '0x0') AssertionError: '0x0, op at entry=42' != '0x0' ====================================================================== FAIL: test_builtin_function (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 579, in test_builtin_function self.assertEqual(gdb_repr, '') AssertionError: 'op at entry=' != '' ====================================================================== FAIL: test_builtin_method (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 586, in test_builtin_method (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7fbe200, for file , line 1, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f46f0, flags=0x7fffffffe140, locals={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, filename=0x544076 "", mod=0x832e80) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe140, locals={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', 'sys': , '__doc__': None, '__package__': None}, start=257, str=0x7ed010 "import sys; print sys.stdout.readlines\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "import sys; print sys.stdout.readlines\n", flags=flags at entry=0x7fffffffe140) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_builtins_help (test.test_gdb.PrettyPrintTests) Ensure that the new-style class _Helper in site.py can be handled ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 486, in test_builtins_help msg='Unexpected rendering %r' % gdb_repr) AssertionError: Unexpected rendering 'op at entry=<_Helper at remote 0x7ffff7ee3dd0>' ====================================================================== FAIL: test_classic_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of classic class instances ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 360, in test_classic_class msg='Unexpected classic-class rendering %r' % gdb_repr) AssertionError: Unexpected classic-class rendering 'op at entry=' ====================================================================== FAIL: test_corrupt_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a corrupt ob_type is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 453, in test_corrupt_ob_type expvalue=42) File "Python-2.7.9/Lib/test/test_gdb.py", line 432, in assertSane (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=42, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f861f8, for file , line 1, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f46e0, flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, filename=0x544076 "", mod=0x832da8) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, start=257, str=0x7ed010 "print 42\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "print 42\n", flags=flags at entry=0x7fffffffe160) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_corrupt_tp_flags (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_flags is handled ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 459, in test_corrupt_tp_flags expvalue=42) File "Python-2.7.9/Lib/test/test_gdb.py", line 432, in assertSane (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=42, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f861f8, for file , line 1, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f46e0, flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, filename=0x544076 "", mod=0x832da8) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, start=257, str=0x7ed010 "print 42\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "print 42\n", flags=flags at entry=0x7fffffffe160) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_corrupt_tp_name (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_name is handled ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 465, in test_corrupt_tp_name expvalue=42) File "Python-2.7.9/Lib/test/test_gdb.py", line 432, in assertSane (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=42, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f861f8, for file , line 1, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8830, globals=globals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, locals=locals at entry={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f46e0, flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, filename=0x544076 "", mod=0x832da8) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe160, locals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, globals={'__builtins__': , '__name__': '__main__', '__doc__': None, '__package__': None}, start=257, str=0x7ed010 "print 42\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "print 42\n", flags=flags at entry=0x7fffffffe160) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_dicts (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of dictionaries ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 251, in test_dicts self.assertGdbRepr({}) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry={}' != '{}' ====================================================================== FAIL: test_exceptions (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 337, in test_exceptions "exceptions.RuntimeError('I am an error',)") AssertionError: "op at entry=exceptions.RuntimeError('I am an error',)" != "exceptions.RuntimeError('I am an error',)" ====================================================================== FAIL: test_frozensets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of frozensets ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 318, in test_frozensets self.assertGdbRepr(frozenset()) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=frozenset([])' != 'frozenset([])' ====================================================================== FAIL: test_int (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "int" values ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 230, in test_int self.assertGdbRepr(42) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=42' != '42' ====================================================================== FAIL: test_lists (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of lists ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 257, in test_lists self.assertGdbRepr([]) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=[]' != '[]' ====================================================================== FAIL: test_long (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "long" values ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 238, in test_long self.assertGdbRepr(0L) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=0L' != '0L' ====================================================================== FAIL: test_modern_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of new-style class instances ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 372, in test_modern_class msg='Unexpected new-style class rendering %r' % gdb_repr) AssertionError: Unexpected new-style class rendering 'op at entry=' ====================================================================== FAIL: test_selfreferential_dict (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a dict doesn't lead proxyval ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 507, in test_selfreferential_dict self.assertEqual(gdb_repr, "{'foo': {'bar': {...}}}") AssertionError: "op at entry={'foo': {'bar': {...}}}" != "{'foo': {'bar': {...}}}" ====================================================================== FAIL: test_selfreferential_list (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a list doesn't lead proxyval ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 494, in test_selfreferential_list self.assertEqual(gdb_repr, '[3, 4, 5, [...]]') AssertionError: 'op at entry=[3, 4, 5, [...]]' != '[3, 4, 5, [...]]' ====================================================================== FAIL: test_selfreferential_new_style_instance (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 533, in test_selfreferential_new_style_instance (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=) at remote 0x7ffff7f64310>' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=) at remote 0x7ffff7f64310>, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=) at remote 0x7ffff7f64310>, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at remote 0x7ffff7f64310>) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=) at remote 0x7ffff7f64310>, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f17050, for file , line 6, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8bb0, globals=globals at entry={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, locals=locals at entry={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8bb0, globals=globals at entry={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, locals=locals at entry={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f4710, flags=0x7fffffffe120, locals={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, globals={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, filename=0x544076 "", mod=0x833078) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe120, locals={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, globals={'foo': ) at remote 0x7ffff7f64310>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, start=257, str=0x7ed010 "\nclass Foo(object):\n pass\nfoo = Foo()\nfoo.an_attr = foo\nprint foo\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "\nclass Foo(object):\n pass\nfoo = Foo()\nfoo.an_attr = foo\nprint foo\n", flags=flags at entry=0x7fffffffe120) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_selfreferential_old_style_instance (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 520, in test_selfreferential_old_style_instance (gdb_repr, gdb_output)) AssertionError: Unexpected gdb representation: 'op at entry=) at remote 0x7ffff7f8a6c8>' Breakpoint 1 at 0x45e3a0: PyObject_Print. (2 locations) [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, PyObject_Print (op=op at entry=) at remote 0x7ffff7f8a6c8>, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 329 { #0 PyObject_Print (op=op at entry=) at remote 0x7ffff7f8a6c8>, fp=0x7ffff74ab800 <_IO_2_1_stdout_>, flags=flags at entry=1) at Objects/object.c:329 #1 0x000000000043a13b in file_PyObject_Print (f=0x7ffff7fb7150, f=0x7ffff7fb7150, flags=1, op=) at remote 0x7ffff7f8a6c8>) at Objects/fileobject.c:110 #2 PyFile_WriteObject (v=v at entry=) at remote 0x7ffff7f8a6c8>, f=f at entry=, flags=flags at entry=1) at Objects/fileobject.c:2579 #3 0x00000000004b4bc5 in PyEval_EvalFrameEx (f=f at entry=Frame 0x7ffff7f17050, for file , line 6, in (), throwflag=throwflag at entry=0) at Python/ceval.c:1771 #4 0x00000000004b800b in PyEval_EvalCodeEx (co=co at entry=0x7ffff7fa8bb0, globals=globals at entry={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, locals=locals at entry={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3265 #5 0x00000000004b8119 in PyEval_EvalCode (co=co at entry=0x7ffff7fa8bb0, globals=globals at entry={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, locals=locals at entry={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}) at Python/ceval.c:667 #6 0x00000000004e0f0f in run_mod (arena=0x7f4710, flags=0x7fffffffe130, locals={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, globals={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, filename=0x544076 "", mod=0x833030) at Python/pythonrun.c:1371 #7 PyRun_StringFlags (flags=0x7fffffffe130, locals={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, globals={'foo': ) at remote 0x7ffff7f8a6c8>, '__builtins__': , '__package__': None, '__name__': '__main__', 'Foo': , '__doc__': None}, start=257, str=0x7ed010 "\nclass Foo:\n pass\nfoo = Foo()\nfoo.an_attr = foo\nprint foo\n") at Python/pythonrun.c:1334 #8 PyRun_SimpleStringFlags (command=command at entry=0x7ed010 "\nclass Foo:\n pass\nfoo = Foo()\nfoo.an_attr = foo\nprint foo\n", flags=flags at entry=0x7fffffffe130) at Python/pythonrun.c:975 #9 0x00000000004157d0 in Py_Main (argc=4, argv=) at Modules/main.c:584 #10 0x00007ffff7112fe0 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000000000414e3e in _start () ====================================================================== FAIL: test_sets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of sets ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 299, in test_sets self.assertGdbRepr(set()) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=set([])' != 'set([])' ====================================================================== FAIL: test_singletons (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of True, False and None ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 245, in test_singletons self.assertGdbRepr(True) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=True' != 'True' ====================================================================== FAIL: test_strings (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of strings ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 262, in test_strings self.assertGdbRepr('') File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: "op at entry=''" != "''" ====================================================================== FAIL: test_subclassing_list (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a list subclass ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 385, in test_subclassing_list msg='Unexpected new-style class rendering %r' % gdb_repr) AssertionError: Unexpected new-style class rendering 'op at entry=' ====================================================================== FAIL: test_subclassing_tuple (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a tuple subclass ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 399, in test_subclassing_tuple msg='Unexpected new-style class rendering %r' % gdb_repr) AssertionError: Unexpected new-style class rendering 'op at entry=' ====================================================================== FAIL: test_truncation (test.test_gdb.PrettyPrintTests) Verify that very long output is truncated ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/test test_gdb failed -- multiple errors occurred Downloads/Python-2.7.9/Lib/test/test_gdb.py", line 553, in test_truncation "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, " AssertionError: 'op at entry=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226...(truncated)' != '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226...(truncated)' ====================================================================== FAIL: test_tuples (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of tuples ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 269, in test_tuples self.assertGdbRepr(tuple()) File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: 'op at entry=()' != '()' ====================================================================== FAIL: test_unicode (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of unicode values ---------------------------------------------------------------------- Traceback (most recent call last): File "Python-2.7.9/Lib/test/test_gdb.py", line 276, in test_unicode self.assertGdbRepr(u'') File "Python-2.7.9/Lib/test/test_gdb.py", line 226, in assertGdbRepr self.assertEqual(gdb_repr, repr(val)) AssertionError: "op at entry=u''" != "u''" ---------------------------------------------------------------------- Ran 45 tests in 7.045s FAILED (failures=29, skipped=12) 1 test failed: test_gdb ---------- components: Tests messages: 235293 nosy: vlee priority: normal severity: normal status: open title: Python 2.7.9 test_gdb fails on Fedora 21 type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 01:14:53 2015 From: report at bugs.python.org (Vinson Lee) Date: Tue, 03 Feb 2015 00:14:53 +0000 Subject: [New-bugs-announce] [issue23381] Python 2.7.9+ test_gdb regression on Ubuntu 10.04 Message-ID: <1422922493.9.0.93892499408.issue23381@psf.upfronthosting.co.za> New submission from Vinson Lee: Python 2.7.9+ test_gdb regressed on Ubuntu 10.04. 063d966b78f0c0b7cf4c937991bf883c563f574e is the first bad commit commit 063d966b78f0c0b7cf4c937991bf883c563f574e Author: Serhiy Storchaka Date: Sat Jan 31 11:48:36 2015 +0200 Issue #22765: Fixed test_gdb failures. Supressed unexpected gdb output. Patch by Bohuslav Kabrda. :040000 040000 dd420a96366b568ad8ae5e7c88759d743b29584c 0eb56f3c2d8d731985ae93258170e00571ae9a35 M Lib bisect run success $ gdb --version GNU gdb (GDB) 7.1-ubuntu Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: . $ ./python Lib/test/regrtest.py -v test_gdb == CPython 2.7.9+ (default, Feb 2 2015, 15:48:27) [GCC 4.4.3] == Linux-2.6.32-71-generic-x86_64-with-debian-squeeze-sid little-endian == cpython/build/test_python_29238 Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0) [1/1] test_gdb test_NULL_instance_dict (test.test_gdb.PrettyPrintTests) Ensure that a PyInstanceObject with with a NULL in_dict is handled ... FAIL test_NULL_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with NULL ob_type is handled gracefully ... FAIL test_NULL_ptr (test.test_gdb.PrettyPrintTests) Ensure that a NULL PyObject* is handled gracefully ... FAIL test_builtin_function (test.test_gdb.PrettyPrintTests) ... FAIL test_builtin_method (test.test_gdb.PrettyPrintTests) ... FAIL test_builtins_help (test.test_gdb.PrettyPrintTests) Ensure that the new-style class _Helper in site.py can be handled ... FAIL test_classic_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of classic class instances ... FAIL test_corrupt_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a corrupt ob_type is handled gracefully ... FAIL test_corrupt_tp_flags (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_flags is handled ... FAIL test_corrupt_tp_name (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_name is handled ... FAIL test_dicts (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of dictionaries ... FAIL test_exceptions (test.test_gdb.PrettyPrintTests) ... FAIL test_frames (test.test_gdb.PrettyPrintTests) ... FAIL test_frozensets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of frozensets ... FAIL test_getting_backtrace (test.test_gdb.PrettyPrintTests) ... FAIL test_int (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "int" values ... FAIL test_lists (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of lists ... FAIL test_long (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "long" values ... FAIL test_modern_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of new-style class instances ... FAIL test_selfreferential_dict (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a dict doesn't lead proxyval ... FAIL test_selfreferential_list (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a list doesn't lead proxyval ... FAIL test_selfreferential_new_style_instance (test.test_gdb.PrettyPrintTests) ... FAIL test_selfreferential_old_style_instance (test.test_gdb.PrettyPrintTests) ... FAIL test_sets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of sets ... FAIL test_singletons (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of True, False and None ... FAIL test_strings (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of strings ... FAIL test_subclassing_list (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a list subclass ... FAIL test_subclassing_tuple (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a tuple subclass ... FAIL test_truncation (test.test_gdb.PrettyPrintTests) Verify that very long output is truncated ... FAIL test_tuples (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of tuples ... FAIL test_unicode (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of unicode values ... FAIL test_basic_command (test.test_gdb.PyListTests) Verify that the "py-list" command works ... skipped 'Python was compiled with optimizations' test_one_abs_arg (test.test_gdb.PyListTests) Verify the "py-list" command with one absolute argument ... skipped 'Python was compiled with optimizations' test_two_abs_args (test.test_gdb.PyListTests) Verify the "py-list" command with two absolute arguments ... skipped 'Python was compiled with optimizations' test_down_at_bottom (test.test_gdb.StackNavigationTests) Verify handling of "py-down" at the bottom of the stack ... skipped 'test requires py-up/py-down commands' test_pyup_command (test.test_gdb.StackNavigationTests) Verify that the "py-up" command works ... skipped 'test requires py-up/py-down commands' test_up_at_top (test.test_gdb.StackNavigationTests) Verify handling of "py-up" at the top of the stack ... skipped 'test requires py-up/py-down commands' test_up_then_down (test.test_gdb.StackNavigationTests) Verify "py-up" followed by "py-down" ... skipped 'test requires py-up/py-down commands' test_basic_command (test.test_gdb.PyBtTests) Verify that the "py-bt" command works ... skipped 'Python was compiled with optimizations' test_basic_command (test.test_gdb.PyPrintTests) Verify that the "py-print" command works ... skipped 'Python was compiled with optimizations' test_print_after_up (test.test_gdb.PyPrintTests) ... skipped 'test requires py-up/py-down commands' test_printing_builtin (test.test_gdb.PyPrintTests) ... skipped 'Python was compiled with optimizations' test_printing_global (test.test_gdb.PyPrintTests) ... skipped 'Python was compiled with optimizations' test_basic_command (test.test_gdb.PyLocalsTests) ... skipped 'Python was compiled with optimizations' test_locals_after_up (test.test_gdb.PyLocalsTests) ... skipped 'test requires py-up/py-down commands' ====================================================================== FAIL: test_NULL_instance_dict (test.test_gdb.PrettyPrintTests) Ensure that a PyInstanceObject with with a NULL in_dict is handled ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 495, in test_NULL_instance_dict exptype='Foo') File "cpython/Lib/test/test_gdb.py", line 433, in assertSane cmds_after_breakpoint=cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_NULL_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with NULL ob_type is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 466, in test_NULL_ob_type 'set op->ob_type=0') File "cpython/Lib/test/test_gdb.py", line 433, in assertSane cmds_after_breakpoint=cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_NULL_ptr (test.test_gdb.PrettyPrintTests) Ensure that a NULL PyObject* is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 458, in test_NULL_ptr 'backtrace']) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_builtin_function (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 597, in test_builtin_function gdb_repr, gdb_output = self.get_gdb_repr('print len') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_builtin_method (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 601, in test_builtin_method gdb_repr, gdb_output = self.get_gdb_repr('import sys; print sys.stdout.readlines') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_builtins_help (test.test_gdb.PrettyPrintTests) Ensure that the new-style class _Helper in site.py can be handled ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 502, in test_builtins_help gdb_repr, gdb_output = self.get_gdb_repr('print __builtins__.help', import_site=True) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_classic_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of classic class instances ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 376, in test_classic_class print foo''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_corrupt_ob_type (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a corrupt ob_type is handled gracefully ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 472, in test_corrupt_ob_type expvalue=42) File "cpython/Lib/test/test_gdb.py", line 433, in assertSane cmds_after_breakpoint=cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_corrupt_tp_flags (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_flags is handled ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 478, in test_corrupt_tp_flags expvalue=42) File "cpython/Lib/test/test_gdb.py", line 433, in assertSane cmds_after_breakpoint=cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_corrupt_tp_name (test.test_gdb.PrettyPrintTests) Ensure that a PyObject* with a type with corrupt tp_name is handled ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 484, in test_corrupt_tp_name expvalue=42) File "cpython/Lib/test/test_gdb.py", line 433, in assertSane cmds_after_breakpoint=cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_dicts (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of dictionaries ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 270, in test_dicts self.assertGdbRepr({}) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_exceptions (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 354, in test_exceptions ''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_frames (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 615, in test_frames cmds_after_breakpoint=['print (PyFrameObject*)(((PyCodeObject*)op)->co_zombieframe)'] File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_frozensets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of frozensets ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 337, in test_frozensets self.assertGdbRepr(frozenset()) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_getting_backtrace (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 237, in test_getting_backtrace gdb_output = self.get_stack_trace('print 42') File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_int (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "int" values ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 249, in test_int self.assertGdbRepr(42) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_lists (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of lists ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 276, in test_lists self.assertGdbRepr([]) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_long (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of various "long" values ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 257, in test_long self.assertGdbRepr(0L) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_modern_class (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of new-style class instances ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 388, in test_modern_class print foo''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_selfreferential_dict (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a dict doesn't lead proxyval ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 524, in test_selfreferential_dict self.get_gdb_repr("a = {} ; b = {'bar':a} ; a['foo'] = b ; print a") File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_selfreferential_list (test.test_gdb.PrettyPrintTests) Ensure that a reference loop involving a list doesn't lead proxyval ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 511, in test_selfreferential_list self.get_gdb_repr("a = [3, 4, 5] ; a.append(a) ; print a") File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_selfreferential_new_style_instance (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 548, in test_selfreferential_new_style_instance print foo''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_selfreferential_old_style_instance (test.test_gdb.PrettyPrintTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 535, in test_selfreferential_old_style_instance print foo''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_sets (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of sets ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 318, in test_sets self.assertGdbRepr(set()) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_singletons (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of True, False and None ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 264, in test_singletons self.assertGdbRepr(True) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_strings (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of strings ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 281, in test_strings self.assertGdbRepr('') File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_subclassing_list (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a list subclass ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 401, in test_subclassing_list print foo''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_subclassing_tuple (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of an instance of a tuple subclass ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 415, in test_subclassing_tuple print foo''') File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_truncation (test.test_gdb.PrettyPrintTests) Verify that very long output is truncated ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 570, in test_truncation gdb_repr, gdb_output = self.get_gdb_repr('print range(1000)') File "cpython/Lib/test/test_gdb.py", linetest test_gdb failed -- multiple errors occurred 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_tuples (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of tuples ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 288, in test_tuples self.assertGdbRepr(tuple()) File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ====================================================================== FAIL: test_unicode (test.test_gdb.PrettyPrintTests) Verify the pretty-printing of unicode values ---------------------------------------------------------------------- Traceback (most recent call last): File "cpython/Lib/test/test_gdb.py", line 295, in test_unicode self.assertGdbRepr(u'') File "cpython/Lib/test/test_gdb.py", line 244, in assertGdbRepr cmds_after_breakpoint) File "cpython/Lib/test/test_gdb.py", line 213, in get_gdb_repr import_site=import_site) File "cpython/Lib/test/test_gdb.py", line 198, in get_stack_trace self.assertEqual(unexpected_errlines, []) AssertionError: Lists differ: ['Undefined set print command:... != [] First list contains 1 additional elements. First extra element 0: Undefined set print command: "entry-values no". Try "help set print". - ['Undefined set print command: "entry-values no". Try "help set print".'] + [] ---------------------------------------------------------------------- Ran 45 tests in 3.121s FAILED (failures=31, skipped=14) 1 test failed: test_gdb ---------- components: Tests messages: 235311 nosy: serhiy.storchaka, vlee priority: normal severity: normal status: open title: Python 2.7.9+ test_gdb regression on Ubuntu 10.04 type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 09:20:03 2015 From: report at bugs.python.org (miles) Date: Tue, 03 Feb 2015 08:20:03 +0000 Subject: [New-bugs-announce] [issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown Message-ID: <1422951603.17.0.63584283635.issue23382@psf.upfronthosting.co.za> New submission from miles: Maybe can not shutdown ThreadPoolExecutor when call the method shutdown. Though the variable of _shutdown is set to true in the method of shutdown, it may also reads the variable of _shutdown from cpu cache in the method of _worker, and the worst case is that it could see an out-of-date value of _shutdown forever. so need to acquire lock before reading the variable of _shutdown to make sure see an up-to-date value. the following is the new code: def _worker(executor_reference, work_queue): try: while True: work_item = work_queue.get(block=True) if work_item is not None: work_item.run() continue executor = executor_reference() shutdown = False with executor._shutdown_lock.acquire(): shutdown = executor._shutdown # Exit if: # - The interpreter is shutting down OR # - The executor that owns the worker has been collected OR # - The executor that owns the worker has been shutdown. if _shutdown or executor is None or shutdown: # Notice other workers work_queue.put(None) return del executor except BaseException: _base.LOGGER.critical('Exception in worker', exc_info=True) def shutdown(self, wait=True): with self._shutdown_lock: self._shutdown = True self._work_queue.put(None) if wait: for t in self._threads: t.join() ---------- components: 2to3 (2.x to 3.x conversion tool) messages: 235319 nosy: miles priority: normal severity: normal status: open title: Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown versions: Python 3.2 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 09:42:53 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 03 Feb 2015 08:42:53 +0000 Subject: [New-bugs-announce] [issue23383] Clean up bytes formatting Message-ID: <1422952973.11.0.84169569208.issue23383@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch makes following changes to bytes formatting code: 1) Fixes a bug when PyUnicode_AsEncodedObject() returns non-bytes byte-like object. 2) Avoid few non-neccessary memory allocations and copyings (converting bytearray or ASCII string to bytes). 3) Clean up the code. ---------- components: Interpreter Core files: bytes_format.patch keywords: patch messages: 235320 nosy: Arfrever, eric.smith, ethan.furman, haypo, nascheme, ncoghlan, python-dev, serhiy.storchaka, vadmium priority: normal severity: normal stage: patch review status: open title: Clean up bytes formatting type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file37996/bytes_format.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 10:05:39 2015 From: report at bugs.python.org (Valeriy Osipov) Date: Tue, 03 Feb 2015 09:05:39 +0000 Subject: [New-bugs-announce] [issue23384] urllib.proxy_bypass_registry slow down under Windows if website has no reverse DNS and Fiddler is runing Message-ID: <1422954339.65.0.892974237114.issue23384@psf.upfronthosting.co.za> New submission from Valeriy Osipov: Environment: OS Windows 8, Fiddler is running To reproduce the issue launch this code snippet: import requests import datetime import urllib print datetime.datetime.now() #print requests.get('http://torgi.gov.ru/lotSearchArchive.html?bidKindId=2') #print urllib.proxy_bypass_registry('torgi.gov.ru') print datetime.datetime.now() This on? GET requests to this wesite takes 6-7 seconds. Now comment the following code from the urllib.py from the proxy_bypass_registry function: try: fqdn = socket.getfqdn(rawHost) if fqdn != rawHost: host.append(fqdn) except socket.error: pass Now the same GET request takes 2 second. It is normal because this website takes 2 seconds for response. ---------- components: Windows messages: 235322 nosy: aristotel, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: urllib.proxy_bypass_registry slow down under Windows if website has no reverse DNS and Fiddler is runing versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 10:49:55 2015 From: report at bugs.python.org (nivin) Date: Tue, 03 Feb 2015 09:49:55 +0000 Subject: [New-bugs-announce] [issue23385] snmp - mib error Message-ID: <1422956995.0.0.309413347292.issue23385@psf.upfronthosting.co.za> New submission from nivin: Hi , I have created one snmp mib , and tried to send a sample trap using a script written in python using pysnmp. but i got error as follows ,'pysnmp.smi.error.SmiError: MIB file "mymib.py[co]" not found in search path'. Can any one help me please? I'm attaching the script i used and mib file. Thank you ---------- components: Build files: snmp_script.txt messages: 235327 nosy: nivin priority: normal severity: normal status: open title: snmp - mib error type: compile error versions: Python 2.7 Added file: http://bugs.python.org/file37998/snmp_script.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 10:50:22 2015 From: report at bugs.python.org (nivin) Date: Tue, 03 Feb 2015 09:50:22 +0000 Subject: [New-bugs-announce] [issue23386] snmp - mib error Message-ID: <1422957022.99.0.675855776064.issue23386@psf.upfronthosting.co.za> New submission from nivin: Hi , I have created one snmp mib , and tried to send a sample trap using a script written in python using pysnmp. but i got error as follows ,'pysnmp.smi.error.SmiError: MIB file "mymib.py[co]" not found in search path'. Can any one help me please? I'm attaching the script i used and mib file. Thank you ---------- components: Build files: snmp_script.txt messages: 235328 nosy: nivin priority: normal severity: normal status: open title: snmp - mib error type: compile error versions: Python 2.7 Added file: http://bugs.python.org/file37999/snmp_script.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 12:11:44 2015 From: report at bugs.python.org (Berker Peksag) Date: Tue, 03 Feb 2015 11:11:44 +0000 Subject: [New-bugs-announce] [issue23387] test_urllib2 fails with HTTP Error 502: Bad Gateway Message-ID: <1422961904.97.0.978548371135.issue23387@psf.upfronthosting.co.za> New submission from Berker Peksag: ====================================================================== ERROR: test_issue16464 (test.test_urllib2.MiscTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_urllib2.py", line 1491, in test_issue16464 opener.open(request, "1".encode("us-ascii")) File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/urllib/request.py", line 469, in open response = meth(req, response) File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/urllib/request.py", line 579, in http_response 'http', request, response, code, msg, hdrs) File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/urllib/request.py", line 507, in error return self._call_chain(*args) File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/urllib/request.py", line 441, in _call_chain result = func(*args) File "/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/urllib/request.py", line 587, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 502: Bad Gateway http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.x/builds/2140/steps/test/logs/stdio ---------- components: Tests files: httperror.diff keywords: patch messages: 235336 nosy: berker.peksag, orsenthil priority: normal severity: normal stage: patch review status: open title: test_urllib2 fails with HTTP Error 502: Bad Gateway type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38001/httperror.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 12:24:02 2015 From: report at bugs.python.org (cameris) Date: Tue, 03 Feb 2015 11:24:02 +0000 Subject: [New-bugs-announce] [issue23388] datetime.strftime('%s') does not take timezone into account Message-ID: <1422962642.68.0.522001893601.issue23388@psf.upfronthosting.co.za> New submission from cameris: The following occured on linux with python 3.4.2, the machines local timezone is tz2: >>> import datetime >>> tz1 = datetime.timezone.utc; tz2 = datetime.timezone(datetime.timedelta(seconds=3600)); tz3 = datetime.timezone(datetime.timedelta(seconds=18000)) >>> d1 = datetime.datetime.now(tz=tz1); d2 = datetime.datetime.now(tz=tz2); d3 = datetime.datetime.now(tz=tz3) >>> d1.timestamp(), d2.timestamp(), d3.timestamp() (1422962091.603168, 1422962091.603181, 1422962091.603185) >>> d1.strftime('%s'), d2.strftime('%s'), d3.strftime('%s') ('1422958491', '1422962091', '1422976491') Or in other words: >>> d1.strftime('%s') == str(int(d1.timestamp())), d2.strftime('%s') == str(int(d2.timestamp())), d3.strftime('%s') == str(int(d3.timestamp())) (False, True, False) Expected result of the last line would be (True, True, True). ---------- components: Library (Lib) messages: 235338 nosy: cameris priority: normal severity: normal status: open title: datetime.strftime('%s') does not take timezone into account type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 3 18:55:40 2015 From: report at bugs.python.org (Alistair Lynn) Date: Tue, 03 Feb 2015 17:55:40 +0000 Subject: [New-bugs-announce] [issue23389] pkgutil.find_loader raises an ImportError on PEP 420 implicit namespace packages Message-ID: <1422986140.16.0.155842945597.issue23389@psf.upfronthosting.co.za> New submission from Alistair Lynn: The documentation states that __spec__ is None in?and only in?__main__. That this happens also for PEP 420 implicit namespace packages appears to be the cause of the error. ---------- components: Interpreter Core messages: 235357 nosy: alynn priority: normal severity: normal status: open title: pkgutil.find_loader raises an ImportError on PEP 420 implicit namespace packages type: crash versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 4 00:44:57 2015 From: report at bugs.python.org (Stefan Krah) Date: Tue, 03 Feb 2015 23:44:57 +0000 Subject: [New-bugs-announce] [issue23390] make profile-opt: test_distutils failure Message-ID: <1423007097.16.0.224366200931.issue23390@psf.upfronthosting.co.za> Changes by Stefan Krah : ---------- nosy: skrah priority: normal severity: normal status: open title: make profile-opt: test_distutils failure type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 4 06:06:50 2015 From: report at bugs.python.org (Martin Panter) Date: Wed, 04 Feb 2015 05:06:50 +0000 Subject: [New-bugs-announce] [issue23391] Documentation of EnvironmentError (OSError) arguments disappeared Message-ID: <1423026410.5.0.815744351944.issue23391@psf.upfronthosting.co.za> New submission from Martin Panter: Seems to have been removed in revision 097f4fda61a4, for PEP 3151. The older EnvironmentError documentation talks about creating the exception with two and three constructor arguments, however I don?t see this in the new documentation. Is this usage meant to be deprecated, or still allowed? Either way, I think the documentation should mention it. ---------- assignee: docs at python components: Documentation messages: 235372 nosy: docs at python, vadmium priority: normal severity: normal status: open title: Documentation of EnvironmentError (OSError) arguments disappeared versions: Python 3.3, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 4 08:33:53 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 04 Feb 2015 07:33:53 +0000 Subject: [New-bugs-announce] [issue23392] Add tests for marshal FILE* API Message-ID: <1423035233.28.0.238042061902.issue23392@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Marshal C API functions that works with FILE* (PyMarshal_WriteLongToFile, PyMarshal_WriteObjectToFile, PyMarshal_ReadLongFromFile, PyMarshal_ReadShortFromFile, PyMarshal_ReadObjectFromFile, PyMarshal_ReadLastObjectFromFile) are not used now in CPython and are not tested. Proposed patch adds tests for them. ---------- assignee: serhiy.storchaka components: Tests files: test_marshal_file_api.patch keywords: patch messages: 235381 nosy: kristjan.jonsson, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Add tests for marshal FILE* API type: enhancement versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38010/test_marshal_file_api.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 4 20:45:04 2015 From: report at bugs.python.org (=?utf-8?q?S=C3=A9bastien_Gallou?=) Date: Wed, 04 Feb 2015 19:45:04 +0000 Subject: [New-bugs-announce] [issue23393] [Windows] Unable to link with Python in debug configuration Message-ID: <1423079104.29.0.130380017527.issue23393@psf.upfronthosting.co.za> New submission from S?bastien Gallou: Hi all, I installed Python (2.7.9) as binaries under Windows. I have trouble trying to compile my application embedding Python, in debug configuration. I have exactly the same problem as described here : http://www.cmake.org/pipermail/cmake/2013-January/053125.html The include file python.h selects automaticaly the library file to link with, but python27_d.lib (needed in debug configuration) is not provided by the installer. I don't find if this issue was already supported. Thanks, S?bastien Gallou ---------- components: Build, Library (Lib), Windows messages: 235397 nosy: sgallou, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: [Windows] Unable to link with Python in debug configuration type: compile error versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 4 22:42:16 2015 From: report at bugs.python.org (=?utf-8?q?Fran=C3=A7ois_Trahan?=) Date: Wed, 04 Feb 2015 21:42:16 +0000 Subject: [New-bugs-announce] [issue23394] No garbage collection at end of main thread Message-ID: <1423086136.45.0.600991266786.issue23394@psf.upfronthosting.co.za> New submission from Fran?ois Trahan: When reaching the end of a script, there is no garbage collection done if another thread is running. If you have cyclic references between objects that would be elligible for collection under one of which a __del__ would terminate that thread, execution will hang at the end of the "main thread". We ended up in this situation; adding "gc.collect" at the end of our script solves the problem, but this is a library and it is not reasonnable to assume all our clients will properly ensure every execution path properly forces a garbage collection. Suggestion: at the end of a thread, or at least main thread, force a collection. ---------- components: Interpreter Core messages: 235406 nosy: Fran?ois.Trahan priority: normal severity: normal status: open title: No garbage collection at end of main thread versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 5 00:50:50 2015 From: report at bugs.python.org (Thomas Kluyver) Date: Wed, 04 Feb 2015 23:50:50 +0000 Subject: [New-bugs-announce] [issue23395] _thread.interrupt_main() errors if SIGINT handler in SIG_DFL, SIG_IGN Message-ID: <1423093850.42.0.241477195496.issue23395@psf.upfronthosting.co.za> New submission from Thomas Kluyver: In tracking down an obscure error we were seeing, we boiled it down to this test case for thread.interrupt_main(): import signal, threading, _thread, time signal.signal(signal.SIGINT, signal.SIG_DFL) # or SIG_IGN def thread_run(): _thread.interrupt_main() t = threading.Thread(target=thread_run) t.start() time.sleep(10) This fails with an error message "TypeError: 'int' object is not callable", and a traceback completely disconnected from the cause of the error, presumably because it's not coming from the usual Python stack. The problem appears to be that interrupt_main sets (in the C code) Handlers[SIGINT].tripped, which is only expected to occur when the handler is a Python function. When PyErr_CheckSignals() runs, it tries to call Handlers[SIGINT].func as a Python function, but it's a Python integer, causing the error. I think the fix for this is to check what Handlers[sig_num].func is, either in trip_signal() before setting Handlers[sig_num].tripped, or in PyErr_CheckSignals before calling it. I can work on a patch if desired, but I'm not brilliant at C. ---------- components: Library (Lib) messages: 235412 nosy: minrk, takluyver priority: normal severity: normal status: open title: _thread.interrupt_main() errors if SIGINT handler in SIG_DFL, SIG_IGN type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 5 02:12:24 2015 From: report at bugs.python.org (John Boersma) Date: Thu, 05 Feb 2015 01:12:24 +0000 Subject: [New-bugs-announce] [issue23396] Wrong print for 2.7.9 Message-ID: <1423098744.16.0.128886256638.issue23396@psf.upfronthosting.co.za> New submission from John Boersma: In the tutorial for 2.7.9, in the section on quotes and the escape character, there is the following example text: >>> '"Isn\'t," she said.' '"Isn\'t," she said.' >>> print '"Isn\'t," she said.' "Isn't," she said. >>> s = 'First line.\nSecond line.' # \n means newline >>> s # without print(), \n is included in the output 'First line.\nSecond line.' >>> print s # with print, \n produces a new line First line. Second line. Note the print() in a comment. Isn't that Python 3 syntax? Should just be print for 2.7, I believe. ---------- assignee: docs at python components: Documentation messages: 235414 nosy: docs at python, johnboersma priority: normal severity: normal status: open title: Wrong print for 2.7.9 type: enhancement versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 5 09:24:45 2015 From: report at bugs.python.org (Berker Peksag) Date: Thu, 05 Feb 2015 08:24:45 +0000 Subject: [New-bugs-announce] [issue23397] PEP 431 implementation Message-ID: <1423124685.79.0.871732052041.issue23397@psf.upfronthosting.co.za> New submission from Berker Peksag: I have an outdated and WIP patch to implement PEP 431 on GitHub: https://github.com/berkerpeksag/cpython Hopefully, I will complete my work before beta 1. ---------- assignee: berker.peksag components: Library (Lib) messages: 235423 nosy: berker.peksag priority: normal severity: normal stage: needs patch status: open title: PEP 431 implementation type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 5 19:29:54 2015 From: report at bugs.python.org (Sergiy) Date: Thu, 05 Feb 2015 18:29:54 +0000 Subject: [New-bugs-announce] [issue23398] calendar.monthrange for February 2015 Message-ID: <1423160994.09.0.552874281605.issue23398@psf.upfronthosting.co.za> New submission from Sergiy: >>>print(calendar.monthrange(2015,2)) (6, 28) February has only 5 weeks. ---------- components: Extension Modules messages: 235440 nosy: skarpovsky priority: normal severity: normal status: open title: calendar.monthrange for February 2015 type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 6 01:03:01 2015 From: report at bugs.python.org (Barry A. Warsaw) Date: Fri, 06 Feb 2015 00:03:01 +0000 Subject: [New-bugs-announce] [issue23399] venv should create relative symlinks where possible Message-ID: <1423180981.46.0.156182357728.issue23399@psf.upfronthosting.co.za> New submission from Barry A. Warsaw: There is a subtle behavior difference between virtualenv and pyvenv. When you create a venv with virtualenv, the symbolic links files /bin are relative, while they are absolute with pyvenv. This means that virtual environments created with virtualenv can be relocated (modulo the #! lines of the script, but let's not worry about that for now), but virtual environments created with pyvenv cannot be relocated. With pyvenv, you also have /lib64 pointing to an absolute path. AFAICT, there's no good reason why the symlink targets must be absolute paths. Relative paths work just fine for virtualenv and they should work fine for pyvenv. This patch changes the lib64 and /bin/* symlinks to be relative. There should be no change when symlinks are disabled or are unavailable. One remaining question is whether /bin/python itself should be relative. In virtualenv, even with something like `-p /usr/bin/python3.4` you still get a relative link. /bin/python points outside of the venv, so the question is whether the $PATH-dependent behavior of virtualenv is worthwhile or not. My patch does not change this symlink. ---------- assignee: vinay.sajip components: Library (Lib) files: venv.patch keywords: patch messages: 235457 nosy: barry, dstufft, vinay.sajip priority: normal severity: normal stage: patch review status: open title: venv should create relative symlinks where possible type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38023/venv.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 6 10:03:19 2015 From: report at bugs.python.org (Ole Streicher) Date: Fri, 06 Feb 2015 09:03:19 +0000 Subject: [New-bugs-announce] [issue23400] Inconsistent behaviour of multiprocessing.Queue() if sem_open is not implemented Message-ID: <1423213399.39.0.868704647623.issue23400@psf.upfronthosting.co.za> New submission from Ole Streicher: On Debian Hurd, there is no sem_open implementation. When I try there to create a Queue, I get different errors, depending on the Python version: import multiprocessing q = multiprocessing.Queue() gives on Python 2.7.9: maxsize = 0 def Queue(maxsize=0): ''' Returns a queue object ''' > from multiprocessing.queues import Queue /usr/lib/python2.7/multiprocessing/__init__.py:217: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __all__ = ['Queue', 'SimpleQueue', 'JoinableQueue'] import sys import os import threading import collections import time import atexit import weakref from Queue import Empty, Full import _multiprocessing from multiprocessing import Pipe > from multiprocessing.synchronize import Lock, BoundedSemaphore, Semaphore, Condition /usr/lib/python2.7/multiprocessing/queues.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Condition', 'Event' ] import threading import os import sys from time import time as _time, sleep as _sleep import _multiprocessing from multiprocessing.process import current_process from multiprocessing.util import Finalize, register_after_fork, debug from multiprocessing.forking import assert_spawning, Popen # Try to import the mp.synchronize module cleanly, if it fails # raise ImportError for platforms lacking a working sem_open implementation. # See issue 3770 try: from _multiprocessing import SemLock except (ImportError): raise ImportError("This platform lacks a functioning sem_open" + " implementation, therefore, the required" + " synchronization primitives needed will not" + > " function, see issue 3770.") E ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. /usr/lib/python2.7/multiprocessing/synchronize.py:59: ImportError and on 3.4.2: self = , maxsize = 0 def Queue(self, maxsize=0): '''Returns a queue object''' from .queues import Queue > return Queue(maxsize, ctx=self.get_context()) /usr/lib/python3.4/multiprocessing/context.py:101: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , maxsize = 0 def __init__(self, maxsize=0, *, ctx): if maxsize <= 0: > maxsize = _multiprocessing.SemLock.SEM_VALUE_MAX E AttributeError: 'module' object has no attribute 'SemLock' /usr/lib/python3.4/multiprocessing/queues.py:38: AttributeError Both traces are actually copied from Debian build logs: 2.7.9: https://buildd.debian.org/status/fetch.php?pkg=python-astropy&arch=hurd-i386&ver=1.0~rc1-1&stamp=1422724845 3.4.2: https://buildd.debian.org/status/fetch.php?pkg=python-astropy&arch=hurd-i386&ver=1.0~rc1-2&stamp=1423153281 Neither for 2.7.9 nor for 3.4.2 this behaviour is documented. Also, I would expect to have a NotImplementedError and not an ImportError or an AttributeError in such a case. Please: * document the behaviour of multiprocessing.Queue() in the case of an absent sem_open for both Python2 and Python3 * Consider raising NotImplementedError or being consistent in some other way. ---------- components: Library (Lib) messages: 235474 nosy: olebole priority: normal severity: normal status: open title: Inconsistent behaviour of multiprocessing.Queue() if sem_open is not implemented type: behavior versions: Python 2.7, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 6 10:58:55 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 06 Feb 2015 09:58:55 +0000 Subject: [New-bugs-announce] [issue23401] Add pickle support of Mapping views Message-ID: <1423216735.87.0.773240969368.issue23401@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch adds pickle support of general Mapping views. See also issue23264. ---------- components: Library (Lib) messages: 235475 nosy: alexandre.vassalotti, pitrou, rhettinger, serhiy.storchaka, stutzbach priority: normal severity: normal stage: patch review status: open title: Add pickle support of Mapping views type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 6 19:21:40 2015 From: report at bugs.python.org (Adam) Date: Fri, 06 Feb 2015 18:21:40 +0000 Subject: [New-bugs-announce] [issue23402] dynload_shlib does not close ldl handles when the interpreter is shut down Message-ID: <1423246900.58.0.18702391948.issue23402@psf.upfronthosting.co.za> New submission from Adam: I think dynload_shlib (and maybe some of the other non-ldl dynamic library loaders?) should close the libraries when the interpreter is shut down. Currently the handles are not ever closed and in ldl's case sometimes leaked. The reason I desire this behavior is I have Python opening a shared library that I also open (all within the same process), and I want to be able to reload the library at runtime (via dlclose() + dlopen()) by shutting down the Python interpreter, dlclose()/dlopen(), and re-starting Python on the other side, however having the hanging reference to the library within the interpreter is preventing my dlclose() call from unloading the library. I have attached a patch for dynload_shlib.c that tracks all handles returned by dlopen() and will close them properly when the interpreter is shut down. ---------- components: Interpreter Core files: python-dlopen.diff keywords: patch messages: 235490 nosy: Adam priority: normal severity: normal status: open title: dynload_shlib does not close ldl handles when the interpreter is shut down type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file38028/python-dlopen.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 6 22:28:18 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 06 Feb 2015 21:28:18 +0000 Subject: [New-bugs-announce] [issue23403] Use pickle protocol 4 by default? Message-ID: <1423258098.35.0.366066771775.issue23403@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Why not use pickle protocol 4 by default? It allows to pickle some objects which are not pickleable with lower protocols and is more efficient. ---------- components: Library (Lib) messages: 235498 nosy: alexandre.vassalotti, pitrou, serhiy.storchaka priority: normal severity: normal status: open title: Use pickle protocol 4 by default? type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 6 23:46:05 2015 From: report at bugs.python.org (Vinson Lee) Date: Fri, 06 Feb 2015 22:46:05 +0000 Subject: [New-bugs-announce] [issue23404] Python 3.5 does not build with Python 2.6. Message-ID: <1423262765.96.0.527396909.issue23404@psf.upfronthosting.co.za> New submission from Vinson Lee: Is there a minimum Python requirement to build Python? Python 3.5 does not build with Python 2.6. Python 3.4, Python 3.3, and Python 2.7 build with Python 2.6 so this is recent change in build requirements. For example, this build failure occurs on CentOS 6. python ./Parser/asdl_c.py -h Include ./Parser/Python.asdl Traceback (most recent call last): File "./Parser/asdl_c.py", line 6, in import asdl File "Parser/asdl.py", line 36 builtin_types = {'identifier', 'string', 'bytes', 'int', 'object', 'singleton'} ^ SyntaxError: invalid syntax ---------- components: Build messages: 235505 nosy: vlee priority: normal severity: normal status: open title: Python 3.5 does not build with Python 2.6. type: compile error versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 7 11:48:02 2015 From: report at bugs.python.org (Ned Deily) Date: Sat, 07 Feb 2015 10:48:02 +0000 Subject: [New-bugs-announce] [issue23405] Tools/freeze "make" gets missing file error with unix shared builds Message-ID: <1423306082.41.0.550476230992.issue23405@psf.upfronthosting.co.za> New submission from Ned Deily: While investigating another freeze-related issue, I found that Tools/freeze seems to not work when used with an installed shared unix build. The symptom is that the linkage step in the freeze-produced Makefile fails with: gcc -pthread -Xlinker -export-dynamic config.o frozen.o M___future__.o [...] M_zipfile.o /py/3x/unix/root/lib/python3.4/config-3.4dm/libpython3.4dm.so -lpthread -ldl -lutil -lm -o hello gcc: error: /py/3x/unix/root/f/../lib/python3.4/config-3.4dm/libpython3.4dm.so: No such file or directory Makefile:505: recipe for target 'hello' failed make: *** [hello] Error 1 The problem is that the freeze linkage step is looking for the python library with a shared library extension (in this case, '.so') in the config-3.xmm directory but the library is installed as a static archive ('.a') there: ./lib/python3.4/config-3.4dm/libpython3.4dm.a I'm unfamiliar with freeze and its history but it looks like the problem was introduced by the changes for Issue11824, in particular this: - libs = [os.path.join(binlib, 'libpython$(VERSION).a')] + libs = [os.path.join(binlib, '$(LDLIBRARY)')] The Python Makefile target "libainstall" installs the Python library in the config* dir: @if test -d $(LIBRARY); then :; else \ if test "$(PYTHONFRAMEWORKDIR)" = no-framework; then \ if test "$(SHLIB_SUFFIX)" = .dll; then \ $(INSTALL_DATA) $(LDLIBRARY) $(DESTDIR)$(LIBPL) ; \ else \ $(INSTALL_DATA) $(LIBRARY) $(DESTDIR)$(LIBPL)/$(LIBRARY) ; \ $(RANLIB) $(DESTDIR)$(LIBPL)/$(LIBRARY) ; \ fi; \ else \ echo Skip install of $(LIBRARY) - use make frameworkinstall; \ fi; \ fi So, if I understand the cases correctly, for non-".dll" cases, the library is installed in the config* directory as file name $LIBRARY. But the changed code in the freeze-generated Makefile looks for the library as $LDLIBRARY. For static Python builds, $LIBRARY and $LDLIBRARY are (always?) the same, but for shared builds, they are (usually) not, for example: LDLIBRARY = "libpython3.4dm.so" LIBRARY = "libpython3.4dm.a" I'm not sure what the complete solution should be but clearly what is there now is not right for shared unix builds. ---------- components: Demos and Tools messages: 235518 nosy: lemburg, loewis, meador.inge, ned.deily priority: normal severity: normal stage: needs patch status: open title: Tools/freeze "make" gets missing file error with unix shared builds versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 7 13:55:26 2015 From: report at bugs.python.org (Abraham Smith) Date: Sat, 07 Feb 2015 12:55:26 +0000 Subject: [New-bugs-announce] [issue23406] interning and list comprehension leads to unexpected behavior Message-ID: <1423313726.28.0.32804996181.issue23406@psf.upfronthosting.co.za> New submission from Abraham Smith: Some students were working on matrix routines for practice. The following code: >>> L = [ [0]*3 ]*3 >>> for i in range(3): ... for j in range(3): ... if i==j: L[i][j]=1 was expected to return [[1,0,0],[0,1,0],[0,0,1]] but it returned [[1,1,1],[1,1,1],[1,1,1]] because the list [0]*3 was being interned silently, so all three rows were the same memory! To see this, I did >>> map(id, L) [139634871681464, 139634871681464, 139634871681464] On the other hand >>> M=[ [ 0 for i in range(3) ] for j in range(3) ] does not intern: >>> map(id, L) [139634871631672, 139634871681608, 139634871681680] so the above loop works as expected. This is true in both python 2.7 and 3.4. This is very confusing to users! If this intern behavior with [0]*3 is intended, it should be documented more clearly, because this is something that new students of python might encounter right away when playing with the language's list methods. I didn't see any reference to interning in the discussion of lists in the standard library reference. Moreover, I also could not find any reference to the automatic interning of mutable objects, such as lists. Personally, I cannot see any reason to silently and automatically intern a mutable object; however, if this behavior is really desired, it should be documented. ---------- assignee: docs at python components: Documentation messages: 235520 nosy: Abraham.Smith, docs at python priority: normal severity: normal status: open title: interning and list comprehension leads to unexpected behavior versions: Python 2.7, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 7 21:59:23 2015 From: report at bugs.python.org (Craig Holmquist) Date: Sat, 07 Feb 2015 20:59:23 +0000 Subject: [New-bugs-announce] [issue23407] os.walk always follows Windows junctions Message-ID: <1423342763.81.0.179122512075.issue23407@psf.upfronthosting.co.za> New submission from Craig Holmquist: os.walk follows Windows junctions even if followlinks is False: >>> import os >>> appdata = os.environ['LOCALAPPDATA'] >>> for root, dirs, files in os.walk(appdata, followlinks=False): ... print(root) C:\Users\Test\AppData\Local C:\Users\Test\AppData\Local\Apple C:\Users\Test\AppData\Local\Apple\Apple Software Update C:\Users\Test\AppData\Local\Apple Computer C:\Users\Test\AppData\Local\Apple Computer\iTunes C:\Users\Test\AppData\Local\Application Data C:\Users\Test\AppData\Local\Application Data\Apple C:\Users\Test\AppData\Local\Application Data\Apple\Apple Software Update C:\Users\Test\AppData\Local\Application Data\Apple Computer C:\Users\Test\AppData\Local\Application Data\Apple Computer\iTunes C:\Users\Test\AppData\Local\Application Data\Application Data C:\Users\Test\AppData\Local\Application Data\Application Data\Apple C:\Users\Test\AppData\Local\Application Data\Application Data\Apple\Apple Software Update C:\Users\Test\AppData\Local\Application Data\Application Data\Apple Computer C:\Users\Test\AppData\Local\Application Data\Application Data\Apple Computer\iTunes C:\Users\Test\AppData\Local\Application Data\Application Data\Application Data C:\Users\Test\AppData\Local\Application Data\Application Data\Application Data\Apple C:\Users\Test\AppData\Local\Application Data\Application Data\Application Data\Apple\Apple Software Update C:\Users\Test\AppData\Local\Application Data\Application Data\Application Data\Apple Computer C:\Users\Test\AppData\Local\Application Data\Application Data\Application Data\Apple Computer\iTunes C:\Users\Test\AppData\Local\Application Data\Application Data\Application Data\Application Data [...] For directory symbolic links, os.walk seems to have the correct behavior. However, Windows 7 (at least) employs junctions instead of symlinks in situations like the default user profile layout, i.e. the "Application Data" junction shown above. I also noticed that, for junctions, os.path.islink returns False but os.stat and os.lstat return different results. ---------- components: Library (Lib) messages: 235531 nosy: craigh priority: normal severity: normal status: open title: os.walk always follows Windows junctions type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 10:05:46 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 08 Feb 2015 09:05:46 +0000 Subject: [New-bugs-announce] [issue23408] Pickle tests use incorrect test data Message-ID: <1423386346.84.0.92397011471.issue23408@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Pickle tests use incorrect test data for protocols 0-2. This data is not compatible with Python 2 because refers to Python 3 modules builtins and copyreg (instead of __builtin__ and copy_reg). Proposed patch fixes this (data was regenerated by pythontester). Allso corrected dissassembly dumps. ---------- components: Tests files: test_pickle_data.patch keywords: patch messages: 235551 nosy: alexandre.vassalotti, pitrou, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Pickle tests use incorrect test data type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38040/test_pickle_data.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 11:21:17 2015 From: report at bugs.python.org (Martin Panter) Date: Sun, 08 Feb 2015 10:21:17 +0000 Subject: [New-bugs-announce] [issue23410] Document more BaseHTTPRequestHandler attributes Message-ID: <1423390877.64.0.272904715009.issue23410@psf.upfronthosting.co.za> New submission from Martin Panter: [Padding to avoid Error: 'utf8' codec can't decode bytes in position 189-190: invalid continuation byte] This is a patch to document two attributes of http.server.BaseHTTPRequestHandler: ?close_connection? and ?requestline?. Normally these are set by the handle_one_request() method, but when overriding that method, you need to know to set these attributes. ---------- assignee: docs at python components: Documentation messages: 235554 nosy: docs at python, vadmium priority: normal severity: normal status: open title: Document more BaseHTTPRequestHandler attributes type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 12:38:46 2015 From: report at bugs.python.org (Martin Panter) Date: Sun, 08 Feb 2015 11:38:46 +0000 Subject: [New-bugs-announce] [issue23411] Update urllib.parse.__all__ Message-ID: <1423395526.83.0.535584531827.issue23411@psf.upfronthosting.co.za> New submission from Martin Panter: +"DefragResult", "ParseResult", "SplitResult", +"DefragResultBytes", "ParseResultBytes", "SplitResultBytes"] Also adds test case. ---------- components: Library (Lib) files: urllib.parse-all.patch keywords: patch messages: 235556 nosy: vadmium priority: normal severity: normal status: open title: Update urllib.parse.__all__ type: behavior Added file: http://bugs.python.org/file38044/urllib.parse-all.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 16:43:35 2015 From: report at bugs.python.org (Richard Dymond) Date: Sun, 08 Feb 2015 15:43:35 +0000 Subject: [New-bugs-announce] [issue23412] importlib sometimes fails to import a recently created module Message-ID: <1423410215.95.0.524604601959.issue23412@psf.upfronthosting.co.za> New submission from Richard Dymond: importlib.import_module() sometimes fails to import a module that has just been written to the filesystem, aborting with an ImportError. Example output when executing the attached file with Python 3.3 or 3.4: Wrote tmpwbzb35.py Successfully imported tmpwbzb35 Wrote tmp34c6qs.py Traceback (most recent call last): File "importerror.py", line 12, in importlib.import_module(modname) File "/home/rjd/Python/Python3.3/lib/python3.3/importlib/__init__.py", line 88, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1577, in _gcd_import File "", line 1558, in _find_and_load File "", line 1522, in _find_and_load_unlocked ImportError: No module named 'tmp34c6qs' I have been unable to reproduce this behaviour with Python 2.7 or Python 3.2. ---------- files: importerror.py messages: 235557 nosy: rjdymond priority: normal severity: normal status: open title: importlib sometimes fails to import a recently created module type: behavior versions: Python 3.3, Python 3.4 Added file: http://bugs.python.org/file38046/importerror.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 20:13:12 2015 From: report at bugs.python.org (Martynas Brijunas) Date: Sun, 08 Feb 2015 19:13:12 +0000 Subject: [New-bugs-announce] [issue23413] Incorrect division result Message-ID: <1423422792.29.0.39296794097.issue23413@psf.upfronthosting.co.za> New submission from Martynas Brijunas: Dear Python team, when dividing 3 by 7, I expect to get the following result: 0.428571428571428571428571428571428571428571428571428571428571... What I am getting in the Python interpreter is this: >>> 3 / 7 0.42857142857142855 Which in my opinion is incorrect. ---------- messages: 235564 nosy: Martynas.Brijunas priority: normal severity: normal status: open title: Incorrect division result type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 22:38:58 2015 From: report at bugs.python.org (mattip) Date: Sun, 08 Feb 2015 21:38:58 +0000 Subject: [New-bugs-announce] [issue23414] seek(count, whence) accepts bogus whence on windows, python2.7 Message-ID: <1423431538.2.0.390923477638.issue23414@psf.upfronthosting.co.za> New submission from mattip: f=open('abc.txt', 'w+') f.seek(0, 42) does not raise an exception on windows, python2.7 ---------- components: Windows messages: 235568 nosy: mattip, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: seek(count, whence) accepts bogus whence on windows, python2.7 type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 8 23:13:04 2015 From: report at bugs.python.org (Larry Hastings) Date: Sun, 08 Feb 2015 22:13:04 +0000 Subject: [New-bugs-announce] [issue23415] add-to-pydotorg does not support .exe installers for Windows Message-ID: <1423433584.82.0.0803149544554.issue23415@psf.upfronthosting.co.za> New submission from Larry Hastings: Steve is using new technology to make the installers for Windows. He generates four installers now: * .exe for Win32 * -amd64.exe for Win64 * -webinstall.exe for Win32 web-based installers * -amd64-webinstall.exe for Win64 web-based installers add-to-pydotorg does not recognize these. I attempted to hack it up to add support for them. All I did was modify the "file_descriptions" structure, adding new lines for each of the four new types. I believe my edits worked fine, but it failed to add three of the four installers. The first .exe it processed worked (the plain ".exe", the Win32 non-web-based installer). But the last three failed with the error message: Sorry, this request could not be processed. Please try again later. ---------- assignee: georg.brandl components: Build messages: 235571 nosy: georg.brandl, larry, steve.dower priority: normal severity: normal stage: needs patch status: open title: add-to-pydotorg does not support .exe installers for Windows type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 05:03:57 2015 From: report at bugs.python.org (Martin Panter) Date: Mon, 09 Feb 2015 04:03:57 +0000 Subject: [New-bugs-announce] [issue23416] Make urllib.parse.SplitResult etc arguments optional Message-ID: <1423454637.96.0.780742641159.issue23416@psf.upfronthosting.co.za> New submission from Martin Panter: This would be a simple API enhancement and would allow easier building of URLs, like >>> SplitResult("rtp", address, query=urlencode(query)).geturl() "rtp://localhost:5004?rtcpport=5005" It seems the best way to do this at the moment is annoyingly verbose: SplitResult("rtp", address, path="", query=urlencode(query), fragment="").geturl() The way hinted by the documentation can leave an ugly empty query string: >>> query = () >>> "rtp://%s?%s" % (address, urlencode(query)) "rtp://localhost:5004?" This enhancement would also allow easy parsing of usernames, ports, etc: >>> SplitResult(netloc="[::1]:0").hostname "::1" >>> SplitResult(netloc="[::1]:0").port 0 Looking at the code, I think this could be implemented by adding an explicit constructor to each of the concrete classes, with arguments defaulting to "" or b"" as appropriate. ---------- components: Library (Lib) messages: 235584 nosy: vadmium priority: normal severity: normal status: open title: Make urllib.parse.SplitResult etc arguments optional type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 07:03:01 2015 From: report at bugs.python.org (George Schizas) Date: Mon, 09 Feb 2015 06:03:01 +0000 Subject: [New-bugs-announce] [issue23417] Windows 8.1 and Windows Server 2012 R2 are not displayed properly Message-ID: <1423461781.68.0.225997431705.issue23417@psf.upfronthosting.co.za> New submission from George Schizas: Python on Windows can now can understand that it's on Windows 8.1 and Windows Server 2012 R2, but platform.py hasn't been updated, and claims it's on "post2012Server": >>> import platform >>> print(platform.win32_ver()) ('post2012Server', '6.3.9600', '', 'Multiprocessor Free') The function win32_ver (which most of the platform library relies upon) should be changed, to include Windows 8.1 and Windows Server 2012 R2. I've attached a patch file that does that ---------- components: Library (Lib), Windows files: Python 3.5a1 - platform.py.patch keywords: patch messages: 235592 nosy: gschizas, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows 8.1 and Windows Server 2012 R2 are not displayed properly type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file38050/Python 3.5a1 - platform.py.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 08:03:43 2015 From: report at bugs.python.org (Martin Panter) Date: Mon, 09 Feb 2015 07:03:43 +0000 Subject: [New-bugs-announce] [issue23418] Keep http.server.__all__ up to date Message-ID: <1423465423.93.0.840073300907.issue23418@psf.upfronthosting.co.za> New submission from Martin Panter: Here is a patch that adds the Simple/CGIHTTPRequestHandler classes to __all__, and a test case that should help keep it up to date in the future. ---------- components: Library (Lib) files: http.server-all.patch keywords: patch messages: 235594 nosy: vadmium priority: normal severity: normal status: open title: Keep http.server.__all__ up to date type: behavior Added file: http://bugs.python.org/file38051/http.server-all.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 12:00:44 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 09 Feb 2015 11:00:44 +0000 Subject: [New-bugs-announce] [issue23419] Faster default __reduce__ for classes without __init__ Message-ID: <1423479644.09.0.947345755494.issue23419@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch makes faster default __reduce__ implementation for the case when there is no non-trivial __init__ defined (e.g. for named tuples). In this case __reduce__ will return (cls, newargs) instead of (copyreg.__newobj__, (cls,) + newargs). >>> pickletools.dis(pickletools.optimize(pickle.dumps(turtle.Vec2D(12, 34), 3))) Before: 0: \x80 PROTO 3 2: c GLOBAL 'turtle Vec2D' 16: K BININT1 12 18: K BININT1 34 20: \x86 TUPLE2 21: \x81 NEWOBJ 22: . STOP After: 0: \x80 PROTO 3 2: c GLOBAL 'turtle Vec2D' 16: K BININT1 12 18: K BININT1 34 20: \x86 TUPLE2 21: R REDUCE 22: . STOP Pickled size is the same, but pickling is faster. The benefit is in avoiding of importing copyreg.__newobj__ and allocating new tuple (cls,) + newargs. Microbenchmarks results: $ ./python -m timeit -s "import pickle; from turtle import Vec2D; a = [Vec2D(i, i+0.1) for i in range(1000)]" -- "pickle.dumps(a)" Before: 100 loops, best of 3: 16.3 msec per loop After: 100 loops, best of 3: 15.2 msec per loop $ ./python -m timeit -s "import copy; from turtle import Vec2D; a = [Vec2D(i, i+0.1) for i in range(1000)]" -- "copy.deepcopy(a)" Before: 10 loops, best of 3: 96.6 msec per loop After: 10 loops, best of 3: 88.7 msec per loop ---------- components: Interpreter Core files: object_reduce_no_init.patch keywords: patch messages: 235600 nosy: alexandre.vassalotti, pitrou, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Faster default __reduce__ for classes without __init__ type: performance versions: Python 3.5 Added file: http://bugs.python.org/file38052/object_reduce_no_init.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 13:05:42 2015 From: report at bugs.python.org (Robert Kuska) Date: Mon, 09 Feb 2015 12:05:42 +0000 Subject: [New-bugs-announce] [issue23420] python -m cProfile -s fails with non informative message Message-ID: <1423483542.47.0.873331461969.issue23420@psf.upfronthosting.co.za> New submission from Robert Kuska: Originaly reported here: https://bugzilla.redhat.com/show_bug.cgi?id=1160640 I've forgotten to add the sort value to the -s option of cProfile which results in a traceback instead of user friendly error message. In the example below hello.py just prints a "Hello World": $ python -m cProfile -s hello.py Traceback (most recent call last): File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code exec code in run_globals File "/usr/lib64/python2.6/cProfile.py", line 190, in main() File "/usr/lib64/python2.6/cProfile.py", line 185, in main parser.print_usage() File "/usr/lib64/python2.6/optparse.py", line 1597, in print_usage print >>file, self.get_usage() File "/usr/lib64/python2.6/optparse.py", line 1583, in get_usage self.expand_prog_name(self.usage)) File "/usr/lib64/python2.6/optparse.py", line 1560, in expand_prog_name return s.replace("%prog", self.get_prog_name()) File "/usr/lib64/python2.6/optparse.py", line 1555, in get_prog_name return os.path.basename(sys.argv[0]) IndexError: list index out of range Tested with python2.7, python3.4 with the same result. Attached patch adds `choices` for sort option of cProfile. > $ python -m cProfile -s sdds.py Usage: cProfile.py [-o output_file_path] [-s sort] scriptfile [arg] ... cProfile.py: error: option -s: invalid choice: 'sdds.py' (choose from 'cumulative', 'module', 'ncalls', 'pcalls', 'file', 'line', 'name', 'calls', 'stdname', 'nfl', 'filename', 'cumtime', 'time', 'tottime') ---------- components: Extension Modules files: sort-choices.patch keywords: patch messages: 235602 nosy: rkuska priority: normal severity: normal status: open title: python -m cProfile -s fails with non informative message type: enhancement versions: Python 2.7, Python 3.4 Added file: http://bugs.python.org/file38053/sort-choices.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 14:03:37 2015 From: report at bugs.python.org (wdv4758h) Date: Mon, 09 Feb 2015 13:03:37 +0000 Subject: [New-bugs-announce] [issue23421] tarfile module does not correctly choose compression algorithms Message-ID: <1423487017.12.0.640987709425.issue23421@psf.upfronthosting.co.za> New submission from wdv4758h: The command "python -m tarfile -c test.tar.bz2 test/" should create a file that is compressed by bzip2, but actually the detection of compression algorithm that should be used is broken (for gz, xz, bz2). fix it by prepending a dot to the keys of the dictionary ---------- components: Library (Lib) files: fix_compressions_dict.patch hgrepos: 297 keywords: patch messages: 235604 nosy: wdv4758h priority: normal severity: normal status: open title: tarfile module does not correctly choose compression algorithms type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file38056/fix_compressions_dict.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 16:45:08 2015 From: report at bugs.python.org (Brett Cannon) Date: Mon, 09 Feb 2015 15:45:08 +0000 Subject: [New-bugs-announce] [issue23422] Clarify docs for importlib.import_module() Message-ID: <1423496708.39.0.508635169879.issue23422@psf.upfronthosting.co.za> New submission from Brett Cannon: * Link to the function from importlib.__import__() * Link to importlib.invalidate_caches() * Linkify mention in docs for importlib.util.find_spec() ---------- assignee: brett.cannon components: Documentation messages: 235616 nosy: brett.cannon priority: low severity: normal status: open title: Clarify docs for importlib.import_module() versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 20:39:39 2015 From: report at bugs.python.org (Mark Baker) Date: Mon, 09 Feb 2015 19:39:39 +0000 Subject: [New-bugs-announce] [issue23423] XPath Support in ElementTree doc omission Message-ID: <1423510779.4.0.551912157105.issue23423@psf.upfronthosting.co.za> New submission from Mark Baker: The list of XPath supported features in section 20.5.2.2. "Supported XPath syntax" on page https://docs.python.org/3.4/library/xml.etree.elementtree.html does not list the use of a predicate based on an element value (it only list predicates based on an attribute value. However, testing in 3.4 shows that a predicate based on an element value also works. >>> import xml.etree.ElementTree as etree >>> x = 'bing' >>> y = etree.XML(x) # Find by attribute value >>> z = y.find('bar/baz[@y="bang"]') >>> print(z) # Find by element value >>> z = y.find('bar/[baz="bing"]') >>> print(z) # Find fails with incorrect element value >>> z = y.find('bar/[baz="bong"]') >>> z >>> print(z) None Below the line that says: [tag] Selects all elements that have a child named tag. Only immediate children are supported. It should say something like: [tag="value"] Selects all elements that have a child named tag with the given value. Only immediate children are supported. ---------- assignee: docs at python components: Documentation messages: 235627 nosy: docs at python, mbakeranalecta priority: normal severity: normal status: open title: XPath Support in ElementTree doc omission type: enhancement versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 21:37:19 2015 From: report at bugs.python.org (Grzegorz Abramczyk) Date: Mon, 09 Feb 2015 20:37:19 +0000 Subject: [New-bugs-announce] [issue23424] Unicode character ends interactive session Message-ID: <1423514239.73.0.0631467799432.issue23424@psf.upfronthosting.co.za> New submission from Grzegorz Abramczyk: Inputing some Unicode characters (like '??????...') causes interactive session to abort. When console session is set to use UTF-8 code page (65001) after diacritic character appears in string the session abruptly ends. Looking into debug output it looks like some cleanup is performed but there are no error messages indicating what caused problem. Problem spotted on Windows 10 (technical preview) but I may try to replicate it on some released operating system. --- C:\>chcp 1250 Active code page: 1250 C:\>python -i Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> '?' '?' >>> exit() C:\>chcp 65001 Active code page: 65001 C:\>python -i Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> '?' C:\ ---------- components: Unicode, Windows files: -v.txt messages: 235629 nosy: AGrzes, ezio.melotti, haypo, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Unicode character ends interactive session type: crash versions: Python 3.4 Added file: http://bugs.python.org/file38061/-v.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 21:55:10 2015 From: report at bugs.python.org (albertjan) Date: Mon, 09 Feb 2015 20:55:10 +0000 Subject: [New-bugs-announce] [issue23425] Windows getlocale unix-like with french, german, portuguese, spanish Message-ID: <1423515310.7.0.637877209066.issue23425@psf.upfronthosting.co.za> New submission from albertjan: getlocale() is supposed to (?) return a locale two-tuple in a platform-specific notation. However, in *Windows* 7 64, with Python 3.4, 3.3 and 2.7 a *unix-like*, abbreviated, lang_territory notation is used for french, german, portuguese, spanish. In other words: In these four cases, the output of setlocale is not equal to ".".join(locale.getlocale()) ## Code that demonstrates the differences from __future__ import print_function import locale import collections import pprint languages = ("chinese czech danish dutch english finnish french german greek " "hungarian icelandic italian japanese korean norwegian polish " "portuguese russian slovak spanish swedish turkish") d = collections.defaultdict(list) t = collections.namedtuple("Locale", "lang setlocale getlocale") for language in languages.split(): sloc = locale.setlocale(locale.LC_ALL, language) gloc = locale.getlocale() record = t(language, sloc, gloc) if gloc[0][2] == "_": d["unix-like"].append(record) else: d["windows-like"].append(record) pprint.pprint(dict(d)) ## output n:\>C:\Miniconda3\python.exe N:\temp\loc.py ------------------------------------------------------------- {'unix-like': [Locale(lang='french', setlocale='French_France.1252', getlocale=('fr_FR', 'cp1252')), Locale(lang='german', setlocale='German_Germany.1252', getlocale=('de_DE', 'cp1252')), Locale(lang='portuguese', setlocale='Portuguese_Brazil.1252', getlocale=('pt_BR', 'cp1252')), Locale(lang='spanish', setlocale='Spanish_Spain.1252', getlocale=('es_ES', 'cp1252'))], ------------------------------------------------------------- 'windows-like': [Locale(lang='chinese', setlocale="Chinese (Simplified)_People's Republic of China.936", getlocale=("Chinese (Simplified)_People's Republic of China", '936')), Locale(lang='czech', setlocale='Czech_Czech Republic.1250', getlocale=('Czech_Czech Republic', '1250')), Locale(lang='danish', setlocale='Danish_Denmark.1252', getlocale=('Danish_Denmark', '1252')), Locale(lang='dutch', setlocale='Dutch_Netherlands.1252', getlocale=('Dutch_Netherlands', '1252')), Locale(lang='english', setlocale='English_United States.1252', getlocale=('English_United States', '1252')), Locale(lang='finnish', setlocale='Finnish_Finland.1252', getlocale=('Finnish_Finland', '1252')), Locale(lang='greek', setlocale='Greek_Greece.1253', getlocale=('Greek_Greece', '1253')), Locale(lang='hungarian', setlocale='Hungarian_Hungary.1250', getlocale=('Hungarian_Hungary', '1250')), Locale(lang='icelandic', setlocale='Icelandic_Iceland.1252', getlocale=('Icelandic_Iceland', '1252')), Locale(lang='italian', setlocale='Italian_Italy.1252', getlocale=('Italian_Italy', '1252')), Locale(lang='japanese', setlocale='Japanese_Japan.932', getlocale=('Japanese_Japan', '932')), Locale(lang='korean', setlocale='Korean_Korea.949', getlocale=('Korean_Korea', '949')), Locale(lang='norwegian', setlocale='Norwegian (Bokm?l)_Norway.1252', getlocale=('Norwegian (Bokm?l)_Norway', '1252')), Locale(lang='polish', setlocale='Polish_Poland.1250', getlocale=('Polish_Poland', '1250')), Locale(lang='russian', setlocale='Russian_Russia.1251', getlocale=('Russian_Russia', '1251')), Locale(lang='slovak', setlocale='Slovak_Slovakia.1250', getlocale=('Slovak_Slovakia', '1250')), Locale(lang='swedish', setlocale='Swedish_Sweden.1252', getlocale=('Swedish_Sweden', '1252')), Locale(lang='turkish', setlocale='Turkish_Turkey.1254', getlocale=('Turkish_Turkey', '1254'))]} ---------- components: Library (Lib) messages: 235630 nosy: fomcl at yahoo.com priority: normal severity: normal status: open title: Windows getlocale unix-like with french, german, portuguese, spanish type: behavior versions: Python 2.7, Python 3.3, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 22:06:45 2015 From: report at bugs.python.org (Alexander Belopolsky) Date: Mon, 09 Feb 2015 21:06:45 +0000 Subject: [New-bugs-announce] [issue23426] run_setup is broken in distutils Message-ID: <1423516005.28.0.423247861021.issue23426@psf.upfronthosting.co.za> New submission from Alexander Belopolsky: With the following simple setup.py $ cat setup.py from distutils.core import setup from distutils.command.install import install class install1(install): sub_commands = install.sub_commamnds setup(name='bug', cmdclass={'install': install1}) I get >>> from distutils.core import run_setup >>> run_setup('setup.py') Traceback (most recent call last): File "", line 1, in File "Lib/distutils/core.py", line 216, in run_setup exec(f.read(), g, l) File "", line 4, in File "", line 5, in install1 NameError: name 'install' is not defined Furthermore, on an even simpler setup.py: $ cat setup1.py from distutils.core import setup setup(name='bug') run_setup runs, but clobbers sys.argv: >>> from distutils.core import run_setup >>> run_setup('setup1.py', script_args=['--name']) bug >>> import sys;sys.argv ['setup1.py', '--name'] ---------- components: Distutils messages: 235632 nosy: belopolsky, dstufft, eric.araujo priority: normal severity: normal status: open title: run_setup is broken in distutils type: behavior versions: Python 2.7, Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 22:09:28 2015 From: report at bugs.python.org (Jan-Philip Gehrcke) Date: Mon, 09 Feb 2015 21:09:28 +0000 Subject: [New-bugs-announce] [issue23427] Python should expose command when invoked with -c Message-ID: <1423516168.08.0.69949754755.issue23427@psf.upfronthosting.co.za> New submission from Jan-Philip Gehrcke: When Python is invoked with the `-c command` switch, the command string does not get exposed in sys.argv: $ python -c "import sys; print(sys.argv)" ['-c'] $ python -c "import sys; print(sys.argv)" arg1 ['-c', 'arg1'] The command string does not get exposed anywhere, AFAIK, so it is inaccessible from within Python programs. There might be application scenarios in which it is useful to access the command string, such as for debugging purposes. One scenario is when a Python session should be able to "re-spawn" itself in a subprocess (I came across this question on StackOverflow: http://stackoverflow.com/q/28412903/145400) I propose to make the command string accessible. If you agree that it might make sense, the question is *how/where* to expose it. One possible way is to retain it in sys.argv, as in this example: $ python -c "import sys; print(sys.argv)" "arg1" ['-c', 'import sys; print(sys.argv)', 'arg1'] The current sys.argv docs say > If the command was executed using the -c command line option to > the interpreter, argv[0] is set to the string '-c'. This sentence could then be adjusted to "[...], argv[0] is set to the string '-c', and argv[1] contains the command." This method breaks existing applications that are started with the -c method and that consume command line arguments in a sys.argv[1:] fashion. The tests in Lib/test/test_cmd_line.py all pass, however. A second method would be to change sys.argv[0] from '-c' to '-c command'. This would break existing applications that check for sys.argv[0] == 'c'. A third method would be to leave sys.argv as it is, and expose the command with a new attribute in the sys module. I have attached a patch for variant 1 (passes all tests in Lib/test/test_cmd_line.py), to demonstrate which code is affected: the translation from the "real" argv to sys' argv is triggered in Modules/main.c. The patch does not change behavior of '-m' (it's funny, however, that the current version of main.c at first replaces the module string with '-m', whereas the runpy module later on replaces '-m' with the path to the module file anyway.). As a side node, I figure that the sys.argv documentation should be adjusted to properly reflect the -m behavior, which is: $ ./python -m testmodule foo testmodule sys.argv: ['/data/local/pythondev/pythontip/cpython/testmodule.py', 'foo'] Let me hear your comments, and I am willing to work on code and doc patches, thanks! ---------- assignee: docs at python components: Documentation, Interpreter Core files: sys_argv_cmd.patch keywords: patch messages: 235633 nosy: docs at python, georg.brandl, haypo, jgehrcke, pitrou priority: normal severity: normal status: open title: Python should expose command when invoked with -c type: enhancement versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file38065/sys_argv_cmd.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 22:34:27 2015 From: report at bugs.python.org (STINNER Victor) Date: Mon, 09 Feb 2015 21:34:27 +0000 Subject: [New-bugs-announce] [issue23428] Use the monotonic clock for thread conditions on POSIX platforms Message-ID: <1423517667.9.0.82075564578.issue23428@psf.upfronthosting.co.za> New submission from STINNER Victor: Python 3.5 now requires a monotonic clock to start and has the C function _PyTime_monotonic(). Python/condvar.h and Python/thread_pthread.h should use the monotonic clock CLOCK_MONOTONIC, not the system clock CLOCK_REALTIME. See the PEP 418 for the rationale. Most platforms support pthread_condattr_setclock(CLOCK_MONOTONIC), except Mac OS X and old versions of Android. The glib looks to use pthread_cond_timedwait_relative_np() for Mac OS X: https://mail.gnome.org/archives/commits-list/2014-February/msg07782.html Note: Android had non-standard pthread_cond_timedwait_monotonic() and pthread_cond_timedwait_monotonic_np() functions. Android is not a official supported platform, and newer Android version now support pthread_condattr_setclock(). I prefer to not support old Android versions (yet). https://android-review.googlesource.com/#/c/83881/ -- For Windows, SleepConditionVariableSRW() is used on Windows 7 and newer, otherwise WaitForSingleObjectEx() is used. By the way, the check looks to be done during the compilation. I should check which Windows version is used to build Python... SleepConditionVariableSRW() and WaitForSingleObjectEx() both take a relative timeout, so they don't use (directly) the system clock. I don't see any required change for Windows. According to the PEP 418: "WaitForSingleObject() uses the same timer as GetTickCount() with the same precision." Hum, it is not possible to interrupt such wait() with CTRL+c? time.sleep() is implemented with WaitForSingleObjectEx() with _PyOS_SigintEvent(). It doesn't look to be the case here. ---------- components: Interpreter Core messages: 235637 nosy: haypo priority: normal severity: normal status: open title: Use the monotonic clock for thread conditions on POSIX platforms versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 9 23:40:25 2015 From: report at bugs.python.org (Justin) Date: Mon, 09 Feb 2015 22:40:25 +0000 Subject: [New-bugs-announce] [issue23429] -5**4 returns -625 instead of 625 Message-ID: <1423521625.37.0.588744177258.issue23429@psf.upfronthosting.co.za> New submission from Justin: C:\Users\Justin>python Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:15:05) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print(-5**4) -625 >>> #note...this should be 625 and not a negative 625 ---------- messages: 235642 nosy: gilbe024 priority: normal severity: normal status: open title: -5**4 returns -625 instead of 625 type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 04:34:08 2015 From: report at bugs.python.org (Martin Panter) Date: Tue, 10 Feb 2015 03:34:08 +0000 Subject: [New-bugs-announce] [issue23430] socketserver.BaseServer.handle_error() should not catch exiting exceptions Message-ID: <1423539248.06.0.0190891390955.issue23430@psf.upfronthosting.co.za> New submission from Martin Panter: I propose changing the socket servers to not suppress exceptions that are meant to exit the interpreter. This is most applicable to single threaded servers, but my patch does the same thing for multithreading servers. It no longer catches exceptions that are not derived from the Exception class, such as KeyboardInterrupt and SystemExit. The shutdown_request() method is still called in all cases though. I also added a test for the forking server?s handle_error() method. ---------- components: Library (Lib) files: socketserver-exit.patch keywords: patch messages: 235662 nosy: vadmium priority: normal severity: normal status: open title: socketserver.BaseServer.handle_error() should not catch exiting exceptions type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38074/socketserver-exit.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 08:09:16 2015 From: report at bugs.python.org (ww0115) Date: Tue, 10 Feb 2015 07:09:16 +0000 Subject: [New-bugs-announce] [issue23431] Idle Application Not Responding Message-ID: <1423552156.48.0.822437580956.issue23431@psf.upfronthosting.co.za> New submission from ww0115: I recently downloaded Python 3.4 on my Mac (running on Yosemite 10.10.2) and every time I try and open Idle, the icon just keeps jumping and the application never opens. When I click on the icon after it stops jumping it just says "Application Not Responding". I've also tried downloading ActiveTCL to see if it helps but it doesn't as that application doesn't open either. Any suggestions? ---------- components: IDLE messages: 235675 nosy: ww0115 priority: normal severity: normal status: open title: Idle Application Not Responding versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 13:29:15 2015 From: report at bugs.python.org (Berker Peksag) Date: Tue, 10 Feb 2015 12:29:15 +0000 Subject: [New-bugs-announce] [issue23432] Duplicate content in SystemExit documentation Message-ID: <1423571355.15.0.388380550498.issue23432@psf.upfronthosting.co.za> New submission from Berker Peksag: >From https://docs.python.org/3/library/exceptions.html#SystemExit Also, this exception derives directly from BaseException and not Exception, since it is not technically an error. and The exception inherits from BaseException instead of Exception so that it is not accidentally caught by code that catches Exception. This allows the exception to properly propagate up and cause the interpreter to exit. Here is a patch to remove the duplicate content and document the code attribute properly. ---------- assignee: docs at python components: Documentation files: systemexit.diff keywords: patch messages: 235684 nosy: berker.peksag, docs at python priority: normal severity: normal stage: patch review status: open title: Duplicate content in SystemExit documentation versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38082/systemexit.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 15:14:31 2015 From: report at bugs.python.org (Matthias Klose) Date: Tue, 10 Feb 2015 14:14:31 +0000 Subject: [New-bugs-announce] [issue23433] undefined behaviour in faulthandler.c, exposed by GCC 5 Message-ID: <1423577671.31.0.124602995451.issue23433@psf.upfronthosting.co.za> New submission from Matthias Klose: richi: https://github.com/nemomobile-packages/python3/blob/master/Modules/faulthandler.c#L903 richi: LD_LIBRARY_PATH=/builddir/build/BUILD/Python-3.4.2/build/debug/ /builddir/build/BUILD/Python-3.4.2/build/debug/python -E -c 'import faulthandler; faulthandler.enable(); faulthandler._stack_overflow()' i.e. what Jakub says richi: the function certainly shouldn't return address of a local variable; dunno what would happen if you just cast that to an integer though richi: and it better should do something to avoid tail calls there richi: the if (sp < min_sp || max_sp < sp) is also undefined behavior ah, I get python segfaults building some extensions instead (but can't reproduce locally...) jakub: so what's your fix? richi: I don't have a fix, we just documented it not to be a gcc fault, we'll leave fixing to the package maintainer ah, I see richi: dunno if e.g. uintptr_t x; memcpy (&x, &sp, sizeof (x)); would DTRT and be portable enough richi: and then of course pass uintptr_t min_sp/max_sp, compare the x against that etc. well, just (uintptr_t)&buffer should be enough ---------- components: Extension Modules messages: 235685 nosy: doko priority: normal severity: normal status: open title: undefined behaviour in faulthandler.c, exposed by GCC 5 type: crash versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 15:54:47 2015 From: report at bugs.python.org (Myroslav Opyr) Date: Tue, 10 Feb 2015 14:54:47 +0000 Subject: [New-bugs-announce] [issue23434] RFC6266 support Message-ID: <1423580087.81.0.367611213854.issue23434@psf.upfronthosting.co.za> New submission from Myroslav Opyr: cgi.FieldStorage has problems parsing the multipart/form-data request with file fields with non-latin filenames. It drops the filename parameter formatted according to RFC6266 [1] (most modern browsers do). There is already python implementation for that RFC in rfc6266 module [2]. Ref: [1] https://tools.ietf.org/html/rfc6266 [2] https://pypi.python.org/pypi/rfc6266 ---------- components: Library (Lib) messages: 235688 nosy: Myroslav.Opyr priority: normal severity: normal status: open title: RFC6266 support type: enhancement versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 19:29:41 2015 From: report at bugs.python.org (Ivailo Monev) Date: Tue, 10 Feb 2015 18:29:41 +0000 Subject: [New-bugs-announce] [issue23435] installation with full path as prefix incomplete Message-ID: <1423592981.49.0.0894192959587.issue23435@psf.upfronthosting.co.za> New submission from Ivailo Monev: Installing Python with prefix that equals "/" seems to not install the dymic libraries like _collections. The steps I performed to install it: ./configure --prefix=/ \ --with-threads \ --enable-shared \ --enable-ipv6 \ --with-system-ffi \ --with-system-expat make make install I saw that during the installation that the sharedinstall rule from Makefile.pre (originally defined in Makefile.pre.in) fails to remove $(DESTDIR)$(DESTSHARED)/_sysconfigdata.py* so I manually created the leading directory and touched the file it was looking for (I could've modified Makefile.pre too) and it actually worked - lib-dynload was created and the .so files where installed in it. I have tried using empty string as prefix but that causes runtime issues since - Python is not able to find the site-packages directory. Cheers! ---------- messages: 235699 nosy: Ivailo.Monev priority: normal severity: normal status: open title: installation with full path as prefix incomplete _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 20:30:43 2015 From: report at bugs.python.org (Kevin Rocard) Date: Tue, 10 Feb 2015 19:30:43 +0000 Subject: [New-bugs-announce] [issue23436] xml.dom.minidom.Element.ownerDocument is hiden Message-ID: <1423596643.73.0.960398071264.issue23436@psf.upfronthosting.co.za> New submission from Kevin Rocard: Extracted from xml.dom.minidom: ~~~ Node(...): ... ownerDocument = None ... Element(Node): __slots__=('ownerDocument', ...) ... ~~~ As Element declares an ownerDocument attribute in __slots__, Node's ownerDocument attribute is hidden: ~~~ class B: b=1; class D(B): __slots__={'b'} D().b -> AttributeError ~~~ This leads to a strange behaviour were accessing a base attribute fails with an attribute error. Should the Node.ownerDocument attribute not be removed? Or its name removed from the Element.__slots__ list? Ie have the attribute in the base or the derivative, but not both. Independent note: says: > When inheriting from a class without __slots__ [Node], the __dict__ attribute of that class will always be accessible, so a __slots__ definition in the subclass [Element] is meaningless. So as for as I understand Element.__slots__ does not reduce the Element() footprint (it was introduced for that). ---------- components: Library (Lib) messages: 235700 nosy: krocard priority: normal severity: normal status: open title: xml.dom.minidom.Element.ownerDocument is hiden type: enhancement versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 21:46:26 2015 From: report at bugs.python.org (Paul Moore) Date: Tue, 10 Feb 2015 20:46:26 +0000 Subject: [New-bugs-announce] [issue23437] Make user scripts directory versioned on Windows Message-ID: <1423601186.31.0.692326413397.issue23437@psf.upfronthosting.co.za> New submission from Paul Moore: Patch to make the user scripts directory on Windows %APPDATA%\Python\PythonXY\Scripts rather than %APPDATA%\Python\Scripts. See the thread "PEP 370 - per-user scripts directory on Windows" (Feb 10 2015) on python-dev for discussion, but essentially this is to ensure that different Python versions cannot overwrite each others' package scripts when packages are installed into the user site directory. Not tested, as I don't yet have Visual Studio 2015 installed. I'll try to obtain and install it when I get the chance. ---------- assignee: steve.dower components: Windows files: userscripts.patch keywords: needs review, patch messages: 235702 nosy: pmoore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Make user scripts directory versioned on Windows type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38085/userscripts.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 23:01:22 2015 From: report at bugs.python.org (pyed) Date: Tue, 10 Feb 2015 22:01:22 +0000 Subject: [New-bugs-announce] [issue23438] HTMLParser don't know how to deal with 'ampersand' Message-ID: <1423605682.36.0.595483679722.issue23438@psf.upfronthosting.co.za> New submission from pyed: the attached file use example from : https://docs.python.org/3/library/html.parser.html and it will show different cases where HTMLParser fail to parse '&' and '&' ---------- components: Library (Lib) files: htmlparser_bugs.py messages: 235714 nosy: pyed priority: normal severity: normal status: open title: HTMLParser don't know how to deal with 'ampersand' type: behavior versions: Python 3.4 Added file: http://bugs.python.org/file38089/htmlparser_bugs.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 10 23:41:51 2015 From: report at bugs.python.org (Martin Panter) Date: Tue, 10 Feb 2015 22:41:51 +0000 Subject: [New-bugs-announce] [issue23439] Fixed http.client.__all__ and added a test Message-ID: <1423608111.44.0.417026855918.issue23439@psf.upfronthosting.co.za> New submission from Martin Panter: This patch was split off my patch for Issue 3566, since it should be less controversial. It adds the HTTPMessage class and the parse_headers() function to __all__. I?m not too sure on the status of the parse_headers() function. It is not mentioned in the ?http.client? documentation, but is referenced by the ?http.server? module?s BaseHTTPRequestHandler.headers entry. Perhaps it should be left unexported? ---------- components: Library (Lib) messages: 235719 nosy: vadmium priority: normal severity: normal status: open title: Fixed http.client.__all__ and added a test type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 08:42:51 2015 From: report at bugs.python.org (Martin Panter) Date: Wed, 11 Feb 2015 07:42:51 +0000 Subject: [New-bugs-announce] [issue23440] Extend http.server.SimpleHTTPRequestHandler testing Message-ID: <1423640571.78.0.653754704207.issue23440@psf.upfronthosting.co.za> New submission from Martin Panter: These are some additions to the ?http.server? module tests. It is an updated version of index-test.2.patch that I posted at Issue 23255. * Tests that index.html is served, rather than an automatic directory listing * Tests that there is no extra data sent after the response ---------- components: Tests files: simple-http-testing.patch keywords: patch messages: 235730 nosy: vadmium priority: normal severity: normal status: open title: Extend http.server.SimpleHTTPRequestHandler testing type: enhancement Added file: http://bugs.python.org/file38091/simple-http-testing.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 11:21:44 2015 From: report at bugs.python.org (Armin Rigo) Date: Wed, 11 Feb 2015 10:21:44 +0000 Subject: [New-bugs-announce] [issue23441] rlcompleter: tab on empty prefix => insert spaces Message-ID: <1423650104.74.0.902857510979.issue23441@psf.upfronthosting.co.za> New submission from Armin Rigo: In the interactive prompt: >>> if 1: ... Pressing tab here should just insert 4 spaces. It makes no sense to display all 500 possible "completions" of the empty word, and using tab to indent is a very common feature of any editor with a Python mode. Patch attached. ---------- components: Library (Lib) files: rlcompleter_tab2space.diff keywords: patch messages: 235733 nosy: arigo priority: normal severity: normal status: open title: rlcompleter: tab on empty prefix => insert spaces type: enhancement versions: Python 3.5, Python 3.6 Added file: http://bugs.python.org/file38093/rlcompleter_tab2space.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 12:46:11 2015 From: report at bugs.python.org (Martin Panter) Date: Wed, 11 Feb 2015 11:46:11 +0000 Subject: [New-bugs-announce] [issue23442] http.client.REQUEST_HEADER_FIELDS_TOO_LARGE renamed in 3.5 Message-ID: <1423655171.77.0.0216146047129.issue23442@psf.upfronthosting.co.za> New submission from Martin Panter: This is a regression caused by the new HTTPStatus enum from Issue 21793. RFC 6585 uses the plural ?Fields?, so maybe the new enum symbol needs renaming. $ python3.4 Python 3.4.2 (default, Oct 8 2014, 13:44:52) [GCC 4.9.1 20140903 (prerelease)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import http.client >>> http.client.REQUEST_HEADER_FIELDS_TOO_LARGE 431 >>> http.client.REQUEST_HEADER_FIELD_TOO_LARGE Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'REQUEST_HEADER_FIELD_TOO_LARGE' >>> $ ./python Python 3.5.0a0 (qbase qtip simple-http-testing.patch tip:f3fadbfb10ba, Feb 11 2015, 07:18:07) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import http.client >>> http.client.REQUEST_HEADER_FIELDS_TOO_LARGE Traceback (most recent call last): File "", line 1, in AttributeError: module 'http.client' has no attribute 'REQUEST_HEADER_FIELDS_TOO_LARGE' >>> http.client.REQUEST_HEADER_FIELD_TOO_LARGE ---------- components: Library (Lib) messages: 235737 nosy: demian.brecht, vadmium priority: normal severity: normal status: open title: http.client.REQUEST_HEADER_FIELDS_TOO_LARGE renamed in 3.5 type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 15:19:22 2015 From: report at bugs.python.org (Ken Marsh) Date: Wed, 11 Feb 2015 14:19:22 +0000 Subject: [New-bugs-announce] [issue23443] XMLRPCLIB Exception uses str not class or instance Message-ID: <1423664362.4.0.817157506262.issue23443@psf.upfronthosting.co.za> New submission from Ken Marsh: xmlrpclib.Fault: :exceptions must be classes or instances, not str" Exception handling appears to be improperly coded. Occurs when far side gives an unexpected response to an RPC call. ---------- components: XML messages: 235748 nosy: kmarsh priority: normal severity: normal status: open title: XMLRPCLIB Exception uses str not class or instance type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 16:50:15 2015 From: report at bugs.python.org (Thomas Chiroux) Date: Wed, 11 Feb 2015 15:50:15 +0000 Subject: [New-bugs-announce] [issue23444] HCI Bluetooth socket bind error on an arm crosscompiler environment Message-ID: <1423669815.59.0.783666779084.issue23444@psf.upfronthosting.co.za> New submission from Thomas Chiroux: This bug bellow occurs only on my crosscompiled environment on arm (marvell armada 166): arm-pxa168-linux-gnueabi It does not seems to be a cross-compile issue: python compiles without problem and all unittests pass on the target device. description and first clues --------------------------- The problem is easyly reproducted using this script: #!/usr/bin/python import socket sock = socket.socket(socket.AF_BLUETOOTH, socket.SOCK_RAW, socket.BTPROTO_HCI) sock.bind((0,)) which raises the following exception when run on the target device: Traceback (most recent call last): File "./test_bt.py", line 4, in sock.bind((0,)) OSError: [Errno 22] Invalid argument This does not give much clues, but strace does (i've filtered to display the two significant parts of strace) socket(PF_BLUETOOTH, SOCK_RAW|SOCK_CLOEXEC, 1) = 3 bind(3, {sa_family=AF_UNSPEC, sa_data="\0\0\360\35\251\266s\316(U\3\0\0\0"}, 6) = -1 EINVAL (Invalid argument) (on a working environment, including arm, like a raspberry pi, strace gives the following result (and no traceback of course): socket(PF_BLUETOOTH, SOCK_RAW|SOCK_CLOEXEC, 1) = 3 bind(3, {sa_family=AF_BLUETOOTH, sa_data="\0\0\0\0\0\0X\352\243\266\0\24\265\266"}, 6) = 0 So, on the armada166, between the socket creation and the bind we lost the socket family (AF_UNSPEC instead of AF_BLUETOOTH). And That's why bind returns invalid argument. socketmodule and PyArg_ParseTuple --------------------------------- Now let's look at Modules/socketmodule.c: After some digging, i've found that the problem is in getsockaddrarg, in the AF_BLUETOOTH / BTPROTO_HCI case and more precisely on this line: https://hg.python.org/cpython/file/ab2c023a9432/Modules/socketmodule.c#l1449 reproducted here: if (!PyArg_ParseTuple(args, "i", &_BT_HCI_MEMB(addr, dev))) { When we execute the PyArg_ParseTuple, the addr->hci_family is crunched (by zeros with my previous python sample). At this same place, i've done the following test: char buffer[8]; memset(buffer, 0x55, 8); if (!PyArg_ParseTuple(args, "i", buffer) { PyErr_SetString(PyExc_OSError, "getsockaddrarg: " "wrong format"); return 0; } printf("CL: %d %d %d %d %d %d %d %d\n", buffer[0], buffer[1], buffer[2], buffer[3], buffer[4], buffer[5], buffer[6], buffer[7]); memset(buffer, 0xAA, 8); if (!PyArg_ParseTuple(args, "i", buffer+1) { PyErr_SetString(PyExc_OSError, "getsockaddrarg: " "wrong format"); return 0; } printf("CL+1: %d %d %d %d %d %d %d %d\n", buffer[0], buffer[1], buffer[2], buffer[3], buffer[4], buffer[5], buffer[6], buffer[7]); memset(buffer, 0xBB, 8); if (!PyArg_ParseTuple(args, "i", buffer+2) { PyErr_SetString(PyExc_OSError, "getsockaddrarg: " "wrong format"); return 0; } printf("CL+2: %d %d %d %d %d %d %d %d\n", buffer[0], buffer[1], buffer[2], buffer[3], buffer[4], buffer[5], buffer[6], buffer[7]); memset(buffer, 0xcc, 8); if (!PyArg_ParseTuple(args, "i", buffer+3) { PyErr_SetString(PyExc_OSError, "getsockaddrarg: " "wrong format"); return 0; } printf("CL+3: %d %d %d %d %d %d %d %d\n", buffer[0], buffer[1], buffer[2], buffer[3], buffer[4], buffer[5], buffer[6], buffer[7]); and the result is: CL: 0 0 0 0 85 85 85 85 CL+1: 0 0 0 0 170 170 170 170 CL+2: 0 0 0 0 187 187 187 187 CL+3: 0 0 0 0 204 204 204 204 (WTF ??) in a working environnement (tested on raspberry B+ / python 3.4.2 locally compiled) result is what we should expect: CL: 0 0 0 0 85 85 85 85 CL+1: 170 0 0 0 0 170 170 170 CL+2: 187 187 0 0 0 0 187 187 CL+3: 204 204 204 0 0 0 0 204 So on my box if PyArg_ParseTuple write on &addr->hci_dev if write 4 bytes from &addr->hci_family which is 2 bytes BEFORE &addr->hci_dev At this time I can not understand how it's possible. Remarks and patch ----------------- Now I have several remarks and a working patch. * remark/question 1: why does PyArg_ParseTuple parse an int when addr->hci_dev is an unsigned short ? even in normal situation, when it works, writing on &addr->hci_dev overflow on the next two bytes which are btw addr->channel (more on that later) * remark/question 2: i've tried to dig more deeply inside PyArg_ParseTuple and found another odd thing, but I did not try to change it without knowing what I do: in Python/getargs.c, in convertsimple, int parsing result is not casted before returned: here: https://hg.python.org/cpython/file/ab2c023a9432/Python/getargs.c#l690 (ival is a long). In all other cases (short, unsigned short, char, usigned char), they are casted before return. [disclosure: i've tested to add the cast and relaunched my bind test, it did not change anything, but it's still strange for me] * Now a working patch: here below and attached a working patch which results on a good socket bind, but now completely satisfiying: --- Python-3.4.2/Modules/socketmodule.c 2014-10-08 10:18:15.000000000 +0200 +++ CC_PYTHON/Python-3.4.2/Modules/socketmodule.c 2015-02-11 15:42:35.173455634 +0100 @@ -1446,11 +1446,12 @@ getsockaddrarg(PySocketSockObject *s, Py return 0; #else _BT_HCI_MEMB(addr, family) = AF_BLUETOOTH; - if (!PyArg_ParseTuple(args, "i", &_BT_HCI_MEMB(addr, dev))) { + if (!PyArg_ParseTuple(args, "H", &_BT_HCI_MEMB(addr, dev))) { PyErr_SetString(PyExc_OSError, "getsockaddrarg: " "wrong format"); return 0; } + _BT_HCI_MEMB(addr, channel) = HCI_CHANNEL_RAW; #endif *len_ret = sizeof *addr; return 1; in short: I parse now an unsigned short instead of parsing an int which gives me a two bytes long elements which is stored well on addr->hci_dev without overloading addr->hci_family. But this modification alone is not enough: addr->hci_channel needed a good value. that's why i added _BT_HCI_MEMB(addr, channel) = HCI_CHANNEL_RAW; which sets addr->hci_channel to zero. (without this line, any value could be here) And that led me to another question/problem: how is hci_channel normally handled ? It does not seems to be a valid parameter of bind; behaviour without the patch will erase hci_channel while storing int value in hci_dev, so theorically we can assign a well defined int value in our bind method to both assign the wanted value in hci_dev and hci_channel, but it does not seems to be a proper way to do it. ---------- components: Extension Modules files: bluetooth_bind_arm.patch keywords: patch messages: 235751 nosy: Thomas.Chiroux priority: normal severity: normal status: open title: HCI Bluetooth socket bind error on an arm crosscompiler environment type: behavior versions: Python 3.4 Added file: http://bugs.python.org/file38097/bluetooth_bind_arm.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 17:28:42 2015 From: report at bugs.python.org (Antoine Pitrou) Date: Wed, 11 Feb 2015 16:28:42 +0000 Subject: [New-bugs-announce] [issue23445] Use -Og for debug builds Message-ID: <1423672122.39.0.391181135666.issue23445@psf.upfronthosting.co.za> New submission from Antoine Pitrou: Recent gcc versions have an optimization level named "-Og". It enables all optimizations that do not break debugging. Here is a patch that uses it on debug builds. Without the patch I get the following time for the whole test suite: $ time ./python -m test -j12 [...] real 3m51.670s user 11m53.039s sys 0m28.300s With the patch it goes down to: real 2m36.378s user 7m13.743s sys 0m29.728s So the test suite is 33% faster to pass. ---------- components: Build, Tests files: og.patch keywords: patch messages: 235754 nosy: pitrou priority: normal severity: normal stage: patch review status: open title: Use -Og for debug builds type: enhancement versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38098/og.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 18:30:15 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Wed, 11 Feb 2015 17:30:15 +0000 Subject: [New-bugs-announce] [issue23446] Use PyMem_New instead of PyMem_Malloc Message-ID: <1423675815.31.0.721977402363.issue23446@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch replaces PyMem_Malloc with PyMem_New if the former is used in the form PyMem_Malloc(len * sizeof(type)). This can fix possible overflow errors and makes the code cleaner. ---------- components: Extension Modules, Interpreter Core files: pymem_new.patch keywords: patch messages: 235758 nosy: benjamin.peterson, haypo, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Use PyMem_New instead of PyMem_Malloc type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38099/pymem_new.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 19:38:45 2015 From: report at bugs.python.org (Antonio Cota) Date: Wed, 11 Feb 2015 18:38:45 +0000 Subject: [New-bugs-announce] [issue23447] Relative imports with __all__ attribute Message-ID: <1423679925.56.0.995999383567.issue23447@psf.upfronthosting.co.za> New submission from Antonio Cota: That's the situation: a/ __init__.py first.py second.py #init.py __all__ = ['second', 'first'] print('i\'m starting the directory') #first.py print('hi, i\'m the first') from . import * #second.py print('hi, i\'m the second') >From the interactive prompt: >>> import a.first i'm starting the directory hi, i'm the first hi, i'm the second Traceback (most recent call last): File "", line 1, in File "/home/antox/Scrivania/a/first.py", line 2, in from . import * AttributeError: module 'a' has no attribute 'first' It's pretty weird. ---------- messages: 235761 nosy: antox priority: normal severity: normal status: open title: Relative imports with __all__ attribute type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 11 19:40:23 2015 From: report at bugs.python.org (Neil Gierman) Date: Wed, 11 Feb 2015 18:40:23 +0000 Subject: [New-bugs-announce] [issue23448] urllib2 needs to remove scope from IPv6 address when creating Host header Message-ID: <1423680023.83.0.120732364895.issue23448@psf.upfronthosting.co.za> New submission from Neil Gierman: Using a scoped IPv6 address with urllib2 creates an invalid Host header that Apache will not accept. IP = "fe80::0000:0000:0000:0001%eth0" req = urllib2.Request("http://[" + IP + "]/") req.add_header('Content-Type', 'application/json') res = urllib2.urlopen(req, json.dumps(data)) Apache will reject the above request because the Host header is "[fe80::0000:0000:0000:0001%eth0]". This behavior was reported to Apache at https://issues.apache.org/bugzilla/show_bug.cgi?id=35122 and the Apache devs will not fix this as there are new RFCs prohibiting scopes in the Host header. Firefox had the same issue and their fix was to strip out the scope from the Host header: https://bugzilla.mozilla.org/show_bug.cgi?id=464162 and http://hg.mozilla.org/mozilla-central/rev/bb80e727c531. My suggestion is to change urllib2.py's do_request_ method from: if not request.has_header('Host'): request.add_unredirected_header('Host', sel_host) to: if not request.has_header('Host'): request.add_unredirected_header('Host', re.compile(r"%.*$").sub("", sel_host, 1)) I have not tested this patch to urllib2.py however I am now using similar logic in my code to override the Host header when I create my request: IP = "fe80::0000:0000:0000:0001%eth0" req = urllib2.Request("http://[" + IP + "]/") req.add_header('Host', '[' + re.compile(r"%.*").sub("", IP, 1) + ']') req.add_header('Content-Type', 'application/json') res = urllib2.urlopen(req, json.dumps(data)) ---------- components: Library (Lib) messages: 235762 nosy: ngierman priority: normal severity: normal status: open title: urllib2 needs to remove scope from IPv6 address when creating Host header type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 00:11:09 2015 From: report at bugs.python.org (Mark Lawrence) Date: Wed, 11 Feb 2015 23:11:09 +0000 Subject: [New-bugs-announce] [issue23449] Fatal errors rebuilding 3.5 from Visual Studio Windows 8.1 64 bit Message-ID: <1423696269.22.0.462965146464.issue23449@psf.upfronthosting.co.za> New submission from Mark Lawrence: If I select "Rebuild Solution" I get this:- error C1083: Cannot open include file: 'tcl.h': No such file or directory C:\cpython\Modules\_tkinter.c error C1083: Cannot open include file: 'tcl.h': No such file or directory C:\cpython\Modules\tkappinit.c I'm assuming that this should never happen. ---------- components: Build, Windows messages: 235776 nosy: BreamoreBoy, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Fatal errors rebuilding 3.5 from Visual Studio Windows 8.1 64 bit type: compile error versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 00:21:04 2015 From: report at bugs.python.org (Mark Lawrence) Date: Wed, 11 Feb 2015 23:21:04 +0000 Subject: [New-bugs-announce] [issue23450] Possible loss of data warnings building 3.5 Visual Studio Windows 8.1 64 bit Message-ID: <1423696864.41.0.743234937427.issue23450@psf.upfronthosting.co.za> New submission from Mark Lawrence: The attached file lists many of the warnings, but note there may be more as tkinter didn't build and is subject to #23449. ---------- components: Build, Windows files: PossibleLossOfData.txt messages: 235777 nosy: BreamoreBoy, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Possible loss of data warnings building 3.5 Visual Studio Windows 8.1 64 bit type: compile error versions: Python 3.5 Added file: http://bugs.python.org/file38100/PossibleLossOfData.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 00:34:06 2015 From: report at bugs.python.org (Mark Lawrence) Date: Wed, 11 Feb 2015 23:34:06 +0000 Subject: [New-bugs-announce] [issue23451] Deprecation warnings building 3.5 Visual Studio Windows 8.1 64 bit Message-ID: <1423697646.35.0.470486317043.issue23451@psf.upfronthosting.co.za> New submission from Mark Lawrence: Raised as a placeholder. ---------- components: Build, Windows files: Deprecations.txt messages: 235778 nosy: BreamoreBoy, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Deprecation warnings building 3.5 Visual Studio Windows 8.1 64 bit type: compile error versions: Python 3.5 Added file: http://bugs.python.org/file38101/Deprecations.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 07:45:59 2015 From: report at bugs.python.org (Mark Lawrence) Date: Thu, 12 Feb 2015 06:45:59 +0000 Subject: [New-bugs-announce] [issue23452] Build errors using VS Express 2013 in win32 mode Message-ID: <1423723559.86.0.395515662536.issue23452@psf.upfronthosting.co.za> New submission from Mark Lawrence: I came across these while looking at #23449. The output is in the attached file as it's cleaner that way. ---------- components: Build, Windows files: Win32BuildErrors.txt messages: 235801 nosy: BreamoreBoy, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Build errors using VS Express 2013 in win32 mode type: compile error versions: Python 3.5 Added file: http://bugs.python.org/file38105/Win32BuildErrors.txt _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 12:19:58 2015 From: report at bugs.python.org (Carl Chenet) Date: Thu, 12 Feb 2015 11:19:58 +0000 Subject: [New-bugs-announce] [issue23453] Opening a stream with tarfile.open() triggers a TypeError: can't concat bytes to str error Message-ID: <1423739998.49.0.706967373594.issue23453@psf.upfronthosting.co.za> New submission from Carl Chenet: I'm trying to use a tar stream to a Python tarfile object but each time I do have a TypeError: can't concat bytes to str error Here is my test: -----8<----- #!/usr/bin/python3.4 import tarfile import sys tarobj = tarfile.open(mode='r|', fileobj=sys.stdin) print(tarobj) tarobj.close() -----8<----- $ tar cvf test.tar.gz tests/ tests/ tests/foo1 tests/foo/ tests/foo/bar $ tar -O -xvf test.tar | ./tarstream.py tests/ tests/foo1 tests/foo/ tests/foo/bar Traceback (most recent call last): File "./tarstream.py", line 6, in tarobj = tarfile.open(mode='r|', fileobj=sys.stdin) File "/usr/lib/python3.4/tarfile.py", line 1578, in open t = cls(name, filemode, stream, **kwargs) File "/usr/lib/python3.4/tarfile.py", line 1470, in __init__ self.firstmember = self.next() File "/usr/lib/python3.4/tarfile.py", line 2249, in next tarinfo = self.tarinfo.fromtarfile(self) File "/usr/lib/python3.4/tarfile.py", line 1082, in fromtarfile buf = tarfile.fileobj.read(BLOCKSIZE) File "/usr/lib/python3.4/tarfile.py", line 535, in read buf = self._read(size) File "/usr/lib/python3.4/tarfile.py", line 543, in _read return self.__read(size) File "/usr/lib/python3.4/tarfile.py", line 569, in __read self.buf += buf TypeError: can't concat bytes to str Regards, Carl Chenet ---------- components: Library (Lib) messages: 235808 nosy: chaica_ priority: normal severity: normal status: open title: Opening a stream with tarfile.open() triggers a TypeError: can't concat bytes to str error type: crash versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 15:31:58 2015 From: report at bugs.python.org (Ulrich Dorsch) Date: Thu, 12 Feb 2015 14:31:58 +0000 Subject: [New-bugs-announce] [issue23454] plistlib and xml.parsers.expat python3 problems with strings and bytes Message-ID: <1423751518.98.0.629522641345.issue23454@psf.upfronthosting.co.za> New submission from Ulrich Dorsch: TypeError: startswith first arg must be str or a tuple of str, not bytes In line 558 of plistlib.py at the beginnging of "def _is_fmt_xml(header)" is the problem, caused by the use of the byte arguments defined in line 555 (prefixes = (b' _______________________________________ From report at bugs.python.org Thu Feb 12 19:48:54 2015 From: report at bugs.python.org (Andrew Dalke) Date: Thu, 12 Feb 2015 18:48:54 +0000 Subject: [New-bugs-announce] [issue23455] file iterator "deemed broken"; can resume after StopIteration Message-ID: <1423766934.46.0.3718920034.issue23455@psf.upfronthosting.co.za> New submission from Andrew Dalke: The file iterator is "deemed broken". As I don't think it should be made non-broken, I suggest the documentation should be changed to point out when file iteration is broken. I also think the term 'broken' is a label with needlessly harsh connotations and should be softened. The iterator documentation uses the term 'broken' like this (quoting here from https://docs.python.org/3.4/library/stdtypes.html): Once an iterator?s __next__() method raises StopIteration, it must continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken. (Older versions comment "This constraint was added in Python 2.3; in Python 2.2, various iterators are broken according to this rule.") An IOBase is supposed to support the iterator protocol (says https://docs.python.org/3.4/library/io.html#io.IOBase ). However, it does not, nor does the documentation say that it's broken in the face of a changing file (eg, when another process appends to a log file). % ./python.exe Python 3.5.0a1+ (default:4883f9046b10, Feb 11 2015, 04:30:46) [GCC 4.8.4] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> f = open("empty") >>> next(f) Traceback (most recent call last): File "", line 1, in StopIteration >>> >>> ^Z Suspended % echo "Hello!" >> empty % fg ./python.exe >>> next(f) 'Hello!\n' This is apparently well-known behavior, as I've come across several references to it on various Python-related lists, including this one from Miles in 2008: https://mail.python.org/pipermail/python-list/2008-September/491920.html . Strictly speaking, file objects are broken iterators: Fredrik Lundh in the same thread ( https://mail.python.org/pipermail/python-list/2008-September/521090.html ) says: it's a design guideline, not an absolute rule The 7+ years of 'broken' behavior in Python suggests that /F is correct. But while 'broken' could be considered a meaningless label, it carries with it some rather negative connotations. It sounds like developers are supposed to make every effort to avoid broken code, when that's not something Python itself does. It also means that my code can be called "broken" solely because it assumed Python file iterators are non-broken. I am not happy when people say my code is broken. It is entirely reasonable that a seek(0) would reset the state and cause next(it) to not continue to raise a StopIteration exception. However, errors can arise when using Python file objects, as an iterator, to parse a log file or any other files which are appended to by another process. Here's an example of code that can break. It extracts the first and last elements of an iterator; more specifically, the first and last lines of a file. If there are no lines it returns None for both values; and if there's only one line then it returns the same line as both values. def get_first_and_last_elements(it): first = last = next(it, None) for last in it: pass return first, last This code expects a non-broken iterator. If passed a file, and the file were 1) initially empty when the next() was called, and 2) appended to by the time Python reaches the for loop, then it's possible for first value to be None while last is a string. This is unexpected, undocumented, and may lead to subtle errors. There are work-arounds, like ensuring that the StopIteration only occurs once: def get_first_and_last_elements(it): first = last = next(it, None) if last is not None: for last in it: pass return first, last but much existing code expects non-broken iterators, such as the Python example implementation at https://docs.python.org/2/library/itertools.html#itertools.dropwhile . (I have a reproducible failure using it, a fork(), and a file iterator with a sleep() if that would prove useful.) Another option is to have a wrapper around file object iterators to keep raising StopIteration, like: def safe_iter(it): yield from it # -or- (line for line in file_iter) but people need to know to do this with file iterators or other potentially broken iterators. The current documentation does not say when file iterators are broken, and I don't know which other iterators are also broken. I realize this is a tricky issue. I don't think it's possible now to change the file's StopIteration behavior. I expect that there is code which depends on the current brokenness, the ability to seek() and re-iterate is useful, and the idea that next() returns text if and only if readline() is not empty is useful and well-entrenched. Pypy has the same behavior as CPython so any change will take some time to propagate to the other implementations. Instead, I'm fine with a documentation change in io.html . It currently says: IOBase (and its subclasses) support the iterator protocol, meaning that an IOBase object can be iterated over yielding the lines in a stream. Lines are defined slightly differently depending on whether the stream is a binary stream (yielding bytes), or a text stream (yielding unicode strings). See readline() below. I suggest adding something like: The file iterator does not completely follow the iterator protocol. If new data is added to the file after the iterator raises a StopIteration then next(file) will resume returning lines. The safest way to iterate over lines in a log file or other changing file is use a generator comprehension: (line for line in file) The iterator may also resume after using seek() to move the file position. You'll note that I failed to use the term "broken". This should really start The file iterator is broken. I find that term rather harsh, and since broken iterators are acceptable in Python, I suggest toning down or qualifying the use of "broken" in stdtypes.html. I have no suggestions for an improved version. ---------- assignee: docs at python components: Documentation messages: 235850 nosy: dalke, docs at python priority: normal severity: normal status: open title: file iterator "deemed broken"; can resume after StopIteration versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 12 22:59:58 2015 From: report at bugs.python.org (STINNER Victor) Date: Thu, 12 Feb 2015 21:59:58 +0000 Subject: [New-bugs-announce] [issue23456] asyncio: add missing @coroutine decorators Message-ID: <1423778398.48.0.478290226002.issue23456@psf.upfronthosting.co.za> New submission from STINNER Victor: coroutine_decorator.patch adds missing @coroutine decorator to coroutine functions and methods in the asyncio module. I'm not sure that it's ok to add @coroutine to __iter__() methods. At least, test_asyncio pass. ---------- components: asyncio files: coroutine_decorator.patch keywords: patch messages: 235857 nosy: gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: asyncio: add missing @coroutine decorators versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38118/coroutine_decorator.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 13 03:47:56 2015 From: report at bugs.python.org (Dwight) Date: Fri, 13 Feb 2015 02:47:56 +0000 Subject: [New-bugs-announce] [issue23457] make test failures Message-ID: <1423795676.28.0.704579624562.issue23457@psf.upfronthosting.co.za> New submission from Dwight: Hi, Looking for assistance in figuring out what caused the following test failures and how to fix the problems. Built and run on an IBM pSeries system running AIX 7.1. Appreciate any help I can get. I am not a software developer. I am compiling this because I need this to build things I need to build firefox. Would really appreciate some help! (Mozilla.org and IBM not interested in providing a working browser for my system.) Bye, Dwight ***Failed Test the were run in verbose mode test_locale failed test_io failed test_ssl failed test_faulthandler test_httpservers test_socket failed test_fileio failed test_distutils failed test_asyncio failed test_mmap failed test_resource failed test_posix failed ---------- components: Tests files: PYTHONFailedTest.log messages: 235871 nosy: dcrs6000 at gmail.com priority: normal severity: normal status: open title: make test failures versions: Python 3.4 Added file: http://bugs.python.org/file38124/PYTHONFailedTest.log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 13 09:31:46 2015 From: report at bugs.python.org (STINNER Victor) Date: Fri, 13 Feb 2015 08:31:46 +0000 Subject: [New-bugs-announce] [issue23458] [2.7] random: make the file descriptor non-inheritable (on POSIX) Message-ID: <1423816306.11.0.133617972899.issue23458@psf.upfronthosting.co.za> New submission from STINNER Victor: Attached patch tries to make the private random file descriptor non-inheritable. It should fix the following issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197376 I tried to write an unit test, but since the PEP 446 is not implemented, unexpected file descriptors are inherited. The test should maybe be run in a subprocess to not inherit all file descriptors created by other unit tests. Note: I removed the stat.S_ISDOOR(st.st_mode) check from Lib/test/subprocessdata/fd_status.py, because stat.S_ISDOOR is not defined in Python 2.7. ---------- components: Interpreter Core files: pep446_random.patch keywords: patch messages: 235880 nosy: alex, haypo, pitrou priority: normal severity: normal status: open title: [2.7] random: make the file descriptor non-inheritable (on POSIX) versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38126/pep446_random.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 13 13:32:39 2015 From: report at bugs.python.org (STINNER Victor) Date: Fri, 13 Feb 2015 12:32:39 +0000 Subject: [New-bugs-announce] [issue23459] Linux: expose the new execveat() syscall Message-ID: <1423830759.75.0.255479243543.issue23459@psf.upfronthosting.co.za> New submission from STINNER Victor: execveat() was added to Linux in kernel 3.19: http://man7.org/linux/man-pages/man2/execveat.2.html It may be interesting to expose it in Python. "The primary aim of adding an execveat syscall is to allow an implementation of fexecve(3) that does not rely on the /proc filesystem. The current glibc version of fexecve(3) is implemented via /proc, which causes problems in sandboxed or otherwise restricted environments." http://lwn.net/Articles/600344/ ---------- components: Extension Modules messages: 235889 nosy: haypo priority: normal severity: normal status: open title: Linux: expose the new execveat() syscall versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 13 18:15:21 2015 From: report at bugs.python.org (Ian Kelly) Date: Fri, 13 Feb 2015 17:15:21 +0000 Subject: [New-bugs-announce] [issue23460] Decimals do not obey ':g' exponential notation formatting rules Message-ID: <1423847721.13.0.666007782647.issue23460@psf.upfronthosting.co.za> New submission from Ian Kelly: >>> '{:g}'.format(D('0.000001')) '0.000001' Formatted with '{:e}', the exponent would be -6, so per the formatting rules described under the 'g' specifier at https://docs.python.org/3/library/string.html#format-specification-mini-language the above should be formatted using exponential notation. ---------- components: Library (Lib) messages: 235904 nosy: ikelly priority: normal severity: normal status: open title: Decimals do not obey ':g' exponential notation formatting rules type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 14 00:10:48 2015 From: report at bugs.python.org (Paul Moore) Date: Fri, 13 Feb 2015 23:10:48 +0000 Subject: [New-bugs-announce] [issue23461] Building on windows modifies importlib.h Message-ID: <1423869048.38.0.453011208999.issue23461@psf.upfronthosting.co.za> New submission from Paul Moore: When building Python (cpython repository, tip revision on the default branch) using Visual Studio 2015 (version downloaded 12/02/2015) I get a message in the build: C:\Work\Projects\cpython\PCbuild\_freeze_importlib.vcxproj(98,5): error : importlib.h has been updated. You will need to rebuild pythoncore to see the changes. This is using the command "PCBuild\build.bat -p x64". When I check hg status after the build, Python\importlib.h shows as modified. (I have 2 changed files in the repo, one a documentation file and the other launcher.c. Neither change should affect importlib). ---------- messages: 235922 nosy: pmoore priority: normal severity: normal status: open title: Building on windows modifies importlib.h versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 14 09:45:26 2015 From: report at bugs.python.org (Jeremy Nicola) Date: Sat, 14 Feb 2015 08:45:26 +0000 Subject: [New-bugs-announce] [issue23462] All os.exec*e variants crash on Windows Message-ID: <1423903526.7.0.282245151833.issue23462@psf.upfronthosting.co.za> New submission from Jeremy Nicola: On Windows 7, using python 3.4.2 32 bits MSVCv.1600 or 3.4.1 64 bits, all the os.exec*e variants crash: os.execle('C:/Python34/python.exe','Python.exe','-V',{}) os.execve('C:/Python34/python.exe',['python.exe','-V'],{}) os.execlpe('C:/Python34/python.exe','python.exe','-V',{}) os.execvpe('C:/Python34/python.exe',['python.exe','-V'],{}) Without any error message, windows will just open a "Python.exe has stopped working" window, be the scripts run from an interactive shell or invoking python script.py On the other hand, os.execl('C:/Python34/python.exe','Python.exe','-V') os.execve('C:/Python34/python.exe',['python.exe','-V']) os.execlpe('C:/Python34/python.exe','python.exe','-V') os.execvpe('C:/Python34/python.exe',['python.exe','-V']) will work perfectly. ---------- components: Windows messages: 235952 nosy: nicolaje, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: All os.exec*e variants crash on Windows type: crash versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 14 12:54:00 2015 From: report at bugs.python.org (Tom Edwards) Date: Sat, 14 Feb 2015 11:54:00 +0000 Subject: [New-bugs-announce] [issue23463] Incorrect behaviour when opening files containing colons on Windows Message-ID: <1423914840.96.0.470956533091.issue23463@psf.upfronthosting.co.za> New submission from Tom Edwards: Consider this script: f = open("bug>test.txt",'w') f.write("hello") f.close() On Windows the first line will throw an OSError exception because the character '>' is not valid in an NTFS filename. This is correct. Now consider this script: f = open("bug:test.txt",'w') f.write("hello") f.close() This script will complete without error, and f.write will return 5. This despite the colon character also being invalid in NTFS filenames! The output of the second script is an empty file called "bug" in the working directory. I expect it to throw the same exception as the first script. ---------- components: IO messages: 235964 nosy: Artfunkel priority: normal severity: normal status: open title: Incorrect behaviour when opening files containing colons on Windows type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 14 15:35:14 2015 From: report at bugs.python.org (A. Jesse Jiryu Davis) Date: Sat, 14 Feb 2015 14:35:14 +0000 Subject: [New-bugs-announce] [issue23464] Remove or deprecate JoinableQueue in asyncio docs Message-ID: <1423924514.61.0.0196289031224.issue23464@psf.upfronthosting.co.za> New submission from A. Jesse Jiryu Davis: asyncio.JoinableQueue was once a distinct subclass of asyncio.Queue, now it's just a deprecated alias. Once this is merged into CPython: https://code.google.com/p/tulip/issues/detail?id=220 ...then remove or deprecate JoinableQueue in asyncio-sync.rst ---------- components: asyncio messages: 235971 nosy: emptysquare, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: Remove or deprecate JoinableQueue in asyncio docs versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 14 15:59:15 2015 From: report at bugs.python.org (Paul Moore) Date: Sat, 14 Feb 2015 14:59:15 +0000 Subject: [New-bugs-announce] [issue23465] Implement PEP 486 - Make the Python Launcher aware of virtual environments Message-ID: <1423925955.39.0.302643429195.issue23465@psf.upfronthosting.co.za> New submission from Paul Moore: Implementation of PEP 486 (Make the Python Launcher aware of virtual environments). Tested manually on my local PC - there aren't currently any tests for the launcher that I can see (and I'm not entirely sure how I'd write such a test) so the patch includes code and docs but no tests. ---------- assignee: steve.dower files: pep486.patch keywords: patch messages: 235974 nosy: pmoore, steve.dower, vinay.sajip priority: normal severity: normal stage: patch review status: open title: Implement PEP 486 - Make the Python Launcher aware of virtual environments type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38139/pep486.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 15 19:21:46 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 15 Feb 2015 18:21:46 +0000 Subject: [New-bugs-announce] [issue23466] Inconsistency between str and bytes formatting of integers Message-ID: <1424024506.86.0.140677708328.issue23466@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: PEP 461 says that all numeric bytes formatting codes will work as they do for str. In particular b"%x" % val is equivalent to ("%x" % val).encode("ascii"). But this is wrong with current implementation: >>> '%x' % 3.14 Traceback (most recent call last): File "", line 1, in TypeError: %x format: an integer is required, not float >>> b'%x' % 3.14 b'3' The same is for %X, %o and %c. Raising TypeError on non-integer input to %c, %o, %x, and %X was added in issue19995. ---------- components: Interpreter Core messages: 236056 nosy: ethan.furman, serhiy.storchaka priority: normal severity: normal status: open title: Inconsistency between str and bytes formatting of integers type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 15 19:33:27 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 15 Feb 2015 18:33:27 +0000 Subject: [New-bugs-announce] [issue23467] Improve byte formatting compatibility between Py2 and Py3 Message-ID: <1424025207.37.0.294145986044.issue23467@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: The main motivation of PEP 461 was to help ease migration from, and/or have a single code base with, Python 2. But bytes formatting don't support the %r code supported in Python 2. Instead it supports the %a code which is exactly equivalent to the %r code in Python 2 but doesn't supported in Python 2. So it is not so easy to migrate from or have a single code base with Python 2 it the code uses the %r opcode. As far as bytes formatting supports the %s code (an alias to %b) purely for compatibility with Python 2, it would be good to support the %r as an alias to %a. ---------- components: Interpreter Core messages: 236057 nosy: ethan.furman, gvanrossum, serhiy.storchaka priority: normal severity: normal status: open title: Improve byte formatting compatibility between Py2 and Py3 type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 15 19:46:48 2015 From: report at bugs.python.org (Stoneagee) Date: Sun, 15 Feb 2015 18:46:48 +0000 Subject: [New-bugs-announce] [issue23468] ISO 8601 datetime process Message-ID: <1424026008.92.0.265262924683.issue23468@psf.upfronthosting.co.za> New submission from Stoneagee: Code: import dateutil.parser from datetime import datetime a='0001-01-01T00:00:00+02:00' b='1964-01-01T00:00:00+02:00' ia=dateutil.parser.parse(a) ib=dateutil.parser.parse(b) print a print ia print b print ib output: 0001-01-01T00:00:00+02:00 2001-01-01 00:00:00+02:00 1964-01-01T00:00:00+02:00 1964-01-01 00:00:00+02:00 On wiki ISO 8601, under Combined date and time representations: If a time zone designator is required, it follows the combined date and time. For example "2007-04-05T14:30Z" or "2007-04-05T12:30-02:00" I know you may have to fall back template on date such as 1/1/1, but this is ISO and according to my reading, it does not require one. BTW, my input comes from public posted information. Thanks, ---------- messages: 236059 nosy: Stoneagee priority: normal severity: normal status: open title: ISO 8601 datetime process type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 16 01:59:02 2015 From: report at bugs.python.org (Berker Peksag) Date: Mon, 16 Feb 2015 00:59:02 +0000 Subject: [New-bugs-announce] [issue23469] Delete Misc/*.wpr files Message-ID: <1424048342.99.0.00819785192711.issue23469@psf.upfronthosting.co.za> New submission from Berker Peksag: Config/plugin/readme files for Emacs and Vim in Misc have been deleted. The only remaining ones are for Wing IDE. Here is a patch to remove these files. ---------- files: wpr.diff keywords: patch messages: 236082 nosy: berker.peksag priority: normal severity: normal stage: patch review status: open title: Delete Misc/*.wpr files versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38153/wpr.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 16 09:18:36 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 16 Feb 2015 08:18:36 +0000 Subject: [New-bugs-announce] [issue23470] OpenBSD buildbot uses wrong stdlib Message-ID: <1424074716.69.0.694325234594.issue23470@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Looks as OpenBSD buildbot imports xml.sax.saxutils from wrong place. http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%202.7/builds/511/steps/test/logs/stdio ====================================================================== ERROR: test_xmlgen_encoding_bytes (test.test_sax.StringXmlgenTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/python-builds/2.7.borja-openbsd-x86/build/Lib/test/test_sax.py", line 297, in test_xmlgen_encoding_bytes gen.characters(u"\u20ac".encode(encoding)) File "/usr/local/lib/python2.7/site-packages/_xmlplus/sax/saxutils.py", line 309, in characters writetext(self._out, content) File "/usr/local/lib/python2.7/site-packages/_xmlplus/sax/saxutils.py", line 188, in writetext stream.write(escape(text, entities)) File "/home/python-builds/2.7.borja-openbsd-x86/build/Lib/codecs.py", line 357, in write data, consumed = self.encode(object, self.errors) File "/home/python-builds/2.7.borja-openbsd-x86/build/Lib/encodings/iso8859_15.py", line 12, in encode return codecs.charmap_encode(input,errors,encoding_table) UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 0: ordinal not in range(128) saxutils.py is imported from /usr/local/lib/python2.7/ instead of the build directory. The error that causes UnicodeDecodeError was fixed in issue17606. May be other test failures have the same cause. ---------- components: Tests messages: 236094 nosy: serhiy.storchaka priority: normal severity: normal status: open title: OpenBSD buildbot uses wrong stdlib type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 16 09:30:29 2015 From: report at bugs.python.org (Daniel) Date: Mon, 16 Feb 2015 08:30:29 +0000 Subject: [New-bugs-announce] [issue23471] 404 Not Found when downloading Python 3.4.3rc1 Documentation Message-ID: <1424075429.03.0.707318894375.issue23471@psf.upfronthosting.co.za> New submission from Daniel: Via Chrome on Android 4.4. ---------- assignee: docs at python components: Documentation messages: 236095 nosy: docs at python, stringsonfire priority: normal severity: normal status: open title: 404 Not Found when downloading Python 3.4.3rc1 Documentation type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 17 15:43:49 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 17 Feb 2015 14:43:49 +0000 Subject: [New-bugs-announce] [issue23472] Setup locales on buildbots Message-ID: <1424184229.01.0.47372627635.issue23472@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Tests use following locales: test_types: en_US.UTF-8 test_codecs: tr_TR test_decimal: ps_AF test_float: fr_FR or de_DE test_imaplib: de_DE or fr_FR test_locale: (en_US.UTF-8, en_US.ISO-8859-1, en_US.US-ASCII, or en_US), tr_TR test_pickle: de_DE or fr_FR test_re: en_US.ISO-8859-1, en_US.UTF-8 test_strptime: en_US.UTF-8, de_DE.UTF-8 test_time: fr_FR test_unicode: de_DE or fr_FR And test__locale uses locales from the long list. So it would be good to setup following locales on all buildbots: en_US, en_US.UTF-8, en_US.ISO-8859-1, en_US.ISO-8859-1, en_US.US-ASCII, de_DE, de_DE.UTF-8, fr_FR, tr_TR, ps_AF ---------- components: Tests messages: 236135 nosy: ezio.melotti, michael.foord, pitrou, serhiy.storchaka priority: normal severity: normal status: open title: Setup locales on buildbots type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 17 17:52:36 2015 From: report at bugs.python.org (Zack) Date: Tue, 17 Feb 2015 16:52:36 +0000 Subject: [New-bugs-announce] [issue23473] Allow namedtuple to be JSON encoded as dict Message-ID: <1424191956.27.0.511838281489.issue23473@psf.upfronthosting.co.za> New submission from Zack: We used to be able to override _iterencode prior to 2.7 to get our namedtuples to be encoded as dict using json.dump(s) but now we can not. Namedtuples are automatically encoded as list but it would be more logical and convenient to have them encoded as dict ---------- messages: 236139 nosy: Zack--, ezio.melotti, pitrou, rhettinger priority: normal severity: normal status: open title: Allow namedtuple to be JSON encoded as dict type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 17 19:34:06 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 17 Feb 2015 18:34:06 +0000 Subject: [New-bugs-announce] [issue23474] Enhance locale testing Message-ID: <1424198046.6.0.777613508402.issue23474@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch enhance locale testing. test__locale is converted to support unittest discovery. When there are no suitable locales (e.g. there is only POSIX locale) tests are reported as skipped. Tested thousands_sep with non-english locales. Tested non-ascii values of decimal_point and thousands_sep with the ps_AF locale. In test_locale Turkish locale test no longer break a line. ---------- components: Tests files: test__locale.patch keywords: patch messages: 236141 nosy: lemburg, loewis, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Enhance locale testing type: enhancement versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38159/test__locale.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 17 22:40:00 2015 From: report at bugs.python.org (STINNER Victor) Date: Tue, 17 Feb 2015 21:40:00 +0000 Subject: [New-bugs-announce] [issue23475] asyncio: reference leak in test_close_kill_running() Message-ID: <1424209200.5.0.815525755061.issue23475@psf.upfronthosting.co.za> New submission from STINNER Victor: The test_close_kill_running() test of test.test_asyncio.test_subprocess leaks references. It may be a reference cycle. ---------------------- haypo at selma$ ./python -Wall -b -m test.regrtest -R 3:3:refleaks -m test_close_kill_running test_asyncio [1/1] test_asyncio beginning 6 repetitions 123456 ...... test_asyncio leaked [388, 388, 388] references, sum=1164 test_asyncio leaked [84, 85, 85] memory blocks, sum=254 1 test failed: test_asyncio ---------------------- See also the issue #23353. Note: test_close_dont_kill_finished() doesn't leak. ---------- components: asyncio messages: 236152 nosy: gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: asyncio: reference leak in test_close_kill_running() versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 18 02:07:17 2015 From: report at bugs.python.org (John Nagle) Date: Wed, 18 Feb 2015 01:07:17 +0000 Subject: [New-bugs-announce] [issue23476] SSL cert verify fail for "www.verisign.com" Message-ID: <1424221637.85.0.898475945708.issue23476@psf.upfronthosting.co.za> New submission from John Nagle: SSL certificate verification fails for "www.verisign.com" when using the cert list from Firefox. Other sites ("google.com", "python.org") verify fine. This may be related to a known, and fixed, OpenSSL bug. See: http://rt.openssl.org/Ticket/Display.html?id=2732&user=guest&pass=guest https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1014640 Some versions of OpenSSL are known to be broken for cases where there multiple valid certificate trees. This happens when one root cert is being phased out in favor of another, and cross-signing is involved. Python ships with its own copy of OpenSSL on Windows. Tests for "www.verisign.com" Win7, x64: Python 2.7.9 with OpenSSL 1.0.1j 15 Oct 2014. FAIL Python 3.4.2 with OpenSSL 1.0.1i 6 Aug 2014. FAIL openssl s_client -OpenSSL 1.0.1h 5 Jun 2014 FAIL Ubuntu 14.04 LTS, x64, using distro's versions of Python: Python 2.7.6 - test won't run, needs create_default_context Python 3.4.0 with OpenSSL 1.0.1f 6 Jan 2014. FAIL openssl s_client OpenSSL 1.0.1f 6 Jan 2014 PASS That's with the same cert file in all cases. The OpenSSL version for Python programs comes from ssl.OPENSSL_VERSION. The Linux situation has me puzzled. On Linux, Python is supposedly using the system version of OpenSSL. The versions match. Why do Python and the OpenSSL command line client disagree? Different options passed to OpenSSL by Python? A simple test program and cert file are attached. Please try this in your environment. ---------- components: Library (Lib) files: ssltest.py messages: 236158 nosy: nagle priority: normal severity: normal status: open title: SSL cert verify fail for "www.verisign.com" versions: Python 2.7, Python 3.4 Added file: http://bugs.python.org/file38165/ssltest.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 18 15:57:17 2015 From: report at bugs.python.org (Alex Shkop) Date: Wed, 18 Feb 2015 14:57:17 +0000 Subject: [New-bugs-announce] [issue23477] Increase coverage for wsgiref module Message-ID: <1424271437.72.0.0459822564302.issue23477@psf.upfronthosting.co.za> New submission from Alex Shkop: Added test for wssgiref.simple_server to check that environ argument contains correct headers, query string and path. This code wasn't covered in tests previously. ---------- components: Tests files: wsgiref_test_environ.patch keywords: patch messages: 236176 nosy: ashkop priority: normal severity: normal status: open title: Increase coverage for wsgiref module versions: Python 3.5 Added file: http://bugs.python.org/file38169/wsgiref_test_environ.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 18 18:50:09 2015 From: report at bugs.python.org (P Yap) Date: Wed, 18 Feb 2015 17:50:09 +0000 Subject: [New-bugs-announce] [issue23478] A list arg with default value inside function got appended each time the function is called Message-ID: <1424281809.58.0.308788589404.issue23478@psf.upfronthosting.co.za> New submission from P Yap: I have a function (test) with a list variable APP and declared its default as an empty list [], while APP is not a global variable, if I execute the same function multiple times, each time the APP will get appended. To workaround this problem, I need to do APP.pop() inside the function or explicitly called the function with an argument (test([])). del APP or reassign APP=[] inside the function does not resolve the problem same thing happens if the function is defined as an method inside a class. Here is a little test program for testing: def test(APP=[]): if len(APP) == 0: APP.append('1abc') print "APP=", APP APP.append('2def') class test1 (object): def t1(self, abc=[]): abc.append('abc') print abc if __name__ == '__main__': print "class test" t = test1() i = 0 while i < 3: t.t1() i += 1 print "Test function::" i = 0 while i < 3: test() i += 1 Here are the output: class test ['abc'] ['abc', 'abc'] ['abc', 'abc', 'abc'] Test function:: APP= ['1abc'] APP= ['1abc', '2def'] APP= ['1abc', '2def', '2def'] ---------- components: Interpreter Core messages: 236185 nosy: yappyta priority: normal severity: normal status: open title: A list arg with default value inside function got appended each time the function is called type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 18 22:56:15 2015 From: report at bugs.python.org (Mahmoud Hashemi) Date: Wed, 18 Feb 2015 21:56:15 +0000 Subject: [New-bugs-announce] [issue23479] str.format() breaks object duck typing Message-ID: <1424296575.91.0.194467251728.issue23479@psf.upfronthosting.co.za> New submission from Mahmoud Hashemi: While porting some old code, I found some interesting misbehavior in the new-style string formatting. When formatting objects which support int and float conversion, old-style percent formatting works great, but new-style formatting explodes hard. Here's a basic example: class MyType(object): def __init__(self, func): self.func = func def __float__(self): return float(self.func()) print '%f' % MyType(lambda: 3) # Output (python2 and python3): 3.000000 print '{:f}'.format(MyType(lambda: 3)) # Output (python2): # Traceback (most recent call last): # File "tmp.py", line 28, in # print '{:f}'.format(MyType(lambda: 3)) # ValueError: Unknown format code 'f' for object of type 'str' # # Output (python3.4): # Traceback (most recent call last): # File "tmp.py", line 30, in # print('{:f}'.format(MyType(lambda: 3))) # TypeError: non-empty format string passed to object.__format__ And the same holds true for int and so forth. I would expect these behaviors to be the same between the two formatting styles, and tangentially, expect a more python2-like error message for the python 3 case. ---------- messages: 236192 nosy: mahmoud priority: normal severity: normal status: open title: str.format() breaks object duck typing type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 18 23:23:57 2015 From: report at bugs.python.org (Zorigt Bazarragchaa) Date: Wed, 18 Feb 2015 22:23:57 +0000 Subject: [New-bugs-announce] [issue23480] Minor typo Message-ID: <1424298237.21.0.795168205464.issue23480@psf.upfronthosting.co.za> New submission from Zorigt Bazarragchaa: I found a syntax type in your demo example on https://docs.python.org/2/library/json.html >>> print json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True) {"a": 0, "b": 0, "c": 0} The correct return should be a 'str' type not json. Therefore: >>> print json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True) '{"a": 0, "b": 0, "c": 0}' ---------- components: Demos and Tools messages: 236195 nosy: zorigt priority: normal severity: normal status: open title: Minor typo _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 19 01:54:56 2015 From: report at bugs.python.org (Alex Gaynor) Date: Thu, 19 Feb 2015 00:54:56 +0000 Subject: [New-bugs-announce] [issue23481] SSL module should not offer RC4 based cipher suites for clients by default Message-ID: <1424307296.49.0.474882830169.issue23481@psf.upfronthosting.co.za> New submission from Alex Gaynor: In addition to the security concerns, it is now a violation of RFC7465 to offer a cipher suite with RC4 in a ClientHello: https://tools.ietf.org/html/rfc7465 ---------- components: Library (Lib) files: rc4.diff keywords: patch messages: 236202 nosy: alex, christian.heimes, dstufft, giampaolo.rodola, janssen, pitrou priority: normal severity: normal status: open title: SSL module should not offer RC4 based cipher suites for clients by default versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38176/rc4.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 19 02:10:19 2015 From: report at bugs.python.org (Steve Dower) Date: Thu, 19 Feb 2015 01:10:19 +0000 Subject: [New-bugs-announce] [issue23482] sqlite3_d.dll is not included in installer Message-ID: <1424308219.39.0.206852062326.issue23482@psf.upfronthosting.co.za> New submission from Steve Dower: This file is necessary for _sqlite3 to be importable with the debug binaries. ---------- assignee: steve.dower components: Installation, Windows messages: 236204 nosy: steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: sqlite3_d.dll is not included in installer type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 19 09:53:54 2015 From: report at bugs.python.org (Scholes C) Date: Thu, 19 Feb 2015 08:53:54 +0000 Subject: [New-bugs-announce] [issue23483] python 2.7 builds using icc Message-ID: <1424336034.12.0.170693655501.issue23483@psf.upfronthosting.co.za> New submission from Scholes C: HI, can you please look into this ? thanks. icc builds of python 2.7 seem to have issues handling nan, inf, etc $ /usr/local/python-2.7.6/bin/python Python 2.7.6 (default, Jan 10 2014, 12:14:02) [GCC Intel(R) C++ gcc 4.1 mode] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print float('nan') 0.0 $ /usr/local/python-2.6.6-64bit/bin/python Python 2.6.6 (r266:84292, Oct 14 2010, 15:47:19) [GCC Intel(R) C++ gcc 4.1 mode] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print float('nan') nan I tried both ?fp-model strict and ?fp-model precise compiler options as suggested by http://bugs.python.org/issue21167, but neither seems to resolve other situations like the one with atan2 below: $ LD_LIBRARY_PATH=/dat/sharefolder_scratch/python-build ./python Python 2.7.9 (default, Feb 18 2015, 19:58:37) [GCC Intel(R) C++ gcc 4.1 mode] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print float('nan') nan >>> import math >>> print math.atan2(0, float('nan')) 0.0 $ /usr/local/python-2.6.6-64bit/bin/python Python 2.6.6 (r266:84292, Oct 14 2010, 15:47:19) [GCC Intel(R) C++ gcc 4.1 mode] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import math >>> print math.atan2(0, float('nan')) nan ---------- components: Interpreter Core messages: 236213 nosy: Scholes.C priority: normal severity: normal status: open title: python 2.7 builds using icc type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 19 12:31:14 2015 From: report at bugs.python.org (Teodor Dima) Date: Thu, 19 Feb 2015 11:31:14 +0000 Subject: [New-bugs-announce] [issue23484] SemLock acquire() keyword arg 'blocking' is invalid Message-ID: <1424345474.95.0.767809945273.issue23484@psf.upfronthosting.co.za> New submission from Teodor Dima: The keyword for 'blocking' in the documentation for multiprocessing.Lock.acquire() (and all synchronization objects dependent on SemLock) differs from its implementation at Modules/_multiprocessing/semaphore.c:70 - https://docs.python.org/3.4/library/threading.html#threading.Lock.acquire ---------- components: Extension Modules messages: 236215 nosy: td priority: normal severity: normal status: open title: SemLock acquire() keyword arg 'blocking' is invalid type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 19 14:32:55 2015 From: report at bugs.python.org (STINNER Victor) Date: Thu, 19 Feb 2015 13:32:55 +0000 Subject: [New-bugs-announce] [issue23485] PEP 475: handle EINTR in the select and selectors module Message-ID: <1424352775.64.0.420442648443.issue23485@psf.upfronthosting.co.za> New submission from STINNER Victor: The implementation of the PEP 475 has to modify the following functions to restart on EINTR (and recompute the timeout): * select.select() * select.poll() * select.epoll.poll() * select.devpoll.poll() * select.kqueue.control() * selectors.SelectSelector.select() and other selector classes See also issues #18885 and #23285. ---------- components: Extension Modules messages: 236216 nosy: haypo, neologix priority: normal severity: normal status: open title: PEP 475: handle EINTR in the select and selectors module versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 19 22:59:13 2015 From: report at bugs.python.org (Craig Holmquist) Date: Thu, 19 Feb 2015 21:59:13 +0000 Subject: [New-bugs-announce] [issue23486] Enum comparisons are 20x slower than comparing equivalent ints Message-ID: <1424383153.37.0.19682507186.issue23486@psf.upfronthosting.co.za> New submission from Craig Holmquist: Running the attached test script: $ time python test.py enum real 0m6.546s user 0m6.530s sys 0m0.007s $ time python test.py int real 0m0.384s user 0m0.377s sys 0m0.000s I encountered this with a script that yielded a sequence of objects (potentially a few hundred thousand of them) and categorized them with instances of an Enum subclass. The consumer of that iteration processes each object with a switch-case-like comparison of the category, checking it sequentially against each instance of the Enum. This seems like a fairly common use case. >From cProfile it looks like EnumMeta.__getattr__ and _is_dunder are the main bottlenecks: [...] 7/1 0.000 0.000 0.000 0.000 abc.py:194(__subclasscheck__) 1 0.000 0.000 0.001 0.001 enum.py:1() 3 0.000 0.000 0.000 0.000 enum.py:132() 2000021 0.988 0.000 0.988 0.000 enum.py:16(_is_dunder) 19 0.000 0.000 0.000 0.000 enum.py:24(_is_sunder) 2000002 1.825 0.000 2.813 0.000 enum.py:241(__getattr__) 17 0.000 0.000 0.000 0.000 enum.py:282(__setattr__) 3 0.000 0.000 0.000 0.000 enum.py:342(_get_mixins_) 3 0.000 0.000 0.000 0.000 enum.py:387(_find_new_) [...] ---------- components: Library (Lib) files: test.py messages: 236234 nosy: craigh priority: normal severity: normal status: open title: Enum comparisons are 20x slower than comparing equivalent ints type: performance versions: Python 3.4 Added file: http://bugs.python.org/file38177/test.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 08:24:03 2015 From: report at bugs.python.org (BJ Dierkes) Date: Fri, 20 Feb 2015 07:24:03 +0000 Subject: [New-bugs-announce] [issue23487] argparse: add_subparsers 'action' broken Message-ID: <1424417043.22.0.71313795028.issue23487@psf.upfronthosting.co.za> New submission from BJ Dierkes: Related: http://bugs.python.org/issue9253 I came across issue9253 in trying to implement a default action for a subparser namespace. In the absence of a 'default' option, I thought that it may be possible by adding an 'action' to 'add_subparsers'. Per the documentation this should be possible: https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_subparsers [QUOTE] action - the basic type of action to be taken when this argument is encountered at the command line [/QUOTE] That said, custom actions on 'add_subparsers' doesn't appear to work at all: import argparse class CustomAction(argparse.Action): def __call__(self, parser, namespace, values, option_string=None): print('Inside CustomAction') setattr(namespace, self.dest, values) root_parser = argparse.ArgumentParser(prog='myapp') sub_parser = root_parser.add_subparsers(dest='commands', action=CustomAction) args = root_parser.parse_args() Produces: $ python argtest.py --help Traceback (most recent call last): File "argtest.py", line 46, in sub_parser = root_parser.add_subparsers(dest='commands', action=CustomAction) File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1661, in add_subparsers action = parsers_class(option_strings=[], **kwargs) TypeError: __init__() got an unexpected keyword argument 'prog' Erroneous documentation maybe? Tested the same on Python 2.7 and 3.3. ---------- components: Library (Lib) messages: 236254 nosy: derks priority: normal severity: normal status: open title: argparse: add_subparsers 'action' broken type: behavior versions: Python 2.7, Python 3.3 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 11:18:42 2015 From: report at bugs.python.org (Raymond Hettinger) Date: Fri, 20 Feb 2015 10:18:42 +0000 Subject: [New-bugs-announce] [issue23488] Random objects twice as big as necessary on 64-bit builds Message-ID: <1424427522.46.0.066480034722.issue23488@psf.upfronthosting.co.za> New submission from Raymond Hettinger: The Modules/_randommodule.c implements the 32-bit version of the MersenneTwister and its struct uses (unsigned long) for each of the 624 elements of the state vector. On a 32-bit build, the unsigned longs are 4 bytes. However, on a 64-bit build, they are 8 bytes each eventhough only the bottom 32-bits are used. This causes the random object to be twice as big as necessary. sys.getsizeof(_random.Random()) reports 5016 bytes. This wastes memory, grinds the cache, and slows performance. The (unsigned long) declaration should probably be replaced with (uint32_t). ---------- assignee: rhettinger messages: 236262 nosy: rhettinger, serhiy.storchaka priority: normal severity: normal status: open title: Random objects twice as big as necessary on 64-bit builds type: resource usage versions: Python 2.7, Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 12:48:28 2015 From: report at bugs.python.org (juj) Date: Fri, 20 Feb 2015 11:48:28 +0000 Subject: [New-bugs-announce] [issue23489] atexit handlers are not executed when using multiprocessing.Pool.map. Message-ID: <1424432908.71.0.0710990700334.issue23489@psf.upfronthosting.co.za> New submission from juj: When Multiprocessing.Pool.map is used for a script that registers atexit handlers, the atexit handlers are not executed when the pool threads quit. STR: 1. Run attached file in Python 2.7 with 'python task_spawn.py' 2. Observe the printed output. Observed: Console prints: CREATED TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_qef8r_ CREATED TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_axi9tt CREATED TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_vx6fmu task1 task2 ATEXIT: REMOVING TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_qef8r_ Expected: Console should print: CREATED TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_qef8r_ CREATED TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_axi9tt CREATED TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_vx6fmu task1 task2 ATEXIT: REMOVING TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_vx6fmu ATEXIT: REMOVING TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_axi9tt ATEXIT: REMOVING TEMP DIRECTORY c:\users\clb\appdata\local\temp\temp_qef8r_ ---------- components: Library (Lib) files: task_spawn.py messages: 236273 nosy: juj priority: normal severity: normal status: open title: atexit handlers are not executed when using multiprocessing.Pool.map. type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file38185/task_spawn.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 13:06:49 2015 From: report at bugs.python.org (paul) Date: Fri, 20 Feb 2015 12:06:49 +0000 Subject: [New-bugs-announce] [issue23490] allocation (and overwrite) of a 0 byte buffer Message-ID: <1424434009.7.0.146767591235.issue23490@psf.upfronthosting.co.za> New submission from paul: # Bug # --- # # Py_UNICODE * # PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size) # { # ... # #endif # wchar_t *w; # wchar_t *wchar_end; # # ... # 1 _PyUnicode_WSTR(unicode) = (wchar_t *) PyObject_MALLOC(sizeof(wchar_t) * # (_PyUnicode_LENGTH(unicode) + 1)); # ... # w = _PyUnicode_WSTR(unicode); # 2 wchar_end = w + _PyUnicode_LENGTH(unicode); # # if (PyUnicode_KIND(unicode) == PyUnicode_1BYTE_KIND) { # one_byte = PyUnicode_1BYTE_DATA(unicode); # 3 for (; w < wchar_end; ++one_byte, ++w) # *w = *one_byte; # /* null-terminate the wstr */ # 4 *w = 0; # } # # 1. if length(unicode)==2**30-1, then malloced buffer has size equal to # 4*(2^30-1+1)=2^32 == 0 (modulo 2^32) # 2. wchar_end is equal to w-4 because of pointer arithmetic (nonexplicit # multiplication by 4) # 3. w > wchar_end, so we don't enter the loop # 4. 4 byte write to a 0 size buffer # # GDB output # ---------- # # 3860 _PyUnicode_WSTR(unicode) = (wchar_t *) PyObject_MALLOC(sizeof(wchar_t) * # ... # (gdb) print sizeof(wchar_t)*(((PyASCIIObject*)(unicode))->length+1) # $21 = 0 # ... # (gdb) n # 3868 w = _PyUnicode_WSTR(unicode); # (gdb) n # 3869 wchar_end = w + _PyUnicode_LENGTH(unicode); # (gdb) n # 3871 if (PyUnicode_KIND(unicode) == PyUnicode_1BYTE_KIND) { # (gdb) print w # $22 = 0x805fc028 L"\xfbfbfbfb\xced00000" # (gdb) print wchar_end # $23 = 0x805fc024 L"\xfbfbfb6f\xfbfbfbfb\xced00000" # ... # 3876 *w = 0; # # ) # OS info # ------- # # % ./python -V # Python 3.4.1 # # % uname -a # Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 15:31:16 UTC 2013 i686 i686 i386 GNU/Linux import locale s='a'*(2**30-1) locale.strxfrm(s) ---------- files: poc_strxfrm.py messages: 236275 nosy: pkt priority: normal severity: normal status: open title: allocation (and overwrite) of a 0 byte buffer type: crash versions: Python 3.4 Added file: http://bugs.python.org/file38186/poc_strxfrm.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 13:52:51 2015 From: report at bugs.python.org (Paul Moore) Date: Fri, 20 Feb 2015 12:52:51 +0000 Subject: [New-bugs-announce] [issue23491] PEP 441 - Improving Python Zip Application Support Message-ID: <1424436771.29.0.353849461373.issue23491@psf.upfronthosting.co.za> New submission from Paul Moore: This is the patch for PEP 441 (Zip Application Support). Steve, could you check the installer changes, please? I haven't managed to get a setup where I can test the installer, and I'm not aware of any WiX coding tools, so I just edited the XML files by hand with grep and vim, and checked that Tools/buildmsi.bat doesn't error in anything I wrote... (There's a CRC check error in TCL, but that was there before I made changes) light.exe : error LGHT0216: An unexpected Win32 exception with error code 0x17 occurred while accessing file 'C:\Work\Projects\cpython\externals\tcltk64\lib\tcl8.6\tzdata\America\Detroit': Data error (cyclic redundancy check) [C:\Work\Projects\cpython\Tools\msi\tcltk\tcltk.wixproj] Also, I don't *think* I need to do anything for the new files in Doc\Lib, Lib and Lib\test to get picked up, but I'm not sure how to check that. ---------- assignee: steve.dower components: Library (Lib) files: pep441.patch keywords: patch messages: 236276 nosy: pmoore, steve.dower priority: normal severity: normal status: open title: PEP 441 - Improving Python Zip Application Support type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38187/pep441.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 15:40:11 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 20 Feb 2015 14:40:11 +0000 Subject: [New-bugs-announce] [issue23492] Argument Clinic: improve generated parser for 1-argument functions Message-ID: <1424443211.95.0.366719358355.issue23492@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch improve generated parsers for functions with single positional argument. Now they always generated as METH_O and PyArg_Parse() is used to parse single argument. To avoid code churn in this and following changes it would be worth to extract all generated code in separated files. ---------- components: Build files: clinic_meth_o.patch keywords: patch messages: 236288 nosy: larry, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Argument Clinic: improve generated parser for 1-argument functions type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38188/clinic_meth_o.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 17:25:56 2015 From: report at bugs.python.org (Wouter Bolsterlee) Date: Fri, 20 Feb 2015 16:25:56 +0000 Subject: [New-bugs-announce] [issue23493] optimize sort_keys in json module by using operator.itemgetter() Message-ID: <1424449556.4.0.766318893285.issue23493@psf.upfronthosting.co.za> New submission from Wouter Bolsterlee: The JSON encoder uses a lambda function for the sort(key=...) invocation used to sort the keys in a JSON object in case sort_keys=True is passed: https://hg.python.org/cpython/file/46bfddb14cbe/Lib/json/encoder.py#l352 Instead of having a lambda, operator.itemgetter(0) can be used here, which is a minor performance gain. It should be noted that many other internals of the JSON encoder have also been fine-tuned (manually) for performance reasons. ---------- components: Library (Lib) messages: 236302 nosy: wbolster priority: normal severity: normal status: open title: optimize sort_keys in json module by using operator.itemgetter() versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 20:46:14 2015 From: report at bugs.python.org (Gil Shotan) Date: Fri, 20 Feb 2015 19:46:14 +0000 Subject: [New-bugs-announce] [issue23494] adding timedelta to datetime object is not timezone aware Message-ID: <1424461574.53.0.989762969133.issue23494@psf.upfronthosting.co.za> New submission from Gil Shotan: I encountered a strange bug involving timezone aware datetime objects and timedelta objects. The crux of the matter is that daylight savings time is considered a different timezone, and therefore the timezone of a datetime object is date dependent. However, adding a timedelta object to a datetime object seems to do the simple thing which is preserve the timezone object. The bug manifests itself whenever adding a timedelta object crosses a daylight savings time boundary. The following code illustrates the problem. Note that the transition between PDT (Pacific daylight time) and PST (Pacific savings time) occurs on March 9th (ish) >>> from datetime import datetime, timedelta >>> import pytz >>> tz = pytz.timezone('America/Los_Angeles') >>> before = tz.localize(datetime(year=2015, month=3, day=8)) >>> before datetime.datetime(2015, 3, 8, 0, 0, tzinfo=) >>> # notice PST timezone >>> after_right = tz.localize(datetime(year=2015, month=3, day=10)) >>> after_right datetime.datetime(2015, 3, 10, 0, 0, tzinfo=) >>> # notice PDT timezone >>> after_wrong = before + timedelta(days=2) >>> after_wrong datetime.datetime(2015, 3, 10, 0, 0, tzinfo=) >>> # when calculated this way, the timezone remains at PST ---------- components: Distutils messages: 236326 nosy: dstufft, eric.araujo, gilsho priority: normal severity: normal status: open title: adding timedelta to datetime object is not timezone aware type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 20 21:45:53 2015 From: report at bugs.python.org (Steven Barker) Date: Fri, 20 Feb 2015 20:45:53 +0000 Subject: [New-bugs-announce] [issue23495] The writer.writerows method should be documented as accepting any iterable (not only a list) Message-ID: <1424465153.78.0.58816335685.issue23495@psf.upfronthosting.co.za> New submission from Steven Barker: The documentation for the csv.writer.writerows method says that it expects "a list of row objects", when it really will accept any iterable that yields rows (such as a generator). While it's often nice for code to be more accepting than the documented requirements, I think the docs in this case should state that writerows() expects an iterable, rather than misinforming users that a list is required. This documentation issue was brought up in a Stack Overflow question here: http://stackoverflow.com/questions/28636848/csv-writer-writerows-takes-iterator I expect the necessary documentation patch will be pretty trivial, and if nobody else gets to it first, I will try to provide one when I have enough time to update my cpython checkout (not soon, alas). ---------- assignee: docs at python components: Documentation messages: 236328 nosy: Steven.Barker, docs at python priority: normal severity: normal status: open title: The writer.writerows method should be documented as accepting any iterable (not only a list) type: enhancement versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 21 21:46:57 2015 From: report at bugs.python.org (Cyd Haselton) Date: Sat, 21 Feb 2015 20:46:57 +0000 Subject: [New-bugs-announce] [issue23496] Steps for Android Native Build of Python 3.4.2 Message-ID: <1424551617.0.0.291471450783.issue23496@psf.upfronthosting.co.za> New submission from Cyd Haselton: This is a (hopefully) complete list of patches and modifications required to natively build Python 3.4.2 on an Android device where patches == listed issues in this bug tracker modifications (mods) == unsubmitted patches and/or edits made to source CAVEATS Build was completed on a tablet running Android 4.4.2 and within the KBOX environment; an app that simulates a Linux filesystem by using a ported version of fakechroot. Details of the environment can be found here: http://kevinboone.net/kbox2_how_it_works.html. NOTE: During the Python build it was necessary to patch this environment with a modified version of fakechroot; the KBOX version currently available as of 2.21.2015 may not contain the modified fakechroot. CONFIGURE && POST-CONFIGURE && MAKE The configure command was run with the following parameters --enable-shared --without-ensurepip --prefix=/usr/python After running configure. changes were made to pyconfig.h to account for various platform limitations and to Setup to ensure that a) all modules specified were built as shared and b) modules that used symbols from libpython were explicitly linked to said library. See attached pyconfig.h and Setup. Make fails when the newly built python binary tries to execute and run setup.py. This is resolved by copying the shared library from the build directory to /lib in the KBOX environment. Otherwise make && make install runs without errors provided patches/mods are applied. I did not run make test or the equivalent for this build. PATCHES/MODS The attached patches from the following issues must be applied: http://bugs.python.org/issue20305 http://bugs.python.org/issue16353 (NOTE: use patch with 'defpath' not 'confstr', which is not supported in Android) http://bugs.python.org/issue21668 The following modifications to source must be made: The instructions under 'Code Changes' at this link: https://code.google.com/p/python-for-android/wiki/CrossCompilingPython The attached patch, contributed by Ryan Gonzales (patch.diff) The changes listed below by file (apologies for formatting) /* patch for strdup segfault issue in readline.c */ #define RESTORE_LOCALE(sl) { if (sl) { setlocale(LC_CTYPE, sl); free(sl); } } char *_locale = setlocale(LC_CTYPE, NULL); char *saved_locale; if(!_locale) _locale = ""; saved_locale = strdup(_locale); char *saved_locale = setlocale(LC_CTYPE, NULL); if (saved_locale != NULL) { saved_locale = strdup(saved_locale); if (!saved_locale) Py_FatalError("not enough memory to save locale"); } /* fix for Lib/ctypes/__init.py__ */ 30 DEFAULT_MODE == RTLD_GLOBAL /* patch for python.c */ /* here line for oldloc is commented out */ 24 int i, res; 25 /* char *oldloc; */ 26 #ifdef __FreeBSD__ /* here oldloc lines commented out */ 46 47 /* oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL)); 48 * if (!oldloc) { 49 * fprintf(stderr, "out of memory\n"); 50 * return 1; 51 * 52 * } 53 */ /* patch for pythonrun.c 307, --309-310, ++311-313 --1009-1012 ++1013-1014 */ 305 return get_codec_name(codeset); 306 #else 307 char* m; 308 PyErr_SetNone(PyExc_NotImplementedError); 309 /* return NULL; */ 310 /* char* m; */ 311 m = malloc(6); 312 strcpy(m, "ascii"); 313 return m; 314 #endif 315 } 1002 static int 1003 initfsencoding(PyInterpreterState *interp) 1004 { 1005 PyObject *codec; 1006 1007 if (Py_FileSystemDefaultEncoding == NULL) 1008 { 1009 /* Py_FileSystemDefaultEncoding = get_locale_encoding(); 1010 * if (Py_FileSystemDefaultEncoding == NULL) 1011 * Py_FatalError("Py_Initialize: Unable to get the locale encoding 1011 "); 1012 */ 1013 Py_FileSystemDefaultEncoding = malloc(6); 1014 strcpy(Py_FileSystemDefaultEncoding, "ascii"); 1015 Py_HasFileSystemDefaultEncoding = 0; 1016 interp->fscodec_initialized = 1; 1017 return 0; 1018 } 1019 /* changes to Lib/plat-linux/DLFCN.py */ 76 # Included from bits/dlfcn.h 77 # Changed for Android 78 RTLD_LAZY = 1 79 RTLD_NOW = 0 80 #RTLD_BINDING_MASK = 0x3 81 #RTLD_NOLOAD = 0x00004 82 RTLD_GLOBAL = 2 83 RTLD_LOCAL = 0 84 #RTLD_NODELETE = 0x01000 85 86 # Also from Android's dlfcn.h 87 RTLD_DEFAULT ((void*) 0xffffffff) 88 RTLD_NEXT ((void*) 0xfffffffe) 89 ---------- components: Build files: patch.diff keywords: patch messages: 236389 nosy: chaselton priority: normal severity: normal status: open title: Steps for Android Native Build of Python 3.4.2 versions: Python 3.4 Added file: http://bugs.python.org/file38201/patch.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 22 01:57:55 2015 From: report at bugs.python.org (Myles Dear) Date: Sun, 22 Feb 2015 00:57:55 +0000 Subject: [New-bugs-announce] [issue23497] textwrap.dedent doesn't work properly with embedded strings containing linefeeds Message-ID: <1424566675.14.0.0615952821634.issue23497@psf.upfronthosting.co.za> New submission from Myles Dear: The textwrap.dedent function does not work when the string to dedent itself contains embedded strings that contain newline characters. https://docs.python.org/3.4/library/textwrap.html?highlight=dedent#textwrap.dedent states that this function "can be used to make triple-quoted strings line up with the left edge of the display". It just so happens that my triple-quoted string itself contains a single-quoted string with newline characters. I would have expected textwrap.dedent to ignore these newline characters inside single or double quoted strings contained in the larger triple-quoted string. The semantics of this bug may be slightly different than https://bugs.python.org/issue19479 "textwrap.dedent doesn't work properly with strings containing CRLF", so I'm raising a new issue. I am seeing this in a module I'm writing that emits synthetic python code that is subsequently passed back into the interpreter via "exec". Python 3.4.1 (default, Nov 12 2014, 13:34:29) [GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from textwrap import dedent >>> s = ''' ... 'my_key' : r'my value which contains \n character' ... ''' >>> s "\n 'my_key' : r'my value which contains \n character'\n" >>> dedent(s) "\n 'my_key' : r'my value which contains \ncharacter'\n" >>> ---------- messages: 236396 nosy: mdear priority: normal severity: normal status: open title: textwrap.dedent doesn't work properly with embedded strings containing linefeeds _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 22 01:58:10 2015 From: report at bugs.python.org (Martin Panter) Date: Sun, 22 Feb 2015 00:58:10 +0000 Subject: [New-bugs-announce] [issue23498] Expose http.cookiejar.split_header_words() Message-ID: <1424566690.92.0.444204836552.issue23498@psf.upfronthosting.co.za> New submission from Martin Panter: I propose to document the split_header_words() so that it can be used to parse various kinds of HTTP-based header fields. Perhaps it should live in a more general module like ?http?, or ?email.policy.HTTP? (hinted in Issue 3609). Perhaps there is also room for finding a better name, such as parse_header_attributes() or something, since splitting space-separated words is not its most important property. The function takes a series of header field values, as returned from Message.get_all(failobj=()). The field values may be separate strings and may also be comma-separated. It parses space- or semicolon-separated name=value attributes from each field value. Examples: RFC 2965 Set-Cookie2 fields: >>> cookies = ( ... 'Cookie1="VALUE";Version=1;Discard, Cookie2="Same field";Version=1', ... 'Cookie3="Separate header field";Version=1', ... ) >>> pprint(http.cookiejar.split_header_words(cookies)) [[('Cookie1', 'VALUE'), ('Version', '1'), ('Discard', None)], [('Cookie2', 'Same field'), ('Version', '1')], [('Cookie3', 'Separate header field'), ('Version', '1')]] RTSP 1.0 (RFC 2326) Transport header field: >>> transport = 'RTP/AVP;unicast;mode="PLAY, RECORD", RTP/AVP/TCP;interleaved=0-1' >>> pprint(http.cookiejar.split_header_words((transport,))) [[('RTP/AVP', None), ('unicast', None), ('mode', 'PLAY, RECORD')], [('RTP/AVP/TCP', None), ('interleaved', '0-1')]] The parsing of spaces seems to be an attempt to parse headers like WWW-Authenticate, although it mixes up the parameters when given this example from RFC 7235: >>> auth = 'Newauth realm="apps", type=1, title="Login to \\"apps\\"", Basic realm="simple"' >>> pprint(http.cookiejar.split_header_words((auth,))) [[('Newauth', None), ('realm', 'apps')], [('type', '1')], [('title', 'Login to "apps"')], [('Basic', None), ('realm', 'simple')]] Despite that, the function is still very useful for parsing many kinds of header fields that use semicolons. All the alternatives in the standard library that I know of have disadvantages: * cgi.parse_header() does not split comma-separated values apart, and ignores any attribute without an equals sign, such as ?Discard? and ?unicast? above * email.message.Message.get_params() and get_param() do not split comma-separated values either, and parsing header values other than the first one in a Message object is awkward * email.headerregistry.ParameterizedMIMEHeader looks relevant, but I couldn?t figure out how to use it ---------- components: Library (Lib) messages: 236397 nosy: vadmium priority: normal severity: normal status: open title: Expose http.cookiejar.split_header_words() type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 22 11:03:27 2015 From: report at bugs.python.org (SilentGhost) Date: Sun, 22 Feb 2015 10:03:27 +0000 Subject: [New-bugs-announce] [issue23499] Minor grammar issue in mimetypes.rst Message-ID: <1424599407.74.0.534363562696.issue23499@psf.upfronthosting.co.za> New submission from SilentGhost: Just a minor grammar issue: missing "be" in mimetypes.rst ---------- assignee: docs at python components: Documentation files: mimetypes.diff keywords: patch messages: 236402 nosy: SilentGhost, docs at python, eric.araujo, ezio.melotti, georg.brandl priority: normal severity: normal stage: patch review status: open title: Minor grammar issue in mimetypes.rst versions: Python 3.5 Added file: http://bugs.python.org/file38202/mimetypes.diff _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 22 11:37:42 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 22 Feb 2015 10:37:42 +0000 Subject: [New-bugs-announce] [issue23500] Argument Clinic: multiple macro definition Message-ID: <1424601462.93.0.790792182733.issue23500@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Argument Clinic generates multiple definitions in non-block mode for functions with alternating signature. See example file. If FOO is not defines, SPAM_METHODDEF is defined twice. First time in: #ifndef SPAM_METHODDEF #define SPAM_METHODDEF #endif /* !defined(SPAM_METHODDEF) */ Second time in: #if !defined(FOO) ... #define SPAM_METHODDEF \ ---------- components: Build, Demos and Tools messages: 236403 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Argument Clinic: multiple macro definition type: behavior versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sun Feb 22 12:04:48 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Sun, 22 Feb 2015 11:04:48 +0000 Subject: [New-bugs-announce] [issue23501] Argument Clinic: generate code into separate files by default Message-ID: <1424603088.54.0.245767727537.issue23501@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Proposed patch makes Argument Clinic to generate code into separate files by default. This will help maintaining in the case of enhancements of Argument Clinic generation (such as issue23492). See thread on http://comments.gmane.org/gmane.comp.python.devel/151585 . There is an issue with multiple macro definition in the posix module (see issue23500). ---------- components: Build, Demos and Tools files: clinic_file.patch keywords: patch messages: 236406 nosy: larry, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Argument Clinic: generate code into separate files by default type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38204/clinic_file.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 23 20:21:48 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 23 Feb 2015 19:21:48 +0000 Subject: [New-bugs-announce] [issue23502] pprint: added support for mapping proxy Message-ID: <1424719308.61.0.00451137981633.issue23502@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Mapping proxy is quite often used in building . For example types __dict__ are mapping proxy. >>> bool.__dict__ mappingproxy({'__or__': , '__doc__': 'bool(x) -> bool\n\nReturns True when the argument x is true, False otherwise.\nThe builtins True and False are the only two instances of the class bool.\nThe class bool is a subclass of the class int, and cannot be subclassed.', '__xor__': , '__and__': , '__rxor__': , '__repr__': , '__new__': , '__rand__': , '__str__': , '__ror__': }) Mapping proxy is not a subclass of dict, so pprint doesn't print it pretty. Proposed patch adds support for mapping proxies in pprint. >>> from pprint import pprint as pp >>> pp(bool.__dict__) mappingproxy({'__and__': , '__doc__': 'bool(x) -> bool\n' '\n' 'Returns True when the argument x is true, False ' 'otherwise.\n' 'The builtins True and False are the only two ' 'instances of the class bool.\n' 'The class bool is a subclass of the class int, and ' 'cannot be subclassed.', '__new__': , '__or__': , '__rand__': , '__repr__': , '__ror__': , '__rxor__': , '__str__': , '__xor__': }) ---------- components: Library (Lib) files: pprint_mappingproxy.patch keywords: patch messages: 236460 nosy: fdrake, pitrou, rhettinger, serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: pprint: added support for mapping proxy type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38215/pprint_mappingproxy.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 23 22:18:56 2015 From: report at bugs.python.org (Dwight Guth) Date: Mon, 23 Feb 2015 21:18:56 +0000 Subject: [New-bugs-announce] [issue23503] undefined behavior in Objects/obmalloc.c Message-ID: <1424726336.65.0.769638302679.issue23503@psf.upfronthosting.co.za> New submission from Dwight Guth: According to the ISO C standard (http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf) section 6.5.6 paragraph 8, the behavior of the C addition operator is undefined if the result of the operand does not either fall inside the same array object that is being indexed, or one index past the end of the array object. In Objects/obmalloc.c line 841, the macros PT(0) through PT(7) expand to references to array indexes before the beginning of the usedpools array. As a result, this initializer causes this file to be undefined. ---------- components: Interpreter Core messages: 236465 nosy: dwight.guth priority: normal severity: normal status: open title: undefined behavior in Objects/obmalloc.c type: behavior _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Mon Feb 23 23:14:05 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Mon, 23 Feb 2015 22:14:05 +0000 Subject: [New-bugs-announce] [issue23504] Add __all__ into types Message-ID: <1424729645.56.0.912988413489.issue23504@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Currently pydoc outputs only two classes and two functions in the types module. Proposed patch add __all__ into the types module, so that pydoc will output builtin class deliberately added into the types module. ---------- components: Library (Lib) files: types___all__.patch keywords: patch messages: 236467 nosy: serhiy.storchaka priority: normal severity: normal stage: patch review status: open title: Add __all__ into types type: behavior versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38216/types___all__.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 01:11:53 2015 From: report at bugs.python.org (Yassine ABOUKIR) Date: Tue, 24 Feb 2015 00:11:53 +0000 Subject: [New-bugs-announce] [issue23505] Urlparse insufficient validation leads to open redirect Message-ID: <1424736713.95.0.74935935546.issue23505@psf.upfronthosting.co.za> New submission from Yassine ABOUKIR: The module urlparse lacks proper validation of the input leading to open redirect vulnerability. The issue is that URLs do not survive the round-trip through `urlunparse(urlparse(url))`. Python sees `/////foo.com` as a URL with no hostname or scheme and a path of `//foo.com`, but when it reconstructs the URL after parsing, it becomes `//foo.com`. This can be practically exploited this way : http://example.com/login?next=/////evil.com The for fix this would be for `urlunparse()` to serialize paths with two leading slashes as '/%2F', at least when `scheme` and `netloc` are empty. ---------- components: Library (Lib) messages: 236470 nosy: yaaboukir priority: normal severity: normal status: open title: Urlparse insufficient validation leads to open redirect type: security _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 05:51:45 2015 From: report at bugs.python.org (Stan) Date: Tue, 24 Feb 2015 04:51:45 +0000 Subject: [New-bugs-announce] [issue23506] Data Structure of Dict not changing its value as expected Message-ID: <1424753505.27.0.912963667947.issue23506@psf.upfronthosting.co.za> New submission from Stan: Expected the following code to change the value of the variable, it is not, comment are appreciated at this point. GUMMIE_BEARS = '' arr2 = { 'GUMMIE_BEARS' : GUMMIE_BEARS } arr2['GUMMIE_BEARS'] = 'hello world' print arr2['GUMMIE_BEARS'] print 'and this too' + GUMMIE_BEARS + '<---should be' # point #1 Expected output at point #1 should have 'hello world' contained within the string, it does not however. # I note this description: # Mutable variables ? such as dictionaries and lists ? are passed by reference, # and so if your function accepts mutable argument, it may modify the contents of that mutable variable # outside the scope of the function. Thank You. ---------- messages: 236476 nosy: opensource3 priority: normal severity: normal status: open title: Data Structure of Dict not changing its value as expected type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 09:08:32 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Tue, 24 Feb 2015 08:08:32 +0000 Subject: [New-bugs-announce] [issue23507] Tuple creation is two slow Message-ID: <1424765312.55.0.269687406134.issue23507@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: Currently tuples creating uses free lists. But this optimization is not enough. When reuse cached tuple for arguments of repeatedly called functions, the performance of some builtins can be significantly increased. $ ./python -m timeit -s "f = lambda x: x" -s "s = list(range(1000))" -- "list(filter(f, s))" Unpatched: 1000 loops, best of 3: 773 usec per loop Patched: 1000 loops, best of 3: 558 usec per loop $ ./python -m timeit -s "f = lambda x: x" -s "s = list(range(1000))" -- "list(map(f, s))" Unpatched: 1000 loops, best of 3: 689 usec per loop Patched: 1000 loops, best of 3: 556 usec per loop $ ./python -m timeit -s "f = lambda x: x" -s "s = list(range(1000))" -- "sorted(s, key=f)" Unpatched: 1000 loops, best of 3: 758 usec per loop Patched: 1000 loops, best of 3: 550 usec per loop The same effect can be achieved for itertools functions. I don't propose to commit this complicated patch, but these results can be used as a guide to the optimization of tuple creating. It is surprising to me that this patch has any effect at all. ---------- components: Interpreter Core files: reuse_argtuples.patch keywords: patch messages: 236480 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Tuple creation is two slow type: performance versions: Python 3.5 Added file: http://bugs.python.org/file38223/reuse_argtuples.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 09:56:18 2015 From: report at bugs.python.org (Gregory P. Smith) Date: Tue, 24 Feb 2015 08:56:18 +0000 Subject: [New-bugs-announce] [issue23508] Document & unittest the subprocess.getstatusoutput() status value Message-ID: <1424768178.86.0.938424229154.issue23508@psf.upfronthosting.co.za> New submission from Gregory P. Smith: The changes from http://bugs.python.org/issue10197 caused subprocess.getstatusoutput() to start returning a subprocess returncode style status for the child process instead of a POSIX style status. before that bug's changes: >>> getstatusoutput("exit 1")[0] 256 >>> getstatusoutput("kill -TERM $$")[0] 15 after that bug's changes: >>> getstatusoutput("exit 1")[0] 1 >>> getstatusoutput("kill -TERM $$")[0] -15 This behavior was entirely untested so it was missed when fixing issue10197. Given the new behavior has shipped in a stable Python release starting with (I believe) 3.3.4 and all 3.4 releases I do not think it should be changed. But we should document it with proper versionchanged notices. ---------- messages: 236482 nosy: gregory.p.smith priority: normal severity: normal status: open title: Document & unittest the subprocess.getstatusoutput() status value _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 10:43:42 2015 From: report at bugs.python.org (=?utf-8?q?J=C3=B6rn_Hees?=) Date: Tue, 24 Feb 2015 09:43:42 +0000 Subject: [New-bugs-announce] [issue23509] Counter.__iadd__ falls back to __add__ with bad performance impact Message-ID: <1424771022.43.0.932025143765.issue23509@psf.upfronthosting.co.za> New submission from J?rn Hees: This caught me when i worked on a stats script and updated c = Counter() in a loop with c += Counter(temp_dict). After profiling, i found out that my script spent 2 minutes in Counter.__add__ which seemed terribly slow (each invocation was in the ms not ?s range). Looking into this i found out that there is no implementation for Counter.__iadd__ (or __isub__) which i just assumed to be based on Counter.update (Counter.subtract). Here's some timing output: In [1]: from collections import Counter In [2]: a = range(1000) In [3]: b = range(1000) In [4]: ab = a+b In [5]: len(ab) Out[5]: 2000 In [6]: %timeit c=Counter(a) 1000 loops, best of 3: 361 ?s per loop In [7]: %timeit c=Counter(ab) 1000 loops, best of 3: 734 ?s per loop In [8]: %timeit c=Counter(a) ; c += Counter(b) 1000 loops, best of 3: 1.51 ms per loop In [9]: %timeit c=Counter(a) ; c.update(b) 1000 loops, best of 3: 741 ?s per loop Counter.update is way faster than +=, even if you re-add the overhead to create a new Counter: In [10]: %timeit c=Counter(a) ; c.update(Counter(b)) 1000 loops, best of 3: 1.16 ms per loop In [11]: %timeit c=Counter(a) ; c.update({x:1 for x in b}) 1000 loops, best of 3: 844 ?s per loop Would it be welcome if i add __iadd__ and __isub__ which just invoke update and subtract? One reason not to have this is the current __add__ and __sub__ behavior of min values > 0, which from a set-theoretic standpoint makes sense, but has a certain potential of confusing developers who just use it as a simplistic Stats module. ---------- messages: 236484 nosy: joern priority: normal severity: normal status: open title: Counter.__iadd__ falls back to __add__ with bad performance impact type: performance versions: Python 2.7, Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 14:27:54 2015 From: report at bugs.python.org (Cezary Wagner) Date: Tue, 24 Feb 2015 13:27:54 +0000 Subject: [New-bugs-announce] [issue23510] multiprocessing bug Message-ID: <1424784474.12.0.228863603483.issue23510@psf.upfronthosting.co.za> New submission from Cezary Wagner: 'with' code with manager will not work: AttributeError: 'SyncManager' object has no attribute 'shutdown' Test code: # coding=utf-8 import multiprocessing import multiprocessing.managers import logging def callback(result): print multiprocessing.current_process().name, 'callback', result def worker(io_lock, value): # error raise RuntimeError() result = value + 1 with io_lock: print multiprocessing.current_process().name, value, result return result def main(): manager = multiprocessing.managers.SyncManager() with manager: io_lock = manager.Lock() pool = multiprocessing.Pool(multiprocessing.cpu_count()) for i in range(10): print pool.apply_async(worker, args=(io_lock, i), callback = callback) pool.close() pool.join() if __name__ == '__main__': logging.basicConfig(level = logging.DEBUG) main() See this code from multiprocessing.managers.SyncManager: class SyncManager(BaseManager): pass And now see 'with' methods in Basemanager: def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): self.shutdown() ---------- components: Library (Lib) messages: 236490 nosy: Cezary.Wagner priority: normal severity: normal status: open title: multiprocessing bug type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 16:15:02 2015 From: report at bugs.python.org (Baptiste Mispelon) Date: Tue, 24 Feb 2015 15:15:02 +0000 Subject: [New-bugs-announce] [issue23511] Broken code example in email module documentation Message-ID: <1424790902.68.0.738942973553.issue23511@psf.upfronthosting.co.za> New submission from Baptiste Mispelon: The first code example at https://docs.python.org/3.5/library/email-examples.html throws an `AttributeError` because `MIMEText`'s constructor expects a `str` object, not a `bytes` one: >>> # Import smtplib for the actual sending function ... import smtplib >>> >>> # Import the email modules we'll need ... from email.mime.text import MIMEText >>> >>> # Open a plain text file for reading. For this example, assume that ... # the text file contains only ASCII characters. ... fp = open(textfile, 'rb') >>> # Create a text/plain message ... msg = MIMEText(fp.read()) Traceback (most recent call last): File "", line 2, in File "/usr/lib/python3.4/email/mime/text.py", line 33, in __init__ _text.encode('us-ascii') AttributeError: 'bytes' object has no attribute 'encode' ---------- assignee: docs at python components: Documentation messages: 236503 nosy: bmispelon, docs at python priority: normal severity: normal status: open title: Broken code example in email module documentation _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 16:25:11 2015 From: report at bugs.python.org (Edward D'Souza) Date: Tue, 24 Feb 2015 15:25:11 +0000 Subject: [New-bugs-announce] [issue23512] List of builtins is not alphabetical on https://docs.python.org/2/library/functions.html Message-ID: <1424791511.8.0.507561074794.issue23512@psf.upfronthosting.co.za> New submission from Edward D'Souza: The list of built-in functions at the top of https://docs.python.org/2/library/functions.html is not alphabetical. Specifically, (apply, coerce, intern, buffer) allow appear out of order at the end of the list, instead of where they should be alphabetically. ---------- assignee: docs at python components: Documentation messages: 236505 nosy: docs at python, edsouza priority: normal severity: normal status: open title: List of builtins is not alphabetical on https://docs.python.org/2/library/functions.html type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 18:01:43 2015 From: report at bugs.python.org (Cezary Wagner) Date: Tue, 24 Feb 2015 17:01:43 +0000 Subject: [New-bugs-announce] [issue23513] Add support for classes/object model in multiprocessing/pickle Message-ID: <1424797303.36.0.545069200968.issue23513@psf.upfronthosting.co.za> New submission from Cezary Wagner: Currently I trying to write multiprocessing program but it is impossible to write it in Python since I can not use many features and write unpythonic code. Simple example - how it should work: class Thing(object): class Result(object): def __init__(self, something): self.something = something def worker(self): return self.Result(1) def master(self): pool.apply_async(self.worker) Now I must do artificial code - not need for anybody and not solving any problem like this - artificial syntax :) class Result(object): def __init__(self, something): self.something = something def call_work_since_can_not_be_called_normally_by_multiprocessing_from_class(self): result = self.worker() return result class Thing(object): def worker(self): return Result(1) # how to overload it??? - no more inheritance def master(self): ... pool.apply_async(worker, args=(self,)) I my opinion that code is ugly and against python Zen. Why to not fix multiprocessing and pickle to support class.class or class.method - make code simpler and allow more people use parallelism in Python way not procedural way. ---------- components: Library (Lib) messages: 236516 nosy: Cezary.Wagner priority: normal severity: normal status: open title: Add support for classes/object model in multiprocessing/pickle type: enhancement versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 18:12:19 2015 From: report at bugs.python.org (Cezary Wagner) Date: Tue, 24 Feb 2015 17:12:19 +0000 Subject: [New-bugs-announce] [issue23514] multiprocessing documentation - little more examples for parallel computing Message-ID: <1424797939.13.0.206434175912.issue23514@psf.upfronthosting.co.za> New submission from Cezary Wagner: I am little confused with multiprocessing documentation since it does not contain enough examples. Parallel computing is not simple by design so documentation can be little improved to make it simpler. No pattern to use pool with classes. No pattern to exchange objects between processes. Understanding usage of Queue and SimpleQueue or JoinableQueue is not very clear. Current documentation in majority is good reference with very rare examples / patterns. ---------- assignee: docs at python components: Documentation messages: 236519 nosy: Cezary.Wagner, docs at python priority: normal severity: normal status: open title: multiprocessing documentation - little more examples for parallel computing _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 20:32:12 2015 From: report at bugs.python.org (Tim Peters) Date: Tue, 24 Feb 2015 19:32:12 +0000 Subject: [New-bugs-announce] [issue23515] Bad logic in timsort's merge_collapse Message-ID: <1424806332.96.0.559675082856.issue23515@psf.upfronthosting.co.za> New submission from Tim Peters: Some researchers found an error in the logic of merge_collapse, explained here, and with corrected code shown in section 3.2: http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it/ This affects all current versions of Python. However, I marked the priority "low" because, as the article also notes, there's currently no machine in existence with enough memory to hold an array large enough for a contrived input to trigger an overflow of the pending-runs stack. It should be fixed anyway, and their suggested fix looks good to me. ---------- components: Interpreter Core messages: 236533 nosy: tim.peters priority: low severity: normal stage: needs patch status: open title: Bad logic in timsort's merge_collapse type: crash _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 22:58:00 2015 From: report at bugs.python.org (Leonardo Tancredi) Date: Tue, 24 Feb 2015 21:58:00 +0000 Subject: [New-bugs-announce] [issue23516] requests: parse_url() mishandles special characters when the URL specifies authentication credentials Message-ID: <1424815080.16.0.743473330916.issue23516@psf.upfronthosting.co.za> New submission from Leonardo Tancredi: I was running pip install with the --proxy switch to authenticate to a proxy server with user "user" and password "pass?word", when I noticed it fails. It seems to fail when the password contains some special characters, v.g., ? and #. Here's the exception I saw: Exception: Traceback (most recent call last): File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/basecommand.py", line 232, in main status = self.run(options, args) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/commands/install.py", line 339, in run requirement_set.prepare_files(finder) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/req/req_set.py", line 333, in prepare_files upgrade=self.upgrade, File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/index.py", line 305, in find_requirement page = self._get_page(main_index_url, req) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/index.py", line 783, in _get_page return HTMLPage.get_page(link, req, session=self.session) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/index.py", line 872, in get_page "Cache-Control": "max-age=600", File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/sessions.py", line 473, in get return self.request('GET', url, **kwargs) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/download.py", line 365, in request return super(PipSession, self).request(method, url, *args, **kwargs) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/sessions.py", line 461, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/sessions.py", line 573, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/cachecontrol/adapter.py", line 43, in send resp = super(CacheControlAdapter, self).send(request, **kw) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/adapters.py", line 337, in send conn = self.get_connection(request.url, proxies) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/adapters.py", line 245, in get_connection proxy_manager = self.proxy_manager_for(proxy) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/adapters.py", line 155, in proxy_manager_for **proxy_kwargs) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/packages/urllib3/poolmanager.py", line 265, in proxy_from_url return ProxyManager(proxy_url=url, **kw) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/packages/urllib3/poolmanager.py", line 210, in __init__ proxy = parse_url(proxy_url) File "/usr/local/lib/python3.3/site-packages/pip-6.0.8-py3.3.egg/pip/_vendor/requests/packages/urllib3/util/url.py", line 185, in parse_url raise LocationParseError(url) pip._vendor.requests.packages.urllib3.exceptions.LocationParseError: Failed to parse: user:pass AFAICT the problem lies in function parse_url() in url.py because it assumes that there cannot exist neither a ? nor a # between the :// and the next / . This does not hold, because a URL can include a username and a password right there, as in http://user:pass?word at host/path. Here's the offending piece of code: if '://' in url: scheme, url = url.split('://', 1) # Find the earliest Authority Terminator # (http://tools.ietf.org/html/rfc3986#section-3.2) url, path_, delim = split_first(url, ['/', '?', '#']) It's funny that this snippet violates precisely the specification given in that comment (RFC3986 section 3.2), because it clearly states that this string can contain a userinfo field: authority = [ userinfo "@" ] host [ ":" port ] For some reason, urlencoding the password did not help either, the error message did not change. ---------- components: Extension Modules messages: 236550 nosy: leotan priority: normal severity: normal status: open title: requests: parse_url() mishandles special characters when the URL specifies authentication credentials type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Tue Feb 24 23:38:42 2015 From: report at bugs.python.org (Tommaso Barbugli) Date: Tue, 24 Feb 2015 22:38:42 +0000 Subject: [New-bugs-announce] [issue23517] datetime.utcfromtimestamp parses timestamps incorrectly Message-ID: <1424817522.64.0.693607620573.issue23517@psf.upfronthosting.co.za> New submission from Tommaso Barbugli: Hi, I am porting a library from python 2.7 to 3.4 and I noticed that the behaviour of datetime.utcfromtimestamp is not consistent between the two versions. For example on python 2.7.5 datetime.utcfromtimestamp(1424817268.274) returns a datetime with 274000 microseconds the same code in python 3.4 returns a datetime with 273999 microseconds. ---------- components: Library (Lib) messages: 236552 nosy: tbarbugli priority: normal severity: normal status: open title: datetime.utcfromtimestamp parses timestamps incorrectly type: behavior versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 25 03:58:19 2015 From: report at bugs.python.org (Zack Weinberg) Date: Wed, 25 Feb 2015 02:58:19 +0000 Subject: [New-bugs-announce] [issue23518] Misplaced diagnostic caret for some SyntaxErrors Message-ID: <1424833099.8.0.762005260502.issue23518@psf.upfronthosting.co.za> New submission from Zack Weinberg: I tripped over a couple of SyntaxError cases where the diagnostic caret is misplaced. >>> While x: File "", line 1 While x: ^ SyntaxError: invalid syntax The caret should point to the capital W in 'While'. >>> for x in the range(10): File "", line 1 for x in the range(10): ^ SyntaxError: invalid syntax The caret should point to the 'the'. ---------- messages: 236560 nosy: zwol priority: normal severity: normal status: open title: Misplaced diagnostic caret for some SyntaxErrors _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 25 12:00:40 2015 From: report at bugs.python.org (Mathieu Pasquet) Date: Wed, 25 Feb 2015 11:00:40 +0000 Subject: [New-bugs-announce] [issue23519] using asyncio.iscoroutinefunction() on a functools.partial object Message-ID: <1424862040.11.0.0579325382624.issue23519@psf.upfronthosting.co.za> New submission from Mathieu Pasquet: Using iscoroutinefunction() on an object returned by functools.partial() should return True if the function wrapped was a coroutine function. (a recursive loop like the one in asyncio/events.py get_function_source() may be what needs to be done) ---------- components: asyncio messages: 236569 nosy: gvanrossum, haypo, mathieui, yselivanov priority: normal severity: normal status: open title: using asyncio.iscoroutinefunction() on a functools.partial object type: enhancement versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 25 14:30:03 2015 From: report at bugs.python.org (Alexey) Date: Wed, 25 Feb 2015 13:30:03 +0000 Subject: [New-bugs-announce] [issue23520] test_os failed (python-3.4.3) Message-ID: <1424871003.6.0.446487630162.issue23520@psf.upfronthosting.co.za> New submission from Alexey: Just one test failed for python-3.4.3 ....... test_invalid_offset (test.test_os.TestSendfile) ... FAIL ....... ---------- files: test_os_log messages: 236574 nosy: avoevodkin priority: normal severity: normal status: open title: test_os failed (python-3.4.3) versions: Python 3.4 Added file: http://bugs.python.org/file38232/test_os_log _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 25 18:05:47 2015 From: report at bugs.python.org (Alexander Belopolsky) Date: Wed, 25 Feb 2015 17:05:47 +0000 Subject: [New-bugs-announce] [issue23521] OverflowError from timedelta * float in datetime.py Message-ID: <1424883947.81.0.425363902916.issue23521@psf.upfronthosting.co.za> New submission from Alexander Belopolsky: >>> import sys >>> sys.modules['_datetime'] = None >>> from datetime import timedelta >>> timedelta(seconds=1)*0.6112295 Traceback (most recent call last): File "", line 1, in File "/Users/a/Work/cpython/Lib/datetime.py", line 519, in __mul__ return self * a / b File "/Users/a/Work/cpython/Lib/datetime.py", line 516, in __mul__ self._microseconds * other) File "/Users/a/Work/cpython/Lib/datetime.py", line 411, in __new__ raise OverflowError("timedelta # of days is too large: %d" % d) OverflowError: timedelta # of days is too large: 63720670102 C implementation is unaffected: >>> from datetime import timedelta >>> timedelta(seconds=1)*0.6112295 datetime.timedelta(0, 0, 611229) ---------- assignee: belopolsky components: Library (Lib) messages: 236602 nosy: belopolsky priority: normal severity: normal stage: needs patch status: open title: OverflowError from timedelta * float in datetime.py type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Wed Feb 25 19:40:46 2015 From: report at bugs.python.org (Jake) Date: Wed, 25 Feb 2015 18:40:46 +0000 Subject: [New-bugs-announce] [issue23522] Misleading note in Statistics module documentation Message-ID: <1424889646.1.0.160219400081.issue23522@psf.upfronthosting.co.za> New submission from Jake: In the statistics module documentation, there is a note that states that "The mean is strongly affected by outliers and is not a robust estimator for central location: the mean is not necessarily a typical example of the data points. For more robust, although less efficient, measures of central location, see median() and mode()" https://docs.python.org/3/library/statistics.html While I appreciate the intention, this is quite misleading. The implication is that the mean, median and mode are different ways to estimate one "central location", however, in reality they are very different things (albeit which refer to a similar notion). The sample mean is an unbiased estimator of the true mean but it need not be unbiased as an estimator of the true median or modes and vice versa for the median and mode. To make this clearer I would rephrase to "The mean is strongly affected by outliers and is not necessarily representative of the central tendency of the data. For cases with large outliers or very low sample size, see median() and mode()" Apologies if this is seen as frivolous, but statistics can be hard enough to remain very clear about even when the words are used precisely. ---------- assignee: docs at python components: Documentation messages: 236612 nosy: Journeyman08, docs at python priority: normal severity: normal status: open title: Misleading note in Statistics module documentation type: enhancement versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 01:23:35 2015 From: report at bugs.python.org (=?utf-8?b?5aSP54aZ5Li0?=) Date: Thu, 26 Feb 2015 00:23:35 +0000 Subject: [New-bugs-announce] [issue23523] cmath.atanh has wrong output Message-ID: <1424910215.48.0.703419414911.issue23523@psf.upfronthosting.co.za> New submission from ???: When I was using cmath.atanh, I found that this function generates wrong output. For example: >>> import cmath >>> cmath.atanh(-1.89) (-0.5888951591901462+1.5707963267948966j) However, the correct output should be: (-0.5888951591901462-1.5707963267948966j) Clearly, the sign of the imaginary part is wrong. ---------- components: Library (Lib) messages: 236636 nosy: ??? priority: normal severity: normal status: open title: cmath.atanh has wrong output type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 06:54:17 2015 From: report at bugs.python.org (Steve Dower) Date: Thu, 26 Feb 2015 05:54:17 +0000 Subject: [New-bugs-announce] [issue23524] Use _set_thread_local_invalid_parameter_handler in posixmodule Message-ID: <1424930057.56.0.00430774738557.issue23524@psf.upfronthosting.co.za> New submission from Steve Dower: (Now that VS 2015 CTP 6 is public, here's the second half of #23314.) In posixmodule.c is a big section with the following comment: /* Microsoft CRT in VS2005 and higher will verify that a filehandle is * valid and raise an assertion if it isn't. * Normally, an invalid fd is likely to be a C program error and therefore * an assertion can be useful, but it does contradict the POSIX standard * which for write(2) states: * "Otherwise, -1 shall be returned and errno set to indicate the error." * "[EBADF] The fildes argument is not a valid file descriptor open for * writing." ... * The structures below must be updated for each version of visual studio * according to the file internal.h in the CRT source, until MS comes * up with a less hacky way to do this. * (all of this is to avoid globally modifying the CRT behaviour using * _set_invalid_parameter_handler() and _CrtSetReportMode()) */ We now have the less hacky way to avoid globally modifying the IPH, which is a new _set_thread_local_invalid_parameter_handler() function. (#23314 has hopefully already dealt with _CrtSetReportMode().) The attached patch adds new macros within posixmodule.c to handle VS 2015 builds and use the new function. I have left the old code there as a transitional measure while people migrate to the new compiler, but also have no plans to remove it. The new macros _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH should surround any code that may trigger the invalid parameter handler. The _Py_VERIFY_FD macro calls the _PyVerify_fd function when necessary - when the IPH can be suppressed, this should be a no-op, and when the IPH cannot be suppressed it must be a useful/safe check. The _PyVerify_fd function remains defined in all cases, but when it's possible to safely call into the CRT to validate the handle we now do that instead of the old approach. The only visible change should be that in debug builds you will see assertion dialogs if an invalid FD is encountered, which can be safely ignored. The test suite disables these dialogs already, so buildbots should be unaffected. Prior discussions (#3545 and #4804) have decided that it's important to leave these dialogs enabled for debug builds, and there is no way (and won't be any way) to disable these on a per-thread basis. ---------- assignee: steve.dower components: Windows keywords: patch messages: 236648 nosy: haypo, steve.dower, tim.golden, zach.ware priority: high severity: normal stage: patch review status: open title: Use _set_thread_local_invalid_parameter_handler in posixmodule type: crash versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 07:03:25 2015 From: report at bugs.python.org (Andrew Barnert) Date: Thu, 26 Feb 2015 06:03:25 +0000 Subject: [New-bugs-announce] [issue23525] isbuiltin, isroutine, etc. Message-ID: <1424930605.81.0.317229186778.issue23525@psf.upfronthosting.co.za> New submission from Andrew Barnert: The documentation and implementation for `inspect.isbuiltin` and related functions do not match. While #4968 attempted to clarify things to make the documentation match the behavior, the actual committed changes are still wrong. `isbuiltin` says "Return true if the object is a built-in function or a bound built-in method", but it returns false for bound built-in special methods (e.g., `[].__add__`). That's because it's implemented as `isinstance(types.BuiltinFunctionType)`, but bound special methods are a different type (`method-wrapper`). Terry Reedy's suggested fix from #4968 ("... or a bound built-in non-special method") would fix this, but the committed change didn't. The docstring has a further problem; it says that anything that passes will have a `__self__` with "instance to which a method is bound, or None", but for functions, the `__self__` is usually the module the `PyMethodDef` comes from, not `None`. (For example, `sum.__self__` is the`'builtins` module.) `isroutine` says "Return true if the object is a user-defined or built-in function or method." (The docstring instead says "? any kind of function or method.") But it returns false for built-in bound special methods (`[].__add__`). That's because it's implemented as `isbuiltin` or `isfunction` or `ismethod` or `ismethoddescriptor`; while `isbuiltin` picks up functions and bound non-special methods, and `ismethoddescriptor` picks up unbound special and non-special methods, nothing picks up bound special methods. A doc-only fix for this, along with Terry's suggested fix for `isbuiltin`, would be "Return true if the object is a user-defined function or method, or a built-in function, unbound method, or bound non-special method." That sounds horrible, but that's because it actually is a horrible test in the first place. A better fix would be to make `isbuiltin` handle both kinds of bound method. Then `isroutine` is correct as written, and both are documented correctly. (Note that there is no type in `types` corresponding to `method-wrapper`, so either `types` or `inspect` would have to add something like `BuiltinSpecialMethodType = type([].__add__)`.) It would be even better to fix this at the source and clean up the types of builtin functions and methods, but that boat sailed with 3.0, so... ---------- assignee: docs at python components: Documentation messages: 236649 nosy: abarnert, docs at python priority: normal severity: normal status: open title: isbuiltin, isroutine, etc. versions: Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 08:35:36 2015 From: report at bugs.python.org (Alex Shkop) Date: Thu, 26 Feb 2015 07:35:36 +0000 Subject: [New-bugs-announce] [issue23526] Silence resource warnings in test_httplib Message-ID: <1424936136.32.0.667262366966.issue23526@psf.upfronthosting.co.za> New submission from Alex Shkop: Three resource warnings are present in test_httplib. Patch closes HTTPSConnection in test_networked_noverification, test_networked_trusted_by_default_cert, test_networked_good_cert. ---------- components: Tests files: test_httplib_warnings.patch keywords: patch messages: 236652 nosy: ashkop priority: normal severity: normal status: open title: Silence resource warnings in test_httplib versions: Python 3.5 Added file: http://bugs.python.org/file38240/test_httplib_warnings.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 11:01:36 2015 From: report at bugs.python.org (Alex Shkop) Date: Thu, 26 Feb 2015 10:01:36 +0000 Subject: [New-bugs-announce] [issue23527] test_smtpnet uses incorrect port for STARTTLS Message-ID: <1424944896.48.0.616589540337.issue23527@psf.upfronthosting.co.za> New submission from Alex Shkop: test_smtpnet uses port 25 for STARTTLS, whereas gmail exposes STARTTLS SMTP over port 587. ---------- components: Tests files: test_smtpnet_starttls_port.patch keywords: patch messages: 236658 nosy: ashkop priority: normal severity: normal status: open title: test_smtpnet uses incorrect port for STARTTLS versions: Python 3.5 Added file: http://bugs.python.org/file38242/test_smtpnet_starttls_port.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 11:22:06 2015 From: report at bugs.python.org (Martin Panter) Date: Thu, 26 Feb 2015 10:22:06 +0000 Subject: [New-bugs-announce] [issue23528] Limit decompressed data when reading from GzipFile Message-ID: <1424946126.98.0.486244279604.issue23528@psf.upfronthosting.co.za> New submission from Martin Panter: This is a patch I originally posted at Issue 15955, but am moving it to a separate issue so there is less confusion. GzipFile.read() etc is susceptible to decompression bombing. My patch tests and fixes that, making use of the existing ?max_length? parameter in the ?zlib? module. The rest of Issue 15955 is about enhancing the bzip and LZMA modules to support limited decompression, but since the zlib module can already limit the decompressed data, I think this gzip patch should be considered as a bug fix rather than enhancement, e.g. the fix for Issue 16043 (gzip decoding for XML RPC module) assumed GzipFile.read() is limited. ---------- components: Library (Lib) files: gzip-bomb.patch keywords: patch messages: 236659 nosy: nikratio, vadmium priority: normal severity: normal status: open title: Limit decompressed data when reading from GzipFile type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38243/gzip-bomb.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 12:06:26 2015 From: report at bugs.python.org (Martin Panter) Date: Thu, 26 Feb 2015 11:06:26 +0000 Subject: [New-bugs-announce] [issue23529] Limit decompressed data when reading from LZMAFile and BZ2File Message-ID: <1424948786.21.0.575757839036.issue23529@psf.upfronthosting.co.za> New submission from Martin Panter: This is a follow-on from Issue 15955, which has added low-level support for limiting the amount of data from the LZMA and bzip decompressors. The high-level LZMAFile and BZ2File reader APIs need to make use of the new low level max_length parameter. I am starting off with a patch for LZMAFile, based on a patch I posted to Issue 15955. I split out a _RawReader class that does the actual decompress() calls, and then wrapped that in a BufferedReader. The LZMAFile then just delegates the read methods to the BufferedReader. This avoids needing any special code to implement buffering, readline(), etc. This involved some changes in the API though: * LZMAFile now uses BufferedReader.peek(). The current implementation seems appropriate, but I am not comfortable with the current specification in the documentation, which says it is allowed to not return any useful data. See Issue 5811. * read() now accepts size=None, because BufferedReader does. I had to change a test case for this. * BufferedReader.seek() raises a different exception for invalid ?whence? Once the LZMAFile patch sees a bit of review and looks like it might be acceptable, I plan to change the BZ2File class in a similar manner. ---------- components: Library (Lib) files: LZMAFile.v2.patch keywords: patch messages: 236663 nosy: nikratio, vadmium priority: normal severity: normal status: open title: Limit decompressed data when reading from LZMAFile and BZ2File type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file38245/LZMAFile.v2.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 15:30:54 2015 From: report at bugs.python.org (Julian Taylor) Date: Thu, 26 Feb 2015 14:30:54 +0000 Subject: [New-bugs-announce] [issue23530] os and multiprocessing.cpu_count do not respect cpuset/affinity Message-ID: <1424961054.13.0.0830068196807.issue23530@psf.upfronthosting.co.za> New submission from Julian Taylor: multiprocessing.cpu_count and os.cpu_count which are often used to determine how many processes one can run in parallel do not respect the cpuset which may limit the process to only a subset of online cpus leading to heavy oversubscription in e.g. containerized environments: $ taskset -c 0 python3.4 -c 'import multiprocessing; print(multiprocessing.cpu_count())' 32 $ taskset -c 0 python3.4 -c 'import os; print(os.cpu_count())' 32 While the correct result here should be 1. This requires programs to have to use less portable methods like forking to gnu nproc or having to read the /proc filesystem. Having a keyword argument to switch between online and available cpus would be fine too. ---------- components: Library (Lib) messages: 236671 nosy: jtaylor priority: normal severity: normal status: open versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 20:58:35 2015 From: report at bugs.python.org (johnkw) Date: Thu, 26 Feb 2015 19:58:35 +0000 Subject: [New-bugs-announce] [issue23531] SSL operations cause entire process to hang Message-ID: <1424980715.38.0.880892658959.issue23531@psf.upfronthosting.co.za> New submission from johnkw: SSL operations cause entire process to hang. At a minimum this includes the SSL read that goes on after a socket is connected. It may apply to many other (or all) SSL socket operations. ---------- components: Library (Lib) files: sslbug.py messages: 236701 nosy: johnkw priority: normal severity: normal status: open title: SSL operations cause entire process to hang type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file38251/sslbug.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Thu Feb 26 23:55:54 2015 From: report at bugs.python.org (Rick Otten) Date: Thu, 26 Feb 2015 22:55:54 +0000 Subject: [New-bugs-announce] [issue23532] regex "|" behavior differs from documentation Message-ID: <1424991354.19.0.598419279567.issue23532@psf.upfronthosting.co.za> Changes by Rick Otten : ---------- components: Regular Expressions nosy: Rick Otten, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: regex "|" behavior differs from documentation type: behavior versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 01:42:04 2015 From: report at bugs.python.org (vinod sharma) Date: Fri, 27 Feb 2015 00:42:04 +0000 Subject: [New-bugs-announce] [issue23533] MacOSX Python child process running urlopen crashed when tkMessageBox is imported Message-ID: <1424997724.04.0.676051015042.issue23533@psf.upfronthosting.co.za> New submission from vinod sharma: I've have an application which uses tkMessageBox module, multiprocessing module and urllib2 module. For some unknown reason, python child process is crashing silently while fetching a url. This only happens when tkMessageBox module is imported. Sample code to reproduce the problem is attached. My operating system is MacOS 10.10 ---------- components: Tkinter files: crashreproduce.py messages: 236721 nosy: vinod sharma priority: normal severity: normal status: open title: MacOSX Python child process running urlopen crashed when tkMessageBox is imported type: crash versions: Python 2.7 Added file: http://bugs.python.org/file38255/crashreproduce.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 07:57:30 2015 From: report at bugs.python.org (Maxime Belanger) Date: Fri, 27 Feb 2015 06:57:30 +0000 Subject: [New-bugs-announce] [issue23534] `test_longdouble` fails on Mac when using system libffi (version 3.1) Message-ID: <1425020250.66.0.941803975171.issue23534@psf.upfronthosting.co.za> New submission from Maxime Belanger: Greetings, We're compiling a custom version of Python 2.7.9 for the Mac (building on OS X 10.9 with Xcode 6.1), and we're instructing Python to use a vanilla copy of `libffi` (that we've also compiled ourselves) using the `--with-system-ffi` flag. We're running the regression test suite for sanity purposes and we've noticed this error in `test_ctypes`'s `test_longdouble` method: ====================================================================== FAIL: test_longdouble (ctypes.test.test_callbacks.Callbacks) ---------------------------------------------------------------------- Traceback (most recent call last): File "../Python.framework/Versions/2.7/lib/python2.7/ctypes/test/test_callbacks.py", line 88, in test_longdouble self.check_type(c_longdouble, 3.14) File "../Python.framework/Versions/2.7/lib/python2.7/ctypes/test/test_callbacks.py", line 24, in check_type self.assertEqual(result, arg) AssertionError: 0.0 != 3.14 I'm reasonably convinced this is caused by the version of `libffi` we're using. It seems that if the `--with-system-ffi` flag isn't set, an internal version of `libffi` is used instead. However, given that version 3.1 is said to be the version Python 2.7.9 is "tracking", this feels like a bug to me. Also, it seems that trying to hit this code path in production code could trigger a crash. ---------- components: Tests, ctypes messages: 236729 nosy: Maxime Belanger priority: normal severity: normal status: open title: `test_longdouble` fails on Mac when using system libffi (version 3.1) type: crash versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 13:20:32 2015 From: report at bugs.python.org (Eugene Bright) Date: Fri, 27 Feb 2015 12:20:32 +0000 Subject: [New-bugs-announce] [issue23535] os.path.join() wrong concatenation of "C:" on Windows Message-ID: <1425039632.15.0.00321481955838.issue23535@psf.upfronthosting.co.za> New submission from Eugene Bright: Hello! I found strange os.path.join() behavior on Windows. It works fine in common case. >>> os.path.join("C", "filename") 'C\\filename' But if first argument is "C:" there are no backslashes added at all! >>> os.path.join("C:", "filename") 'C:filename' But I expect two inserted backslashes... >>> sys.version '3.4.1 |Anaconda 2.1.0 (64-bit)| (default, Sep 24 2014, 18:32:42) [MSC v.1600 64 bit (AMD64)]' Is there a bug? Thanks! ---------- messages: 236739 nosy: Eugene Bright priority: normal severity: normal status: open title: os.path.join() wrong concatenation of "C:" on Windows _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 13:49:28 2015 From: report at bugs.python.org (Piotr Dobrogost) Date: Fri, 27 Feb 2015 12:49:28 +0000 Subject: [New-bugs-announce] [issue23536] Add explicit information on config file format not supporting filters Message-ID: <1425041368.98.0.00469006830804.issue23536@psf.upfronthosting.co.za> New submission from Piotr Dobrogost: It would be helpful to make it clear in section "Configuration file format" that it's not possible to configure filters through configuration file as opposed to dictionary passed to dictConfig() method. I found this clearly stated in Pyramid docs at http://docs.pylonsproject.org/docs/pyramid/en/latest/narr/logging.html ? "For more advanced filtering, the logging module provides a logging.Filter object; however it cannot be used directly from the configuration file." ---------- assignee: docs at python components: Documentation messages: 236741 nosy: docs at python, piotr.dobrogost, vinay.sajip priority: normal severity: normal status: open title: Add explicit information on config file format not supporting filters type: enhancement versions: Python 2.7, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 17:14:29 2015 From: report at bugs.python.org (Martin Richard) Date: Fri, 27 Feb 2015 16:14:29 +0000 Subject: [New-bugs-announce] [issue23537] BaseSubprocessTransport includes two unused methods Message-ID: <1425053669.13.0.0502434719545.issue23537@psf.upfronthosting.co.za> New submission from Martin Richard: base_subprocess.BaseSuprocessTransport implements _make_write_subprocess_pipe_proto and _make_read_subprocess_pipe_proto. Both are private and both raise NotImplementedError. However, when I grep in tulip sources for those methods, they are never called nor overridden by subclasses of BaseSuprocessTransport. Shouldn't they be removed? ---------- components: asyncio messages: 236777 nosy: gvanrossum, haypo, martius, yselivanov priority: normal severity: normal status: open title: BaseSubprocessTransport includes two unused methods _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 18:28:47 2015 From: report at bugs.python.org (Link Mauve) Date: Fri, 27 Feb 2015 17:28:47 +0000 Subject: [New-bugs-announce] [issue23538] New Windows installer in 3.5.0a1 breaks compatibility with Wine Message-ID: <1425058127.83.0.0230104914567.issue23538@psf.upfronthosting.co.za> New submission from Link Mauve: The previous msi installer was working fine with `wine msiexec /i python*.msi`, but the new exe-based one fails with an unreadable error in latest wine. ---------- components: Windows messages: 236802 nosy: Link Mauve, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: New Windows installer in 3.5.0a1 breaks compatibility with Wine type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 18:30:43 2015 From: report at bugs.python.org (Demian Brecht) Date: Fri, 27 Feb 2015 17:30:43 +0000 Subject: [New-bugs-announce] [issue23539] Content-length not set for HTTP methods expecting body when body is None Message-ID: <1425058243.4.0.797359828619.issue23539@psf.upfronthosting.co.za> New submission from Demian Brecht: #14721 solved setting the Content-Length header for 0-length bodies. However, it doesn't account for cases where body is None (reported by James Rutherford here: http://bugs.python.org/issue14721#msg236600). One method of solving this might be something like this: _METHODS_EXPECTING_BODIES = {'OPTIONS', 'POST', 'PUT', 'PATCH'} if method.upper() in _METHODS_EXPECTING_BODIES and \ 'content-length' not in header_names: self._set_content_length(body) (_set_content_length would have to be updated in order to allow for None) This ensures that Content-Length will not be set for methods not expecting a body. RFC 7230, Section 3.3.2: A user agent SHOULD NOT send a Content-Length header field when the request message does not contain a payload body and the method semantics do not anticipate such a body. ---------- components: Library (Lib) messages: 236803 nosy: demian.brecht priority: normal severity: normal stage: needs patch status: open title: Content-length not set for HTTP methods expecting body when body is None type: behavior versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 19:11:10 2015 From: report at bugs.python.org (Martin Richard) Date: Fri, 27 Feb 2015 18:11:10 +0000 Subject: [New-bugs-announce] [issue23540] Proposal for asyncio: SubprocessTransport.detach() to detach a process from a transport Message-ID: <1425060670.32.0.0609117457316.issue23540@psf.upfronthosting.co.za> New submission from Martin Richard: I would like to add a detach() method to base_suprocess.BaseSuprocessTransport, which would release the underlying Popen object to the user, pretty much like socket.detach() detaches a socket object and returns the fd. The rationale is the following: the lifetime of a subprocess started using a loop is bound to that loop, or require to clause the loop without terminating the process which leads to resource leaks (the stdin/stdout pipes can't be closed). It may be useful in some cases. For instance, I create a fork of a process running a loop which started one or more subprocesses. In the child processus, I'd like to close the pipes and free the transport objects by calling: proc = transport.detach() transport.close() proc.stdin.close() proc.stdout.close() proc.stderr.close() The process is still running, in the parent process, everything looks like before the fork, the child can forget about the parent loop without fearing resource leaks. It is somewhat related to http://bugs.python.org/issue21998 (Support fork). I propose a patch which adds BaseSubprocessTransport.detach(), a specialized version for _UnixSubprocessTransport taking care of removing the callbacks from the ChildWatcher and a detach method for the pipes transports for unix and proactor. ---------- components: asyncio files: add-detach-to-subprocess_transport.patch keywords: patch messages: 236808 nosy: gvanrossum, haypo, martius, yselivanov priority: normal severity: normal status: open title: Proposal for asyncio: SubprocessTransport.detach() to detach a process from a transport type: enhancement versions: Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file38263/add-detach-to-subprocess_transport.patch _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 21:18:35 2015 From: report at bugs.python.org (Ceridwen) Date: Fri, 27 Feb 2015 20:18:35 +0000 Subject: [New-bugs-announce] [issue23541] Re module's match() fails to halt on a particular input Message-ID: <1425068315.25.0.0642664750871.issue23541@psf.upfronthosting.co.za> New submission from Ceridwen: Attached is a three-line script whose third line, a call to match() for a compiled regex, fails to halt on Python 2.7 and on 3.4 (after changing ur in front of the regex string to r to accommodate the change in Unicode handling). I found it by stepping through the debugger so I suspect the problem is in the C backend, not the Python. ---------- components: Regular Expressions files: mwe.py messages: 236829 nosy: ceridwen, ezio.melotti, mrabarnett priority: normal severity: normal status: open title: Re module's match() fails to halt on a particular input type: behavior versions: Python 2.7, Python 3.4 Added file: http://bugs.python.org/file38266/mwe.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 21:43:48 2015 From: report at bugs.python.org (Shakeel Mohamed) Date: Fri, 27 Feb 2015 20:43:48 +0000 Subject: [New-bugs-announce] [issue23542] Update PEP 476 for using urllib2.build_opener() Message-ID: <1425069828.0.0.476725699175.issue23542@psf.upfronthosting.co.za> New submission from Shakeel Mohamed: In the "Opting Out" section of PEP 476, there's a workaround for disabling SSL cert verification. I think it would be worth updating the section to include the following workaround when using urllib2.build_opener(), like so: unverified_ssl_handler = urllib2.HTTPSHandler(context=ssl._create_unverified_context()) opener = urllib2.build_opener(unverified_ssl_handler) ---------- assignee: docs at python components: Documentation messages: 236835 nosy: Shakeel Mohamed, docs at python priority: normal severity: normal status: open title: Update PEP 476 for using urllib2.build_opener() versions: Python 2.7 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 22:13:19 2015 From: report at bugs.python.org (Rosa Maria) Date: Fri, 27 Feb 2015 21:13:19 +0000 Subject: [New-bugs-announce] [issue23543] encoding error trying to save string to file Message-ID: <1425071599.87.0.548743835483.issue23543@psf.upfronthosting.co.za> New submission from Rosa Maria: I made a program to read which files are in a windows folder, and saves in a file in order to print it, but when it tries to write in a file the following error appears: UnicodeEncodeError: 'charmap' codec can't encode character '\u2010' in position 8: character maps to I extpected that being Python-3 an utf8 native, I do'n have this problems. I send the Python script and some examples of files to read. One of the failures example is the file named: 'LKC.6558?100?HD?P?101_C.xlsx\n' which appears in windows as: 'LKC 6558 100 HD P 101_C.xlsx\n' ---------- components: Unicode files: Redear_Carpetotas.py messages: 236841 nosy: Gravitania, ezio.melotti, haypo priority: normal severity: normal status: open title: encoding error trying to save string to file type: crash versions: Python 3.4 Added file: http://bugs.python.org/file38268/Redear_Carpetotas.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 22:25:35 2015 From: report at bugs.python.org (Andrew Harrington) Date: Fri, 27 Feb 2015 21:25:35 +0000 Subject: [New-bugs-announce] [issue23544] Idle stacker viewer crash OSX Message-ID: <1425072335.83.0.576854566256.issue23544@psf.upfronthosting.co.za> New submission from Andrew Harrington: 1. In idle 3.4.3, OSX 10.10, active state Tk 8.5.17, starting data: Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin 2. start debugger, open very simple demo file, attached: 3. run, so debugger starts, step into main() 4. In debug menu, select Stack Viewer 5. hangs (apple swirling color wheel forever). Same error is repeatable. ---------- components: IDLE files: goodScope.py messages: 236842 nosy: Andrew.Harrington priority: normal severity: normal status: open title: Idle stacker viewer crash OSX type: crash versions: Python 3.4 Added file: http://bugs.python.org/file38269/goodScope.py _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Fri Feb 27 23:29:43 2015 From: report at bugs.python.org (Serhiy Storchaka) Date: Fri, 27 Feb 2015 22:29:43 +0000 Subject: [New-bugs-announce] [issue23545] Turn on extra warnings on GCC Message-ID: <1425076183.98.0.20319163057.issue23545@psf.upfronthosting.co.za> New submission from Serhiy Storchaka: What if make GCC emit extra warnings? Adding following options: -Wall -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers will make GCC emit all warnings except few types that would produce too many warnings. With these options the compilation is almost clean on 32-bit Linux. ---------- components: Build messages: 236851 nosy: serhiy.storchaka priority: normal severity: normal status: open title: Turn on extra warnings on GCC type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 08:59:27 2015 From: report at bugs.python.org (Liam Marsh) Date: Sat, 28 Feb 2015 07:59:27 +0000 Subject: [New-bugs-announce] [issue23546] windows, IDLE and pep 397 Message-ID: <1425110367.06.0.532868119558.issue23546@psf.upfronthosting.co.za> New submission from Liam Marsh: hello, pep 397 describes a "Python launcher for the Windows platform. A Python launcher is a single executable which uses a number of heuristics to locate a Python executable and launch it with a specified command line." Problem: that affects only the "open" file action behavior, and not the "EDIT with idle" file action behavior, and then it is the last installed IDLE who wins. the problem is not the IDLE changes themselves, but that one version's IDLE can only run the edited script in its version, and ignoring the shebang. Could the windows installer install a third executable (for ex: pyi.exe) which launch the correct IDLE according to the shebang line of the edited file? Or maybe it should be done using "pyw.exe /edit "... Than you for reading this and have a nice day/evening! ---------- components: IDLE, Windows messages: 236870 nosy: Liam.Marsh, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: windows, IDLE and pep 397 type: enhancement versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 15:36:25 2015 From: report at bugs.python.org (Michael Bevilacqua-Linn) Date: Sat, 28 Feb 2015 14:36:25 +0000 Subject: [New-bugs-announce] [issue23547] Engineering at Google in 2015 In-Reply-To: Message-ID: New submission from Michael Bevilacqua-Linn: Hey thanks so much for reaching out! I'm not really interested in leaving Philly or my current job at the moment though... Thanks! MBL On Fri, Feb 27, 2015 at 4:48 AM, Margaret O'Reilly wrote: > Hi Michael, > > I hope you are well. > > > My name is Margaret and I am part of the Google Technical Recruiting > team. I tried reaching you about a month ago and just wanted to check back > in to see if you might be interested in exploring a potential engineering > role at Google? > > With your experience and our current open roles, we may have a role that?s > a good fit for you. If you are interested in hearing more, please let me > know :-) > > > If however this isn?t something you?re interested in, just let me know and > we?ll be sure not to spam. > > > Thanks and looking forward to your response. > > > Regards, > > > Margaret. > > > > > On Thu, Jan 29, 2015 at 10:58 AM, Margaret O'Reilly > wrote: > >> Hi Michael, >> >> I hope all is well. My name is Margaret and I am currently working with >> Google's Recruitment team. I found your profile online and you interest and >> experience in building scalable distributed systems caught my eye as we >> have a team in Google which I think would be a great match for you! >> >> I know my colleague Derrick got in touch with you in 2014 and relocation >> wasn't an option. I wanted to get back in touch again to see if now might >> be a more convenient time perhaps? If so, I would be delighted to hear from >> you and I'll happily arrange a call at your convenience! >> >> Thanks and looking forward to hearing from you! >> >> Kind Regards, >> >> Margaret >> >> Paxos Made Live - An Engineering Perspective >> >> Bigtable: A Distributed Storage System for Structured Data >> >> > > > > -- > Best Regards, > > Margaret > > Margaret O' Reilly | Technical Sourcer | margaretor at google.com | +353 1 > 519 3003 > ---------- messages: 236889 nosy: Michael.Bevilacqua-Linn priority: normal severity: normal status: open title: Engineering at Google in 2015 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 17:13:01 2015 From: report at bugs.python.org (Jack O'Connor) Date: Sat, 28 Feb 2015 16:13:01 +0000 Subject: [New-bugs-announce] [issue23548] TypeError in event loop finalizer, new in Python 3.4.3 Message-ID: <1425139981.48.0.523334940185.issue23548@psf.upfronthosting.co.za> New submission from Jack O'Connor: This toy program: import asyncio @asyncio.coroutine def main(): p = yield from asyncio.create_subprocess_shell('echo hi') yield from p.wait() asyncio.get_event_loop().run_until_complete(main()) Produces this output on Arch Linux under Python 3.4.3 (but not 3.4.2): hi Exception ignored in: > Traceback (most recent call last): File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/base_events.py", line 361, in __del__ File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/unix_events.py", line 57, in close File "/home/jacko/Downloads/Python-3.4.3/Lib/asyncio/unix_events.py", line 138, in remove_signal_handler TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object ---------- components: asyncio messages: 236892 nosy: gvanrossum, haypo, oconnor663, yselivanov priority: normal severity: normal status: open title: TypeError in event loop finalizer, new in Python 3.4.3 type: crash versions: Python 3.4 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 17:37:41 2015 From: report at bugs.python.org (Eli Bendersky) Date: Sat, 28 Feb 2015 16:37:41 +0000 Subject: [New-bugs-announce] [issue23549] heapq docs should be more precise about how to access the smallest element Message-ID: <1425141461.42.0.47354110207.issue23549@psf.upfronthosting.co.za> New submission from Eli Bendersky: The heapq documentation has this paragraph after the doc of nsmallest: The latter two functions perform best for smaller values of n. For larger values, it is more efficient to use the sorted() function. Also, when n==1, it is more efficient to use the built-in min() and max() functions. This is confusing as it suggests to use min() on a heap to find the minimal element. heap[0] is the minimal element, but this is only mentioned at the very top - so you need to read the doc of the entire module to find it. Nothing in the docs of methods suggests it. ---------- assignee: docs at python components: Documentation messages: 236895 nosy: docs at python, eli.bendersky priority: normal severity: normal status: open title: heapq docs should be more precise about how to access the smallest element type: behavior versions: Python 2.7, Python 3.4, Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 19:21:10 2015 From: report at bugs.python.org (Hammerite) Date: Sat, 28 Feb 2015 18:21:10 +0000 Subject: [New-bugs-announce] [issue23550] Add to unicodedata a function to query the "Quick_Check" property for a character Message-ID: <1425147670.05.0.762030398058.issue23550@psf.upfronthosting.co.za> New submission from Hammerite: Unicode Standard Annex #15 (http://unicode.org/reports/tr15/#Stable_Code_Points) describes how each character in Unicode, for each of the four normalisation forms, has a "Quick_Check" value that aids in determining whether a given string is in that normalisation form. It goes on to describe, in section 9.1, how this "Quick_Check" value may be used to optimise the concatenation of a string onto a normalised string to produce another normalised string: normalisation need only be performed from the last "stable" character in the left-hand string onwards, where a character is "stable" if it has the "Quick_Check" property and has a canonical combining class of 0. This will generally be more efficient than re-running the normalisation algorithm on the entire concatenated string, if the strings involved are long. The unicodedata standard-library module does not, in my understanding, expose this information. I would like to see a new function added that allows us to determine whether a given character has the "Quick_Check" property for a given normalisation form. This function might accept two parameters, the first of which is a string indicating the normalisation form and the second of which is the character being tested (similar to unicodedata.normalize()). Suppose we have a need to accept text data, receiving chunks of it at a time, and every time we receive a new chunk we need to append it to the string so far and also make sure that the resulting string is normalised to a particular normalisation form (NFD say). This implies that we would like to be able to concatenate the new chunk (which may not be normalised) onto the string "so far" (which is) and have the result be normalised - but without re-doing normalisation of the whole string over again, as this might be inefficient. From the linked UAX, this might be achieved like this, where unicodedata.quick_check() is the requested function: def concat (s1, s2): LSCP = len(s1) # Last stable character position while LSCP > 0: LSCP -= 1 if unicodedata.combining(s1[LSCP]) == 0 and unicodedata.quick_check('NFD', s1[LSCP]): break return s1[:LSCP] + unicodedata.normalize('NFD', s1[LSCP:] + s2) ---------- components: Library (Lib), Unicode messages: 236901 nosy: Hammerite, ezio.melotti, haypo priority: normal severity: normal status: open title: Add to unicodedata a function to query the "Quick_Check" property for a character type: enhancement _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 21:35:52 2015 From: report at bugs.python.org (Raymond Hettinger) Date: Sat, 28 Feb 2015 20:35:52 +0000 Subject: [New-bugs-announce] [issue23551] IDLE to provide menu options for using PIP Message-ID: <1425155752.55.0.0736033483828.issue23551@psf.upfronthosting.co.za> New submission from Raymond Hettinger: In teaching Python, I find that many Windows users are command-line challenged and have difficulties using and accessing PIP. In the spirit of what Tortoise has done for version control, I propose that menu options be added for PIP. PyPI (Python Package Index) --------------------------- * Ensure pip / Upgrade pip * Show installed packages (freeze) (and give options to update or remove) * Install from requirements file (install -r) * Search for package * Install package * Upgrade package (install -U) * Uninstall package * Change default repository (default pypi.python.org) * Change default install directory (in case the user doesn't have admin rights) This change would be a major step forward in making the PyPi accessible by people who lack the skills or inclination to wrestle with PIP. I would love to be able to start a class with a fresh Python download from python.org and effortlessly install requests and other tools without users having to fire-up a terminal window and wrestle with the various parts. ---------- assignee: terry.reedy components: IDLE messages: 236906 nosy: rhettinger, terry.reedy priority: normal severity: normal stage: needs patch status: open title: IDLE to provide menu options for using PIP type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________ From report at bugs.python.org Sat Feb 28 21:42:52 2015 From: report at bugs.python.org (Raymond Hettinger) Date: Sat, 28 Feb 2015 20:42:52 +0000 Subject: [New-bugs-announce] [issue23552] Have timeit warn about runs that are not independent of each other Message-ID: <1425156172.0.0.931116052614.issue23552@psf.upfronthosting.co.za> New submission from Raymond Hettinger: IPython 3.0 added a useful feature that we ought to consider for inclusion either in timeit.repeat() or in the command-line interface: """Using %timeit prints warnings if there is at least a 4x difference in timings between the slowest and fastest runs, since this might meant that the multiple runs are not independent of one another.""" ---------- components: Library (Lib) messages: 236908 nosy: rhettinger priority: normal severity: normal status: open title: Have timeit warn about runs that are not independent of each other type: enhancement versions: Python 3.5 _______________________________________ Python tracker _______________________________________