Mailman 3 March 2013 - Python-Dev

noob contributions to unit tests
by Sean Felipe Wolfe 02 May '13

02 May '13

Hey everybody how are you all :) I am an intermediate-level python coder looking to get help out. I've been reading over the dev guide about helping increase test coverage --> http://docs.python.org/devguide/coverage.html And also the third-party code coverage referenced in the devguide page: http://coverage.livinglogic.de/ I'm seeing that according to the coverage tool, two of my favorite libraries, urllib/urllib2, have no unit tests? Is that correct or am I reading it wrong? If that's correct it seems like a great place perhaps for me to cut my teeth and I would be excited to learn and help out here. And of course any thoughts or advice for an aspiring Python contributor would be appreciated. Of course the dev guide gives me plenty of good info. Thanks! -- A musician must make music, an artist must paint, a poet must write, if he is to be ultimately at peace with himself. - Abraham Maslow

5 5

Introducing Electronic Contributor Agreements
by Brian Curtin 15 Apr '13

15 Apr '13

The full announcement is at http://blog.python.org/2013/03/introducing-electronic-contributor.html, but a summary follows. We've now moved to an electronic Contributor License Agreement form at http://www.python.org/psf/contrib/contrib-form/ which will hopefully ease the signing and sending of forms for our potential contributors. The form shows the required fields whether you're signing as an individual or a representative of an organization, and removes the need to print, scan, fax, etc. When a new contributor fills in the form, they are emailed a copy of the form and asked to confirm the email address that they used (and received that copy at). Upon confirming, the signed form is sent to the PSF Administrator and filed away. The signature can either be generated from your typed name, or you can draw or upload your actual written signature if you choose.

15 30

How to fix the incorrect shared library extension on linux for 3.2 and newer?
by Matthias Klose 10 Apr '13

10 Apr '13

This is http://bugs.python.org/issue16754, affecting Linux systems only, and only those which don't provide static libraries. PEP 3149 did change the SO macro to include the ABI tag, although the SO macro is used to search for shared system libraries too. E.g. searching for the jpeg library search for a file libjpeg.cpython3.3m.so, which is not found. If the static library libjpeg.a is found, it is taken, and linked as -ljpeg, which then links with the shared library. The patch in the issue now makes a distinction between EXT_SUFFIX and SHLIB_SUFFIX, and restores the value of SO to SHLIB_SUFFIX. Now this could break users of sysconfig.get_config_var('SO'), however I don't see a better way than to restore the original behaviour and advise people to use the new config variables. Matthias

6 5

Semantics of __int__(), __index__()
by Mark Shannon 06 Apr '13

06 Apr '13

Hi all, I was looking into http://bugs.python.org/issue17576 and I found that the semantics of __int__() and __index__() are not precisely defined in the documentation and that the implementation (CPython 3.4a) has some odd behaviour. Defining two classes: class Int1(int): def __init__(self, val=0): print("new %s" % self.__class__) class Int2(Int1): def __int__(self): return self and two instances i1 = Int1() i2 = Int2() we get the following behaviour: >>> type(int(i1)) <class 'int'> I would have expected 'Int1' >>> type(int(i2)) new <class '__main__.Int2'> <class '__main__.Int2'> Why is a new Int2 being created? operator.index does similar things. So, 1. Should type(int(x)) be exactly int, or is any subclass OK? 2. Should type(index(x)) be exactly int, or is any subclass OK? 3. Should int(x) be defined as int_check(x.__int__())? 4. Should operator.index(x) be defined as index_check(x.__index__())? where: def int_check(x): if is_int(x): return x else: raise TypeError(...) def index_check(x): if is_index(x): return x else: raise TypeError(...) The definition of is_int(x) and is_index(x) follow from the answers to 1 and 2. I had previously assumed (and would expect) that the answers were: 1. Any subclass is OK 2. Ditto 3. Yes 4. Yes Which means that def is_int(x): return int in type(x).__mro__ is_index = is_int Cheers, Mark.

23 56

Slides from today's parallel/async Python talk
by Trent Nelson 05 Apr '13

05 Apr '13

Just posted the slides for those that didn't have the benefit of attending the language summit today: https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alter… Trent.

10 27

Are undocumented exceptions considered bugs?
by Stefan Bucur 03 Apr '13

03 Apr '13

Hi, I'm not sure this is the right place to ask this question, but I thought I'd give it a shot since it also concerns the Python standard library. I'm writing an automated test case generation tool for Python programs that explores all possible execution paths through a program. When applying this tool on Python's 2.7.3 urllib package, it discovered input strings for which the urllib.urlopen(url) call would raise a TypeError. For instance: urllib.urlopen('\x00\x00\x00') [...] File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 86, in urlopen return opener.open(url) File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 207, in open return getattr(self, name)(url) File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 462, in open_file return self.open_local_file(url) File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 474, in open_local_file stats = os.stat(localname) TypeError: must be encoded string without NULL bytes, not str In the urllib documentation it is only mentioned that the IOError is raised when the connection cannot be established. Since the input passed is a string (and not some other type), is the TypeError considered a bug (either in the documentation, or in the implementation)? Thanks a lot, Stefan

3 3

A bit about the GIL
by Alfredo Solano Martínez 01 Apr '13

01 Apr '13

Hi, I know this may be tiresome by now, so feel free to ignore, but I'd like to share with the list an idea about the GIL, more specifically the reference counting mechanism. Simply put, make the reference counter a sharded one. That is, separate it into several subcounters, in this case one for each thread. The logic would then be something like this: - when increasing the refcount, a thread writes only to its own subcounter, creating one first if necessary. - similarly, when decreasing the refcount, there is no need to access other subcounters until that subcounter reaches zero. - when a subcounter gets to zero, delete it, and read the other subcounters to check if it was the last one. - delete the object only if there are no more subcounters. Contention could then be reduced to a minimum, since a thread only needs to read other subcounters when its own reaches zero or wants the total value. Depending on the implementation it might help with false sharing too, as subcounters may or may not be in the same cache-line. Unfortunately, in a crude test of mine there is already a severe performance degradation, and that is without rwlocks. I've used a basic linked list, and changed the INCREF/DECREF macros to functions to accommodate the extra logic so it may not be the best approach (too many dereferences). Does this makes sense to anyone? Thanks, Alfredo PS: At the very least, it might be another reason to keep the GIL.

4 3

mingw32 port
by Matthias Klose 31 Mar '13

31 Mar '13

[No, I'm not interested in the port myself] patches for a mingw32 port are floating around on the web and the python bug tracker, although most of them as a set of patches in one issue addressing several things, and which maybe outdated for the trunk. at least for me re-reading a big patch in a new version is more work than having the patches in different issues. So proposing to break down the patches in independent ones, dealing with: - mingw32 support (excluding any cross-build support). tested with a native build with srcdir == builddir. The changes for distutils mingw32 support should be a separate patch. Who could review these? Asked Martin, but didn't get a reply yet. - patches to cross-build for mingw32. - patches to deal with a srcdir != builddir configuration, where the srcdir is read-only (please don't mix with the cross-build support). All work should be done on the tip/trunk. So ok to close issue16526, issue3871, issue3754 and suggest in the reports to start over with more granular changes? Matthias

3 2

Re: [Python-Dev] [Python-checkins] cpython (2.7): Add an itertools recipe showing how to use t.__copy__().
by Brett Cannon 31 Mar '13

31 Mar '13

"Upcomping" -> "upcoming" On Mar 31, 2013 2:38 AM, "raymond.hettinger" <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/1026b1d47f30 > changeset: 83037:1026b1d47f30 > branch: 2.7 > parent: 83034:e044d22d2f61 > user: Raymond Hettinger <python(a)rcn.com> > date: Sat Mar 30 23:37:57 2013 -0700 > summary: > Add an itertools recipe showing how to use t.__copy__(). > > files: > Doc/library/itertools.rst | 12 ++++++++++++ > 1 files changed, 12 insertions(+), 0 deletions(-) > > > diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst > --- a/Doc/library/itertools.rst > +++ b/Doc/library/itertools.rst > @@ -828,6 +828,18 @@ > indices = sorted(random.randrange(n) for i in xrange(r)) > return tuple(pool[i] for i in indices) > > + def tee_lookahead(t, i): > + """Inspect the i-th upcomping value from a tee object > + while leaving the tee object at its current position. > + > + Raise an IndexError if the underlying iterator doesn't > + have enough values. > + > + """ > + for value in islice(t.__copy__(), i, None): > + return value > + raise IndexError(i) > + > Note, many of the above recipes can be optimized by replacing global > lookups > with local variables defined as default values. For example, the > *dotproduct* recipe can be written as:: > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > >

1 0

Re: [Python-Dev] py2.7: dictobject not properly resizing
by Micha Gorelick 31 Mar '13

31 Mar '13

True, but your example mechanism of getting a shrink event is purely based on ma_fill. This is happening because your last loop is increasing ma_fill to the point where it thinks it needs to resize because of the load factor and then it calculates the new size based on ma_used. The comment that I pointed out from the code seems to imply that simply having ma_fill >> ma_used will trigger a resize. The two conditions for a resize are definitely not equivalent! Micha ----------------------------- http://micha.gd/ http://github.com/mynameisfiber/

1 1