Mailman 3 October 2012 - Python-Dev

Split unicodeobject.c into subfiles?
by Victor Stinner Nov. 17, 2012

Nov. 17, 2012

Hi, I would like to split the huge unicodeobject.c file into smaller files. It's just the longest C file of CPython: 14,849 lines. I don't know exactly how to split it, but first I would like to know if you would agree with the idea. Example: - Objects/unicode/codecs.c - Objects/unicode/mod_format.c - Objects/unicode/methods.c - Objects/unicode/operators.c - etc. I don't know if it's better to use a subdirectory, or use a prefix for new files: Objects/unicode_methods.c, Objects/unicode_codecs.c, etc. There is already a Python/codecs.c file for example (same filename). I would like to split the unicodeobject.c because it's hard to navigate in this huge file between all functions, variables, types, macros, etc. It's hard to add new code and to fix bugs. For example, the implementation of str%args takes 1000 lines, 2 types and 10 functions (since my refactor yesterday, in Python 3.3 the main function is 500 lines long :-)). I only see one argument against such refactoring: it will be harder to backport/forwardport bugfixes. Victor

10 23

[BUG] Trailing spaces in pretty-printed JSON
by Leo Nov. 15, 2012

Nov. 15, 2012

Use this script on a json file and observe all the trailing spaces generated. (screenshot attached.) #!/usr/bin/env python """ Pretty print json file. """ if __name__ == '__main__': import sys import json if '-h' in sys.argv or '--help' in sys.argv: print "Usage: ppjson <FILE>" exit(0) assert sys.argv[1], "No file provided" with open(sys.argv[1]) as f: print json.dumps(json.load(f), indent=4)

3 3

accept the wheel PEPs 425, 426, 427
by Daniel Holth Nov. 13, 2012

Nov. 13, 2012

I'd like to submit the Wheel PEPs 425 (filename metadata), 426 (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has been stable since May and we are preparing a patch to support it in pip, but we need to earn consensus before including it in the most widely used installer. Wheel is a binary packaging format that is mostly based on the phenomenal work done by 'packaging' as PEP 376 et al. The resulting packages solve packaging problems by being installable without setup.py or any variation of distutils, lxml can be installed in 0.7 seconds, and a simple installer is just "unzip" inside site-packages. Wheel installers know about the major version number of the spec itself, and will refuse to install version 2.0 wheels if they do not understand them, so the format can be changed down the line. Let me know what I need to do to get it accepted, if anything needs to be added or revised, or, preferably, that it is awesome and you want to use it ASAP. Thanks, Daniel Holth

12 42

Improve error message "UnboundLocalError: local variable referenced before assignment"
by anatoly techtonik Nov. 12, 2012

Nov. 12, 2012

Here is the code: ---[cut]----------------------------- DEBUG = [] FONT_NAMES = [] def names(): if len(DEBUG): print(len(DEBUG)) if len(FONT_NAMES): print(len(FONT_NAMES)) if len(FONT_NAMES)==0: FONT_NAMES = "query()" names() ---[cut]----------------------------- Here is the output: Traceback (most recent call last): File "globalocal.py", line 13, in <module> names() File "globalocal.py", line 8, in names if len(FONT_NAMES): UnboundLocalError: local variable 'FONT_NAMES' referenced before assignment As you may see there is inconsistency between handling of line 6 - "if len(DEBUG):" and line 8 - "if len(FONT_NAMES):". This is very magical and hard to troubleshoot. I wonder if this message can be improved with a pointer to the concept on when global variables become local? -- anatoly t.

12 14

Python 3.3 vs. Python 2.7 benchmark results (again, but this time more solid numbers)
by Brett Cannon Nov. 3, 2012

Nov. 3, 2012

I re-ran the unladen benchmarks on my work machine and w/o the -b option flipped on (i.e. more thorough benchmark numbers). I figured I would share them now instead of after my PyCon Argentina talk in case people decide to dig into the results now, find a pathological problem in CPython, and then fix it before I give my presentation (if you have trouble running a benchmark or it isn't available in the repo because it's one I hacked together, just ask and I can help you run the benchmark if you want to try to speed things up). I have colour-coded benchmarks based on whether it is faster or slower in Python 3.3 (sorry for those of you who hate HTML email). But the tl;dr message is that Python 3.3 looks good compared to Python 2.7 (the median benchmark score is 5% slower). Worst benchmark is nosite_startup, best is telco. The benchmarks people might want to analyze (i.e. more than 20% slower in Python 3.3) are mako_v2, threaded_count, normal_startup, iterative_count, pathlib, formatted_logging, and simple_logging. ########################################### Report on Linux 3.2.5-gg987 #1 SMP Fri Sep 14 02:36:36 PDT 2012 x86_64 x86_64 Total CPU cores: 12 ### 2to3 ### 9.320000 -> 8.980000: 1.04x faster ### call_method ### Min: 0.417756 -> 0.355247: 1.18x faster Avg: 0.419688 -> 0.356382: 1.18x faster Significant (t=92.85) Stddev: 0.00604 -> 0.00577: 1.0479x smaller Timeline: b'http://tinyurl.com/8soruze' ### call_method_slots ### Min: 0.417611 -> 0.358451: 1.17x faster Avg: 0.420761 -> 0.359676: 1.17x faster Significant (t=88.70) Stddev: 0.00605 -> 0.00588: 1.0291x smaller Timeline: b'http://tinyurl.com/8uu8234' ### call_method_unknown ### Min: 0.459057 -> 0.359327: 1.28x faster Avg: 0.462929 -> 0.360410: 1.28x faster Significant (t=137.99) Stddev: 0.00698 -> 0.00583: 1.1969x smaller Timeline: b'http://tinyurl.com/9mo7h24' ### call_simple ### Min: 0.341689 -> 0.265289: 1.29x faster Avg: 0.343003 -> 0.266503: 1.29x faster Significant (t=124.20) Stddev: 0.00555 -> 0.00511: 1.0859x smaller Timeline: b'http://tinyurl.com/9pnn7q4' ### chameleon ### Min: 0.072232 -> 0.062713: 1.15x faster Avg: 0.074588 -> 0.064261: 1.16x faster Significant (t=33.74) Stddev: 0.00284 -> 0.00245: 1.1599x smaller Timeline: b'http://tinyurl.com/8my8afl' ### chaos ### Min: 0.313727 -> 0.367015: 1.17x slower Avg: 0.317568 -> 0.371473: 1.17x slower Significant (t=-26.72) Stddev: 0.00962 -> 0.01053: 1.0942x larger Timeline: b'http://tinyurl.com/9y2u6kh' ### django ### Min: 0.798331 -> 0.855461: 1.07x slower Avg: 0.801109 -> 0.860996: 1.07x slower Significant (t=-87.43) Stddev: 0.00336 -> 0.00348: 1.0356x larger Timeline: b'http://tinyurl.com/9sf95pq' ### fannkuch ### Min: 1.364705 -> 1.327680: 1.03x faster Avg: 1.380412 -> 1.337467: 1.03x faster Significant (t=10.48) Stddev: 0.02056 -> 0.02040: 1.0077x smaller Timeline: b'http://tinyurl.com/9r2vq6g' ### fastpickle ### Min: 0.763479 -> 0.805715: 1.06x slower Avg: 0.770036 -> 0.810855: 1.05x slower Significant (t=-12.73) Stddev: 0.01618 -> 0.01589: 1.0180x smaller Timeline: b'http://tinyurl.com/9rvqo4s' ### fastunpickle ### Min: 0.588694 -> 0.663616: 1.13x slower Avg: 0.596622 -> 0.672418: 1.13x slower Significant (t=-23.22) Stddev: 0.01503 -> 0.01752: 1.1656x larger Timeline: b'http://tinyurl.com/9eggn34' ### float ### Min: 0.363234 -> 0.344408: 1.05x faster Avg: 0.376159 -> 0.354165: 1.06x faster Significant (t=8.76) Stddev: 0.01282 -> 0.01227: 1.0455x smaller Timeline: b'http://tinyurl.com/8d6rcb8' ### formatted_logging ### Min: 0.330988 -> 0.400309: 1.21x slower Avg: 0.335522 -> 0.408920: 1.22x slower Significant (t=-33.48) Stddev: 0.00989 -> 0.01194: 1.2076x larger Timeline: b'http://tinyurl.com/9ll7dqk' ### genshi ### Min: 0.229140 -> 0.251766: 1.10x slower Avg: 0.232124 -> 0.257252: 1.11x slower Significant (t=-40.24) Stddev: 0.00516 -> 0.00564: 1.0925x larger Timeline: b'http://tinyurl.com/9dpuuaw' ### go ### Min: 0.632778 -> 0.710382: 1.12x slower Avg: 0.636143 -> 0.716748: 1.13x slower Significant (t=-37.61) Stddev: 0.00186 -> 0.01504: 8.0815x larger Timeline: b'http://tinyurl.com/8s7vw74' ### hexiom2 ### Min: 150.982155 -> 154.702444: 1.02x slower Avg: 151.194622 -> 154.780953: 1.02x slower Significant (t=-15.83) Stddev: 0.30047 -> 0.11103: 2.7063x smaller Timeline: b'http://tinyurl.com/8rkkduv' ### iterative_count ### Min: 0.117036 -> 0.156752: 1.34x slower Avg: 0.120802 -> 0.172218: 1.43x slower Significant (t=-34.92) Stddev: 0.00542 -> 0.00889: 1.6422x larger Timeline: b'http://tinyurl.com/9x9rtnk' ### json_dump_v2 ### Min: 3.449868 -> 3.522645: 1.02x slower Avg: 3.467124 -> 3.541902: 1.02x slower Significant (t=-13.20) Stddev: 0.02701 -> 0.02960: 1.0959x larger Timeline: b'http://tinyurl.com/8bsz64a' ### json_load ### Min: 0.981740 -> 0.567611: 1.73x faster Avg: 0.986729 -> 0.572975: 1.72x faster Significant (t=128.95) Stddev: 0.01796 -> 0.01386: 1.2955x smaller Timeline: b'http://tinyurl.com/93txokx' ### mako_v2 ### Min: 0.083660 -> 0.243323: 2.91x slower Avg: 0.084634 -> 0.247875: 2.93x slower Significant (t=-821.55) Stddev: 0.00193 -> 0.00400: 2.0737x larger Timeline: b'http://tinyurl.com/98n9fab' ### meteor_contest ### Min: 0.257992 -> 0.232116: 1.11x faster Avg: 0.262581 -> 0.236684: 1.11x faster Significant (t=14.31) Stddev: 0.00916 -> 0.00894: 1.0243x smaller Timeline: b'http://tinyurl.com/8tpjt43' ### nbody ### Min: 0.375414 -> 0.293685: 1.28x faster Avg: 0.379489 -> 0.299794: 1.27x faster Significant (t=42.71) Stddev: 0.00997 -> 0.00864: 1.1537x smaller Timeline: b'http://tinyurl.com/96aqtod' ### normal_startup ### Min: 0.360002 -> 0.593214: 1.65x slower Avg: 0.386755 -> 0.600625: 1.55x slower Significant (t=-134.28) Stddev: 0.01055 -> 0.00395: 2.6704x smaller Timeline: b'http://tinyurl.com/9td8pna' ### nqueens ### Min: 0.300390 -> 0.363904: 1.21x slower Avg: 0.304282 -> 0.368003: 1.21x slower Significant (t=-37.41) Stddev: 0.00813 -> 0.00888: 1.0920x larger Timeline: b'http://tinyurl.com/9zxyfcu' ### pathlib ### Min: 0.106088 -> 0.138693: 1.31x slower Avg: 0.107279 -> 0.139885: 1.30x slower Significant (t=-133.12) Stddev: 0.00256 -> 0.00290: 1.1324x larger Timeline: b'http://tinyurl.com/9llvj6a' ### pidigits ### Min: 0.351666 -> 0.341745: 1.03x faster Avg: 0.354743 -> 0.344146: 1.03x faster Significant (t=5.89) Stddev: 0.00965 -> 0.00829: 1.1643x smaller Timeline: b'http://tinyurl.com/8bkgrv4' ### raytrace ### Min: 1.547054 -> 1.641147: 1.06x slower Avg: 1.552614 -> 1.643716: 1.06x slower Significant (t=-286.42) Stddev: 0.00190 -> 0.00120: 1.5920x smaller Timeline: b'http://tinyurl.com/9bmnbsd' ### regex_compile ### Min: 0.494022 -> 0.537924: 1.09x slower Avg: 0.497904 -> 0.541971: 1.09x slower Significant (t=-18.23) Stddev: 0.01177 -> 0.01239: 1.0523x larger Timeline: b'http://tinyurl.com/cvdhrrm' ### regex_effbot ### Min: 0.065431 -> 0.073393: 1.12x slower Avg: 0.069753 -> 0.077338: 1.11x slower Significant (t=-10.61) Stddev: 0.00361 -> 0.00354: 1.0179x smaller Timeline: b'http://tinyurl.com/cupb89m' ### regex_v8 ### Min: 0.071053 -> 0.081441: 1.15x slower Avg: 0.075075 -> 0.086167: 1.15x slower Significant (t=-12.44) Stddev: 0.00359 -> 0.00518: 1.4455x larger Timeline: b'http://tinyurl.com/d9ly6x3' ### simple_logging ### Min: 0.325386 -> 0.395093: 1.21x slower Avg: 0.330235 -> 0.399825: 1.21x slower Significant (t=-34.22) Stddev: 0.00952 -> 0.01077: 1.1317x larger Timeline: b'http://tinyurl.com/8sbqv85' ### startup_nosite ### Min: 0.082137 -> 0.453112: 5.52x slower Avg: 0.129994 -> 0.459361: 3.53x slower Significant (t=-276.85) Stddev: 0.01114 -> 0.00419: 2.6585x smaller Timeline: b'http://tinyurl.com/932mzal' ### telco ### Min: 0.810000 -> 0.010000: 81.00x faster Avg: 0.823600 -> 0.015200: 54.18x faster Significant (t=284.37) Stddev: 0.01946 -> 0.00505: 3.8556x smaller Timeline: http://chart.apis.google.com/chart?cht=lc&chs=700x400&chxt=x,y,x,y&chxr=1,0… ### threaded_count ### Min: 0.140653 -> 0.173500: 1.23x slower Avg: 0.152514 -> 0.270779: 1.78x slower Significant (t=-49.87) Stddev: 0.00605 -> 0.01564: 2.5837x larger Timeline: b'http://tinyurl.com/9w4u7el' ### unpack_sequence ### Min: 0.000077 -> 0.000067: 1.15x faster Avg: 0.000081 -> 0.000069: 1.18x faster Significant (t=1163.57) Stddev: 0.00000 -> 0.00000: 1.7412x larger Timeline: b'http://tinyurl.com/8qdcjcr' The following not significant results are hidden, use -v to show them: html5lib, richards, silent_logging, spectral_norm.

17 36

Re: [Python-Dev] [Python-checkins] cpython: Issue #15478: Fix test_os on Windows (os.chown is missing)
by Jeremy Kloth Oct. 31, 2012

Oct. 31, 2012

On Tue, Oct 30, 2012 at 6:07 PM, victor.stinner <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/01cc9fb52887 > changeset: 80068:01cc9fb52887 > user: Victor Stinner <victor.stinner(a)gmail.com> > date: Wed Oct 31 01:04:10 2012 +0100 > summary: > Issue #15478: Fix test_os on Windows (os.chown is missing) > > files: > Lib/test/test_os.py | 3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > > diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py > --- a/Lib/test/test_os.py > +++ b/Lib/test/test_os.py > @@ -2069,7 +2069,6 @@ > funcs = [ > (os.chdir,), > (os.chmod, 0o777), > - (os.chown, 0, 0), > (os.lchown, 0, 0), > (os.listdir,), > (os.lstat,), > @@ -2081,6 +2080,8 @@ > (os.truncate, 0), > (os.unlink,), > ] > + if hasattr(os, "chown"): > + funcs.append((os.chown, 0, 0)) > if sys.platform == "win32": > funcs.extend(( > (os._getfullpathname,), Also missing on Windows is os.lchown.

2 1

Sign of bytes
by anatoly techtonik Oct. 31, 2012

Oct. 31, 2012

Hi, I wonder why Python uses signed chars for bytes http://docs.python.org/2/library/ctypes.html#ctypes.c_byte This is a Java thing, but Java doesn't have unsigned types at all http://en.wikipedia.org/wiki/Criticism_of_Java#Unsigned_integer_types Windows implements BYTE as unsigned char, and it is in the same line as WORD, DWORD etc. The way you look at memory contents in assembly. byte type is also unsigned in .NET platform for all languages implementes, and also has a sbyte counterpart. When working with bytes in decimal system it is much more convenient to operate with strictly positive values than with -128 - 127 (or is it -127 to 128?) -- anatoly t.

3 3

Segmentaion fault with wrongly set PYTHONPATH on Windows
by anatoly techtonik Oct. 31, 2012

Oct. 31, 2012

Could anybody reopen http://bugs.python.org/issue8766 ? I can't. Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test). > set PYTHONHOME=C:\ > python BTW, what is the role of PYTHONPATH on Windows? Is it a path for %INSTALLDIR%\Lib\site-packages? -- anatoly t.

8 14

Python Bug Day this Saturday announced
by Éric Araujo Oct. 30, 2012

Oct. 30, 2012

Hi everybody, I just sent the announcement for the bug day to python-list (apparently pending approval), core-mentorship and montrealpython. Core developers who plan on being on IRC can add themselves to the list on http://wiki.python.org/moin/PythonBugDay so that people can connect nicknames with people. The list by Petri at http://piratepad.net/pyconfi-sprint-issues can still be updated. Otherwise we’ll fall back to the usual roundup query for easy bugs. Cheers!

2 1

configure produces a non-working Makefile in some scenarios, due to ASDLGEN
by Vinay Sajip Oct. 29, 2012

Oct. 29, 2012

In some scenarios, configure produces a Makefile which fails because ASDLGEN doesn't point to a working Python. In particular, it seems to assume that if there is an executable called e.g. "python3.4" on the path, then that will be a system Python. In my perhaps unusual but IMO perfectly valid setup, I have various Python repos set up like so: $HOME/projects/python | +- default | +- 3.3 | + and so on for 3.2, 3.1, 2.7 In order to facilitate testing some script against multiple Python versions, I have shell scripts in $HOME/bin named python3.4, python3.3 etc. which just run the programs in the relevant directories in $HOME/projects/python/. (I know I can do this using aliases etc., but I think that's beside the point.) When I run configure in the repo for the default branch, it appears to look for a Python on the path named python3.4, finds one in $HOME/bin, and thus generates an ASDLGEN value of "python3.4". If I happen to have a built Python in the default repo, then the script in $HOME/bin will successfully run that, and all appears well; but if I clean the default repo and re-run make, it fails at the ASDLGEN step, because the $HOME/bin/python3.4 script fails, due to not being able to invoke $HOME/projects/python/default/python. Is this a bug in configure, or is my configuration regarded as too perverse to support? Regards, Vinay Sajip

4 5