Mailman 3 August 2009 - Python-Dev

[RELEASED] Python 3.1.1
by Benjamin Peterson 16 Aug '09

16 Aug '09

On behalf of the Python development team, I'm happy to announce the first bugfix release of the Python 3.1 series, Python 3.1.1. This bug fix release fixes many normal bugs and several critical ones including potential data corruption in the io library. Python 3.1 focuses on the stabilization and optimization of the features and changes that Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. Other features include an ordered dictionary implementation, a condensed syntax for nested with statements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in the Python distribution. Please note the Windows and Mac binaries are not available yet but will be in the coming days. To download Python 3.1.1 visit: http://www.python.org/download/releases/3.1.1/ The 3.1 documentation can be found at: http://docs.python.org/3.1 Bugs can always be reported to: http://bugs.python.org Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 3.1.1's contributors)

1 0

Re: [Python-Dev] random number generator state
by Greg Ewing 16 Aug '09

16 Aug '09

Scott David Daniels wrote: > No, I don't really need MT. The others would be fine. > I'd love further details. The one I've been working with is due to Pierre L'Ecuyer [1] and is known as MRG32k3a. It's a combined multiple recursive linear congruential generator with 6 words of state. The formulas are r1[i] = (a12 * r1[i-2] + a13 * r1[i-3]) % m1 r2[i] = (a21 * r2[i-1] + a23 * r2[i-3]) % m2 r[i] = (r1[i] - r2[i]) * m1 where m1 = 2**32 - 209 m2 = 2**32 - 22835 a12 = 1403580 a13 = -810728 a21 = 527612 a23 = -1370589 If you consider the state to be made up of two 3-word state vectors, then there are two 3x3 matrices which map a given state onto the next state. So to jump ahead n steps in the sequence, you raise these matrices to the power of n. I've attached some code implementing this generator together with the jumping-ahead. (Sorry it's in C++, I hadn't discovered Python when I wrote it.) [1] Pierre L'Ecuyer, Good Parameters and Implementations for Combined Multiple Recursive Random Number Generators, Operations Research v47 no1 Jan-Feb 1999 http://www.iro.umontreal.ca/~lecuyer/myftp/papers/combmrg2.ps -- Greg /* * cmr_random_generator.C * ====================== * * Combined Multiple Recursive random number generator. * * This is an implementation of Pierre L'Ecuyer's * MRG32k3a generator, described in: * * Pierre L'Ecuyer, Good Parameters and Implementations for * Combined Multiple Recursive Random Number Generators, * Operations Research v47 no1 Jan-Feb 1999 */ #include "cmr_random_generator.H" static const double norm = 2.328306549295728e-10, m1 = 4294967087.0, m2 = 4294944443.0, a12 = 1403580.0, a13 = -810728.0, a21 = 527612.0, a23 = -1370589.0; static double a[2][3][3] = { { {0.0, 1.0, 0.0}, {0.0, 0.0, 1.0}, {a13, a12, 0.0} }, { {0.0, 1.0, 0.0}, {0.0, 0.0, 1.0}, {a23, 0.0, a21} } }; static double m[2] = { m1, m2 }; static double init_s[2][3] = { {1.0, 1.0, 1.0}, {1.0, 1.0, 1.0} }; inline static double mod(double x, double m) { long k = (long)(x / m); x -= k * m; if (x < 0.0) x += m; return x; } /* * Initialisation */ CMRRandomGenerator::CMRRandomGenerator() { for (int i = 0; i <= 1; i++) for (int j = 0; j <= 2; j++) s[i][j] = init_s[i][j]; } /* * Advance CMRG one step and return next number */ double CMRRandomGenerator::Next() { double p1 = mod(a12 * s[0][1] + a13 * s[0][0], m1); s[0][0] = s[0][1]; s[0][1] = s[0][2]; s[0][2] = p1; double p2 = mod(a21 * s[1][2] + a23 * s[1][0], m2); s[1][0] = s[1][1]; s[1][1] = s[1][2]; s[1][2] = p2; double p = p1 - p2; if (p < 0.0) p += m1; return (p + 1) * norm; } typedef unsigned long long Int64; typedef Int64 CMRG_Vector[3]; typedef Int64 CMRG_Matrix[3][3]; static Int64 ftoi(double x, double m) { if (x >= 0.0) return Int64(x); else return Int64((long double)x + (long double)m); } static double itof(Int64 i, Int64 m) { return i; } static void v_ftoi(double u[], CMRG_Vector v, double m) { for (int i = 0; i <= 2; i++) v[i] = ftoi(u[i], m); } static void v_itof(CMRG_Vector u, double v[], Int64 m) { for (int i = 0; i <= 2; i++) v[i] = itof(u[i], m); } static void v_copy(CMRG_Vector u, CMRG_Vector v) { for (int i = 0; i <= 2; i++) v[i] = u[i]; } static void m_ftoi(double a[][3], CMRG_Matrix b, double m) { for (int i = 0; i <= 2; i++) for (int j = 0; j <= 2; j++) b[i][j] = ftoi(a[i][j], m); } static void m_copy(CMRG_Matrix a, CMRG_Matrix b) { for (int i = 0; i <= 2; i++) for (int j = 0; j <= 2; j++) b[i][j] = a[i][j]; } static void mv_mul(CMRG_Matrix a, CMRG_Vector u, CMRG_Vector v, Int64 m) { CMRG_Vector w; int i, j; for (i = 0; i <= 2; i++) { w[i] = 0; for (j = 0; j <= 2; j++) w[i] = (a[i][j] * u[j] + w[i]) % m; } v_copy(w, v); } static void mm_mul(CMRG_Matrix a, CMRG_Matrix b, CMRG_Matrix c, Int64 m) { CMRG_Matrix d; int i, j, k; for (i = 0; i <= 2; i++) { for (j = 0; j <= 2; j++) { d[i][j] = 0; for (k = 0; k <= 2; k++) d[i][j] = (a[i][k] * b[k][j] + d[i][j]) % m; } } m_copy(d, c); } /* * Advance the CMRG by n*2^e steps */ void CMRRandomGenerator::Advance(unsigned long n, unsigned int e) { CMRG_Matrix B[2]; CMRG_Vector S[2]; Int64 M[2]; int i; for (i = 0; i <= 1; i++) { m_ftoi(a[i], B[i], m[i]); v_ftoi(s[i], S[i], m[i]); M[i] = Int64(m[i]); } while (e--) { for (i = 0; i <= 1; i++) mm_mul(B[i], B[i], B[i], M[i]); } while (n) { if (n & 1) for (i = 0; i <= 1; i++) mv_mul(B[i], S[i], S[i], M[i]); n >>= 1; if (n) for (i = 0; i <= 1; i++) mm_mul(B[i], B[i], B[i], M[i]); } for (i = 0; i <= 1; i++) v_itof(S[i], s[i], M[i]); } /* * cmr_random_generator.H * ====================== * * Combined Multiple Recursive random number generator. */ #ifndef cmr_random_generator_H #define cmr_random_generator_H class CMRRandomGenerator { public: CMRRandomGenerator(); double Next(); void Advance(unsigned long mant, unsigned int exp); protected: double s[2][3]; }; #endif

1 0

random number generator state
by Scott David Daniels 16 Aug '09

16 Aug '09

I find I have a need in randomized testing for a shorter version of getstate, even if it _is_ slower to restore. When running exhaustive tests, a failure report should show the start state of the generator. Unfortunately, our current state includes a 625-element array. I want a state that can be read off a report and typed in to reproduce the state. Something a bit like the initial seed, a count of cycle calls, and a few other things. So, in addition to .getstate() and .setstate(...), I'd at least need to have .get_slow_state() and possibly expand what .setstate(...) takes. However, a call to .setstate should reset the counter or all is for naught. That means I need to change the results of .getstate, thus giving me three kinds of input to .setstate: old, new-short, and new-long. In trying to get this to work, I found what might be a bug: code says mt[0] = 0x80000000UL; /* MSB is 1; assuring non-zero initial array */ but probably should be: mt[0] |= 0x80000000UL; /* MSB is 1; assuring non-zero initial array */ In checking into that issue, I went to the original Mersenne-Twister code, and I see the original authors are pursuing a newer generator, dSFMT. I now have a dilemma. Should I continue the work on the original M-T code (which is now seeming problematic for compatibility) or simply make a new generator with similar calls using dSFMT and put the new feature in that where there is no compatibility problem. Which would be more useful for the Python community? --Scott David Daniels Scott.Daniels(a)Acm.Org

4 4

Updating tests in branches
by Frank Wierzbicki 16 Aug '09

16 Aug '09

I plan on updating the Python unit tests with tests from Jython that turn out to be generic Python tests. Should I be putting these tests into trunk and 3k or should I also put them into the 2.6 and 3.1 maintenance branches as well? Regards, -Frank

2 2

Tweaking AST lineno and col_offset
by Frank Wierzbicki 14 Aug '09

14 Aug '09

Hi all, Off and on I have been directly comparing Jython's AST with Python's AST and generally working towards making them as close to identical as possible. There are a couple of places where I haven't "fixed" Jython because it looks to me like Jython has slightly better offsets. One example: for a,b in c: pass The Tuple node "a,b" ends up with a col_offset of 0 (the position of the "for") where Jython has the col_offset as 4 (the position of "a"). Jython's result is more consistent with other Tuple node col_offset results. I have a local patch that changes the CPython col_offset to match Jython's, but before I submit a patch I thought I'd ask here if there is support for this sort of change and if I should continue to find col_offset and lineno results that look fishy to me, or should I just change Jython's results to match (one way or another, things will be much easier for me to test if they match). Also, would this be a change that would be considered a backwards incompatibility? In other words, would patches like this be allowed in 2.6/3.1 or only in 2.7/3.2. Regards, -Frank

3 7

(try-except) conditional expression similar to (if-else) conditional (PEP 308)
by Jeff McAninch 14 Aug '09

14 Aug '09

I'm new to this list, so please excuse me if this topic has been discussed, but I didn't see anything similar in the archives. I very often want something like a try-except conditional expression similar to the if-else conditional. An example of the proposed syntax might be: x = float(string) except float('nan') or possibly x = float(string) except ValueError float('nan') Here's a simple example: Converting a large list of strings to floats where there may be errors that I want returned as nan's. Currently I would write the function: def safe_float_function(string): try: result = float(string) except: result = float('nan') return result and get my list of floats using the list comprehension: xs = [ safe_float_function(string) for string in strings ] With a try-except conditional I would instead define the following lambda: safe_float_conditional = lambda string : float(string) except float('nan') leading to: xs = [ safe_float_conditional(string) for string in strings ] My understanding is that the second would be faster at run time, and, like if-else conditional expressions, possibly more easily read by the human. Again, please excuse me if this has been discussed previously. If so, I'd appreciate being pointed to the discussion. Please also excuse me if for there is some currently (pre-python 3.0) idiom that I could use to efficiently get this same behaviour. If so, I'd appreciate being educated. Thanks, Jeff McAninch -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch(a)lanl.gov ==========================

17 27

expy: an expressway to extend Python
by Yingjie Lan 14 Aug '09

14 Aug '09

Hi All, This is to announce the initial release of expy 0.1.0. More details at http://expy.sourceforge.net/ Thanks! Yingjie

3 3

Summary of Python tracker Issues
by Python tracker 14 Aug '09

14 Aug '09

ACTIVITY SUMMARY (08/07/09 - 08/14/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2329 open (+26) / 16195 closed ( +8) / 18524 total (+34) Open issues with patches: 925 Average duration of open issues: 659 days. Median duration of open issues: 414 days. Open Issues Breakdown open 2297 (+26) pending 31 ( +0) Issues Created Or Reopened (34) _______________________________ TarFile.getmembers fails at struct.unpack: unpack requires a str 08/07/09 http://bugs.python.org/issue6669 created srid Printing the 'The Python Tutorial' 08/08/09 http://bugs.python.org/issue6670 created brimac webbrowser.py doesn't respect xfce default browser 08/09/09 http://bugs.python.org/issue6671 created ava1ar patch Add Mingw recognition to pyport.h to allow building extensions 08/09/09 http://bugs.python.org/issue6672 created f0k Py3.1 hangs in coroutine and eats up all memory 08/09/09 CLOSED http://bugs.python.org/issue6673 created scoder Fatal error: deallocating None 08/10/09 CLOSED http://bugs.python.org/issue6674 created shashi inf == inf (wrong IEEE 754 behaviour) 08/10/09 CLOSED http://bugs.python.org/issue6675 created Cyborg16 expat parser throws Memory Error when parsing multiple files 08/10/09 http://bugs.python.org/issue6676 created realpolitik Place the term "delete" within the documentation for os.remove() 08/10/09 http://bugs.python.org/issue6677 created mcow inspect.currentframe documentation omits optional depth paramete 08/10/09 http://bugs.python.org/issue6678 created llimllib obsolete paragraph in re doc for re.sub 08/10/09 CLOSED http://bugs.python.org/issue6679 created MLModel Python 3.1 fails to build when db.h contains non-UTF-8 character 08/10/09 CLOSED http://bugs.python.org/issue6680 created Arfrever patch email.parser clips trailing \n of multipart/mixed part if part e 08/10/09 http://bugs.python.org/issue6681 created gvanrossum Default traceback does not handle PEP302 loaded modules 08/11/09 http://bugs.python.org/issue6682 created anders.blomdell(a)control.lth.se smtplib authentication - try all mechanisms 08/11/09 http://bugs.python.org/issue6683 created shubes "x / 1" and "x * 1" should return x 08/11/09 CLOSED http://bugs.python.org/issue6684 created mrjbq7 CGI module documentation references method 'toupper'; should be 08/11/09 http://bugs.python.org/issue6685 created troy xml.sax.xmlreader.XMLReader.getProperty (xml.sax.handler.propert 08/11/09 http://bugs.python.org/issue6686 created cms103 patch Move the special-case for integer objects out of PyBytes_FromObj 08/11/09 http://bugs.python.org/issue6687 created alexandre.vassalotti patch, 26backport Optimize PyBytes_FromObject. 08/11/09 http://bugs.python.org/issue6688 created alexandre.vassalotti patch subprocess doesn't pass arguments correctly on Linux when shell= 08/12/09 http://bugs.python.org/issue6689 created davidfraser patch BUILD_SET followed by COMPARE_OP (in) can be optimized if all it 08/12/09 http://bugs.python.org/issue6690 created alex Support for nested classes and function for pyclbr 08/12/09 http://bugs.python.org/issue6691 created gpolo patch asyncore kqueue support 08/13/09 http://bugs.python.org/issue6692 created Ikinoki New functions in site.py to get user/global site packages paths 08/13/09 http://bugs.python.org/issue6693 created tarek patch itertools documentation still contains references to ifilterfals 08/13/09 CLOSED http://bugs.python.org/issue6694 created alex.morega PyXXX_ClearFreeList for dict, set, and list 08/13/09 http://bugs.python.org/issue6695 created matthiastroffaes patch Profile objects should be documented 08/13/09 http://bugs.python.org/issue6696 created honeyman Python 3.1 segfaults when invalid UTF-8 characters are passed fr 08/13/09 http://bugs.python.org/issue6697 created Arfrever IDLE no longer opens only an edit window when configured to do s 08/13/09 http://bugs.python.org/issue6698 created gpolo IDLE: Warn user about overwriting a file that has a newer versio 08/13/09 http://bugs.python.org/issue6699 created gpolo patch inspect.getsource() returns incorrect source lines 08/14/09 http://bugs.python.org/issue6700 created gagenellina patch Make custom xmlrpc extension easier 08/14/09 http://bugs.python.org/issue6701 created bogdan.opanchuk patch Tkinter: modify xview of entry widget 08/14/09 CLOSED http://bugs.python.org/issue6702 created paolo Issues Now Closed (22) ______________________ xview/yview of Tix.Grid is broken 705 days http://bugs.python.org/issue1135 gpolo patch IDLE - use enumerate instead of zip(count(), ...) 399 days http://bugs.python.org/issue3344 gpolo patch subprocess fails in select when descriptors are large 392 days http://bugs.python.org/issue3392 benjamin.peterson patch Idle doesn't obey the new improved warnings arguements 327 days http://bugs.python.org/issue3926 gpolo patch Idle hangs when given a nonexistent filename. 207 days http://bugs.python.org/issue4985 gpolo patch IDLE to support reindent.py 190 days http://bugs.python.org/issue5150 gpolo Python 3 pdb: shows internal code, breakpoints don't work 78 days http://bugs.python.org/issue6126 georg.brandl patch Missing Shell menu in Linux IDLE 67 days http://bugs.python.org/issue6168 gpolo Tkinter.Entry: fix for xview and some doc clarifications 73 days http://bugs.python.org/issue6180 gpolo patch test_ttk_guionly buildbot test crash: Tcl_FinalizeNotifier: noti 18 days http://bugs.python.org/issue6527 gpolo 2to3 fails to fix test.test_support 13 days http://bugs.python.org/issue6583 benjamin.peterson 2to3 test_print_function_option fails on Windows 14 days http://bugs.python.org/issue6599 benjamin.peterson patch Desire python.org documentation link to user contribution wiki ( 5 days http://bugs.python.org/issue6660 keenethery re.findall does not always return a list of strings 2 days http://bugs.python.org/issue6663 pitrou Py3.1 hangs in coroutine and eats up all memory 2 days http://bugs.python.org/issue6673 alexandre.vassalotti Fatal error: deallocating None 0 days http://bugs.python.org/issue6674 benjamin.peterson inf == inf (wrong IEEE 754 behaviour) 0 days http://bugs.python.org/issue6675 marketdickinson obsolete paragraph in re doc for re.sub 2 days http://bugs.python.org/issue6679 georg.brandl Python 3.1 fails to build when db.h contains non-UTF-8 character 2 days http://bugs.python.org/issue6680 benjamin.peterson patch "x / 1" and "x * 1" should return x 0 days http://bugs.python.org/issue6684 rhettinger itertools documentation still contains references to ifilterfals 0 days http://bugs.python.org/issue6694 georg.brandl Tkinter: modify xview of entry widget 0 days http://bugs.python.org/issue6702 gpolo Top Issues Most Discussed (10) ______________________________ 15 Regexp 2.7 (modifications to current re 2.2.2) 486 days open http://bugs.python.org/issue2636 5 Python 3 pdb: shows internal code, breakpoints don't work 78 days closed http://bugs.python.org/issue6126 5 urllib.quote() escapes characters unnecessarily and contrary to 486 days open http://bugs.python.org/issue2637 4 inf == inf (wrong IEEE 754 behaviour) 0 days closed http://bugs.python.org/issue6675 4 Py3.1 hangs in coroutine and eats up all memory 2 days closed http://bugs.python.org/issue6673 4 subprocess fails in select when descriptors are large 392 days closed http://bugs.python.org/issue3392 3 PyXXX_ClearFreeList for dict, set, and list 1 days open http://bugs.python.org/issue6695 3 subprocess doesn't pass arguments correctly on Linux when shell 2 days open http://bugs.python.org/issue6689 3 xml.sax.xmlreader.XMLReader.getProperty (xml.sax.handler.proper 3 days open http://bugs.python.org/issue6686 3 email.parser clips trailing \n of multipart/mixed part if part 4 days open http://bugs.python.org/issue6681

1 0

Tkinter: modify xview of entry widget
by paolo.fragu＠libero.it 14 Aug '09

14 Aug '09

Hi, I'm Paolo from Italy and I'm a python user. I wish to propose a useful and smart method modify in Tkinter Library: Previously to scroll this widget we had to write an external function (recalling xview_moveto and xview_scroll). With my method this operation is cleared and the same as all other widgets (just have to call xview). ---------------------------------------------------------- Modify Proposal: ---------------------------------------------------------- Change the method xview of entry so it works as all widget scrollable, and it's compatible with 'old' xview. So to scroll entry widget: entry_widget['xscrollcommand']=scroll_widget.set scroll_widget['command']=entry_widget.xview The change in module Tkinter is: def xview(self,*args): """Query and change horizontal position of the view.""" #modify if not args: return self._getdoubles(self.tk.call(self._w, 'xview')) #old code index=args[0] self.tk.call(self._w, 'xview', index) ---------------------------------------------------------- I wish that this implementation could be integrated in Tkinter, and I remain at disposal for any question or further information. Waiting for your response, Best regards Paolo Fraguglia

3 2

[RELEASED] Python 3.1.1 Release Candidate
by Benjamin Peterson 13 Aug '09

13 Aug '09

On behalf of the Python development team, I'm pleased to announce the first release candidate of Python 3.1.1. This bug fix release fixes many normal bugs and several critical ones including potential data corruption in the io library. The final version should be out within the next week. Python 3.1 focuses on the stabilization and optimization of the features and changes that Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. Other features include an ordered dictionary implementation, a condensed syntax for nested with statements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in the Python distribution. To download Python 3.1.1 visit: http://www.python.org/download/releases/3.1/ The 3.1 documentation can be found at: http://docs.python.org/3.1 Bugs can always be reported to: http://bugs.python.org Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 3.1.1's contributors)

1 0