Mailman 3 November 2012 - Python-Dev

Testing the tests by modifying the ordering of dict items.
by Mark Shannon Dec. 22, 2012

Dec. 22, 2012

Hi, Python code should not depend upon the ordering of items in a dict. Unfortunately it seems that a number of tests in the standard library do just that. Changing PyDict_MINSIZE from 8 to either 4 or 16 causes the following tests to fail: test_dis test_email test_inspect test_nntplib test_packaging test_plistlib test_pprint test_symtable test_trace test_sys also fails, but this is a legitimate failure in sys.getsizeof() Changing the collision resolution function from f(n) = 5n + 1 to f(n) = n + 1 results in the same failures, except for test_packaging and test_symtable which pass. Finally, changing the seed in unicode_hash() from (implicit) 0 to an arbitrary value (12345678) causes the above tests to fail plus: test_json test_set test_ttk_textonly test_urllib test_urlparse I think this is a real issue as the unicode_hash() function is likely to change soon due to http://bugs.python.org/issue13703. Should I: 1. Submit one big bug report? 2. Submit a bug report for each "failing" test separately? 3. Ignore it, since the tests only fail when I start messing about? Cheers, Mark.

4 4

Understanding the buffer API
by Jeff Allen Dec. 18, 2012

Dec. 18, 2012

I'm implementing the buffer API and some of memoryview for Jython. I have read with interest, and mostly understood, the discussion in Issue #10181 that led to the v3.3 re-implementation of memoryview and much-improved documentation of the buffer API. Although Jython is targeting v2.7 at the moment, and 1-D bytes (there's no Jython NumPy), I'd like to lay a solid foundation that benefits from the recent CPython work. I hope that some of the complexity in memoryview stems from legacy considerations I don't have to deal with in Jython. I am puzzled that PEP 3118 makes some specifications that seem unnecessary and complicate the implementation. Would those who know the API inside out answer a few questions? My understanding is this: When a consumer requests a buffer from the exporter it specifies using flags how it intends to navigate it. If the buffer actually needs more apparatus than the consumer proposes, this raises an exception. If the buffer needs less apparatus than the consumer proposes, the exporter has to supply what was asked for. For example, if the consumer sets PyBUF_STRIDES, and the buffer can only be navigated by using suboffsets (PIL-style) this raises an exception. Alternatively, if the consumer sets PyBUF_STRIDES, and the buffer is just a simple byte array, the exporter has to supply shape and strides arrays (with trivial values), since the consumer is going to use those arrays. Is there any harm is supplying shape and strides when they were not requested? The PEP says: "PyBUF_ND ... If this is not given then shape will be NULL". It doesn't stipulate that strides will be null if PyBUF_STRIDES is not given, but the library documentation says so. suboffsets is different since even when requested, it will be null if not needed. Similar, but simpler, the PEP says "PyBUF_FORMAT ... If format is not explicitly requested then the format must be returned as NULL (which means "B", or unsigned bytes)". What would be the harm in returning "B"? One place where this really matters is in the implementation of memoryview. PyMemoryView requests a buffer with the flags PyBUF_FULL_RO, so even a simple byte buffer export will come with shape, strides and format. A consumer (of the memoryview's buffer API) might specify PyBUF_SIMPLE: according to the PEP I can't simply give it the original buffer since required fields (that the consumer will presumably not access) are not NULL. In practice, I'd like to: what could possibly go wrong? Jeff Allen

4 13

Summary of Python tracker Issues
by Python tracker Dec. 3, 2012

Dec. 3, 2012

ACTIVITY SUMMARY (2012-11-23 - 2012-11-30) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3792 ( -7) closed 24566 (+48) total 28358 (+41) Open issues with patches: 1653 Issues opened (27) ================== #8824: Improve documentation of exec http://bugs.python.org/issue8824 reopened by mark.dickinson #9400: multiprocessing.pool.AsyncResult.get() messes up exceptions http://bugs.python.org/issue9400 reopened by sbt #15587: IDLE is pixelated on the Macbook Pro with Retina Display http://bugs.python.org/issue15587 reopened by Tyler.Crompton #16541: tk_setPalette doesn't accept keyword parameters http://bugs.python.org/issue16541 opened by HJarausch #16543: improve TypeError messages for missing arguments (meta issue) http://bugs.python.org/issue16543 opened by ezio.melotti #16544: Add external link to ast docs http://bugs.python.org/issue16544 opened by asvetlov #16547: IDLE raises an exception in tkinter after fresh file's text ha http://bugs.python.org/issue16547 opened by ebfe #16550: pickletools.py treats 32bit lengths as signed, but pickle.py a http://bugs.python.org/issue16550 opened by serhiy.storchaka #16551: Cleanup the pure Python pickle implementation http://bugs.python.org/issue16551 opened by serhiy.storchaka #16554: The description of the argument of MAKE_FUNCTION and MAKE_CLOS http://bugs.python.org/issue16554 opened by daniel.urban #16555: Add es_cu to locale aliases http://bugs.python.org/issue16555 opened by Leiser.Fernández.Gallo #16557: PEP 380 isn't reflected in the Functional Programming HOWTO http://bugs.python.org/issue16557 opened by msmhrt #16561: Windows installer doesn't use UAC, then crashes http://bugs.python.org/issue16561 opened by Redoute #16562: Optimize dict equality test http://bugs.python.org/issue16562 opened by rhettinger #16564: email.generator.BytesGenerator fails with bytes payload http://bugs.python.org/issue16564 opened by Alexander.Kruppa #16565: Increase Py_AddPendingCall array size http://bugs.python.org/issue16565 opened by felipecruz #16566: Structure._anonymous_ should not allow strings http://bugs.python.org/issue16566 opened by techtonik #16568: allow constructors to be documented separately from class http://bugs.python.org/issue16568 opened by chris.jerdonek #16569: Preventing errors of simultaneous access in zipfile http://bugs.python.org/issue16569 opened by serhiy.storchaka #16572: Bad multi-inheritance support in some libs like threading or m http://bugs.python.org/issue16572 opened by thomas.chiroux #16574: clarify policy on updates to final peps http://bugs.python.org/issue16574 opened by chris.jerdonek #16575: ctypes: unions as arguments http://bugs.python.org/issue16575 opened by arigo #16576: ctypes: structure with bitfields as argument http://bugs.python.org/issue16576 opened by arigo #16577: Suspect test.test_codeccallbacks.test_mutatingdecodehandler http://bugs.python.org/issue16577 opened by amaury.forgeotdarc #16579: .pyw disturb multiprocessing behavior http://bugs.python.org/issue16579 opened by Alex.stein #16580: Add examples to int.to_bytres and int.from_bytes http://bugs.python.org/issue16580 opened by paddy3118 #16581: define "PEP editor" in PEP 1 http://bugs.python.org/issue16581 opened by chris.jerdonek Most recent 15 issues with no replies (15) ========================================== #16581: define "PEP editor" in PEP 1 http://bugs.python.org/issue16581 #16580: Add examples to int.to_bytres and int.from_bytes http://bugs.python.org/issue16580 #16575: ctypes: unions as arguments http://bugs.python.org/issue16575 #16561: Windows installer doesn't use UAC, then crashes http://bugs.python.org/issue16561 #16557: PEP 380 isn't reflected in the Functional Programming HOWTO http://bugs.python.org/issue16557 #16551: Cleanup the pure Python pickle implementation http://bugs.python.org/issue16551 #16550: pickletools.py treats 32bit lengths as signed, but pickle.py a http://bugs.python.org/issue16550 #16516: argparse types (and actions) must be hashable http://bugs.python.org/issue16516 #16509: sqlite3 docs do not explain check_same_thread http://bugs.python.org/issue16509 #16494: Add a method on importlib.SourceLoader for creating bytecode f http://bugs.python.org/issue16494 #16492: Add a load_parents argument to importlib.find_loader() http://bugs.python.org/issue16492 #16486: Add context manager support to aifc module http://bugs.python.org/issue16486 #16463: test_timeout failure on the RHEL buildbot http://bugs.python.org/issue16463 #16450: test_missing_localfile masks problems in urlopen http://bugs.python.org/issue16450 #16429: Emit SyntaxWarning for code that risks UnboundLocalError http://bugs.python.org/issue16429 Most recent 15 issues waiting for review (15) ============================================= #16572: Bad multi-inheritance support in some libs like threading or m http://bugs.python.org/issue16572 #16569: Preventing errors of simultaneous access in zipfile http://bugs.python.org/issue16569 #16562: Optimize dict equality test http://bugs.python.org/issue16562 #16554: The description of the argument of MAKE_FUNCTION and MAKE_CLOS http://bugs.python.org/issue16554 #16551: Cleanup the pure Python pickle implementation http://bugs.python.org/issue16551 #16550: pickletools.py treats 32bit lengths as signed, but pickle.py a http://bugs.python.org/issue16550 #16543: improve TypeError messages for missing arguments (meta issue) http://bugs.python.org/issue16543 #16537: Python’s setup.py raises a ValueError when self.extensions i http://bugs.python.org/issue16537 #16525: wave file module does not support 32bit float format http://bugs.python.org/issue16525 #16523: attrgetter and itemgetter signatures in docs need cleanup http://bugs.python.org/issue16523 #16515: TypeError message incorrect for max() with one keyword arg http://bugs.python.org/issue16515 #16512: imghdr doesn't support jpegs with an ICC profile http://bugs.python.org/issue16512 #16511: IDLE configuration file: blank height and width fields trip up http://bugs.python.org/issue16511 #16510: Using appropriate checks in tests http://bugs.python.org/issue16510 #16507: Patch selectmodule.c to support WSAPoll on Windows http://bugs.python.org/issue16507 Top 10 most discussed issues (10) ================================= #16543: improve TypeError messages for missing arguments (meta issue) http://bugs.python.org/issue16543 21 msgs #16518: add "buffer protocol" to glossary http://bugs.python.org/issue16518 15 msgs #16574: clarify policy on updates to final peps http://bugs.python.org/issue16574 15 msgs #4591: 32-bits unsigned user/group identifier http://bugs.python.org/issue4591 11 msgs #4945: json checks True/False by identity, not boolean value http://bugs.python.org/issue4945 11 msgs #16547: IDLE raises an exception in tkinter after fresh file's text ha http://bugs.python.org/issue16547 9 msgs #7976: warnings should provide a public API for accessing its option http://bugs.python.org/issue7976 8 msgs #16565: Increase Py_AddPendingCall array size http://bugs.python.org/issue16565 8 msgs #16566: Structure._anonymous_ should not allow strings http://bugs.python.org/issue16566 8 msgs #11175: allow argparse FileType to accept encoding and errors argument http://bugs.python.org/issue11175 6 msgs Issues closed (45) ================== #1827: svnversion_init() doesn't support svn urls in sandbox/trunk http://bugs.python.org/issue1827 closed by christian.heimes #1977: Python reinitialization test http://bugs.python.org/issue1977 closed by christian.heimes #2039: Pymalloc patch for int and float objects http://bugs.python.org/issue2039 closed by christian.heimes #3410: platform.version() don't work as expected in Vista in portugue http://bugs.python.org/issue3410 closed by ezio.melotti #4473: POP3 missing support for starttls http://bugs.python.org/issue4473 closed by pitrou #9011: ast_for_factor unary minus optimization changes AST http://bugs.python.org/issue9011 closed by mark.dickinson #9176: module termios doesn't build on HP-UX http://bugs.python.org/issue9176 closed by skrah #10259: Entry text not set if all of 'Font', 'Foreground' and 'Justify http://bugs.python.org/issue10259 closed by gpolo #11076: Iterable argparse Namespace http://bugs.python.org/issue11076 closed by asvetlov #12848: pickle.py treats 32bit lengths as signed, but _pickle.c as uns http://bugs.python.org/issue12848 closed by pitrou #14478: Decimal hashing very slow, could be cached http://bugs.python.org/issue14478 closed by mark.dickinson #15990: solidify argument/parameter terminology http://bugs.python.org/issue15990 closed by chris.jerdonek #16205: update :class:`str` references to link to the str type section http://bugs.python.org/issue16205 closed by chris.jerdonek #16209: add a "class str" entry to the docs http://bugs.python.org/issue16209 closed by chris.jerdonek #16323: Wrong C API documentation for locale encoding http://bugs.python.org/issue16323 closed by asvetlov #16333: Trailing whitespace in json dump when using indent http://bugs.python.org/issue16333 closed by ezio.melotti #16339: Document "exec(stmt, global_dict, local_dict)" form in Python http://bugs.python.org/issue16339 closed by mark.dickinson #16423: urllib data URL http://bugs.python.org/issue16423 closed by pitrou #16464: urllib.request: opener not resetting content-length http://bugs.python.org/issue16464 closed by asvetlov #16477: tarfile fails to close file handles in case of exception http://bugs.python.org/issue16477 closed by asvetlov #16483: Make int(float('inf')) raise ValueError rather than OverflowEr http://bugs.python.org/issue16483 closed by mark.dickinson #16519: site.venv() should use abspath(executable) http://bugs.python.org/issue16519 closed by python-dev #16521: logging.basicConfig creates empty file when using handlers http://bugs.python.org/issue16521 closed by python-dev #16524: File access not always working with Python for Windows 32 bits http://bugs.python.org/issue16524 closed by terry.reedy #16530: documentation of os.wait3 http://bugs.python.org/issue16530 closed by ezio.melotti #16532: AMD64 Windows 7 build failures http://bugs.python.org/issue16532 closed by pitrou #16534: -Olimit: unsupported option: warnings and ./configure failures http://bugs.python.org/issue16534 closed by skrah #16540: Make itertools count, cycle, and repeat objects subscriptable http://bugs.python.org/issue16540 closed by rhettinger #16542: http//bugs.python/joko.suwito http://bugs.python.org/issue16542 closed by ezio.melotti #16545: ast.FunctionDef sets a bad value for kw_defaults when keyword- http://bugs.python.org/issue16545 closed by brett.cannon #16546: ast.YieldFrom needlessly has its expr value as optional http://bugs.python.org/issue16546 closed by mark.dickinson #16548: os.system won't run any command and there is no error message http://bugs.python.org/issue16548 closed by r.david.murray #16549: regression: -m json.tool module is broken http://bugs.python.org/issue16549 closed by ezio.melotti #16552: os.path.basename() docs should link to os.path.split() http://bugs.python.org/issue16552 closed by chris.jerdonek #16553: named kwd form of OrderedDict ctor yield random key order http://bugs.python.org/issue16553 closed by benjamin.peterson #16556: Update string.Formatter.vformat documentation to say "**kwargs http://bugs.python.org/issue16556 closed by ezio.melotti #16558: multiprocessing fails to raise exception with parameters http://bugs.python.org/issue16558 closed by sbt #16559: Update JSON tests http://bugs.python.org/issue16559 closed by serhiy.storchaka #16560: Python sighandlers delayed for no reason http://bugs.python.org/issue16560 closed by zdenek.pavlas #16563: re.match loops forever on simple regexp http://bugs.python.org/issue16563 closed by mark.dickinson #16567: Implementing .= for variable operator http://bugs.python.org/issue16567 closed by amaury.forgeotdarc #16570: Absolute imports fail to take full path into account? http://bugs.python.org/issue16570 closed by brett.cannon #16571: Iterating over inconsistently-indented code block causes execu http://bugs.python.org/issue16571 closed by mark.dickinson #16573: 2to3 should treat enumerate like sorted for zip, map, filter, http://bugs.python.org/issue16573 closed by python-dev #16578: Regular expressions with empty named groups break isname check http://bugs.python.org/issue16578 closed by Gabriel.Rodríguez.Alberich

6 6

type vs. class terminology
by Chris Jerdonek Nov. 30, 2012

Nov. 30, 2012

I would like to know when we should use "class" in the Python 3 documentation, and when we should use "type." Are these terms synonymous in Python 3, and do we have a preference for which to use and when? I'm sure this has been discussed before. But if this terminology issue has already been resolved, the resolution doesn't seem to be reflected in the docs. For example, the glossary entries for type and class don't reference each other. Thoughts? --Chris

7 6

Socket timeout and completion based sockets
by Kristján Valur Jónsson Nov. 28, 2012

Nov. 28, 2012

Regarding the recent discussion on python-ideas about asyncronous IO, I'd like to ask a question about python socket's Timeout feature. Specifically this: Is it a documented or a guaranteed feature that a send/receive operation that times out with a socket.timeout error is re-startable on that socket? The reason I ask is that depending on the implementation, a timeout may leave a socket in an undefined state. As it is implemented in the standard cPython implementation, the timeout feature is done with an internal select() call. Thus, if the select() call times out, socket send/receive is not even called, so a retry is possible without issue. However, other implementations of python sockets, e.g. ones that rely on IO completion, may not have the luxury of using select. For example, on Windows, there is no way to abort an IOCP socket call, so a timeout must be implemented by aborting the wait. Dealing with the resulting race can be an interesting challenge. K

5 11

Setting project home path the best way
by Christian Tismer Nov. 28, 2012

Nov. 28, 2012

Hi friends, I have a project that has its root somewhere on my machine. This project has many folders and contains quite some modules. There is a common root of the module tree, and I want to use - either absolute imports - relative imports with '.' Problem: - I want to run any module inside the heirarchy from the command-line - this should work, regardless what my 'cwd' is - this should work with or without virtualenv. So far, things work fine with virtualenv, because sys.executable is in the project module tree. Without virtualenv, this is not so. But I hate to make settings like PYTHONPATH, because these are not permanent. . Question: How should I define my project root dir in a unique way, without setting an environment variable? What is the lest intrusive way to spell that? Reason: I'd like to make things work correctly and unambigously when I call a script inside the module heirarchy. Things are not fixed: there exist many checkouts In the file system, and each should know where to search its home/root in the tree. Is this elegantly possible to deduce from the actually executed script file? Cheers - chris Sent from my Ei4Steve

9 32

Handling support for newer OS features at run time
by Trent Nelson Nov. 28, 2012

Nov. 28, 2012

The hackiest part of my WSAPoll patch is this: --- a/PC/pyconfig.h Sun Nov 18 10:42:42 2012 +0000 +++ b/PC/pyconfig.h Sun Nov 18 17:27:29 2012 -0500 @@ -158,12 +158,12 @@ /* set the version macros for the windows headers */ #ifdef MS_WINX64 /* 64 bit only runs on XP or greater */ -#define Py_WINVER 0x0501 /* _WIN32_WINNT_WINXP */ -#define Py_NTDDI NTDDI_WINXP +#define Py_WINVER 0x0600 /* _WIN32_WINNT_WINXP */ +#define Py_NTDDI NTDDI_WIN6 #else /* Python 2.6+ requires Windows 2000 or greater */ -#define Py_WINVER 0x0500 /* _WIN32_WINNT_WIN2K */ -#define Py_NTDDI NTDDI_WIN2KSP4 +#define Py_WINVER 0x0600 /* _WIN32_WINNT_WIN2K */ +#define Py_NTDDI NTDDI_WIN6 #endif Basically, because of the way our build is currently configured, I had to bump the minimum supported Windows version from XP to Vista just to get access to the WSAPoll headers. (Issue: http://bugs.python.org/issue16507 Patch: http://bugs.python.org/file28038/wsapoll.patch) Ideally, a Windows binary should make WSAPoll/select.poll() available if running on Vista or above, without impacting the ability to run on XP. I don't think we've currently got the ability to do this, right? Is there a precedent set anywhere else? I suspect it's not as much of an issue on *NIX platforms as you'll typically compile from source. Windows, not so much. Thoughts? A single binary that dynamically loads applicable modules seems cleaner but will obviously take more work. Other approach would be to start distributing multiple versions of Python based on the underlying Windows version. Not the nicest approach. Trent.

4 6

New Contributor Experience in Python and other FOSS Communities - A Survey
by Brian Curtin Nov. 27, 2012

Nov. 27, 2012

Hi all, Along with a number of other free and open communities, Python is being included in a survey of new contributors since January 2010. The survey is being done by Kevin Carillo, a PhD student at Victoria University of Wellington who is studying various approaches that projects use to gain and work with new contributors. If you have written patches, reviewed issues, or done anything to contribute to the development of Python and you started this after January 2010, I hope you can spare around 20 minutes to complete this survey: https://limesurvey.sim.vuw.ac.nz/index.php?sid=65151&lang=en There's a longer blog post up at http://blog.python.org/2012/11/new-contributor-experience-in-python.html if you would like a bit more information. On behalf of Kevin Carillo, I thank you for your time and consideration of this survey. Brian Curtin

1 1

Tru64 support
by Andrew Svetlov Nov. 27, 2012

Nov. 27, 2012

Unittests for subprocess module has weird comment: We cannot use os.path.realpath to canonicalize the path, since it doesn't expand Tru64 {memb} strings. See bug 1063571. http://hg.python.org/cpython/file/82ae284cd5f1/Lib/test/test_subprocess.py#… is link to comment, http://bugs.python.org/issue1063571 — issue in bugtracker. Is it still up to date? Bug has been created in 2004. I don't see Tru64 in list of available buildbots. Do we need to care about this platform? And how to make sure what existing code works fine for that? Or maybe we can drop support for Tru64 as well as we already does it for, say, VMS? -- Thanks, Andrew Svetlov

10 20

logging and rotation
by Matthias Bernt Nov. 25, 2012

Nov. 25, 2012

Dear mailing list, I'm using the logging module and write my log messages via the FileHandler. I just realized that using an external log rotation mechanism does not work. That is, new messages are not added to the file after rotation. In my opinion external log rotate mechanisms should work with the standard file handler. websearch pointed me to TimedRotatingFileHandler . But this only seems like a workaround. For instance I would like to get my log file mailed once a week. But it seems difficult to sync the cron job doing this and the logging-rotation mechanism. I think TimedRotatingFileHandler is important, e.g. for windows users. But for *nux users this seems to be unnecessary. Just want to gather some opinions before filing a bug. Regards, Matthias

4 3