Mailman 3 March 2013 - Python-Dev

Idle, site.py, and the release candidates
by Terry Jan Reedy 31 Mar '13

31 Mar '13

While trying to test the patch for http://bugs.python.org/issue5492 on Windows, I discovered that quit() and exit() in the Idle Shell are now disabled, it seems, for all versions on all systems rather than just sometimes on Linux. The problem is a change in idlelib that invalidated an assumption made in site.py. Revs 81718-81721 for http://bugs.python.org/issue9290 changed idlelib.PyShell.PseudoFile (line 1277 in 3.3) to subclass io.TextIOBase, which subclasses IOBase. This gave PseudoFile and its subclasses a .fileno instance method attribute that raises io.UnsupportedOperation: fileno. This is not a bug since the doc for io.IOBase.fileno says: "Return the underlying file descriptor (an integer) of the stream if it exists. An OSError is raised if the IO object does not use a file descriptor." (the particular error raised is not an issue here). This is the code for Quitter.__call__ in site.py (line 368 in 3.3): def __call__(self, code=None): # Shells like IDLE catch the SystemExit, but listen when # stdin wrapper is closed. try: fd = -1 if hasattr(sys.stdin, "fileno"): fd = sys.stdin.fileno() if fd != 0: # Don't close stdin if it wraps fd 0 sys.stdin.close() except: pass raise SystemExit(code) The incorrect assumption is that if sys.stdin.fileno exits but raises, the call did not come from a shell that needs .close called. I do not know enough about other circumstances in which stdin.fileno would do something other than return 0 to be sure of what the proper fix would be. (I increasingly dislike bare excepts as they hide the thinking and knowledge of the original programmer. What exception was expected that should be passed away?) Given that the callable constants exit and quit and are optionally suppressed on startup and are not standard builtins http://docs.python.org/3/library/constants.html#constants-added-by-the-site… it would be alright with me to ignore this regression and release as scheduled. But I though people should be aware of it. -- Terry Jan Reedy

4 6

py2.7: dictobject not properly resizing
by Micha Gorelick 30 Mar '13

30 Mar '13

I was taking a look at dictobject.c and realized that the logic controlling whether a resizedict will occur in dict_set_item_by_hash_or_entry disallows for the shrinking of a dictionary. This is contrary to what the comments directly above say: (http://hg.python.org/cpython/file/f3032825f637/Objects/dictobject.c#l771) 771 /* If we added a key, we can safely resize. Otherwise just return! 772 * If fill >= 2/3 size, adjust size. Normally, this doubles or 773 * quaduples the size, but it's also possible for the dict to shrink 774 * (if ma_fill is much larger than ma_used, meaning a lot of dict 775 * keys have been * deleted). The "bug" occures in the following conditional since we exit out of the function without checking the relative magnitudes of ma_filled to ma_used. Instead, we only check if we still have a correct loading factor (and the "don't resize on modification" bit). This can be fixed by changing the following conditional on line 785 to: if (mp->ma_used <= n_used || (mp->ma_fill*3 < (mp->ma_mask+1)*2 && mp->ma_used*5 > mp->ma_fill)) The factor of 5 was chosen arbitrarily... I'm sure with some benchmarking we could tune it to an optimal value for the internal use of dictionaries. However, before I put this effort in I was wondering if this is in fact desired behavior or if it is indeed a bug. At the very least, the comments should be updated to reflect the actual resizing dynamics of the dictionary. Micha ----------------------------- http://micha.gd/ http://github.com/mynameisfiber/

3 2

Accepting PEP 434, Idle Enhancement Exception
by Nick Coghlan 30 Mar '13

30 Mar '13

I am accepting Todd Rovito's and Terry Reedy's PEP 434, officially declaring IDLE to be an application bundled with Python, with the contents of "Lib/idlelib" exempt from the usual "no new features in maintenance releases" rule. As stated in the PEP, this isn't carte blanche to do major rewrites in maintenance releases, merely acknowledgement that, when in doubt, we better serve our users by treating IDLE as a bundled application and making it behave consistently across all supported versions than we do by treating it as a library first and an application second. Hopefully this clarification, and the stated goal of supporting IDLE as a high quality cross-platform default starting point for new Python users that aren't already accustomed to the command line and editing text files directly, will make it easier for the IDLE developers to focus on making IDLE excel at that task. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 9

Re: [Python-Dev] cpython (2.7): Fix typos and clear up one very odd bit of wording as pointed out by
by Antoine Pitrou 30 Mar '13

30 Mar '13

On Sat, 30 Mar 2013 09:39:00 +0100 (CET) gregory.p.smith <python-checkins(a)python.org> wrote: > > The workarounds and modifications are not included in patch releases as they > break backward compatibility. This is not entirely true. It was perfectly possible to give the *options* to change behaviour, but the patch wasn't ready. Regards Antoine.

1 0

Writing importers and path hooks
by Paul Moore 29 Mar '13

29 Mar '13

On 27 March 2013 21:19, Bradley M. Froehle <brad.froehle(a)gmail.com> wrote: > I implemented just such a path hook ---- zipimporter plus the magic required > for C extensions --- as a challenge to myself to learn more about the Python > import mechanisms. > > See https://github.com/bfroehle/pydzipimport. Apologies for hijacking the thread, but it's interesting that you implemented your hook like this. I notice that you didn't use any of the importlib functionality in doing so. Was there a particular reason? I ask because a few days ago, I was writing a very similar importer, as I wanted to try a proof of concept importer based on the new importlib stuff (which is intended to make writing custom importers easier), and I really struggled to get something working. It seems to me that the importlib documentation doesn't help much for people trying to import path hooks. But it might be just me. Does anyone have an example of a simple importlib-based finder/loader? That would be a huge help for me. In return for any pointers, I'll look at putting together a doc patch to clarify how to use importlib to build your own path hooks :-) Thanks, Paul

3 6

Summary of Python tracker Issues
by Python tracker 29 Mar '13

29 Mar '13

ACTIVITY SUMMARY (2013-03-22 - 2013-03-29) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3887 (-21) closed 25461 (+72) total 29348 (+51) Open issues with patches: 1724 Issues opened (46) ================== #6671: webbrowser doesn't respect xfce default browser http://bugs.python.org/issue6671 reopened by eric.araujo #17295: __slots__ on PyVarObject subclass http://bugs.python.org/issue17295 reopened by haypo #17523: Additional tests for the os module. http://bugs.python.org/issue17523 opened by willweaver #17525: os.getcwd() fails on cifs share http://bugs.python.org/issue17525 opened by dcuddihy #17526: inspect.findsource raises undocumented error for code objects http://bugs.python.org/issue17526 opened by Nils.Bruin #17527: PATCH as valid request method in wsgiref.validator http://bugs.python.org/issue17527 opened by lsbardel #17528: Implement dumps/loads for lru_cache http://bugs.python.org/issue17528 opened by frafra #17529: fix os.sendfile() documentation regarding the type of file des http://bugs.python.org/issue17529 opened by neologix #17530: pprint could use line continuation for long bytes literals http://bugs.python.org/issue17530 opened by pitrou #17532: IDLE: Always include "Options" menu on MacOSX http://bugs.python.org/issue17532 opened by roger.serwy #17533: test_xpickle fails with "cannot import name precisionbigmemtes http://bugs.python.org/issue17533 opened by ned.deily #17534: unittest keeps references to test cases alive http://bugs.python.org/issue17534 opened by ezio.melotti #17535: IDLE: Add an option to show line numbers along the left side o http://bugs.python.org/issue17535 opened by Todd.Rovito #17536: update browser list with additional browser names http://bugs.python.org/issue17536 opened by doko #17537: csv.DictReader should fail if >1 column has the same name http://bugs.python.org/issue17537 opened by doko #17538: Document XML Vulnerabilties http://bugs.python.org/issue17538 opened by dstufft #17539: Use the builtins module in the unittest.mock.patch example http://bugs.python.org/issue17539 opened by berker.peksag #17540: logging formatter support 'style' key in dictionary config http://bugs.python.org/issue17540 opened by monson #17544: regex code re-raises exceptions on success http://bugs.python.org/issue17544 opened by Zdeněk.Pavlas #17545: os.listdir and os.path.join inconsistent on empty path http://bugs.python.org/issue17545 opened by babou #17546: Document the circumstances where the locals() dict gets update http://bugs.python.org/issue17546 opened by techtonik #17547: "checking whether gcc supports ParseTuple __format__... " erro http://bugs.python.org/issue17547 opened by dmalcolm #17548: unittest.mock: test_create_autospec_unbound_methods is skipped http://bugs.python.org/issue17548 opened by haypo #17549: Some exceptions not highlighted in exceptions documentation. http://bugs.python.org/issue17549 opened by Ramchandra Apte #17551: Windows - accessing drive with nothing mounted forces user int http://bugs.python.org/issue17551 opened by bobjalex #17552: socket.sendfile() http://bugs.python.org/issue17552 opened by giampaolo.rodola #17553: Note that distutils’ bdist_rpm command is not used to build http://bugs.python.org/issue17553 opened by Sean.Carolan #17554: Compact output for regrtest http://bugs.python.org/issue17554 opened by ezio.melotti #17555: Creating new processes after importing multiprocessing.manager http://bugs.python.org/issue17555 opened by Marc.Brünink #17557: test_getgroups of test_posix can fail on OS X 10.8 if more tha http://bugs.python.org/issue17557 opened by ned.deily #17558: gdb debugging python frames in optimised interpreters http://bugs.python.org/issue17558 opened by mcobden #17560: problem using multiprocessing with really big objects? http://bugs.python.org/issue17560 opened by mrjbq7 #17561: Add socket.create_server_sock() convenience function http://bugs.python.org/issue17561 opened by giampaolo.rodola #17563: Excessive resizing of dicts when used as a cache http://bugs.python.org/issue17563 opened by Mark.Shannon #17564: test_urllib2_localnet fails http://bugs.python.org/issue17564 opened by Mark.Shannon #17565: segfaults during serialization http://bugs.python.org/issue17565 opened by eddiewrc #17566: Document that importlib.abc.Loader.module_repr is abstract and http://bugs.python.org/issue17566 opened by brett.cannon #17567: Clarify importlib.abc.PathEntryFinder.find_loader() docs http://bugs.python.org/issue17567 opened by brett.cannon #17568: re: Infinite loop with repeated empty alternative http://bugs.python.org/issue17568 opened by ericp #17569: urllib2 urlopen truncates https pages after 32768 characters http://bugs.python.org/issue17569 opened by jhp7e #17570: Improve devguide Windows instructions http://bugs.python.org/issue17570 opened by ezio.melotti #17571: broken links on Lib/datetime.py docstring http://bugs.python.org/issue17571 opened by tshepang #17572: strptime exception context http://bugs.python.org/issue17572 opened by Claudiu.Popa #17573: add ElementTree XML processing benchmark to benchmark suite http://bugs.python.org/issue17573 opened by scoder #17574: pysetup failing with "OSError: [Errno 18] Invalid cross-device http://bugs.python.org/issue17574 opened by vaab #17575: HTTPConnection.send http://bugs.python.org/issue17575 opened by dspublic(a)freemail.hu Most recent 15 issues with no replies (15) ========================================== #17575: HTTPConnection.send http://bugs.python.org/issue17575 #17573: add ElementTree XML processing benchmark to benchmark suite http://bugs.python.org/issue17573 #17572: strptime exception context http://bugs.python.org/issue17572 #17571: broken links on Lib/datetime.py docstring http://bugs.python.org/issue17571 #17570: Improve devguide Windows instructions http://bugs.python.org/issue17570 #17568: re: Infinite loop with repeated empty alternative http://bugs.python.org/issue17568 #17567: Clarify importlib.abc.PathEntryFinder.find_loader() docs http://bugs.python.org/issue17567 #17566: Document that importlib.abc.Loader.module_repr is abstract and http://bugs.python.org/issue17566 #17563: Excessive resizing of dicts when used as a cache http://bugs.python.org/issue17563 #17558: gdb debugging python frames in optimised interpreters http://bugs.python.org/issue17558 #17557: test_getgroups of test_posix can fail on OS X 10.8 if more tha http://bugs.python.org/issue17557 #17547: "checking whether gcc supports ParseTuple __format__... " erro http://bugs.python.org/issue17547 #17540: logging formatter support 'style' key in dictionary config http://bugs.python.org/issue17540 #17539: Use the builtins module in the unittest.mock.patch example http://bugs.python.org/issue17539 #17532: IDLE: Always include "Options" menu on MacOSX http://bugs.python.org/issue17532 Most recent 15 issues waiting for review (15) ============================================= #17573: add ElementTree XML processing benchmark to benchmark suite http://bugs.python.org/issue17573 #17572: strptime exception context http://bugs.python.org/issue17572 #17563: Excessive resizing of dicts when used as a cache http://bugs.python.org/issue17563 #17561: Add socket.create_server_sock() convenience function http://bugs.python.org/issue17561 #17555: Creating new processes after importing multiprocessing.manager http://bugs.python.org/issue17555 #17554: Compact output for regrtest http://bugs.python.org/issue17554 #17552: socket.sendfile() http://bugs.python.org/issue17552 #17547: "checking whether gcc supports ParseTuple __format__... " erro http://bugs.python.org/issue17547 #17540: logging formatter support 'style' key in dictionary config http://bugs.python.org/issue17540 #17539: Use the builtins module in the unittest.mock.patch example http://bugs.python.org/issue17539 #17538: Document XML Vulnerabilties http://bugs.python.org/issue17538 #17536: update browser list with additional browser names http://bugs.python.org/issue17536 #17535: IDLE: Add an option to show line numbers along the left side o http://bugs.python.org/issue17535 #17529: fix os.sendfile() documentation regarding the type of file des http://bugs.python.org/issue17529 #17527: PATCH as valid request method in wsgiref.validator http://bugs.python.org/issue17527 Top 10 most discussed issues (10) ================================= #17546: Document the circumstances where the locals() dict gets update http://bugs.python.org/issue17546 24 msgs #17561: Add socket.create_server_sock() convenience function http://bugs.python.org/issue17561 14 msgs #17538: Document XML Vulnerabilties http://bugs.python.org/issue17538 13 msgs #17560: problem using multiprocessing with really big objects? http://bugs.python.org/issue17560 13 msgs #17554: Compact output for regrtest http://bugs.python.org/issue17554 9 msgs #17536: update browser list with additional browser names http://bugs.python.org/issue17536 8 msgs #17522: Add api PyGILState_Check http://bugs.python.org/issue17522 7 msgs #17564: test_urllib2_localnet fails http://bugs.python.org/issue17564 7 msgs #2704: IDLE: Patch to make PyShell behave more like a Terminal interf http://bugs.python.org/issue2704 6 msgs #7511: msvc9compiler.py: ValueError when trying to compile with VC Ex http://bugs.python.org/issue7511 6 msgs Issues closed (56) ================== #4022: 2.6 dependent on c:\python26\ on windows http://bugs.python.org/issue4022 closed by georg.brandl #4159: Table about Standard Encodings is cut off at the bottom - 35 e http://bugs.python.org/issue4159 closed by python-dev #4653: Patch to fix typos in C code http://bugs.python.org/issue4653 closed by gregory.p.smith #5135: Expose simplegeneric function in functools module http://bugs.python.org/issue5135 closed by terry.reedy #5445: codecs.StreamWriter.writelines problem when passed generator http://bugs.python.org/issue5445 closed by benjamin.peterson #5970: sys.exc_info leaks into a generator http://bugs.python.org/issue5970 closed by georg.brandl #6310: Windows "App Paths" key is not checked when installed for curr http://bugs.python.org/issue6310 closed by georg.brandl #6661: Transient test_multiprocessing failure (test_active_children) http://bugs.python.org/issue6661 closed by georg.brandl #7300: Unicode arguments in str.format() http://bugs.python.org/issue7300 closed by georg.brandl #7565: Increasing resource.RLIMIT_NOFILE has no effect http://bugs.python.org/issue7565 closed by georg.brandl #8552: msilib can't create large CAB files http://bugs.python.org/issue8552 closed by r.david.murray #8906: Document TestCase attributes in class docstring http://bugs.python.org/issue8906 closed by ezio.melotti #8911: regrtest.main should have a test skipping argument http://bugs.python.org/issue8911 closed by ezio.melotti #9672: test_xpickle fails on Windows: invokes pythonx.y instead of py http://bugs.python.org/issue9672 closed by georg.brandl #9986: PDF files of python docs have text missing http://bugs.python.org/issue9986 closed by georg.brandl #10211: BufferObject doesn't support new buffer interface http://bugs.python.org/issue10211 closed by kristjan.jonsson #10359: ISO C cleanup http://bugs.python.org/issue10359 closed by gregory.p.smith #11087: Speeding up the interpreter with a few lines of code http://bugs.python.org/issue11087 closed by pitrou #12207: Document ast.PyCF_ONLY_AST http://bugs.python.org/issue12207 closed by eric.araujo #14468: Update cloning guidelines in devguide http://bugs.python.org/issue14468 closed by ezio.melotti #15052: Outdated comments in build_ssl.py http://bugs.python.org/issue15052 closed by loewis #15611: devguide: add "core mentors" area to Experts Index http://bugs.python.org/issue15611 closed by brett.cannon #16204: PyBuffer_FillInfo returns 'B' buffer, whose behavior has chang http://bugs.python.org/issue16204 closed by skrah #16676: Segfault under Python 3.3 after PyType_GenericNew http://bugs.python.org/issue16676 closed by benjamin.peterson #16692: Support TLS 1.1 and TLS 1.2 http://bugs.python.org/issue16692 closed by pitrou #16880: Importing "imp" will fail if dynamic loading not supported http://bugs.python.org/issue16880 closed by brett.cannon #17025: reduce multiprocessing.Queue contention http://bugs.python.org/issue17025 closed by neologix #17100: rotating an ordereddict http://bugs.python.org/issue17100 closed by rhettinger #17150: pprint could use line continuation for long string literals http://bugs.python.org/issue17150 closed by pitrou #17316: Add Django 1.5 to benchmarks http://bugs.python.org/issue17316 closed by brett.cannon #17317: Benchmark driver should calculate actual benchmark count in -h http://bugs.python.org/issue17317 closed by brett.cannon #17323: Disable [X refs, Y blocks] ouput in debug builds http://bugs.python.org/issue17323 closed by ezio.melotti #17329: Document unittest.SkipTest http://bugs.python.org/issue17329 closed by ezio.melotti #17389: Optimize Event.wait() http://bugs.python.org/issue17389 closed by pitrou #17425: Update OpenSSL versions in Windows builds http://bugs.python.org/issue17425 closed by loewis #17433: stdlib generator-like iterators don't forward send/throw http://bugs.python.org/issue17433 closed by terry.reedy #17438: json.load docs should mention that it always return unicode http://bugs.python.org/issue17438 closed by ezio.melotti #17447: str.identifier shouldn't accept Python keywords http://bugs.python.org/issue17447 closed by rhettinger #17479: Fix test discovery for test_io.py http://bugs.python.org/issue17479 closed by ezio.melotti #17488: subprocess.Popen bufsize=0 parameter behaves differently in Py http://bugs.python.org/issue17488 closed by gregory.p.smith #17489: random.Random implements __getstate__() and __reduce__() http://bugs.python.org/issue17489 closed by rhettinger #17504: Dropping duplicated docstring explanation of what Mocks' side_ http://bugs.python.org/issue17504 closed by ezio.melotti #17510: assertEquals deprecated in test_program.py (unittest) http://bugs.python.org/issue17510 closed by ezio.melotti #17516: Dead code should be removed http://bugs.python.org/issue17516 closed by haypo #17519: unittest should not try to run abstract classes http://bugs.python.org/issue17519 closed by michael.foord #17521: fileConfig() disables any previously-used "named" loggers, eve http://bugs.python.org/issue17521 closed by python-dev #17524: Problem to run a method http://bugs.python.org/issue17524 closed by brett.cannon #17531: test_grp and test_pwd fail with 32-bit builds on OS X systems http://bugs.python.org/issue17531 closed by ned.deily #17541: Importing `webbrowser` module gives NameError in Python 2.7.4r http://bugs.python.org/issue17541 closed by ezio.melotti #17550: --enable-profiling does nothing (shell syntax bug in configure http://bugs.python.org/issue17550 closed by python-dev #17556: os.path.join() converts None to '' by default http://bugs.python.org/issue17556 closed by georg.brandl #17559: str.is* implementation seem suboptimal for single character st http://bugs.python.org/issue17559 closed by georg.brandl #17562: spam http://bugs.python.org/issue17562 closed by ezio.melotti #444582: Finding programs in PATH, adding shutil.which http://bugs.python.org/issue444582 closed by eric.araujo #616013: cPickle documentation incomplete http://bugs.python.org/issue616013 closed by georg.brandl #886488: popen2 on Windows does not check _fdopen return value http://bugs.python.org/issue886488 closed by terry.reedy

1 0

Safely importing zip files with C extensions
by Vinay Sajip 29 Mar '13

29 Mar '13

> This quote is here to stop GMane complaining that I'm top-posting. Ignore. I've already posted this to distutils-sig, but thought that it might be of interest to readers here as it relates to importing C extensions ... zipimport is great, but there can be issues importing software that contains C extensions. But the new wheel format (PEP 427) may give us a better way of importing zip files containing C extensions. Since wheels are .zip files, they can sometimes be used to provide functionality without needing to be installed. But whereas .zip files contain no convention for indicating compatibility with a particular Python, wheels do contain this compatibility information. Thus, it is possible to check if a wheel can be directly imported from, and the wheel support in distlib allows you to take advantage of this using the mount() and unmount() methods. When you mount a wheel, its absolute path name is added to sys.path, allowing the Python code in it to be imported. (A DistlibException is raised if the wheel isn't compatible with the Python which calls the mount() method.) You don't need mount() just to add the wheel's name to sys.path, or to import pure-Python wheels, of course. But the mount() method goes further than just enabling Python imports - any C extensions in the wheel are also made available for import. For this to be possible, the wheel has to be built with additional metadata about extensions - a JSON file called EXTENSIONS which serialises an extension mapping dictionary. This maps extension module names to the names in the wheel of the shared libraries which implement those modules. Running unmount() on the wheel removes its absolute pathname from sys.path and makes its C extensions, if any, also unavailable for import. Wheels built with the new "distil" tool contain the EXTENSIONS metadata, so can be mounted complete with C extensions: $ distil download -d /tmp simplejson Downloading simplejson-3.1.2.tar.gz to /tmp/simplejson-3.1.2 63KB @ 73 KB/s 100 % Done: 00:00:00 Unpacking ... done. $ distil package --fo=wh -d /tmp /tmp/simplejson-3.1.2/ The following packages were built: /tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl $ python Python 2.7.2+ (default, Jul 20 2012, 22:15:08) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from distlib.wheel import Wheel >>> w = Wheel('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl') >>> w.mount() >>> import simplejson._speedups >>> dir(simplejson._speedups) ['__doc__', '__file__', '__loader__', '__name__', '__package__', 'encode_basestring_ascii', 'make_encoder', 'make_scanner', 'scanstring'] >>> simplejson._speedups.__file__ '/home/vinay/.distlib/dylib-cache/simplejson/_speedups.so' >>> Does anyone see any problems with this approach to importing C extensions from zip files? Regards, Vinay Sajip

10 13

Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue 17538: Document XML vulnerabilties
by Ezio Melotti 28 Mar '13

28 Mar '13

Hi, On Tue, Mar 26, 2013 at 6:53 PM, christian.heimes <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/e87364449954 > changeset: 82973:e87364449954 > branch: 2.7 > parent: 82963:d321885ff8f3 > user: Christian Heimes <christian(a)cheimes.de> > date: Tue Mar 26 17:53:05 2013 +0100 > summary: > Issue 17538: Document XML vulnerabilties > > [...] > > diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst > new file mode 100644 > --- /dev/null > +++ b/Doc/library/xml.rst > @@ -0,0 +1,131 @@ > +.. _xml: > + > +XML Processing Modules > +====================== > + > +.. module:: xml > + :synopsis: Package containing XML processing modules > +.. sectionauthor:: Christian Heimes <christian(a)python.org> > +.. sectionauthor:: Georg Brandl <georg(a)python.org> > + > + > +Python's interfaces for processing XML are grouped in the ``xml`` package. > + > +.. warning:: > + > + The XML modules are not secure against erroneous or maliciously > + constructed data. If you need to parse untrusted or unauthenticated data see > + :ref:`xml-vulnerabilities`. > + > +It is important to note that modules in the :mod:`xml` package require that > +there be at least one SAX-compliant XML parser available. The Expat parser is > +included with Python, so the :mod:`xml.parsers.expat` module will always be > +available. > + > +The documentation for the :mod:`xml.dom` and :mod:`xml.sax` packages are the > +definition of the Python bindings for the DOM and SAX interfaces. > + > +The XML handling submodules are: > + > +* :mod:`xml.etree.ElementTree`: the ElementTree API, a simple and lightweight Something is missing here ^ > + > +.. > + > +* :mod:`xml.dom`: the DOM API definition > +* :mod:`xml.dom.minidom`: a lightweight DOM implementation > +* :mod:`xml.dom.pulldom`: support for building partial DOM trees > + > +.. > + > +* :mod:`xml.sax`: SAX2 base classes and convenience functions > +* :mod:`xml.parsers.expat`: the Expat parser binding > + > + > +.. _xml-vulnerabilities: > + > [...] > + > +defused packages > +---------------- > + > +`defusedxml`_ is a pure Python package with modified subclasses of all stdlib > +XML parsers that prevent any potentially malicious operation. The courses of > +action are recommended for any server code that parses untrusted XML data. This last sentence doesn't make much sense to me. Is it even correct? > The > +package also ships with example exploits and an extended documentation on more > +XML exploits like xpath injection. > + > +`defusedexpat`_ provides a modified libexpat and patched replacment s/replacment/replacement/ > +:mod:`pyexpat` extension module with countermeasures against entity expansion > +DoS attacks. Defusedexpat still allows a sane and configurable amount of entity > +expansions. The modifications will be merged into future releases of Python. > + > +The workarounds and modifications are not included in patch releases as they > +break backward compatibility. After all inline DTD and entity expansion are > +well-definied XML features. s/definied/defined/ > + > + > +.. _defusedxml: <https://pypi.python.org/pypi/defusedxml/> > +.. _defusedexpat: <https://pypi.python.org/pypi/defusedexpat/> > +.. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs > +.. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb > +.. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition > [...] Best Regards, Ezio Melotti

1 0

Can I introspect/reflect to get arguments exec()?
by Rocky Bernstein 28 Mar '13

28 Mar '13

[asked on comp.lang.python but no takers. So I'm bumping it up a notch.] I have ported my Python debugger pydbgr to Python3. See [1] or [2]. Inside the debugger, when there is an exec() somewhere in the call stack, I'd like to be able to retrieve the string parameter. With this, the debugger can show part of the string in a call stack. Or it can show the text when the frame is set to that exec() frame. Going further, the debugger could write the exec string out to a temporary file. And when reporting locations, it could report not just something like "<string> line 4", but also give that temporary file name which a front-end could use as well. So consider this code using inspect.getargvalues() and inspect.currentframe(): import inspect def my_exec(string): show_args(inspect.currentframe()) # simulate exec(string) def show_args(frame): print(inspect.getargvalues(frame)) my_exec("show_args(inspect.currentframe())") exec("show_args(inspect.currentframe())") When run this is the output: python3 exec-args.py ArgInfo(args=['string'], varargs=None, keywords=None, locals={'string': 'show_args(inspect.currentframe())'}) ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec': <function my_exec at 0xb6f828ec>,, ... In a different setting, CPython byte-code assembly that gets generated for running exec() is: 25 88 LOAD_GLOBAL 10 (exec) 91 LOAD_CONST 4 ('show_args(inspect.currentframe())') --> 94 CALL_FUNCTION 1 97 POP_TOP What's going on? Also, I have the same question for CPython 2.6 or Python 2.7. Thanks! [1] http://code.google.com/p/pydbgr/ [2] http://code.google.com/p/python3-trepan/

3 5

Can we triple quoted string as a comment?
by Victor Stinner 28 Mar '13

28 Mar '13

Hi, I just realized that the Python peephole optimizer removes useless instructions like numbers and strings between other instructions, without raising an error nor emiting an error. Example: $ python -Wd -c 'print "Hello"; "World"' Hello As part of my astoptimizer project, I wrote a function to detect such useless instructions which emit a warning. I opened the following issue to report what I found: http://bugs.python.org/issue17516 Different modules use long strings as comments. What is the "official" policy about such strings? Should we use strings or comments? (IMO a comment should be used instead.) Victor

8 7

Python-Dev March 2013