Mailman 3 December 2013 - Python-Dev

tracemalloc: add an optional memory limit
by Victor Stinner Dec. 9, 2013

Dec. 9, 2013

Hi, The PEP 454 (tracemalloc module) has been implemented in Python 3.4 beta 1. Previously, I also wrote a pyfailmalloc project to test how Python behaves on memory allocation failures. I found various bugs using this tool. I propose to add an optional memory limit feature to the tracemallocmodule. It would allow to set an arbitrary limit (ex: 100 MB) to check how your application behaves on systems with a few free memory (ex: embedded devices). What do you think of this enhancement? Would it fit Python 3.4 timeline? It's a new feature, but for a module which was introduced in Python 3.4. I wanted to propose directly the feature in the PEP, but Python was not ready for that (Python failed to handle correctly memory allocation failures). On Linux, it's possible to limit globally the "address space" of a process, but this value is not convinient. For example, the address space includes shared memory and read-only memory mappings of files, whereas this memory should not reduce the memory available for other applications. tracemalloc only counts memory directly allocated by Python and so private memory which reduces directly the memory available for other applications. Technically, it's interesting to implement this feature in tracemalloc. No new code should be added to count how many bytes were allocated by Python, tracemalloc must already do that (indirectly). It's simple to modify tracemalloc to make malloc() fails if an allocation would make the counter greater than the limit. I wrote a patch in the following issue to add an optional memory_limit parameter to tracemalloc.start(): http://bugs.python.org/issue19817 The patch is short: Doc/library/tracemalloc.rst | 6 +++- Lib/test/test_tracemalloc.py | 11 +++++++- Modules/_tracemalloc.c | 56 ++++++++++++++++++++++++++++++++++++------- 3 files changed, 63 insertions(+), 10 deletions(-) -- I tried to fix all bugs in Python to handle correctly memory allocation failures. When the memory limit is very low (less than 1024 bytes), there are still some corner cases where Python may crash. These corner cases can be fixed. For example, I proposed a patch to make PyErr_NoMemory() more reliable (which is probably the most critical bug of remaining bugs): http://bugs.python.org/issue19835 I fixed a reference leak in the unittest module (#19880) to reduce the risk of PyErr_NoMemory() crash when I use the Python test suite to test the memory limit feature (see issue #19817). I'm not sure that PyErr_NoMemory() bug affects real applications. Victor

3 3

Summary of Python tracker Issues
by Python tracker Dec. 6, 2013

Dec. 6, 2013

ACTIVITY SUMMARY (2013-11-29 - 2013-12-06) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4323 ( +6) closed 27359 (+70) total 31682 (+76) Open issues with patches: 1982 Issues opened (56) ================== #10717: Multiprocessing module cannot call instance methods across pro http://bugs.python.org/issue10717 reopened by alexandre.vassalotti #16549: regression: -m json.tool module is broken http://bugs.python.org/issue16549 reopened by Arfrever #19835: Add a MemoryError singleton to fix an unlimited loop when the http://bugs.python.org/issue19835 opened by haypo #19837: Wire protocol encoding for the JSON module http://bugs.python.org/issue19837 opened by ncoghlan #19838: test.test_pathlib.PosixPathTest.test_touch_common fails on Fre http://bugs.python.org/issue19838 opened by Claudiu.Popa #19840: The is no way to tell shutil.move to ignore metadata http://bugs.python.org/issue19840 opened by zodalahtathi #19841: ConfigParser PEP issues http://bugs.python.org/issue19841 opened by Ivailo.Monev #19843: Wait for multiple sub-processes to terminate http://bugs.python.org/issue19843 opened by giampaolo.rodola #19846: print() and write() are relying on sys.getfilesystemencoding() http://bugs.python.org/issue19846 opened by Sworddragon #19847: Setting the default filesystem-encoding http://bugs.python.org/issue19847 opened by Sworddragon #19851: imp.reload problem with submodule http://bugs.python.org/issue19851 opened by ronaldoussoren #19855: uuid._find_mac fails if an executable not in /sbin or /usr/sbi http://bugs.python.org/issue19855 opened by serhiy.storchaka #19856: Possible bug in shutil.move() on Windows http://bugs.python.org/issue19856 opened by serhiy.storchaka #19857: test_imaplib doesn't always reap SocketServer thread http://bugs.python.org/issue19857 opened by neologix #19858: Make pickletools.optimize aware of the MEMOIZE opcode. http://bugs.python.org/issue19858 opened by alexandre.vassalotti #19861: Update What's New for Python 3.4 http://bugs.python.org/issue19861 opened by serhiy.storchaka #19862: Unclear xpath caching for custom namespaces http://bugs.python.org/issue19862 opened by valeriy.nov #19863: Missing function attributes in 2.7 docs. http://bugs.python.org/issue19863 opened by mark.dickinson #19864: multiprocessing Proxy docs need locking semantics explained http://bugs.python.org/issue19864 opened by maxpolk #19865: create_unicode_buffer() fails on non-BMP strings on Windows http://bugs.python.org/issue19865 opened by gergely.erdelyi #19866: tests aifc, sunau and wave failures on a fresh Win64 installat http://bugs.python.org/issue19866 opened by francismb #19867: pickletools.OpcodeInfo.code is a string http://bugs.python.org/issue19867 opened by pitrou #19869: BaseCookie does not complain if a non RFC compliant cookie hea http://bugs.python.org/issue19869 opened by florianpilz #19870: Backport Cookie fix to 2.7 (httponly / secure flag) http://bugs.python.org/issue19870 opened by florianpilz #19871: json module won't parse a float that starts with a decimal poi http://bugs.python.org/issue19871 opened by picomancer #19873: There is a duplicate function in Lib/test/test_pathlib.py http://bugs.python.org/issue19873 opened by vajrasky #19875: test_getsockaddrarg occasional failure http://bugs.python.org/issue19875 opened by pitrou #19876: selectors (and asyncio?): document behaviour on closed files/s http://bugs.python.org/issue19876 opened by haypo #19878: bz2.BZ2File.__init__() cannot be called twice with non-existen http://bugs.python.org/issue19878 opened by Level #19879: imageop: bug in error handler http://bugs.python.org/issue19879 opened by Level #19880: unittest: on failure, TestCase.run() keeps a reference to the http://bugs.python.org/issue19880 opened by haypo #19883: Integer overflow in zipimport.c http://bugs.python.org/issue19883 opened by christian.heimes #19884: Importing readline produces erroneous output http://bugs.python.org/issue19884 opened by bkabrda #19885: lzma segfault when __init__ with non-existent file after execu http://bugs.python.org/issue19885 opened by vajrasky #19886: Better estimated memory requirements for bigmem tests http://bugs.python.org/issue19886 opened by serhiy.storchaka #19887: Path.resolve() ENAMETOOLONG on pathologic symlinks http://bugs.python.org/issue19887 opened by serhiy.storchaka #19888: Three argument type() super call sets __name__ but not __qualn http://bugs.python.org/issue19888 opened by zkrynicki #19890: Typo in multiprocessing docs http://bugs.python.org/issue19890 opened by Antony.Lee #19891: Exiting Python REPL prompt with user without home directory th http://bugs.python.org/issue19891 opened by vajrasky #19893: Python cApi memory problem. Py_Initialize memory leak http://bugs.python.org/issue19893 opened by rstarostecki #19894: zipfile ignores deflate level settings in zipinfo object http://bugs.python.org/issue19894 opened by rmilne #19895: Cryptic error when subclassing multiprocessing classes http://bugs.python.org/issue19895 opened by Antony.Lee #19896: Exposing "q" and "Q" to multiprocessing.sharedctypes http://bugs.python.org/issue19896 opened by Antony.Lee #19897: Use python as executable instead of python3 in Python 2 docs http://bugs.python.org/issue19897 opened by berker.peksag #19898: No tests for dequereviter_new http://bugs.python.org/issue19898 opened by christian.heimes #19899: No test for thread.interrupt_main() http://bugs.python.org/issue19899 opened by christian.heimes #19900: improve pickle intro http://bugs.python.org/issue19900 opened by pitrou #19901: tests fail due to unsupported SO_REUSEPORT when building Pytho http://bugs.python.org/issue19901 opened by RubyTuesdayDONO #19902: logging docs don't document integer constants http://bugs.python.org/issue19902 opened by follower #19903: Idle: Use inspect.signature for calltips http://bugs.python.org/issue19903 opened by terry.reedy #19904: Add 128-bit integer support to struct http://bugs.python.org/issue19904 opened by fil #19905: Add 128-bit integer support to ctypes http://bugs.python.org/issue19905 opened by fil #19906: Typo in urllib documentation http://bugs.python.org/issue19906 opened by Claudiu.Popa #19907: gettext - Non ascii chars in header http://bugs.python.org/issue19907 opened by Michael.Müller #19909: Best practice docs for new SSL features http://bugs.python.org/issue19909 opened by christian.heimes #19910: Document that compile() source may be bytes http://bugs.python.org/issue19910 opened by gvanrossum Most recent 15 issues with no replies (15) ========================================== #19910: Document that compile() source may be bytes http://bugs.python.org/issue19910 #19909: Best practice docs for new SSL features http://bugs.python.org/issue19909 #19907: gettext - Non ascii chars in header http://bugs.python.org/issue19907 #19906: Typo in urllib documentation http://bugs.python.org/issue19906 #19905: Add 128-bit integer support to ctypes http://bugs.python.org/issue19905 #19903: Idle: Use inspect.signature for calltips http://bugs.python.org/issue19903 #19902: logging docs don't document integer constants http://bugs.python.org/issue19902 #19899: No test for thread.interrupt_main() http://bugs.python.org/issue19899 #19898: No tests for dequereviter_new http://bugs.python.org/issue19898 #19897: Use python as executable instead of python3 in Python 2 docs http://bugs.python.org/issue19897 #19896: Exposing "q" and "Q" to multiprocessing.sharedctypes http://bugs.python.org/issue19896 #19895: Cryptic error when subclassing multiprocessing classes http://bugs.python.org/issue19895 #19894: zipfile ignores deflate level settings in zipinfo object http://bugs.python.org/issue19894 #19890: Typo in multiprocessing docs http://bugs.python.org/issue19890 #19867: pickletools.OpcodeInfo.code is a string http://bugs.python.org/issue19867 Most recent 15 issues waiting for review (15) ============================================= #19906: Typo in urllib documentation http://bugs.python.org/issue19906 #19900: improve pickle intro http://bugs.python.org/issue19900 #19897: Use python as executable instead of python3 in Python 2 docs http://bugs.python.org/issue19897 #19896: Exposing "q" and "Q" to multiprocessing.sharedctypes http://bugs.python.org/issue19896 #19894: zipfile ignores deflate level settings in zipinfo object http://bugs.python.org/issue19894 #19887: Path.resolve() ENAMETOOLONG on pathologic symlinks http://bugs.python.org/issue19887 #19886: Better estimated memory requirements for bigmem tests http://bugs.python.org/issue19886 #19883: Integer overflow in zipimport.c http://bugs.python.org/issue19883 #19880: unittest: on failure, TestCase.run() keeps a reference to the http://bugs.python.org/issue19880 #19878: bz2.BZ2File.__init__() cannot be called twice with non-existen http://bugs.python.org/issue19878 #19876: selectors (and asyncio?): document behaviour on closed files/s http://bugs.python.org/issue19876 #19863: Missing function attributes in 2.7 docs. http://bugs.python.org/issue19863 #19855: uuid._find_mac fails if an executable not in /sbin or /usr/sbi http://bugs.python.org/issue19855 #19843: Wait for multiple sub-processes to terminate http://bugs.python.org/issue19843 #19841: ConfigParser PEP issues http://bugs.python.org/issue19841 Top 10 most discussed issues (10) ================================= #18885: handle EINTR in the stdlib http://bugs.python.org/issue18885 17 msgs #19876: selectors (and asyncio?): document behaviour on closed files/s http://bugs.python.org/issue19876 14 msgs #19837: Wire protocol encoding for the JSON module http://bugs.python.org/issue19837 13 msgs #7105: weak dict iterators are fragile because of unpredictable GC ru http://bugs.python.org/issue7105 12 msgs #19888: Three argument type() super call sets __name__ but not __qualn http://bugs.python.org/issue19888 10 msgs #19717: resolve() fails when the path doesn't exist http://bugs.python.org/issue19717 7 msgs #19834: Unpickling exceptions pickled by Python 2 http://bugs.python.org/issue19834 7 msgs #19847: Setting the default filesystem-encoding http://bugs.python.org/issue19847 7 msgs #19871: json module won't parse a float that starts with a decimal poi http://bugs.python.org/issue19871 7 msgs #19891: Exiting Python REPL prompt with user without home directory th http://bugs.python.org/issue19891 7 msgs Issues closed (68) ================== #2295: cPickle corner case - docs or bug? http://bugs.python.org/issue2295 closed by alexandre.vassalotti #2799: Remove _PyUnicode_AsString(), rework _PyUnicode_AsStringAndSiz http://bugs.python.org/issue2799 closed by alexandre.vassalotti #3657: pickle can pickle the wrong function http://bugs.python.org/issue3657 closed by alexandre.vassalotti #3693: Obscure array.array error message http://bugs.python.org/issue3693 closed by alexandre.vassalotti #6477: Pickling of NoneType raises PicklingError http://bugs.python.org/issue6477 closed by alexandre.vassalotti #9269: Cannot pickle self-referencing sets http://bugs.python.org/issue9269 closed by alexandre.vassalotti #9276: pickle should support methods http://bugs.python.org/issue9276 closed by alexandre.vassalotti #9709: test_distutils warning: initfunc exported twice on Windows http://bugs.python.org/issue9709 closed by skrah #11349: _pickle should implement the module finalisation protocol http://bugs.python.org/issue11349 closed by alexandre.vassalotti #11480: Cannot copy a class with a metaclass other than type http://bugs.python.org/issue11480 closed by alexandre.vassalotti #11854: __or__ et al instantiate subclass of set without calling __ini http://bugs.python.org/issue11854 closed by mark.dickinson #13520: Patch to make pickle aware of __qualname__ http://bugs.python.org/issue13520 closed by alexandre.vassalotti #15667: PEP 3121, 384 refactoring applied to pickle module http://bugs.python.org/issue15667 closed by alexandre.vassalotti #15798: subprocess.Popen() fails if 0, 1 or 2 descriptor is closed http://bugs.python.org/issue15798 closed by gregory.p.smith #16231: pickle persistent_id return value http://bugs.python.org/issue16231 closed by alexandre.vassalotti #17897: Optimize unpickle prefetching http://bugs.python.org/issue17897 closed by serhiy.storchaka #18015: python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or http://bugs.python.org/issue18015 closed by alexandre.vassalotti #18073: pickle.Unpickler may read too many bytes, causing hangs with b http://bugs.python.org/issue18073 closed by alexandre.vassalotti #18699: What is Future.running() for in PEP 3148 / concurrent.futures. http://bugs.python.org/issue18699 closed by gvanrossum #18840: Tutorial recommends pickle module without any warning of insec http://bugs.python.org/issue18840 closed by pitrou #18843: Py_FatalError (msg=0x7f0e3b373232 "bad leading pad byte") at P http://bugs.python.org/issue18843 closed by neologix #18994: Inside fcntl module, we does not check the return code of all_ http://bugs.python.org/issue18994 closed by neologix #19088: TypeError with pickle in embedded python3.3 when starting mult http://bugs.python.org/issue19088 closed by alexandre.vassalotti #19138: doctest.IGNORE_EXCEPTION_DETAIL doesn't match when no detail e http://bugs.python.org/issue19138 closed by tim.peters #19296: Compiler warning when compiling dbm module http://bugs.python.org/issue19296 closed by christian.heimes #19454: devguide: Document what a "platform support" is http://bugs.python.org/issue19454 closed by gvanrossum #19509: No SSL match_hostname() in ftp, imap, nntp, pop, smtp modules http://bugs.python.org/issue19509 closed by christian.heimes #19595: Silently skipped test in test_winsound http://bugs.python.org/issue19595 closed by zach.ware #19665: test_logging fails with SMTPHandler timeout http://bugs.python.org/issue19665 closed by python-dev #19707: Check if unittest.mock needs updating for PEP 451 http://bugs.python.org/issue19707 closed by brett.cannon #19726: BaseProtocol is not an ABC http://bugs.python.org/issue19726 closed by gvanrossum #19741: tracemalloc: tracemalloc_log_alloc() doesn't check _Py_HASHTAB http://bugs.python.org/issue19741 closed by python-dev #19754: pickletools.optimize doesn't reframe correctly http://bugs.python.org/issue19754 closed by alexandre.vassalotti #19757: _tracemalloc.c: compiler warning with gil_state http://bugs.python.org/issue19757 closed by python-dev #19780: Pickle 4 frame headers optimization http://bugs.python.org/issue19780 closed by pitrou #19781: No SSL match_hostname() in ftplib http://bugs.python.org/issue19781 closed by christian.heimes #19782: No SSL match_hostname() in imaplib http://bugs.python.org/issue19782 closed by christian.heimes #19783: No SSL match_hostname() in nntplib http://bugs.python.org/issue19783 closed by christian.heimes #19784: No SSL match_hostname() in poplib http://bugs.python.org/issue19784 closed by christian.heimes #19785: No SSL match_hostname() in smtplib http://bugs.python.org/issue19785 closed by christian.heimes #19789: Improve wording of how to "undo" a call to logging.disable(lvl http://bugs.python.org/issue19789 closed by python-dev #19800: Write more detailed framing tests http://bugs.python.org/issue19800 closed by pitrou #19814: Document prefix matching behavior of argparse, particularly fo http://bugs.python.org/issue19814 closed by eli.bendersky #19827: Optimize socket.settimeout() and socket.setblocking(): avoid s http://bugs.python.org/issue19827 closed by python-dev #19831: tracemalloc: stop the module later at Python shutdown http://bugs.python.org/issue19831 closed by python-dev #19833: asyncio: patches to document EventLoop and add more examples http://bugs.python.org/issue19833 closed by haypo #19836: selectors: improve examples section http://bugs.python.org/issue19836 closed by gvanrossum #19839: bz2: regression wrt supporting files with trailing garbage aft http://bugs.python.org/issue19839 closed by python-dev #19842: selectors: refactor BaseSelector implementation http://bugs.python.org/issue19842 closed by gvanrossum #19844: os.path.split fails on windows path http://bugs.python.org/issue19844 closed by christian.heimes #19845: Compiling Python on Windows docs http://bugs.python.org/issue19845 closed by zach.ware #19848: Misleading error on creating already existed symlink http://bugs.python.org/issue19848 closed by serhiy.storchaka #19849: selectors behaviour on EINTR undocumented http://bugs.python.org/issue19849 closed by neologix #19850: asyncio: limit EINTR occurrences with SA_RESTART http://bugs.python.org/issue19850 closed by neologix #19852: Misplaced private API method in pathlib.py http://bugs.python.org/issue19852 closed by pitrou #19853: Add support for Bitrig to 2.7 http://bugs.python.org/issue19853 closed by brett.cannon #19854: Add support for Bitrig to 3.4 http://bugs.python.org/issue19854 closed by brett.cannon #19859: functools.lru_cache keeps objects alive forever http://bugs.python.org/issue19859 closed by rhettinger #19860: asyncio: Add context manager to BaseEventLoop? http://bugs.python.org/issue19860 closed by gvanrossum #19868: Importing humanhash and uuid causes a segmentation fault crash http://bugs.python.org/issue19868 closed by ned.deily #19872: Remove unused imports in pathlib http://bugs.python.org/issue19872 closed by pitrou #19874: test_logging failures on Snow Leopard http://bugs.python.org/issue19874 closed by vinay.sajip #19877: test_resolve_dot of test_pathlib.py fails on Windows Vista http://bugs.python.org/issue19877 closed by pitrou #19881: Fix bigmem pickle tests http://bugs.python.org/issue19881 closed by alexandre.vassalotti #19882: Closing a socket when makefile() is used http://bugs.python.org/issue19882 closed by pitrou #19889: Revision information missing in Python 2.6.9 http://bugs.python.org/issue19889 closed by barry #19892: register.send_metadata fails with EOFError: EOF when reading a http://bugs.python.org/issue19892 closed by neaj #19908: pathlib.PureWindowsPath doesn't join relative paths correctly http://bugs.python.org/issue19908 closed by serhiy.storchaka

1 0

Python 3 is five years old
by Brett Cannon Dec. 6, 2013

Dec. 6, 2013

On 2008-12-03, Python 3.0.0 was released by Barry. I was actually on IRC the night when it happened, with Guido leaving his office across the hall from mine. We gave each other a high-five and then we both went home. Not exactly a party, but I know I at least breathed a sigh of relief that Python 3 was finally real. On this anniversary I just wanted to thank everyone who contributed to making Python 3 happen. Thanks to the core devs who worked hard to make Python better for everyone, even in the face of people yelling at us that we were crazy, killing Python, etc. Thanks to those in the community who stuck by the dev team and had faith we knew what we were doing and have continued to help everyone move forward and off of Python 2 to realize how much more pleasant Python 3 is to work with. At this point I'm willing to say I'm happy with how Python 3 has turned out and how the community has supported it. Python 3.3 was a great release and I think Python 3.4 will be as well. I see no reason to think that the language and community's forward momentum will not continue, leading to a day where Python 2 only exists for people stuck on RHEL. =)

4 3

Re: [Python-Dev] Python-Dev Digest, Vol 125, Issue 5
by Fil Mackay Dec. 6, 2013

Dec. 6, 2013

Hi Antoine, > I've found that libffi does support this type, but sadly ctypes and cffi > do > > not. Adding to ctypes does not seem to be trivial, since the description > of > > an integer type is limited to a single character ("q" in the case of long > > long). "q" is considered to be a length of 8, whereas what I really want > is > > the integer type "ll" which is correctly considered length of 16. > > Aren't you talking about the struct module? In ctypes, it seems it > would be sufficient to add a "c_int128" type (and/or "c_uint128"). > Adding a c_int128 type seems somewhat non trivial though. The specific-sized c_intX types are assigned to the named equivalents (c_long etc.). Getting a named type that equates to 128-bit (on OSX at least anyway) is the problem. The : class c_something(_SimpleCData): _type_ = "?" _check_size(c_something) The type "q" seems to relate to "long long" ("q"uad word?) - but that has a size of 8 for me. I don't know: what "something" should be, and what "?" should be. I am guessing stay away from the "long long long" schenanigans and just define c_int128 directly: class c_int128(_SimpleCData): _type_ = "?" .. but there still seems to lack a _type_ that will relate to a 128-bit integer.. OK, it seems that adding a new "?" that relates to 128-bit requires the change to struct module. This seems to be in lib-dynload and is written in C not python? > Can anyone give me some pointers as to why this has not been added to > date, > > and the best line of attack? > > Probably because noone needed it until now? > > We have a comprehensive guide if you want to contribute a patch: > http://docs.python.org/devguide/ > > The first step would be to open an enhancement request on the > issue tracker, the discussion will then move on there: > http://bugs.python.org/ > (if you want to enhance both the ctypes and struct modules, please open > a separate issue for each) > OK, thanks. I had been put off by the "bugs" title since this is an enhancement.. will do. Regards, Fil.

1 0

One-line abstractmethod function?
by Allen Li Dec. 5, 2013

Dec. 5, 2013

Hello Python devs, As a regular Python user, I find the abc module useful for making Python's duck typing more explicit. In particular, I ofen use it like a Java interface or C header, to provide methods to implement for a given "duck type". 90% of the time, it ends up looking something like this: class Foo(metaclass=abc.ABCMeta): @abc.abstractmethod def f1(self): raise NotImplementedError @staticmethod @abc.abstractmethod def f2(arg1): raise NotImplementedError ... What if there was a function, say make_abstract_method (better name pending), so that the above could be written like: class Foo(metaclass=abc.ABCMeta): f1 = abc.make_abstract_method('f1', ['self']) f2 = staticmethod(abc.make_abstract_method('f2', ['arg1'])) ... I think that it would make ABC definitions a lot more compact for many use cases, but I welcome any criticisms against this idea. Allen Li

6 10

test_bind_port and test_find_unused_port fail due to missing SO_REUSEPORT when building Python 3.3.2-r2 (from portage) on 3.7.10-gentoo-r1 kernel
by Reuben Garrett Dec. 5, 2013

Dec. 5, 2013

Greetings, learned Python devs! Apologies in advance if my message is misdirected. I'm building Python 3.3.2-r2 from Gentoo's Portage tree [1] and encountered two failed tests which probably should not have been attempted on my OS (Gentoo 3.7.10): test_bind_port and test_find_unused_port both use the SO_REUSEPORT socket option which, to the best of my knowledge is only available since kernel version 3.9. I was able to build by skipping the tests — but I believe the tests are there for a reason, and it would be best if a test that is known to fail should be skipped (or replaced with a fallback that is likely to succeed). Issue # 16594 [3] may be related (not sure). Is it possible to detect the kernel version and skip (or modify) these tests if SO_REUSEPORT is not available? If I'm missing something obvious (or should be reporting this elsewhere), please educate me so I can better respect your attention next time around :] +=== excerpt of portage build log: ====================================================================== ERROR: test_bind_port (test.test_support.TestSupport) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.3.2/Lib/test/test_support.py", line 87, in test_bind_port support.bind_port(s) File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.3.2/Lib/test/support.py", line 548, in bind_port if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT) == 1: OSError: [Errno 92] Protocol not available ====================================================================== ERROR: test_find_unused_port (test.test_support.TestSupport) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.3.2/Lib/test/test_support.py", line 80, in test_find_unused_port port = support.find_unused_port() File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.3.2/Lib/test/support.py", line 522, in find_unused_port port = bind_port(tempsock) File "/var/tmp/portage/dev-lang/python-3.3.2-r2/work/Python-3.3.2/Lib/test/support.py", line 548, in bind_port if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT) == 1: OSError: [Errno 92] Protocol not available ===+ [1]: https://packages.gentoo.org/package/dev-lang/python [2]: https://lwn.net/Articles/542629/ [3]: http://bugs.python.org/issue16594 -- Best regards, Reuben Garrett

2 2

Running the unit test as root/administrator
by Vajrasky Kok Dec. 5, 2013

Dec. 5, 2013

Greetings, comrades! Having handled these two issues personally: http://bugs.python.org/issue19877 - test related with symlink fails on Windows Vista with administrator account (in Windows NT 6, only account in administrator group can create symlink) http://bugs.python.org/issue18678 - bug in spwd module, which can be used only by root I believe there is a virtue of running the unit test as root/administrator to catch these errors automatically. Currently there is no unit test for spwd module. What do you say, comrades? --Vajrasky

7 10

128 bit integer support
by Fil Mackay Dec. 5, 2013

Dec. 5, 2013

I have been doing some research on getting "int128_t" supported in Python, and have hit a snag: I've found that libffi does support this type, but sadly ctypes and cffi do not. Adding to ctypes does not seem to be trivial, since the description of an integer type is limited to a single character ("q" in the case of long long). "q" is considered to be a length of 8, whereas what I really want is the integer type "ll" which is correctly considered length of 16. These musings are on OSX. Can anyone give me some pointers as to why this has not been added to date, and the best line of attack? Regards, Fil.

3 2

PEP 0404 and VS 2010
by Christian Tismer Dec. 2, 2013

Dec. 2, 2013

Howdy friends, according to pep 404, there will never be an official Python 2.8. The migration path is from 2.7 to 3.x. I agree with this strategy in almost all consequences but this one: Many customers are forced to stick with Python 2.X because of other products, but they require a Python 2.X version which can be compiled using Visual Studio 2010 or better. This is considered an improvement and not a bug fix, where I disagree. So I see many Python 2.7 repositories on BB/GH which are hacked to support VS 2010, but they are not converted with the same quality in mind that fits our standards. My question ----------- I have created a very clean Python 2.7.6+ based CPython with the Stackless additions, that compiles with VS 2010, using the adapted project structure of Python 3.3.X, and I want to publish that on the Stackless website as the official "Stackless Python 2.8". If you consider Stackless as official ;-) . This compiler change is currently the only deviation from CPython 2.7, but we may support a few easy back-ports on customer demand. We don'd add any random stuff, of course. My question is if that is safe to do: Can I rely on PEP 404 that the "Python 2.8" namespace never will clash with CPython? And if not, what do you suggest then? It will be submitted by end of November, thanks for your quick responses! all the best -- Chris -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

22 65

Re: [Python-Dev] [Python-checkins] cpython (3.3): Issue #19728: Fix sys.getfilesystemencoding() documentation
by Nick Coghlan Dec. 2, 2013

Dec. 2, 2013

On 2 Dec 2013 21:18, "victor.stinner" <python-checkins(a)python.org> wrote: > > http://hg.python.org/cpython/rev/b231e0c3fd26 > changeset: 87692:b231e0c3fd26 > branch: 3.3 > parent: 87690:7d3297f127ae > user: Victor Stinner <victor.stinner(a)gmail.com> > date: Mon Dec 02 12:16:46 2013 +0100 > summary: > Issue #19728: Fix sys.getfilesystemencoding() documentation This doesn't appear to be the right issue number (ironically, this number refers to the PEP 453 Windows installer one I was trying to reference when I got a commit message wrong the other day and ended up referencing an asyncio bug instead). Cheers, Nick. > > files: > Doc/library/sys.rst | 5 ++--- > 1 files changed, 2 insertions(+), 3 deletions(-) > > > diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst > --- a/Doc/library/sys.rst > +++ b/Doc/library/sys.rst > @@ -409,7 +409,7 @@ > * On Mac OS X, the encoding is ``'utf-8'``. > > * On Unix, the encoding is the user's preference according to the result of > - nl_langinfo(CODESET), or ``'utf-8'`` if ``nl_langinfo(CODESET)`` failed. > + nl_langinfo(CODESET). > > * On Windows NT+, file names are Unicode natively, so no conversion is > performed. :func:`getfilesystemencoding` still returns ``'mbcs'``, as > @@ -420,8 +420,7 @@ > * On Windows 9x, the encoding is ``'mbcs'``. > > .. versionchanged:: 3.2 > - On Unix, use ``'utf-8'`` instead of ``None`` if ``nl_langinfo(CODESET)`` > - failed. :func:`getfilesystemencoding` result cannot be ``None``. > + :func:`getfilesystemencoding` result cannot be ``None`` anymore. > > > .. function:: getrefcount(object) > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins >

2 1