Mailman 3 July 2012 - Python-Dev

Unbinding of methods
by M Stefan July 19, 2012

July 19, 2012

Hey, As part of pickle4, I found it interesting to add the possibility of pickling bound functions (instance methods). This is done by pickling f.__self__ and f.__func__ separately, and then adding a BIND opcode to tie them together. While this appears to work fine for python methods (non-builtin), some issues arise with builtins. These are partly caused because not all builtin function types support __func__, partly because not all of them fill __module__ when they should and partly because … [View More]

4 5

Use function names instead of functions for os.supports_dir_fd?
by Victor Stinner July 19, 2012

July 19, 2012

Hi, Python 3.3 introduced os.supports_dir_fd to check if some os functions do accept a file descriptor instead of a path. The problem is that os.supports_dir_fd is a list of functions, not a list of function names. If os functions are monkey patched, you cannot test anymore if a function supports file descriptor. Monkey patching is a common practice in Python. test_os.py replaces os.exec*() functions temporary for example. It's also inconsistent with the new time.get_clock_info() function … [View More]

5 5

Re: [Python-Dev] [Python-checkins] cpython: Close #15387: inspect.getmodulename() now uses a new
by Jim Jewett July 18, 2012

July 18, 2012

Why is inspect.getmoduleinfo() deprecated? Is it just to remove circular dependencies? FWIW, I much prefer an API like: tell_me_about(object) to one like: for test_data in (X, Y, Z): usable = tester(object, test_data) if valid(usable): return possible_results[test_data] and to me, inspect.getmoduleinfo(path) looks like the first, while checking the various import.machinery.*SUFFIXES looks like the second. -jJ On 7/18/12, nick.coghlan <python-… [View More]checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/af7961e1c362 > changeset: 78161:af7961e1c362 > user: Nick Coghlan <ncoghlan(a)gmail.com> > date: Wed Jul 18 23:14:57 2012 +1000 > summary: > Close #15387: inspect.getmodulename() now uses a new > importlib.machinery.all_suffixes() API rather than the deprecated > inspect.getmoduleinfo() > > files: > Doc/library/importlib.rst | 13 ++++++++++++- > Doc/library/inspect.rst | 15 ++++++++++++--- > Lib/importlib/machinery.py | 4 ++++ > Lib/inspect.py | 11 +++++++++-- > Misc/NEWS | 3 +++ > 5 files changed, 40 insertions(+), 6 deletions(-) > > > diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst > --- a/Doc/library/importlib.rst > +++ b/Doc/library/importlib.rst > @@ -533,12 +533,23 @@ > > .. attribute:: EXTENSION_SUFFIXES > > - A list of strings representing the the recognized file suffixes for > + A list of strings representing the recognized file suffixes for > extension modules. > > .. versionadded:: 3.3 > > > +.. func:: all_suffixes() > + > + Returns a combined list of strings representing all file suffixes for > + Python modules recognized by the standard import machinery. This is a > + helper for code which simply needs to know if a filesystem path > + potentially represents a Python module (for example, > + :func:`inspect.getmodulename`) > + > + .. versionadded:: 3.3 > + > + > .. class:: BuiltinImporter > > An :term:`importer` for built-in modules. All known built-in modules > are > diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst > --- a/Doc/library/inspect.rst > +++ b/Doc/library/inspect.rst > @@ -198,9 +198,18 @@ > .. function:: getmodulename(path) > > Return the name of the module named by the file *path*, without > including the > - names of enclosing packages. This uses the same algorithm as the > interpreter > - uses when searching for modules. If the name cannot be matched > according to the > - interpreter's rules, ``None`` is returned. > + names of enclosing packages. The file extension is checked against all > of > + the entries in :func:`importlib.machinery.all_suffixes`. If it matches, > + the final path component is returned with the extension removed. > + Otherwise, ``None`` is returned. > + > + Note that this function *only* returns a meaningful name for actual > + Python modules - paths that potentially refer to Python packages will > + still return ``None``. > + > + .. versionchanged:: 3.3 > + This function is now based directly on :mod:`importlib` rather than > the > + deprecated :func:`getmoduleinfo`. > > > .. function:: ismodule(object) > diff --git a/Lib/importlib/machinery.py b/Lib/importlib/machinery.py > --- a/Lib/importlib/machinery.py > +++ b/Lib/importlib/machinery.py > @@ -13,3 +13,7 @@ > from ._bootstrap import ExtensionFileLoader > > EXTENSION_SUFFIXES = _imp.extension_suffixes() > + > +def all_suffixes(): > + """Returns a list of all recognized module suffixes for this process""" > + return SOURCE_SUFFIXES + BYTECODE_SUFFIXES + EXTENSION_SUFFIXES > diff --git a/Lib/inspect.py b/Lib/inspect.py > --- a/Lib/inspect.py > +++ b/Lib/inspect.py > @@ -450,8 +450,15 @@ > > def getmodulename(path): > """Return the module name for a given file, or None.""" > - info = getmoduleinfo(path) > - if info: return info[0] > + fname = os.path.basename(path) > + # Check for paths that look like an actual module file > + suffixes = [(-len(suffix), suffix) > + for suffix in importlib.machinery.all_suffixes()] > + suffixes.sort() # try longest suffixes first, in case they overlap > + for neglen, suffix in suffixes: > + if fname.endswith(suffix): > + return fname[:neglen] > + return None > > def getsourcefile(object): > """Return the filename that can be used to locate an object's source. > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -41,6 +41,9 @@ > Library > ------- > > +- Issue #15397: inspect.getmodulename() is now based directly on importlib > + via a new importlib.machinery.all_suffixes() API. > + > - Issue #14635: telnetlib will use poll() rather than select() when > possible > to avoid failing due to the select() file descriptor limit. > > > -- > Repository URL: http://hg.python.org/cpython > [View Less]

2 1

PyBytes_Join
by Eli Bendersky July 18, 2012

July 18, 2012

Hi, PyUnicode_Join is a public C API function. However, its cousin for the bytes object is tucked privately in Objects/bytesobject.c as _PyBytes_Join. Is there any harm in exposing it publicly? Eli

1 1

io.BytesIO slower than monkey-patching io.RawIOBase
by Eli Bendersky July 17, 2012

July 17, 2012

While working on #1767933, Serhiy came up with an observation that "monkey-patching" one of the base classes of io is faster than using BytesIO when in need of a file-like object for writing into. I've distilled it into this standalone test: import io data = [b'a'*10, b'bb'*5, b'ccc'*5] * 10000 def withbytesio(): bio = io.BytesIO() for i in data: bio.write(i) return bio.getvalue() def monkeypatching(): mydata = [] file = io.RawIOBase() file.writable = lambda:… [View More]

5 5

early startup error reporting failure
by Kristján Valur Jónsson July 16, 2012

July 16, 2012

Hi there. I've been busy taking the current beta candidate and merging it into the stackless repo. As expected, things don't just go smoothly and there are the usual startup errors, this being a rather intrusive patch and all that. However, I found that early startup errors were not being reported correctly, so I had do make some changes to fix that. I'm not sure these are the correct fixes, so I'd like to start this here and see if anyone feels responsible. Right: The initial error … [View More]occurs here: if (PyImport_ImportFrozenModule("_frozen_importlib") <= 0) { Py_FatalError("Py_Initialize: can't import _frozen_importlib"); My problem was that the actual exception was not being reported along with the FatalError message. Digging around a bit, I found the cause here: fileobject.c, PyFile_WriteString() } else if (!PyErr_Occurred()) { That is, this function declines to write anything if there is an exception present. My quick and dirty fix was to remove this test and just print even with a present exception. That fixes the issue. But perhaps the _correct_ way is to suppress the exception higher up in the callchain, which is this: > python33_d.dll!PyFile_WriteString(const char * s, _object * f) Line 179 C python33_d.dll!PyTraceBack_Print(_object * v, _object * f) Line 415 + 0x11 bytes C python33_d.dll!print_exception(_object * f, _object * value) Line 1748 + 0x12 bytes C python33_d.dll!print_exception_recursive(_object * f, _object * value, _object * seen) Line 1889 C python33_d.dll!PyErr_Display(_object * exception, _object * value, _object * tb) Line 1913 C python33_d.dll!sys_excepthook(_object * self, _object * args) Line 197 C python33_d.dll!PyCFunction_Call(_object * func, _object * arg, _object * kw) Line 99 + 0x46 bytes C python33_d.dll!PyObject_Call(_object * func, _object * arg, _object * kw) Line 2149 + 0x48 bytes C python33_d.dll!PyEval_CallObjectWithKeywords(_object * func, _object * arg, _object * kw) Line 4584 C python33_d.dll!PyErr_PrintEx(int set_sys_last_vars) Line 1686 + 0x12 bytes C python33_d.dll!Py_FatalError(const char * msg) Line 2358 C Perhaps error should be fetched and restored in PyTraceback_Print(), since it already does some exception juggling, obviously assuming that an exception state can be present that it is worthwhile to preserve. Ok, then I came to the second issue. When printing the tracebacks, this early in the process, I hit upon this code, in traceback.c, tb_displayline(), I made this change (line 344): - return _Py_DisplaySourceLine(f, filename, lineno, 4); + /* ignore IO errors here, IO may not be ready yet */ + if ( _Py_DisplaySourceLine(f, filename, lineno, 4) ) + PyErr_Clear(); + return err; This early in the process, IO cannot be imported so it is impossible to output source line. The source line output is a "bonus" feature anyway and we shouldn't, IMHO, fail outputting tracebacks if we cannot read the code. The actual failure was importing the IO library. Perhaps an alternative fix, then, is to fix the _Py_DisplaySourceLine() so that it deals with failure to import IO in the same way as failure to read the file, i.e. just returns a "success" value of 0. With these changes, I was able to successfully output the error. Hopefully I will be able to debug it too :) Any thoughts? [View Less]

1 1

Re: [Python-Dev] cpython: Actually initialize __main__.__loader__ with loader instances, not the
by Antoine Pitrou July 15, 2012

July 15, 2012

On Sun, 15 Jul 2012 11:10:50 +0200 (CEST) nick.coghlan <python-checkins(a)python.org> wrote: > tstate = PyThreadState_GET(); > interp = tstate->interp; > - loader = PyObject_GetAttrString(interp->importlib, loader_name); > + loader_type = PyObject_GetAttrString(interp->importlib, loader_name); > + if (loader_type == NULL) { > + return -1; > + } > + loader = PyObject_CallFunction(loader_type, "ss", "__main__", filename); I … [View More]

1 0

Re: [Python-Dev] cpython: Take the first step in resolving the messy pkgutil vs importlib edge cases by
by Antoine Pitrou July 15, 2012

July 15, 2012

On Sun, 15 Jul 2012 10:10:07 +0200 (CEST) nick.coghlan <python-checkins(a)python.org> wrote: > > int > +set_main_loader(PyObject *d, const char *filename, const char *loader_name) > +{ This function should be static. Regards Antoine. -- Software development and contracting: http://pro.pitrou.net

1 0

Summary of Python tracker Issues
by Python tracker July 13, 2012

July 13, 2012

ACTIVITY SUMMARY (2012-07-06 - 2012-07-13) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3520 (+25) closed 23603 (+57) total 27123 (+82) Open issues with patches: 1485 Issues opened (58) ================== #14826: urllib2.urlopen fails to load URL http://bugs.python.org/issue14826 reopened by rosslagerwall #15264: … [View More]

1 0

Fwd: How to recruit open-source contributors – The Story of Data
by Steve Holden July 12, 2012

July 12, 2012

Interesting article about Gentoo's approach to developer recruitment from GSoC: http://redmonk.com/dberkholz/2012/07/10/how-to-recruit-open-source-contribu… S -- Steve Holden steve(a)holdenweb.com, Holden Web, LLC http://holdenweb.com/ Python classes (and much more) through the web http://oreillyschool.com/ Conferences and technical event management at http://theopenbastion.com/ Next: DjangoCon US Sep 6-8, Washington DC http://djangocon.us/

2 1