Mailman 3 February 2015 - Python-Dev

Re: [Python-Dev] cpython: Issue #23152: Renames attribute_data_to_stat to _Py_attribute_data_to_stat
by Serhiy Storchaka Feb. 22, 2015

Feb. 22, 2015

On 21.02.15 20:04, steve.dower wrote: > https://hg.python.org/cpython/rev/307713759a62 > changeset: 94720:307713759a62 > parent: 94718:4f6f4aa0d80f > user: Steve Dower <steve.dower(a)microsoft.com> > date: Sat Feb 21 10:04:10 2015 -0800 > summary: > Issue #23152: Renames attribute_data_to_stat to _Py_attribute_data_to_stat What about time_t_to_FILE_TIME?

2 5

Generate all Argument Clinic code into separate files
by Serhiy Storchaka Feb. 22, 2015

Feb. 22, 2015

Currently for some files converted to use Argument Clinic the generated code is written into a separate file, for other files it is inlined. I'm going to make a number of small enhancement to Argument Clinic to generate faster code for parsing arguments (see for example issue23492). Every such step will produce large diffs for generated code and will create code churn if it is inlined and mixed up with handwritten code. It would be better when only generated files will be changed. So I suggest to move all inlined generated code in separate file. What are you think?

3 3

Enum is really serializable using json?
by Facundo Batista Feb. 22, 2015

Feb. 22, 2015

We have this issue closed as resolved: http://bugs.python.org/issue18264 It's called "enum.IntEnum is not compatible with JSON serialisation", and it looks that after a long conversation they added proper support for it in Py3.4. However, this is still failing: Python 3.4.2 (default, Oct 8 2014, 13:18:07) [GCC 4.9.1] on linux >>> import enum, json >>> Foo = enum.Enum('Foo', 'bar baz') >>> json.dumps(Foo.bar) Traceback (most recent call last): ... TypeError: <Foo.bar: 1> is not JSON serializable Am I failing to understand the bug tracker saying the patch was applied for Py3.4? (thought it was more polite to ask here than to reopen the whole thread in the issue tracker) Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

3 3

TypeError messages
by Serhiy Storchaka Feb. 22, 2015

Feb. 22, 2015

Different patterns for TypeError messages are used in the stdlib: expected X, Y found expected X, found Y expected X, but Y found expected X instance, Y found X expected, not Y expect X, not Y need X, Y found X is required, not Y Z must be X, not Y Z should be X, not Y and more. What the pattern is most preferable? Some messages use the article before X or Y. Should the article be used or omitted? Some messages (only in C) truncate actual type name (%.50s, %.80s, %.200s, %.500s). Should type name be truncated at all and for how limit? Type names newer truncated in TypeError messages raised in Python code. Some messages enclose actual type name with single quotes ('%s', '%.200s'). Should type name be quoted? It is uncommon if type name contains spaces.

12 23

[Python-checkins] cpython: Issue #23152: Implement _Py_fstat() to support files larger than 2 GB on
by Ben Hoyt Feb. 21, 2015

Feb. 21, 2015

When merging some changes while working on scandir, I noticed a minor issue with this commit: https://hg.python.org/cpython/rev/4f6f4aa0d80f The definition of "struct win32_stat" has been moved to fileutils.h and renamed to "struct _Py_stat_struct", which is fine -- however, the old "struct win32_stat" definition is still present (but unused) in posixmodule.c. So I think the old "struct win32_stat { ... }" definition can simply be removed from posixmodule.c now. Also, unrelated to this commit, I notice the _Py_attribute_data_to_stat function (was attribute_data_to_stat) can't fail and always returns 0, and all callers ignore its return value anyway. Can it be changed to return void? -Ben

3 2

Prefixes and namespaces
by Serhiy Storchaka Feb. 21, 2015

Feb. 21, 2015

/* Namespaces are one honking great idea -- let's do more of those! */ There are two ways to avoid name conflicts: prefixes and namespaces. Programming languages that lacks namespaces (such as C) need to use prefixes. For example: PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23. Python used the same prefixed names when reflect C constants to module level Python globals. But when convert integer constants to IntEnum, is it needed to preserve common prefix? Or may be we can drop it, because enum class name plays its role? class Protocol(IntEnum): PROTOCOL_SSLv2 = ... PROTOCOL_SSLv3 = ... PROTOCOL_SSLv23 = ... or class Protocol(IntEnum): SSLv2 = ... SSLv3 = ... SSLv23 = ... ? Protocol.PROTOCOL_SSLv2 or Protocol.SSLv2?

4 4

Idea: __length_hint__ wrapper in itertools
by Carlos Pita Feb. 21, 2015

Feb. 21, 2015

Hi all, python now supports the __length_hint__ method but not for every use case that could potentially benefit from it, v.g.: 1) Some common builtins like map don't set the hint. 2) Generators. 3) Application specific hints that, by definition, can't be predicted by a general strategy. I know 1 and 2 were discussed at some length in the past, but still there is no agreement about what to do with them. In the meantime, and because of 3, what do you think about adding a simple iterator wrapper to itertools which would allow to provide the hint by the user without any significant performance lost (__next__ in the wrapper could be just the original __next__). AFAIK there is no such thing in the standard library or anywhere else and it seems pretty easy to implement, although maybe I'm completely wrong. Say: hinted_iter = it.length_hint(iter, hint) Cheers -- Carlos

2 1

PEP 441 - Improving Python ZIP Application Support
by Paul Moore Feb. 21, 2015

Feb. 21, 2015

I'm looking at putting together a patch for CPython to implement PEP 441. In doing so, there are a few issues that I'd like to raise with the PEP. These are all to do with the supporting app "pyzaa" (IIRC, Nick proposed renaming this to "pyzapp", which I like, but it's not a big deal either way). 1. The PEP says that the application will warn if you try to add C extensions to the archive. In practice, this will be quite hard to do - extensions vary across platforms, and it's not only C extensions that won't work from an archive (consider a DLL with a pure-python ctypes wrapper). We could check the filenames for "known" extensions (maybe using importlib.machinery.EXTENSION_SUFFIXES, but that varies by platform) but I think that's going to be fragile. IMO, the costs aren't worth the benefits, and I'd like to remove this proposal and simply document that applications packed up with pyzaa need to be tested to ensure they work from a zipfile. 2. The option to compile Python source seems to me to be of limited use. It ties the archive to a particular Python version (more precisely, only one version gets the benefits of precompiled bytecode). It's also tricky to implement, particularly the option to compile an existing archive, because there standard library py_compile and compileall modules work on the filesystem, and for compiling files in an archive it would be better to build the bytecode in memory (no need to use tempfiles). I'd prefer to say that if users want to include bytecode, they can precompile on disk (using compileall) for as many versions as they want. FWIW, Daniel's standalone pyzaa program doesn't currently include either of the above features. As well as removing things, I'd like to add a programmable API for the utility. At the moment I only have a rough idea of how that would work, but I'm thinking of a PyZApp class, which can be created from a filename or an open stream (like a ZipFile object) and which has methods to set the shebang line, setting the main entry point, and adding files or directories. Does that need to go into the PEP or is it OK to review the API as part of the patch? I don't mind either way, but my normal coding process tends to be to fiddle with the API as I code it up and use it in the main app, so I'd normally leave documenting the final API till it's written... If the above seems like a reasonable plan, I'll work up a patch with the intention that it gets approved and implemented in time for 3.5. Paul

14 60

[X-POST] GSoC 2015 ideas
by Kushal Das Feb. 21, 2015

Feb. 21, 2015

Hi all, In case anyone wants to mentor a student for GSoC 2015, or have an idea on which students can work on during GSoC, please start putting them up in [1]. I am yet to get the edit access, I will put up the initial template text there. You can even send the idea as a reply to this thread. The template for the ideas is given below: == Project name == * '''Project description''': Make sure you have a high-level description that any student can understand, as well as deeper details * '''Skills''': programming languages? specific domain knowledge? * '''Difficulty level''': Easy/Intermediate/Hard classification (students ask for this info frequently to help them narrow down their choices) * '''Related Readings/Links''': was there a mailing list discussion about this topic? standards you want the students to read first? bugs/feature requests? * '''Potential mentors''': A list of mentors likely to be involved with this project, so students know who to look for on IRC/mailing lists if they have questions. (If you've had trouble with students overwhelming specific mentors, feel free to re-iterate here if students should contact the mailing list to reach all mentors.) [1] https://wiki.python.org/moin/SummerOfCode/2015/python-core Kushal -- Fedora Cloud Engineer CPython Core Developer Director Python Software Foundation http://kushaldas.in

1 0

Summary of Python tracker Issues
by Python tracker Feb. 20, 2015

Feb. 20, 2015

ACTIVITY SUMMARY (2015-02-13 - 2015-02-20) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4770 (-20) closed 30484 (+54) total 35254 (+34) Open issues with patches: 2239 Issues opened (24) ================== #23018: Add version info to python http://bugs.python.org/issue23018 reopened by serhiy.storchaka #23460: Decimals do not obey ':g' exponential notation formatting rule http://bugs.python.org/issue23460 opened by ikelly #23462: All os.exec*e variants crash on Windows http://bugs.python.org/issue23462 opened by nicolaje #23464: Remove or deprecate JoinableQueue in asyncio docs http://bugs.python.org/issue23464 opened by emptysquare #23465: Implement PEP 486 - Make the Python Launcher aware of virtual http://bugs.python.org/issue23465 opened by pmoore #23466: PEP 461: Inconsistency between str and bytes formatting of int http://bugs.python.org/issue23466 opened by serhiy.storchaka #23467: Improve byte formatting compatibility between Py2 and Py3 http://bugs.python.org/issue23467 opened by serhiy.storchaka #23469: Delete Misc/*.wpr files http://bugs.python.org/issue23469 opened by berker.peksag #23470: OpenBSD buildbot uses wrong stdlib http://bugs.python.org/issue23470 opened by serhiy.storchaka #23471: 404 Not Found when downloading Python 3.4.3rc1 Documentation http://bugs.python.org/issue23471 opened by stringsonfire #23472: Setup locales on buildbots http://bugs.python.org/issue23472 opened by serhiy.storchaka #23476: SSL cert verify fail for "www.verisign.com" http://bugs.python.org/issue23476 opened by nagle #23477: Increase coverage for wsgiref module http://bugs.python.org/issue23477 opened by ashkop #23484: SemLock acquire() keyword arg 'blocking' is invalid http://bugs.python.org/issue23484 opened by td #23485: PEP 475: handle EINTR in the select and selectors module http://bugs.python.org/issue23485 opened by haypo #23486: Enum member lookup is 20x slower than normal class attribute l http://bugs.python.org/issue23486 opened by craigh #23487: argparse: add_subparsers 'action' broken http://bugs.python.org/issue23487 opened by derks #23488: Random objects twice as big as necessary on 64-bit builds http://bugs.python.org/issue23488 opened by rhettinger #23489: atexit handlers are not executed when using multiprocessing.Po http://bugs.python.org/issue23489 opened by juj #23490: allocation (and overwrite) of a 0 byte buffer http://bugs.python.org/issue23490 opened by pkt #23491: PEP 441 - Improving Python Zip Application Support http://bugs.python.org/issue23491 opened by pmoore #23492: Argument Clinic: improve generated parser for 1-argument funct http://bugs.python.org/issue23492 opened by serhiy.storchaka #23493: optimize sort_keys in json module by using operator.itemgetter http://bugs.python.org/issue23493 opened by wbolster #433028: SRE: (?flag:...) is not supported http://bugs.python.org/issue433028 reopened by serhiy.storchaka Most recent 15 issues with no replies (15) ========================================== #23493: optimize sort_keys in json module by using operator.itemgetter http://bugs.python.org/issue23493 #23492: Argument Clinic: improve generated parser for 1-argument funct http://bugs.python.org/issue23492 #23487: argparse: add_subparsers 'action' broken http://bugs.python.org/issue23487 #23485: PEP 475: handle EINTR in the select and selectors module http://bugs.python.org/issue23485 #23484: SemLock acquire() keyword arg 'blocking' is invalid http://bugs.python.org/issue23484 #23470: OpenBSD buildbot uses wrong stdlib http://bugs.python.org/issue23470 #23469: Delete Misc/*.wpr files http://bugs.python.org/issue23469 #23464: Remove or deprecate JoinableQueue in asyncio docs http://bugs.python.org/issue23464 #23459: Linux: expose the new execveat() syscall http://bugs.python.org/issue23459 #23456: asyncio: add missing @coroutine decorators http://bugs.python.org/issue23456 #23455: file iterator "deemed broken"; can resume after StopIteration http://bugs.python.org/issue23455 #23444: HCI Bluetooth socket bind error on an arm crosscompiled enviro http://bugs.python.org/issue23444 #23443: XMLRPCLIB Exception uses str not class or instance http://bugs.python.org/issue23443 #23436: xml.dom.minidom.Element.ownerDocument is hiden http://bugs.python.org/issue23436 #23423: XPath Support in ElementTree doc omission http://bugs.python.org/issue23423 Most recent 15 issues waiting for review (15) ============================================= #23492: Argument Clinic: improve generated parser for 1-argument funct http://bugs.python.org/issue23492 #23491: PEP 441 - Improving Python Zip Application Support http://bugs.python.org/issue23491 #23490: allocation (and overwrite) of a 0 byte buffer http://bugs.python.org/issue23490 #23488: Random objects twice as big as necessary on 64-bit builds http://bugs.python.org/issue23488 #23486: Enum member lookup is 20x slower than normal class attribute l http://bugs.python.org/issue23486 #23477: Increase coverage for wsgiref module http://bugs.python.org/issue23477 #23469: Delete Misc/*.wpr files http://bugs.python.org/issue23469 #23466: PEP 461: Inconsistency between str and bytes formatting of int http://bugs.python.org/issue23466 #23465: Implement PEP 486 - Make the Python Launcher aware of virtual http://bugs.python.org/issue23465 #23458: [2.7] random: make the file descriptor non-inheritable (on POS http://bugs.python.org/issue23458 #23456: asyncio: add missing @coroutine decorators http://bugs.python.org/issue23456 #23451: Deprecation warnings building 3.5 Visual Studio Windows 8.1 64 http://bugs.python.org/issue23451 #23444: HCI Bluetooth socket bind error on an arm crosscompiled enviro http://bugs.python.org/issue23444 #23441: rlcompleter: tab on empty prefix => insert spaces http://bugs.python.org/issue23441 #23440: Extend http.server.SimpleHTTPRequestHandler testing http://bugs.python.org/issue23440 Top 10 most discussed issues (10) ================================= #22107: tempfile module misinterprets access denied error on Windows http://bugs.python.org/issue22107 16 msgs #21998: asyncio: support fork http://bugs.python.org/issue21998 8 msgs #22928: HTTP header injection in urrlib2/urllib/httplib/http.client http://bugs.python.org/issue22928 8 msgs #23465: Implement PEP 486 - Make the Python Launcher aware of virtual http://bugs.python.org/issue23465 7 msgs #23486: Enum member lookup is 20x slower than normal class attribute l http://bugs.python.org/issue23486 7 msgs #21257: Document parse_headers function of http.client http://bugs.python.org/issue21257 6 msgs #3566: httplib persistent connections violate MUST in RFC2616 sec 8.1 http://bugs.python.org/issue3566 5 msgs #20699: Document that binary IO classes work with bytes-likes objects http://bugs.python.org/issue20699 5 msgs #23425: Windows getlocale unix-like with french, german, portuguese, s http://bugs.python.org/issue23425 5 msgs #23452: Build errors using VS Express 2013 in win32 mode http://bugs.python.org/issue23452 5 msgs Issues closed (53) ================== #4773: HTTPMessage not documented and has inconsistent API across Py2 http://bugs.python.org/issue4773 closed by berker.peksag #9099: multiprocessing/win32: WindowsError: [Error 0] Success on Pipe http://bugs.python.org/issue9099 closed by davin #9119: Python download page needs to mention crypto code in Windows i http://bugs.python.org/issue9119 closed by berker.peksag #9913: Misc/SpecialBuilds.txt is out of date http://bugs.python.org/issue9913 closed by belopolsky #12782: Multiple context expressions do not support parentheses for co http://bugs.python.org/issue12782 closed by barry #14559: (2.7.3 Regression) PC\8.0 directory can no longer be used to http://bugs.python.org/issue14559 closed by steve.dower #14672: Windows installer: add desktop shortcut(s) http://bugs.python.org/issue14672 closed by steve.dower #14910: argparse: disable abbreviation http://bugs.python.org/issue14910 closed by berker.peksag #16111: Python 2.7.3 Windows MSI installer installs the VC++ 9 dlls di http://bugs.python.org/issue16111 closed by loewis #16361: HTTPS/TLS Problem in Python 3.3 http://bugs.python.org/issue16361 closed by loewis #17033: RPM spec file has old config_binsuffix value http://bugs.python.org/issue17033 closed by benjamin.peterson #17753: test_zipfile: requires write access to test and email.test http://bugs.python.org/issue17753 closed by serhiy.storchaka #19105: pprint doesn't use all width http://bugs.python.org/issue19105 closed by serhiy.storchaka #19681: test_repr (test.test_functools.TestPartialC) failures http://bugs.python.org/issue19681 closed by serhiy.storchaka #19792: pathlib does not support symlink in Windows XP http://bugs.python.org/issue19792 closed by steve.dower #20833: scripts\pydocgui.pyw out of date http://bugs.python.org/issue20833 closed by berker.peksag #21548: pydoc -k IndexError on empty docstring http://bugs.python.org/issue21548 closed by berker.peksag #21849: Fix multiprocessing for non-ascii data http://bugs.python.org/issue21849 closed by serhiy.storchaka #21898: .hgignore: Missing ignores for Eclipse/pydev http://bugs.python.org/issue21898 closed by berker.peksag #22235: httplib: TypeError with file() object in ssl.py http://bugs.python.org/issue22235 closed by r.david.murray #22633: Memory disclosure/buffer overread via bug in Py_FrozenMain http://bugs.python.org/issue22633 closed by python-dev #22735: Fix various crashes exposed through mro() customization http://bugs.python.org/issue22735 closed by berker.peksag #22844: test_gdb failure on Debian Wheezy for Z http://bugs.python.org/issue22844 closed by serhiy.storchaka #22883: Get rid of references to PyInt in Py3 sources http://bugs.python.org/issue22883 closed by serhiy.storchaka #22885: Arbitrary code execution vulnerability due to unchecked eval() http://bugs.python.org/issue22885 closed by serhiy.storchaka #23096: Implementation-depended pickling floats with protocol 0 http://bugs.python.org/issue23096 closed by serhiy.storchaka #23146: Incosistency in pathlib between / and \ http://bugs.python.org/issue23146 closed by pitrou #23169: Reflect that PreReq and BuildPreReq are deprecated in the late http://bugs.python.org/issue23169 closed by benjamin.peterson #23233: TypeError in ./setup.py http://bugs.python.org/issue23233 closed by serhiy.storchaka #23239: SSL match_hostname does not accept IP Address http://bugs.python.org/issue23239 closed by pitrou #23408: Pickle tests use incorrect test data http://bugs.python.org/issue23408 closed by serhiy.storchaka #23410: Document more BaseHTTPRequestHandler attributes http://bugs.python.org/issue23410 closed by python-dev #23418: Keep http.server.__all__ up to date http://bugs.python.org/issue23418 closed by berker.peksag #23422: Clarify docs for importlib.import_module() http://bugs.python.org/issue23422 closed by brett.cannon #23424: Unicode character ends interactive session http://bugs.python.org/issue23424 closed by terry.reedy #23437: Make user scripts directory versioned on Windows http://bugs.python.org/issue23437 closed by python-dev #23439: Fixed http.client.__all__ and added a test http://bugs.python.org/issue23439 closed by berker.peksag #23442: http.client.REQUEST_HEADER_FIELDS_TOO_LARGE renamed in 3.5 http://bugs.python.org/issue23442 closed by berker.peksag #23446: Use PyMem_New instead of PyMem_Malloc http://bugs.python.org/issue23446 closed by serhiy.storchaka #23450: Possible loss of data warnings building 3.5 Visual Studio Wind http://bugs.python.org/issue23450 closed by serhiy.storchaka #23457: make test failures http://bugs.python.org/issue23457 closed by dcrs6000(a)gmail.com #23461: Building on windows modifies importlib.h http://bugs.python.org/issue23461 closed by steve.dower #23463: Incorrect behaviour when opening files containing colons on Wi http://bugs.python.org/issue23463 closed by tim.golden #23468: ISO 8601 datetime process http://bugs.python.org/issue23468 closed by belopolsky #23473: Allow namedtuple to be JSON encoded as dict http://bugs.python.org/issue23473 closed by serhiy.storchaka #23474: Enhance locale testing http://bugs.python.org/issue23474 closed by serhiy.storchaka #23475: asyncio: reference leak in test_close_kill_running() http://bugs.python.org/issue23475 closed by haypo #23478: A list arg with default value inside function got appended eac http://bugs.python.org/issue23478 closed by benjamin.peterson #23479: str.format() breaks object duck typing http://bugs.python.org/issue23479 closed by r.david.murray #23480: Minor typo http://bugs.python.org/issue23480 closed by SilentGhost #23481: SSL module should not offer RC4 based cipher suites for client http://bugs.python.org/issue23481 closed by python-dev #23482: sqlite3_d.dll is not included in installer http://bugs.python.org/issue23482 closed by python-dev #23483: python 2.7 builds using icc http://bugs.python.org/issue23483 closed by SilentGhost

1 0

str(IntEnum)
by Demian Brecht Feb. 20, 2015

Feb. 20, 2015

While working on a bug in the issue tracker, I came across something that I thought was a little odd around the behaviour of IntEnum. Should the behaviour of an instance of an IntEnum not be symmetric to an int where possible? For example: >>> class MyEnum(IntEnum): ... FOO = 1 ... >>> MyEnum.FOO == 1 True >>> MyEnum.FOO * 3 == 3 True >>> str(MyEnum.FOO) == str(1) False In my mind, the string representation here should be “1” and not the label. Was this simply an oversight of the specialized IntEnum implementation, or was there a concrete reason for this that I’m not seeing?

2 1

Re: [Python-Dev] cpython: Improve struct cache locality by bring commonly accessed fields close together.
by Antoine Pitrou Feb. 20, 2015

Feb. 20, 2015

On Fri, 20 Feb 2015 08:50:10 +0000 raymond.hettinger <python-checkins(a)python.org> wrote: > https://hg.python.org/cpython/rev/0ba4bcc0a7a6 > changeset: 94696:0ba4bcc0a7a6 > user: Raymond Hettinger <python(a)rcn.com> > date: Fri Feb 20 00:50:04 2015 -0800 > summary: > Improve struct cache locality by bring commonly accessed fields close together. > > files: > Modules/_randommodule.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c > --- a/Modules/_randommodule.c > +++ b/Modules/_randommodule.c > @@ -78,8 +78,8 @@ > > typedef struct { > PyObject_HEAD > + int index; > unsigned long state[N]; > - int index; > } RandomObject; I may be missing something something, but I don't see how that improves cache locality. The index is only ever looked up when the state is also looked up, and since the state is used sequentially there are equal chances for the state lookup to be near the end rather than near the front. Regards Antoine.

1 0

Python 3.4 RPM on AIX
by Blaxton Feb. 18, 2015

Feb. 18, 2015

hi I am having problem building Python RPM on AIX and I think there are several bugs in Python .spec files. README file says download the bz2 version, but only gziped and xz version available from Python website so had to convert the .tgz file to bz2 and copy it over to SOURCES directory. %define config_binsuffix was set to 2.6 and not sure why it should be 2.6 when I am working on 3.4 version. had to change it to to pass the error. %define config_binsuffix none Misc/cheatsheet seems to be decommissioned in new version of Python but still in spec file so commented it. The last error that I am receiving and can't figure the resolution is belowwhy it won't create the required direcotory ?Processing files: python-devel-3.4.2-AIX71 File not found by glob: /var/opt/freeware/tmp/python-3.4.2-root/usr/include/python3.4/*.h File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/lib/python3.4/config PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2 Processing files: python-tools-3.4.2-AIX71 File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/pydoc File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/smtpd.py PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2-AIX71 running rpm -bl option to see missing files are below: rpm -bl /opt/freeware/src/packages/SPECS/python-3.4.spec Processing files: python-3.4.2-AIX71 File not found by glob: /var/opt/freeware/tmp/python-3.4.2-root/opt/freeware/man/man1/python.1* File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/include/python3.4 Processing files: python-devel-3.4.2-AIX71 File not found by glob: /var/opt/freeware/tmp/python-3.4.2-root/usr/include/python3.4/*.h File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/lib/python3.4/config PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2 Processing files: python-tools-3.4.2-AIX71 File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/pydoc File not found: /var/opt/freeware/tmp/python-3.4.2-root/usr/bin/smtpd.py PreReq: rpmlib(VersionedDependencies) <= 3.0.3-1 python = 3.4.2-AIX71 Any idea as how I can fix this ?

5 7

Re: [Python-Dev] [Python-checkins] cpython (3.4): Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
by Stefan Krah Feb. 16, 2015

Feb. 16, 2015

On Mon, Feb 16, 2015 at 11:35:52AM +0000, serhiy.storchaka wrote: > diff --git a/Modules/_testbuffer.c b/Modules/_testbuffer.c > --- a/Modules/_testbuffer.c > +++ b/Modules/_testbuffer.c > @@ -850,7 +850,7 @@ > Py_ssize_t *dest; > Py_ssize_t x, i; > > - dest = PyMem_Malloc(len * (sizeof *dest)); > + dest = PyMem_New(Py_ssize_t, len); > if (dest == NULL) { > PyErr_NoMemory(); > return NULL; This, too, was already protected by len == ndim <= 64. Stefan Krah

2 1

PEP 370 - per-user scripts directory on Windows
by Paul Moore Feb. 16, 2015

Feb. 16, 2015

tl;dr - Having a shared per-user scripts directory on Windows (%APPDATA%/Python/Scripts) causes a number of potential issues when users install the same package in multiple Python versions. I'd like to suggest that this be changed to a versioned directory (%APPDATA%/PythonXY/Scripts) to match all other Python directories on Windows. This issue originally started as a discussion on the pip tracker around making --user the default for pip. See https://github.com/pypa/pip/issues/1668 for details. However, because the user scripts directory is shared across all Python versions, this poses a number of issues: 1. If we "pip install" something like pytest, which version gets the name "py.test.exe" works on a "last install wins" basis, which is not ideal. 2. The above behaviour (and more generally the shared site scripts directory) does not respect the user's "Make this my default Python" choice from the installer, in potentially surprising ways. 3. Uninstalling a package which has been installed in 2 pythons will likely fail, because either the checksum of an installed script does not match what's in the RECORD file, or because a file that should be present has already been uninstalled via a different Python version [1]. I suggest that the per-user scripts directory should be versioned, in exactly the same way as every other Python-related directory on Windows. So we would have %APPDATA%\PythonXY\Scripts alongside %APPDATA%\PythonXY\site-packages. This is a familiar pattern for Windows users, and should not cause significant confusion. It would be good to get this added for Python 3.5, *before* making user installs the default (needed for 3.5 as the default install location for 3.5 will need admin privileges to write). Some additional points: 1. This is a Windows-specific proposal, because Unix has always had scripts installed in a common location (e.g., /usr/local/bin) and commonly-understood solutions already exist on that platform (versioned command names, python -m, system package management, etc). However, these solutions would be a significant change for Windows users. 2. Packages which recommend a user install specifically so that their commands are available regardless of which Python is on PATH would have to find an alternative solution. I doubt there are many such packages, but this should be considered. 3. This proposal does not affect the Python launcher for Windows. People using the launcher exclusively may be currently using "py -m pip" (or whatever) to launch applications, in which case the change of user script directory won't affect them. 4. I am not aware of any tools that rely on the location of the user scripts directory. Such scripts should be using sysconfig in any case. 5. The problems mentioned exist in current Python versions, but haven't been flagged up, presumably because very few people use the user install location - precisely because it's not the default in pip. Changing the pip default would cause more people to encounter the issues, but I don't propose backporting this change, as it's not a bug fix. If it becomes a major issue for pip users on older versions, we can either reconsider this, or pip can decide on its own workaround. Comments? If this is acceptable, I would be willing to prepare a patch for Python 3.5 implementing this. Paul [1] Checking of checksums is not currently implemented, AFAIK, but it is the purpose of the checksums in the RECORD file, and could be implemented in future versions of pip.

11 60

PEP 486: Make the Python Launcher aware of virtual environments
by Paul Moore Feb. 15, 2015

Feb. 15, 2015

On 12 February 2015 at 14:46, Paul Moore <p.f.moore(a)gmail.com> wrote: > If py.exe detected when the environment variable VIRTUAL_ENV was set, > and used that virtualenv as the default Python rather than the > "system" python it normally used, this would be perfect. > > I think I'll write a PEP for this. Do people think it's a reasonable idea? OK, here we go. https://www.python.org/dev/peps/pep-0486/ Reproduced inline below for ease of reading / commenting. It's also available on github at https://github.com/pfmoore/peps should people want to submit corrections or anything there. PEP: 486 Title: Make the Python Launcher aware of virtual environments Version: $Revision$ Last-Modified: $Date$ Author: Paul Moore <p.f.moore(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Feb-2015 Python-Version: 3.5 Post-History: Abstract ======== The Windows installers for Python include a launcher that locates the correct Python interpreter to run (see PEP 397). However, the launcher is not aware of virtual environments (virtualenv [1]_ or PEP 405 based), and so cannot be used to run commands from the active virtualenv. This PEP proposes making the launcher "virtualenv aware". This means that when run without specifying an explicit Python interpreter to use, the launcher will use the currently active virtualenv, if any, before falling back to the configured default Python. Rationale ========= Windows users with multiple copies of Python installed need a means of selecting which one to use. The Python launcher provides this facility by means of a ``py`` command that can be used to run either a configured "default" Python or a specific interpreter, by means of command line arguments. So typical usage would be:: # Run the Python interactive interpreter py # Execute an installed module py -m pip install pytest py -m pytest When using virtual environments, the ``py`` launcher is unaware that a virtualenv is active, and will continue to use the system Python. So different command invocations are needed to run the same commands in a virtualenv:: # Run the Python interactive interpreter python # Execute an installed module (these could use python -m, # which is longer to type but is a little mopre similar to the # launcher approach) pip install pytest py.test Having to use different commands is is error-prone, and in many cases the error is difficult to spot immediately. The PEP proposes making the ``py`` command usable with virtual environments, so that the first form of command can be used in all cases. Implementation ============== Both ``virtualenv`` and the core ``venv`` module set an environment variable ``VIRTUAL_ENV`` when activating a virtualenv. This PEP proposes that the launcher checks for the ``VIRTUAL_ENV`` environment variable whenever it would run the "default" Python interpreter for the system (i.e., when no specific version flags such as ``py -2.7`` are used) and if present, run the Python interpreter for the virtualenv rather than the default system Python. The "default" Python interpreter referred to above is (as per PEP 397) either the latest version of Python installed on the system, or a version configured via the ``py.ini`` configuration file. When the user specifies an explicit Python version on the command line, this will always be used (as at present). Impact on Script Launching ========================== As well as interactive use, the launcher is used as the Windows file association for Python scripts. In that case, a "shebang" (``#!``) line at the start of the script is used to identify the interpreter to run. A fully-qualified path can be used, or a version-specific Python (``python3`` or ``python2``, or even ``python3.5``), or the generic ``python``, which means to use the default interpreter. With the proposed change, scripts that start with ``#!python`` (or one of its equivalent forms) will be run using an active virtualenv. This is a change in behaviour, although it will only affect users running scripts from the command line with a virtualenv activated. Under Unix, the ``#!/usr/bin/env python`` shebang line will use the version of Python found on ``$PATH``, whereas ``#!/usr/bin/python`` will use the system Python. In order to match Unix behaviour as closely as possible, it is proposed that the two shebang forms:: #!/usr/bin/env python #!python use an active virtualenv, if present, whereas the forms:: #!/usr/bin/python #!/usr/local/bin/python use *only* the default system Python, and ignore the ``VIRTUAL_ENV`` environment variable. Exclusions ========== The PEP makes no attempt to promote the use of the launcher for running Python on Windows. Most existing documentation assumes the user of ``python`` as the command to run Python, and (for example) ``pip`` to run an installed Python command. This documentation is not expected to change, and users who choose to manage their ``PATH`` environment variable can continue to use this form. The focus of this PEP is purely on allowing users who prefer to use the launcher when dealing with their system Python installations, to be able to continue to do so when using virtual environments. References ========== .. [1] https://virtualenv.pypa.io/ Copyright ========= This document has been placed in the public domain.

3 5

Re: [Python-Dev] subclassing builtin data structures
by Isaac Schwabacher Feb. 15, 2015

Feb. 15, 2015

On 15-02-13, Guido van Rossum wrote: > Are you willing to wait 10 days for an answer? I'm out of round tuits for a while. IIUC, the argument is that the Liskov Substitution Principle is a statement about how objects of a subtype behave relative to objects of a supertype, and it doesn't apply to constructors because they aren't behaviors of existing objects. So other overriding methods *should* be able to handle the same inputs that the respective overridden methods do, but constructors don't need to. Even though __init__ is written as an instance method, it seems like it's "morally" a part of the class method __new__ that's only split off for convenience. If this message is unclear, it's because I don't really understand this myself and I'm trying to articulate my best understanding of what's been said on this thread and those it links to. ijs > On Fri, Feb 13, 2015 at 10:22 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com(javascript:main.compose()> wrote: > > > > > On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky <alexander.belopolsky@gmail.com(javascript:main.compose()> wrote: > > >> > > >> FWIW you're wrong when you claim that "a constructor is no different from any other method". Someone else should probably explain this (it's an old argument that's been thoroughly settled). > > > > > > > > > Well, the best answer I've got in the past [1] was "ask on python-dev since Guido called the operator overriding expectation." :-) > > > > > > And let me repost this bit of history [1]: > > > > Here is the annotated pre-r82065 code: > > > > 39876 gvanrossum def __add__(self, other): > > 39876 gvanrossum if isinstance(other, timedelta): > > 39928 gvanrossum return self.__class__(self.__days + other.__days, > > 39876 gvanrossum self.__seconds + other.__seconds, > > 39876 gvanrossum self.__microseconds + other.__microseconds) > > 40207 tim_one return NotImplemented > > 39876 gvanrossum > > > > > > > > [1] http://bugs.python.org/issue2267#msg125979 > > > > > > > > > -- > --Guido van Rossum (python.org/~guido(http://python.org/~guido))

9 15

Can I replace distutils.msvccompiler?
by Steve Dower Feb. 15, 2015

Feb. 15, 2015

I'm about to start doing a rework on distutils.msvc[9]compiler.MSVCCompiler for Python 3.5 to be able to handle forward-compatibility better. Right now I've tweaked msvc9compiler enough for VS 2015, but it won't handle later VS versions automatically. Is there any reason to keep the old msvccompiler and msvc9compiler modules around? If possible I'd like to drop them completely and just have one implementation (probably distutils.msvccompiler.MSVCCompiler again). Obviously it will be behaviourally compatible with 3.4 and compatible with the CCompiler interface, but it may not be function-for-function compatible with distutils.msvc9compiler (e.g. get_build_version is not useful). Do we need a deprecation period? msvccompiler is mentioned in the docs, but not documented, and msvc9compiler is not even mentioned. Thanks, Steve

1 0

PEP 471 (scandir): Add a new DirEntry.inode() method?
by Victor Stinner Feb. 14, 2015

Feb. 14, 2015

Hi, TL;DR: on POSIX, is it useful to know the inode number (st_ino) without the device number (st_dev)? While reading feedback on the Python 3.5 alpha 1 release, I saw a comment saying that the current design of os.scandir() (PEP 471) doesn't fit a very specific usecase where the inode number is needed: "Ah, turns out we needed even more optimizations than that is able to give us; in particular, the underlying system readdir call gives us the inode number, which we need to compare against a cache of hard links, in order to avoid having to stat the underlying files if we've already done so on another hard link. It looks like the DirEntry API used here only includes the path and name, not the inode number, without invoking another stat call, and we needed to optimize out that extra stat call." https://www.reddit.com/r/Python/comments/2synry/so_8_peps_are_currently_bei… Since the C function readdir() provides the inode number (d_ino field of the dirent structure), I propose add a new DirEntry.inode() method. *** Now the real question: is it useful to know the inode number (st_ino) without the device number (st_dev)? *** On POSIX, you can still get the st_dev from DirEntry.stat(), but it always require a system call. So you loose the whole purpose of DirEntry (no extra syscall). I wrote a script script check_stdev.py, attached to this email, to check if all entries of a directory have the same st_dev value than the directory itself: - same for /usr/bin, /usr/lib, /tmp, /proc, ... - different for /dev What about "union" file systems like UnionFS or thinks like "mount -o bind"? Can someone test? Does anyone have some information? So the answer looks to be: it's useful for all directories except of /dev. Example: --- /dev/hugepages st_dev is different: 35 vs 5 /dev/mqueue st_dev is different: 13 vs 5 /dev/pts st_dev is different: 11 vs 5 /dev/shm st_dev is different: 17 vs 5 --- On POSIX, DirEntry.inode() just exposes the d_ino value from readdir(). On Windows, FirstFindFileW/FindFindFileW returns almost a full stat_result structure, except of st_ino, st_dev and st_nlink fields which are set to 0. So DirEntry.inode() has to call os.lstat() to read the inode number. The inode number will be cached by DirEntry.inode() in the DirEntry object, but the os.lstat() result is dropped. On Windows, I don't want to cache the full os.lstat() result from DirEntry.inode() into DirEntry to replace the previous incomplete stat_result from FirstFindFileW/FindFindFileW, because DirEntry.stat() would return a different result (st_ino, st_dev, st_nlink fields set or not) depending if the inode() methode was called or not. Note: scandir-6.patch of http://bugs.python.org/issue22524 contains an implementation of os.scandir() with DirEntry.inode(), if you want to play. Victor

8 14

PEP 471 (scandir): Poll to choose the implementation (full C or C+Python)
by Victor Stinner Feb. 14, 2015

Feb. 14, 2015

Hi, TL,DR: are you ok to add 800 lines of C code for os.scandir(), 4x faster than os.listdir() when the file type is checked? I accepted the PEP 471 (os.scandir) a few months ago, but it is not implement yet in Python 3.5, because I didn't make a choice on the implementation. Ben Hoyt wrote different implementations: - full C: os.scandir() and DirEntry are written in C (no change on os.py) - C+Python: os._scandir() (wrapper for opendir/readdir and FindFirstFileW/FindNextFileW) in C, DirEntry in Python - ctypes: os.scandir() and DirEntry fully implemented in Python I'm not interested by the ctypes implementation. It's useful for a third party project hosted at PyPI, but for CPython I prefer to wrap C functions using C code. In short, the C implementation is faster than the C+Python implementation. The issue #22524 (*) is full of benchmark numbers. IMO the most interesting benchmark is to compare os.listdir() + os.stat() versus os.scandir() + Direntry.is_dir(). Let me try to summarize results of this benchmark: * C implementation: scandir is at least 3.5x faster than listdir, up to 44.6x faster on Windows * C+Python implementation: scandir is not really faster than listdir, between 1.3x and 1.4x faster (*) http://bugs.python.org/issue22524 Ben Hoyt reminded me that os.scandir() (PEP 471) doesn't add any new feature: pathlib already provides a nice API on top of os and os.path modules. (You may even notice that DirEntry a much fewer methods ;-)) The main (only?) purpose of the PEP is performance. If os.scandir() is "only" 1.4x faster, I don't think that it is interesting to use os.scandir() in an application. I guess that all applications/libraries will want to keep compatibility with Python 3.4 and older and so will anyway have to duplicate the code to use os.listdir() + os.stat(). So is it worth to duplicate code for such small speedup? Now I see 3 choices: - take the full C implementation, because it's much faster (at least 3.4x faster!) - reject the whole PEP 471 (not nice), because it adds too much code for a minor speedup (not true on Windows: up to 44x faster!) - take the C+Python implementation, because maintenance matters more than performances (only 1.3x faster, sorry) => IMO the best option is to take the C implementation. What do you think? I'm concerned by the length of the C code: the full C implementations adds ~800 lines of C code to posixmodule.c. This file is already the longest C file in CPython. I don't want to make it longer, but I'm not motived to start to split it. Last time I proposed to split a file (unicodeobject.c), some developers complained that it makes search harder. I don't understand this, there are so many tools to navigate in C code. But it was enough for me to give up on this idea. A alternative is to add a new _scandir.c module to host the new C code, and share some code with posixmodule.c: remove "static" keyword from required C functions (functions to convert Windows attributes to a os.stat_result object). That's a reasonable choice. What do you think? FYI I ran the benchmark on different hardware (SSD, HDD, tmpfs), file systems (ext4, tmpfs, NFS/ext4), operating systems (Linux, Windows). Victor

11 19

subclassing builtin data structures
by Ethan Furman Feb. 14, 2015

Feb. 14, 2015

I suspect the last big hurdle to making built-in data structures nicely subclassable is the insistence of such types to return new instances as the base class instead of the derived class. In case that wasn't clear ;) --> class MyInt(int): ... def __repr__(self): ... return 'MyInt(%d)' % self ... --> m = MyInt(42) --> m MyInt(42) --> m + 1 43 --> type(m+1) <class 'int'> Besides the work it would take to rectify this, I imagine the biggest hurdle would be the performance hit in always looking up the type of self. Has anyone done any preliminary benchmarking? Are there other concerns? -- ~Ethan~

12 27

Re: [Python-Dev] subclassing builtin data structures
by Isaac Schwabacher Feb. 14, 2015

Feb. 14, 2015

Hmm... super *is* a problem, because if we super, we can end up asking superclasses that we then won't know how to mimic. So what we really want is to move the superclasses we can mimic to the front of the MRO. If none of those can indirectly mimic the base class, then we try the other classes in the MRO to see if we can get a partial upgrade. On 15-02-13, Neil Girdhar wrote: > I think it works as Isaac explained if __make_me__ is an instance method that also accepts the calling class type. > > On Fri, Feb 13, 2015 at 8:12 PM, Ethan Furman <Python-Dev(a)python.org <ethan(a)stoneleaf.us')" target="1">ethan(a)stoneleaf.us> wrote: > > > On 02/13/2015 02:31 PM, Serhiy Storchaka wrote: > > > On 13.02.15 05:41, Ethan Furman wrote: > > >> So there are basically two choices: > > >> > > >> 1) always use the type of the most-base class when creating new instances > > >> > > >> pros: > > >> - easy > > >> - speedy code > > >> - no possible tracebacks on new object instantiation > > >> > > >> cons: > > >> - a subclass that needs/wants to maintain itself must override all > > >> methods that create new instances, even if the only change is to > > >> the type of object returned > > >> > > >> 2) always use the type of self when creating new instances > > >> > > >> pros: > > >> - subclasses automatically maintain type > > >> - much less code in the simple cases [1] > > >> > > >> cons: > > >> - if constructor signatures change, must override all methods which > > >> create new objects > > > > > > And switching to (2) would break existing code which uses subclasses with constructors with different signature (e.g. > > > defaultdict). > > > > I don't think defaultdict is a good example -- I don't see any methods on it that return a new dict, default or > > otherwise. So if this change happened, defaultdict would have to have its own __add__ and not rely on dict's __add__. > > > > > > > The third choice is to use different specially designed constructor. > > > > > > class A(int): > > > > > > --> class A(int): > > > ... def __add__(self, other): > > > ... return self.__make_me__(int(self) + int(other)) > > > > > > ... def __repr__(self): > > > ... return 'A(%d)' % self > > > > How would this help in the case of defaultdict? __make_me__ is a class method, but it needs instance info to properly > > create a new dict with the same default factory. > > > > -- > > ~Ethan~ > > > > > > _______________________________________________ > > Python-Dev mailing list > > https://mail.python.org/mailman/listinfo/python-dev(javascript:main.compose('new', 't=Python-Dev(a)python.org> > > <a href=) > > Unsubscribe: https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com > > > >

1 0

Re: [Python-Dev] subclassing builtin data structures
by Isaac Schwabacher Feb. 14, 2015

Feb. 14, 2015

On 15-02-13, Neil Girdhar wrote: > Unlike a regular method, you would never need to call super since you should know everyone that could be calling you. Typically, when you call super, you have something like this: > > A < B, C > > > B < D > > > so you end up with > > > mro: A, B, C, D > > > And then when A calls super and B calls super it gets C which it doesn't know about. But C calls super and gets D. The scenario I'm concerned with is that A knows how to mimic B's constructor and B knows how to mimic D's, but A doesn't know about D. So D asks A if it knows how to mimic D's constructor, and it says no. Via super, B gets a shot, and it does know, so it translates the arguments to D's constructor into arguments to B's constructor, and again asks A if it knows how to handle them. Then A says yes, translates the args, and constructs an A. If C ever gets consulted, it responds "I don't know a thing" and calls super. > But in the case of make_me, it's someone like C who is calling make_me. If it gets a method in B, then that's a straight-up bug. make_me needs to be reimplemented in A as well, and A would never delegate up since other classes in the mro chain (like B) might not know about C. This scheme (as I've written it) depends strongly on all the classes in the MRO having __make_me__ methods with this very precisely defined structure: test base against yourself, then any superclasses you care to mimic, then call super. Any antisocial superclass ruins everyone's party. > Best, > Neil > > > On Fri, Feb 13, 2015 at 7:00 PM, Isaac Schwabacher <alexander.belopolsky(a)gmail.com <ischwabacher(a)wisc.edu')" target="1">ischwabacher(a)wisc.edu> wrote: > > > On 15-02-13, Neil Girdhar wrote: > > > I personally don't think this is a big enough issue to warrant any changes, but I think Serhiy's solution would be the ideal best with one additional parameter: the caller's type. Something like > > > > > > def __make_me__(self, cls, *args, **kwargs) > > > > > > > > > and the idea is that any time you want to construct a type, instead of > > > > > > > > > self.__class__(assumed arguments…) > > > > > > > > > where you are not sure that the derived class' constructor knows the right argument types, you do > > > > > > > > > def SomeCls: > > > def some_method(self, ...): > > > return self.__make_me__(SomeCls, assumed arguments…) > > > > > > > > > Now the derived class knows who is asking for a copy. In the case of defaultdict, for example, he can implement __make_me__ as follows: > > > > > > > > > def __make_me__(self, cls, *args, **kwargs): > > > if cls is dict: return default_dict(self.default_factory, *args, **kwargs) > > > return default_dict(*args, **kwargs) > > > > > > > > > essentially the caller is identifying himself so that the receiver knows how to interpret the arguments. > > > > > > > > > Best, > > > > > > > > > Neil > > > > Such a method necessarily involves explicit switching on classes... ew. > > Also, to make this work, a class needs to have a relationship with its superclass's superclasses. So in order for DefaultDict's subclasses not to need to know about dict, it would need to look like this: > > > > class DefaultDict(dict): > > ....@classmethod # instance method doesn't make sense here > > ....def __make_me__(cls, base, *args, **kwargs): # make something like base(*args, **kwargs) > > ........# when we get here, nothing in cls.__mro__ above DefaultDict knows how to construct an equivalent to base(*args, **kwargs) using its own constructor > > ........if base is DefaultDict: > > ............return DefaultDict(*args, **kwargs) # if DefaultDict is the best we can do, do it > > ........elif base is dict: > > ............return cls.__make_me__(DefaultDict, None, *args, **kwargs) # subclasses that know about DefaultDict but not dict will intercept this > > ........else: > > ............super(DefaultDict, cls).__make_me__(base, *args, **kwargs) # we don't know how to make an equivalent to base.__new__(*args, **kwargs), so keep looking > > > > I don't even think this is guaranteed to construct an object of class cls corresponding to a base(*args, **kwargs) even if it were possible, since multiple inheritance can screw things up. You might need to have an explicit list of "these are the superclasses whose constructors I can imitate", and have the interpreter find an optimal path for you. > > > > > On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky <http://stackoverflow.com/questions/5490824/should-constructors-comply-with-…'new', 't=alexander.belopolsky(a)gmail.com>(java_script:main.compose()> wrote: > > > > > > > > > > > On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar <mistersheik(a)gmail.com <mistersheik(a)gmail.com>(java_script:main.compose()> wrote: > > > > > > > > > Interesting: > > Not every language allows you to call self.__class__(). In the languages that don't you can get away with incompatible constructor signatures. > > > > > > > > > > > > However, let me try to focus the discussion on a specific issue before we go deep into OOP theory. > > > > > > > > > > > > With python's standard datetime.date we have: > > > > > > > > > > > > >>> from datetime import * > > > > >>> class Date(date): > > > > ... pass > > > > ... > > > > >>> Date.today() > > > > Date(2015, 2, 13) > > > > >>> Date.fromordinal(1) > > > > Date(1, 1, 1) > > > > > > > > > > > > Both .today() and .fromordinal(1) will break in a subclass that redefines __new__ as follows: > > > > > > > > > > > > >>> class Date2(date): > > > > ... def __new__(cls, ymd): > > > > ... return date.__new__(cls, *ymd) > > > > ... > > > > >>> Date2.today() > > > > Traceback (most recent call last): > > > > File "<stdin>", line 1, in <module> > > > > TypeError: __new__() takes 2 positional arguments but 4 were given > > > > >>> Date2.fromordinal(1) > > > > Traceback (most recent call last): > > > > File "<stdin>", line 1, in <module> > > > > TypeError: __new__() takes 2 positional arguments but 4 were given > > > > > > > > > > > > > > > > > > > > Why is this acceptable, but we have to sacrifice the convenience of having Date + timedelta > > > > return Date to make it work with Date2: > > > > > > > > > > > > >>> Date2((1,1,1)) + timedelta(1) > > > > datetime.date(1, 1, 2) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >

2 1

Re: [Python-Dev] subclassing builtin data structures
by Isaac Schwabacher Feb. 14, 2015

Feb. 14, 2015

On 15-02-13, Neil Girdhar wrote: > I personally don't think this is a big enough issue to warrant any changes, but I think Serhiy's solution would be the ideal best with one additional parameter: the caller's type. Something like > > def __make_me__(self, cls, *args, **kwargs) > > > and the idea is that any time you want to construct a type, instead of > > > self.__class__(assumed arguments…) > > > where you are not sure that the derived class' constructor knows the right argument types, you do > > > def SomeCls: > def some_method(self, ...): > return self.__make_me__(SomeCls, assumed arguments…) > > > Now the derived class knows who is asking for a copy. In the case of defaultdict, for example, he can implement __make_me__ as follows: > > > def __make_me__(self, cls, *args, **kwargs): > if cls is dict: return default_dict(self.default_factory, *args, **kwargs) > return default_dict(*args, **kwargs) > > > essentially the caller is identifying himself so that the receiver knows how to interpret the arguments. > > > Best, > > > Neil Such a method necessarily involves explicit switching on classes... ew. Also, to make this work, a class needs to have a relationship with its superclass's superclasses. So in order for DefaultDict's subclasses not to need to know about dict, it would need to look like this: class DefaultDict(dict): ....@classmethod # instance method doesn't make sense here ....def __make_me__(cls, base, *args, **kwargs): # make something like base(*args, **kwargs) ........# when we get here, nothing in cls.__mro__ above DefaultDict knows how to construct an equivalent to base(*args, **kwargs) using its own constructor ........if base is DefaultDict: ............return DefaultDict(*args, **kwargs) # if DefaultDict is the best we can do, do it ........elif base is dict: ............return cls.__make_me__(DefaultDict, None, *args, **kwargs) # subclasses that know about DefaultDict but not dict will intercept this ........else: ............super(DefaultDict, cls).__make_me__(base, *args, **kwargs) # we don't know how to make an equivalent to base.__new__(*args, **kwargs), so keep looking I don't even think this is guaranteed to construct an object of class cls corresponding to a base(*args, **kwargs) even if it were possible, since multiple inheritance can screw things up. You might need to have an explicit list of "these are the superclasses whose constructors I can imitate", and have the interpreter find an optimal path for you. > On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky <alexander.belopolsky@gmail.com(javascript:main.compose()> wrote: > > > > > On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar <mistersheik@gmail.com(javascript:main.compose()> wrote: > > > > > Interesting: http://stackoverflow.com/questions/5490824/should-constructors-comply-with-… > > > > > > > > > Let me humbly conjecture that the people who wrote the top answers have background in less capable languages than Python. > > > > > > Not every language allows you to call self.__class__(). In the languages that don't you can get away with incompatible constructor signatures. > > > > > > However, let me try to focus the discussion on a specific issue before we go deep into OOP theory. > > > > > > With python's standard datetime.date we have: > > > > > > >>> from datetime import * > > >>> class Date(date): > > ... pass > > ... > > >>> Date.today() > > Date(2015, 2, 13) > > >>> Date.fromordinal(1) > > Date(1, 1, 1) > > > > > > Both .today() and .fromordinal(1) will break in a subclass that redefines __new__ as follows: > > > > > > >>> class Date2(date): > > ... def __new__(cls, ymd): > > ... return date.__new__(cls, *ymd) > > ... > > >>> Date2.today() > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > TypeError: __new__() takes 2 positional arguments but 4 were given > > >>> Date2.fromordinal(1) > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > TypeError: __new__() takes 2 positional arguments but 4 were given > > > > > > > > > > Why is this acceptable, but we have to sacrifice the convenience of having Date + timedelta > > return Date to make it work with Date2: > > > > > > >>> Date2((1,1,1)) + timedelta(1) > > datetime.date(1, 1, 2) > > > > > > > > > > > > > > > > > > > >

2 1

Building on Windows - importlib.h changed
by Paul Moore Feb. 13, 2015

Feb. 13, 2015

I'm working on a patch for the Python launcher. I built Python (current tip, on MS Windows, with VS 2015), and I've just noticed that hg status shows: >>hg status -mard M Doc\using\windows.rst M PC\launcher.c M Python\importlib.h I didn't change importlib.h, and I don't see that the changes I did make would affect importlib. I presume I'm OK *not* including the importlib.h change in my patch? Paul PS By the way, the Windows build process is beautifully clean these days. Many thanks to Steve Dower and whoever else has worked on streamlining it!

3 4