Mailman 3 February 2016 - Python-Dev

Py_SETREF again
by Serhiy Storchaka 12 Feb '16

12 Feb '16

Sorry to bringing this up again. I was hoping we were done with that. When discussing the name of the Py_SETREF macro I was supposed to add a pair of macros: for Py_DECREF and Py_XDECREF. But I got a lot of opinions to be limited to only one macro. On 28.02.14 15:58, Kristján Valur Jónsson wrote: > Also, for the equivalence to hold there is no separate Py_XSETREF, the X > behaviour is implied, which I favour. Enough of this X-proliferation > already! On 16.12.15 16:53, Random832 wrote: > I think "SET" names imply that it's safe if the original > reference is NULL. This isn't an objection to the names, but if > it is given one of those names I think it should use Py_XDECREF. It was my initial intension. But then I had got a number of voices for single macros. On 16.12.15 23:16, Victor Stinner wrote: > I would prefer a single macro to avoid bugs, I don't think that such > macro has a critical impact on performances. It's more designed for > safety, no? On 17.12.15 08:22, Nick Coghlan wrote: >> 1. Py_SETREF > > +1 if it always uses Py_XDECREF on the previous value (as I'd expect > this to work even if the previous value was NULL) There was no (besides my) clearly expressed vote for two macros. As a result I have replaced both Py_DECREF and Py_XDECREF with the macro that always uses Py_XDECREF. Now Raymond, who was not involved in the previous discussions, expressed the view that we should to rename Py_SETREF to Py_XSETREF and add new Py_SETREF that uses Py_DECREF for using in the code that used Py_DECREF previously. [1] We should discuss the need for this, and may be re-discuss the names for the macros. [1] http://bugs.python.org/issue26200

3 3

Windows: Remove support of bytes filenames in the os module?
by Victor Stinner 11 Feb '16

11 Feb '16

Hi, Since 3.3, functions of the os module started to emit DeprecationWarning when called with bytes filenames. The rationale is quite simple: Windows native type for filenames is Unicode, and the Windows has a weird behaviour when you use bytes. For example, os.listdir(b'.') gives you paths which cannot be used with open() on filenames which are not encodable the ANSI code page. Unencodable characters are replaced with "?". The following issue was opened to document this weird behaviour (but the doc was never completed): "Document that bytes OS API can returns unusable results on Windows" http://bugs.python.org/issue16700 When the new os.scandir() API was designed, I asked to *not* support bytes filenames since they are "broken by design". https://www.python.org/dev/peps/pep-0471/ Recently, an user complained that os.walk() doesn't work with bytes on Windows anymore: "Regression: os.walk now using os.scandir() breaks bytes filenames on windows" http://bugs.python.org/issue25911 Serhiy Storchaka just pushed a change to reintroduce support bytes support on Windows in os.walk(), but I would prefer to do the *opposite*: drop supports for bytes filenames on Windows. Are we brave enough to force users to use the "right" type for filenames? -- On Python 2, it wasn't possible to use Unicode for filenames, many functions fail badly with Unicode, especially when you mix bytes and Unicode. On Python 3, Unicode is the "natural" types, most Python functions prefer Unicode, and the PEP 383 (surrogateescape) allows to safetely use Unicode on UNIX even with undecodable filenames (invalid bytes are stored as Unicode surrogate characters). Victor

14 39

PEP 515: Underscores in Numeric Literals
by Georg Brandl 11 Feb '16

11 Feb '16

Hey all, based on the feedback so far, I revised the PEP. There is now a much simpler rule for allowed underscores, with no exceptions. This made the grammar simpler as well. --------------------------------------------------------------------------- PEP: 515 Title: Underscores in Numeric Literals Version: $Revision$ Last-Modified: $Date$ Author: Georg Brandl Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 10-Feb-2016 Python-Version: 3.6 Abstract and Rationale ====================== This PEP proposes to extend Python's syntax so that underscores can be used in integral, floating-point and complex number literals. This is a common feature of other modern languages, and can aid readability of long literals, or literals whose value should clearly separate into parts, such as bytes or words in hexadecimal notation. Examples:: # grouping decimal numbers by thousands amount = 10_000_000.0 # grouping hexadecimal addresses by words addr = 0xDEAD_BEEF # grouping bits into bytes in a binary literal flags = 0b_0011_1111_0100_1110 # making the literal suffix stand out more imag = 1.247812376e-15_j Specification ============= The current proposal is to allow one or more consecutive underscores following digits and base specifiers in numeric literals. The production list for integer literals would therefore look like this:: integer: decimalinteger | octinteger | hexinteger | bininteger decimalinteger: nonzerodigit (digit | "_")* | "0" ("0" | "_")* nonzerodigit: "1"..."9" digit: "0"..."9" octinteger: "0" ("o" | "O") "_"* octdigit (octdigit | "_")* hexinteger: "0" ("x" | "X") "_"* hexdigit (hexdigit | "_")* bininteger: "0" ("b" | "B") "_"* bindigit (bindigit | "_")* octdigit: "0"..."7" hexdigit: digit | "a"..."f" | "A"..."F" bindigit: "0" | "1" For floating-point and complex literals:: floatnumber: pointfloat | exponentfloat pointfloat: [intpart] fraction | intpart "." exponentfloat: (intpart | pointfloat) exponent intpart: digit (digit | "_")* fraction: "." intpart exponent: ("e" | "E") ["+" | "-"] intpart imagnumber: (floatnumber | intpart) ("j" | "J") Alternative Syntax ================== Underscore Placement Rules -------------------------- Instead of the liberal rule specified above, the use of underscores could be limited. Common rules are (see the "other languages" section): * Only one consecutive underscore allowed, and only between digits. * Multiple consecutive underscore allowed, but only between digits. A less common rule would be to allow underscores only every N digits (where N could be 3 for decimal literals, or 4 for hexadecimal ones). This is unnecessarily restrictive, especially considering the separator placement is different in different cultures. Different Separators -------------------- A proposed alternate syntax was to use whitespace for grouping. Although strings are a precedent for combining adjoining literals, the behavior can lead to unexpected effects which are not possible with underscores. Also, no other language is known to use this rule, except for languages that generally disregard any whitespace. C++14 introduces apostrophes for grouping, which is not considered due to the conflict with Python's string literals. [1]_ Behavior in Other Languages =========================== Those languages that do allow underscore grouping implement a large variety of rules for allowed placement of underscores. This is a listing placing the known rules into three major groups. In cases where the language spec contradicts the actual behavior, the actual behavior is listed. **Group 1: liberal** This group is the least homogeneous: the rules vary slightly between languages. All of them allow trailing underscores. Some allow underscores after non-digits like the ``e`` or the sign in exponents. * D [2]_ * Perl 5 (underscores basically allowed anywhere, although docs say it's more restricted) [3]_ * Rust (allows between exponent sign and digits) [4]_ * Swift (although textual description says "between digits") [5]_ **Group 2: only between digits, multiple consecutive underscores** * C# (open proposal for 7.0) [6]_ * Java [7]_ **Group 3: only between digits, only one underscore** * Ada [8]_ * Julia (but not in the exponent part of floats) [9]_ * Ruby (docs say "anywhere", in reality only between digits) [10]_ Implementation ============== A preliminary patch that implements the specification given above has been posted to the issue tracker. [11]_ Open Questions ============== This PEP currently only proposes changing the literal syntax. The following extensions are open for discussion: * Allowing underscores in string arguments to the ``Decimal`` constructor. It could be argued that these are akin to literals, since there is no Decimal literal available (yet). * Allowing underscores in string arguments to ``int()`` with base argument 0, ``float()`` and ``complex()``. References ========== .. [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3499.html .. [2] http://dlang.org/spec/lex.html#integerliteral .. [3] http://perldoc.perl.org/perldata.html#Scalar-value-constructors .. [4] http://doc.rust-lang.org/reference.html#number-literals .. [5] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swif… .. [6] https://github.com/dotnet/roslyn/issues/216 .. [7] https://docs.oracle.com/javase/7/docs/technotes/guides/language/underscores… .. [8] http://archive.adaic.com/standards/83lrm/html/lrm-02-04.html#2.4 .. [9] http://docs.julialang.org/en/release-0.4/manual/integers-and-floating-point… .. [10] http://ruby-doc.org/core-2.3.0/doc/syntax/literals_rdoc.html#label-Numbers .. [11] http://bugs.python.org/issue26331 Copyright ========= This document has been placed in the public domain.

11 13

Re: [Python-Dev] fullOfEels, assistant program for writing Python extension modules in C
by Hugh Fisher 11 Feb '16

11 Feb '16

On Fri, Feb 12, 2016 at 3:30 AM, Nathaniel Smith <njs(a)pobox.com> wrote: > You're almost certainly aware of this, but just to double check since you > don't mention it in the email: cython is also a great tool for handling > similar situations. Not quite the same since in addition to generating all > the boilerplate for you it then lets you use almost-python to actually write > the C implementations as well, and I understand that with your tool you > write the actual implementations in C. But probably also worth considering > in cases where you'd consider this tool, so wanted to make sure it was on > your radar. Yes, cython is a fine tool and I wouldn't try to dissuade anyone from using it if it works for them. FullOfEels is for when the implementation should be hidden altogether. Most often this is because of cross-platform differences or coding horrors, but could also be handy for teaching when it's easier to just give students plain Python modules to look at. Thanks for replying. -- cheers, Hugh Fisher

1 0

fullOfEels, assistant program for writing Python extension modules in C
by Hugh Fisher 11 Feb '16

11 Feb '16

I've written a Python program named fullOfEels to speed up the first stages of writing Python extension modules in C. It is not a replacement for SWIG, SIP, or ctypes. It's for the case where you want to work in the opposite direction, specifying a Python API and then writing an implementation in C. (A small niche maybe, but I hope it isn't just me who sometimes works this way.) The input is a Python module specifying what it should do but not how, with all the functions, classes, and methods being just pass. The output is a pair of .h and .c files with all the boilerplate C code required: module initialization, class type structs, C method functions and method tables. Downloadable from https://bitbucket.org/hugh_fisher/fullofeels All feedback and suggestions welcome. -- cheers, Hugh Fisher

2 1

why we have both re.match and re.string?
by Luca Sangiacomo 10 Feb '16

10 Feb '16

Hi, I hope the question is not too silly, but why I would like to understand the advantages of having both re.match() and re.search(). Wouldn't be more clear to have just one function with one additional parameters like this: re.search(regexp, text, from_beginning=True|False) ? In this way we prevent, as written in the documentation, people writing ".*" in front of the regexp used with re.match() Thanks.

3 2

Experiences with Creating PEP 484 Stub Files
by Phil Thompson 10 Feb '16

10 Feb '16

I've been adding support to the SIP wrapper generator for automatically generating PEP 484 compatible stub files so that future versions of PyQt can be shipped with them. By way of feedback I thought I'd share my experience, confusions and suggestions. There are a number of things I'd like to express but cannot find a way to do so... - objects that implement the buffer protocol - type objects - slice objects - capsules - sequences of fixed size (ie. specified in the same way as Tuple) - distinguishing between instance and class attributes. The documentation is incomplete - there is no mention of Set or Tuple for example. I found the documentation confusing regarding Optional. Intuitively it seems to be the way to specify arguments with default values. However it is explained in terms of (for example) Union[str, None] and I (intuitively but incorrectly) read that as meaning "a str or None" as opposed to "a str or nothing". bytes can be used as shorthand for bytes, bytearray and memoryview - but what about objects that really only support bytes? Shouldn't the shorthand be called something like AnyBytes? Is there any recommended way to test the validity and completeness of stub files? What's the recommended way to parse them? Phil

5 10

Re: [Python-Dev] Windows: Remove support of bytes filenames in the os module?
by Victor Stinner 09 Feb '16

09 Feb '16

Le mercredi 10 février 2016, Steve Dower <python(a)stevedower.id.au> a écrit : > > I really don't like the idea of not being able to use bytes in cross > platform code. Unless it's become feasible to use Unicode for lossless > filenames on Linux - last I heard it wasn't. > The point of my email is that even on Python 3, users kept bad habits because of Python 2. *Yes*, you can use Unicode filenames on all platforms on Python 3 since 2009 thanks to the following PEP: https://www.python.org/dev/peps/pep-0383/ In my first email, I mentioned a bug report of an user still using bytes filenames on Windows with Python 3. It is on the Blender project which *only* supports Python 3. Or maybe I missed something huge which really force Blender to use bytes??? But if a few functions still require bytes, I would suggest to use instead os.fsencode() for them. It's more much convenient to handle filenames as Unicode on Python 3. Victor

1 0

Issue #26204: compiler now emits a SyntaxWarning on constant statement
by Victor Stinner 09 Feb '16

09 Feb '16

Hi, I changed the Python compiler to ignore any kind "constant expressions", whereas it only ignored strings and integers before: http://bugs.python.org/issue26204 The compiler now also emits a SyntaxWarning on such case. IMHO the warning can help to detect bugs for developers who just learnt Python. The warning is *not* emited for strings, since triple quoted strings are a common syntax for multiline comments. The warning is *not* emited neither for ellispis (...) since "f(): ..." is a legit syntax for abstract function. Are you ok with the new warning? New behaviour: haypo@smithers$ ./python Python 3.6.0a0 (default:759a975e1230, Feb 8 2016, 18:21:23) >>> def f(): ... False ... <stdin>:2: SyntaxWarning: ignore constant statement >>> import dis; dis.dis(f) 2 0 LOAD_CONST 0 (None) 3 RETURN_VALUE Old behaviour: haypo@smithers$ python3 Python 3.4.3 (default, Jun 29 2015, 12:16:01) >>> def f(): ... False ... >>> import dis; dis.dis(f) 2 0 LOAD_CONST 1 (False) 3 POP_TOP 4 LOAD_CONST 0 (None) 7 RETURN_VALUE Before strings and numbers were already ignored. Example: haypo@smithers$ python3 Python 3.4.3 (default, Jun 29 2015, 12:16:01) >>> def f(): ... 123 ... >>> import dis; dis.dis(f) 2 0 LOAD_CONST 0 (None) 3 RETURN_VALUE Victor

18 38

Improving docs for len() of set
by Ben Hoyt 08 Feb '16

08 Feb '16

Hi folks, Just a suggestion for a documentation tweak. Currently the docs for len() on a set say this: .. describe:: len(s) Return the cardinality of set *s*. I'm a relatively seasoned programmer, but I don't really have a maths background, and I didn't know what "cardinality" meant. I could kind of grok it by context, but could we change this to something like the following? .. describe:: len(s) Return the number of elements in set *s* (cardinality of *s*). Happy to open a bugs.python.org issue on this, but wanted to get general consensus first. -Ben

5 6