Mailman 3 August 2016 - Python-ideas

Sequence views
by Serhiy Storchaka 04 Aug '16

04 Aug '16

Maybe it's time to add a new module for sequence-specific functions (seqtools?). It should contain at least two classes or fabric functions: 1. A view that represents a sliced subsequence. Lazy equivalent of seq[start:end:step]. This feature is implemented in third-party module dataview [1]. 2. A view that represents a linear sequence as 2D array. Iterating this view emits non-intersecting chunks of the sequence. For example it can be used for representing the bytes object as a sequence of 1-byte bytes objects (as in 2.x), a generalized alternative to iterbytes() from PEP 467 [2]. Neither itertools nor collections modules look good place for these features, since they are not concrete classes and work only with sequences, not general iterables or iterators. On other side, mappingproxy and ChainMap look close, maybe new module should be oriented not on sequences, but on views. [1] https://pypi.python.org/pypi/dataview [2] https://www.python.org/dev/peps/pep-0467

8 11

proper naming of Enum members
by Ethan Furman 04 Aug '16

04 Aug '16

There are currently a few locations in the stdlib, such as http and socket, that are now using Enums to replace constants; those names are all upper-case -- those aren't the names I am speaking of. The names I am speaking of are those in brand-new enumerations where we have full control. As an example: class FederalHoliday(AutoNumberEnum): NewYear = "First day of the year.", 'absolute', JANUARY, 1 MartinLutherKingJr = "Birth of Civil Rights leader.", 'relative', JANUARY, MONDAY, 3 President = "Birth of George Washington", 'relative', FEBRUARY, MONDAY, 3 Memorial = "Memory of fallen soldiers", 'relative', MAY, MONDAY, 5 Independence = "Declaration of Independence", 'absolute', JULY, 4 Labor = "American Labor Movement", 'relative', SEPTEMBER, MONDAY, 1 Columbus = "Americas discovered", 'relative', OCTOBER, MONDAY, 2 Veterans = "Recognition of Armed Forces service", 'relative', NOVEMBER, 11, 1 Thanksgiving = "Day of Thanks", 'relative', NOVEMBER, THURSDAY, 4 Christmas = "Birth of Jesus Christ", 'absolute', DECEMBER, 25 def __init__(self, doc, type, month, day, occurance=None): self.__doc__ = doc self.type = type self.month = month self.day = day self.occurance = occurance def date(self, year): """ returns the observed date of the holiday for `year` """" ... @classmethod def next_business_day(cls, date, days=1): """ Return the next `days` business day from date. """ ... @classmethod def count_business_days(cls, date1, date2): """ Return the number of business days between 'date1' and 'date2'. """ ... @classmethod def year(cls, year): """ Return a list of the actual FederalHoliday dates for `year`. """ ... Take the name "NewYear": if it had been a global constant I would have named it "NEWYEAR"; if it had been a normal class attribute I would have named it "new_year"; however, being an Enum member, it is neither of those things. <context switch> I've written some custom data types as part of my dbf package, and a few of them have instances that are singletons that are created in the global (okay, module) namespace, and for them I followed Python's lead in naming singletons: Python has used Title Case in such things as None, True, and False, so I followed suit and named mine -- Null, NullDate, NullTime, NullDateTime, etc. </context switch> Given my past history with using and creating singleton objects, I followed suit when creating my own Enum classes. I was recently queried about my apparent break with PEP 8 for naming Enum members, to which I replied: > Considering the strange beast that an Enum is, there is not much precedent for it anywhere. > > Consider: > > - Enum is a class > - but it is a container > - and can be iterated over > - and it has a length (which can be zero) > - but it's always True in a boolean sense > > - Enum members are instances of the Enum class > - but are pre-created > - and new ones cannot be created > - but are available as attributes on the class > > Given all that I have been using Title case (or CamelCase) to name the members as it helps > distinguish an Enum member from an ordinary attribute (which Enum classes can also have). I forgot to include in that reply that I think CamelCase also helps to emphasize the special singleton nature of Enum members. My question for the community: Your thoughts/opinions of my reasoning, and if you don't agree then which casing choice would you recommend and use, and why? (Reminder: this question does not include Enums whose names are replacements for existing constants and so the names cannot be changed.) -- ~Ethan~

8 8

Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484
by אלעזר 04 Aug '16

04 Aug '16

> Date: Wed, 3 Aug 2016 16:53:55 -0700 > From: Guido van Rossum > > On Wed, Aug 3, 2016 at 3:02 PM, Pavol Lisy <pavol.lisy(a)gmail.com> wrote: > > > def fnc(): > > global a: list > > I'm not proposing to add such syntax, and the right place for the type > of a would be at the global level, not on the `global` syatement. > If 'a' is only read, it looks like a sensible way to state that this function should be called only when 'a' is a list. It's not the same as a global type declaration. More like a parameter type hint. If 'a' is written to, it is very similar to the 'if else' example. (which, if I may add, is usually defined as the meet of the assertions. If we talk about abstract interpretation frameworks)

1 0

Making the stdlib consistent again
by Ralph Broenink 04 Aug '16

04 Aug '16

Hi python-ideas, As you all know, the Python stdlib can sometimes be a bit of an inconsistent mess that can be surprising in how it names things. This is mostly caused by the fact that several modules were developed before the introduction of PEP-8, and now we're stuck with the older naming within these modules. It has been said and discussed in the past [1][2] that the stdlib is in fact inconsistent, but fixing this has almost always been disregarded as being too painful (after all, we don't want a new Python 3 all over again). However, this way, we will never move away from these inconsistencies. Perhaps this is fine, but I think we should at least consider providing function and class names that are unsurprising for developers. While maintaining full backwards compatibility, my idea is that we should offer consistently named aliases in -eventually- all stdlib modules. For instance, with Python 2.6, the threading module received this treatment, but unfortunately this was not expanded to all modules. What am I speaking of precisely? I have done a quick survey of the stdlib and found the following examples. Please note, this is a highly opinionated list; some names may have been chosen with a very good reason, and others are just a matter of taste. Hopefully you agree with at least some of them: * The CamelCasing in some modules are the most obvious culprits, e.g. logging and unittest. There is obviously an issue regarding subclasses and methods that are supposed to be overridden, but I feel we could make it work. * All lower case class names, such as collections.defaultdict and collections.deque, should be CamelCased. Another example is datetime, which uses names such as timedelta instead of TimeDelta. * Inconsistent names all together, such as re.sub, which I feel should be re.replace (cf. str.replace). But also re.finditer and re.findall, but no re.find. * Names that do not reflect actual usage, such as ssl.PROTOCOL_SSLv23, which can in fact not be used as client for SSLv2. * Underscore usage, such as tarfile.TarFile.gettarinfo (should it not be get_tar_info?), http.client.HTTPConnection.getresponse vs set_debuglevel, and pathlib.Path.samefile vs pathlib.Path.read_text. And is it pkgutil.iter_modules or is it pathlib.Path.iterdir (or re.finditer)? * Usage of various abbreviations, such as in filecmp.cmp * Inconsistencies between similar modules, e.g. between tarfile.TarFile.add and zipfile.ZipFile.write. These are just some examples of inconsistent and surprising naming I could find, other categories are probably also conceivable. Another subject for reconsideration would be attribute and argument names, but I haven't looked for those in my quick survey. For all of these inconsistencies, I think we should make a 'consistently' named alternative, and alias the original variant with them (or the other way around), without defining a deprecation timeline for the original names. This should make it possible to eventually make the stdlib consistent, Pythonic and unsurprising. What would you think of such an effort? Regards, Ralph Broenink [1] https://mail.python.org/pipermail/python-ideas/2010-January/006755.html [2] https://mail.python.org/pipermail/python-dev/2009-March/086646.html

18 29

size of the installation of Python on mobile devices
by Xavier de Gaye 03 Aug '16

03 Aug '16

Issue 26852 [1] proposes an enhancement to reduce the size of the Python installation by using a sourceless distribution. It seems that discussion on this problem (the size of the Python installation on mobile devices) should be moved here. IMHO there is no need to debate on the fact that a full install of Python on a mobile device today is a problem. As a reference, here is a raw estimation of the sizes of the different installs: full install: 111M without the test suite: 53M without the test suite and as a sourceless distribution: 23M same as above but also without ensurepip: 14M It would be useful to find the references to the previous discussions related to sourceless distributions, especially the reasons why their use is discouraged. Xavier [1] http://bugs.python.org/issue26852

11 28

Re: [Python-ideas] Python-ideas Digest, Vol 117, Issue 13
by Arek Bulski 02 Aug '16

02 Aug '16

I am moving the discussion on file-like methods to GitHub. Mailing list is not very usable IMHO. Please switch to that. https://github.com/python/peps/issues/66 pozdrawiam, Arkadiusz Bulski 2016-08-02 7:02 GMT+02:00 <python-ideas-request(a)python.org>: > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > > Today's Topics: > > 1. Re: Trial balloon: adding variable type declarations in > support of PEP 484 (Steven D'Aprano) > 2. Re: Trial balloon: adding variable type declarations in > support of PEP 484 (Steven D'Aprano) > 3. Re: Proposing new file-like object methods (eryk sun) > 4. Re: Trial balloon: adding variable type declarations in > support of PEP 484 (Guido van Rossum) > 5. Re: Trial balloon: adding variable type declarations in > support of PEP 484 (Guido van Rossum) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 2 Aug 2016 12:55:09 +1000 > From: Steven D'Aprano <steve(a)pearwood.info> > To: python-ideas(a)python.org > Subject: Re: [Python-ideas] Trial balloon: adding variable type > declarations in support of PEP 484 > Message-ID: <20160802025509.GD6608(a)ando.pearwood.info> > Content-Type: text/plain; charset=us-ascii > > On Mon, Aug 01, 2016 at 02:31:16PM -0700, Guido van Rossum wrote: > > PEP 484 doesn't change Python's syntax. Therefore it has no good > > syntax to offer for declaring the type of variables, and instead you > > have to write e.g. > > > > a = 0 # type: float > > b = [] # type: List[int] > > c = None # type: Optional[str] > > > > I'd like to address this in the future, and I think the most elegant > > syntax would be to let you write these as follows: > > > > a: float = 0 > > b: List[int] = [] > > c: Optional[str] = None > > > Those examples look reasonable to me. > > > [...] > > Second, when these occur in a class body, they can define either class > > variables or instance variables. Do we need to be able to specify > > which? > > I would think so. Consider the case that you have Class.spam and > Class().spam which may not be the same type. E.g. the class attribute > (representing the default value used by all instances) might be a > mandatory int, while the instance attribute might be Optional[int]. > > > > Third, there's an annoying thing with tuples/commas here. On the one > > hand, in a function declaration, we may see (a: int = 0, b: str = ''). > > On the other hand, in an assignment, we may see > > > > a, b = 0, '' > > > > Suppose we wanted to add types to the latter. Would we write this as > > > > a, b: int, str = 0, '' > > > > or as > > > > a: int, b: str = 0, '' > > Require parens around the name:hint. > > (a:int), (b:str) = 0, '' > > Or just stick to a type hinting comment :-) > > > What about this case? > > spam, eggs = [1, 2.0, 'foo'], (1, '') > [a, b, c], [d, e] = spam, eggs > > That becomes: > > [(a:int), (b:float), (c:str)], [(x:int), (y:str)] = spam, eggs > > which is bearable, but just unpleasant enough to discourage people from > doing it unless they really need to. > > > -- > Steve > > > ------------------------------ > > Message: 2 > Date: Tue, 2 Aug 2016 12:57:12 +1000 > From: Steven D'Aprano <steve(a)pearwood.info> > To: python-ideas(a)python.org > Subject: Re: [Python-ideas] Trial balloon: adding variable type > declarations in support of PEP 484 > Message-ID: <20160802025711.GE6608(a)ando.pearwood.info> > Content-Type: text/plain; charset=us-ascii > > On Tue, Aug 02, 2016 at 07:40:39AM +1000, Chris Angelico wrote: > > > Additional case, unless it's patently obvious to someone with more 484 > > experience than I: what happens with chained assignment? > > > > a = b = c = 0 > > > > Does each variable get separately tagged, or does one tag apply to all? > > (a:int) = (b:Optional[int]) = c = 0 > > `a` is declared to always be an int, `b` declared to be an int or None, > and `c` is not declared. > > > > -- > Steve > > > ------------------------------ > > Message: 3 > Date: Tue, 2 Aug 2016 04:33:53 +0000 > From: eryk sun <eryksun(a)gmail.com> > To: python-ideas(a)python.org > Subject: Re: [Python-ideas] Proposing new file-like object methods > Message-ID: > < > CACL+1au+bX_tKQvxDJC1SBLSObKbqAUPjj9dpmcr9s1izq0Cng(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Tue, Aug 2, 2016 at 2:00 AM, Random832 <random832(a)fastmail.com> wrote: > > On Mon, Aug 1, 2016, at 20:27, Wes Turner wrote: > >> file.pread? > > > > Wouldn't that be rather like having os.fstatat? > > > > For the concept in general, I'm concerned that the equivalent > > functionality on windows may require that the I/O be done > > asynchronously, or that the file have been opened in a special > > way. > > Updating the file position is all or nothing. In synchronous mode, > which is what the CRT's low I/O implementation uses, the file position > is always updated. However, one can atomically set the file position > before a read by passing an overlapped with an offset to ReadFile, > which in turn sets the ByteOffset argument of the underlying > NtReadFile [1] system call. > > [1]: https://msdn.microsoft.com/en-us/library/ff567072 > > For example: > > import _winapi > import ctypes > from ctypes import wintypes > > kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) > > ULONG_PTR = wintypes.WPARAM > > class OVERLAPPED(ctypes.Structure): > class _OFF_PTR(ctypes.Union): > class _OFFSET(ctypes.Structure): > _fields_ = (('Offset', wintypes.DWORD), > ('OffsetHigh', wintypes.DWORD)) > _fields_ = (('_offset', _OFFSET), > ('Pointer', wintypes.LPVOID)) > _anonymous_ = ('_offset',) > _fields_ = (('Internal', ULONG_PTR), > ('InternalHigh', ULONG_PTR), > ('_off_ptr', _OFF_PTR), > ('hEvent', wintypes.HANDLE)) > _anonymous_ = ('_off_ptr',) > > with open('test.txt', 'wb') as f: > f.write(b'0123456789') > > h = _winapi.CreateFile('test.txt', 0x80000000, 3, 0, 3, 0, 0) > buf = (ctypes.c_char * 5)() > n = (wintypes.DWORD * 1)() > fp = (wintypes.LARGE_INTEGER * 1)() > > ov = (OVERLAPPED * 1)() > ov[0].Offset = 5 > > > >>> kernel32.ReadFile(h, buf, 5, n, ov) > 1 > >>> buf[:] > b'56789' > >>> kernel32.SetFilePointerEx(h, 0, fp, 1) > 1 > >>> fp[0] > 10 > > Note that after reading 5 bytes at an offset of 5, the current file > position is 10. With pread() it would still be at 0. The file position > can be tracked and set atomically before each read or write, but I > don't think this is practical unless it's part of a larger project to > use the Windows API directly. > > > ------------------------------ > > Message: 4 > Date: Mon, 1 Aug 2016 21:57:12 -0700 > From: Guido van Rossum <guido(a)python.org> > To: "Steven D'Aprano" <steve(a)pearwood.info> > Cc: Python-Ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] Trial balloon: adding variable type > declarations in support of PEP 484 > Message-ID: > < > CAP7+vJJLz2wRPpkrtb8j8v7vTRFqjbw_b4a5pwAv0-5HyKNf4w(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > The parentheses really strike me as too much complexity. You should > just split it up into multiple lines, or use a semicolon. > > On Mon, Aug 1, 2016 at 7:57 PM, Steven D'Aprano <steve(a)pearwood.info> > wrote: > > On Tue, Aug 02, 2016 at 07:40:39AM +1000, Chris Angelico wrote: > > > >> Additional case, unless it's patently obvious to someone with more 484 > >> experience than I: what happens with chained assignment? > >> > >> a = b = c = 0 > >> > >> Does each variable get separately tagged, or does one tag apply to all? > > > > (a:int) = (b:Optional[int]) = c = 0 > > > > `a` is declared to always be an int, `b` declared to be an int or None, > > and `c` is not declared. > > > > > > > > -- > > Steve > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > > > ------------------------------ > > Message: 5 > Date: Mon, 1 Aug 2016 22:02:33 -0700 > From: Guido van Rossum <guido(a)python.org> > To: "Steven D'Aprano" <steve(a)pearwood.info> > Cc: Python-Ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] Trial balloon: adding variable type > declarations in support of PEP 484 > Message-ID: > < > CAP7+vJJHYLwzbnOqrbqD3vboktXG-Rp+8n-MYxm8uf_Ge2m_yQ(a)mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Regarding class variables: in my experience instance variables way > outnumber class variables, except when the latter are used as defaults > for instance variables. > > See e.g. this bit of code from mypy (and many others, actually): > https://github.com/python/mypy/blob/master/mypy/nodes.py#L154. All of > these declare instance variables. Many of them have no need for a > default in the class, but must give one anyway or else there's nothing > to put the type comment on -- you can't write > > defs # type: List[Statement] > > (it would be a NameError) and you certainly don't want to write > > defs = [] # type: List[Statement] > > (else the gods of shared mutable state will curse you) so you have to > make do with > > defs = None # type: List[Statement] > > But this would be totally reasonable as > > defs: List[Statement] > > under my proposal. > > On Mon, Aug 1, 2016 at 7:55 PM, Steven D'Aprano <steve(a)pearwood.info> > wrote: > > On Mon, Aug 01, 2016 at 02:31:16PM -0700, Guido van Rossum wrote: > >> PEP 484 doesn't change Python's syntax. Therefore it has no good > >> syntax to offer for declaring the type of variables, and instead you > >> have to write e.g. > >> > >> a = 0 # type: float > >> b = [] # type: List[int] > >> c = None # type: Optional[str] > >> > >> I'd like to address this in the future, and I think the most elegant > >> syntax would be to let you write these as follows: > >> > >> a: float = 0 > >> b: List[int] = [] > >> c: Optional[str] = None > > > > > > Those examples look reasonable to me. > > > > > > [...] > >> Second, when these occur in a class body, they can define either class > >> variables or instance variables. Do we need to be able to specify > >> which? > > > > I would think so. Consider the case that you have Class.spam and > > Class().spam which may not be the same type. E.g. the class attribute > > (representing the default value used by all instances) might be a > > mandatory int, while the instance attribute might be Optional[int]. > > > > > >> Third, there's an annoying thing with tuples/commas here. On the one > >> hand, in a function declaration, we may see (a: int = 0, b: str = ''). > >> On the other hand, in an assignment, we may see > >> > >> a, b = 0, '' > >> > >> Suppose we wanted to add types to the latter. Would we write this as > >> > >> a, b: int, str = 0, '' > >> > >> or as > >> > >> a: int, b: str = 0, '' > > > > Require parens around the name:hint. > > > > (a:int), (b:str) = 0, '' > > > > Or just stick to a type hinting comment :-) > > > > > > What about this case? > > > > spam, eggs = [1, 2.0, 'foo'], (1, '') > > [a, b, c], [d, e] = spam, eggs > > > > That becomes: > > > > [(a:int), (b:float), (c:str)], [(x:int), (y:str)] = spam, eggs > > > > which is bearable, but just unpleasant enough to discourage people from > > doing it unless they really need to. > > > > > > -- > > Steve > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > > > ------------------------------ > > End of Python-ideas Digest, Vol 117, Issue 13 > ********************************************* >

2 1

Proposing new file-like object methods
by Arek Bulski 02 Aug '16

02 Aug '16

I would like to see few methods added to file() objects: - read(size, *, offset=None) so os.pread() be used going around file description position - write(data, *, offset=None) analog os.pwrite() - flush(sync=False) so os.fsync(f.fileno()) can optionally follow For text files, offset would probably be not supported or count in characters. New methods could be added like readp/writep/fsync instead of modifying current methods, its debatable. Please tell me how to go about pushing this mainstream.

4 3

Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484
by אלעזר 02 Aug '16

02 Aug '16

On 08/01/2016 02:31 PM, Guido van Rossum wrote: > (I've considered a 'var' keyword in the past, but there just are too > many variables named 'var' in my code. :-) > > There's the obvious 'local', completing 'global' and 'nonlocal'. - Elazar

1 0