From abarnert at yahoo.com  Sun Dec  1 00:42:31 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 30 Nov 2013 15:42:31 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <5298390B.30506@canterbury.ac.nz>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
Message-ID: <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>

On Nov 28, 2013, at 22:49, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> spir wrote:
>> (But then, possibly, some would wonder why this new ord(s,i) is not a string method ;-)
> 
> For the same reason that the existing ord() function isn't
> a string method, whatever that is!

http://docs.python.org/2/faq/design.html#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list

The short version is: historical reasons, and no reason to change it compelling enough to be worth the costs in backward compat, bikeshedding, etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131130/ffee5576/attachment-0001.html>

From guido at python.org  Sun Dec  1 00:48:57 2013
From: guido at python.org (Guido van Rossum)
Date: Sat, 30 Nov 2013 15:48:57 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
Message-ID: <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>

On Sat, Nov 30, 2013 at 3:42 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Nov 28, 2013, at 22:49, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> spir wrote:
>
> (But then, possibly, some would wonder why this new ord(s,i) is not a
> string method ;-)
>
>
> For the same reason that the existing ord() function isn't
> a string method, whatever that is!
>
>
>
> http://docs.python.org/2/faq/design.html#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list
>
> The short version is: historical reasons, and no reason to change it
> compelling enough to be worth the costs in backward compat, bikeshedding,
> etc.
>

That FAQ entry does not do the real motivation justice. While OO "purists"
may argue about it, to me it's obvious that in quite a few cases the
function spelling is more readable than the method spelling. I found it
more readable 23 years ago, and I still find it more readable today. So
it's not historical reasons to me, even if not everybody agrees (obviously
:-).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131130/da2be6e1/attachment.html>

From tjreedy at udel.edu  Sun Dec  1 02:23:17 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 30 Nov 2013 20:23:17 -0500
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
 <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
Message-ID: <l7e324$709$1@ger.gmane.org>

On 11/30/2013 6:48 PM, Guido van Rossum wrote:
>
> On Sat, Nov 30, 2013 at 3:42 PM, Andrew Barnert
> <abarnert at yahoo.com
> <mailto:abarnert at yahoo.com>> wrote:
>
>     On Nov 28, 2013, at 22:49, Greg Ewing
>     <greg.ewing at canterbury.ac.nz
>     <mailto:greg.ewing at canterbury.ac.nz>> wrote:
>
>>     spir wrote:
>>>     (But then, possibly, some would wonder why this new ord(s,i) is
>>>     not a string method ;-)

ord and chr are inverse functions. Would you have chr be an int method?
(Even if yes, history comes into play here as ints did not have methods 
until 2.2(?, or close to that).

>>     For the same reason that the existing ord() function isn't
>>     a string method, whatever that is!
>
>     http://docs.python.org/2/faq/design.html#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list
>
>     The short version is: historical reasons, and no reason to change it
>     compelling enough to be worth the costs in backward compat,
>     bikeshedding, etc.
>
>
> That FAQ entry does not do the real motivation justice. While OO
> "purists" may argue about it, to me it's obvious that in quite a few
> cases the function spelling is more readable than the method spelling. I
> found it more readable 23 years ago, and I still find it more readable
> today. So it's not historical reasons to me, even if not everybody
> agrees (obviously :-).

Sometimes functions are more functional (pun intented) than methods. 
len(ob) checks that ob.__len__() returns an int or something that can be 
'interpreted' as such. reversed(itable) checks for itable.__reversed__ 
before using a default version.

-- 
Terry Jan Reedy


From greg at krypto.org  Sun Dec  1 03:03:21 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 30 Nov 2013 18:03:21 -0800
Subject: [Python-ideas] Replacing the if __name__ == "__main__" idiom
 (was Re: making a module callable)
In-Reply-To: <CAAZsQLDk03g-GPg1tJZmzzpRtR_O4+2jKKmXLHMKs3hmSjr2QA@mail.gmail.com>
References: <CAN8d9gmrYBQ7-jgEspoJ-tKo_-vZfDpzaPa-8gW4jSWz-sMTLA@mail.gmail.com>
 <20131125071932.GA65531@cskk.homeip.net>
 <CADiSq7cRkMisYe8dRPL_Hc2a9M9AOP1aBKaPV=Y1kdDqJxyweA@mail.gmail.com>
 <20131125141220.GE2085@ando> <20131125144244.1cb160f0@anarchist>
 <CAP7+vJL6P+bh1JZ_Hbu9+cGZS+CGzS2SpYmOnQ0uOvQF=Wi8ig@mail.gmail.com>
 <CADiSq7eXKxgdaA2xeEgv_rBP2T+NGtBNk2bboHT_-pNpG2mT5w@mail.gmail.com>
 <CAP7+vJJHZdEqa_n44EA6KfANSUNZiARMyuDsGQwnGfHZJoN2wQ@mail.gmail.com>
 <20131125171603.1812e9fd@anarchist>
 <CAP7+vJ+fdcbZ6UNcqWu3TYrKTvOZkZ=aqkt7EWrXDz4+RDPAYA@mail.gmail.com>
 <CAHVvXxT10Jab=ShNk8Jje-Vz13ci6n8-k1L-LqbG2kiOC2OcmA@mail.gmail.com>
 <CAMjeLr84XXmiynurD7r6OpTskX=N+zjeu_TeDYKrTJS2n2VvsQ@mail.gmail.com>
 <CADiSq7dBXCAL-N=xHq0Y-dNuzWp-km68pm1fFCHJLw7VVLXApw@mail.gmail.com>
 <5294CD94.6050008@gmail.com> <D9FD9A58-468F-49DA-982B-039B0B0E37FF@yahoo.com>
 <CAAZsQLDk03g-GPg1tJZmzzpRtR_O4+2jKKmXLHMKs3hmSjr2QA@mail.gmail.com>
Message-ID: <CAGE7PNL5ZSQv2Yd8-bcg478koEFgPt6EAzz+h1uhzV=Q=d0jaw@mail.gmail.com>

On Wed, Nov 27, 2013 at 9:58 AM, Gregory Salvan <apieum at gmail.com> wrote:

> my cent:
> I think it's most often a bad practice (whereas it's convenient) to mix
> "executable" and "importable" code.
> Providing an easier way to do it, may encourage a bad practice.
>
> I would prefer a solution which encourage separation of "executable" and
> "importable" code.
>

The only way to do that is to force people to put their executable code in
a __main__.py file and prevent class and function definitions within
that...  Gross.

People need the test for main or not (imported or not) because they write
unit tests for _all_ of their code, including their main program.  Just
because it can be imported does not mean it is a library that anyone
outside of their tests _should_ import. Sure, there are some python stdlib
modules and others that work as an importable library or via python -m
modulename to run a program but I wouldn't call that the common case.  I
would not want that to stop working either.

-gps



>
>
>
> 2013/11/26 Andrew Barnert <abarnert at yahoo.com>
>
>> On Nov 26, 2013, at 8:34, Alan Cristhian Ruiz <alan.cristh at gmail.com>
>> wrote:
>>
>> > I think the need to change * if __ name__ == "__main__": * is
>> capricious and obsessive. The current rule is better than anything that has
>> been suggested so far. I never had any problems with the * if __ name__ ==
>> "__main__": *. Also in python there are other things much more difficult to
>> learn and use, such as metaclasses.
>>
>> Although I agree with your main point, I don't think that's a very good
>> argument.
>>
>> __main__ is something novices have to learn early and use in code
>> regularly; metaclasses are something only experienced developers use, and
>> not that often (and that's even if you count using stdlib metaclasses to,
>> e.g., create ABCs, which doesn't really require you to understand how they
>> work). It's perfectly reasonable for an "expert" feature to be more
>> difficult to learn than a novice feature.
>>
>> Also, Python doesn't have a queue of improvements to be scheduled to a
>> team of developers. Things get improved if someone is motivated enough to
>> write the code and drive the idea to consensus and/or BDFL approval. So,
>> improving this would have very little bearing on improving things you care
>> about more.
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131130/4eab0db5/attachment.html>

From greg at krypto.org  Sun Dec  1 03:04:05 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 30 Nov 2013 18:04:05 -0800
Subject: [Python-ideas] Replacing the if __name__ == "__main__" idiom
 (was Re: making a module callable)
In-Reply-To: <CADiSq7dN=btPKgRtvX0_gmeP-tsZqhd9=_2KGuL13HUExPYqGw@mail.gmail.com>
References: <CAN8d9gmrYBQ7-jgEspoJ-tKo_-vZfDpzaPa-8gW4jSWz-sMTLA@mail.gmail.com>
 <20131125071932.GA65531@cskk.homeip.net>
 <CADiSq7cRkMisYe8dRPL_Hc2a9M9AOP1aBKaPV=Y1kdDqJxyweA@mail.gmail.com>
 <20131125141220.GE2085@ando>
 <CADiSq7dN=btPKgRtvX0_gmeP-tsZqhd9=_2KGuL13HUExPYqGw@mail.gmail.com>
Message-ID: <CAGE7PNL=jPpZ60ac6t3s9yQCqOoxqd6nC17r5wzXfrOxDWfNng@mail.gmail.com>

On Mon, Nov 25, 2013 at 6:22 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 26 November 2013 00:12, Steven D'Aprano <steve at pearwood.info> wrote:
>
> <A wonderful summary>
>
> I'm wondering if we should add a link to Steven's post from
> http://www.python.org/dev/peps/pep-0299/ (and perhaps even update the
> PEP text itself)
>
> As the status of PEP 299 shows, Guido has rejected the idea of a
> special main function before, but I think Steven's post does a better
> job of spelling out "Why not?" than any of the previous discussions.
>

+1  (if you haven't already done it)

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131130/8456d7d9/attachment-0001.html>

From abarnert at yahoo.com  Sun Dec  1 05:33:08 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 30 Nov 2013 20:33:08 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
 <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
Message-ID: <6751972E-5222-4B02-AD87-3FBC404C1317@yahoo.com>

On Nov 30, 2013, at 15:48, Guido van Rossum <guido at python.org> wrote:

> 
> On Sat, Nov 30, 2013 at 3:42 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> On Nov 28, 2013, at 22:49, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> 
>>> spir wrote:
>>>> (But then, possibly, some would wonder why this new ord(s,i) is not a string method ;-)
>>> 
>>> For the same reason that the existing ord() function isn't
>>> a string method, whatever that is!
>> 
>> http://docs.python.org/2/faq/design.html#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list
>> 
>> The short version is: historical reasons, and no reason to change it compelling enough to be worth the costs in backward compat, bikeshedding, etc.
> 
> That FAQ entry does not do the real motivation justice. While OO "purists" may argue about it, to me it's obvious that in quite a few cases the function spelling is more readable than the method spelling. I found it more readable 23 years ago, and I still find it more readable today. So it's not historical reasons to me, even if not everybody agrees (obviously :-).

I kind of read it the other way around--making _too many_ things methods is the historical quirk, because at some point (probably after making str and tuple and so on into full types) you and the other core devs went a bit farther with the OO-style "everything should be a method if at all possible" than you would have earlier or later in history.

But I may have been assuming too much. A lot of other "multi-paradigm" languages went through a phase like that and have a historical legacy, like C++'s ridiculous string class and JavaScript's excess of Array methods. But Python does seen to have ended up with a lot fewer quirky choices than those languages. Still, I think the various index/find/count/etc. methods didn't all need to be methods.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131130/ae31ecf8/attachment.html>

From ncoghlan at gmail.com  Sun Dec  1 06:03:51 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 1 Dec 2013 15:03:51 +1000
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <6751972E-5222-4B02-AD87-3FBC404C1317@yahoo.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
 <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
 <6751972E-5222-4B02-AD87-3FBC404C1317@yahoo.com>
Message-ID: <CADiSq7ciPJaJDNwcLYUL3DR1U0mFcayho0QiPMrvUs5tX5P4Bg@mail.gmail.com>

On 1 December 2013 14:33, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Nov 30, 2013, at 15:48, Guido van Rossum <guido at python.org> wrote:
> That FAQ entry does not do the real motivation justice. While OO "purists"
> may argue about it, to me it's obvious that in quite a few cases the
> function spelling is more readable than the method spelling. I found it more
> readable 23 years ago, and I still find it more readable today. So it's not
> historical reasons to me, even if not everybody agrees (obviously :-).
>
> I kind of read it the other way around--making _too many_ things methods is
> the historical quirk, because at some point (probably after making str and
> tuple and so on into full types) you and the other core devs went a bit
> farther with the OO-style "everything should be a method if at all possible"
> than you would have earlier or later in history.
>
> But I may have been assuming too much. A lot of other "multi-paradigm"
> languages went through a phase like that and have a historical legacy, like
> C++'s ridiculous string class and JavaScript's excess of Array methods. But
> Python does seen to have ended up with a lot fewer quirky choices than those
> languages. Still, I think the various index/find/count/etc. methods didn't
> all need to be methods.

One of the other things to keep in mind is that many of the builtin
function vs method choices predated the iterator protocol. A few have
since been updated to work with it (list.sort -> sorted, list.reverse
-> reversed), but others live on as Sequence ABC methods rather than
as builtins or itertools.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From abarnert at yahoo.com  Sun Dec  1 06:53:54 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 30 Nov 2013 21:53:54 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <CADiSq7ciPJaJDNwcLYUL3DR1U0mFcayho0QiPMrvUs5tX5P4Bg@mail.gmail.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
 <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
 <6751972E-5222-4B02-AD87-3FBC404C1317@yahoo.com>
 <CADiSq7ciPJaJDNwcLYUL3DR1U0mFcayho0QiPMrvUs5tX5P4Bg@mail.gmail.com>
Message-ID: <AEC8ACC4-4016-43AD-96C4-23475955C86B@yahoo.com>

On Nov 30, 2013, at 21:03, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 1 December 2013 14:33, Andrew Barnert <abarnert at yahoo.com> wrote:
>> On Nov 30, 2013, at 15:48, Guido van Rossum <guido at python.org> wrote:
>> That FAQ entry does not do the real motivation justice. While OO "purists"
>> may argue about it, to me it's obvious that in quite a few cases the
>> function spelling is more readable than the method spelling. I found it more
>> readable 23 years ago, and I still find it more readable today. So it's not
>> historical reasons to me, even if not everybody agrees (obviously :-).
>> 
>> I kind of read it the other way around--making _too many_ things methods is
>> the historical quirk, because at some point (probably after making str and
>> tuple and so on into full types) you and the other core devs went a bit
>> farther with the OO-style "everything should be a method if at all possible"
>> than you would have earlier or later in history.
>> 
>> But I may have been assuming too much. A lot of other "multi-paradigm"
>> languages went through a phase like that and have a historical legacy, like
>> C++'s ridiculous string class and JavaScript's excess of Array methods. But
>> Python does seen to have ended up with a lot fewer quirky choices than those
>> languages. Still, I think the various index/find/count/etc. methods didn't
>> all need to be methods.
> 
> One of the other things to keep in mind is that many of the builtin
> function vs method choices predated the iterator protocol. A few have
> since been updated to work with it (list.sort -> sorted, list.reverse
> -> reversed), but others live on as Sequence ABC methods rather than
> as builtins or itertools.

Sure, that's the part of "historical reasons" that seems accurate. There are a few methods that would be functions if they were added today (e.g., because the iterator protocol didn't exist), and a few functions that might be methods (e.g., because some of the standard types weren't class-like). Many of them would be the same even if they were added today; it's only the handful of exceptions that people even think to ask about.

If you read the FAQ as saying that Python should be Java/Ruby-style pure OO (or that it should be Haskell-style pure functional), but isn't because of historical reasons, then it's unfair and incorrect. But I just read it as saying the exact balance of methods vs. functions might turn out slightly different.


From denis.spir at gmail.com  Sun Dec  1 09:15:41 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 01 Dec 2013 09:15:41 +0100
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
Message-ID: <529AF02D.5080007@gmail.com>

On 12/01/2013 12:42 AM, Andrew Barnert wrote:
> On Nov 28, 2013, at 22:49, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
>> spir wrote:
>>> (But then, possibly, some would wonder why this new ord(s,i) is not a string method ;-)
>>
>> For the same reason that the existing ord() function isn't
>> a string method, whatever that is!
>
> http://docs.python.org/2/faq/design.html#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list
>
> The short version is: historical reasons, and no reason to change it compelling enough to be worth the costs in backward compat, bikeshedding, etc.

I'm very ok with extending the builtin func ord to get an index param: ord(i). 
Solves the issue without trouble, and backward compatibile. What do you think?

Denis

From denis.spir at gmail.com  Sun Dec  1 09:21:59 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 01 Dec 2013 09:21:59 +0100
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
 <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
Message-ID: <529AF1A7.4090803@gmail.com>

On 12/01/2013 12:48 AM, Guido van Rossum wrote:
> That FAQ entry does not do the real motivation justice. While OO "purists"
> may argue about it, to me it's obvious that in quite a few cases the
> function spelling is more readable than the method spelling. I found it
> more readable 23 years ago, and I still find it more readable today. So
> it's not historical reasons to me, even if not everybody agrees (obviously
> :-).

I was told ord() is a func because it works only on single-code (historically 
single-byte) strings (yes, ord throws an error otherwise, I just checked!).

 From a personal, nearly esthetic, pov, I'd rather make ord a method [s.ord(i)], 
and join not [join(strings, sep)]. But I do _not_ want to argue on this ;-).

Denis

From denis.spir at gmail.com  Sun Dec  1 09:37:24 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 01 Dec 2013 09:37:24 +0100
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <6751972E-5222-4B02-AD87-3FBC404C1317@yahoo.com>
References: <52960290.6090809@gmail.com> <52972C49.1040909@gmail.com>
 <20131129004527.GZ2085@ando> <52981EB6.50003@canterbury.ac.nz>
 <52983764.9010204@gmail.com> <5298390B.30506@canterbury.ac.nz>
 <A9135027-C00F-4C79-9596-9151B20BA676@yahoo.com>
 <CAP7+vJ+n68YAj2Q3ONaBP4aXVyL9mFBTVQdfDoO5JDY0540e-A@mail.gmail.com>
 <6751972E-5222-4B02-AD87-3FBC404C1317@yahoo.com>
Message-ID: <529AF544.9020808@gmail.com>

On 12/01/2013 05:33 AM, Andrew Barnert wrote:
> But I may have been assuming too much. A lot of other "multi-paradigm" languages went through a phase like that and have a historical legacy, like C++'s ridiculous string class and JavaScript's excess of Array methods. But Python does seen to have ended up with a lot fewer quirky choices than those languages. Still, I think the various index/find/count/etc. methods didn't all need to be methods.

Something must be good to be criticisable... Python is clean, so its quircks 
so-to-say jump at one's eyes. When facing C++, or similar mess, I don't know 
where to start the critics ;-)

Denis

From denis.spir at gmail.com  Sun Dec  1 13:45:52 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 01 Dec 2013 13:45:52 +0100
Subject: [Python-ideas] start, test, init
Message-ID: <529B2F80.1070605@gmail.com>

Hello,

This is a proposal and opinion I have thought at sending for a long time 
already, but did not because it is not exactly proper to Python (there are a few 
more of this category ;-). What decided me is the thread about the idiom "if 
__name__ == '__main__':", which had much success! Below a short summary or my 
views, followed by a longer series of comments.

Summary:
There should be 3 top-level functions predefined by the language:
* 'start' : start func for a module run as an app (or 'main')
* 'test'  : main func of a module's test suite
* 'init'  : module init code, rather when imported
In my view, all 3 correspond to clearly definite & distinct functionalities. 
Each of them provides clarity and all together permit getting rid of "lost code" 
roaming around at the top-level of modules; which I dislike, even for scripts; 
however, the proposal does not force anyone to follow such a style. Every module 
using such functions is then a set of definitions: assignments, def's, classes, 
plus imports and such. Possibly 1, 2, 3 of them are language-defined main 
functions. Very clean...

The execution logics would be:
* if imported module:
     ~ run init if any
     ~ else, run nothing
* if executed module:
     ~ run test, if any (see below)
     ~ else, run start, if any
     ~ else, run nothing (error?)

What do you think?


=== why start? ===

It looks nice to have hello world reduced to:

     print("Hello, world!")

However, this is a very superficial nicety. I prefere the (very big!) 
complication of:

     def start():
         print("Hello, world!")

This code is self-understanding even for a novice programmer and nicely 
introduces some structural feature of the language. [0]

This function 'start' would have an optional argument for command-line args; it 
may indeed also return an exit code.

About the name 'start', well, it's a question of meaning; and to avoid confusion 
I'd rather reserved 'main' for a package's main module. Also, every of these 3 
funcs is a 'main' one, depending on the actual case. But I would *not* fight on 
this point, call it 'main' if you like. (In assembly, it's commonly start as 
well: both ends of the expressivity scale meet here ;-).

When compiling with start, in standard the language may also automagically strip 
out development or control instructions, like assertions. (There also may be a 
differenciated debug print command... but that's another point. Also different 
error messages for end-users.)


=== why test? ===

If you practice testing (by code), you know why. Nearly all of my modules end up 
ending with:

# ===   t e s t   =======================

def test_abc ():
     ...
def test_def ():
     ...
def test_ghi ():
     ...
...
def test ():
     test_abc()
     test_def()
     test_ghi()
     ...
if __name__ == '__main__':
     test()

Then, I comment out the last 2 lines when all runs fine; or else, if it's the 
main module of an app, I replace it with a call to start.

This is a second form or running a module as a stand-alone prog (from the 
command-line). Actually, I guess, anyone practicing a minimum of testing may 
so-to-say press the 'test' button constantly, far more often than we lauch an 
app in usage or trial mode (if ever it's an app). We switch to normal usage 
execution once only per development phase, when all is fine and we prepare a 
user release. This is why, in standard, when both exist, 'test' has precedence 
over 'start'. There may be a builtin config var (eg __main__) to set (eg 
__main__ = start), or any other to switch to start (but preferably from code, 
see also [1]).

This function 'test' _may_ have an optional argument for command-line args; it 
may also return an exit code, here meaning test success / failure (why not 
number of failures?). Args may be handy to drive testing differently: exact 
funcs to run, form of output, depth of testing..., in a purely user-defined way 
(no language-defined meaning, else we'd never stop arguing on the topic; a 
typical bikeshed issue). [1]


=== why init? ===

There is much less use, in my personal practice, for such an init function 
typically run when a module is imported. (I know it from the D language.) But I 
guess whenever we need it, having it is very nice. I simulate it (1) to init 
program elements from external data (2) to import, scan, process, big data files 
like unicode tables, avoiding huge code-data files and/or security issues (3) 
for a usage similar to compile-time computations in static langs that provide 
that. Anyway, it is a clearly defined functionality, just like start & test.


=== flexibility ===

An init func may be run by test or start, conditionnally or not.
A test func may be run by init or start; maybe self-testing just once on first 
launch of a just-installed app or package.


=== alternative ===

Apart from clarity and practicality (see below), such flexibility is also why I 
prefere such predefined function names to a proposal by Steven D'Aprano, on the 
mentionned thread, of builtin conditionals like:
if __main__:
     ...
if is_main():
     ...
if is_main:
     ...

We could however trivially extend this proposal with eg is_tested & is_imported 
builtin conditionals. Why not chose the direct simplicity of builtin func names, 
though? This is a common Python usage (it would also introduce novices to this 
idea of special func names, then they are ready for "magic methods").

Another issue is practicle: writing or not such a function, or chosing between 
test and start, is something we control from code itself. To launch tests for 
instance, there is thus no need to run a module with a special command-line 
option, or to modify it in our favorite programming editor's settings, or to 
change environment variables or python config files... all pretty annoying 
things to do (and I never remember which is to be done, where, how exactly...). 
Instead, predefined conditionnals depend on such settings or data external to 
the code. [2]


=== why simple names? ===

Again, I would not fight for simple names; it may well in fact be preferable to 
have weird names with underscores. In my view, the need in python for names like 
__all__ or __str__ is due to the fact that there is no difference in the 
language between defining and redefining a symbol. If Python used eg ':=' to 
redefine symbols, then:

foo = ...	# error, symbol 'foo' already exists
foo := ...	# ok, intentional redefinition of builtin 'foo'

Waiting for this, it may indeed be preferable to have weird names.


=== trial mode ===

There is one mode not covered by this: what I call trial, as opposed to in-code 
tests. This corresponds more or less to a beta-version: a simulation of normal 
usage by developpers or (power-)users, to further find bugs or more generally 
control the software as is. I don't know of cases where this would require a 
special dedicated main func: this is rather opposite to the idea of checking the 
program really as is. But there may be code parts interpreted differently  or 
conditionnally compiled, differently from the case of an actual user release: 
assertions, debug prints, stats, benchmarking, profiling... all kinds of 
"meta-programming" issues.


=== all together ===

''' d o c
'''

import foo
__all__ = ...

code...
code...
code...

def test_abc ():
     ...
def test_def ():
     ...
def test_ghi ():
     ...
...
def test ():
     test_abc()
     test_def()
     test_ghi()
     ...

def init ():
     ...

# __main__ = start
def start (args):
     init()
     # process args
     ...


What do you think? (bis)

Denis

[0] Python may lack such higher-level code structure, more generally; this is an 
open question.

[1] This feature is for command-line args fans, but I prefere to control testing 
from code itself. For information, I have 2 std test config params, which I 
define in code; whether to:
* write successful checks also, for comparison
* continue on check failures, or else stop
(They affect a custom check func, similar to an assert, used for testing only.) 
Turning them to off transforms a module's diagnosis test suite into (part of) a 
regression test suite.

[2] More generally, I want to control all what concerns code and its development 
from code itself. Command-line args and such (better command-line args only) are 
good only for very special needs like producing doc, a parse-tree, compile-only, 
profiling, etc. I definitely dislike languages that force people to constantly 
change the command line to compile/run or worse change env vars or edit config 
files (and I don't even evoque makefiles and such...).


From ncoghlan at gmail.com  Sun Dec  1 14:07:58 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 1 Dec 2013 23:07:58 +1000
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <CADiSq7eT=KE8T+YCs7euyH+o6yh8CqcS+jHMGrJR_+2VxrUFmA@mail.gmail.com>

The fundamental problem with this idea (outside of backwards compatibility
issues) is that requiring people to write functions completely misses the
core reason that scripting languages are easier to learn than those with a
declarative compile time syntax: it's far easier to use a function than it
is to define one.

More on that topic:
http://www.curiousefficiency.org/posts/2011/08/scripting-languages-and-suitable.html

Aside from a couple of hints to the tokenizer and compiler in the file
header, Python has no declarative mode - this is a feature, not a bug.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131201/dd26b312/attachment.html>

From abarnert at yahoo.com  Sun Dec  1 14:49:58 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 1 Dec 2013 05:49:58 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <272F2B9E-13B7-4C50-9BBC-22C6F72A4672@yahoo.com>

On Dec 1, 2013, at 4:45, spir <denis.spir at gmail.com> wrote:

> Every module using such functions is then a set of definitions: assignments, def's, classes, plus imports and such. Possibly 1, 2, 3 of them are language-defined main functions. Very clean...
> 
> The execution logics would be:
> * if imported module:
>    ~ run init if any
>    ~ else, run nothing
> * if executed module:
>    ~ run test, if any (see below)
>    ~ else, run start, if any
>    ~ else, run nothing (error?)
> 
> What do you think?

I assume you realize that function and class definitions, assignments, etc. are code, and there is no separate execution phase for defining code vs. running code in Python.

So, are you proposing that top-level code only be allowed to run some subset of the language, or that it just be encouraged to do so as a convention?

Meanwhile, your proposal to make hello world a two-liner instead of a one-liner may not seem that big a deal, but it means that every novice has to learn how to define functions before they can write their first program. And it means that everyone who wants to use Python as an admin/scripting language in place of perl or awk or whatever has to write that extra line of code, making Python less usable. And so on.

And meanwhile, in realistic large programs, you're already going to define and call the function anyway, so you're only saving the call. I don't think saving 1 line in 100-line scripts is worth adding 1 line to 1-line scripts.

From denis.spir at gmail.com  Sun Dec  1 15:13:14 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 01 Dec 2013 15:13:14 +0100
Subject: [Python-ideas] start, test, init
In-Reply-To: <272F2B9E-13B7-4C50-9BBC-22C6F72A4672@yahoo.com>
References: <529B2F80.1070605@gmail.com>
 <272F2B9E-13B7-4C50-9BBC-22C6F72A4672@yahoo.com>
Message-ID: <529B43FA.10207@gmail.com>

On 12/01/2013 02:49 PM, Andrew Barnert wrote:
> On Dec 1, 2013, at 4:45, spir <denis.spir at gmail.com> wrote:
>
>> Every module using such functions is then a set of definitions: assignments, def's, classes, plus imports and such. Possibly 1, 2, 3 of them are language-defined main functions. Very clean...
>>
>> The execution logics would be:
>> * if imported module:
>>     ~ run init if any
>>     ~ else, run nothing
>> * if executed module:
>>     ~ run test, if any (see below)
>>     ~ else, run start, if any
>>     ~ else, run nothing (error?)
>>
>> What do you think?
>
> I assume you realize that function and class definitions, assignments, etc. are code, and there is no separate execution phase for defining code vs. running code in Python.
>
> So, are you proposing that top-level code only be allowed to run some subset of the language, or that it just be encouraged to do so as a convention?
>
> Meanwhile, your proposal to make hello world a two-liner instead of a one-liner may not seem that big a deal, but it means that every novice has to learn how to define functions before they can write their first program. And it means that everyone who wants to use Python as an admin/scripting language in place of perl or awk or whatever has to write that extra line of code, making Python less usable. And so on.
>
> And meanwhile, in realistic large programs, you're already going to define and call the function anyway, so you're only saving the call. I don't think saving 1 line in 100-line scripts is worth adding 1 line to 1-line scripts.

Well, from the initial post:
	"however, the proposal does not force anyone to follow such a style"

Nothing prevents you to write hello-world the usual way. Nothing prevents using 
Python as a calculator, neither, nore as a programmer-friendly Bash; however, 
for the latter case, i'd still use 'test' and 'start'. Instead, it is a 
structural addition to the language; a backward compatible one if ever special 
names, like '__test__', are used rather than my preferred ones, like 'test'.

The last point of your post (saving a func call) has nothing to do with the 
proposal or its spirit. It is about code clarity and high-level structuration, 
replacing "if __name=='__main__'", allowing code flow logics to be all cleanly 
placed in funcs, such things...
(did you reply too fast, maybe?)

denis

From guido at python.org  Sun Dec  1 17:52:31 2013
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Dec 2013 08:52:31 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B43FA.10207@gmail.com>
References: <529B2F80.1070605@gmail.com>
 <272F2B9E-13B7-4C50-9BBC-22C6F72A4672@yahoo.com>
 <529B43FA.10207@gmail.com>
Message-ID: <CAP7+vJKdVoZVcp9sG7qpXxFM8MwzQa_yT9Htk2dfAkoCirHK8g@mail.gmail.com>

spir,

If you are serious about your idea I would recommend that you try to build
a library that implements your convention. It's a clear that you have put a
lot of thought in your proposal. It is not so clear that it will be an
improvement. It is also not so clear that there won't be implementation
roadblocks. However, it *is* clear that most people do not think it is
worth the effort. You can be sure that the effort would be significant --
not just in coding it up, but in documentation, education, getting people
to adopt it, and so on. So, since this is open source, if you really, truly
believe in your idea, build it. You will definitely learn something. You
may end up rejecting your own idea. Or you may find a better way. Or you
may pave the road for adoption. There is only one way to find out.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131201/518201e4/attachment.html>

From ron3200 at gmail.com  Sun Dec  1 21:32:52 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 01 Dec 2013 14:32:52 -0600
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <l7g6d9$nt8$1@ger.gmane.org>



On 12/01/2013 06:45 AM, spir wrote:
> # __main__ = start
> def start (args):
>      init()
>      # process args
>      ...
>
>
> What do you think? (bis)

I think you are describing good programming practices for larger modules. 
And I think most people who have been programming in python for any length 
of time develop some form of what you are describing.

The only difference is that they wouldn't need the conditional logic at the 
bottom of the module.  But that also serves as bit of self documentation as 
to what the module or script does, is more flexible, and only adds a line 
or two if the rest is put into functions or classes.


As for the testing case... I'd like for python to have a -t option that 
only sets a global name __test__ to True.  Then that covers most of your 
use cases without adding a whole lot.  Sometimes less is more.

    if __test__:
        test()
    elif __name__ == '__main__':
        main()

That's just one possible combination of using __test__ and __name__.

Cheers,
    Ron


From ncoghlan at gmail.com  Sun Dec  1 22:13:56 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 2 Dec 2013 07:13:56 +1000
Subject: [Python-ideas] start, test, init
In-Reply-To: <l7g6d9$nt8$1@ger.gmane.org>
References: <529B2F80.1070605@gmail.com>
	<l7g6d9$nt8$1@ger.gmane.org>
Message-ID: <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>

On 2 Dec 2013 06:34, "Ron Adam" <ron3200 at gmail.com> wrote:
>
>
>
> On 12/01/2013 06:45 AM, spir wrote:
>>
>> # __main__ = start
>> def start (args):
>>      init()
>>      # process args
>>      ...
>>
>>
>> What do you think? (bis)
>
>
> I think you are describing good programming practices for larger modules.
And I think most people who have been programming in python for any length
of time develop some form of what you are describing.
>
> The only difference is that they wouldn't need the conditional logic at
the bottom of the module.  But that also serves as bit of self
documentation as to what the module or script does, is more flexible, and
only adds a line or two if the rest is put into functions or classes.
>
>
> As for the testing case... I'd like for python to have a -t option that
only sets a global name __test__ to True.  Then that covers most of your
use cases without adding a whole lot.  Sometimes less is more.
>
>    if __test__:
>        test()
>    elif __name__ == '__main__':
>        main()
>
> That's just one possible combination of using __test__ and __name__.

Oh, interesting. Such an option could also alter -O and -OO to keep assert
statements.

I'd love to see that idea elaborated further, as there are a variety of
questions around how it might work in practice (Compile time constant like
__debug__? Always False builtin shadowed in the main module? If it works
like __debug__ and/or affects -O and -OO, does it need a new naming
convention in the pycache directory?)

Cheers,
Nick.

>
> Cheers,
>    Ron
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/1ca6d027/attachment.html>

From ethan at stoneleaf.us  Sun Dec  1 21:55:58 2013
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 01 Dec 2013 12:55:58 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <20131129001248.GY2085@ando>
References: <52960290.6090809@gmail.com> <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
Message-ID: <529BA25E.6000509@stoneleaf.us>

On 11/28/2013 04:12 PM, Steven D'Aprano wrote:
>
> And some benchmarks:
>
> py> from timeit import Timer
> py> setup = "from __main__ import match"
> py> t1 = Timer("match('abcdef', 'cde', 2, -1)", setup)
> py> t2 = Timer("s[2:-1] == 'cde'", "s = 'abcdef'")
> py> min(t1.repeat(repeat=5))
> 1.2987589836120605
> py> min(t2.repeat(repeat=5))
> 0.25656223297119141
>
> Slicing is about three times faster.

Wouldn't that be 5 times faster?

I tried to get startswith to be quicker, and compared a `s.startswith('cde', 2)` (so no function call, no plethora of 
other objects being created and destroyed), and slicing was still twice as fast.

--
~Ethan~

From ethan at stoneleaf.us  Sun Dec  1 22:12:10 2013
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 01 Dec 2013 13:12:10 -0800
Subject: [Python-ideas] Replacing the if __name__ == "__main__" idiom
 (was Re: making a module callable)
In-Reply-To: <CAGE7PNL5ZSQv2Yd8-bcg478koEFgPt6EAzz+h1uhzV=Q=d0jaw@mail.gmail.com>
References: <CAN8d9gmrYBQ7-jgEspoJ-tKo_-vZfDpzaPa-8gW4jSWz-sMTLA@mail.gmail.com>
 <20131125071932.GA65531@cskk.homeip.net>
 <CADiSq7cRkMisYe8dRPL_Hc2a9M9AOP1aBKaPV=Y1kdDqJxyweA@mail.gmail.com>
 <20131125141220.GE2085@ando> <20131125144244.1cb160f0@anarchist>
 <CAP7+vJL6P+bh1JZ_Hbu9+cGZS+CGzS2SpYmOnQ0uOvQF=Wi8ig@mail.gmail.com>
 <CADiSq7eXKxgdaA2xeEgv_rBP2T+NGtBNk2bboHT_-pNpG2mT5w@mail.gmail.com>
 <CAP7+vJJHZdEqa_n44EA6KfANSUNZiARMyuDsGQwnGfHZJoN2wQ@mail.gmail.com>
 <20131125171603.1812e9fd@anarchist>
 <CAP7+vJ+fdcbZ6UNcqWu3TYrKTvOZkZ=aqkt7EWrXDz4+RDPAYA@mail.gmail.com>
 <CAHVvXxT10Jab=ShNk8Jje-Vz13ci6n8-k1L-LqbG2kiOC2OcmA@mail.gmail.com>
 <CAMjeLr84XXmiynurD7r6OpTskX=N+zjeu_TeDYKrTJS2n2VvsQ@mail.gmail.com>
 <CADiSq7dBXCAL-N=xHq0Y-dNuzWp-km68pm1fFCHJLw7VVLXApw@mail.gmail.com>
 <5294CD94.6050008@gmail.com> <D9FD9A58-468F-49DA-982B-039B0B0E37FF@yahoo.com>
 <CAAZsQLDk03g-GPg1tJZmzzpRtR_O4+2jKKmXLHMKs3hmSjr2QA@mail.gmail.com>
 <CAGE7PNL5ZSQv2Yd8-bcg478koEFgPt6EAzz+h1uhzV=Q=d0jaw@mail.gmail.com>
Message-ID: <529BA62A.3030204@stoneleaf.us>

On 11/30/2013 06:03 PM, Gregory P. Smith wrote:
> On Wed, Nov 27, 2013 at 9:58 AM, Gregory Salvan wrote:
>
>> my cent:
>> I think it's most often a bad practice (whereas it's convenient) to mix "executable" and "importable" code.
>> Providing an easier way to do it, may encourage a bad practice.
>>
>> I would prefer a solution which encourage separation of "executable" and "importable" code.
>
> The only way to do that is to force people to put their executable code in a __main__.py file and prevent class and
> function definitions within that...  Gross.
>
> People need the test for main or not (imported or not) because they write unit tests for _all_ of their code, including
> their main program.  Just because it can be imported does not mean it is a library that anyone outside of their tests
> _should_ import. Sure, there are some python stdlib modules and others that work as an importable library or via python
> -m modulename to run a program but I wouldn't call that the common case.  I would not want that to stop working either.

+1 to gps' statements.

--
~Ethan~

From ethan at stoneleaf.us  Sun Dec  1 22:18:35 2013
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 01 Dec 2013 13:18:35 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <529BA7AB.6010507@stoneleaf.us>

If somebody wants this type of behavior it is easy enough to put in the `if __name__ == "__main__"` section.  I do not 
see that adding it to the language buys us anything.

--
~Ethan~

From tjreedy at udel.edu  Sun Dec  1 23:16:05 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 01 Dec 2013 17:16:05 -0500
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <l7gces$kjc$1@ger.gmane.org>

On 12/1/2013 7:45 AM, spir wrote:

> === why test? ===
>
> If you practice testing (by code), you know why. Nearly all of my
> modules end up ending with:
>
> # ===   t e s t   =======================
>
> def test_abc ():
>      ...
> def test_def ():
>      ...
> def test_ghi ():
>      ...
> ...
> def test ():
>      test_abc()
>      test_def()
>      test_ghi()
>      ...
> if __name__ == '__main__':
>      test()

The style above is obsolete.  The older test.regrtest module (private, 
intended for CPython test suite only) required a test_main function in 
which one collected together the test classes, etc, but one might forget 
something. The newer unittest module does the test discovery for us, so 
no master test function is needed. Just put
unittest.main() at the bottom instead of test(). Much easier.

-- 
Terry Jan Reedy


From bruce at leapyear.org  Sun Dec  1 22:55:55 2013
From: bruce at leapyear.org (Bruce Leban)
Date: Sun, 1 Dec 2013 13:55:55 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
References: <529B2F80.1070605@gmail.com> <l7g6d9$nt8$1@ger.gmane.org>
 <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
Message-ID: <CAGu0AnuNhu+ixsuvtDKTU9SpwCHXE9_fhrsh+SuVG=+YE0X9GA@mail.gmail.com>

On Sun, Dec 1, 2013 at 1:13 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> > As for the testing case... I'd like for python to have a -t option that
> only sets a global name __test__ to True.  Then that covers most of your
> use cases without adding a whole lot.  Sometimes less is more.
> >
> >    if __test__:
>

I like this idea but the specific proposal means __test__ needs to be
defined in all modules. A simple alternative:

if __name__ == '__test__':


--- Bruce
SALE! My brother's puzzle company Puzzazz <http://www.puzzazz.com> is
having a BlackFriday/CyberMonday sale http://j.mp/182R4kG - FREE puzzle
books and BOGO
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131201/b9060684/attachment.html>

From steve at pearwood.info  Mon Dec  2 01:02:00 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 2 Dec 2013 11:02:00 +1100
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <529BA25E.6000509@stoneleaf.us>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us>
Message-ID: <20131202000200.GR2085@ando>

On Sun, Dec 01, 2013 at 12:55:58PM -0800, Ethan Furman wrote:

> >py> from timeit import Timer
> >py> setup = "from __main__ import match"
> >py> t1 = Timer("match('abcdef', 'cde', 2, -1)", setup)
> >py> t2 = Timer("s[2:-1] == 'cde'", "s = 'abcdef'")
> >py> min(t1.repeat(repeat=5))
> >1.2987589836120605
> >py> min(t2.repeat(repeat=5))
> >0.25656223297119141
> >
> >Slicing is about three times faster.
> 
> Wouldn't that be 5 times faster?

I'm reminded of a joke. What's 9/2?

The mathematician says it's 4.5. The scientist says it's 4.5 ? 0.1. The 
engineer says it's between 4 and 5, but we'll call it 6 to be on the 
safe side.

Apparently I'm an engineer at heart :-)

> I tried to get startswith to be quicker, and compared a 
> `s.startswith('cde', 2)` (so no function call, no plethora of other objects 
> being created and destroyed), and slicing was still twice as fast.

Did you try it with huge strings? Say, a million Unicode characters?


-- 
Steven

From guido at python.org  Mon Dec  2 01:21:40 2013
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Dec 2013 16:21:40 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <20131202000200.GR2085@ando>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us> <20131202000200.GR2085@ando>
Message-ID: <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>

On Sun, Dec 1, 2013 at 4:02 PM, Steven D'Aprano <steve at pearwood.info> wrote:

>
> I'm reminded of a joke. What's 9/2?
>
> The mathematician says it's 4.5. The scientist says it's 4.5 ? 0.1. The
> engineer says it's between 4 and 5, but we'll call it 6 to be on the
> safe side.
>

I think you missed one. The programmer says it's 4. :-)


> Apparently I'm an engineer at heart :-)
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131201/db506c1c/attachment.html>

From ron3200 at gmail.com  Mon Dec  2 01:55:28 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 01 Dec 2013 18:55:28 -0600
Subject: [Python-ideas] start, test, init
In-Reply-To: <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
References: <529B2F80.1070605@gmail.com>	<l7g6d9$nt8$1@ger.gmane.org>
 <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
Message-ID: <l7glpm$djt$1@ger.gmane.org>



On 12/01/2013 03:13 PM, Nick Coghlan wrote:
>  > As for the testing case... I'd like for python to have a -t option that
> only sets a global name __test__ to True.  Then that covers most of your
> use cases without adding a whole lot.  Sometimes less is more.
>  >
>  >    if __test__:
>  >        test()
>  >    elif __name__ == '__main__':
>  >        main()
>  >
>  > That's just one possible combination of using __test__ and __name__.
>
> Oh, interesting. Such an option could also alter -O and -OO to keep assert
> statements.

You could use the __test__ flag as the key to remove asserts rather than 
-O, -OO, and __debug__.  Is that what you are thinking?

If so, I think it would be equivalent to this.

      assert (invariant_condition) if __test__ else 1, fail_message



> I'd love to see that idea elaborated further, as there are a variety of
> questions around how it might work in practice (Compile time constant like
> __debug__?

Isn't __debug__ set at Python's compile time, not module compile time?

I think that makes __debug__ good for testing C extensions and python's own 
C code, but it's too static to be very useful in python modules.


Always False builtin shadowed in the main module?

That would limit the __test__ check to the __main__ module.  I'd like to be 
able to have that check available anywhere.

For example if I wanted to insert some print functions and use the __test__ 
flag to turn them on when I need to see them...

        if __test__:
            print(this)
            print(that)

The idea could go one step further and allow a value to be passed with the 
-t flag.  Then you could select tests based on that value.



> If it works
> like __debug__ and/or affects -O and -OO, does it need a new naming
> convention in the pycache directory?)

I don't think it should effect -O or -OO in any significant way.  And -O or 
-OO shouldn't effect __test__ either.


I think the easiest way to implement it is to just to have __test__ set in 
the modules attributes at load/import time. If it's always to be available, 
then why not?  I consider testing an imortant part of any program, so 
having __test__ in a module doesn't seem out of place to me.  If 
anything... maybe if it's there... it's more likely to be used.  ;-)


I also like how easy it is to access.  For many small scripts, including an 
argument parser is more work than I want to do, but including a few small 
tests to call if "__test__ == True" is perfect for those cases.

     if __test__:
         import doctest
         doctest.testmod()

     elif __name__ == '__main__':
         main()


And I think it makes using __name__ here seem more natIt may also work as a 
name in the sys module.  sys.__test__, but I prefer the simpler spelling of 
just __test__ because it looks nicer and allows for setting __test__ at the 
top of a module.ural.

Ok, maybe that one is just me.   It just seems, to me, some of the oddness 
comes from always seeing it spelled *exactly the same everywhere*. So the 
"if __name__ == '__main__':"  line just begs for being reduced to something 
more minimal.

Of course that does not mean it should be changes just that we will 
consider doing that to anything that stands out as being duplicated a lot. 
  It's just what good programmers do. ;-)

Cheers,
    Ron





















From python at mrabarnett.plus.com  Mon Dec  2 02:07:40 2013
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 02 Dec 2013 01:07:40 +0000
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us> <20131202000200.GR2085@ando>
 <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>
Message-ID: <529BDD5C.4040409@mrabarnett.plus.com>

On 02/12/2013 00:21, Guido van Rossum wrote:
> On Sun, Dec 1, 2013 at 4:02 PM, Steven D'Aprano <steve at pearwood.info
> <mailto:steve at pearwood.info>> wrote:
>
>
>     I'm reminded of a joke. What's 9/2?
>
>     The mathematician says it's 4.5. The scientist says it's 4.5 ? 0.1. The
>     engineer says it's between 4 and 5, but we'll call it 6 to be on the
>     safe side.
>
>
> I think you missed one. The programmer says it's 4. :-)
>
No. The programmer used to say it was 4, but now says it's 4.5, give or
take a little rounding error. :-)

>     Apparently I'm an engineer at heart :-)
>


From dreamingforward at gmail.com  Mon Dec  2 02:19:36 2013
From: dreamingforward at gmail.com (Mark Janssen)
Date: Sun, 1 Dec 2013 17:19:36 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <CAMjeLr-oDXtNigYkV8-j-i2ymA4gKdqeGNgK1LiTtHZOPQoWBQ@mail.gmail.com>

> There should be 3 top-level functions predefined by the language:
> * 'start' : start func for a module run as an app (or 'main')
> * 'test'  : main func of a module's test suite
> * 'init'  : module init code, rather when imported

1)  I'd make "main" a keyword, used to denote the program start point,
and let the interpreter bail-out if it encounters more than one "main"
section.  Yes that keyword would only be used for that one, simple
purpose, but it *is* a special purpose.  This would also solve the
circular import issues, because now you have a well-defined *root* in
which to construct a graph and check for cycles.
2)  I think testing should also be more "built-into" the interpreter
environment.  I like the idea of a python "-t" flag or something which
makes it runs doctests.  In which case, I'd also what a test()
built-in, so that I can use it *within* the interpreter environment
also, making sure invariants are preserved when modifying and re-using
code.
3) with a "main" keyword, this wouldn't be absolutely necessary.
You'll know where your top-level scope is.

-- 
MarkJ
Tacoma, Washington

From breamoreboy at yahoo.co.uk  Mon Dec  2 02:45:30 2013
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 02 Dec 2013 01:45:30 +0000
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <529BDD5C.4040409@mrabarnett.plus.com>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us> <20131202000200.GR2085@ando>
 <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>
 <529BDD5C.4040409@mrabarnett.plus.com>
Message-ID: <l7gong$9a8$1@ger.gmane.org>

On 02/12/2013 01:07, MRAB wrote:
> On 02/12/2013 00:21, Guido van Rossum wrote:
>> On Sun, Dec 1, 2013 at 4:02 PM, Steven D'Aprano
>> <steve at pearwood.info
>> <mailto:steve at pearwood.info>> wrote:
>>
>>
>>     I'm reminded of a joke. What's 9/2?
>>
>>     The mathematician says it's 4.5. The scientist says it's 4.5 ?
>> 0.1. The
>>     engineer says it's between 4 and 5, but we'll call it 6 to be on the
>>     safe side.
>>
>>
>> I think you missed one. The programmer says it's 4. :-)
>>
> No. The programmer used to say it was 4, but now says it's 4.5, give or
> take a little rounding error. :-)
>

The newbie says it should be 4.5 but gets 4.499999999999999999999 so 
raises issue on bug tracker stating that Python can't do arithmetic 
properly.

In [5]: 9/2
Out[5]: 4.5

Blast, lousy example :)

-- 
Python is the second best programming language in the world.
But the best has yet to be invented.  Christian Tismer

Mark Lawrence


From ron3200 at gmail.com  Mon Dec  2 02:53:32 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 01 Dec 2013 19:53:32 -0600
Subject: [Python-ideas] start, test, init
In-Reply-To: <l7glpm$djt$1@ger.gmane.org>
References: <529B2F80.1070605@gmail.com>	<l7g6d9$nt8$1@ger.gmane.org>
 <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
 <l7glpm$djt$1@ger.gmane.org>
Message-ID: <l7gp6i$ej4$1@ger.gmane.org>



On 12/01/2013 06:55 PM, Ron Adam wrote:
>
> And I think it makes using __name__ here seem more natIt may also work as a
> name in the sys module.  sys.__test__, but I prefer the simpler spelling of
> just __test__ because it looks nicer and allows for setting __test__ at the
> top of a module.ural.

Hmmm... still have that annoying bug of having stuff I cut pasted back in 
at random places in Thunderbird.

That should have read...

      "And I think it makes using __name__ here seem more natural."


The part that was pasted in was a paragraph I deleted considering using 
sys.__test__, but I decided it wasn't really needed.

Cheers,
    Ron


From ethan at stoneleaf.us  Mon Dec  2 06:28:07 2013
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 01 Dec 2013 21:28:07 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <l7glpm$djt$1@ger.gmane.org>
References: <529B2F80.1070605@gmail.com>	<l7g6d9$nt8$1@ger.gmane.org>
 <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
 <l7glpm$djt$1@ger.gmane.org>
Message-ID: <529C1A67.3010509@stoneleaf.us>

On 12/01/2013 04:55 PM, Ron Adam wrote:
>
> Isn't __debug__ set at Python's compile time, not module compile time?

No.  __debug__ is true when Python [1] is started normally, and False when started with -O or -OO.

--
~Ethan~

[1] Cpython to be specific; and there may be other flags that affect __debug__.

From ncoghlan at gmail.com  Mon Dec  2 08:37:37 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 2 Dec 2013 17:37:37 +1000
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <l7gong$9a8$1@ger.gmane.org>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us> <20131202000200.GR2085@ando>
 <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>
 <529BDD5C.4040409@mrabarnett.plus.com> <l7gong$9a8$1@ger.gmane.org>
Message-ID: <CADiSq7fGEUi6JjABebkkRxt4SxBPgWx0uOvjdRz81LdD7HT3PQ@mail.gmail.com>

On 2 December 2013 11:45, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> The newbie says it should be 4.5 but gets 4.499999999999999999999 so raises
> issue on bug tracker stating that Python can't do arithmetic properly.
>
> In [5]: 9/2
> Out[5]: 4.5
>
> Blast, lousy example :)

Getting a modern Python to fall into that trap at all is actually
quite difficult - the way str.__repr__ works was changed a while ago
to favour the nearest terminating approximation that is represented
using the same IEEE754 bit pattern :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From abarnert at yahoo.com  Mon Dec  2 09:42:13 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 2 Dec 2013 00:42:13 -0800
Subject: [Python-ideas] start, test, init
In-Reply-To: <CAMjeLr-oDXtNigYkV8-j-i2ymA4gKdqeGNgK1LiTtHZOPQoWBQ@mail.gmail.com>
References: <529B2F80.1070605@gmail.com>
 <CAMjeLr-oDXtNigYkV8-j-i2ymA4gKdqeGNgK1LiTtHZOPQoWBQ@mail.gmail.com>
Message-ID: <A55C4BCC-B1EF-40A4-B64F-7B2FED9569A3@yahoo.com>

On Dec 1, 2013, at 17:19, Mark Janssen <dreamingforward at gmail.com> wrote:

> 1)  I'd make "main" a keyword, used to denote the program start point,
> and let the interpreter bail-out if it encounters more than one "main"
> section.  Yes that keyword would only be used for that one, simple
> purpose, but it *is* a special purpose.  This would also solve the
> circular import issues, because now you have a well-defined *root* in
> which to construct a graph and check for cycles.

How does that solve circular import issues? There's already a well-defined root, __main__, today. And if there's a cycle, it's a cycle whether it involves the root or not. And if you're talking about moving imports into a function instead of at the top level, people already do that today, and it wouldn't work any better.

From abarnert at yahoo.com  Mon Dec  2 09:45:06 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 2 Dec 2013 00:45:06 -0800
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <CADiSq7fGEUi6JjABebkkRxt4SxBPgWx0uOvjdRz81LdD7HT3PQ@mail.gmail.com>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us> <20131202000200.GR2085@ando>
 <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>
 <529BDD5C.4040409@mrabarnett.plus.com> <l7gong$9a8$1@ger.gmane.org>
 <CADiSq7fGEUi6JjABebkkRxt4SxBPgWx0uOvjdRz81LdD7HT3PQ@mail.gmail.com>
Message-ID: <B89D4DC2-AA6F-4591-89F4-0EC14244D7EA@yahoo.com>

On Dec 1, 2013, at 23:37, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 2 December 2013 11:45, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> The newbie says it should be 4.5 but gets 4.499999999999999999999 so raises
>> issue on bug tracker stating that Python can't do arithmetic properly.
>> 
>> In [5]: 9/2
>> Out[5]: 4.5
>> 
>> Blast, lousy example :)
> 
> Getting a modern Python to fall into that trap at all is actually
> quite difficult - the way str.__repr__ works was changed a while ago
> to favour the nearest terminating approximation that is represented
> using the same IEEE754 bit pattern :)

I think the trap you're looking for is "9/2 == 4.5". Except that one doesn't fail either.

From denis.spir at gmail.com  Mon Dec  2 12:36:21 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 12:36:21 +0100
Subject: [Python-ideas] start, test, init
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <529C70B5.2050308@gmail.com>

On 12/01/2013 01:45 PM, spir wrote:
> [much stuff]

About flexibility, I spoke of intra-module flexibility, but forgot the 
inter-module side: a module's 'test' or 'start' (I think in particular at 
regression test suites, with a std 'test' name for unit tests).

denis

From denis.spir at gmail.com  Mon Dec  2 12:42:12 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 12:42:12 +0100
Subject: [Python-ideas] start, test, init -- in lib
In-Reply-To: <529B2F80.1070605@gmail.com>
References: <529B2F80.1070605@gmail.com>
Message-ID: <529C7214.2040708@gmail.com>

Hello,

Below is a minimal simulation of the proposal's core principal. Using my best 
knowledge of Python to have things light for the client, the necessity for it to 
make use of globals() and sys (see below, end of mod.py) nevertheless removes 
much of the proposal's nicety.

Anyway. It establishes the control logic as expressed in the proposal:

* if imported module:
     ~ run 'init' if any
     ~ else, run nothing
* if executed module:
     ~ run 'test', if any (see below)
     ~ else, run 'start', if any
     ~ else, run nothing (error?)

If both test & start exist, start is run if the magic var __run_fn__ == start.
The implementation also forwards command-line args to either test or start.

Below code:
* I called the control logic module 'top_fn', with a func of same name.
* To test it, a module 'mod'.
* To test the case of import, a module 'main'.

Try it by:
* running mod as is (with some command-line args): should run test
* running mod after uncommenting __run_fun__: should run start
* running main, importing mod: should run init

Non-nice aspects:
* The obligation to include in client module:
     import sys
     from topfn import topfn
     topfn(globals(), sys.argv)
   This should be compiler magic, certainly (eg make C main be the right func).
* No further possible compiler magic about differential compilation or such
   (see original proposal).

To be clear, the most important feature of the proposal is to establish standard 
magic names for high-level software structure, which are correctly taken into 
account by the language (read: compiler, probably). If this feature exists, it 
will be used, then recommanded.

Thank you,
Denis

==============================================================================
# topfn.py

from types import FunctionType

def topfn(env, argv):
     ''' Control which top function is executed in importing module.
     '''
     # relevant vars:
     name    = env["__name__"]               # know if module was run as app
     init    = env.get("init", None)         # init func
     test    = env.get("test", None)         # test func
     start   = env.get("start", None)        # start func for app run
     run_fn  = env.get("__run_fn__", None)   # choice between test & start

     # control logic:
     if name == "__main__":
         if isinstance(start, FunctionType) and run_fn == start:
             start(argv)
         elif isinstance(test, FunctionType):
             test(argv)
     elif isinstance(init, FunctionType):
         init()

==============================================================================
# mod.py

print("in mod")

def init ():
     print("Executing init.")

def test1 ():
     print("Executing test1.")
def test2 ():
     print("Executing test1.")
def test3 ():
     print("Executing test1.")
def test (argv):
     print("Executing test with argv: %s." % argv)
     test1()
     test2()
     test3()

def start (argv):
     print("Executing start with argv: %s." % argv)
#~ __run_fn__ = start

import sys
from topfn import topfn
topfn(globals(), sys.argv)

==============================================================================
# main.py

print("in main")
import mod

From denis.spir at gmail.com  Mon Dec  2 12:55:42 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 12:55:42 +0100
Subject: [Python-ideas] string codes & substring equality
In-Reply-To: <529BDD5C.4040409@mrabarnett.plus.com>
References: <529632E5.80907@mrabarnett.plus.com>
 <CACac1F9vXPnR+Y-i7ZzGVMPf=s8in9-h3mezC9U3sVWC8+A--Q@mail.gmail.com>
 <52966558.4030002@gmail.com> <529671F0.5040200@stoneleaf.us>
 <CAPTjJmrqPUH8=OYiz59LmThO1wpP9zbg2xWJsBXjgjhrwAG=zw@mail.gmail.com>
 <529685A0.3010305@stoneleaf.us> <20131128105308.GW2085@ando>
 <5297ACAA.2070803@stoneleaf.us> <20131129001248.GY2085@ando>
 <529BA25E.6000509@stoneleaf.us> <20131202000200.GR2085@ando>
 <CAP7+vJLJAiLStPTLh94VV-aSCX6sQ9=njTicBUr=MSaXyskPAA@mail.gmail.com>
 <529BDD5C.4040409@mrabarnett.plus.com>
Message-ID: <529C753E.5090801@gmail.com>

On 12/02/2013 02:07 AM, MRAB wrote:
>> I think you missed one. The programmer says it's 4. :-)
>>
> No. The programmer used to say it was 4, but now says it's 4.5, give or
> take a little rounding error. :-)

[off topic]

Yop, I just discover that: which division is executed (integral or real) is now 
determined by operator. Great reparation of a long-time Python bug, in my view:

python 3.3.1:

spir at ospir:~$ python3
Python 3.3.1 (default, Sep 25 2013, 19:29:01)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 9/2
4.5
>>> 9//2
4
>>> 9.0/2.0
4.5
>>> 9.0//2.0
4.0
>>>

python 2.7.4:

spir at ospir:~$ python2
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 9/2
4
>>> 9//2
4
>>> 9.0/2.0
4.5
>>> 9.0//2.0
4.0

[There remains a slight inconsistency, does't it? real div always returns a 
float independently of args' types, while integral division return a float if 
float args; is it in purpose? anyway, we'll survive it.]

Denis

From ncoghlan at gmail.com  Mon Dec  2 13:28:25 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 2 Dec 2013 22:28:25 +1000
Subject: [Python-ideas] start, test, init
In-Reply-To: <529C70B5.2050308@gmail.com>
References: <529B2F80.1070605@gmail.com>
	<529C70B5.2050308@gmail.com>
Message-ID: <CADiSq7f0L=+4MFhjjT-Og+8+Q+pOXCszuP-TNGDq4deQP3Lj3g@mail.gmail.com>

On 2 Dec 2013 21:37, "spir" <denis.spir at gmail.com> wrote:
>
> On 12/01/2013 01:45 PM, spir wrote:
>>
>> [much stuff]
>
>
> About flexibility, I spoke of intra-module flexibility, but forgot the
inter-module side: a module's 'test' or 'start' (I think in particular at
regression test suites, with a std 'test' name for unit tests).

That's already covered by unittest test discovery. Higher level structure
beyond the individual module is more the province of distutils-sig than it
is python-ideas or python-dev.

Cheers,
Nick.

>
> denis
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/f653ba4f/attachment.html>

From denis.spir at gmail.com  Mon Dec  2 13:35:31 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 13:35:31 +0100
Subject: [Python-ideas] start, test, init
In-Reply-To: <l7g6d9$nt8$1@ger.gmane.org>
References: <529B2F80.1070605@gmail.com> <l7g6d9$nt8$1@ger.gmane.org>
Message-ID: <529C7E93.6030602@gmail.com>

On 12/01/2013 09:32 PM, Ron Adam wrote:
>
>
> On 12/01/2013 06:45 AM, spir wrote:
>> # __main__ = start
>> def start (args):
>>      init()
>>      # process args
>>      ...
>>
>>
>> What do you think? (bis)
>
> I think you are describing good programming practices for larger modules. And I
> think most people who have been programming in python for any length of time
> develop some form of what you are describing.
>
> The only difference is that they wouldn't need the conditional logic at the
> bottom of the module.  But that also serves as bit of self documentation as to
> what the module or script does, is more flexible, and only adds a line or two if
> the rest is put into functions or classes.
>
>
> As for the testing case... I'd like for python to have a -t option that only
> sets a global name __test__ to True.  Then that covers most of your use cases
> without adding a whole lot.  Sometimes less is more.
>
>     if __test__:
>         test()
>     elif __name__ == '__main__':
>         main()
>
> That's just one possible combination of using __test__ and __name__.

That is all right, in my view; except maybe the point on flexibility (the 
solution of std func names is highly flexible). However, this does not address 2 
related issues:
* Why has the idiom "(el)if __name__ == '__main__':" has to be that obscure? and 
involve unnecessary knwoledge of Python arcanes (for a newcomer)
* Having a std (albeit not required) high-level software structure is a net gain 
for the community of Python programmers (instead of everyone doing their own 
way... when they do).

Denis

From denis.spir at gmail.com  Mon Dec  2 13:37:20 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 13:37:20 +0100
Subject: [Python-ideas] start, test, init
In-Reply-To: <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
References: <529B2F80.1070605@gmail.com>	<l7g6d9$nt8$1@ger.gmane.org>
 <CADiSq7ciye37eT64OiBg_joJDo9zE7i3sd009vbRcwaK4ncLnw@mail.gmail.com>
Message-ID: <529C7F00.5050100@gmail.com>

On 12/01/2013 10:13 PM, Nick Coghlan wrote:
> On 2 Dec 2013 06:34, "Ron Adam" <ron3200 at gmail.com> wrote:
>>
>>
>>
>> On 12/01/2013 06:45 AM, spir wrote:
>>>
>>> # __main__ = start
>>> def start (args):
>>>       init()
>>>       # process args
>>>       ...
>>>
>>>
>>> What do you think? (bis)
>>
>>
>> I think you are describing good programming practices for larger modules.
> And I think most people who have been programming in python for any length
> of time develop some form of what you are describing.
>>
>> The only difference is that they wouldn't need the conditional logic at
> the bottom of the module.  But that also serves as bit of self
> documentation as to what the module or script does, is more flexible, and
> only adds a line or two if the rest is put into functions or classes.
>>
>>
>> As for the testing case... I'd like for python to have a -t option that
> only sets a global name __test__ to True.  Then that covers most of your
> use cases without adding a whole lot.  Sometimes less is more.
>>
>>     if __test__:
>>         test()
>>     elif __name__ == '__main__':
>>         main()
>>
>> That's just one possible combination of using __test__ and __name__.
>
> Oh, interesting. Such an option could also alter -O and -OO to keep assert
> statements.
>
> I'd love to see that idea elaborated further, as there are a variety of
> questions around how it might work in practice (Compile time constant like
> __debug__? Always False builtin shadowed in the main module? If it works
> like __debug__ and/or affects -O and -OO, does it need a new naming
> convention in the pycache directory?)

This is very similar to my proposal, except that you'd control that from the 
code. No need to change command-line args.

Denis

From daniel at daniel-watkins.co.uk  Mon Dec  2 13:39:48 2013
From: daniel at daniel-watkins.co.uk (Daniel Watkins)
Date: Mon, 2 Dec 2013 12:39:48 +0000
Subject: [Python-ideas] start, test, init
In-Reply-To: <529C7E93.6030602@gmail.com>
References: <529B2F80.1070605@gmail.com> <l7g6d9$nt8$1@ger.gmane.org>
 <529C7E93.6030602@gmail.com>
Message-ID: <20131202123948.GI22882@daniel-watkins.co.uk>

On Mon, Dec 02, 2013 at 01:35:31PM +0100, spir wrote:
> * Why has the idiom "(el)if __name__ == '__main__':" has to be that
> obscure? and involve unnecessary knwoledge of Python arcanes (for a
> newcomer)

It seems to me that having to know the string "if __name__ ==
'__main__'" is about the same level of arcana as knowing the name of a
magic function that does the same.

> * Having a std (albeit not required) high-level software structure
> is a net gain for the community of Python programmers (instead of
> everyone doing their own way... when they do).

"if __name__ == '__main__'" is, by necessity, standard across all Python
programs.  And if I want to run the tests for a project, I would expect
'./setup.py test' to run them for me.

If a project has more than one entry point, how would I know which one
to run to get the tests to work?  Or would they all have to implement
the test function identically?


Dan

From denis.spir at gmail.com  Mon Dec  2 13:48:47 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 13:48:47 +0100
Subject: [Python-ideas] start, test, init
In-Reply-To: <CAMjeLr-oDXtNigYkV8-j-i2ymA4gKdqeGNgK1LiTtHZOPQoWBQ@mail.gmail.com>
References: <529B2F80.1070605@gmail.com>
 <CAMjeLr-oDXtNigYkV8-j-i2ymA4gKdqeGNgK1LiTtHZOPQoWBQ@mail.gmail.com>
Message-ID: <529C81AF.4090105@gmail.com>

On 12/02/2013 02:19 AM, Mark Janssen wrote:
>> There should be 3 top-level functions predefined by the language:
>> * 'start' : start func for a module run as an app (or 'main')
>> * 'test'  : main func of a module's test suite
>> * 'init'  : module init code, rather when imported
>
> 1)  I'd make "main" a keyword, used to denote the program start point,
> and let the interpreter bail-out if it encounters more than one "main"
> section.  Yes that keyword would only be used for that one, simple
> purpose, but it *is* a special purpose.  This would also solve the
> circular import issues, because now you have a well-defined *root* in
> which to construct a graph and check for cycles.
> 2)  I think testing should also be more "built-into" the interpreter
> environment.  I like the idea of a python "-t" flag or something which
> makes it runs doctests.  In which case, I'd also what a test()
> built-in, so that I can use it *within* the interpreter environment
> also, making sure invariants are preserved when modifying and re-using
> code.
> 3) with a "main" keyword, this wouldn't be absolutely necessary.
> You'll know where your top-level scope is.

I guess that is close to my point of view, if I undertand yours correctly: there 
could be an minimal amount of high-level software structure, at least about 
different running modes (exec, test, import), built into the language.

Denis


From ron3200 at gmail.com  Mon Dec  2 16:37:37 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Mon, 02 Dec 2013 09:37:37 -0600
Subject: [Python-ideas] start, test, init
In-Reply-To: <529C7E93.6030602@gmail.com>
References: <529B2F80.1070605@gmail.com> <l7g6d9$nt8$1@ger.gmane.org>
 <529C7E93.6030602@gmail.com>
Message-ID: <l7i9fn$mto$1@ger.gmane.org>



On 12/02/2013 06:35 AM, spir wrote:
>>
>> As for the testing case... I'd like for python to have a -t option that only
>> sets a global name __test__ to True.  Then that covers most of your use
>> cases
>> without adding a whole lot.  Sometimes less is more.
>>
>>     if __test__:
>>         test()
>>     elif __name__ == '__main__':
>>         main()
>>
>> That's just one possible combination of using __test__ and __name__.
>
> That is all right, in my view; except maybe the point on flexibility (the
> solution of std func names is highly flexible). However, this does not
> address 2 related issues:
> * Why has the idiom "(el)if __name__ == '__main__':" has to be that
> obscure? and involve unnecessary knwoledge of Python arcanes (for a newcomer)


I don't see it as obscure or arcane to compare a string to variable which 
bound to a string.  It's a very common and basic operation.

The only extra knowledge you need is to know that the __name__ attribute is 
set to "__main__" in the main module.  And that enables us to check if 
"this" is the main module in a direct way.

 >>> dir()
['__builtins__', '__doc__', '__loader__', '__name__', '__package__']
 >>> __name__
'__main__'
 >>> __name__ == '__main__'
True

The difference between python and compiled languages that must use 
functions to get things started, is that python is already executing the 
code, so it a matter of avoiding some parts at different times.

 > * Having a std (albeit not required) high-level software structure is a net
 > gain for the community of Python programmers (instead of everyone doing
 > their own way... when they do).

Most beginner books on python explain objects and name binding fairly 
early.  Knowing a module is also an object and has a __name__ attribute is 
something a python programmer should learn very early.  It helps them learn 
and understand how everything works in a more direct way. So rather than 
adding ... "You been doing this for a while, but many people do this other 
way ... ".  So it would need to be explained anyway.

Cheers,
    Ron


From apieum at gmail.com  Mon Dec  2 16:55:22 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Mon, 2 Dec 2013 16:55:22 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
Message-ID: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>

Hi,
I've made a try of a function that check object/class members for duck
typing.

For now I've basically called it isducktype(X, A)
it returns true if:
- X has all attributes of A
- X methods have same number of arguments than corresponding A methods

or if A.__ducktypecheck__(X) returns True

Behaviour looks like isinstance and issubclass (support tuples...).

To test it:
The patch with example is here:
https://gist.github.com/apieum/7751279

Or you can use my branch here:
https://bitbucket.org/gsalvan/cpython/commits/branch/default


Any opinion about a feature like this ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/405b9a83/attachment.html>

From guido at python.org  Mon Dec  2 16:59:25 2013
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Dec 2013 07:59:25 -0800
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
Message-ID: <CAP7+vJ+6F-v49N=YdP4OWc+3PdAPEZsUq2Z=a_sfLxm=wAaN_w@mail.gmail.com>

Sounds like a cute thing to post on PyPI and see how popular it becomes.
You have until the 3.5 release cycle starts to gather votes and
improvements. There are lots of edge cases I could see going wrong, so I'm
kind of skeptical of having this in the stdlib without strong caveats about
its usefulness. Maybe it could go in the inspect module, which is
traditionally full of heuristics.


On Mon, Dec 2, 2013 at 7:55 AM, Gregory Salvan <apieum at gmail.com> wrote:

> Hi,
> I've made a try of a function that check object/class members for duck
> typing.
>
> For now I've basically called it isducktype(X, A)
> it returns true if:
> - X has all attributes of A
> - X methods have same number of arguments than corresponding A methods
>
> or if A.__ducktypecheck__(X) returns True
>
> Behaviour looks like isinstance and issubclass (support tuples...).
>
> To test it:
> The patch with example is here:
> https://gist.github.com/apieum/7751279
>
> Or you can use my branch here:
> https://bitbucket.org/gsalvan/cpython/commits/branch/default
>
>
> Any opinion about a feature like this ?
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/8a0f8982/attachment.html>

From phd at phdru.name  Mon Dec  2 17:11:48 2013
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 2 Dec 2013 17:11:48 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
Message-ID: <20131202161148.GA19607@phdru.name>

Hi!

On Mon, Dec 02, 2013 at 04:55:22PM +0100, Gregory Salvan <apieum at gmail.com> wrote:
> For now I've basically called it isducktype(X, A)
> it returns true if:
> - X has all attributes of A

   Isn't the requirement too strong? When I need a file-like object I
seldom need more than .read() and .write() methods, and even of those I
seldom need both at once -- I usually need one or the other.
   Testing for strict conformance is not duck typing IMO -- it's
interface.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From rosuav at gmail.com  Mon Dec  2 17:22:51 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 3 Dec 2013 03:22:51 +1100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <20131202161148.GA19607@phdru.name>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
Message-ID: <CAPTjJmoo=csyH=nmmhhO=tt=ipLF68Pm8ZHwv8SJO3cMUgiObQ@mail.gmail.com>

On Tue, Dec 3, 2013 at 3:11 AM, Oleg Broytman <phd at phdru.name> wrote:
> On Mon, Dec 02, 2013 at 04:55:22PM +0100, Gregory Salvan <apieum at gmail.com> wrote:
>> For now I've basically called it isducktype(X, A)
>> it returns true if:
>> - X has all attributes of A
>
>    Isn't the requirement too strong? When I need a file-like object I
> seldom need more than .read() and .write() methods, and even of those I
> seldom need both at once -- I usually need one or the other.
>    Testing for strict conformance is not duck typing IMO -- it's
> interface.

But you could have a "ReadableFile" that has a .read() method and
whatever else you need, thus using that class as a sort of interface.
With a little careful scripting and introspection, you might even be
able to craft that by source code analysis, which would be extremely
cool (eg you run this script over your source code and it notes that
elements of this list are indexed with these five keywords, so it
constructs a class with those five so you can check what goes into the
list). Whether it's actually _useful_ or not remains to be seen, but
it would certainly be cool.

ChrisA

From apieum at gmail.com  Mon Dec  2 17:33:10 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Mon, 2 Dec 2013 17:33:10 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <20131202161148.GA19607@phdru.name>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
Message-ID: <CAAZsQLChasSQ6ODV8+TkrEq1mKUaEoSaREaNjceY0h_G4X6SUQ@mail.gmail.com>

Ok for pypi. (I've never tried to make python C extension, I feel it's
going to be epic :) )

@Oleg
you can override default behaviour with __ducktypecheck__
it's not as strict as an interface as var names can be differents and can
have defaults.
then instead of declaring an interface with "read" and "write", you just
create a base class with these methods.

What solution do you suggest ?
I've made a function that returns public members only, maybe this method
can be exposed in the api too.




2013/12/2 Oleg Broytman <phd at phdru.name>

> Hi!
>
> On Mon, Dec 02, 2013 at 04:55:22PM +0100, Gregory Salvan <apieum at gmail.com>
> wrote:
> > For now I've basically called it isducktype(X, A)
> > it returns true if:
> > - X has all attributes of A
>
>    Isn't the requirement too strong? When I need a file-like object I
> seldom need more than .read() and .write() methods, and even of those I
> seldom need both at once -- I usually need one or the other.
>    Testing for strict conformance is not duck typing IMO -- it's
> interface.
>
> Oleg.
> --
>      Oleg Broytman            http://phdru.name/            phd at phdru.name
>            Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/570b016d/attachment.html>

From rosuav at gmail.com  Mon Dec  2 17:38:26 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 3 Dec 2013 03:38:26 +1100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLChasSQ6ODV8+TkrEq1mKUaEoSaREaNjceY0h_G4X6SUQ@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
 <CAAZsQLChasSQ6ODV8+TkrEq1mKUaEoSaREaNjceY0h_G4X6SUQ@mail.gmail.com>
Message-ID: <CAPTjJmr-quvmpF5R6NGLjS_Hrh_C6p5ab=wrUByq0=k-T73-Fw@mail.gmail.com>

On Tue, Dec 3, 2013 at 3:33 AM, Gregory Salvan <apieum at gmail.com> wrote:
> Ok for pypi. (I've never tried to make python C extension, I feel it's going
> to be epic :)

Is there any reason this can't be implemented in pure Python? It'd be
a lot easier to figure things out that way. Maybe a C implementation
could come later, but for a first cut, this ought to be possible in
Python.

ChrisA

From apieum at gmail.com  Mon Dec  2 17:50:57 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Mon, 2 Dec 2013 17:50:57 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAPTjJmr-quvmpF5R6NGLjS_Hrh_C6p5ab=wrUByq0=k-T73-Fw@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
 <CAAZsQLChasSQ6ODV8+TkrEq1mKUaEoSaREaNjceY0h_G4X6SUQ@mail.gmail.com>
 <CAPTjJmr-quvmpF5R6NGLjS_Hrh_C6p5ab=wrUByq0=k-T73-Fw@mail.gmail.com>
Message-ID: <CAAZsQLBDfaS1=e+nAX4is+iBZ8KnXk25Su1fT4wSA=arVjBvng@mail.gmail.com>

Yes it can be done in python.
It's actually in C, essentially because I wanted to learn python internal,
and as it's quite similar to isinstance and issubclass.


2013/12/2 Chris Angelico <rosuav at gmail.com>

> On Tue, Dec 3, 2013 at 3:33 AM, Gregory Salvan <apieum at gmail.com> wrote:
> > Ok for pypi. (I've never tried to make python C extension, I feel it's
> going
> > to be epic :)
>
> Is there any reason this can't be implemented in pure Python? It'd be
> a lot easier to figure things out that way. Maybe a C implementation
> could come later, but for a first cut, this ought to be possible in
> Python.
>
> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/1fc1cc81/attachment.html>

From storchaka at gmail.com  Mon Dec  2 18:36:09 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 02 Dec 2013 19:36:09 +0200
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
Message-ID: <l7igdb$h0o$1@ger.gmane.org>

02.12.13 17:55, Gregory Salvan ???????(??):
> Hi,
> I've made a try of a function that check object/class members for duck
> typing.
>
> For now I've basically called it isducktype(X, A)
> it returns true if:
> - X has all attributes of A
> - X methods have same number of arguments than corresponding A methods
>
> or if A.__ducktypecheck__(X) returns True
>
> Behaviour looks like isinstance and issubclass (support tuples...).

class FooReader:
     def read(self, n=None): ...

class BarReader:
     def read(self, amount=-1): ...

class BazReader:
     def read(self, size=-1, chars=-1, firstline=False): ...

Instances of all three classes can be used where needed an object with 
the read() method which are called with one positional argument or 
without arguments. Is it possible to use isducktype() for such case?



From phd at phdru.name  Mon Dec  2 18:54:35 2013
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 2 Dec 2013 18:54:35 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLChasSQ6ODV8+TkrEq1mKUaEoSaREaNjceY0h_G4X6SUQ@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
 <CAAZsQLChasSQ6ODV8+TkrEq1mKUaEoSaREaNjceY0h_G4X6SUQ@mail.gmail.com>
Message-ID: <20131202175435.GA22609@phdru.name>

On Mon, Dec 02, 2013 at 05:33:10PM +0100, Gregory Salvan <apieum at gmail.com> wrote:
> @Oleg
> you can override default behaviour with __ducktypecheck__
> it's not as strict as an interface as var names can be differents and can
> have defaults.
> then instead of declaring an interface with "read" and "write", you just
> create a base class with these methods.
> 
> What solution do you suggest ?
> I've made a function that returns public members only, maybe this method
> can be exposed in the api too.
> 
> 2013/12/2 Oleg Broytman <phd at phdru.name>
> 
> > Hi!
> >
> > On Mon, Dec 02, 2013 at 04:55:22PM +0100, Gregory Salvan <apieum at gmail.com>
> > wrote:
> > > For now I've basically called it isducktype(X, A)
> > > it returns true if:
> > > - X has all attributes of A
> >
> >    Isn't the requirement too strong? When I need a file-like object I
> > seldom need more than .read() and .write() methods, and even of those I
> > seldom need both at once -- I usually need one or the other.
> >    Testing for strict conformance is not duck typing IMO -- it's
> > interface.

   I don't suggest any solution as I don't have a problem to solve with
interfaces. You are going to implement another interface checker in
addition to existing ones?
https://pypi.python.org/pypi?%3Aaction=search&term=interface&submit=search
   No objections from me.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From elazarg at gmail.com  Mon Dec  2 19:04:19 2013
From: elazarg at gmail.com (=?UTF-8?B?15DXnNei15bXqA==?=)
Date: Mon, 2 Dec 2013 20:04:19 +0200
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <20131202161148.GA19607@phdru.name>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
Message-ID: <CAPw6O2SMyM-oDrdJQgx8iPaaDJwtooonNrjDxAQn3XR2hHjmYg@mail.gmail.com>

On Dec 2, 2013 6:12 PM, "Oleg Broytman" <phd at phdru.name> wrote:
>
> Hi!
>
> On Mon, Dec 02, 2013 at 04:55:22PM +0100, Gregory Salvan <apieum at gmail.com>
wrote:
> > For now I've basically called it isducktype(X, A)
> > it returns true if:
> > - X has all attributes of A
>
>    Isn't the requirement too strong? When I need a file-like object I
> seldom need more than .read() and .write() methods, and even of those I
> seldom need both at once -- I usually need one or the other.
>    Testing for strict conformance is not duck typing IMO -- it's
> interface.
Interfaces are for nominal languages. Technically it should be called
x.is_structurally_subtype()

Elazar _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/fb90de44/attachment.html>

From phd at phdru.name  Mon Dec  2 19:10:52 2013
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 2 Dec 2013 19:10:52 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAPw6O2SMyM-oDrdJQgx8iPaaDJwtooonNrjDxAQn3XR2hHjmYg@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <20131202161148.GA19607@phdru.name>
 <CAPw6O2SMyM-oDrdJQgx8iPaaDJwtooonNrjDxAQn3XR2hHjmYg@mail.gmail.com>
Message-ID: <20131202181052.GA23868@phdru.name>

On Mon, Dec 02, 2013 at 08:04:19PM +0200, ?????????? <elazarg at gmail.com> wrote:
> On Dec 2, 2013 6:12 PM, "Oleg Broytman" <phd at phdru.name> wrote:
> > On Mon, Dec 02, 2013 at 04:55:22PM +0100, Gregory Salvan <apieum at gmail.com>
> wrote:
> > > For now I've basically called it isducktype(X, A)
> > > it returns true if:
> > > - X has all attributes of A
> >
> >    Isn't the requirement too strong? When I need a file-like object I
> > seldom need more than .read() and .write() methods, and even of those I
> > seldom need both at once -- I usually need one or the other.
> >    Testing for strict conformance is not duck typing IMO -- it's
> > interface.
> Interfaces are for nominal languages. Technically it should be called
> x.is_structurally_subtype()

   I'm sure the terminology in Python is quite stable. There are
interfaces (there are many implementations, there is one rejected PEP
and one deferred PEP) and there is duck typing. Formalizing duck typing
is IMO constructing an interface.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From abarnert at yahoo.com  Mon Dec  2 19:42:22 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 2 Dec 2013 10:42:22 -0800
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
Message-ID: <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>

Shouldn't this be tied to ABCs rather than redesigning and reimplementing most of ABC to add a little bit on top?

Sent from a random iPhone

On Dec 2, 2013, at 7:55, Gregory Salvan <apieum at gmail.com> wrote:

> Hi,
> I've made a try of a function that check object/class members for duck typing.
> 
> For now I've basically called it isducktype(X, A)
> it returns true if:
> - X has all attributes of A
> - X methods have same number of arguments than corresponding A methods
> 
> or if A.__ducktypecheck__(X) returns True
> 
> Behaviour looks like isinstance and issubclass (support tuples...).
> 
> To test it:
> The patch with example is here:
> https://gist.github.com/apieum/7751279
> 
> Or you can use my branch here:
> https://bitbucket.org/gsalvan/cpython/commits/branch/default
> 
> 
> Any opinion about a feature like this ?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/7989bb57/attachment.html>

From denis.spir at gmail.com  Mon Dec  2 19:51:13 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 02 Dec 2013 19:51:13 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
Message-ID: <529CD6A1.8080202@gmail.com>

On 12/02/2013 04:55 PM, Gregory Salvan wrote:
> I've made a try of a function that check object/class members for duck
> typing.
>
> For now I've basically called it isducktype(X, A)
> it returns true if:
> - X has all attributes of A
> - X methods have same number of arguments than corresponding A methods

Just a question:
I wonder about actual usefulness. For a strict safe check you'd also want to 
control method param types and names, not only number. Indeed, param types are 
out of question here, but what about param names? Since you implement that in C, 
is it at all possible to get method param names? (Well, in fact, maybe even 
Python's self-examination power allows that, does it?)

Denis

From zuo at chopin.edu.pl  Tue Dec  3 00:10:47 2013
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Tue, 03 Dec 2013 00:10:47 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
Message-ID: <821805489b4fb932405a0f26749201dd@chopin.edu.pl>

02.12.2013 19:42, Andrew Barnert wrote:
> Shouldn't this be tied to ABCs rather than redesigning and
> reimplementing most of ABC to add a little bit on top?

+1.

*j


From apieum at gmail.com  Tue Dec  3 02:21:15 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Tue, 3 Dec 2013 02:21:15 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <l7igdb$h0o$1@ger.gmane.org>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <l7igdb$h0o$1@ger.gmane.org>
Message-ID: <CAAZsQLDosSLkqr15QuiCDy9L0BEne7D4QQW_kE+r+u+oZOUf5Q@mail.gmail.com>

@Serhiy
Not really (only with FooReader, BarReader) as it checks if functions
co_argcount and co_kwonlyargcount are equals.
In this case it requires BazReader.__ducktypecheck__ to be implemented.

I had a doubt when doing this, I agree with you isducktype(BazReader,
FooReader) can return True (whereas  isducktype(FooReader, BazReader)
should stay false)
This can easily be changed.

@Oleg:
The fact there is a lot of interface implementations means certainly it's
something needed.
Each implementation I've seen add burden on objects whereas the only thing
personnally I need, is to be sure I'll be able to call some methods off
injected objects.

@spir:
Names can easily be checked, but I can't see why doing this, it add
constraints to developpers without adding safety.

@Andrew +1
Maybe inspect is more appropriate as it provides ArgSpec and getmembers ?

My code is crappy, I've just do it like that to train me in C then tought
it might be interesting to get feedback.


2013/12/2 Serhiy Storchaka <storchaka at gmail.com>

> 02.12.13 17:55, Gregory Salvan ???????(??):
>
>  Hi,
>> I've made a try of a function that check object/class members for duck
>> typing.
>>
>> For now I've basically called it isducktype(X, A)
>> it returns true if:
>> - X has all attributes of A
>> - X methods have same number of arguments than corresponding A methods
>>
>> or if A.__ducktypecheck__(X) returns True
>>
>> Behaviour looks like isinstance and issubclass (support tuples...).
>>
>
> class FooReader:
>     def read(self, n=None): ...
>
> class BarReader:
>     def read(self, amount=-1): ...
>
> class BazReader:
>     def read(self, size=-1, chars=-1, firstline=False): ...
>
> Instances of all three classes can be used where needed an object with the
> read() method which are called with one positional argument or without
> arguments. Is it possible to use isducktype() for such case?
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131203/da9facc3/attachment-0001.html>

From abarnert at yahoo.com  Tue Dec  3 08:56:37 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 2 Dec 2013 23:56:37 -0800
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLDosSLkqr15QuiCDy9L0BEne7D4QQW_kE+r+u+oZOUf5Q@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <l7igdb$h0o$1@ger.gmane.org>
 <CAAZsQLDosSLkqr15QuiCDy9L0BEne7D4QQW_kE+r+u+oZOUf5Q@mail.gmail.com>
Message-ID: <341911A7-D99C-4BE5-A158-6901864AE09E@yahoo.com>

On Dec 2, 2013, at 17:21, Gregory Salvan <apieum at gmail.com> wrote:

> @Andrew +1
> Maybe inspect is more appropriate as it provides ArgSpec and getmembers ?

Sure, using inspect for the implementation _also_ makes sense. But my point is more about the interface to your new thing. 

Your thing checks most of the same stuff that ABCs check, but maybe not all of it, and also adds checking for compatible argspecs. (I believe that's the only new feature it adds; am I right?)

Meanwhile, it uses a new isducktype function instead of working with normal issubclass/isinstance, and a completely different way of registering explicit compliance, and so on.

Instead, why not just extend ABCMeta with a new subclass that just adds the check for compatible argspecs (maybe using inspect, as you say) and takes advantage of all the other stuff ABC already does? Besides being less code,  which isn't that big of a deal, it's less of a radical conceptual change, which is a bigger deal, and it means that all kinds of code that relies on isinstance today can work with no changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131202/b9d3446c/attachment.html>

From ncoghlan at gmail.com  Tue Dec  3 14:05:52 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 3 Dec 2013 23:05:52 +1000
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
Message-ID: <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>

On 3 December 2013 04:42, Andrew Barnert <abarnert at yahoo.com> wrote:
> Shouldn't this be tied to ABCs rather than redesigning and reimplementing
> most of ABC to add a little bit on top?

This seems like an apropos place for this link:
http://docs.python.org/3/library/abc#abc.ABCMeta.__subclasshook__

That's the existing "formalised ducktyping" hook, that already allows
things like the following:

>>> class MyClass:
...     def __len__(self):
...         return 0
...
>>> from collections.abc import Sized
>>> isinstance(MyClass(), Sized)
True
>>> issubclass(MyClass, Sized)
True

The advantage ABCs have over ducktyping alone is that subclassing and
explicit registration allow ambiguities like the one between the
Sequence and Mapping interfaces to be resolved (you can't always just
ducktype those, as they have the same methods - they differ only in
how the __*item__ methods handle slice objects).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From apieum at gmail.com  Wed Dec  4 06:22:06 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Wed, 4 Dec 2013 06:22:06 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
 <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
Message-ID: <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>

Sorry Andrew, I'm not sure to understand all your point.
I don't believe it reimplements things of ABC, or I don't see what.

I agree ABC solution is the cleanest, whereas I would be able to check
functions, stay on a runtime checking and being free to define or not a
kind of protocol.

Before reading your comments on ABC, I've released the python version of
"isducktype": https://github.com/apieum/ducktype
I've modified some behaviours within your comments and made it more stable.

It would be interesting to have the version wich use ABCMeta, so I'm going
to implement it.
I've thought to name it SignedMeta, or ProtocolMeta, any opinion ?







2013/12/3 Nick Coghlan <ncoghlan at gmail.com>

> On 3 December 2013 04:42, Andrew Barnert <abarnert at yahoo.com> wrote:
> > Shouldn't this be tied to ABCs rather than redesigning and reimplementing
> > most of ABC to add a little bit on top?
>
> This seems like an apropos place for this link:
> http://docs.python.org/3/library/abc#abc.ABCMeta.__subclasshook__
>
> That's the existing "formalised ducktyping" hook, that already allows
> things like the following:
>
> >>> class MyClass:
> ...     def __len__(self):
> ...         return 0
> ...
> >>> from collections.abc import Sized
> >>> isinstance(MyClass(), Sized)
> True
> >>> issubclass(MyClass, Sized)
> True
>
> The advantage ABCs have over ducktyping alone is that subclassing and
> explicit registration allow ambiguities like the one between the
> Sequence and Mapping interfaces to be resolved (you can't always just
> ducktype those, as they have the same methods - they differ only in
> how the __*item__ methods handle slice objects).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131204/aaf3ddb8/attachment.html>

From abarnert at yahoo.com  Wed Dec  4 09:19:30 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 4 Dec 2013 00:19:30 -0800
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
 <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
 <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>
Message-ID: <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>

On Dec 3, 2013, at 21:22, Gregory Salvan <apieum at gmail.com> wrote:

> Sorry Andrew, I'm not sure to understand all your point.
> I don't believe it reimplements things of ABC, or I don't see what.

Almost everything you did reimplements features of ABCs. The builtin isinstance and issubclass functions can check whether you implement the methods of a type, and you can override that by explicitly declaring that you implement the type. That's the core feature set of your proposal, and of ABCs; you've just implemented the same thing with a different API, plus and minus a few minor details.

If you haven't read PEP 3119, you really should. (It's also worth reading rejected predecessors like 245/246 and successors like 3124, and of course the pre-existing third party stuff like PyProtocols and the Interface types in Twisted and Zope.)

> I agree ABC solution is the cleanest, whereas I would be able to check functions, stay on a runtime checking and being free to define or not a kind of protocol.

I don't understand what you mean here. ABCs can check methods. They do so at runtime. You're free to define or not define any protocol you want.

What you have to add is checking the argspecs of the methods rather than just their names. That's a great idea, but it's a much better idea as an addition to what's already there than as a complete parallel API to very similar functionality. And if this were added to the stdlib, I'm pretty sure it would be a modification/extension of the abc module, not a competing and similar but not identical module.

> Before reading your comments on ABC, I've released the python version of "isducktype": https://github.com/apieum/ducktype
> I've modified some behaviours within your comments and made it more stable.
> 
> It would be interesting to have the version wich use ABCMeta, so I'm going to implement it. 
> I've thought to name it SignedMeta, or ProtocolMeta, any opinion ?

First, it sounds like you're focusing on the wrong part of the name--ABCMeta is a metaclass for implementing ABCs; leaving out the ABC part sounds like you're designing a metaclass for something completely unrelated.

Less seriously, ProtocolMeta sounds like it's going to add more PyProtocols-style functionality (in particular, adapters from one protocol to another), but there's only so many synonyms available and they're all taken, so I don't think that's a major problem. As for SignedMeta, I don't even know what that's meant to imply. Is it to do with method signatures? If so, I think "signature" is a better word; you don't really think about anything being "signed" in a method signature; that's dragging in associations from a different meaning of the word.

But I don't have any better suggestions. Protocol, interface, prototype, type, abstract type, abstract base... As I said, all of the names have already been used up, repeatedly, and none of them have any particular connotations that would explain how your module, metaclass, function, etc. differ from abc. 

In fact, I'd probably just come up with a stupid package name like abc2 and stick in modules named abc, collections.abc, and numbers and expose names identical to the stdlib ones, so I could play around with it by just using abc2.isinstance, abc2.abc.ABCMeta, abc2.collections.abc.Mapping, etc. Or I could just toss some "from abc2 import"s around. But for any serious use beyond playing around, that would be quite confusing.

> 2013/12/3 Nick Coghlan <ncoghlan at gmail.com>
>> On 3 December 2013 04:42, Andrew Barnert <abarnert at yahoo.com> wrote:
>> > Shouldn't this be tied to ABCs rather than redesigning and reimplementing
>> > most of ABC to add a little bit on top?
>> 
>> This seems like an apropos place for this link:
>> http://docs.python.org/3/library/abc#abc.ABCMeta.__subclasshook__
>> 
>> That's the existing "formalised ducktyping" hook, that already allows
>> things like the following:
>> 
>> >>> class MyClass:
>> ...     def __len__(self):
>> ...         return 0
>> ...
>> >>> from collections.abc import Sized
>> >>> isinstance(MyClass(), Sized)
>> True
>> >>> issubclass(MyClass, Sized)
>> True
>> 
>> The advantage ABCs have over ducktyping alone is that subclassing and
>> explicit registration allow ambiguities like the one between the
>> Sequence and Mapping interfaces to be resolved (you can't always just
>> ducktype those, as they have the same methods - they differ only in
>> how the __*item__ methods handle slice objects).
>> 
>> Cheers,
>> Nick.
>> 
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131204/693f8e95/attachment-0001.html>

From ncoghlan at gmail.com  Wed Dec  4 13:20:56 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 4 Dec 2013 22:20:56 +1000
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
 <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
 <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>
 <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>
Message-ID: <CADiSq7d8rNG+8ES7TJuR=9OfnFgT7aNPfxw-HvK8Vo4XQ1Rtng@mail.gmail.com>

On 4 December 2013 18:19, Andrew Barnert <abarnert at yahoo.com> wrote:
> I don't understand what you mean here. ABCs can check methods. They do so at
> runtime. You're free to define or not define any protocol you want.
>
> What you have to add is checking the argspecs of the methods rather than
> just their names. That's a great idea, but it's a much better idea as an
> addition to what's already there than as a complete parallel API to very
> similar functionality. And if this were added to the stdlib, I'm pretty sure
> it would be a modification/extension of the abc module, not a competing and
> similar but not identical module.

It should also be based on the rich function inspect.signature API
(introduced in http://www.python.org/dev/peps/pep-0362/), not on the
older low level interfaces (the function signature API has been
backported to Python 2.x as the funcsigs module on PyPI).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Wed Dec  4 17:09:43 2013
From: brett at python.org (Brett Cannon)
Date: Wed, 4 Dec 2013 11:09:43 -0500
Subject: [Python-ideas] 2 more admins for the list
Message-ID: <CAP1=2W6QAuy4o+ORSt9K3vU1ouooiK9QjTHJ-n9VGZ8CUA7J=w@mail.gmail.com>

FYI both Ethan Furman and Tal Einat have joined Titus and myself in
administering this list.

And while I'm thinking about it, just a reminder about how to let us know
about CoC violations. Please email/cc python-ideas-owner@ when you think
there has been an infraction. There is enough volume on the list you can't
assume we read every email. We also try to stay objective on this topic so
chances are we won't bring up any CoC issues in order to stay neutral.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131204/52d8b74e/attachment.html>

From apieum at gmail.com  Thu Dec  5 07:48:35 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Thu, 5 Dec 2013 07:48:35 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
 <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
 <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>
 <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>
Message-ID: <CAAZsQLD+H6CtjH5QWRx-Tq3mS912ZcnPkQVEunxe3FPwB5WwOA@mail.gmail.com>

Thank you for taking time to answer.
I trust you, but I've sometimes the feeling we're talking about different
things.

I didn't know PEP 3124 and PEP 0362, they are great ressources. Signature
is particularly usefull, it's a pity it's not documented, I'll suggest
something further.

I'm ok with what you said (I talked effectively about signatures).
The matter I see, is that attributes and methods can be set dynamically
(that's why I talked about runtime), either I don't manage these cases, or
I have to extend ABCMeta and override __instancecheck__ and
__subclasscheck__.
The second solution would make duplicate code unless I extract ABCMeta
cache and registry operations in a dedicated object.
This would be a lot of change and despite _abc_* attributes are protected,
I'm afraid some library use them.

I've made an example of failure to illustrate dynamic setting matter:
https://gist.github.com/apieum/7800992
Maybe it's more effective for attributes which are often defined in
__init__ uniquely.

Whereas all of this is really interesting we are quite far from the initial
target which was simply to avoid these cases:
>>> class MyClass:
...     def __len__(self, enum):
...         return 0
...
>>> from collections.abc import Sized
>>> isinstance(MyClass(), Sized)
True
>>> issubclass(MyClass, Sized)
True
>>> len(MyClass())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __len__() missing 1 required positional argument: 'enum'


I'll dig if I can find a better proposal considering all you suggestion.




2013/12/4 Andrew Barnert <abarnert at yahoo.com>

> On Dec 3, 2013, at 21:22, Gregory Salvan <apieum at gmail.com> wrote:
>
> Sorry Andrew, I'm not sure to understand all your point.
> I don't believe it reimplements things of ABC, or I don't see what.
>
>
> Almost everything you did reimplements features of ABCs. The builtin
> isinstance and issubclass functions can check whether you implement the
> methods of a type, and you can override that by explicitly declaring that
> you implement the type. That's the core feature set of your proposal, and
> of ABCs; you've just implemented the same thing with a different API, plus
> and minus a few minor details.
>
> If you haven't read PEP 3119, you really should. (It's also worth reading
> rejected predecessors like 245/246 and successors like 3124, and of course
> the pre-existing third party stuff like PyProtocols and the Interface types
> in Twisted and Zope.)
>
> I agree ABC solution is the cleanest, whereas I would be able to check
> functions, stay on a runtime checking and being free to define or not a
> kind of protocol.
>
>
> I don't understand what you mean here. ABCs can check methods. They do so
> at runtime. You're free to define or not define any protocol you want.
>
> What you have to add is checking the argspecs of the methods rather than
> just their names. That's a great idea, but it's a much better idea as an
> addition to what's already there than as a complete parallel API to very
> similar functionality. And if this were added to the stdlib, I'm pretty
> sure it would be a modification/extension of the abc module, not a
> competing and similar but not identical module.
>
> Before reading your comments on ABC, I've released the python version of
> "isducktype": https://github.com/apieum/ducktype
> I've modified some behaviours within your comments and made it more stable.
>
> It would be interesting to have the version wich use ABCMeta, so I'm going
> to implement it.
> I've thought to name it SignedMeta, or ProtocolMeta, any opinion ?
>
>
> First, it sounds like you're focusing on the wrong part of the
> name--ABCMeta is a metaclass for implementing ABCs; leaving out the ABC
> part sounds like you're designing a metaclass for something completely
> unrelated.
>
> Less seriously, ProtocolMeta sounds like it's going to add more
> PyProtocols-style functionality (in particular, adapters from one protocol
> to another), but there's only so many synonyms available and they're all
> taken, so I don't think that's a major problem. As for SignedMeta, I don't
> even know what that's meant to imply. Is it to do with method signatures?
> If so, I think "signature" is a better word; you don't really think about
> anything being "signed" in a method signature; that's dragging in
> associations from a different meaning of the word.
>
> But I don't have any better suggestions. Protocol, interface, prototype,
> type, abstract type, abstract base... As I said, all of the names have
> already been used up, repeatedly, and none of them have any particular
> connotations that would explain how your module, metaclass, function, etc.
> differ from abc.
>
> In fact, I'd probably just come up with a stupid package name like abc2
> and stick in modules named abc, collections.abc, and numbers and expose
> names identical to the stdlib ones, so I could play around with it by just
> using abc2.isinstance, abc2.abc.ABCMeta, abc2.collections.abc.Mapping, etc.
> Or I could just toss some "from abc2 import"s around. But for any serious
> use beyond playing around, that would be quite confusing.
>
> 2013/12/3 Nick Coghlan <ncoghlan at gmail.com>
>
>> On 3 December 2013 04:42, Andrew Barnert <abarnert at yahoo.com> wrote:
>> > Shouldn't this be tied to ABCs rather than redesigning and
>> reimplementing
>> > most of ABC to add a little bit on top?
>>
>> This seems like an apropos place for this link:
>> http://docs.python.org/3/library/abc#abc.ABCMeta.__subclasshook__
>>
>> That's the existing "formalised ducktyping" hook, that already allows
>> things like the following:
>>
>> >>> class MyClass:
>> ...     def __len__(self):
>> ...         return 0
>> ...
>> >>> from collections.abc import Sized
>> >>> isinstance(MyClass(), Sized)
>> True
>> >>> issubclass(MyClass, Sized)
>> True
>>
>> The advantage ABCs have over ducktyping alone is that subclassing and
>> explicit registration allow ambiguities like the one between the
>> Sequence and Mapping interfaces to be resolved (you can't always just
>> ducktype those, as they have the same methods - they differ only in
>> how the __*item__ methods handle slice objects).
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131205/14472989/attachment-0001.html>

From ncoghlan at gmail.com  Thu Dec  5 10:46:54 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 5 Dec 2013 19:46:54 +1000
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CAAZsQLD+H6CtjH5QWRx-Tq3mS912ZcnPkQVEunxe3FPwB5WwOA@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
 <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
 <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>
 <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>
 <CAAZsQLD+H6CtjH5QWRx-Tq3mS912ZcnPkQVEunxe3FPwB5WwOA@mail.gmail.com>
Message-ID: <CADiSq7eLx8gB+ZFTBDq6ofro1_Ai4H=0PpoD6TQ7q-xWkrQa3Q@mail.gmail.com>

On 5 December 2013 16:48, Gregory Salvan <apieum at gmail.com> wrote:
> Thank you for taking time to answer.
> I trust you, but I've sometimes the feeling we're talking about different
> things.
>
> I didn't know PEP 3124 and PEP 0362, they are great ressources. Signature is
> particularly usefull, it's a pity it's not documented,

It's covered in the inspect module docs:
http://docs.python.org/3/library/inspect#introspecting-callables-with-the-signature-object

> I'm ok with what you said (I talked effectively about signatures).
> The matter I see, is that attributes and methods can be set dynamically
> (that's why I talked about runtime), either I don't manage these cases, or I
> have to extend ABCMeta and override __instancecheck__ and __subclasscheck__.
> The second solution would make duplicate code unless I extract ABCMeta cache
> and registry operations in a dedicated object.

Use ABCMeta.__subclasshook__, that's what it is for.

It's deliberate that ABCs don't check for instance attributes, and
it's only the automatic caching behaviour (required for reasonable
performance) that prevents the ABC machinery from recognising dynamic
class updates by default. However, you *can* make it recognise updates
by invalidating the caches (by doing an explicit registration on any
ABC), and I'd be open to an "abc.invalidate_caches()" function as a
more obvious way to force cache invalidation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From apieum at gmail.com  Thu Dec  5 16:33:05 2013
From: apieum at gmail.com (Gregory Salvan)
Date: Thu, 5 Dec 2013 16:33:05 +0100
Subject: [Python-ideas] check interfaces, isducktype(X, A)
In-Reply-To: <CADiSq7eLx8gB+ZFTBDq6ofro1_Ai4H=0PpoD6TQ7q-xWkrQa3Q@mail.gmail.com>
References: <CAAZsQLAMaVXXDwkD+e8AaC7JCCLOhibUdaHvMgYYLPBsw_r0Zw@mail.gmail.com>
 <096C000A-37EE-4D41-BB47-40B7A2EE0D56@yahoo.com>
 <CADiSq7f3-vgcLXVJS0LkD-Pwa272nT6f6J9TS53HJkzvkYJjBg@mail.gmail.com>
 <CAAZsQLC9o1X-ewJGejmYJ68Zhi++ANEgJOxmiiF71AkSFUVO8A@mail.gmail.com>
 <487E1BD1-63D1-47E1-8CF8-8A12F2A49FEE@yahoo.com>
 <CAAZsQLD+H6CtjH5QWRx-Tq3mS912ZcnPkQVEunxe3FPwB5WwOA@mail.gmail.com>
 <CADiSq7eLx8gB+ZFTBDq6ofro1_Ai4H=0PpoD6TQ7q-xWkrQa3Q@mail.gmail.com>
Message-ID: <CAAZsQLCFdi_A6srTV-xm1Ap_dEwNX0OxE7JJg+V0qYSwuxD5+g@mail.gmail.com>

>
> It's covered in the inspect module docs:
>
> Oups, I've missed it was introduced in v3.3


> I'd be open to an "abc.invalidate_caches()" function as a
>
more obvious way to force cache invalidation.
>

Does this implementation make sense ?
https://gist.github.com/apieum/7805431
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131205/ab0cef7a/attachment.html>

From ericsnowcurrently at gmail.com  Fri Dec  6 17:29:25 2013
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 6 Dec 2013 09:29:25 -0700
Subject: [Python-ideas] Replacing the if __name__ == "__main__" idiom
 (was Re: making a module callable)
In-Reply-To: <CADiSq7dBXCAL-N=xHq0Y-dNuzWp-km68pm1fFCHJLw7VVLXApw@mail.gmail.com>
References: <CAN8d9gmrYBQ7-jgEspoJ-tKo_-vZfDpzaPa-8gW4jSWz-sMTLA@mail.gmail.com>
 <20131125071932.GA65531@cskk.homeip.net>
 <CADiSq7cRkMisYe8dRPL_Hc2a9M9AOP1aBKaPV=Y1kdDqJxyweA@mail.gmail.com>
 <20131125141220.GE2085@ando> <20131125144244.1cb160f0@anarchist>
 <CAP7+vJL6P+bh1JZ_Hbu9+cGZS+CGzS2SpYmOnQ0uOvQF=Wi8ig@mail.gmail.com>
 <CADiSq7eXKxgdaA2xeEgv_rBP2T+NGtBNk2bboHT_-pNpG2mT5w@mail.gmail.com>
 <CAP7+vJJHZdEqa_n44EA6KfANSUNZiARMyuDsGQwnGfHZJoN2wQ@mail.gmail.com>
 <20131125171603.1812e9fd@anarchist>
 <CAP7+vJ+fdcbZ6UNcqWu3TYrKTvOZkZ=aqkt7EWrXDz4+RDPAYA@mail.gmail.com>
 <CAHVvXxT10Jab=ShNk8Jje-Vz13ci6n8-k1L-LqbG2kiOC2OcmA@mail.gmail.com>
 <CAMjeLr84XXmiynurD7r6OpTskX=N+zjeu_TeDYKrTJS2n2VvsQ@mail.gmail.com>
 <CADiSq7dBXCAL-N=xHq0Y-dNuzWp-km68pm1fFCHJLw7VVLXApw@mail.gmail.com>
Message-ID: <CALFfu7Ba05bPL_LQN23CbdWZOGfOZ18h=LQ+RY8Va3pKEgpbrQ@mail.gmail.com>

On Tue, Nov 26, 2013 at 3:01 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On 26 Nov 2013 10:00, "Mark Janssen" <dreamingforward at gmail.com> wrote:
>>
>> The only other possibility not mentioned thus far is to have a main.py
>> file and force python programs to start from it.
>
> Looking for a __main__.py module (or submodule) is the way directory,
> zipfile and package execution work, so this style is already possible
today
> for anyone that wants or needs it.

It also doesn't work so well for non-packages. :)

That said, I like how __main__.py makes a clear separation between the
package and its script form. As I said in my opening email, I think that
distinction is very fuzzy for modules in general and bites both beginners
and advanced users. For me anything that proposed replacing the current "if
__name__ == '__main__'" idiom should help make the script/module separation
more distinct.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131206/1435822c/attachment.html>

From ericsnowcurrently at gmail.com  Fri Dec  6 17:31:47 2013
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 6 Dec 2013 09:31:47 -0700
Subject: [Python-ideas] Replacing the if __name__ == "__main__" idiom
 (was Re: making a module callable)
In-Reply-To: <CALFfu7Ba05bPL_LQN23CbdWZOGfOZ18h=LQ+RY8Va3pKEgpbrQ@mail.gmail.com>
References: <CAN8d9gmrYBQ7-jgEspoJ-tKo_-vZfDpzaPa-8gW4jSWz-sMTLA@mail.gmail.com>
 <20131125071932.GA65531@cskk.homeip.net>
 <CADiSq7cRkMisYe8dRPL_Hc2a9M9AOP1aBKaPV=Y1kdDqJxyweA@mail.gmail.com>
 <20131125141220.GE2085@ando> <20131125144244.1cb160f0@anarchist>
 <CAP7+vJL6P+bh1JZ_Hbu9+cGZS+CGzS2SpYmOnQ0uOvQF=Wi8ig@mail.gmail.com>
 <CADiSq7eXKxgdaA2xeEgv_rBP2T+NGtBNk2bboHT_-pNpG2mT5w@mail.gmail.com>
 <CAP7+vJJHZdEqa_n44EA6KfANSUNZiARMyuDsGQwnGfHZJoN2wQ@mail.gmail.com>
 <20131125171603.1812e9fd@anarchist>
 <CAP7+vJ+fdcbZ6UNcqWu3TYrKTvOZkZ=aqkt7EWrXDz4+RDPAYA@mail.gmail.com>
 <CAHVvXxT10Jab=ShNk8Jje-Vz13ci6n8-k1L-LqbG2kiOC2OcmA@mail.gmail.com>
 <CAMjeLr84XXmiynurD7r6OpTskX=N+zjeu_TeDYKrTJS2n2VvsQ@mail.gmail.com>
 <CADiSq7dBXCAL-N=xHq0Y-dNuzWp-km68pm1fFCHJLw7VVLXApw@mail.gmail.com>
 <CALFfu7Ba05bPL_LQN23CbdWZOGfOZ18h=LQ+RY8Va3pKEgpbrQ@mail.gmail.com>
Message-ID: <CALFfu7AKATSJ3mzT0N4gzXCgZ4KX_QVAK6sZD8dWrq9Hue+GGA@mail.gmail.com>

On Dec 6, 2013 9:29 AM, "Eric Snow" <ericsnowcurrently at gmail.com> wrote:
> That said, I like how __main__.py makes a clear separation between the
package and its script form. As I said in my opening email, I think that
distinction is very fuzzy for modules in general and bites both beginners
and advanced users. For me anything that proposed replacing the current "if
__name__ == '__main__'" idiom should help make the script/module separation
more distinct.

Regardless, it may be worth adding an additional approach to highlighting
the script/module duality.  We could raise a warning when a module
(findable on sys.path) is run as script unless via the "-m" flag.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131206/a70e2e14/attachment.html>

From mcepl at redhat.com  Fri Dec  6 23:09:46 2013
From: mcepl at redhat.com (=?utf-8?B?TWF0xJtq?= Cepl)
Date: Fri, 6 Dec 2013 23:09:46 +0100
Subject: [Python-ideas] why is not request in httplib pretty printed?
Message-ID: <20131206220940.GA15554@wycliff.ceplovi.cz>

Hi,

looking at 
http://hg.python.org/cpython/file/543c76769c14/Lib/http/client.py#l847 
(but this code has been same since like forever) I see that the 
HTTP request is NOT pretty printed:

     if self.debuglevel > 0:
         print("send:", repr(data))

whereas response in effect (because every header is printed 
separately) is. Wouldn't it be better to pretty print the 
request as well?

Otherwise I get quite unreadable debugging logs like the 
following (notice how much response is more readable than 
request).

Mat?j

matej at wycliff: urllib2_kerberos (next *%)$ python test_trac.py 
send: 'GET /desktopqe-backlog/login HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'
reply: 'HTTP/1.1 401 Authorization Required\r\n'
header: Date: Fri, 06 Dec 2013 17:11:13 GMT
header: Server: Apache
header: WWW-Authenticate: Negotiate
header: WWW-Authenticate: Basic realm="Kerberos Login"
header: Content-Length: 483
header: Connection: close
header: Content-Type: text/html; charset=iso-8859-1
2013-12-06 18:11:15,406 DEBUG inside http_error_401
2013-12-06 18:11:15,407 DEBUG retry count: 1
2013-12-06 18:11:15,407 DEBUG req.get_host() returned someserver.redhat.com
2013-12-06 18:11:15,407 DEBUG authGSSClientInit() succeeded
2013-12-06 18:11:15,486 DEBUG authGSSClientStep() succeeded
2013-12-06 18:11:15,486 DEBUG authGSSClientResponse() succeeded
send: 'GET /desktopqe-backlog/login HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nAuthorization: Negotiate YIICdQYJKoZIhvcSA---some-junk--GZDggzt\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'
reply: 'HTTP/1.1 302 Found\r\n'
header: Date: Fri, 06 Dec 2013 17:11:14 GMT
header: Server: Apache
header: WWW-Authenticate: Negotiate YIGZBgkqhkiG9xIB--other-junk--NOxMdOR/5
header: Location: https://someserver.redhat.com/desktopqe-backlog
header: Content-Length: 0
header: Pragma: no-cache
header: Cache-Control: no-cache
header: Expires: Fri, 01 Jan 1999 00:00:00 GMT
header: Set-Cookie: trac_auth=148d029eb321e9a0782fd30ca3f9d17b; Path=/desktopqe-backlog; secure
header: Connection: close
header: Content-Type: text/plain; charset=UTF-8
send: 'GET /desktopqe-backlog HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Fri, 06 Dec 2013 17:11:15 GMT
header: Server: Apache
header: Cache-Control: must-revalidate
header: Expires: Fri, 01 Jan 1999 00:00:00 GMT
header: Content-Length: 9591
header: Set-Cookie: trac_form_token=a6eda80ba3f4ead82093a0af; Path=/desktopqe-backlog; secure
header: Set-Cookie: trac_session=213441ab7934999739979a04; expires=Thu, 06-Mar-2014 17:11:15 GMT; Path=/desktopqe-backlog; secure
header: Connection: close
header: Content-Type: text/html;charset=utf-8
2013-12-06 18:11:17,190 DEBUG www-authenticate header not found
2013-12-06 18:11:17,190 CRITICAL mutual auth failed. No negotiate header
matej at wycliff: urllib2_kerberos (next *%)$ 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131206/5ec40775/attachment.sig>

From tjreedy at udel.edu  Sat Dec  7 00:07:42 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 06 Dec 2013 18:07:42 -0500
Subject: [Python-ideas] why is not request in httplib pretty printed?
In-Reply-To: <20131206220940.GA15554@wycliff.ceplovi.cz>
References: <20131206220940.GA15554@wycliff.ceplovi.cz>
Message-ID: <l7tlbm$6rv$1@ger.gmane.org>

On 12/6/2013 5:09 PM, Mat?j Cepl wrote:

> http://hg.python.org/cpython/file/543c76769c14/Lib/http/client.py#l847
> (but this code has been same since like forever) I see that the
> HTTP request is NOT pretty printed:
>
>       if self.debuglevel > 0:
>           print("send:", repr(data))
>
> whereas response in effect (because every header is printed
> separately) is.

Some thing have just not been done yet.


> Wouldn't it be better to pretty print the
> request as well?

> Otherwise I get quite unreadable debugging logs like the
> following (notice how much response is more readable than
> request).

> matej at wycliff: urllib2_kerberos (next *%)$ python test_trac.py
> send: 'GET /desktopqe-backlog/login HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'

I agree that the request above should be as nicely printed as the 
response below. If there is not already a tracker issue for this, I 
think you should open one.

> reply: 'HTTP/1.1 401 Authorization Required\r\n'
> header: Date: Fri, 06 Dec 2013 17:11:13 GMT
> header: Server: Apache

-- 
Terry Jan Reedy



From stephen at xemacs.org  Sat Dec  7 02:05:07 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 07 Dec 2013 10:05:07 +0900
Subject: [Python-ideas] Attribute docstrings [was: One-line abstractmethod
	function?]
In-Reply-To: <l7tnhj$v2u$1@ger.gmane.org>
References: <52A0E469.7090904@mrabarnett.plus.com>
 <CAP1=2W402=CZsj5bnWAJp9fpf8FJ3FMfHSaGY4sJ31FUOkXL1g@mail.gmail.com>
 <CAP7+vJLkSSMLNqo=5Bqo0Erf3=D-XzP3bemCLF4e669+aXiyWw@mail.gmail.com>
 <20131205211228.GB27767@avalon.amherst.edu>
 <CAP7+vJJQOjtFfSutYuhMdb+-KdSZ43+aXaK=WW3W=NjRTAS9HQ@mail.gmail.com>
 <20131206020629.GA15017@avalon.amherst.edu>
 <20131206020812.GB15017@avalon.amherst.edu>
 <52A136F4.2090609@stoneleaf.us>
 <CAP7+vJK4dgXWsw6=UnmbzDYXxvxoDc8zjAcT10EUKJuFB9VWaQ@mail.gmail.com>
 <CAAZsQLBfK+_WgZ-SBqQkA6mfg_CLS2BueGtG3oUkx=OxC1c0Mg@mail.gmail.com>
 <20131206104621.GN2085@ando> <l7tnhj$v2u$1@ger.gmane.org>
Message-ID: <87vbz131z0.fsf@uwakimon.sk.tsukuba.ac.jp>

Reply-To set to python-ideas at python.org.

Terry Reedy writes:

 > For data attributes, which are usually mutable, it should be attached to 
 > the attribute *concept*, which is represented by the name, rather than 
 > the current but usually changeable value. Values are usually already 
 > documented either by a value representation or a docstring. This could 
 > be done with a string subclass that is used as needed.
 > 
 > For methods, the value is nearly always constant. When multiple methods 
 > share the same function, they usually also share the same name, and 
 > represent the same concept.

Aside: Properties are which?  Data, or method?  It's also not clear to
me that "def first (self): return self.values[0]" is unlikely to be
used for completely different purposes than getting the head of a
list.  I conclude the considerations above are mostly red herrings.

The important thing, I suppose, is that the names of attributes
defined in a class are not mutable.  This means that their docstrings
can be kept in a completely separate dict (or other string -> string
mapping), which could even be stored in a separate file.  (Emacs Lisp
uses this to good effect.  The DOC file for XEmacs is 1.6MB; for GNU
Emacs it's 2.4MB.)  Of course it has its problems, but they're pretty
minor.



From mcepl at redhat.com  Sat Dec  7 12:04:21 2013
From: mcepl at redhat.com (=?utf-8?B?TWF0xJtq?= Cepl)
Date: Sat, 7 Dec 2013 12:04:21 +0100
Subject: [Python-ideas] why is not request in httplib pretty printed?
In-Reply-To: <l7tlbm$6rv$1@ger.gmane.org>
References: <20131206220940.GA15554@wycliff.ceplovi.cz>
 <l7tlbm$6rv$1@ger.gmane.org>
Message-ID: <20131207110414.GA23789@wycliff.ceplovi.cz>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> I agree that the request above should be as nicely printed as 
> the response below. If there is not already a tracker issue 
> for this, I think you should open one.

I haven't found anything, so I have filed 
http://bugs.python.org/issue19917

Mat?j

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iD8DBQFSovx64J/vJdlkhKwRAlXjAJ42kfsxMbP+TIzwBjqAQFqhEQP08wCeM8Aa
IzhfXXl+iYjyRKKk/mWTir0=
=DdfB
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131207/f96ff8c6/attachment.sig>

From mal at egenix.com  Sat Dec  7 13:21:06 2013
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 07 Dec 2013 13:21:06 +0100
Subject: [Python-ideas] Update the required C compiler for Windows to a
 supported version.
In-Reply-To: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>
References: <3dd00aa18f174c4e93a26f6e806b8439@BLUPR03MB293.namprd03.prod.outlook.com>
Message-ID: <52A312B2.8080200@egenix.com>

Hi Steve,

On 27.11.2013 19:24, Steve Dower wrote:
> Stephen J. Turnbull wrote:
>> Vernon D. Cole writes:
>>
>>> I cannot compile a Python extension module with any Microsoft compiler
>>> I can obtain.
>>
>> Your pain is understood, but it's not simple to address it. 
> 
> FWIW, I'm working on making the compiler easily obtainable. The VS 2008 link that was posted is unofficial, and could theoretically disappear at any time (I'm not in control of that), but the Windows SDK for Windows 7 and .NET 3.5 SP1 (http://www.microsoft.com/en-us/download/details.aspx?id=3138) should be around for as long as Windows 7 is supported. The correct compiler (VC9) is included in this SDK, but unfortunately does not install the vcvarsall.bat file that distutils expects. (Though it's pretty simple to add one that will switch on %1 and call the correct vcvars(86|64|...).bat.)
> 
> The SDK needed for Python 3.3 and 3.4 (VC10) is even worse - there are many files missing. I'm hoping we'll be able to set up some sort of downloadable package/tool that will fix this. While we'd obviously love to move CPython onto our latest compilers, it's simply not possible (for good reason). Python 3.4 is presumably locked to VC10, but hopefully 3.5 will be able to use whichever version is current when that decision is made.

I would expect that Python is not the only OSS tool that requires to
use the same compiler version for extensions as the one used to compile
the main application.

With Microsoft opening up towards OSS software, wouldn't it be better
to keep the express versions available officially for a much longer
period or allow 3rd parties to host the express ISOs ?

If there's anything the Python Software Foundation could do to help
with this, please let me know.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 07 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From g.rodola at gmail.com  Sat Dec  7 14:03:12 2013
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 7 Dec 2013 14:03:12 +0100
Subject: [Python-ideas] Easily reference a single unittest from cmdline
Message-ID: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>

During the development process it is very common to implement a new
functionality, write a test for it and then run tests.
I don't want to run *all* tests though, but only the new one which I just wrote.
Currently unittest module lets you do this via cmdline with:

python -m unittest test.test_module.TestClass.test_method

This is not very practical though as you have to figure out the name
of the module (easy) and the name of the test class (something which I
*never* remember).

Proposal is to allow a wildcard notation like this:

python -m unittest test.test_module.*test_method*
python -m unittest *test_method*

What I expect from unittest is to execute only the test methods
matching "*test_method*".

This is related to an old proposal I raised 3 years ago:
https://mail.python.org/pipermail/python-ideas/2010-August/007992.html
After that dicussion I ended up coming up with a decorator for skipping tests:
http://code.activestate.com/recipes/578234-unittestskip_others-decorator/?in=user-4178764
...but I never find it really practical and I ended up abandoning it.

In retrospective, I think the cmdline is the right place from where
such a thing should be controlled.

Thoughts?

--- Giampaolo
https://code.google.com/p/pyftpdlib/
https://code.google.com/p/psutil/
https://code.google.com/p/pysendfile/

From g.rodola at gmail.com  Sat Dec  7 14:27:58 2013
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 7 Dec 2013 14:27:58 +0100
Subject: [Python-ideas] Easily reference a single unittest from cmdline
In-Reply-To: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>
References: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>
Message-ID: <CAFYqXL8=LTYCuhO2sn6B7vrr1Cf25Hmo6So6nTfuqtbx-FD8Ag@mail.gmail.com>

I just took a look at nosetests.
It provides a similar thing and lets you use a regular expression
(which indeed makes a lot more sense than using wildcards)
It works like this:

nosetests test/test_module -m test_name


--- Giampaolo
https://code.google.com/p/pyftpdlib/
https://code.google.com/p/psutil/
https://code.google.com/p/pysendfile/

On Sat, Dec 7, 2013 at 2:03 PM, Giampaolo Rodola' <g.rodola at gmail.com> wrote:
> During the development process it is very common to implement a new
> functionality, write a test for it and then run tests.
> I don't want to run *all* tests though, but only the new one which I just wrote.
> Currently unittest module lets you do this via cmdline with:
>
> python -m unittest test.test_module.TestClass.test_method
>
> This is not very practical though as you have to figure out the name
> of the module (easy) and the name of the test class (something which I
> *never* remember).
>
> Proposal is to allow a wildcard notation like this:
>
> python -m unittest test.test_module.*test_method*
> python -m unittest *test_method*
>
> What I expect from unittest is to execute only the test methods
> matching "*test_method*".
>
> This is related to an old proposal I raised 3 years ago:
> https://mail.python.org/pipermail/python-ideas/2010-August/007992.html
> After that dicussion I ended up coming up with a decorator for skipping tests:
> http://code.activestate.com/recipes/578234-unittestskip_others-decorator/?in=user-4178764
> ...but I never find it really practical and I ended up abandoning it.
>
> In retrospective, I think the cmdline is the right place from where
> such a thing should be controlled.
>
> Thoughts?
>
> --- Giampaolo
> https://code.google.com/p/pyftpdlib/
> https://code.google.com/p/psutil/
> https://code.google.com/p/pysendfile/

From tjreedy at udel.edu  Sat Dec  7 23:03:05 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 07 Dec 2013 17:03:05 -0500
Subject: [Python-ideas] Easily reference a single unittest from cmdline
In-Reply-To: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>
References: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>
Message-ID: <l805ui$ejs$1@ger.gmane.org>

On 12/7/2013 8:03 AM, Giampaolo Rodola' wrote:
> During the development process it is very common to implement a new
> functionality, write a test for it

Or vice versa ;-)

> and then run tests.
> I don't want to run *all* tests though, but only the new one which I just wrote.

I presume for time reasons. The idlelib test modules I have written so 
far run under a second, so this is not an issue for me. For me, a test 
module would have to run several seconds, at least, to be worth 
switching to a console window instead of hitting F5.

> Currently unittest module lets you do this via cmdline with:
>
> python -m unittest test.test_module.TestClass.test_method
>
> This is not very practical though as you have to figure out the name
> of the module (easy) and the name of the test class (something which I
> *never* remember).

I mostly use ClassnameTest or FunctionnameTest.

> Proposal is to allow a wildcard notation like this:
>
> python -m unittest test.test_module.*test_method*

I think this should better be
python -m unittest "test.test_module.*.test_method"

> python -m unittest *test_method*

I think you left something out here.

> What I expect from unittest is to execute only the test methods
> matching "*test_method*".

Inspired by you second post, how about

python -m unittest test.test_module -t test_method1 test_method2 ...

which gives more flexibility. Of course, test_x will match multiple 
tests if test_x appears in multiple testcase classes. I believe there 
are other proposals for more flexibility in picking which tests to run. 
There may be a tracker issue already.

-- 
Terry Jan Reedy


From amber.yust at gmail.com  Sun Dec  8 05:44:36 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Sun, 08 Dec 2013 04:44:36 +0000
Subject: [Python-ideas] Batching/grouping function for itertools
Message-ID: <5251239173677340489@gmail297201516>

After seeing yet another person asking how to do this on #python (and
having needed to do it in the past myself), I'm wondering why itertools
doesn't have a function to break an iterator up into N-sized chunks.

Existing possible solutions include both the "clever" but somewhat
unreadable...

    batched_iter = zip(*[iter(input_iter)]*n)

...and the long-form...

    def batch(input_iter, n):
        input_iter = iter(input_iter)
        while True:
            yield [input_iter.next() for _ in range(n)]

There doesn't seem, however, to be one clear "right" way to do this. Every
time I come up against this task, I go back to itertools expecting one of
the grouping functions there to cover it, but they don't.

It seems like it would be a natural fit for itertools, and it would
simplify things like processing of file formats that use a consistent
number of lines per entry, et cetera.

~Amber
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131208/6eb6b1e2/attachment-0001.html>

From jeanpierreda at gmail.com  Sun Dec  8 06:14:18 2013
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sat, 7 Dec 2013 21:14:18 -0800
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <5251239173677340489@gmail297201516>
References: <5251239173677340489@gmail297201516>
Message-ID: <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>

On Sat, Dec 7, 2013 at 8:44 PM, Amber Yust <amber.yust at gmail.com> wrote:
> After seeing yet another person asking how to do this on #python (and having
> needed to do it in the past myself), I'm wondering why itertools doesn't
> have a function to break an iterator up into N-sized chunks.

+1. In my experience the grouper recipe in the docs serve less as a
helpful example of how to use itertools and more as a thing to copy
paste. That's what modules are for.


-- Devin

From ncoghlan at gmail.com  Sun Dec  8 08:02:05 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 8 Dec 2013 17:02:05 +1000
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
Message-ID: <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>

On 8 December 2013 15:14, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> On Sat, Dec 7, 2013 at 8:44 PM, Amber Yust <amber.yust at gmail.com> wrote:
>> After seeing yet another person asking how to do this on #python (and having
>> needed to do it in the past myself), I'm wondering why itertools doesn't
>> have a function to break an iterator up into N-sized chunks.
>
> +1. In my experience the grouper recipe in the docs serve less as a
> helpful example of how to use itertools and more as a thing to copy
> paste. That's what modules are for.

The windowing problem is too ill-defined - there are enough degrees of
freedom that any API flexible enough to cover them all is harder to
learn than just building out your own version that works the way you
want it to, and a more restrictive API that *doesn't* cover all the
variants introduces a sharp discontinuity between the "blessed"
variant and the alternatives.

For anyone that thinks the stdlib itertools is too minimalist (I'm not
one of them), than "pip install more-itertools" provides the recipes
from the stdlib docs, as well as a few other precomposed operations.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From taleinat at gmail.com  Sun Dec  8 09:14:42 2013
From: taleinat at gmail.com (Tal Einat)
Date: Sun, 8 Dec 2013 10:14:42 +0200
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
Message-ID: <CALWZvp4Pk19ZY0ZZ3p2Ua9uvjunTcWvLmY0QN1xwZCd9u6+ErQ@mail.gmail.com>

Hello Amber,

These issues -- a batching function in itertools and including the
itertools recipes in the stdlib -- have both been discussed here recently.

Specifically regarding the batching function, I couldn't find the most
recent discussion via a quick search. IIRC the conclusion was what Nick
said: different use-cases require slightly different behaviors, which can
not be elegantly expressed as a single, simple and straight-forward
function. Therefore, it is better to have a basic recipe in the docs, which
everyone can modify according to their needs.

With regard to including the other recipes in the stdlib, I recommend
reading the most recent discussion on the archives [1]. The major argument
against this is that these recipes are easily implemented based on the
existing tools, but having all of them in the stdlib means having to
support them all in the future, including maintaining backwards
compatibility. Supporting stdlib code is considerably harder than having
working examples in the docs.

- Tal

[1] https://mail.python.org/pipermail/python-ideas/2012-July/015714.html


On Sun, Dec 8, 2013 at 9:02 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 8 December 2013 15:14, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> > On Sat, Dec 7, 2013 at 8:44 PM, Amber Yust <amber.yust at gmail.com> wrote:
> >> After seeing yet another person asking how to do this on #python (and
> having
> >> needed to do it in the past myself), I'm wondering why itertools doesn't
> >> have a function to break an iterator up into N-sized chunks.
> >
> > +1. In my experience the grouper recipe in the docs serve less as a
> > helpful example of how to use itertools and more as a thing to copy
> > paste. That's what modules are for.
>
> The windowing problem is too ill-defined - there are enough degrees of
> freedom that any API flexible enough to cover them all is harder to
> learn than just building out your own version that works the way you
> want it to, and a more restrictive API that *doesn't* cover all the
> variants introduces a sharp discontinuity between the "blessed"
> variant and the alternatives.
>
> For anyone that thinks the stdlib itertools is too minimalist (I'm not
> one of them), than "pip install more-itertools" provides the recipes
> from the stdlib docs, as well as a few other precomposed operations.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131208/841442ce/attachment.html>

From steve at pearwood.info  Sun Dec  8 10:25:17 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 8 Dec 2013 20:25:17 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
Message-ID: <20131208092517.GU2085@ando>

On Sun, Dec 08, 2013 at 05:02:05PM +1000, Nick Coghlan wrote:

> The windowing problem is too ill-defined - there are enough degrees of
> freedom that any API flexible enough to cover them all is harder to
> learn than just building out your own version that works the way you
> want it to, and a more restrictive API that *doesn't* cover all the
> variants introduces a sharp discontinuity between the "blessed"
> variant and the alternatives.

Playing Devil's Advocate here, I wonder if that is true though. It seems 
to me that there are two basic windowing variants: sliding windows, and 
discrete windows. That is, given a sequence [a, b, c, d, e, f, g, h, i] 
and a window size of 3, the two obvious, common results are:

# sliding window
(a,b,c), (b,c,d), (c,d,e), (d,e,f), (e,f,g), (f,g,h), (g,h,i)

# discrete windows
(a,b,c), (d,e,f), (g,h,i)


Surely anything else is exotic enough that there is no question about 
leaving it up to the individual programmer.

In the second case, there is a question about what to do with sequences 
that are not a multiple of the window size. Similar to zip(), there are 
two things one might do:

- pad with some given object;
- raise an exception

If you want to just ignore extra items, just catch the exception and 
continue. So that's a maximum of three window functions:

sliding(iterable, window_size)
discrete(iterable, window_size, pad=None)
strict_discrete(iterable, window_size)

or just two, if you combine discrete and strict_discrete:

discrete(iterable, window_size [, pad])
# raise if pad is not given


What other varieties are there? Surely none that are common. Once, for a 
lark, I tried to come up with one that was fully general -- as well as a 
window size, you could specify how far to advance the window each step. 
The sliding variety would advance by 1 each step, the discrete variety 
would advance by the window size. But I never found any reason to use it 
with any other step sizes. Surely anything else is more useful in theory 
than in practice.

(That's three times I've said something is "surely" true, always a sign 
my argument is weak *grin*)

Given that this windowing problem keeps coming up, there's no doubt in 
my mind that it is a useful, if not fundamental, iterator operation. 
Ruby's Enumerable module includes both:

http://ruby-doc.org/core-2.0.0/Enumerable.html

each_cons is what I've been calling a sliding window, and each_slice is 
what I've been calling discrete chunks.



-- 
Steven

From ncoghlan at gmail.com  Sun Dec  8 12:05:18 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 8 Dec 2013 21:05:18 +1000
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <20131208092517.GU2085@ando>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
Message-ID: <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>

On 8 December 2013 19:25, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, Dec 08, 2013 at 05:02:05PM +1000, Nick Coghlan wrote:
>
>> The windowing problem is too ill-defined - there are enough degrees of
>> freedom that any API flexible enough to cover them all is harder to
>> learn than just building out your own version that works the way you
>> want it to, and a more restrictive API that *doesn't* cover all the
>> variants introduces a sharp discontinuity between the "blessed"
>> variant and the alternatives.
>
> Playing Devil's Advocate here, I wonder if that is true though. It seems
> to me that there are two basic windowing variants: sliding windows, and
> discrete windows. That is, given a sequence [a, b, c, d, e, f, g, h, i]
> and a window size of 3, the two obvious, common results are:
>
> # sliding window
> (a,b,c), (b,c,d), (c,d,e), (d,e,f), (e,f,g), (f,g,h), (g,h,i)
>
> # discrete windows
> (a,b,c), (d,e,f), (g,h,i)
>
>
> Surely anything else is exotic enough that there is no question about
> leaving it up to the individual programmer.
>
> In the second case, there is a question about what to do with sequences
> that are not a multiple of the window size. Similar to zip(), there are
> two things one might do:
>
> - pad with some given object;
> - raise an exception
>
> If you want to just ignore extra items, just catch the exception and
> continue. So that's a maximum of three window functions:
>
> sliding(iterable, window_size)
> discrete(iterable, window_size, pad=None)
> strict_discrete(iterable, window_size)
>
> or just two, if you combine discrete and strict_discrete:
>
> discrete(iterable, window_size [, pad])
> # raise if pad is not given
>
> What other varieties are there? Surely none that are common. Once, for a
> lark, I tried to come up with one that was fully general -- as well as a
> window size, you could specify how far to advance the window each step.
> The sliding variety would advance by 1 each step, the discrete variety
> would advance by the window size. But I never found any reason to use it
> with any other step sizes. Surely anything else is more useful in theory
> than in practice.
>
> (That's three times I've said something is "surely" true, always a sign
> my argument is weak *grin*)

I'm biased by a signal processing background where playing games with
data windows and the amount of overlap between samples is a *really*
common technique :)

> Given that this windowing problem keeps coming up, there's no doubt in
> my mind that it is a useful, if not fundamental, iterator operation.
> Ruby's Enumerable module includes both:
>
> http://ruby-doc.org/core-2.0.0/Enumerable.html
>
> each_cons is what I've been calling a sliding window, and each_slice is
> what I've been calling discrete chunks.

The two examples in the itertools docs are currently just pairwise
(sliding window of length 2) and grouper (distinct windows of
arbitrary length, always padded)

The general cases would be:

    def sliding_window(iterable, n):
        """Return a sliding window over the data, introducing one new
item into each window"""
        iterables = tee(iterable, n)
        # Prime the iterables
        for i, itr in iterables:
            for __ in range(i):
                next(itr, None)
        return zip(*iterables)

    def discrete_window(iterable, n, fillvalue=None):
        """Return distinct windows of the data, padding the last
window if necessary"""
        repeated_iterable = [iter(iterable)] * n
        return zip_longest(*repeated_iterable, fillvalue=fillvalue)

Given the padding version of discrete_window, the exception raising
version is just:

    def discrete_window_no_padding(iterable, n):
        sentinel = object()
        for x in discrete_window(iterable, n, sentinel):
            if x[-1] is sentinel: raise ValueError("Ragged final partition")
            yield x

Given the "n-1" overlapping version of sliding window, the "selective
overlap" version (ignoring any leftover data at the end) is just:

    def sliding_window_with_configurable_overlap(iterable, n, new_items=1):
        if new_items == 1:
            return sliding_window(iterable, n)
        return islice(sliding_window(iterable, n), 0, None, new_items)

The main argument in favour of offering sliding_window and
discrete_window as itertools is that they each rely on a sophisticated
trick in iterator state manipulation:

- the sliding window relies on using tee() and then priming the
results to start at the appropriate place in the initial window
- the discrete window relies on using *multiple* reference to a single
iterator and exploiting the fact that the iterator advances each time
a value is retrieved

That's a deeper understanding of the object model than most people
will have, so they're likely to just cargo cult the recipe from the
docs anyway, without really trying to understand exactly how it works.

I guess I'm +0 rather than -1 at this point, but it's really Raymond
that needs to be convinced as module maintainer (other points of note:
such a change wouldn't be possible until Python 3.5 anyway, and it
would also require restructuring itertools to be a hybrid C/Python
module, since writing these in C wouldn't offer any significant
benefits - the inner loops in the pure Python versions are already
using existing high speed iterators).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From storchaka at gmail.com  Sun Dec  8 12:30:56 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 08 Dec 2013 13:30:56 +0200
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <20131208092517.GU2085@ando>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
Message-ID: <l81l8j$kgo$1@ger.gmane.org>

08.12.13 11:25, Steven D'Aprano ???????(??):
> In the second case, there is a question about what to do with sequences
> that are not a multiple of the window size. Similar to zip(), there are
> two things one might do:
>
> - pad with some given object;
> - raise an exception

3) emit last chunk incompleted;
4) skip incomplete chunk.

There is also a question about result's type. Sometimes you need an 
iterator of subsequences (i.e. split string on equal string chunks), 
sometimes an iterator of iterators is enough.

I.e. at least 8 variants are needed.


From steve at pearwood.info  Sun Dec  8 13:16:29 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 8 Dec 2013 23:16:29 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <l81l8j$kgo$1@ger.gmane.org>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando> <l81l8j$kgo$1@ger.gmane.org>
Message-ID: <20131208121629.GV2085@ando>

On Sun, Dec 08, 2013 at 01:30:56PM +0200, Serhiy Storchaka wrote:
> 08.12.13 11:25, Steven D'Aprano ???????(??):
> >In the second case, there is a question about what to do with sequences
> >that are not a multiple of the window size. Similar to zip(), there are
> >two things one might do:
> >
> >- pad with some given object;
> >- raise an exception
> 
> 3) emit last chunk incompleted;

Given a window size of two, and input data [a, b, c], are you suggesting 
a variety that returns this?

(a,b), (c,)

There is no need for a separate function for that. Given a version that 
takes a pad object, if the pad argument is not given, return a partial 
chunk at the end.


> 4) skip incomplete chunk.

The very next sentence in my post references that:

"If you want to just ignore extra items, just catch the exception and
continue."

There is no need for an extra function covering that case.


> There is also a question about result's type. Sometimes you need an 
> iterator of subsequences (i.e. split string on equal string chunks), 
> sometimes an iterator of iterators is enough.

None of the other itertools functions treat strings specially. Why 
should this one? If you want to re-join them into strings, you can do so 
with a trivial wrapper:

(''.join(elements) for elements in group("some string", 3, pad=' '))

ought to do the trick, assuming group returns tuples or lists of 
characters.

Re-combining the iterated-upon elements into the input type is not the 
responsibility of itertools.



-- 
Steven

From barry at python.org  Sun Dec  8 14:13:10 2013
From: barry at python.org (Barry Warsaw)
Date: Sun, 8 Dec 2013 08:13:10 -0500
Subject: [Python-ideas] Easily reference a single unittest from cmdline
References: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>
 <CAFYqXL8=LTYCuhO2sn6B7vrr1Cf25Hmo6So6nTfuqtbx-FD8Ag@mail.gmail.com>
Message-ID: <20131208081310.22aa55a3@anarchist.wooz.org>

On Dec 07, 2013, at 02:27 PM, Giampaolo Rodola' wrote:

>I just took a look at nosetests.
>It provides a similar thing and lets you use a regular expression
>(which indeed makes a lot more sense than using wildcards)
>It works like this:
>
>nosetests test/test_module -m test_name

I use nose2 in several projects and have a nice little plugin that reproduces
the most useful (for me) bits of zope testrunner for specifying test
patterns.  I can usually just do something like

$ nose2 -P test_this_one_thing
$ nose2 -P TestThisWholeClass
$ nose2 -P test_module
$ nose2 -P test_things.rst

http://tinyurl.com/l2trn4a

-Barry

P.S. Yes, nose2 *rocks* :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131208/8e850a6b/attachment.sig>

From ron3200 at gmail.com  Sun Dec  8 16:13:06 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 08 Dec 2013 09:13:06 -0600
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
Message-ID: <l8229o$i20$1@ger.gmane.org>



On 12/08/2013 05:05 AM, Nick Coghlan wrote:
> The main argument in favour of offering sliding_window and
> discrete_window as itertools is that they each rely on a sophisticated
> trick in iterator state manipulation:

How about 2 lower level building blocks that would make these things easier 
to make and think about.


Possibly function to take the next n items of an iterator without advancing it.

Along with a function to advance an iterator n ahead without taking anything.


These would be simpler and easier to maintain, and have a wider range of uses.

Cheers,
    Ron






From stephen at xemacs.org  Sun Dec  8 16:21:35 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 09 Dec 2013 00:21:35 +0900
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <20131208092517.GU2085@ando>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
Message-ID: <87ob4r2wsg.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > What other varieties are there? Surely none that are common. Once, for a 
 > lark, I tried to come up with one that was fully general -- as well as a 
 > window size, you could specify how far to advance the window each step. 
 > The sliding variety would advance by 1 each step, the discrete variety 
 > would advance by the window size. But I never found any reason to use it 
 > with any other step sizes. Surely anything else is more useful in theory 
 > than in practice.

Deseasonalization of serially correlated data where the seasonality is
lower-frequency than the series, and more generally data-mining
techniques that start with relatively coarse steps and refine as they
go along are two that come immediately to mind.


From rosuav at gmail.com  Sun Dec  8 16:22:49 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 9 Dec 2013 02:22:49 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <l8229o$i20$1@ger.gmane.org>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
Message-ID: <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>

On Mon, Dec 9, 2013 at 2:13 AM, Ron Adam <ron3200 at gmail.com> wrote:
> How about 2 lower level building blocks that would make these things easier
> to make and think about.
>
>
> Possibly function to take the next n items of an iterator without advancing
> it.

Fundamentally impossible. Here's a function that returns an iterator:

def d20():
    import random
    while True:
        yield random.randrange(1,21)

dice_roller = d20()

How are you going to take the next n items from dice_roller without
advancing it?

ChrisA

From stephen at xemacs.org  Sun Dec  8 16:34:28 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 09 Dec 2013 00:34:28 +0900
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
Message-ID: <87iouz2w6z.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > How are you going to take the next n items from dice_roller without
 > advancing it?

Memoize.

From rosuav at gmail.com  Sun Dec  8 16:40:30 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 9 Dec 2013 02:40:30 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <87iouz2w6z.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
 <87iouz2w6z.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmp=Z4R15ZT4JhPA2F3nLs_UsUEva9FWpBwqJ4Bb8g_UEg@mail.gmail.com>

On Mon, Dec 9, 2013 at 2:34 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Chris Angelico writes:
>
>  > How are you going to take the next n items from dice_roller without
>  > advancing it?
>
> Memoize.

That's not a building-block then, that's a quite separate feature. In
this particular instance there's no way to distinguish between
"predict the next three but don't advance the iterator" and "advance
the iterator by three and then rewind it the same distance", but
imagine an iterators that blocks for input, or something. You don't
want a purportedly low-level function (from which you derive the more
"usable" functions) doing memoization on that.

ChrisA

From ron3200 at gmail.com  Sun Dec  8 16:40:27 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 08 Dec 2013 09:40:27 -0600
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
Message-ID: <l823t1$2qi$1@ger.gmane.org>



On 12/08/2013 09:22 AM, Chris Angelico wrote:
>> >How about 2 lower level building blocks that would make these things easier
>> >to make and think about.
>> >
>> >
>> >Possibly function to take the next n items of an iterator without advancing
>> >it.
> Fundamentally impossible.

In some cases yes, but you would know if it could work before you chose to 
use these.

 > Here's a function that returns an iterator:
>
> def d20():
>      import random
>      while True:
>          yield random.randrange(1,21)
>
> dice_roller = d20()
>
> How are you going to take the next n items from dice_roller without
> advancing it?

If it is to work with generators too...

The function would need to be a wrapper that keeps a buffer.  And the front 
of the buffer would always be the next to be yielded if there is anything 
in it.

In the case of advancing an generator, without taking the values, it would 
still need to call the __next__ methods on it, but not actually return the 
values.

Yes, it won't be as simple as it sounds. ;-)

cheers,
    Ron












From rosuav at gmail.com  Sun Dec  8 16:46:53 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 9 Dec 2013 02:46:53 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <l823t1$2qi$1@ger.gmane.org>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
 <l823t1$2qi$1@ger.gmane.org>
Message-ID: <CAPTjJmrx3Ryo0Q1me=cggE1L=mpNyNQmrrzF9u2pggQBo1srZg@mail.gmail.com>

On Mon, Dec 9, 2013 at 2:40 AM, Ron Adam <ron3200 at gmail.com> wrote:
> The function would need to be a wrapper that keeps a buffer.  And the front
> of the buffer would always be the next to be yielded if there is anything in
> it.

Which is what Stephen said in his pithy message above. Yes, it's
theoretically possible, but it's not something to do in the general
case. Not something to depend on for a chunker/grouper, which should
be able to pass once over the underlying iterator.

ChrisA

From ron3200 at gmail.com  Sun Dec  8 17:07:07 2013
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 08 Dec 2013 10:07:07 -0600
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CAPTjJmrx3Ryo0Q1me=cggE1L=mpNyNQmrrzF9u2pggQBo1srZg@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
 <l823t1$2qi$1@ger.gmane.org>
 <CAPTjJmrx3Ryo0Q1me=cggE1L=mpNyNQmrrzF9u2pggQBo1srZg@mail.gmail.com>
Message-ID: <l825f1$i62$1@ger.gmane.org>



On 12/08/2013 09:46 AM, Chris Angelico wrote:
> On Mon, Dec 9, 2013 at 2:40 AM, Ron Adam<ron3200 at gmail.com>  wrote:
>> >The function would need to be a wrapper that keeps a buffer.  And the front
>> >of the buffer would always be the next to be yielded if there is anything in
>> >it.
> Which is what Stephen said in his pithy message above. Yes, it's
> theoretically possible, but it's not something to do in the general
> case. Not something to depend on for a chunker/grouper, which should
> be able to pass once over the underlying iterator.

For the windowed chunker that was being discussed and uses that are 
similar, they need to hold references to some yielded items some place. 
This just packages that need in a convenient way.


It almost seems to me that coroutines, which is the counter example you 
suggested, should maybe be a type of their own.  That way, they can be 
tested.  (And raise an error if used in cases like this.)

But that's whole other topic.  Best answer for now is don't do that.

Cheers,
    Ron


From steve at pearwood.info  Sun Dec  8 17:38:55 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 9 Dec 2013 03:38:55 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <l8229o$i20$1@ger.gmane.org>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
Message-ID: <20131208163855.GW15615@ando>

On Sun, Dec 08, 2013 at 09:13:06AM -0600, Ron Adam wrote:

> Possibly function to take the next n items of an iterator without advancing 
> it.

Fundamentally impossible. The best you can do it advance the iterator 
but store the results for later use.


> Along with a function to advance an iterator n ahead without taking 
> anything.

Too trivial to bother with. Just advance the iterator and throw the 
result away.

def advance(it, n):
    for _ in range(n):
        next(it)

You can even do it as a one-linear, at the expense of readability:

{next(it) and None for _ in range(n)}.pop()

(The pop isn't really necessary, I just like the fact that it means the 
expression evaluates as None.)


> These would be simpler and easier to maintain, and have a wider range of 
> uses.

Maybe, maybe not, but they don't solve the problem that people keep 
asking to be solved.



-- 
Steven

From steve at pearwood.info  Sun Dec  8 17:45:19 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 9 Dec 2013 03:45:19 +1100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <87iouz2w6z.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
 <87iouz2w6z.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20131208164519.GX2085@ando>

On Mon, Dec 09, 2013 at 12:34:28AM +0900, Stephen J. Turnbull wrote:
> Chris Angelico writes:
> 
>  > How are you going to take the next n items from dice_roller without
>  > advancing it?
> 
> Memoize.

Er, I don't think so. How does the memoizing cache get those values if 
the underlying iterator isn't advanced? Obviously it can't. 
itertools.tee uses a cache, so we can demonstrate the issue:

py> it = iter("abcde")
py> wrapper = itertools.tee(it, 2)[0]
py> _ = list(wrapper)

If the iterator hasn't advanced, then next(it) should yield 'a'. But:

py> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration


Any sort of "iterator look-ahead" has a number of fundamental problems. 
Despite many requests, those problems are part of the reason why Python 
iterators don't provide a "peek" method to look ahead. Not even to look 
ahead a single value, let alone an arbitrary number of values.

- The cache would require unbounded memory (unless you limit 
  the look-ahead to N values);

- iterators with side-effects would cause those side-effects 
  at the wrong time;

- iterators whose calculated values are time-dependent could 
  be calculated at a different time from when they are returned,
  potentially giving the wrong result.


For something like tee, it is difficult to see any other way other than 
memoisation to get the functionality needed, so we just have to live 
with the limitations. But offering dedicated look-ahead with caching as 
fundamental iterator tools, as Ron suggests, strikes me as completely 
the wrong thing to do if what we actually want is to group items. It 
doesn't solve the problem being asked, since it's still up to the caller 
to make their own grouper tool out of the memoising primitive.


-- 
Steven

From storchaka at gmail.com  Sun Dec  8 18:32:40 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 08 Dec 2013 19:32:40 +0200
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <20131208121629.GV2085@ando>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando> <l81l8j$kgo$1@ger.gmane.org>
 <20131208121629.GV2085@ando>
Message-ID: <l82aep$5qo$1@ger.gmane.org>

08.12.13 14:16, Steven D'Aprano ???????(??):
> On Sun, Dec 08, 2013 at 01:30:56PM +0200, Serhiy Storchaka wrote:
>> 08.12.13 11:25, Steven D'Aprano ???????(??):
>>> In the second case, there is a question about what to do with sequences
>>> that are not a multiple of the window size. Similar to zip(), there are
>>> two things one might do:
>>>
>>> - pad with some given object;
>>> - raise an exception
>>
>> 3) emit last chunk incompleted;
>
> Given a window size of two, and input data [a, b, c], are you suggesting
> a variety that returns this?
>
> (a,b), (c,)
>
> There is no need for a separate function for that. Given a version that
> takes a pad object, if the pad argument is not given, return a partial
> chunk at the end.

You had proposed raise an exception when the pad argument is not given 
in previous message. In any case these are three different cases, and 
you can combine only two of them in one function using "absent argument" 
trick.

>> 4) skip incomplete chunk.
>
> The very next sentence in my post references that:
>
> "If you want to just ignore extra items, just catch the exception and
> continue."
>
> There is no need for an extra function covering that case.

You can't just use this generator in expression (e.g. as an argument to 
list). You need special wrapper which catches en exception. This is 
fourth variant. And if you need just this variant, why it is not in the 
stdlib?

>> There is also a question about result's type. Sometimes you need an
>> iterator of subsequences (i.e. split string on equal string chunks),
>> sometimes an iterator of iterators is enough.
>
> None of the other itertools functions treat strings specially. Why
> should this one?

Because I relatively often need this idiom and almost never need general 
function for iterators. I'm sure a function which splits sequences are 
enough in at least 90% cases when you need grouping function.

> If you want to re-join them into strings, you can do so
> with a trivial wrapper:
>
> (''.join(elements) for elements in group("some string", 3, pad=' '))
>
> ought to do the trick, assuming group returns tuples or lists of
> characters.

This is too slow and verbose and kill benefits of grouping function.


From grosser.meister.morti at gmx.net  Sun Dec  8 18:49:19 2013
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Sun, 08 Dec 2013 18:49:19 +0100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <5251239173677340489@gmail297201516>
References: <5251239173677340489@gmail297201516>
Message-ID: <52A4B11F.7040200@gmx.net>

On 12/08/2013 05:44 AM, Amber Yust wrote:
> After seeing yet another person asking how to do this on #python (and having needed to do it in the past myself), I'm
> wondering why itertools doesn't have a function to break an iterator up into N-sized chunks.
>
> Existing possible solutions include both the "clever" but somewhat unreadable...
>
>      batched_iter = zip(*[iter(input_iter)]*n)
>
> ...and the long-form...
>
>      def batch(input_iter, n):
>          input_iter = iter(input_iter)
>          while True:
>              yield [input_iter.next() for _ in range(n)]
>

This function drops items if the length of the input sequence is not a multiple of n. Fix:

	def batch(it, n):
		it = iter(it)
		while True:
			slice = []
			for _ in range(n):
				try:
					slice.append(it.next())
				except StopIteration:
					if slice:
						yield slice
					return
			yield slice

> There doesn't seem, however, to be one clear "right" way to do this. Every time I come up against this task, I go back
> to itertools expecting one of the grouping functions there to cover it, but they don't.
>
> It seems like it would be a natural fit for itertools, and it would simplify things like processing of file formats that
> use a consistent number of lines per entry, et cetera.
>
> ~Amber
>


From amber.yust at gmail.com  Sun Dec  8 19:06:05 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Sun, 08 Dec 2013 18:06:05 +0000
Subject: [Python-ideas] Batching/grouping function for itertools
References: <5251239173677340489@gmail297201516> <52A4B11F.7040200@gmx.net>
Message-ID: <-8796784632715378232@gmail297201516>

So does zip if the items are of unequal length, and the two code examples I
provided (the one using zip and the long-form one) are equivalent.

On Sun Dec 08 2013 at 9:49:56 AM, Mathias Panzenb?ck <
grosser.meister.morti at gmx.net> wrote:

> On 12/08/2013 05:44 AM, Amber Yust wrote:
> > After seeing yet another person asking how to do this on #python (and
> having needed to do it in the past myself), I'm
> > wondering why itertools doesn't have a function to break an iterator up
> into N-sized chunks.
> >
> > Existing possible solutions include both the "clever" but somewhat
> unreadable...
> >
> >      batched_iter = zip(*[iter(input_iter)]*n)
> >
> > ...and the long-form...
> >
> >      def batch(input_iter, n):
> >          input_iter = iter(input_iter)
> >          while True:
> >              yield [input_iter.next() for _ in range(n)]
> >
>
> This function drops items if the length of the input sequence is not a
> multiple of n. Fix:
>
>         def batch(it, n):
>                 it = iter(it)
>                 while True:
>                         slice = []
>                         for _ in range(n):
>                                 try:
>                                         slice.append(it.next())
>                                 except StopIteration:
>                                         if slice:
>                                                 yield slice
>                                         return
>                         yield slice
>
> > There doesn't seem, however, to be one clear "right" way to do this.
> Every time I come up against this task, I go back
> > to itertools expecting one of the grouping functions there to cover it,
> but they don't.
> >
> > It seems like it would be a natural fit for itertools, and it would
> simplify things like processing of file formats that
> > use a consistent number of lines per entry, et cetera.
> >
> > ~Amber
> >
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131208/08223518/attachment.html>

From breamoreboy at yahoo.co.uk  Sun Dec  8 19:24:28 2013
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 08 Dec 2013 18:24:28 +0000
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <5251239173677340489@gmail297201516>
References: <5251239173677340489@gmail297201516>
Message-ID: <l82dgp$613$1@ger.gmane.org>

On 08/12/2013 04:44, Amber Yust wrote:
> After seeing yet another person asking how to do this on #python (and
> having needed to do it in the past myself), I'm wondering why itertools
> doesn't have a function to break an iterator up into N-sized chunks.
>

As discussed umpteen times previously, there is no way that we can agree 
on "a function" that can meet all of the variations that have been 
proposed on this theme.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence


From yoavglazner at gmail.com  Sun Dec  8 19:57:32 2013
From: yoavglazner at gmail.com (yoav glazner)
Date: Sun, 8 Dec 2013 20:57:32 +0200
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <l82dgp$613$1@ger.gmane.org>
References: <5251239173677340489@gmail297201516> <l82dgp$613$1@ger.gmane.org>
Message-ID: <CAJ78kjO5H81xwWwheB3JFVqaiB6VdQs8zciLTeBA6NmoTaa20A@mail.gmail.com>

Hi

On Sun, Dec 8, 2013 at 8:24 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> On 08/12/2013 04:44, Amber Yust wrote:
>
>> After seeing yet another person asking how to do this on #python (and
>> having needed to do it in the past myself), I'm wondering why itertools
>> doesn't have a function to break an iterator up into N-sized chunks.
>>
>>
> As discussed umpteen times previously, there is no way that we can agree
> on "a function" that can meet all of the variations that have been proposed
> on this theme.
>
>
Maybe if we add this function:
>>> def mod_pad(it, modulo, fillval):
'"".join(mod_pad("hello", 3, fillval="!")) => hello!'
for i, val in enumerate(iter(it)):
yield val
for _ in range(i%modulo):
yield fillval

>>> "".join(mod_pad("hello", 3, fillval="!"))
'hello!'

Then we can make grouper/batcher throw an exception in the case of iter_len
% modulo != 0

grouper(mod_pad("hello", 3, fillval="!"), 3) => hel, lo! (in a iterator...)
grouper('hello', 3) => BOOM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131208/1164efbe/attachment.html>

From ncoghlan at gmail.com  Sun Dec  8 22:56:45 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 9 Dec 2013 07:56:45 +1000
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <CAPTjJmrx3Ryo0Q1me=cggE1L=mpNyNQmrrzF9u2pggQBo1srZg@mail.gmail.com>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando>
 <CADiSq7c6jLbduJrLSjGv=K=+AW=rBpJxzAaOZZbVjbVkd5okaw@mail.gmail.com>
 <l8229o$i20$1@ger.gmane.org>
 <CAPTjJmpi2eRfmYu7-Ab4L=8-ko10kO5UB_bJTCernbwZZ0nTVA@mail.gmail.com>
 <l823t1$2qi$1@ger.gmane.org>
 <CAPTjJmrx3Ryo0Q1me=cggE1L=mpNyNQmrrzF9u2pggQBo1srZg@mail.gmail.com>
Message-ID: <CADiSq7c2=DSxiSFkAQaEgEuLx+cERZans-RJgmMQfbT0=N0FSg@mail.gmail.com>

On 9 Dec 2013 01:47, "Chris Angelico" <rosuav at gmail.com> wrote:
>
> On Mon, Dec 9, 2013 at 2:40 AM, Ron Adam <ron3200 at gmail.com> wrote:
> > The function would need to be a wrapper that keeps a buffer.  And the
front
> > of the buffer would always be the next to be yielded if there is
anything in
> > it.
>
> Which is what Stephen said in his pithy message above. Yes, it's
> theoretically possible, but it's not something to do in the general
> case. Not something to depend on for a chunker/grouper, which should
> be able to pass once over the underlying iterator.

tee() is this building block.

Cheers,
Nick.

>
> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131209/a271e17d/attachment.html>

From tjreedy at udel.edu  Sun Dec  8 23:09:14 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 08 Dec 2013 17:09:14 -0500
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <20131208121629.GV2085@ando>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando> <l81l8j$kgo$1@ger.gmane.org>
 <20131208121629.GV2085@ando>
Message-ID: <l82qlv$bl6$1@ger.gmane.org>

On 12/8/2013 7:16 AM, Steven D'Aprano wrote:
> On Sun, Dec 08, 2013 at 01:30:56PM +0200, Serhiy Storchaka wrote:

>> There is also a question about result's type. Sometimes you need an
>> iterator of subsequences (i.e. split string on equal string chunks),
>> sometimes an iterator of iterators is enough.
>
> None of the other itertools functions treat strings specially. Why
> should this one? If you want to re-join them into strings, you can do so
> with a trivial wrapper:
>
> (''.join(elements) for elements in group("some string", 3, pad=' '))

A large fraction, perhaps over half, of the multiple requests for a 
chunker or grouper function are for sequences, not general iterables, as 
input, with the desired output type being the input type. For this, an 
iterator of *slices* is *far* more efficient. The same function could 
easily handle overlaps. (There are still the possible varieties of short 
slice handling). *Untested*:

def window(seq, size, advance=None, extra='skip'):
   '''Yield successive slices of len size of sequence seq.

   Move window advance items (default = size).
   Extra determines the handling of extra items.
   The options are 'skip' (default), 'keep', and 'raise'.
   '''
   if overlap == None: advance = size
   i, j, n = 0, size, len(seq)
   while j <= n:
     yield seq[i:j]
     i += advance
     j += advance
   if j < n + advance:
     if extra == 'keep':
       yield seq[i:j]
     elif extra == 'raise'
       raise ValueError('extra items')
     else:
       raise ValueError('bad extra')

Having gotten this far, it would be possible to treat the above as a 
fast path for sequences and wrap it in try:except and if len or slice 
fail, fall back to a general iterator version. The result could be a 
builtin rather than itertool.

-- 
Terry Jan Reedy


From grosser.meister.morti at gmx.net  Sun Dec  8 23:26:21 2013
From: grosser.meister.morti at gmx.net (=?UTF-8?B?TWF0aGlhcyBQYW56ZW5iw7Zjaw==?=)
Date: Sun, 08 Dec 2013 23:26:21 +0100
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <-8796784632715378232@gmail297201516>
References: <5251239173677340489@gmail297201516> <52A4B11F.7040200@gmx.net>
 <-8796784632715378232@gmail297201516>
Message-ID: <52A4F20D.2020704@gmx.net>

I see. Well, I woudln't expect that behaviour from such a function.

On 12/08/2013 07:06 PM, Amber Yust wrote:
> So does zip if the items are of unequal length, and the two code examples I provided (the one using zip and the
> long-form one) are equivalent.
>
> On Sun Dec 08 2013 at 9:49:56 AM, Mathias Panzenb?ck <grosser.meister.morti at gmx.net
> <mailto:grosser.meister.morti at gmx.net>> wrote:
>
>     On 12/08/2013 05:44 AM, Amber Yust wrote:
>      > After seeing yet another person asking how to do this on #python (and having needed to do it in the past myself), I'm
>      > wondering why itertools doesn't have a function to break an iterator up into N-sized chunks.
>      >
>      > Existing possible solutions include both the "clever" but somewhat unreadable...
>      >
>      >      batched_iter = zip(*[iter(input_iter)]*n)
>      >
>      > ...and the long-form...
>      >
>      >      def batch(input_iter, n):
>      >          input_iter = iter(input_iter)
>      >          while True:
>      >              yield [input_iter.next() for _ in range(n)]
>      >
>
>     This function drops items if the length of the input sequence is not a multiple of n. Fix:
>
>              def batch(it, n):
>                      it = iter(it)
>                      while True:
>                              slice = []
>                              for _ in range(n):
>                                      try:
>                                              slice.append(it.next())
>                                      except StopIteration:
>                                              if slice:
>                                                      yield slice
>                                              return
>                              yield slice
>
>      > There doesn't seem, however, to be one clear "right" way to do this. Every time I come up against this task, I go
>     back
>      > to itertools expecting one of the grouping functions there to cover it, but they don't.
>      >
>      > It seems like it would be a natural fit for itertools, and it would simplify things like processing of file
>     formats that
>      > use a consistent number of lines per entry, et cetera.
>      >
>      > ~Amber
>      >
>
>     _________________________________________________
>     Python-ideas mailing list
>     Python-ideas at python.org <mailto:Python-ideas at python.org>
>     https://mail.python.org/__mailman/listinfo/python-ideas <https://mail.python.org/mailman/listinfo/python-ideas>
>     Code of Conduct: http://python.org/psf/__codeofconduct/ <http://python.org/psf/codeofconduct/>
>


From antony.lee at berkeley.edu  Mon Dec  9 08:11:08 2013
From: antony.lee at berkeley.edu (Antony Lee)
Date: Sun, 8 Dec 2013 23:11:08 -0800
Subject: [Python-ideas] Automatically turning __prepare__ into a classmethod.
Message-ID: <CAGRr6BEoto8trfMdSQOxaiW=P777MoMkNsbxuQBcvYZKVV5=9w@mail.gmail.com>

Just a small idea...
__new__ is automagically turned into a staticmethod, because other
interpretations don't make sense.  Likewise, perhaps __prepare__ could be
automagically turned into a classmethod?
Antony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131208/08ebe525/attachment-0001.html>

From piotr.skamruk at gmail.com  Mon Dec  9 10:07:35 2013
From: piotr.skamruk at gmail.com (Piotr Skamruk)
Date: Mon, 9 Dec 2013 10:07:35 +0100
Subject: [Python-ideas] Easily reference a single unittest from cmdline
In-Reply-To: <CAFYqXL8=LTYCuhO2sn6B7vrr1Cf25Hmo6So6nTfuqtbx-FD8Ag@mail.gmail.com>
References: <CAFYqXL8VsVu4r2UXs7WLC4M+vRxJNshboAtytUsF1FN5GDL21Q@mail.gmail.com>
 <CAFYqXL8=LTYCuhO2sn6B7vrr1Cf25Hmo6So6nTfuqtbx-FD8Ag@mail.gmail.com>
Message-ID: <CAO_YSZMKh5OtbL_8+e1-0Q9n1CBqGCV0uUCH9TggK4PTmM1keQ@mail.gmail.com>

2013/12/7 Giampaolo Rodola' <g.rodola at gmail.com>:
> I just took a look at nosetests.
> It provides a similar thing and lets you use a regular expression
> (which indeed makes a lot more sense than using wildcards)
> It works like this:
>
> nosetests test/test_module -m test_name
Check http://pytest.org/latest/usage.html#usage
Probably: pytest -k part_of_test_name
is what You are seeking.

If You have a while - please look deeper on pytest - it's really
powerful/user friendly tool.

From musicdenotation at gmail.com  Mon Dec  9 11:07:00 2013
From: musicdenotation at gmail.com (musicdenotation at gmail.com)
Date: Mon, 09 Dec 2013 17:07:00 +0700
Subject: [Python-ideas] Pipe indentation
Message-ID: <52a5966d.41c4440a.6bde.31d4@mx.google.com>

if condition:
| do_this
| if cond2:
| |do_that
else:
| do_this_instead

This has the advantage of you always knowing which nesting level you are.

From rosuav at gmail.com  Mon Dec  9 11:11:09 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 9 Dec 2013 21:11:09 +1100
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
Message-ID: <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>

On Mon, Dec 9, 2013 at 9:07 PM,  <musicdenotation at gmail.com> wrote:
> if condition:
> | do_this
> | if cond2:
> | |do_that
> else:
> | do_this_instead
>
> This has the advantage of you always knowing which nesting level you are.

Alternative suggestion: Tab indentation. Do exactly the same thing,
but replace every pipe with a tab. Then you can set your editor to
show indentation visibly (SciTE does this already).

ChrisA

From daniel at daniel-watkins.co.uk  Mon Dec  9 11:29:29 2013
From: daniel at daniel-watkins.co.uk (Daniel Watkins)
Date: Mon, 9 Dec 2013 10:29:29 +0000
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
Message-ID: <20131209102929.GX22882@daniel-watkins.co.uk>

On Mon, Dec 09, 2013 at 09:11:09PM +1100, Chris Angelico wrote:
> On Mon, Dec 9, 2013 at 9:07 PM,  <musicdenotation at gmail.com> wrote:
> > if condition:
> > | do_this
> > | if cond2:
> > | |do_that
> > else:
> > | do_this_instead
> >
> > This has the advantage of you always knowing which nesting level you are.
> 
> Alternative suggestion: Tab indentation. Do exactly the same thing,
> but replace every pipe with a tab. Then you can set your editor to
> show indentation visibly (SciTE does this already).

Alternative suggestion: indent with 4 spaces.  Do exactly the same
thing, but replace every pipe with 4 spaces.  Then you can set your eyes
to see the visible indentation.

;)


More seriously, if you are at the point where you're getting confused by
levels of indentation, you're probably either (a) nesting too much, or
(b) including too much in indented blocks.  Either of these can be
solved by splitting your code up in to more smaller functions/methods
(which, if you name them well, will have the side-effect of making your
code clearer anyway!).


Cheers,

Dan

From oscar.j.benjamin at gmail.com  Mon Dec  9 11:54:40 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 9 Dec 2013 10:54:40 +0000
Subject: [Python-ideas] Batching/grouping function for itertools
In-Reply-To: <20131208121629.GV2085@ando>
References: <5251239173677340489@gmail297201516>
 <CABicbJKuoFdBaWGVTg3eghSQNwo7yNSHEPUVAyiR5EKfSR3a3g@mail.gmail.com>
 <CADiSq7fAgEgR-88yxhc3QNUkMZTjnOaVwqt_mp-VzprGUVZJ=w@mail.gmail.com>
 <20131208092517.GU2085@ando> <l81l8j$kgo$1@ger.gmane.org>
 <20131208121629.GV2085@ando>
Message-ID: <CAHVvXxQeO=6s8O84xvtzBkO_wn-S7pscJo1DefL=Mm2DGSVeaw@mail.gmail.com>

On 8 December 2013 12:16, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, Dec 08, 2013 at 01:30:56PM +0200, Serhiy Storchaka wrote:
>> 08.12.13 11:25, Steven D'Aprano ???????(??):
>> >In the second case, there is a question about what to do with sequences
>> >that are not a multiple of the window size. Similar to zip(), there are
>> >two things one might do:
>> >
>> >- pad with some given object;
>> >- raise an exception
>>
>> 3) emit last chunk incompleted;
>
> Given a window size of two, and input data [a, b, c], are you suggesting
> a variety that returns this?
>
> (a,b), (c,)

This is the only variant I have ever used. You can see an example use case here:
https://mail.python.org/pipermail//python-ideas/2013-August/022767.html

And this is the implementation I use:

def chunks(iterable, chunksize=100):
    islices = map(islice, repeat(iter(iterable)), repeat(chunksize))
    return takewhile(bool, map(list, islices))


Oscar

From denis.spir at gmail.com  Mon Dec  9 15:00:33 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 09 Dec 2013 15:00:33 +0100
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
Message-ID: <52A5CD01.7010605@gmail.com>

On 12/09/2013 11:07 AM, musicdenotation at gmail.com wrote:
> if condition:
> | do_this
> | if cond2:
> | |do_that
> else:
> | do_this_instead
>
> This has the advantage of you always knowing which nesting level you are.

Some programming editors, like Geany, do that for you :-) if you set the right 
setting (albeit with a "thinissim" greay dotted line).

Denis

From flying-sheep at web.de  Mon Dec  9 15:19:29 2013
From: flying-sheep at web.de (Philipp A.)
Date: Mon, 9 Dec 2013 15:19:29 +0100
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
Message-ID: <CAN8d9gnUe8T4LRLbQNuTXeeeFmWGHikoJq=CXR0tensQtDkbsw@mail.gmail.com>

2013/12/9 Chris Angelico <rosuav at gmail.com>

> Alternative suggestion: Tab indentation. Do exactly the same thing,
> but replace every pipe with a tab. Then you can set your editor to
> show indentation visibly (SciTE does this already).
>
> ChrisA
>

that?s also my favourite way to do it.

some editors can show tabs as an slim unobtrusive slim line and
leading/trailing spaces as middle dots (?)

that way you can not only instantly see all misplaced whitespace
characters, but also indentation levels. (one tab per indentation level is
pretty much as semantically unambiguous as it can get)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131209/44624815/attachment.html>

From skip at pobox.com  Mon Dec  9 15:45:58 2013
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 9 Dec 2013 08:45:58 -0600
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <20131209102929.GX22882@daniel-watkins.co.uk>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
 <20131209102929.GX22882@daniel-watkins.co.uk>
Message-ID: <CANc-5UytjA8VonxSOpxEbDPR2b_-A9VB-bH=QxxuGh+dJv9pmQ@mail.gmail.com>

> More seriously, if you are at the point where you're getting confused by
> levels of indentation, you're probably either (a) nesting too much, or
> (b) including too much in indented blocks.

Or (c), apparently only using one-space indentation. :-)

Skip

From breamoreboy at yahoo.co.uk  Mon Dec  9 16:17:36 2013
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 09 Dec 2013 15:17:36 +0000
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <20131209102929.GX22882@daniel-watkins.co.uk>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
 <20131209102929.GX22882@daniel-watkins.co.uk>
Message-ID: <l84mt2$bv$1@ger.gmane.org>

On 09/12/2013 10:29, Daniel Watkins wrote:
> On Mon, Dec 09, 2013 at 09:11:09PM +1100, Chris Angelico wrote:
>> On Mon, Dec 9, 2013 at 9:07 PM,  <musicdenotation at gmail.com> wrote:
>>> if condition:
>>> | do_this
>>> | if cond2:
>>> | |do_that
>>> else:
>>> | do_this_instead
>>>
>>> This has the advantage of you always knowing which nesting level you are.
>>
>> Alternative suggestion: Tab indentation. Do exactly the same thing,
>> but replace every pipe with a tab. Then you can set your editor to
>> show indentation visibly (SciTE does this already).
>
> Alternative suggestion: indent with 4 spaces.  Do exactly the same
> thing, but replace every pipe with 4 spaces.  Then you can set your eyes
> to see the visible indentation.
>
> ;)
>
>
> More seriously, if you are at the point where you're getting confused by
> levels of indentation, you're probably either (a) nesting too much, or
> (b) including too much in indented blocks.  Either of these can be
> solved by splitting your code up in to more smaller functions/methods
> (which, if you name them well, will have the side-effect of making your
> code clearer anyway!).
>
>
> Cheers,
>
> Dan

Alternative alternative alternative suggestion don't post items like 
this as I've just barfed all over my screen :(

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence


From flying-sheep at web.de  Mon Dec  9 20:22:05 2013
From: flying-sheep at web.de (Philipp A.)
Date: Mon, 9 Dec 2013 20:22:05 +0100
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CANc-5UytjA8VonxSOpxEbDPR2b_-A9VB-bH=QxxuGh+dJv9pmQ@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <CAPTjJmrUzrwB33+JVKO7eeGfxypXEXGEWnqzTFhS4k6=T0wFtg@mail.gmail.com>
 <20131209102929.GX22882@daniel-watkins.co.uk>
 <CANc-5UytjA8VonxSOpxEbDPR2b_-A9VB-bH=QxxuGh+dJv9pmQ@mail.gmail.com>
Message-ID: <CAN8d9g=g_y5H=UY_C7Li5Jes9qF2uHLO5DSHkzYunxUh1Y0rRA@mail.gmail.com>

bad joke, that makes it even less clear.

but @OP: yeah, syntax highlighting is the way to go as mainainung pipes and
spaces/tabs would be too much hassle even with editor support.


2013/12/9 Skip Montanaro <skip at pobox.com>

> > More seriously, if you are at the point where you're getting confused by
> > levels of indentation, you're probably either (a) nesting too much, or
> > (b) including too much in indented blocks.
>
> Or (c), apparently only using one-space indentation. :-)
>
> Skip
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131209/e30a488b/attachment.html>

From cjwelborn at live.com  Tue Dec 10 04:13:20 2013
From: cjwelborn at live.com (Christopher Welborn)
Date: Mon, 09 Dec 2013 21:13:20 -0600
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
Message-ID: <l860s5$51c$1@ger.gmane.org>

On 12/09/2013 04:07 AM, 
musicdenotation at gmail.com wrote:
> if condition:
> | do_this
> | if cond2:
> | |do_that
> else:
> | do_this_instead
>
> This has the advantage of you always knowing which nesting level you are.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>

Here's a link to a screen shot that shows Sublime Text's nesting level 
lines:

http://imgur.com/yRGNCKu

...a lot of editors do this, tab or space.

p.s. - it's not my code in the screen shot, its from the powerline 
module for vim (just grabbed the closest thing with nesting).

-- 

- Christopher Welborn <cjwelborn at live.com>
   http://welbornprod.com


From stephen at xemacs.org  Tue Dec 10 06:21:07 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 10 Dec 2013 14:21:07 +0900
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <l860s5$51c$1@ger.gmane.org>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
Message-ID: <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>

Christopher Welborn writes:

 > Here's a link to a screen shot that shows Sublime Text's nesting level 
 > lines:

 > http://imgur.com/yRGNCKu

 > ...a lot of editors do this, tab or space.

But do they get it Pythonically correct?  Ie, in counting indentation,
1 TAB = 1 SPC, regardless of what it looks like on screen.


From rosuav at gmail.com  Tue Dec 10 06:48:16 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 10 Dec 2013 16:48:16 +1100
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>

On Tue, Dec 10, 2013 at 4:21 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Christopher Welborn writes:
>
>  > Here's a link to a screen shot that shows Sublime Text's nesting level
>  > lines:
>
>  > http://imgur.com/yRGNCKu
>
>  > ...a lot of editors do this, tab or space.
>
> But do they get it Pythonically correct?  Ie, in counting indentation,
> 1 TAB = 1 SPC, regardless of what it looks like on screen.

SciTE gets it Python3ically correct: tab != space regardless of number.

ChrisA

From guido at python.org  Tue Dec 10 07:31:24 2013
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Dec 2013 22:31:24 -0800
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
Message-ID: <CAP7+vJJkfco9iNWCAE4jTTRt4B9FPKObA1hP==VTQAbTmeozuA@mail.gmail.com>

On Mon, Dec 9, 2013 at 9:48 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Dec 10, 2013 at 4:21 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>> Christopher Welborn writes:
>>
>>  > Here's a link to a screen shot that shows Sublime Text's nesting level
>>  > lines:
>>
>>  > http://imgur.com/yRGNCKu
>>
>>  > ...a lot of editors do this, tab or space.
>>
>> But do they get it Pythonically correct?  Ie, in counting indentation,
>> 1 TAB = 1 SPC, regardless of what it looks like on screen.

The default Python 2 rule is actually that 1 TAB == 8 SPC.

> SciTE gets it Python3ically correct: tab != space regardless of number.

Right. And in Python 2 you can get the same effect with python -tt.

-- 
--Guido van Rossum (python.org/~guido)

From stephen at xemacs.org  Tue Dec 10 08:02:47 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 10 Dec 2013 16:02:47 +0900
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
Message-ID: <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:
 > On Tue, Dec 10, 2013 at 4:21 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
 > > Christopher Welborn writes:
 > >
 > >  > Here's a link to a screen shot that shows Sublime Text's nesting level
 > >  > lines:
 > >
 > >  > http://imgur.com/yRGNCKu
 > >
 > >  > ...a lot of editors do this, tab or space.
 > >
 > > But do they get it Pythonically correct?  Ie, in counting indentation,
 > > 1 TAB = 1 SPC, regardless of what it looks like on screen.

Oops, that's TAB = SPC*8 in Python 2.

 > SciTE gets it Python3ically correct: tab != space regardless of
 > number.

But what does that mean?  How does SciTE distinguish between "SPC TAB"
at the beginning of line and just "TAB"?  Where does it place guidelines
given

def f(x):
SPC if x:
SPC SPC print("gotcha")
SPC else:
SPC TAB print("gotcha again")

which is acceptable to Python 3.3?  I suppose the right answer, as
usual, is "don't use both spaces and tabs for indentation in the same
program", but I guess then it really doesn't matter how the editor
handles tabs and spaces.

Oh, BTW, I'm definitely -1 on "pipe indentation" in the source
program, since editors can clearly do it if you want it (I don't)
whether it's present in the source, or not.






From rosuav at gmail.com  Tue Dec 10 08:10:58 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 10 Dec 2013 18:10:58 +1100
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
 <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmpwgVpx6JpyHg2kw=+Uznn_FW1jY1rOOwA5ciH9ufb9tg@mail.gmail.com>

On Tue, Dec 10, 2013 at 6:02 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> But what does that mean?  How does SciTE distinguish between "SPC TAB"
> at the beginning of line and just "TAB"?  Where does it place guidelines
> given
>
> def f(x):
> SPC if x:
> SPC SPC print("gotcha")
> SPC else:
> SPC TAB print("gotcha again")

Hmm, actually I have no idea, so I just checked. Seems I was slightly
wrong in my description; the guides seem to be at fixed column
positions (configurable, but not dynamic). The differentiation between
tab and space is only for its signalling highlight - if you mismatch
indentation in any way, the wrong one is highlighted with a very
visible marker. And that highlight correctly accepts your above
example, though it's a little finicky to type (putting a space, then a
tab, on an otherwise-blank line deletes the space, but one can be
explicitly inserted after the tab's already there). The upshot is that
you get instant feedback if you ever get tabs/spaces wrong, and you
get nice guides if you always indent consistently, and in between is
shooting yourself in the foot anyway.

ChrisA

From stephen at xemacs.org  Tue Dec 10 09:35:43 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 10 Dec 2013 17:35:43 +0900
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAPTjJmpwgVpx6JpyHg2kw=+Uznn_FW1jY1rOOwA5ciH9ufb9tg@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
 <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmpwgVpx6JpyHg2kw=+Uznn_FW1jY1rOOwA5ciH9ufb9tg@mail.gmail.com>
Message-ID: <87wqjd14tc.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > The upshot is that you get instant feedback if you ever get
 > tabs/spaces wrong, and you get nice guides if you always indent
 > consistently, and in between is shooting yourself in the foot
 > anyway.

Sounds like somebody's thought this through pretty carefully!  I'm an
emacs guy, so unlikely to switch, but that description sounds like a
good specification to start with if we add that kind of functionality
to our python-mode.

From solipsis at pitrou.net  Tue Dec 10 14:35:32 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Dec 2013 14:35:32 +0100
Subject: [Python-ideas] Pipe indentation
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20131210143532.4fb0bda8@fsol>

On Tue, 10 Dec 2013 14:21:07 +0900
"Stephen J. Turnbull" <stephen at xemacs.org>
wrote:
> Christopher Welborn writes:
> 
>  > Here's a link to a screen shot that shows Sublime Text's nesting level 
>  > lines:
> 
>  > http://imgur.com/yRGNCKu
> 
>  > ...a lot of editors do this, tab or space.
> 
> But do they get it Pythonically correct?  Ie, in counting indentation,
> 1 TAB = 1 SPC, regardless of what it looks like on screen.

Generally they're purely graphical, i.e. they don't care about the
*logical nesting* of your code, only about how many spaces are
drawn on screen.
(no, I don't know what it does with a non-monospaced font)

Regards

Antoine.



From cjwelborn at live.com  Tue Dec 10 14:55:01 2013
From: cjwelborn at live.com (Christopher Welborn)
Date: Tue, 10 Dec 2013 07:55:01 -0600
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org> <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <l876fb$s92$1@ger.gmane.org>

On 12/09/2013 11:21 PM, Stephen J. Turnbull wrote:
> Christopher Welborn writes:
>
>   > Here's a link to a screen shot that shows Sublime Text's nesting level
>   > lines:
>
>   > http://imgur.com/yRGNCKu
>
>   > ...a lot of editors do this, tab or space.
>
> But do they get it Pythonically correct?  Ie, in counting indentation,
> 1 TAB = 1 SPC, regardless of what it looks like on screen.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>

Whatever you set the tab-spacing to is what it shows up as. I set mine 
to 4 so the pic shows 4. If you set it to 1 or 2 you will see a lot of 
vertical lines.
-- 

- Christopher Welborn <cjwelborn at live.com>
   http://welbornprod.com


From guido at python.org  Tue Dec 10 16:47:40 2013
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Dec 2013 07:47:40 -0800
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <87wqjd14tc.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
 <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmpwgVpx6JpyHg2kw=+Uznn_FW1jY1rOOwA5ciH9ufb9tg@mail.gmail.com>
 <87wqjd14tc.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAP7+vJ+tDTgV0BweXt0iYZe3sORcKz1bXpd0ucRv_c2rd95Gng@mail.gmail.com>

On Tue, Dec 10, 2013 at 12:35 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Chris Angelico writes:
>
>  > The upshot is that you get instant feedback if you ever get
>  > tabs/spaces wrong, and you get nice guides if you always indent
>  > consistently, and in between is shooting yourself in the foot
>  > anyway.
>
> Sounds like somebody's thought this through pretty carefully!  I'm an
> emacs guy, so unlikely to switch, but that description sounds like a
> good specification to start with if we add that kind of functionality
> to our python-mode.

Probably the first time significant thought was put into this was when
I designed the -t/-tt option for Python 2. I don't recall when that
was but it was in a very early Python 2 release. Everything else seems
just derivative to me -- either the same solution, or buggy. :-)

-- 
--Guido van Rossum (python.org/~guido)

From tinchester at gmail.com  Tue Dec 10 19:29:57 2013
From: tinchester at gmail.com (=?UTF-8?Q?Tin_Tvrtkovi=C4=87?=)
Date: Tue, 10 Dec 2013 19:29:57 +0100
Subject: [Python-ideas] Automagically set logger name
Message-ID: <CAEJ-mUo8SS+JjMe9WtqEP8R3PERv9hixUAEsh2Fe_Xnhinq_Lg@mail.gmail.com>

Hello,

usually logger instances are retrieved and initialized with the module
name, using the well-known pattern:

logger = logging.getLogger(__name__)

In Java's very popular log4j library (which is cited as an influence by PEP
282), loggers are usually retrieved and initialized in basically an
identical way:

private final static Logger LOG = Logger.getLogger(<name-of-class>.class);

However, in the upcoming log4j 2 library, a new way is available:

private final static Logger LOG = LogManager.getLogger(); // Returns a
Logger with the name of the calling class. [1]

Basically the method throws an exception, catches it and fishes out the
class name from the stack trace. This is a little less explicit, but
still a more convenient (and less annoying) way of accomplishing an
extremely common pattern.

I was wondering what the core devs make of it:

   - is it a good idea? (In general, and in Python) Is it worth it?
   - is it feasible in Python? (taking into account other implementations
   too)
   - are there any gotchas that would make it worse than the current
   standard?



[1]:
https://svn.apache.org/repos/asf/logging/log4j/log4j2/trunk/log4j-api/src/main/java/org/apache/logging/log4j/LogManager.java
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131210/e9476bca/attachment.html>

From rymg19 at gmail.com  Tue Dec 10 19:32:34 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 10 Dec 2013 12:32:34 -0600
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAP7+vJ+tDTgV0BweXt0iYZe3sORcKz1bXpd0ucRv_c2rd95Gng@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
 <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmpwgVpx6JpyHg2kw=+Uznn_FW1jY1rOOwA5ciH9ufb9tg@mail.gmail.com>
 <87wqjd14tc.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJ+tDTgV0BweXt0iYZe3sORcKz1bXpd0ucRv_c2rd95Gng@mail.gmail.com>
Message-ID: <CAO41-mOgYxs57tc6Gc8VC=gjfDEtf95xVUoAT=MJNYTNQ_wo-g@mail.gmail.com>

I now feel dumbfounded because I never knew the -t/-tt option was there
until now...


On Tue, Dec 10, 2013 at 9:47 AM, Guido van Rossum <guido at python.org> wrote:

> On Tue, Dec 10, 2013 at 12:35 AM, Stephen J. Turnbull
> <stephen at xemacs.org> wrote:
> > Chris Angelico writes:
> >
> >  > The upshot is that you get instant feedback if you ever get
> >  > tabs/spaces wrong, and you get nice guides if you always indent
> >  > consistently, and in between is shooting yourself in the foot
> >  > anyway.
> >
> > Sounds like somebody's thought this through pretty carefully!  I'm an
> > emacs guy, so unlikely to switch, but that description sounds like a
> > good specification to start with if we add that kind of functionality
> > to our python-mode.
>
> Probably the first time significant thought was put into this was when
> I designed the -t/-tt option for Python 2. I don't recall when that
> was but it was in a very early Python 2 release. Everything else seems
> just derivative to me -- either the same solution, or buggy. :-)
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
When your hammer is C++, everything begins to look like a thumb.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131210/5ac9cf0a/attachment.html>

From boehm.matthew at gmail.com  Tue Dec 10 20:51:43 2013
From: boehm.matthew at gmail.com (Matthew Boehm)
Date: Tue, 10 Dec 2013 14:51:43 -0500
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <CAO41-mOgYxs57tc6Gc8VC=gjfDEtf95xVUoAT=MJNYTNQ_wo-g@mail.gmail.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmop-m3kR092bCKcTgdmoUCNms72OcCd+d=aMs-NXzdxVQ@mail.gmail.com>
 <87y53t1948.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPTjJmpwgVpx6JpyHg2kw=+Uznn_FW1jY1rOOwA5ciH9ufb9tg@mail.gmail.com>
 <87wqjd14tc.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAP7+vJ+tDTgV0BweXt0iYZe3sORcKz1bXpd0ucRv_c2rd95Gng@mail.gmail.com>
 <CAO41-mOgYxs57tc6Gc8VC=gjfDEtf95xVUoAT=MJNYTNQ_wo-g@mail.gmail.com>
Message-ID: <CAONmwHEEaJW8oSkdcxohft9CyU58gRZHvzAERfx9j1bBHUCCVw@mail.gmail.com>

Just to throw another editor's hat in the ring, the best way to achieve
this in vim is with the indent-guides plugin
https://github.com/nathanaelkane/vim-indent-guides however in stock vim,
you can use 'listchars' to render a tab as a pipe followed by spaces.

-Matt Boehm


On Tue, Dec 10, 2013 at 1:32 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:

> I now feel dumbfounded because I never knew the -t/-tt option was there
> until now...
>
>
> On Tue, Dec 10, 2013 at 9:47 AM, Guido van Rossum <guido at python.org>wrote:
>
>> On Tue, Dec 10, 2013 at 12:35 AM, Stephen J. Turnbull
>> <stephen at xemacs.org> wrote:
>> > Chris Angelico writes:
>> >
>> >  > The upshot is that you get instant feedback if you ever get
>> >  > tabs/spaces wrong, and you get nice guides if you always indent
>> >  > consistently, and in between is shooting yourself in the foot
>> >  > anyway.
>> >
>> > Sounds like somebody's thought this through pretty carefully!  I'm an
>> > emacs guy, so unlikely to switch, but that description sounds like a
>> > good specification to start with if we add that kind of functionality
>> > to our python-mode.
>>
>> Probably the first time significant thought was put into this was when
>> I designed the -t/-tt option for Python 2. I don't recall when that
>> was but it was in a very early Python 2 release. Everything else seems
>> just derivative to me -- either the same solution, or buggy. :-)
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
>
> --
> Ryan
> When your hammer is C++, everything begins to look like a thumb.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131210/79299cd9/attachment-0001.html>

From antony.lee at berkeley.edu  Wed Dec 11 01:39:45 2013
From: antony.lee at berkeley.edu (Antony Lee)
Date: Tue, 10 Dec 2013 16:39:45 -0800
Subject: [Python-ideas] Slightly changing the signature of the
	inspect.Parameter constructor
Message-ID: <CAGRr6BGxhZPZFYfA=s-eihd=66cXPWMGocru5ZYdqf_hpuYadQ@mail.gmail.com>

It seems to me that POSITIONAL_OR_KEYWORD is the most often used kind of
Parameter (after all, this is the "default" kind of Parameter), so perhaps
the constructor for Parameter could be changed from

def __init__(self, name, kind, *, default=_empty, annotation=_empty,
_partial_kwarg=False):

to

def __init__(self, name, kind=Parameter.POSITIONAL_OR_KEYWORD, *,
default=_empty, annotation=_empty, _partial_kwarg=False):

Any thoughts on that?

Best,
Antony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131210/14698c92/attachment.html>

From Steve.Dower at microsoft.com  Thu Dec 12 00:45:45 2013
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Wed, 11 Dec 2013 23:45:45 +0000
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <20131210143532.4fb0bda8@fsol>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>,<20131210143532.4fb0bda8@fsol>
Message-ID: <a2edd3ea7f00415bafa7b552939d71dd@BLUPR03MB293.namprd03.prod.outlook.com>

A space is always the same width, and tabs are normally rendered as multiple spaces in code editors, so it works just fine. Word wrap is where things get fun... (though I've only had one person complain that my Visual Studio extension doesn't handle it properly... who wraps code?)

Top posted from my Windows Phone
________________________________
From: Antoine Pitrou<mailto:solipsis at pitrou.net>
Sent: ?12/?11/?2013 0:36
To: python-ideas at python.org<mailto:python-ideas at python.org>
Subject: Re: [Python-ideas] Pipe indentation

On Tue, 10 Dec 2013 14:21:07 +0900
"Stephen J. Turnbull" <stephen at xemacs.org>
wrote:
> Christopher Welborn writes:
>
>  > Here's a link to a screen shot that shows Sublime Text's nesting level
>  > lines:
>
>  > http://imgur.com/yRGNCKu
>
>  > ...a lot of editors do this, tab or space.
>
> But do they get it Pythonically correct?  Ie, in counting indentation,
> 1 TAB = 1 SPC, regardless of what it looks like on screen.

Generally they're purely graphical, i.e. they don't care about the
*logical nesting* of your code, only about how many spaces are
drawn on screen.
(no, I don't know what it does with a non-monospaced font)

Regards

Antoine.


_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131211/f9186229/attachment.html>

From abarnert at yahoo.com  Thu Dec 12 02:11:13 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 11 Dec 2013 17:11:13 -0800
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <a2edd3ea7f00415bafa7b552939d71dd@BLUPR03MB293.namprd03.prod.outlook.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org> <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20131210143532.4fb0bda8@fsol>
 <a2edd3ea7f00415bafa7b552939d71dd@BLUPR03MB293.namprd03.prod.outlook.com>
Message-ID: <7BC2F430-08C5-46A2-A2EB-FFF24DB15464@yahoo.com>

On Dec 11, 2013, at 15:45, Steve Dower <Steve.Dower at microsoft.com> wrote:

> A space is always the same width, and tabs are normally rendered as multiple spaces in code editors, so it works just fine. Word wrap is where things get fun... (though I've only had one person complain  that my Visual Studio extension doesn't handle it properly... who wraps code?)

You mean viewing soft-wrapped if it goes beyond the window width? I do that (when I can't just reject or edit the code to fit properly in the first place). Much better than horizontally scrolling, especially in a terminal session. It's ugly no matter what you do, but at least you can see it.

> 
> Top posted from my Windows Phone
> From: Antoine Pitrou
> Sent: ?12/?11/?2013 0:36
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Pipe indentation
> 
> On Tue, 10 Dec 2013 14:21:07 +0900
> "Stephen J. Turnbull" <stephen at xemacs.org>
> wrote:
> > Christopher Welborn writes:
> > 
> >  > Here's a link to a screen shot that shows Sublime Text's nesting level 
> >  > lines:
> > 
> >  > http://imgur.com/yRGNCKu
> > 
> >  > ...a lot of editors do this, tab or space.
> > 
> > But do they get it Pythonically correct?  Ie, in counting indentation,
> > 1 TAB = 1 SPC, regardless of what it looks like on screen.
> 
> Generally they're purely graphical, i.e. they don't care about the
> *logical nesting* of your code, only about how many spaces are
> drawn on screen.
> (no, I don't know what it does with a non-monospaced font)
> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131211/d3f937ae/attachment.html>

From rymg19 at gmail.com  Thu Dec 12 23:05:58 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Thu, 12 Dec 2013 16:05:58 -0600
Subject: [Python-ideas] Pipe indentation
In-Reply-To: <7BC2F430-08C5-46A2-A2EB-FFF24DB15464@yahoo.com>
References: <52a5966d.41c4440a.6bde.31d4@mx.google.com>
 <l860s5$51c$1@ger.gmane.org>
 <871u1l2se4.fsf@uwakimon.sk.tsukuba.ac.jp> <20131210143532.4fb0bda8@fsol>
 <a2edd3ea7f00415bafa7b552939d71dd@BLUPR03MB293.namprd03.prod.outlook.com>
 <7BC2F430-08C5-46A2-A2EB-FFF24DB15464@yahoo.com>
Message-ID: <CAO41-mM973E5N=BminRpsi4d0=2bHvTwv4iyypKhdvVT6Ho1iQ@mail.gmail.com>

I usually span the code across multiple lines if it doesn't fit in the IDLE
window. I save gedit's Word Wrap for when I'm lazy.


On Wed, Dec 11, 2013 at 7:11 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Dec 11, 2013, at 15:45, Steve Dower <Steve.Dower at microsoft.com> wrote:
>
>  A space is always the same width, and tabs are normally rendered as
> multiple spaces in code editors, so it works just fine. Word wrap is where
> things get fun... (though I've only had one person complain that my Visual
> Studio extension doesn't handle it properly... who wraps code?)
>
>
> You mean viewing soft-wrapped if it goes beyond the window width? I do
> that (when I can't just reject or edit the code to fit properly in the
> first place). Much better than horizontally scrolling, especially in a
> terminal session. It's ugly no matter what you do, but at least you can see
> it.
>
>
> Top posted from my Windows Phone
>  ------------------------------
> From: Antoine Pitrou <solipsis at pitrou.net>
> Sent: 12/11/2013 0:36
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Pipe indentation
>
>   On Tue, 10 Dec 2013 14:21:07 +0900
> "Stephen J. Turnbull" <stephen at xemacs.org>
> wrote:
> > Christopher Welborn writes:
> >
> >  > Here's a link to a screen shot that shows Sublime Text's nesting
> level
> >  > lines:
> >
> >  > http://imgur.com/yRGNCKu
> >
> >  > ...a lot of editors do this, tab or space.
> >
> > But do they get it Pythonically correct?  Ie, in counting indentation,
> > 1 TAB = 1 SPC, regardless of what it looks like on screen.
>
> Generally they're purely graphical, i.e. they don't care about the
> *logical nesting* of your code, only about how many spaces are
> drawn on screen.
> (no, I don't know what it does with a non-monospaced font)
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
When your hammer is C++, everything begins to look like a thumb.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131212/36d2366b/attachment.html>

From fcoaps at yahoo.com  Tue Dec 17 14:07:43 2013
From: fcoaps at yahoo.com (f p)
Date: Tue, 17 Dec 2013 05:07:43 -0800 (PST)
Subject: [Python-ideas] Warning on conditionals
Message-ID: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>

Hi, I'm new here,

There are cases of bad use of int or str constants without comparison operators in the if statement as shown in this example:

http://stackoverflow.com/questions/20633818/python-if-elif-always-not-working.

Maybe the case in which the if statement is defeated should rate a warning.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131217/36f19074/attachment.html>

From __peter__ at web.de  Tue Dec 17 14:46:58 2013
From: __peter__ at web.de (Peter Otten)
Date: Tue, 17 Dec 2013 14:46:58 +0100
Subject: [Python-ideas] Warning on conditionals
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
Message-ID: <l8pkim$j0e$1@ger.gmane.org>

f p wrote:

> Hi, I'm new here,
> 
> There are cases of bad use of int or str constants without comparison
> operators in the if statement as shown in this example:
> 
> http://stackoverflow.com/questions/20633818/python-if-elif-always-not-
working.
> 
> Maybe the case in which the if statement is defeated should rate a
> warning.

When a newbie writes

if animal == "cat" or "dog":
    ...

it is clear that he has the wrong idea about how this is evaluated, but in 
the general case

if animal == cat or animal_barks:
    ...

this is not an error. A check would be of very limited use.
That said, there is an external tool that issues a warning:

$ cat tmp.py
import random
a = random.choice("ab")
if a == "b" or "c":
    print "yes"
else:
    assert False, "unreachable"
$ pychecker tmp.py
Processing module tmp (tmp.py)...
yes

Warnings...

tmp.py:3: Using a conditional statement with a constant value (c)



From python at mrabarnett.plus.com  Tue Dec 17 16:14:29 2013
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 17 Dec 2013 15:14:29 +0000
Subject: [Python-ideas] Warning on conditionals
In-Reply-To: <l8pkim$j0e$1@ger.gmane.org>
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
 <l8pkim$j0e$1@ger.gmane.org>
Message-ID: <52B06A55.4080001@mrabarnett.plus.com>

On 17/12/2013 13:46, Peter Otten wrote:
> f p wrote:
>
>> Hi, I'm new here,
>>
>> There are cases of bad use of int or str constants without comparison
>> operators in the if statement as shown in this example:
>>
>> http://stackoverflow.com/questions/20633818/python-if-elif-always-not-
> working.
>>
>> Maybe the case in which the if statement is defeated should rate a
>> warning.
>
> When a newbie writes
>
> if animal == "cat" or "dog":
>      ...
>
> it is clear that he has the wrong idea about how this is evaluated, but in
> the general case
>
> if animal == cat or animal_barks:
>      ...
>
> this is not an error. A check would be of very limited use.
> That said, there is an external tool that issues a warning:
>
> $ cat tmp.py
> import random
> a = random.choice("ab")
> if a == "b" or "c":
>      print "yes"
> else:
>      assert False, "unreachable"
> $ pychecker tmp.py
> Processing module tmp (tmp.py)...
> yes
>
> Warnings...
>
> tmp.py:3: Using a conditional statement with a constant value (c)
>
It's the kind of thing that you could have in IDLE.


From amber.yust at gmail.com  Tue Dec 17 16:34:38 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Tue, 17 Dec 2013 15:34:38 +0000
Subject: [Python-ideas] Warning on conditionals
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
 <l8pkim$j0e$1@ger.gmane.org> <52B06A55.4080001@mrabarnett.plus.com>
Message-ID: <8131640268748924455@gmail297201516>

A better check might be to look for cases where both (a) and/or are used in
a condition and (b) the right-hand side of the and/or is a constant.

On Tue Dec 17 2013 at 7:15:06 AM, MRAB <python at mrabarnett.plus.com> wrote:

> On 17/12/2013 13:46, Peter Otten wrote:
> > f p wrote:
> >
> >> Hi, I'm new here,
> >>
> >> There are cases of bad use of int or str constants without comparison
> >> operators in the if statement as shown in this example:
> >>
> >> http://stackoverflow.com/questions/20633818/python-if-elif-always-not-
> > working.
> >>
> >> Maybe the case in which the if statement is defeated should rate a
> >> warning.
> >
> > When a newbie writes
> >
> > if animal == "cat" or "dog":
> >      ...
> >
> > it is clear that he has the wrong idea about how this is evaluated, but
> in
> > the general case
> >
> > if animal == cat or animal_barks:
> >      ...
> >
> > this is not an error. A check would be of very limited use.
> > That said, there is an external tool that issues a warning:
> >
> > $ cat tmp.py
> > import random
> > a = random.choice("ab")
> > if a == "b" or "c":
> >      print "yes"
> > else:
> >      assert False, "unreachable"
> > $ pychecker tmp.py
> > Processing module tmp (tmp.py)...
> > yes
> >
> > Warnings...
> >
> > tmp.py:3: Using a conditional statement with a constant value (c)
> >
> It's the kind of thing that you could have in IDLE.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131217/4afaab60/attachment.html>

From denis.spir at gmail.com  Tue Dec 17 17:15:42 2013
From: denis.spir at gmail.com (spir)
Date: Tue, 17 Dec 2013 17:15:42 +0100
Subject: [Python-ideas] Warning on conditionals
In-Reply-To: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
Message-ID: <52B078AE.6060806@gmail.com>

On 12/17/2013 02:07 PM, f p wrote:
> Hi, I'm new here,
>
> There are cases of bad use of int or str constants without comparison operators in the if statement as shown in this example:
>
> http://stackoverflow.com/questions/20633818/python-if-elif-always-not-working.
>
> Maybe the case in which the if statement is defeated should rate a warning.

Original code (from stackoverflow) is:

if taxon == "Bracelets":
     catId = "178785"
elif taxon == "Kids Earrings" or "Earrings":
     catId = "177591"
[...more elif's...]

The actual design issue, in my view, is reproducing in programming operator 
priority from math syntax. If there were no such priority (at least, for logical 
operators), then people would have to write:

elif (taxon == "Kids Earrings") or "Earrings":
     catId = "177591"

...which makes semantic error obvious, or at least easy to understand, since in:

elif taxon == ("Kids Earrings" or "Earrings"):
     catId = "177591"

the second operand is a logical value, compared to a 'taxon'.

Without parenthesis grouping, however, a "fuzzy logic" (lol!), 
natural-language-like meaning is well possible, indeed it is the original 
coder's intent.

Related, but distinct, it is well known that priority of logical and/or is quite 
an effective source of bugs (also, the closer binding of not). And I remember a 
study about expressions with operators [*] showing how common bugs about plain 
arithmetic operator priority are (+-*/), as surprising as is. Evident 
conclusion, i guess is that operator priority in programming is bad. (For 
additional reasons, I guess the right choice in programming is prefix syntax 
operations.) [**]

Denis

[*] The purpose of the study, IIRC, was about syntactic complexity of 
operations, but it produced such data about error sources as a side-product.

[**] Actually, the same applies to math syntax, but there a revolution is even 
less probable -- see how reluctant mathematicians are to abandon the funny idea 
of PI beeing "the circle constant". http://tauday.com/

From abarnert at yahoo.com  Tue Dec 17 18:02:20 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 17 Dec 2013 09:02:20 -0800
Subject: [Python-ideas] Warning on conditionals
In-Reply-To: <52B078AE.6060806@gmail.com>
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
 <52B078AE.6060806@gmail.com>
Message-ID: <A2226615-EFB1-4972-B1D0-9FDD5371248B@yahoo.com>

On Dec 17, 2013, at 8:15, spir <denis.spir at gmail.com> wrote:

> On 12/17/2013 02:07 PM, f p wrote:
>> Hi, I'm new here,
>> 
>> There are cases of bad use of int or str constants without comparison operators in the if statement as shown in this example:
>> 
>> http://stackoverflow.com/questions/20633818/python-if-elif-always-not-working.
>> 
>> Maybe the case in which the if statement is defeated should rate a warning.
> 
> Original code (from stackoverflow) is:
> 
> if taxon == "Bracelets":
>    catId = "178785"
> elif taxon == "Kids Earrings" or "Earrings":
>    catId = "177591"
> [...more elif's...]
> 
> The actual design issue, in my view, is reproducing in programming operator priority from math syntax. If there were no such priority (at least, for logical operators), then people would have to write:
> 
> elif (taxon == "Kids Earrings") or "Earrings":
>    catId = "177591"
> 
> ...which makes semantic error obvious, or at least easy to understand, since in:
> 
> elif taxon == ("Kids Earrings" or "Earrings"):
>    catId = "177591"
> 
> the second operand is a logical value, compared to a 'taxon'.

No it isn't. The second operand is the string "Kids Earrings".  Which would be harder to debug, not easier.

> 
> Without parenthesis grouping, however, a "fuzzy logic" (lol!), natural-language-like meaning is well possible, indeed it is the original coder's intent.
> 
> Related, but distinct, it is well known that priority of logical and/or is quite an effective source of bugs (also, the closer binding of not). And I remember a study about expressions with operators [*] showing how common bugs about plain arithmetic operator priority are (+-*/), as surprising as is.
> Evident conclusion, i guess is that operator priority in programming is bad. (For additional reasons, I guess the right choice in programming is prefix syntax operations.) [**]

Operator precedence is surprising once, but makes your life easier once you learn it. Prefix notation is unreadable once, and remains unreadable even after you learn it. It's as high a barrier as postfix (RPN) notation, but without the advantages. (Plenty of engineers use their trusty HP 48 happily. But no one uses prefix notation except programmers in languages like Java and Lisp dialects without infix, and all of them complain about it.) I assume no one is going to suggest turning Python into Forth, so why suggest turning it into a form that has all the disadvantages of both with the advantages of neither?

Anyway, it's easy to write a linter that enforces simple rules like not relying on precedence (forcing explicit parens), and then people (and teams, and teachers) who want to follow those rules can do so.

> 
> Denis
> 
> [*] The purpose of the study, IIRC, was about syntactic complexity of operations, but it produced such data about error sources as a side-product.
> 
> [**] Actually, the same applies to math syntax, but there a revolution is even less probable -- see how reluctant mathematicians are to abandon the funny idea of PI beeing "the circle constant". http://tauday.com/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From abarnert at yahoo.com  Tue Dec 17 18:07:36 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 17 Dec 2013 09:07:36 -0800
Subject: [Python-ideas] Warning on conditionals
In-Reply-To: <52B06A55.4080001@mrabarnett.plus.com>
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
 <l8pkim$j0e$1@ger.gmane.org> <52B06A55.4080001@mrabarnett.plus.com>
Message-ID: <9016F381-0762-41F7-A2B4-E08C7B2E8385@yahoo.com>

On Dec 17, 2013, at 7:14, MRAB <python at mrabarnett.plus.com> wrote:

> On 17/12/2013 13:46, Peter Otten wrote:
>> When a newbie writes
>> 
>> if animal == "cat" or "dog":
>>     ...
>> 
>> it is clear that he has the wrong idea about how this is evaluated, but in
>> the general case
>> 
>> if animal == cat or animal_barks:
>>     ...
>> 
>> this is not an error. A check would be of very limited use.
>> That said, there is an external tool that issues a warning:
>> 
>> $ cat tmp.py
>> import random
>> a = random.choice("ab")
>> if a == "b" or "c":
>>     print "yes"
>> else:
>>     assert False, "unreachable"
>> $ pychecker tmp.py
>> Processing module tmp (tmp.py)...
>> yes
>> 
>> Warnings...
>> 
>> tmp.py:3: Using a conditional statement with a constant value (c)
> It's the kind of thing that you could have in IDLE.

We have multiple different linters and style checkers for a reason. Until one of them is so clearly perfect that there's no need for the others, I think it would be more useful to make it easier to integrate checkers into IDLE than to try to duplicate their functionality.

Also, a pretty high percentage of newbies don't use IDLE, so this wouldn't solve the problem for most people.

From tjreedy at udel.edu  Tue Dec 17 20:55:05 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 17 Dec 2013 14:55:05 -0500
Subject: [Python-ideas] Warning on conditionals
In-Reply-To: <9016F381-0762-41F7-A2B4-E08C7B2E8385@yahoo.com>
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
 <l8pkim$j0e$1@ger.gmane.org> <52B06A55.4080001@mrabarnett.plus.com>
 <9016F381-0762-41F7-A2B4-E08C7B2E8385@yahoo.com>
Message-ID: <l8qa6e$t2k$1@ger.gmane.org>

On 12/17/2013 12:07 PM, Andrew Barnert wrote:
> On Dec 17, 2013, at 7:14, MRAB <python at mrabarnett.plus.com> wrote:

>>> tmp.py:3: Using a conditional statement with a constant value (c)
>> It's the kind of thing that you could have in IDLE.
>
> We have multiple different linters and style checkers for a reason. Until one of them is so clearly perfect that there's no need for the others, I think it would be more useful to make it easier to integrate checkers into IDLE than to try to duplicate their functionality.

There is an issue on the tracker to make it easy to integrate code 
checkers into Idle.


-- 
Terry Jan Reedy


From haoyi.sg at gmail.com  Tue Dec 17 21:01:28 2013
From: haoyi.sg at gmail.com (Haoyi Li)
Date: Tue, 17 Dec 2013 12:01:28 -0800
Subject: [Python-ideas] Warning on conditionals
In-Reply-To: <l8qa6e$t2k$1@ger.gmane.org>
References: <1387285663.29639.YahooMailNeo@web161701.mail.bf1.yahoo.com>
 <l8pkim$j0e$1@ger.gmane.org> <52B06A55.4080001@mrabarnett.plus.com>
 <9016F381-0762-41F7-A2B4-E08C7B2E8385@yahoo.com> <l8qa6e$t2k$1@ger.gmane.org>
Message-ID: <CALruUQJZJK6GFO0dDL4+TB7KvLSCrwAxtn0wCmhzg6VGE4zHoQ@mail.gmail.com>

> But no one uses prefix notation except programmers in languages like Java
and Lisp dialects without infix

And I must say, even in Java and Javascript, the libraries with APIs people
love often are the ones who use heavy method-chaining, like Guava and
JQuery and Underscore. Other than the fact that the method names are
alphanumeric instead of symbolic, method chaining is basically infix
notation too.


On Tue, Dec 17, 2013 at 11:55 AM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 12/17/2013 12:07 PM, Andrew Barnert wrote:
>
>> On Dec 17, 2013, at 7:14, MRAB <python at mrabarnett.plus.com> wrote:
>>
>
>  tmp.py:3: Using a conditional statement with a constant value (c)
>>>>
>>> It's the kind of thing that you could have in IDLE.
>>>
>>
>> We have multiple different linters and style checkers for a reason. Until
>> one of them is so clearly perfect that there's no need for the others, I
>> think it would be more useful to make it easier to integrate checkers into
>> IDLE than to try to duplicate their functionality.
>>
>
> There is an issue on the tracker to make it easy to integrate code
> checkers into Idle.
>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131217/ce8ecaec/attachment.html>

From mikhtonyuk at gmail.com  Sat Dec 21 14:14:05 2013
From: mikhtonyuk at gmail.com (Sergii Mikhtoniuk)
Date: Sat, 21 Dec 2013 15:14:05 +0200
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
Message-ID: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>

Hi all,

I would very much appreciate your opinion on my proposal for improvement of
*concurrent.futures* package.

Comparing to other languages such as Scala and C#, Python?s futures
significantly fall behind in functionality especially in ability to chain
computations and compose different futures without blocking and waiting for
result. New packages continue to emerge (*asyncio*) which provide their own
futures implementation, making composition even more difficult.

Proposed improvement implements Scala-like Future as a monadic construct.
It allows performing multiple kinds of operations on Future?s result
without blocking, enabling reactive programming in Python. It implements
common pattern separating *Future* and *Promise* interface, making it very
easy for 3rd party systems to use futures in their API.

Please have a look at this PEP
draft<https://rawgithub.com/mikhtonyuk/rxpython/master/pep-0000.html>,
and reference implementation <https://github.com/mikhtonyuk/rxpython> (as
separate library).

I?m very interested in:
- How PEPable is this?
- What are your thoughts on backward compatibility (current implementation
does not sacrifice any design points for it, but better compatibility can
be achieved)?
- Thoughts on Future-based APIs in other packages?

Thanks,
Sergii
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131221/f8807ec0/attachment.html>

From phd at phdru.name  Sat Dec 21 14:36:21 2013
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 21 Dec 2013 14:36:21 +0100
Subject: [Python-ideas] PEP draft - Composable futures for reactive
 programming
In-Reply-To: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
Message-ID: <20131221133621.GA23931@phdru.name>

Hi!

On Sat, Dec 21, 2013 at 03:14:05PM +0200, Sergii Mikhtoniuk <mikhtonyuk at gmail.com> wrote:
> Please have a look at this PEP
> draft<https://rawgithub.com/mikhtonyuk/rxpython/master/pep-0000.html>,

   It would be nice to have a text version of the PEP posted to the
mailing list.

> and reference implementation <https://github.com/mikhtonyuk/rxpython> (as
> separate library).

   May I advice to add it to the References section of the PEP?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From ncoghlan at gmail.com  Sat Dec 21 15:12:46 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Dec 2013 00:12:46 +1000
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
Message-ID: <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>

On 21 December 2013 23:14, Sergii Mikhtoniuk <mikhtonyuk at gmail.com> wrote:
> Please have a look at this PEP draft, and reference implementation (as
> separate library).
>
> I?m very interested in:
> - How PEPable is this?

This looks like a really interesting idea, and well worth pursuing as
a PEP for 3.5 (the failure to make the Future/Promise split was
actually noted as a flaw in the concurrent.futures design at the time,
but wasn't seen as serious enough to prevent inclusion of the module.
However, this draft PEP does a decent job of showing the kinds of
operations that the current combined class design can make difficult)

> - What are your thoughts on backward compatibility (current implementation
> does not sacrifice any design points for it, but better compatibility can be
> achieved)?

Yes, I think better compatibility is needed. One possible option would
be to take the path of defining a "concurrent.futures.abc" module that
provides this higher level API, and largely leave the existing
concrete classes alone (aside from adjusting them to fit the ABC).

The main possible issue I see is with the result setting APIs on the
existing Future objects.

There are also a lot of staticmethod declarations that look like they
should probably be classmethod declarations.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guido at python.org  Sat Dec 21 18:59:08 2013
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Dec 2013 09:59:08 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
Message-ID: <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>

I'm pretty worried about this proposal. It seems to encourage
callback-style programming, whereas I would like to get away from it
(that's the main thrust of asyncio/PEP 3156). The proposal reminds me
of Twisted Deferred, which is too complicated for my taste.

Note that asyncio has a composition operation of its own, gather(),
but encourages using standard try/except/else/finally syntax for error
handling, rather than adding Twisted-style errbacks.

Nevertheless it is possible that I am misunderstanding the proposal
(e.g. the distinction between Future and Promise wasn't clear from
reading the draft PEP, nor how it would interact with asyncio). I
recommend that the author work find a way to improve the English
grammar of the proposal, e.g. by working with a native speaker.

On Sat, Dec 21, 2013 at 6:12 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 21 December 2013 23:14, Sergii Mikhtoniuk <mikhtonyuk at gmail.com> wrote:
>> Please have a look at this PEP draft, and reference implementation (as
>> separate library).
>>
>> I?m very interested in:
>> - How PEPable is this?
>
> This looks like a really interesting idea, and well worth pursuing as
> a PEP for 3.5 (the failure to make the Future/Promise split was
> actually noted as a flaw in the concurrent.futures design at the time,
> but wasn't seen as serious enough to prevent inclusion of the module.
> However, this draft PEP does a decent job of showing the kinds of
> operations that the current combined class design can make difficult)
>
>> - What are your thoughts on backward compatibility (current implementation
>> does not sacrifice any design points for it, but better compatibility can be
>> achieved)?
>
> Yes, I think better compatibility is needed. One possible option would
> be to take the path of defining a "concurrent.futures.abc" module that
> provides this higher level API, and largely leave the existing
> concrete classes alone (aside from adjusting them to fit the ABC).
>
> The main possible issue I see is with the result setting APIs on the
> existing Future objects.
>
> There are also a lot of staticmethod declarations that look like they
> should probably be classmethod declarations.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
--Guido van Rossum (python.org/~guido)

From mikhtonyuk at gmail.com  Sat Dec 21 23:48:29 2013
From: mikhtonyuk at gmail.com (Sergii Mikhtoniuk)
Date: Sun, 22 Dec 2013 00:48:29 +0200
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
Message-ID: <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>

Hello Guido, Nick,

Thanks a lot for your valuable feedback.

Indeed there is a lot of overlap between *asyncio* and
*concurrent.futures*packages, so it would be very interesting to hear
your overall thoughts on
role/future of *concurrent* package. Do you consider it rudimentary and
replaceable by *asyncio.Future* completely?

I think the question is even not much about which is your preferred
implementation, but rather do we want having futures as stand-alone package
or not. Do you see all these implementations converging in future?

To me *Future* is a very simple and self-contained primitive, independent
of thread pools, processes, IO, and networking. One thing
*concurrent.futures* package does a very good job at is defining an
isolated namespace for futures, stressing out this clear boundary (let?s
disregard here *ThreadPoolExecutor* and *ProcessPoolExecutor* classes which
for some reason ended up in it too).

So when I think of *schedulers* and *event loops* implementations I see
them as ones that build on top of the *Future* primitive, not providing it
as part of their implementation. What I think is important is that having
unified *Future* class simplifies interoperability between different kinds
of schedulers, not necessarily associated with *asyncio* event loops
(process pools for example).

*Futures* are definitely not the final solution for concurrency problem but
rather a well-established utility for representing async operations. It is
a job of higher layer systems to provide more convenient ways for dealing
with asyncs (*yield from* coroutines, *async/await* rewrites etc.), so I
would not say that futures encourage callback-style programming, it?s
simply a lower-layer functionality. On the contrary, monadic methods for
futures composition (e.g. *map()*, *all()*, *first()* etc.) ensure that no
errors would be lost in the process, so I think they would complement*
yield from* model quite nicely by hiding complexity of state maintenance
from user and reducing the number of back-and-forth communications between
event loop and coroutines.

Besides *Futures*, reactive programming has more utilities to offer, such
as *Observables* (representing asynchronous streams of values). It is also
a very useful abstraction with a rich set of composition strategies
(merging, concatenation, grouping), and may deserve its place in separate
package.

Hope to hear back from you to get better picture on overall design
direction here before jumping to implementation details.

Brief follow-up to your questions:
 - Idea behind the Future/Promise separation is to draw a clean line
between client-facing and scheduler-side APIs respectively. Futures should
not expose any completion methods, which clients should not call anyway.
 - Completely agree with you that Twisted-style callbacks are evil and it
is better to have single code path for getting result or raising exception
 - Sorry for bad grammar in the proposal, it?s an early draft written in 3
AM, so I will definitely improve on it if we decide to move forward.


Thanks,
Sergii


On Sat, Dec 21, 2013 at 7:59 PM, Guido van Rossum <guido at python.org> wrote:

> I'm pretty worried about this proposal. It seems to encourage
> callback-style programming, whereas I would like to get away from it
> (that's the main thrust of asyncio/PEP 3156). The proposal reminds me
> of Twisted Deferred, which is too complicated for my taste.
>
> Note that asyncio has a composition operation of its own, gather(),
> but encourages using standard try/except/else/finally syntax for error
> handling, rather than adding Twisted-style errbacks.
>
> Nevertheless it is possible that I am misunderstanding the proposal
> (e.g. the distinction between Future and Promise wasn't clear from
> reading the draft PEP, nor how it would interact with asyncio). I
> recommend that the author work find a way to improve the English
> grammar of the proposal, e.g. by working with a native speaker.
>
> On Sat, Dec 21, 2013 at 6:12 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On 21 December 2013 23:14, Sergii Mikhtoniuk <mikhtonyuk at gmail.com>
> wrote:
> >> Please have a look at this PEP draft, and reference implementation (as
> >> separate library).
> >>
> >> I?m very interested in:
> >> - How PEPable is this?
> >
> > This looks like a really interesting idea, and well worth pursuing as
> > a PEP for 3.5 (the failure to make the Future/Promise split was
> > actually noted as a flaw in the concurrent.futures design at the time,
> > but wasn't seen as serious enough to prevent inclusion of the module.
> > However, this draft PEP does a decent job of showing the kinds of
> > operations that the current combined class design can make difficult)
> >
> >> - What are your thoughts on backward compatibility (current
> implementation
> >> does not sacrifice any design points for it, but better compatibility
> can be
> >> achieved)?
> >
> > Yes, I think better compatibility is needed. One possible option would
> > be to take the path of defining a "concurrent.futures.abc" module that
> > provides this higher level API, and largely leave the existing
> > concrete classes alone (aside from adjusting them to fit the ABC).
> >
> > The main possible issue I see is with the result setting APIs on the
> > existing Future objects.
> >
> > There are also a lot of staticmethod declarations that look like they
> > should probably be classmethod declarations.
> >
> > Cheers,
> > Nick.
> >
> > --
> > Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131222/5f8415d4/attachment-0001.html>

From guido at python.org  Sun Dec 22 01:26:51 2013
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Dec 2013 16:26:51 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
Message-ID: <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>

On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk <mikhtonyuk at gmail.com> wrote:
> Hello Guido, Nick,
>
> Thanks a lot for your valuable feedback.

You're welcome.

> Indeed there is a lot of overlap between asyncio and concurrent.futures
> packages, so it would be very interesting to hear your overall thoughts on
> role/future of concurrent package. Do you consider it rudimentary and
> replaceable by asyncio.Future completely?

They don't really compare. concurrent.futures is about *threads*.
asyncio.Future is about *avoiding* threads in favor of more
lightweight "tasks" and "coroutines", which in turn are built on top
of lower-level callbacks.

(While I want to get away from callbacks as a programming paradigm,
asyncio uses them at the lower levels both because they are a logical
low-level building block and for interoperability with other
frameworks like Tornado and Twisted.)

> I think the question is even not much about which is your preferred
> implementation, but rather do we want having futures as stand-alone package
> or not. Do you see all these implementations converging in future?

I see them as not even competing. They use different paradigms and
apply to different use cases.

> To me Future is a very simple and self-contained primitive, independent of
> thread pools, processes, IO, and networking.

(Agreed on the I/O and networking part only.)

> One thing concurrent.futures
> package does a very good job at is defining an isolated namespace for
> futures, stressing out this clear boundary (let?s disregard here
> ThreadPoolExecutor and ProcessPoolExecutor classes which for some reason
> ended up in it too).

Actually the executor API is an important and integral part of that
package, and threads underlie everything you can do with its Futures.

> So when I think of schedulers and event loops implementations I see them as
> ones that build on top of the Future primitive, not providing it as part of
> their implementation. What I think is important is that having unified
> Future class simplifies interoperability between different kinds of
> schedulers, not necessarily associated with asyncio event loops (process
> pools for example).

The interoperability is completely missing. I can't tell if you've
used asyncio at all, but the important operation of *waiting* for a
result is fundamentally different there than in concurrent.futures. In
the latter, you just write "x = f.result()" and your thread blocks
until the result is available. In asyncio, you have to write "x =
yield from f.result()" which is a coroutine block that lets other
tasks run in the same thread. ("yield from" in this case is how Python
spells the operation that C# calls "await").

> Futures are definitely not the final solution for concurrency problem but
> rather a well-established utility for representing async operations. It is a
> job of higher layer systems to provide more convenient ways for dealing with
> asyncs (yield from coroutines, async/await rewrites etc.),

But unless you are proposing some kind of radical change to add
compile-time type checking/inference to Python, the rewrite option is
unavailable in Python.

> so I would not
> say that futures encourage callback-style programming,

The concurrent.futures.Future class does not. But unless I misread
your proposal, your extensions do.

> it?s simply a
> lower-layer functionality. On the contrary, monadic

(Say that word one more time and everyone tunes out. :-)

> methods for futures
> composition (e.g. map(), all(), first() etc.) ensure that no errors would be
> lost in the process, so I think they would complement yield from model quite
> nicely by hiding complexity of state maintenance from user and reducing the
> number of back-and-forth communications between event loop and coroutines.

I'm not sure I follow. Again, I'm not sure if you've actually written
any code using asyncio.

TBH I've written a fair number of example programs for asyncio and
I've very rarely felt the need for these composition functions. The
main composition primitive I tend to use is "yield from".

> Besides Futures, reactive programming

What *is* reactive programming? If you're talking about
http://en.wikipedia.org/wiki/Reactive_programming,
I'm not sure that it maps well to Python.

> has more utilities to offer, such as
> Observables (representing asynchronous streams of values). It is also a very
> useful abstraction with a rich set of composition strategies (merging,
> concatenation, grouping), and may deserve its place in separate package.

It all sounds very abstract and academic. :-)

> Hope to hear back from you to get better picture on overall design direction
> here before jumping to implementation details.
>
> Brief follow-up to your questions:
>  - Idea behind the Future/Promise separation is to draw a clean line between
> client-facing and scheduler-side APIs respectively. Futures should not
> expose any completion methods, which clients should not call anyway.

Given that Future and Promise are often used synonymously, using them
to make this distinction sounds confusing. I agree that Futures have
two different APIs, one for the consumer and another for the producer.
But I'm not sure that it's necessary to separate them more strictly --
convention seems good enough to me here. (It's the same with many
communication primitives, like queues and even threads.)

(The one thing that trips people up frequently is that, while
set_result() and set_exception() are producer APIs, cancel() is a
consumer API, and the cancellation signal travels from the consumer to
the producer.)

>  - Completely agree with you that Twisted-style callbacks are evil and it is
> better to have single code path for getting result or raising exception
>  - Sorry for bad grammar in the proposal, it?s an early draft written in 3
> AM, so I will definitely improve on it if we decide to move forward.

No problem!

> Thanks,
> Sergii

-- 
--Guido van Rossum (python.org/~guido)

From ben at bendarnell.com  Sun Dec 22 02:37:26 2013
From: ben at bendarnell.com (Ben Darnell)
Date: Sat, 21 Dec 2013 20:37:26 -0500
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
Message-ID: <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>

On Sat, Dec 21, 2013 at 7:26 PM, Guido van Rossum <guido at python.org> wrote:

> On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk <mikhtonyuk at gmail.com>
> wrote:
>  > Indeed there is a lot of overlap between asyncio and concurrent.futures
> > packages, so it would be very interesting to hear your overall thoughts
> on
> > role/future of concurrent package. Do you consider it rudimentary and
> > replaceable by asyncio.Future completely?
>
> They don't really compare. concurrent.futures is about *threads*.
> asyncio.Future is about *avoiding* threads in favor of more
> lightweight "tasks" and "coroutines", which in turn are built on top
> of lower-level callbacks.
>

concurrent.futures.ThreadPoolExecutor is about threads; the Future class
itself is broader.  When I integrated Futures into Tornado I used
concurrent.futures.Future directly (when available).  asyncio.Future is
just an optimized version of c.f.Future (the optimization comes from
assuming single-threaded usage).  There should at least be a common ABC
between them.


>
> (While I want to get away from callbacks as a programming paradigm,
> asyncio uses them at the lower levels both because they are a logical
> low-level building block and for interoperability with other
> frameworks like Tornado and Twisted.)
>
> > I think the question is even not much about which is your preferred
> > implementation, but rather do we want having futures as stand-alone
> package
> > or not. Do you see all these implementations converging in future?
>
> I see them as not even competing. They use different paradigms and
> apply to different use cases.
>
> > To me Future is a very simple and self-contained primitive, independent
> of
> > thread pools, processes, IO, and networking.
>
> (Agreed on the I/O and networking part only.)
>
> > One thing concurrent.futures
> > package does a very good job at is defining an isolated namespace for
> > futures, stressing out this clear boundary (let?s disregard here
> > ThreadPoolExecutor and ProcessPoolExecutor classes which for some reason
> > ended up in it too).
>
> Actually the executor API is an important and integral part of that
> package, and threads underlie everything you can do with its Futures.
>
> > So when I think of schedulers and event loops implementations I see them
> as
> > ones that build on top of the Future primitive, not providing it as part
> of
> > their implementation. What I think is important is that having unified
> > Future class simplifies interoperability between different kinds of
> > schedulers, not necessarily associated with asyncio event loops (process
> > pools for example).
>
> The interoperability is completely missing. I can't tell if you've
> used asyncio at all, but the important operation of *waiting* for a
> result is fundamentally different there than in concurrent.futures. In
> the latter, you just write "x = f.result()" and your thread blocks
> until the result is available. In asyncio, you have to write "x =
> yield from f.result()" which is a coroutine block that lets other
> tasks run in the same thread. ("yield from" in this case is how Python
> spells the operation that C# calls "await").
>

The way I see it, the fundamental operation on Futures is
add_done_callback.  We then have various higher-level operations that let
us get away from using callbacks directly.  One of these happens to be a
method on Future: the blocking mode of Future.result().  Another is
implemented in asyncio and Tornado, in the ability to "yield" a Future.
 The blocking mode of result() is just a convenience; if asyncio-style
futures had existed first then we could instead have a function like "x =
concurrent.futures.wait_for(f)".  In fact, you could write this wait_for
function today in a way that works for both concurrent and asyncio futures.


This is already interoperable:  Tornado's Resolver interface (
https://github.com/facebook/tornado/blob/master/tornado/netutil.py#L184)
returns a Future, which may be generated by a ThreadPoolExecutor or an
asynchronous wrapper around pycares or twisted.  In the other direction
I've worked on hybrid apps that have one Tornado thread alongside a bunch
of Django threads; in these apps it would work to have a Django thread
block on f.result() for a Future returned by Tornado's AsyncHTTPClient.

-Ben


> > Futures are definitely not the final solution for concurrency problem but
> > rather a well-established utility for representing async operations. It
> is a
> > job of higher layer systems to provide more convenient ways for dealing
> with
> > asyncs (yield from coroutines, async/await rewrites etc.),
>
> But unless you are proposing some kind of radical change to add
> compile-time type checking/inference to Python, the rewrite option is
> unavailable in Python.
>
> > so I would not
> > say that futures encourage callback-style programming,
>
> The concurrent.futures.Future class does not. But unless I misread
> your proposal, your extensions do.
>
> > it?s simply a
> > lower-layer functionality. On the contrary, monadic
>
> (Say that word one more time and everyone tunes out. :-)
>
> > methods for futures
> > composition (e.g. map(), all(), first() etc.) ensure that no errors
> would be
> > lost in the process, so I think they would complement yield from model
> quite
> > nicely by hiding complexity of state maintenance from user and reducing
> the
> > number of back-and-forth communications between event loop and
> coroutines.
>
> I'm not sure I follow. Again, I'm not sure if you've actually written
> any code using asyncio.
>
> TBH I've written a fair number of example programs for asyncio and
> I've very rarely felt the need for these composition functions. The
> main composition primitive I tend to use is "yield from".
>
> > Besides Futures, reactive programming
>
> What *is* reactive programming? If you're talking about
> http://en.wikipedia.org/wiki/Reactive_programming,
> I'm not sure that it maps well to Python.
>
> > has more utilities to offer, such as
> > Observables (representing asynchronous streams of values). It is also a
> very
> > useful abstraction with a rich set of composition strategies (merging,
> > concatenation, grouping), and may deserve its place in separate package.
>
> It all sounds very abstract and academic. :-)
>
> > Hope to hear back from you to get better picture on overall design
> direction
> > here before jumping to implementation details.
> >
> > Brief follow-up to your questions:
> >  - Idea behind the Future/Promise separation is to draw a clean line
> between
> > client-facing and scheduler-side APIs respectively. Futures should not
> > expose any completion methods, which clients should not call anyway.
>
> Given that Future and Promise are often used synonymously, using them
> to make this distinction sounds confusing. I agree that Futures have
> two different APIs, one for the consumer and another for the producer.
> But I'm not sure that it's necessary to separate them more strictly --
> convention seems good enough to me here. (It's the same with many
> communication primitives, like queues and even threads.)
>
> (The one thing that trips people up frequently is that, while
> set_result() and set_exception() are producer APIs, cancel() is a
> consumer API, and the cancellation signal travels from the consumer to
> the producer.)
>
> >  - Completely agree with you that Twisted-style callbacks are evil and
> it is
> > better to have single code path for getting result or raising exception
> >  - Sorry for bad grammar in the proposal, it?s an early draft written in
> 3
> > AM, so I will definitely improve on it if we decide to move forward.
>
> No problem!
>
> > Thanks,
> > Sergii
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131221/5713d9e8/attachment-0001.html>

From guido at python.org  Sun Dec 22 04:45:06 2013
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Dec 2013 19:45:06 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
Message-ID: <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>

There's still the issue that in the threading version, you wait for a
Future by blocking the current thread, while in the asyncio version,
you must use "yield from" to block. For interoperability you would
have to refrain from *any* blocking operations (including "yield
from") so you would only be able to use callbacks. But whether you had
to write "x = f.result()" or "x = concurrent.futures.wait_for(f)",
either way you'd implicitly be blocking the current thread.

Yes, a clever scheduler could run other callbacks while blocking, but
that's not a complete solution, because another callback might do a
similar blocking operation, and whatever that waited for could hold up
the earlier blocking operation, causing an ever-deeper recursion and
of event loop invocations that might never complete. (I've had to
debug this in production code.) To cut through that you'd have to have
some kind of stack-swapping coroutine implementation like gevent, or a
syntactic preprocessor that inserts yield or yield-from operations
(I've heard from people who do this), or you'd need a clairvoyant
scheduler that would know which callbacks won't block.

I like the C# solution, but it depends on static typing so a compiler
can know when to emit the coroutine interactions. That wouldn't work
in Python, unless you made the compiler recognizing the wait_for()
operation by name, which feels unsavory (although we do it for super()
:-).

I guess for extreme interop, callbacks that never block is your only
option anyway, but I'd be sad if we had to to recommend this as the
preferred paradigm, or claim that it is all you need.

--Guido (if I don't respond to this thread for the next two weeks,
it's because I'm on vacation :-)


On Sat, Dec 21, 2013 at 5:37 PM, Ben Darnell <ben at bendarnell.com> wrote:
> On Sat, Dec 21, 2013 at 7:26 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk <mikhtonyuk at gmail.com>
>> wrote:
>> > Indeed there is a lot of overlap between asyncio and concurrent.futures
>> > packages, so it would be very interesting to hear your overall thoughts
>> > on
>> > role/future of concurrent package. Do you consider it rudimentary and
>> > replaceable by asyncio.Future completely?
>>
>> They don't really compare. concurrent.futures is about *threads*.
>> asyncio.Future is about *avoiding* threads in favor of more
>> lightweight "tasks" and "coroutines", which in turn are built on top
>> of lower-level callbacks.
>
>
> concurrent.futures.ThreadPoolExecutor is about threads; the Future class
> itself is broader.  When I integrated Futures into Tornado I used
> concurrent.futures.Future directly (when available).  asyncio.Future is just
> an optimized version of c.f.Future (the optimization comes from assuming
> single-threaded usage).  There should at least be a common ABC between them.
>
>>
>>
>> (While I want to get away from callbacks as a programming paradigm,
>> asyncio uses them at the lower levels both because they are a logical
>> low-level building block and for interoperability with other
>> frameworks like Tornado and Twisted.)
>>
>> > I think the question is even not much about which is your preferred
>> > implementation, but rather do we want having futures as stand-alone
>> > package
>> > or not. Do you see all these implementations converging in future?
>>
>> I see them as not even competing. They use different paradigms and
>> apply to different use cases.
>>
>> > To me Future is a very simple and self-contained primitive, independent
>> > of
>> > thread pools, processes, IO, and networking.
>>
>> (Agreed on the I/O and networking part only.)
>>
>> > One thing concurrent.futures
>> > package does a very good job at is defining an isolated namespace for
>> > futures, stressing out this clear boundary (let?s disregard here
>> > ThreadPoolExecutor and ProcessPoolExecutor classes which for some reason
>> > ended up in it too).
>>
>> Actually the executor API is an important and integral part of that
>> package, and threads underlie everything you can do with its Futures.
>>
>> > So when I think of schedulers and event loops implementations I see them
>> > as
>> > ones that build on top of the Future primitive, not providing it as part
>> > of
>> > their implementation. What I think is important is that having unified
>> > Future class simplifies interoperability between different kinds of
>> > schedulers, not necessarily associated with asyncio event loops (process
>> > pools for example).
>>
>> The interoperability is completely missing. I can't tell if you've
>> used asyncio at all, but the important operation of *waiting* for a
>> result is fundamentally different there than in concurrent.futures. In
>> the latter, you just write "x = f.result()" and your thread blocks
>> until the result is available. In asyncio, you have to write "x =
>> yield from f.result()" which is a coroutine block that lets other
>> tasks run in the same thread. ("yield from" in this case is how Python
>> spells the operation that C# calls "await").
>
>
> The way I see it, the fundamental operation on Futures is add_done_callback.
> We then have various higher-level operations that let us get away from using
> callbacks directly.  One of these happens to be a method on Future: the
> blocking mode of Future.result().  Another is implemented in asyncio and
> Tornado, in the ability to "yield" a Future.  The blocking mode of result()
> is just a convenience; if asyncio-style futures had existed first then we
> could instead have a function like "x = concurrent.futures.wait_for(f)".  In
> fact, you could write this wait_for function today in a way that works for
> both concurrent and asyncio futures.
>
> This is already interoperable:  Tornado's Resolver interface
> (https://github.com/facebook/tornado/blob/master/tornado/netutil.py#L184)
> returns a Future, which may be generated by a ThreadPoolExecutor or an
> asynchronous wrapper around pycares or twisted.  In the other direction I've
> worked on hybrid apps that have one Tornado thread alongside a bunch of
> Django threads; in these apps it would work to have a Django thread block on
> f.result() for a Future returned by Tornado's AsyncHTTPClient.
>
> -Ben
>
>>
>> > Futures are definitely not the final solution for concurrency problem
>> > but
>> > rather a well-established utility for representing async operations. It
>> > is a
>> > job of higher layer systems to provide more convenient ways for dealing
>> > with
>> > asyncs (yield from coroutines, async/await rewrites etc.),
>>
>> But unless you are proposing some kind of radical change to add
>> compile-time type checking/inference to Python, the rewrite option is
>> unavailable in Python.
>>
>> > so I would not
>> > say that futures encourage callback-style programming,
>>
>> The concurrent.futures.Future class does not. But unless I misread
>> your proposal, your extensions do.
>>
>> > it?s simply a
>> > lower-layer functionality. On the contrary, monadic
>>
>> (Say that word one more time and everyone tunes out. :-)
>>
>> > methods for futures
>> > composition (e.g. map(), all(), first() etc.) ensure that no errors
>> > would be
>> > lost in the process, so I think they would complement yield from model
>> > quite
>> > nicely by hiding complexity of state maintenance from user and reducing
>> > the
>> > number of back-and-forth communications between event loop and
>> > coroutines.
>>
>> I'm not sure I follow. Again, I'm not sure if you've actually written
>> any code using asyncio.
>>
>> TBH I've written a fair number of example programs for asyncio and
>> I've very rarely felt the need for these composition functions. The
>> main composition primitive I tend to use is "yield from".
>>
>> > Besides Futures, reactive programming
>>
>> What *is* reactive programming? If you're talking about
>> http://en.wikipedia.org/wiki/Reactive_programming,
>> I'm not sure that it maps well to Python.
>>
>> > has more utilities to offer, such as
>> > Observables (representing asynchronous streams of values). It is also a
>> > very
>> > useful abstraction with a rich set of composition strategies (merging,
>> > concatenation, grouping), and may deserve its place in separate package.
>>
>> It all sounds very abstract and academic. :-)
>>
>> > Hope to hear back from you to get better picture on overall design
>> > direction
>> > here before jumping to implementation details.
>> >
>> > Brief follow-up to your questions:
>> >  - Idea behind the Future/Promise separation is to draw a clean line
>> > between
>> > client-facing and scheduler-side APIs respectively. Futures should not
>> > expose any completion methods, which clients should not call anyway.
>>
>> Given that Future and Promise are often used synonymously, using them
>> to make this distinction sounds confusing. I agree that Futures have
>> two different APIs, one for the consumer and another for the producer.
>> But I'm not sure that it's necessary to separate them more strictly --
>> convention seems good enough to me here. (It's the same with many
>> communication primitives, like queues and even threads.)
>>
>> (The one thing that trips people up frequently is that, while
>> set_result() and set_exception() are producer APIs, cancel() is a
>> consumer API, and the cancellation signal travels from the consumer to
>> the producer.)
>>
>> >  - Completely agree with you that Twisted-style callbacks are evil and
>> > it is
>> > better to have single code path for getting result or raising exception
>> >  - Sorry for bad grammar in the proposal, it?s an early draft written in
>> > 3
>> > AM, so I will definitely improve on it if we decide to move forward.
>>
>> No problem!
>>
>> > Thanks,
>> > Sergii
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>



-- 
--Guido van Rossum (python.org/~guido)

From rosuav at gmail.com  Sun Dec 22 04:53:58 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 22 Dec 2013 14:53:58 +1100
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
Message-ID: <CAPTjJmp6RBD=WJRTah+qCrP9T8dSRF1TOJyG0jW6Ayk69yEVzg@mail.gmail.com>

On Sun, Dec 22, 2013 at 2:45 PM, Guido van Rossum <guido at python.org> wrote:
> To cut through that you'd have to have
> some kind of stack-swapping coroutine implementation like gevent...

Forgive the stupid question, but how is stack-swapping during blocking
calls materially different from threads?

ChrisA

From guido at python.org  Sun Dec 22 05:01:26 2013
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Dec 2013 20:01:26 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAPTjJmp6RBD=WJRTah+qCrP9T8dSRF1TOJyG0jW6Ayk69yEVzg@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAPTjJmp6RBD=WJRTah+qCrP9T8dSRF1TOJyG0jW6Ayk69yEVzg@mail.gmail.com>
Message-ID: <CAP7+vJKCJxDSaKw_Pb8Yzse-jmG16HGS2DebV_jzNnj=9Vru2g@mail.gmail.com>

On Sat, Dec 21, 2013 at 7:53 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Dec 22, 2013 at 2:45 PM, Guido van Rossum <guido at python.org> wrote:
>> To cut through that you'd have to have
>> some kind of stack-swapping coroutine implementation like gevent...
>
> Forgive the stupid question, but how is stack-swapping during blocking
> calls materially different from threads?

It's also known as "green threads". The gevent folks and the Stackless
folks (and a few others) do this and claim it is vastly superior to OS
threads. I believe the main difference is that an OS thread takes up a
relatively large amount of resources in the kernel as well as in user
space (for the stack) while a green thread takes up a comparatively
much smaller amount of space, all in user space -- with the result
that you can have many more green threads than you could have OS
threads, and switching between them will be much faster. The price you
pay is that the kernel doesn't know what you're doing and you have to
intercept all system-call-level I/O to make it non-blocking -- if you
accidentally make a blocking syscall, no other green thread will run.

This may sound like a pure implementation-level distinction, but
implementation is what makes things practical (otherwise we'd all be
using Turing machines or lambda calculus :-).

-- 
--Guido van Rossum (python.org/~guido)

From rosuav at gmail.com  Sun Dec 22 05:25:07 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 22 Dec 2013 15:25:07 +1100
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAP7+vJKCJxDSaKw_Pb8Yzse-jmG16HGS2DebV_jzNnj=9Vru2g@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAPTjJmp6RBD=WJRTah+qCrP9T8dSRF1TOJyG0jW6Ayk69yEVzg@mail.gmail.com>
 <CAP7+vJKCJxDSaKw_Pb8Yzse-jmG16HGS2DebV_jzNnj=9Vru2g@mail.gmail.com>
Message-ID: <CAPTjJmps+QZ3uRuw2AQG0TXy4JKHLOP2SOZdLPmgb67NzgwT6g@mail.gmail.com>

On Sun, Dec 22, 2013 at 3:01 PM, Guido van Rossum <guido at python.org> wrote:
> On Sat, Dec 21, 2013 at 7:53 PM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sun, Dec 22, 2013 at 2:45 PM, Guido van Rossum <guido at python.org> wrote:
>>> To cut through that you'd have to have
>>> some kind of stack-swapping coroutine implementation like gevent...
>>
>> Forgive the stupid question, but how is stack-swapping during blocking
>> calls materially different from threads?
>
> It's also known as "green threads". The gevent folks and the Stackless
> folks (and a few others) do this and claim it is vastly superior to OS
> threads.

Gotcha. I grew up on OS/2 where the threading was lean and mean, so I
just used it. I'd be mildly curious to know how different
implementations of threads compare, and how many of them actually
warrant a "lighter-weight thread" feature like this, but for something
that aims to be cross-platform, I can see the value in doing it.

ChrisA

From abarnert at yahoo.com  Sun Dec 22 06:41:07 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 21 Dec 2013 21:41:07 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
Message-ID: <45EAF7BB-9895-472B-8E6F-5437DB05A474@yahoo.com>

A few things I'd like to see.

Your Future uses a condition variable. That could be a big hit for single threaded uses, and since one of your goals is to make asyncio use the same futures as threaded executors, that might not be acceptable. How hard would it be to allow passing a class (or other factory callable) in place of the default condition when constructing a Promise, or just a "threaded=True" flag. (I realize this is more complicated than it sounds--an event loop can always push callbacks onto a thread pool, or just use a thread pool to implement something that's hard to do in a single thread, like DNS lookup--so you probably also need a way to upgrade a single threaded promise to a thread safe one.)

There's a lot of rationale about why separate futures and promises are important, but no rationale for the specific design. If you explained where you deviated from twisted Deferred, JS Promises/A, etc., and why, it would be a lot easier to evaluate your design.

It's not documented how chaining multiple callbacks in sequence works. Does the second callback get the return value of the first one, or the original one? Can you return a new future (possibly with callbacks already attached, possibly already completed)?

Sent from a random iPhone

On Dec 21, 2013, at 5:14, Sergii Mikhtoniuk <mikhtonyuk at gmail.com> wrote:

> Hi all,
> 
> I would very much appreciate your opinion on my proposal for improvement of concurrent.futures package.
> 
> Comparing to other languages such as Scala and C#, Python?s futures significantly fall behind in functionality especially in ability to chain computations and compose different futures without blocking and waiting for result. New packages continue to emerge (asyncio) which provide their own futures implementation, making composition even more difficult.
> 
> Proposed improvement implements Scala-like Future as a monadic construct. It allows performing multiple kinds of operations on Future?s result without blocking, enabling reactive programming in Python. It implements common pattern separating Future and Promise interface, making it very easy for 3rd party systems to use futures in their API.
> 
> Please have a look at this PEP draft, and reference implementation (as separate library).
> 
> I?m very interested in:
> - How PEPable is this?
> - What are your thoughts on backward compatibility (current implementation does not sacrifice any design points for it, but better compatibility can be achieved)?
> - Thoughts on Future-based APIs in other packages?
> 
> Thanks,
> Sergii
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131221/dfab5294/attachment.html>

From abarnert at yahoo.com  Sun Dec 22 06:50:36 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 21 Dec 2013 21:50:36 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAPTjJmps+QZ3uRuw2AQG0TXy4JKHLOP2SOZdLPmgb67NzgwT6g@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAPTjJmp6RBD=WJRTah+qCrP9T8dSRF1TOJyG0jW6Ayk69yEVzg@mail.gmail.com>
 <CAP7+vJKCJxDSaKw_Pb8Yzse-jmG16HGS2DebV_jzNnj=9Vru2g@mail.gmail.com>
 <CAPTjJmps+QZ3uRuw2AQG0TXy4JKHLOP2SOZdLPmgb67NzgwT6g@mail.gmail.com>
Message-ID: <39F3FE04-2E72-43DC-83E4-D67805AE442A@yahoo.com>

Even OS/2 can't do thousands of threads, so if you want to write a server with one thread per client (or two) you'd still need green threads.

Meanwhile, Windows, Linux, and OS X all have pretty fast thread startup and decent schedulers, but the need for a static-sized stack still means you can't do thousands even on today's computers--especially in 32 bit land (which is depressingly still common on Windows).

Sent from a random iPhone

On Dec 21, 2013, at 20:25, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Dec 22, 2013 at 3:01 PM, Guido van Rossum <guido at python.org> wrote:
>> On Sat, Dec 21, 2013 at 7:53 PM, Chris Angelico <rosuav at gmail.com> wrote:
>>> On Sun, Dec 22, 2013 at 2:45 PM, Guido van Rossum <guido at python.org> wrote:
>>>> To cut through that you'd have to have
>>>> some kind of stack-swapping coroutine implementation like gevent...
>>> 
>>> Forgive the stupid question, but how is stack-swapping during blocking
>>> calls materially different from threads?
>> 
>> It's also known as "green threads". The gevent folks and the Stackless
>> folks (and a few others) do this and claim it is vastly superior to OS
>> threads.
> 
> Gotcha. I grew up on OS/2 where the threading was lean and mean, so I
> just used it. I'd be mildly curious to know how different
> implementations of threads compare, and how many of them actually
> warrant a "lighter-weight thread" feature like this, but for something
> that aims to be cross-platform, I can see the value in doing it.
> 
> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From solipsis at pitrou.net  Sun Dec 22 15:54:15 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Dec 2013 15:54:15 +0100
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAPTjJmp6RBD=WJRTah+qCrP9T8dSRF1TOJyG0jW6Ayk69yEVzg@mail.gmail.com>
 <CAP7+vJKCJxDSaKw_Pb8Yzse-jmG16HGS2DebV_jzNnj=9Vru2g@mail.gmail.com>
Message-ID: <20131222155415.5b8d417e@fsol>

On Sat, 21 Dec 2013 20:01:26 -0800
Guido van Rossum <guido at python.org> wrote:

> On Sat, Dec 21, 2013 at 7:53 PM, Chris Angelico <rosuav at gmail.com> wrote:
> > On Sun, Dec 22, 2013 at 2:45 PM, Guido van Rossum <guido at python.org> wrote:
> >> To cut through that you'd have to have
> >> some kind of stack-swapping coroutine implementation like gevent...
> >
> > Forgive the stupid question, but how is stack-swapping during blocking
> > calls materially different from threads?
> 
> It's also known as "green threads". The gevent folks and the Stackless
> folks (and a few others) do this and claim it is vastly superior to OS
> threads. I believe the main difference is that an OS thread takes up a
> relatively large amount of resources in the kernel as well as in user
> space (for the stack) while a green thread takes up a comparatively
> much smaller amount of space, all in user space -- with the result
> that you can have many more green threads than you could have OS
> threads, and switching between them will be much faster.

Also, in Python 2.x, it avoids the contention cost of the GIL with many
OS threads (cf. the Dave Beazley benchmark :-)).

Regards

Antoine.



From ben at bendarnell.com  Sun Dec 22 23:29:50 2013
From: ben at bendarnell.com (Ben Darnell)
Date: Sun, 22 Dec 2013 17:29:50 -0500
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
Message-ID: <CAFkYKJ7wt7q3N9-RNam1XtLa9UEmakO3OvT-2xVo5mauZBsdzA@mail.gmail.com>

On Sat, Dec 21, 2013 at 10:45 PM, Guido van Rossum <guido at python.org> wrote:

> There's still the issue that in the threading version, you wait for a
> Future by blocking the current thread, while in the asyncio version,
> you must use "yield from" to block. For interoperability you would
> have to refrain from *any* blocking operations (including "yield
> from") so you would only be able to use callbacks. But whether you had
> to write "x = f.result()" or "x = concurrent.futures.wait_for(f)",
> either way you'd implicitly be blocking the current thread.
>

Threaded *consumers* of Futures wait for them by blocking, while
asynchronous consumers wait for them by yielding.  It doesn't matter
whether the *producer* of the Future is threaded or asynchronous (except
that if you know you won't be using threads you can use a faster
thread-unsafe Future implementation).

-Ben


>
> Yes, a clever scheduler could run other callbacks while blocking, but
> that's not a complete solution, because another callback might do a
> similar blocking operation, and whatever that waited for could hold up
> the earlier blocking operation, causing an ever-deeper recursion and
> of event loop invocations that might never complete. (I've had to
> debug this in production code.) To cut through that you'd have to have
> some kind of stack-swapping coroutine implementation like gevent, or a
> syntactic preprocessor that inserts yield or yield-from operations
> (I've heard from people who do this), or you'd need a clairvoyant
> scheduler that would know which callbacks won't block.
>
> I like the C# solution, but it depends on static typing so a compiler
> can know when to emit the coroutine interactions. That wouldn't work
> in Python, unless you made the compiler recognizing the wait_for()
> operation by name, which feels unsavory (although we do it for super()
> :-).
>
> I guess for extreme interop, callbacks that never block is your only
> option anyway, but I'd be sad if we had to to recommend this as the
> preferred paradigm, or claim that it is all you need.
>
> --Guido (if I don't respond to this thread for the next two weeks,
> it's because I'm on vacation :-)
>
>
> On Sat, Dec 21, 2013 at 5:37 PM, Ben Darnell <ben at bendarnell.com> wrote:
> > On Sat, Dec 21, 2013 at 7:26 PM, Guido van Rossum <guido at python.org>
> wrote:
> >>
> >> On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk <
> mikhtonyuk at gmail.com>
> >> wrote:
> >> > Indeed there is a lot of overlap between asyncio and
> concurrent.futures
> >> > packages, so it would be very interesting to hear your overall
> thoughts
> >> > on
> >> > role/future of concurrent package. Do you consider it rudimentary and
> >> > replaceable by asyncio.Future completely?
> >>
> >> They don't really compare. concurrent.futures is about *threads*.
> >> asyncio.Future is about *avoiding* threads in favor of more
> >> lightweight "tasks" and "coroutines", which in turn are built on top
> >> of lower-level callbacks.
> >
> >
> > concurrent.futures.ThreadPoolExecutor is about threads; the Future class
> > itself is broader.  When I integrated Futures into Tornado I used
> > concurrent.futures.Future directly (when available).  asyncio.Future is
> just
> > an optimized version of c.f.Future (the optimization comes from assuming
> > single-threaded usage).  There should at least be a common ABC between
> them.
> >
> >>
> >>
> >> (While I want to get away from callbacks as a programming paradigm,
> >> asyncio uses them at the lower levels both because they are a logical
> >> low-level building block and for interoperability with other
> >> frameworks like Tornado and Twisted.)
> >>
> >> > I think the question is even not much about which is your preferred
> >> > implementation, but rather do we want having futures as stand-alone
> >> > package
> >> > or not. Do you see all these implementations converging in future?
> >>
> >> I see them as not even competing. They use different paradigms and
> >> apply to different use cases.
> >>
> >> > To me Future is a very simple and self-contained primitive,
> independent
> >> > of
> >> > thread pools, processes, IO, and networking.
> >>
> >> (Agreed on the I/O and networking part only.)
> >>
> >> > One thing concurrent.futures
> >> > package does a very good job at is defining an isolated namespace for
> >> > futures, stressing out this clear boundary (let?s disregard here
> >> > ThreadPoolExecutor and ProcessPoolExecutor classes which for some
> reason
> >> > ended up in it too).
> >>
> >> Actually the executor API is an important and integral part of that
> >> package, and threads underlie everything you can do with its Futures.
> >>
> >> > So when I think of schedulers and event loops implementations I see
> them
> >> > as
> >> > ones that build on top of the Future primitive, not providing it as
> part
> >> > of
> >> > their implementation. What I think is important is that having unified
> >> > Future class simplifies interoperability between different kinds of
> >> > schedulers, not necessarily associated with asyncio event loops
> (process
> >> > pools for example).
> >>
> >> The interoperability is completely missing. I can't tell if you've
> >> used asyncio at all, but the important operation of *waiting* for a
> >> result is fundamentally different there than in concurrent.futures. In
> >> the latter, you just write "x = f.result()" and your thread blocks
> >> until the result is available. In asyncio, you have to write "x =
> >> yield from f.result()" which is a coroutine block that lets other
> >> tasks run in the same thread. ("yield from" in this case is how Python
> >> spells the operation that C# calls "await").
> >
> >
> > The way I see it, the fundamental operation on Futures is
> add_done_callback.
> > We then have various higher-level operations that let us get away from
> using
> > callbacks directly.  One of these happens to be a method on Future: the
> > blocking mode of Future.result().  Another is implemented in asyncio and
> > Tornado, in the ability to "yield" a Future.  The blocking mode of
> result()
> > is just a convenience; if asyncio-style futures had existed first then we
> > could instead have a function like "x = concurrent.futures.wait_for(f)".
>  In
> > fact, you could write this wait_for function today in a way that works
> for
> > both concurrent and asyncio futures.
> >
> > This is already interoperable:  Tornado's Resolver interface
> > (https://github.com/facebook/tornado/blob/master/tornado/netutil.py#L184
> )
> > returns a Future, which may be generated by a ThreadPoolExecutor or an
> > asynchronous wrapper around pycares or twisted.  In the other direction
> I've
> > worked on hybrid apps that have one Tornado thread alongside a bunch of
> > Django threads; in these apps it would work to have a Django thread
> block on
> > f.result() for a Future returned by Tornado's AsyncHTTPClient.
> >
> > -Ben
> >
> >>
> >> > Futures are definitely not the final solution for concurrency problem
> >> > but
> >> > rather a well-established utility for representing async operations.
> It
> >> > is a
> >> > job of higher layer systems to provide more convenient ways for
> dealing
> >> > with
> >> > asyncs (yield from coroutines, async/await rewrites etc.),
> >>
> >> But unless you are proposing some kind of radical change to add
> >> compile-time type checking/inference to Python, the rewrite option is
> >> unavailable in Python.
> >>
> >> > so I would not
> >> > say that futures encourage callback-style programming,
> >>
> >> The concurrent.futures.Future class does not. But unless I misread
> >> your proposal, your extensions do.
> >>
> >> > it?s simply a
> >> > lower-layer functionality. On the contrary, monadic
> >>
> >> (Say that word one more time and everyone tunes out. :-)
> >>
> >> > methods for futures
> >> > composition (e.g. map(), all(), first() etc.) ensure that no errors
> >> > would be
> >> > lost in the process, so I think they would complement yield from model
> >> > quite
> >> > nicely by hiding complexity of state maintenance from user and
> reducing
> >> > the
> >> > number of back-and-forth communications between event loop and
> >> > coroutines.
> >>
> >> I'm not sure I follow. Again, I'm not sure if you've actually written
> >> any code using asyncio.
> >>
> >> TBH I've written a fair number of example programs for asyncio and
> >> I've very rarely felt the need for these composition functions. The
> >> main composition primitive I tend to use is "yield from".
> >>
> >> > Besides Futures, reactive programming
> >>
> >> What *is* reactive programming? If you're talking about
> >> http://en.wikipedia.org/wiki/Reactive_programming,
> >> I'm not sure that it maps well to Python.
> >>
> >> > has more utilities to offer, such as
> >> > Observables (representing asynchronous streams of values). It is also
> a
> >> > very
> >> > useful abstraction with a rich set of composition strategies (merging,
> >> > concatenation, grouping), and may deserve its place in separate
> package.
> >>
> >> It all sounds very abstract and academic. :-)
> >>
> >> > Hope to hear back from you to get better picture on overall design
> >> > direction
> >> > here before jumping to implementation details.
> >> >
> >> > Brief follow-up to your questions:
> >> >  - Idea behind the Future/Promise separation is to draw a clean line
> >> > between
> >> > client-facing and scheduler-side APIs respectively. Futures should not
> >> > expose any completion methods, which clients should not call anyway.
> >>
> >> Given that Future and Promise are often used synonymously, using them
> >> to make this distinction sounds confusing. I agree that Futures have
> >> two different APIs, one for the consumer and another for the producer.
> >> But I'm not sure that it's necessary to separate them more strictly --
> >> convention seems good enough to me here. (It's the same with many
> >> communication primitives, like queues and even threads.)
> >>
> >> (The one thing that trips people up frequently is that, while
> >> set_result() and set_exception() are producer APIs, cancel() is a
> >> consumer API, and the cancellation signal travels from the consumer to
> >> the producer.)
> >>
> >> >  - Completely agree with you that Twisted-style callbacks are evil and
> >> > it is
> >> > better to have single code path for getting result or raising
> exception
> >> >  - Sorry for bad grammar in the proposal, it?s an early draft written
> in
> >> > 3
> >> > AM, so I will definitely improve on it if we decide to move forward.
> >>
> >> No problem!
> >>
> >> > Thanks,
> >> > Sergii
> >>
> >> --
> >> --Guido van Rossum (python.org/~guido)
> >> _______________________________________________
> >> Python-ideas mailing list
> >> Python-ideas at python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131222/0710f474/attachment-0001.html>

From guido at python.org  Sun Dec 22 23:42:15 2013
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Dec 2013 14:42:15 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAFkYKJ7wt7q3N9-RNam1XtLa9UEmakO3OvT-2xVo5mauZBsdzA@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAFkYKJ7wt7q3N9-RNam1XtLa9UEmakO3OvT-2xVo5mauZBsdzA@mail.gmail.com>
Message-ID: <CAP7+vJJ4LpyPnejY1CjRpVoX0WKj9WOrTc+drfvanah-QW1-yQ@mail.gmail.com>

Aha. That is clever. I will have to look into the details more, but the
idea is promising. Sorry I didn't see tty his before.
On Dec 22, 2013 12:30 PM, "Ben Darnell" <ben at bendarnell.com> wrote:

> On Sat, Dec 21, 2013 at 10:45 PM, Guido van Rossum <guido at python.org>wrote:
>
>> There's still the issue that in the threading version, you wait for a
>> Future by blocking the current thread, while in the asyncio version,
>> you must use "yield from" to block. For interoperability you would
>> have to refrain from *any* blocking operations (including "yield
>> from") so you would only be able to use callbacks. But whether you had
>> to write "x = f.result()" or "x = concurrent.futures.wait_for(f)",
>> either way you'd implicitly be blocking the current thread.
>>
>
> Threaded *consumers* of Futures wait for them by blocking, while
> asynchronous consumers wait for them by yielding.  It doesn't matter
> whether the *producer* of the Future is threaded or asynchronous (except
> that if you know you won't be using threads you can use a faster
> thread-unsafe Future implementation).
>
> -Ben
>
>
>>
>> Yes, a clever scheduler could run other callbacks while blocking, but
>> that's not a complete solution, because another callback might do a
>> similar blocking operation, and whatever that waited for could hold up
>> the earlier blocking operation, causing an ever-deeper recursion and
>> of event loop invocations that might never complete. (I've had to
>> debug this in production code.) To cut through that you'd have to have
>> some kind of stack-swapping coroutine implementation like gevent, or a
>> syntactic preprocessor that inserts yield or yield-from operations
>> (I've heard from people who do this), or you'd need a clairvoyant
>> scheduler that would know which callbacks won't block.
>>
>> I like the C# solution, but it depends on static typing so a compiler
>> can know when to emit the coroutine interactions. That wouldn't work
>> in Python, unless you made the compiler recognizing the wait_for()
>> operation by name, which feels unsavory (although we do it for super()
>> :-).
>>
>> I guess for extreme interop, callbacks that never block is your only
>> option anyway, but I'd be sad if we had to to recommend this as the
>> preferred paradigm, or claim that it is all you need.
>>
>> --Guido (if I don't respond to this thread for the next two weeks,
>> it's because I'm on vacation :-)
>>
>>
>> On Sat, Dec 21, 2013 at 5:37 PM, Ben Darnell <ben at bendarnell.com> wrote:
>> > On Sat, Dec 21, 2013 at 7:26 PM, Guido van Rossum <guido at python.org>
>> wrote:
>> >>
>> >> On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk <
>> mikhtonyuk at gmail.com>
>> >> wrote:
>> >> > Indeed there is a lot of overlap between asyncio and
>> concurrent.futures
>> >> > packages, so it would be very interesting to hear your overall
>> thoughts
>> >> > on
>> >> > role/future of concurrent package. Do you consider it rudimentary and
>> >> > replaceable by asyncio.Future completely?
>> >>
>> >> They don't really compare. concurrent.futures is about *threads*.
>> >> asyncio.Future is about *avoiding* threads in favor of more
>> >> lightweight "tasks" and "coroutines", which in turn are built on top
>> >> of lower-level callbacks.
>> >
>> >
>> > concurrent.futures.ThreadPoolExecutor is about threads; the Future class
>> > itself is broader.  When I integrated Futures into Tornado I used
>> > concurrent.futures.Future directly (when available).  asyncio.Future is
>> just
>> > an optimized version of c.f.Future (the optimization comes from assuming
>> > single-threaded usage).  There should at least be a common ABC between
>> them.
>> >
>> >>
>> >>
>> >> (While I want to get away from callbacks as a programming paradigm,
>> >> asyncio uses them at the lower levels both because they are a logical
>> >> low-level building block and for interoperability with other
>> >> frameworks like Tornado and Twisted.)
>> >>
>> >> > I think the question is even not much about which is your preferred
>> >> > implementation, but rather do we want having futures as stand-alone
>> >> > package
>> >> > or not. Do you see all these implementations converging in future?
>> >>
>> >> I see them as not even competing. They use different paradigms and
>> >> apply to different use cases.
>> >>
>> >> > To me Future is a very simple and self-contained primitive,
>> independent
>> >> > of
>> >> > thread pools, processes, IO, and networking.
>> >>
>> >> (Agreed on the I/O and networking part only.)
>> >>
>> >> > One thing concurrent.futures
>> >> > package does a very good job at is defining an isolated namespace for
>> >> > futures, stressing out this clear boundary (let?s disregard here
>> >> > ThreadPoolExecutor and ProcessPoolExecutor classes which for some
>> reason
>> >> > ended up in it too).
>> >>
>> >> Actually the executor API is an important and integral part of that
>> >> package, and threads underlie everything you can do with its Futures.
>> >>
>> >> > So when I think of schedulers and event loops implementations I see
>> them
>> >> > as
>> >> > ones that build on top of the Future primitive, not providing it as
>> part
>> >> > of
>> >> > their implementation. What I think is important is that having
>> unified
>> >> > Future class simplifies interoperability between different kinds of
>> >> > schedulers, not necessarily associated with asyncio event loops
>> (process
>> >> > pools for example).
>> >>
>> >> The interoperability is completely missing. I can't tell if you've
>> >> used asyncio at all, but the important operation of *waiting* for a
>> >> result is fundamentally different there than in concurrent.futures. In
>> >> the latter, you just write "x = f.result()" and your thread blocks
>> >> until the result is available. In asyncio, you have to write "x =
>> >> yield from f.result()" which is a coroutine block that lets other
>> >> tasks run in the same thread. ("yield from" in this case is how Python
>> >> spells the operation that C# calls "await").
>> >
>> >
>> > The way I see it, the fundamental operation on Futures is
>> add_done_callback.
>> > We then have various higher-level operations that let us get away from
>> using
>> > callbacks directly.  One of these happens to be a method on Future: the
>> > blocking mode of Future.result().  Another is implemented in asyncio and
>> > Tornado, in the ability to "yield" a Future.  The blocking mode of
>> result()
>> > is just a convenience; if asyncio-style futures had existed first then
>> we
>> > could instead have a function like "x =
>> concurrent.futures.wait_for(f)".  In
>> > fact, you could write this wait_for function today in a way that works
>> for
>> > both concurrent and asyncio futures.
>> >
>> > This is already interoperable:  Tornado's Resolver interface
>> > (
>> https://github.com/facebook/tornado/blob/master/tornado/netutil.py#L184)
>> > returns a Future, which may be generated by a ThreadPoolExecutor or an
>> > asynchronous wrapper around pycares or twisted.  In the other direction
>> I've
>> > worked on hybrid apps that have one Tornado thread alongside a bunch of
>> > Django threads; in these apps it would work to have a Django thread
>> block on
>> > f.result() for a Future returned by Tornado's AsyncHTTPClient.
>> >
>> > -Ben
>> >
>> >>
>> >> > Futures are definitely not the final solution for concurrency problem
>> >> > but
>> >> > rather a well-established utility for representing async operations.
>> It
>> >> > is a
>> >> > job of higher layer systems to provide more convenient ways for
>> dealing
>> >> > with
>> >> > asyncs (yield from coroutines, async/await rewrites etc.),
>> >>
>> >> But unless you are proposing some kind of radical change to add
>> >> compile-time type checking/inference to Python, the rewrite option is
>> >> unavailable in Python.
>> >>
>> >> > so I would not
>> >> > say that futures encourage callback-style programming,
>> >>
>> >> The concurrent.futures.Future class does not. But unless I misread
>> >> your proposal, your extensions do.
>> >>
>> >> > it?s simply a
>> >> > lower-layer functionality. On the contrary, monadic
>> >>
>> >> (Say that word one more time and everyone tunes out. :-)
>> >>
>> >> > methods for futures
>> >> > composition (e.g. map(), all(), first() etc.) ensure that no errors
>> >> > would be
>> >> > lost in the process, so I think they would complement yield from
>> model
>> >> > quite
>> >> > nicely by hiding complexity of state maintenance from user and
>> reducing
>> >> > the
>> >> > number of back-and-forth communications between event loop and
>> >> > coroutines.
>> >>
>> >> I'm not sure I follow. Again, I'm not sure if you've actually written
>> >> any code using asyncio.
>> >>
>> >> TBH I've written a fair number of example programs for asyncio and
>> >> I've very rarely felt the need for these composition functions. The
>> >> main composition primitive I tend to use is "yield from".
>> >>
>> >> > Besides Futures, reactive programming
>> >>
>> >> What *is* reactive programming? If you're talking about
>> >> http://en.wikipedia.org/wiki/Reactive_programming,
>> >> I'm not sure that it maps well to Python.
>> >>
>> >> > has more utilities to offer, such as
>> >> > Observables (representing asynchronous streams of values). It is
>> also a
>> >> > very
>> >> > useful abstraction with a rich set of composition strategies
>> (merging,
>> >> > concatenation, grouping), and may deserve its place in separate
>> package.
>> >>
>> >> It all sounds very abstract and academic. :-)
>> >>
>> >> > Hope to hear back from you to get better picture on overall design
>> >> > direction
>> >> > here before jumping to implementation details.
>> >> >
>> >> > Brief follow-up to your questions:
>> >> >  - Idea behind the Future/Promise separation is to draw a clean line
>> >> > between
>> >> > client-facing and scheduler-side APIs respectively. Futures should
>> not
>> >> > expose any completion methods, which clients should not call anyway.
>> >>
>> >> Given that Future and Promise are often used synonymously, using them
>> >> to make this distinction sounds confusing. I agree that Futures have
>> >> two different APIs, one for the consumer and another for the producer.
>> >> But I'm not sure that it's necessary to separate them more strictly --
>> >> convention seems good enough to me here. (It's the same with many
>> >> communication primitives, like queues and even threads.)
>> >>
>> >> (The one thing that trips people up frequently is that, while
>> >> set_result() and set_exception() are producer APIs, cancel() is a
>> >> consumer API, and the cancellation signal travels from the consumer to
>> >> the producer.)
>> >>
>> >> >  - Completely agree with you that Twisted-style callbacks are evil
>> and
>> >> > it is
>> >> > better to have single code path for getting result or raising
>> exception
>> >> >  - Sorry for bad grammar in the proposal, it?s an early draft
>> written in
>> >> > 3
>> >> > AM, so I will definitely improve on it if we decide to move forward.
>> >>
>> >> No problem!
>> >>
>> >> > Thanks,
>> >> > Sergii
>> >>
>> >> --
>> >> --Guido van Rossum (python.org/~guido)
>> >> _______________________________________________
>> >> Python-ideas mailing list
>> >> Python-ideas at python.org
>> >> https://mail.python.org/mailman/listinfo/python-ideas
>> >> Code of Conduct: http://python.org/psf/codeofconduct/
>> >
>> >
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131222/6671ebad/attachment-0001.html>

From mikhtonyuk at gmail.com  Tue Dec 24 17:56:23 2013
From: mikhtonyuk at gmail.com (Sergii Mikhtoniuk)
Date: Tue, 24 Dec 2013 18:56:23 +0200
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAP7+vJJ4LpyPnejY1CjRpVoX0WKj9WOrTc+drfvanah-QW1-yQ@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAFkYKJ7wt7q3N9-RNam1XtLa9UEmakO3OvT-2xVo5mauZBsdzA@mail.gmail.com>
 <CAP7+vJJ4LpyPnejY1CjRpVoX0WKj9WOrTc+drfvanah-QW1-yQ@mail.gmail.com>
Message-ID: <CAPQPjQEB-eeDZKa0srsS1avMcMSWYB_rn+nzOSyKc5Hf3NfZNQ@mail.gmail.com>

Thanks everyone for your feedback.

Taking all your suggestions into account I have revised my
proposal<https://rawgithub.com/mikhtonyuk/rxpython/asyncio/pep-0000.html>
.

In short, it?s now:
- defines separate Future classes for cooperative and multithreaded cases
in concurrent.futures package
- multithreaded implementation adds thread-safety to basic implementation,
so in cooperative concurrency case there is absolutely no overhead
- cooperative future?s interface is identical to asyncio.future
- asyncio.Future inherits from concurrent.futures.cooperative.Future adding
only methods specific to `yield from`
- adds common composition methods for futures (intended to replace and
enhance asyncio.wait/gather and concurrent.futures.wait)

There?s still some work to be done for backward compatibility of
concurrent.futures.Future, but implementation is almost
ready<https://github.com/mikhtonyuk/rxpython/tree/asyncio>
.

Would really appreciate if you could take a look.


Thanks,
Sergii



On Mon, Dec 23, 2013 at 12:42 AM, Guido van Rossum <guido at python.org> wrote:

> Aha. That is clever. I will have to look into the details more, but the
> idea is promising. Sorry I didn't see tty his before.
> On Dec 22, 2013 12:30 PM, "Ben Darnell" <ben at bendarnell.com> wrote:
>
>> On Sat, Dec 21, 2013 at 10:45 PM, Guido van Rossum <guido at python.org>wrote:
>>
>>> There's still the issue that in the threading version, you wait for a
>>> Future by blocking the current thread, while in the asyncio version,
>>> you must use "yield from" to block. For interoperability you would
>>> have to refrain from *any* blocking operations (including "yield
>>> from") so you would only be able to use callbacks. But whether you had
>>> to write "x = f.result()" or "x = concurrent.futures.wait_for(f)",
>>> either way you'd implicitly be blocking the current thread.
>>>
>>
>> Threaded *consumers* of Futures wait for them by blocking, while
>> asynchronous consumers wait for them by yielding.  It doesn't matter
>> whether the *producer* of the Future is threaded or asynchronous (except
>> that if you know you won't be using threads you can use a faster
>> thread-unsafe Future implementation).
>>
>> -Ben
>>
>>
>>>
>>> Yes, a clever scheduler could run other callbacks while blocking, but
>>> that's not a complete solution, because another callback might do a
>>> similar blocking operation, and whatever that waited for could hold up
>>> the earlier blocking operation, causing an ever-deeper recursion and
>>> of event loop invocations that might never complete. (I've had to
>>> debug this in production code.) To cut through that you'd have to have
>>> some kind of stack-swapping coroutine implementation like gevent, or a
>>> syntactic preprocessor that inserts yield or yield-from operations
>>> (I've heard from people who do this), or you'd need a clairvoyant
>>> scheduler that would know which callbacks won't block.
>>>
>>> I like the C# solution, but it depends on static typing so a compiler
>>> can know when to emit the coroutine interactions. That wouldn't work
>>> in Python, unless you made the compiler recognizing the wait_for()
>>> operation by name, which feels unsavory (although we do it for super()
>>> :-).
>>>
>>> I guess for extreme interop, callbacks that never block is your only
>>> option anyway, but I'd be sad if we had to to recommend this as the
>>> preferred paradigm, or claim that it is all you need.
>>>
>>> --Guido (if I don't respond to this thread for the next two weeks,
>>> it's because I'm on vacation :-)
>>>
>>>
>>> On Sat, Dec 21, 2013 at 5:37 PM, Ben Darnell <ben at bendarnell.com> wrote:
>>> > On Sat, Dec 21, 2013 at 7:26 PM, Guido van Rossum <guido at python.org>
>>> wrote:
>>> >>
>>> >> On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk <
>>> mikhtonyuk at gmail.com>
>>> >> wrote:
>>> >> > Indeed there is a lot of overlap between asyncio and
>>> concurrent.futures
>>> >> > packages, so it would be very interesting to hear your overall
>>> thoughts
>>> >> > on
>>> >> > role/future of concurrent package. Do you consider it rudimentary
>>> and
>>> >> > replaceable by asyncio.Future completely?
>>> >>
>>> >> They don't really compare. concurrent.futures is about *threads*.
>>> >> asyncio.Future is about *avoiding* threads in favor of more
>>> >> lightweight "tasks" and "coroutines", which in turn are built on top
>>> >> of lower-level callbacks.
>>> >
>>> >
>>> > concurrent.futures.ThreadPoolExecutor is about threads; the Future
>>> class
>>> > itself is broader.  When I integrated Futures into Tornado I used
>>> > concurrent.futures.Future directly (when available).  asyncio.Future
>>> is just
>>> > an optimized version of c.f.Future (the optimization comes from
>>> assuming
>>> > single-threaded usage).  There should at least be a common ABC between
>>> them.
>>> >
>>> >>
>>> >>
>>> >> (While I want to get away from callbacks as a programming paradigm,
>>> >> asyncio uses them at the lower levels both because they are a logical
>>> >> low-level building block and for interoperability with other
>>> >> frameworks like Tornado and Twisted.)
>>> >>
>>> >> > I think the question is even not much about which is your preferred
>>> >> > implementation, but rather do we want having futures as stand-alone
>>> >> > package
>>> >> > or not. Do you see all these implementations converging in future?
>>> >>
>>> >> I see them as not even competing. They use different paradigms and
>>> >> apply to different use cases.
>>> >>
>>> >> > To me Future is a very simple and self-contained primitive,
>>> independent
>>> >> > of
>>> >> > thread pools, processes, IO, and networking.
>>> >>
>>> >> (Agreed on the I/O and networking part only.)
>>> >>
>>> >> > One thing concurrent.futures
>>> >> > package does a very good job at is defining an isolated namespace
>>> for
>>> >> > futures, stressing out this clear boundary (let?s disregard here
>>> >> > ThreadPoolExecutor and ProcessPoolExecutor classes which for some
>>> reason
>>> >> > ended up in it too).
>>> >>
>>> >> Actually the executor API is an important and integral part of that
>>> >> package, and threads underlie everything you can do with its Futures.
>>> >>
>>> >> > So when I think of schedulers and event loops implementations I see
>>> them
>>> >> > as
>>> >> > ones that build on top of the Future primitive, not providing it as
>>> part
>>> >> > of
>>> >> > their implementation. What I think is important is that having
>>> unified
>>> >> > Future class simplifies interoperability between different kinds of
>>> >> > schedulers, not necessarily associated with asyncio event loops
>>> (process
>>> >> > pools for example).
>>> >>
>>> >> The interoperability is completely missing. I can't tell if you've
>>> >> used asyncio at all, but the important operation of *waiting* for a
>>> >> result is fundamentally different there than in concurrent.futures. In
>>> >> the latter, you just write "x = f.result()" and your thread blocks
>>> >> until the result is available. In asyncio, you have to write "x =
>>> >> yield from f.result()" which is a coroutine block that lets other
>>> >> tasks run in the same thread. ("yield from" in this case is how Python
>>> >> spells the operation that C# calls "await").
>>> >
>>> >
>>> > The way I see it, the fundamental operation on Futures is
>>> add_done_callback.
>>> > We then have various higher-level operations that let us get away from
>>> using
>>> > callbacks directly.  One of these happens to be a method on Future: the
>>> > blocking mode of Future.result().  Another is implemented in asyncio
>>> and
>>> > Tornado, in the ability to "yield" a Future.  The blocking mode of
>>> result()
>>> > is just a convenience; if asyncio-style futures had existed first then
>>> we
>>> > could instead have a function like "x =
>>> concurrent.futures.wait_for(f)".  In
>>> > fact, you could write this wait_for function today in a way that works
>>> for
>>> > both concurrent and asyncio futures.
>>> >
>>> > This is already interoperable:  Tornado's Resolver interface
>>> > (
>>> https://github.com/facebook/tornado/blob/master/tornado/netutil.py#L184)
>>> > returns a Future, which may be generated by a ThreadPoolExecutor or an
>>> > asynchronous wrapper around pycares or twisted.  In the other
>>> direction I've
>>> > worked on hybrid apps that have one Tornado thread alongside a bunch of
>>> > Django threads; in these apps it would work to have a Django thread
>>> block on
>>> > f.result() for a Future returned by Tornado's AsyncHTTPClient.
>>> >
>>> > -Ben
>>> >
>>> >>
>>> >> > Futures are definitely not the final solution for concurrency
>>> problem
>>> >> > but
>>> >> > rather a well-established utility for representing async
>>> operations. It
>>> >> > is a
>>> >> > job of higher layer systems to provide more convenient ways for
>>> dealing
>>> >> > with
>>> >> > asyncs (yield from coroutines, async/await rewrites etc.),
>>> >>
>>> >> But unless you are proposing some kind of radical change to add
>>> >> compile-time type checking/inference to Python, the rewrite option is
>>> >> unavailable in Python.
>>> >>
>>> >> > so I would not
>>> >> > say that futures encourage callback-style programming,
>>> >>
>>> >> The concurrent.futures.Future class does not. But unless I misread
>>> >> your proposal, your extensions do.
>>> >>
>>> >> > it?s simply a
>>> >> > lower-layer functionality. On the contrary, monadic
>>> >>
>>> >> (Say that word one more time and everyone tunes out. :-)
>>> >>
>>> >> > methods for futures
>>> >> > composition (e.g. map(), all(), first() etc.) ensure that no errors
>>> >> > would be
>>> >> > lost in the process, so I think they would complement yield from
>>> model
>>> >> > quite
>>> >> > nicely by hiding complexity of state maintenance from user and
>>> reducing
>>> >> > the
>>> >> > number of back-and-forth communications between event loop and
>>> >> > coroutines.
>>> >>
>>> >> I'm not sure I follow. Again, I'm not sure if you've actually written
>>> >> any code using asyncio.
>>> >>
>>> >> TBH I've written a fair number of example programs for asyncio and
>>> >> I've very rarely felt the need for these composition functions. The
>>> >> main composition primitive I tend to use is "yield from".
>>> >>
>>> >> > Besides Futures, reactive programming
>>> >>
>>> >> What *is* reactive programming? If you're talking about
>>> >> http://en.wikipedia.org/wiki/Reactive_programming,
>>> >> I'm not sure that it maps well to Python.
>>> >>
>>> >> > has more utilities to offer, such as
>>> >> > Observables (representing asynchronous streams of values). It is
>>> also a
>>> >> > very
>>> >> > useful abstraction with a rich set of composition strategies
>>> (merging,
>>> >> > concatenation, grouping), and may deserve its place in separate
>>> package.
>>> >>
>>> >> It all sounds very abstract and academic. :-)
>>> >>
>>> >> > Hope to hear back from you to get better picture on overall design
>>> >> > direction
>>> >> > here before jumping to implementation details.
>>> >> >
>>> >> > Brief follow-up to your questions:
>>> >> >  - Idea behind the Future/Promise separation is to draw a clean line
>>> >> > between
>>> >> > client-facing and scheduler-side APIs respectively. Futures should
>>> not
>>> >> > expose any completion methods, which clients should not call anyway.
>>> >>
>>> >> Given that Future and Promise are often used synonymously, using them
>>> >> to make this distinction sounds confusing. I agree that Futures have
>>> >> two different APIs, one for the consumer and another for the producer.
>>> >> But I'm not sure that it's necessary to separate them more strictly --
>>> >> convention seems good enough to me here. (It's the same with many
>>> >> communication primitives, like queues and even threads.)
>>> >>
>>> >> (The one thing that trips people up frequently is that, while
>>> >> set_result() and set_exception() are producer APIs, cancel() is a
>>> >> consumer API, and the cancellation signal travels from the consumer to
>>> >> the producer.)
>>> >>
>>> >> >  - Completely agree with you that Twisted-style callbacks are evil
>>> and
>>> >> > it is
>>> >> > better to have single code path for getting result or raising
>>> exception
>>> >> >  - Sorry for bad grammar in the proposal, it?s an early draft
>>> written in
>>> >> > 3
>>> >> > AM, so I will definitely improve on it if we decide to move forward.
>>> >>
>>> >> No problem!
>>> >>
>>> >> > Thanks,
>>> >> > Sergii
>>> >>
>>> >> --
>>> >> --Guido van Rossum (python.org/~guido)
>>> >> _______________________________________________
>>> >> Python-ideas mailing list
>>> >> Python-ideas at python.org
>>> >> https://mail.python.org/mailman/listinfo/python-ideas
>>> >> Code of Conduct: http://python.org/psf/codeofconduct/
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131224/f38b310b/attachment-0001.html>

From wolfgang.maier at biologie.uni-freiburg.de  Sat Dec 21 23:29:14 2013
From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang)
Date: Sat, 21 Dec 2013 14:29:14 -0800 (PST)
Subject: [Python-ideas] thoughts on the new 3.4 statistics module
Message-ID: <45123489-39fd-4bdb-aeab-47443d6f0939@googlegroups.com>

First of all: thank you, Steven and everyone else involved, for taking on 
the task of starting to implement this long-missed (at least by me) feature 
!
I really hope the module will be a success and grow over time.
I have two thoughts at the moment about the implementation that I think may 
be worth discussing, if it hasn't happened yet (I have to admit I did not 
go through all previous posts on this topic, only read the PEP):

First: I am not entirely convinced by when the module raises Errors. In 
some places its undoubtedly justified to raise StatisticsError (like when 
empty sequences are passed to mean()).
On the other hand, should there really be an error, when for example no 
unique value for the mode can be found?
Effectively, that would force users to guard every (!) call to the function 
with try/except. In my opinion, a better choice would be to return 
float('nan') or even better a module-specific object (call it Undefined or 
something) that one can check for. This behavior could, in general, be 
implemented for cases, where input can actually be handled and a result be 
calculated (like a list of values in the mode example), but this result is 
considered "undefined" by the algorithm.

Second: I am not entirely happy with the three different flavors of the 
median function. I *do* know that this has been discussed before, but I'm 
not sure whether *all* alternatives have been considered (the PEP only 
talks about the median.low, median.high syntax, which, in fact, I wouldn't 
like that much either. My suggestion would be to have a resolve parameter, 
by which the behavior of a single median function can be modified.
My main argument here is that as the module will grow in the future there 
will be many more such situations, in which different ways of calculating 
statistics are all totally acceptable and you would want to leave the 
choice to the user (the mode function can already be considered as an 
example: maybe the user would want to have the list of "modes" returned in 
case that no unambiguous value can be calculated; actually the current code 
seems to be prepared for later implementation of this feature because it 
does generate the list, just is not returning it). Now if, in all such 
situations, the solution is to have extra functions the module will soon 
end up completely cluttered with them. If, on the other hand, every 
function that will foreseeably have to handle ambiguous situations had a 
resolve parameter the module structure would be much clearer. In the median 
example you would then call median(data) for the default behavior, arguably 
the interpolation, but median(data, resolve='low') or median(data, 
resolve='high') for the alternative calculations. Statistically educated 
users could then guess, relatively easily, which functions have the resolve 
parameter and a quick look at the function's help could tell them, which 
arguments are accepted.

Finally, let me just point out that these are really just first thoughts 
and I do understand that these are design decisions about which different 
people will have different opinions, but I think now is still a good time 
to discuss them, while with an established and (hopefully :) ) much larger 
module you will not be able to change things that easily anymore.

Hoping for a lively discussion,
Wolfgang
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131221/598d40a5/attachment.html>

From phd at phdru.name  Tue Dec 24 20:13:09 2013
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 24 Dec 2013 20:13:09 +0100
Subject: [Python-ideas] thoughts on the new 3.4 statistics module
In-Reply-To: <45123489-39fd-4bdb-aeab-47443d6f0939@googlegroups.com>
References: <45123489-39fd-4bdb-aeab-47443d6f0939@googlegroups.com>
Message-ID: <20131224191309.GA28949@phdru.name>

Hi!

On Sat, Dec 21, 2013 at 02:29:14PM -0800, Wolfgang <wolfgang.maier at biologie.uni-freiburg.de> wrote:
> First: I am not entirely convinced by when the module raises Errors. In 
> some places its undoubtedly justified to raise StatisticsError (like when 
> empty sequences are passed to mean()).
> On the other hand, should there really be an error, when for example no 
> unique value for the mode can be found?
> Effectively, that would force users to guard every (!) call to the function 
> with try/except.

   Not necessary. The user of the library can combine a few calls in a
function/method and catch one exception for the entire calculation. Or
catch it even higher up the stack.

> In my opinion, a better choice would be to return 
> float('nan') or even better a module-specific object (call it Undefined or 
> something) that one can check for.

   With such special values the user must check every return value. What
is the advantage over catching exceptions?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From steve at pearwood.info  Wed Dec 25 01:47:25 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 25 Dec 2013 11:47:25 +1100
Subject: [Python-ideas] thoughts on the new 3.4 statistics module
In-Reply-To: <45123489-39fd-4bdb-aeab-47443d6f0939@googlegroups.com>
References: <45123489-39fd-4bdb-aeab-47443d6f0939@googlegroups.com>
Message-ID: <20131225004724.GI29356@ando>

Hi Wolfgang, and thanks for the feedback! My responses below.

On Sat, Dec 21, 2013 at 02:29:14PM -0800, Wolfgang wrote:

> First: I am not entirely convinced by when the module raises Errors. In 
> some places its undoubtedly justified to raise StatisticsError (like when 
> empty sequences are passed to mean()).
> On the other hand, should there really be an error, when for example no 
> unique value for the mode can be found?

There was no agreement on the best way to handle data with multiple 
modes, so we went with the simplest version that could work. It's easier 
to add functionality to the standard library than to take it away: 
better to delay putting something in for a release or two, than to put 
it in and then be stuck with the consequences of a poor decision for 
years.

An earlier version of statistics.py included a mode function that let 
you specify the maximum number of modes. That function may eventually be 
added to the module, or made available on PyPI. The version included in 
the standard library implements the basic, school-book version of mode: 
it returns the one unique mode, as calculated by counting distinct 
values, or it fails, and the most Pythonic way to implement failure is 
with an exception.


> Effectively, that would force users to guard every (!) call to the function 
> with try/except. 

No different from any other function. If you think a function might 
fail, then you guard it with try...except.


> In my opinion, a better choice would be to return 
> float('nan') or even better a module-specific object (call it Undefined or 
> something) that one can check for. This behavior could, in general, be 
> implemented for cases, where input can actually be handled and a result be 
> calculated (like a list of values in the mode example), but this result is 
> considered "undefined" by the algorithm.

You can easily get that behaviour with a simple wrapper function:

def my_mode(values):
    try:
        return mode(values)
    except StatisticsError:
        return float('nan')


But I'm not convinced that this is appropriate for nominal data. Would 
you expect that the mode of ['red', 'blue', 'green'] should be a 
floating point NAN? I know I wouldn't.


> Second: I am not entirely happy with the three different flavors of the 
> median function. I *do* know that this has been discussed before, but I'm 
> not sure whether *all* alternatives have been considered (the PEP only 
> talks about the median.low, median.high syntax, which, in fact, I wouldn't 
> like that much either. My suggestion would be to have a resolve parameter, 
> by which the behavior of a single median function can be modified.

For median, I don't believe this is appropriate. As a general rule, if a 
function has a parameter which is usually called with a constant known 
when you write the source code:

    median(data, resolve='middle')  # resolve is known at edit-time

especially if that parameter takes only two or three values, then the 
function probably should be split into two or three separate functions. 
I don't think that there are any common use-cases for selecting the type 
of median at *runtime*:

    kind = get_median_kind()
    median(data, resolve=kind)


but if you can think of any, I'd like to hear them.

However, your general suggestion isn't entirely inappropriate. In my 
research, I learned that there are at least fifteen different 
definitions of quartiles in common use, although some are mathematically 
equivalent. See here:

http://www.amstat.org/publications/jse/v14n3/langford.html

I find six distinct definitions for quartiles, and ten for quantiles/ 
fractiles. R supports nine different quantile versions, Haskell six, and 
SAS also supports multiple versions. (I don't remember how many.) 
Mathematica provides a four-argument parameterized version of Quantile.

With six distinct versions of quartile, and ten of quantile, it's too 
many to provide separate functions for each: too much duplication, too 
much clutter. Most people won't care which quantile they get, so there 
ought to be a sensible default. For those who care about matching some 
particular version (say, that used by Excel, or that used by Texas 
Instruments calculators), there ought to be a parameter that allows you 
to select which version is used. R calls this parameter "type". I don't 
remember what SAS and Haskell call it, but the term I prefer is 
"scheme".

I don't know if statistics.py will ever gain a function for calculating 
quantiles other than the median. I will probably put quantiles and 
quartiles on PyPI first, and if I do, I will follow your suggestion to 
provide a parameter to select the version used (although I'll probably 
call it "scheme" rather than "resolve").


-- 
Steven

From python at mrabarnett.plus.com  Wed Dec 25 04:15:04 2013
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 25 Dec 2013 03:15:04 +0000
Subject: [Python-ideas] thoughts on the new 3.4 statistics module
In-Reply-To: <20131225004724.GI29356@ando>
References: <45123489-39fd-4bdb-aeab-47443d6f0939@googlegroups.com>
 <20131225004724.GI29356@ando>
Message-ID: <52BA4DB8.7040502@mrabarnett.plus.com>

On 25/12/2013 00:47, Steven D'Aprano wrote:
> Hi Wolfgang, and thanks for the feedback! My responses below.
>
> On Sat, Dec 21, 2013 at 02:29:14PM -0800, Wolfgang wrote:
>
>> First: I am not entirely convinced by when the module raises Errors. In
>> some places its undoubtedly justified to raise StatisticsError (like when
>> empty sequences are passed to mean()).
>> On the other hand, should there really be an error, when for example no
>> unique value for the mode can be found?
>
> There was no agreement on the best way to handle data with multiple
> modes, so we went with the simplest version that could work. It's easier
> to add functionality to the standard library than to take it away:
> better to delay putting something in for a release or two, than to put
> it in and then be stuck with the consequences of a poor decision for
> years.
>
> An earlier version of statistics.py included a mode function that let
> you specify the maximum number of modes. That function may eventually be
> added to the module, or made available on PyPI. The version included in
> the standard library implements the basic, school-book version of mode:
> it returns the one unique mode, as calculated by counting distinct
> values, or it fails, and the most Pythonic way to implement failure is
> with an exception.
>
[snip]

Data that has multiple modes is "multimodal" (who said stats was
difficult? :-)), so perhaps there could be a "multimode" function that
returns a list of modes.


From techtonik at gmail.com  Wed Dec 25 08:47:11 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 25 Dec 2013 10:47:11 +0300
Subject: [Python-ideas] Reducing language complexity
Message-ID: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>

One of the tools to reduce language complexity is "explicitness" or
the direct link to help/tutorial/documentation from the concept. The
problem with most concepts in computer languages that they don't have
distinct markers by which you can recognize one feature or another.
For example, you can't recognize that code is generator based or uses
metaclass magic without searching for yield or some references to
metaclass through the source file.

One of the ways to reduce language complexity for new people who read
you code, is to prepare them for advanced concepts that your code uses
beforehand. For example, with the following section:

using generators as yield

         ^^^ name of this language feature and also help reference
                              ^^^ distinct keywords and feature
markers that you enable
--
anatoly t.

From rosuav at gmail.com  Wed Dec 25 09:53:14 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 25 Dec 2013 19:53:14 +1100
Subject: [Python-ideas] Reducing language complexity
In-Reply-To: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
References: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
Message-ID: <CAPTjJmpY_pwWa94Czw6LvNAsALFG68ev0Tp3iwV_Kr07aon-eg@mail.gmail.com>

On Wed, Dec 25, 2013 at 6:47 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> One of the ways to reduce language complexity for new people who read
> you code, is to prepare them for advanced concepts that your code uses
> beforehand. For example, with the following section:
>
> using generators as yield
>
>          ^^^ name of this language feature and also help reference
>                               ^^^ distinct keywords and feature
> markers that you enable

Yes, and it's great that you can choose what the keyword is! We should
announce the usage of this thus:

using using as as
using as as using

and only then use using and as as as and using.

And of course, it's then critical to declare the fact that you're
using built-in functions, in case they confuse people too. And their
parameters might be confusing, too; best to acknowledge the exact
parameter lists that each built-in function uses. Something like this:

using print(*values, sep=' ', end='\n', file=sys.stdout, flush=False)
using open(file, mode='r', buffering=-1, encoding=None,
         errors=None, newline=None, closefd=True, opener=None)

Of course, this could get rather long, so we could break things out
into other files, and put all those 'using' declarations out of the
way. Rather than create new syntax for all this, I recommend we use
specially-formatted comments; that way, older versions of Python won't
be bothered by it. So let's put those two declarations into a file
called stdio.h, and put this at the top of your Python script:

#include <stdio.h>

I'm sure everyone will agree that this is the best thing for Python.

ChrisA

From techtonik at gmail.com  Wed Dec 25 10:03:37 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 25 Dec 2013 12:03:37 +0300
Subject: [Python-ideas] Reducing language complexity
In-Reply-To: <CAPTjJmpY_pwWa94Czw6LvNAsALFG68ev0Tp3iwV_Kr07aon-eg@mail.gmail.com>
References: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
 <CAPTjJmpY_pwWa94Czw6LvNAsALFG68ev0Tp3iwV_Kr07aon-eg@mail.gmail.com>
Message-ID: <CAPkN8xL0e-5ptDNUFzDrVyL+=_JjtLz4HavGuy0h+_uLtBjwqw@mail.gmail.com>

On Wed, Dec 25, 2013 at 11:53 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, Dec 25, 2013 at 6:47 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> One of the ways to reduce language complexity for new people who read
>> you code, is to prepare them for advanced concepts that your code uses
>> beforehand. For example, with the following section:
>>
>> using generators as yield
>>
>>          ^^^ name of this language feature and also help reference
>>                               ^^^ distinct keywords and feature
>> markers that you enable
>
> Yes, and it's great that you can choose what the keyword is! We should
> announce the usage of this thus:
>
> using using as as
> using as as using
>
> and only then use using and as as as and using.
>
> And of course, it's then critical to declare the fact that you're
> using built-in functions, in case they confuse people too. And their
> parameters might be confusing, too; best to acknowledge the exact
> parameter lists that each built-in function uses. Something like this:
>
> using print(*values, sep=' ', end='\n', file=sys.stdout, flush=False)
> using open(file, mode='r', buffering=-1, encoding=None,
>          errors=None, newline=None, closefd=True, opener=None)
>
> Of course, this could get rather long, so we could break things out
> into other files, and put all those 'using' declarations out of the
> way. Rather than create new syntax for all this, I recommend we use
> specially-formatted comments; that way, older versions of Python won't
> be bothered by it. So let's put those two declarations into a file
> called stdio.h, and put this at the top of your Python script:
>
> #include <stdio.h>
>
> I'm sure everyone will agree that this is the best thing for Python.

Unfortunately, the first message with "Language complexity formula" is
deemed to be "not python related" by moderators, so this mail landed
without necessary context. Without understanding what adds to the
language complexity, this discussion looks useless.
--
anatoly t.

From taleinat at gmail.com  Wed Dec 25 10:23:12 2013
From: taleinat at gmail.com (Tal Einat)
Date: Wed, 25 Dec 2013 11:23:12 +0200
Subject: [Python-ideas] Reducing language complexity
In-Reply-To: <CAPkN8xL0e-5ptDNUFzDrVyL+=_JjtLz4HavGuy0h+_uLtBjwqw@mail.gmail.com>
References: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
 <CAPTjJmpY_pwWa94Czw6LvNAsALFG68ev0Tp3iwV_Kr07aon-eg@mail.gmail.com>
 <CAPkN8xL0e-5ptDNUFzDrVyL+=_JjtLz4HavGuy0h+_uLtBjwqw@mail.gmail.com>
Message-ID: <CALWZvp7LwidfmqBu6H_Qc0DU_8DsB+AXHeE-MBm1hs4VvjH-MQ@mail.gmail.com>

On Wed, Dec 25, 2013 at 11:03 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>
> Unfortunately, the first message with "Language complexity formula" is
> deemed to be "not python related" by moderators, so this mail landed
> without necessary context. Without understanding what adds to the
> language complexity, this discussion looks useless.
> --
> anatoly t.

Hi Anatoly,

I'm the moderator who rejected your other message titled "Language
complexity formula".

To be clear, I did not write that the rejected message was "not Python
related". I wrote that it was not relevant specifically to the
Python-Ideas mailing list. As I'm sure you know, this list is for
discussing ideas for the Python language. Your message only suggested
a formula for evaluating the complexity of programming languages,
while containing no ideas or suggestions for the language itself.

If that message was intended as context for this message (titled
"Reducing language complexity"), then you should have included the
context as well as the suggestion in a single message.

To be clear, please do not post several messages which discuss the
same subject, or very similar subjects, in parallel.

Of course, you are free to add context and explanations to the
discussion as you see fit.

Regards,
- Tal Einat

From abarnert at yahoo.com  Wed Dec 25 10:35:17 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 25 Dec 2013 01:35:17 -0800
Subject: [Python-ideas] Reducing language complexity
In-Reply-To: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
References: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
Message-ID: <179F68B2-33D5-41B3-A7E7-6FCD2E90A802@yahoo.com>

When I search for "python generators" instead of "python yield", I get the same docs pages, blog posts, and StackOverflow questions making up 90% of the results, in the same order.

When I search for "generators" instead of "yield" at docs.python.org, I get _worse_ results--a bunch of stuff about the C API and then a slew of random modules--the yield statement, the relevant tutorial sections, etc. are nowhere to be seen.

When I use the built-in help, "yield" gives me a nice overview; "generators" tells me there's no documentation.

So, in what way is "generators" a better term for finding help/tutorial/documentation than "yield"?

And I'm not sure what term for the metaclass feature would be better than "metaclass". (If you were going to suggest "using metaclass as __metaclass__" I might agree that was a small improvement, if Python hadn't fixed that 5 years ago.)

Sent from a random iPhone

On Dec 24, 2013, at 23:47, anatoly techtonik <techtonik at gmail.com> wrote:

> One of the tools to reduce language complexity is "explicitness" or
> the direct link to help/tutorial/documentation from the concept. The
> problem with most concepts in computer languages that they don't have
> distinct markers by which you can recognize one feature or another.
> For example, you can't recognize that code is generator based or uses
> metaclass magic without searching for yield or some references to
> metaclass through the source file.
> 
> One of the ways to reduce language complexity for new people who read
> you code, is to prepare them for advanced concepts that your code uses
> beforehand. For example, with the following section:
> 
> using generators as yield
> 
>         ^^^ name of this language feature and also help reference
>                              ^^^ distinct keywords and feature
> markers that you enable
> --
> anatoly t.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From steve at pearwood.info  Wed Dec 25 11:19:32 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 25 Dec 2013 21:19:32 +1100
Subject: [Python-ideas] Reducing language complexity
In-Reply-To: <CAPkN8xL0e-5ptDNUFzDrVyL+=_JjtLz4HavGuy0h+_uLtBjwqw@mail.gmail.com>
References: <CAPkN8xK7wcGBUXrGKS+HWfwTfgQja2XFHSb=UR_LnpyGXERiTg@mail.gmail.com>
 <CAPTjJmpY_pwWa94Czw6LvNAsALFG68ev0Tp3iwV_Kr07aon-eg@mail.gmail.com>
 <CAPkN8xL0e-5ptDNUFzDrVyL+=_JjtLz4HavGuy0h+_uLtBjwqw@mail.gmail.com>
Message-ID: <20131225101931.GK29356@ando>

On Wed, Dec 25, 2013 at 12:03:37PM +0300, anatoly techtonik wrote:

> Unfortunately, the first message with "Language complexity formula" is
> deemed to be "not python related" by moderators, so this mail landed
> without necessary context. Without understanding what adds to the
> language complexity, this discussion looks useless.

You should write a blog post about the language complexity formula. That 
way people who are motivated by it can read it, while those who don't 
won't need to be bothered by an off-topic discussion on something which 
isn't specific to Python.

As for your suggestion to "prepare" readers for "advanced" concepts:

    using generators as yield


who defines what counts as "advanced"? Python for-loops iterating over 
sequences may seen advanced to a C or Pascal programmer expecting to 
write code like "for i := 1 to 50 do process(seq[i]);". Exceptions may 
seem advanced to Java programmers. Even functions may seem terribly 
advanced to those who have never programmed before. (And I've seen 
them on the tutor list.)

How do you expect this "using" statement to actually help the reader? 
I have never, ever, not even as a total newbie knowing next to nothing 
about Python (or any other language) been in a position where I was 
reading code, came up on a keyword, function or idiom I didn't 
understand, and thought "Oh, if only this was declared at the beginning 
of the file, I would have understood it better!".

On the other hand, I would feel pretty annoyed if I came across a file 
containing

    using flibbit as smudgeon

at the start, spent a lot of time reading up on flibbit and smudgeon, 
then came back to the file and discovered that the hours I had spent was 
completely unnecessary because the part of the file I cared about didn't 
actually use flibbit or smudgeon.

If a reader comes across "advanced" code they don't understand:

def increment(values):
    for value in values:
        yield value+1

isn't it pretty trivial for them to ask "what does yield do?". You 
don't need a "using" statement to lead them to the right search 
terms. If you think the search terms aren't obvious enough and want to 
write code aimed at the least-experienced reader, just use a comment, 
and put it right where it is needed:

# Keywords: yield, generator, Python.
# http://link-to-your-blog-explaining-this
def increment(values):
    for value in values:
        yield value+1


-- 
Steven

From mcepl at redhat.com  Thu Dec 26 23:17:34 2013
From: mcepl at redhat.com (Matej Cepl)
Date: Thu, 26 Dec 2013 23:17:34 +0100
Subject: [Python-ideas] textwrap.TextWrapper width=None (or Inf)
Message-ID: <20131226221733.GA21940@wycliff.ceplovi.cz>

Hi,

I wonder why nobody asked on bugs.python.org for rather obvious 
functionality of being able to reflow a paragraph to one line?  
Meaning, that any paragraph would be stripped of all whitespace 
(etc. ... whatever is configured by the additional parameters of 
the TextWrapper class) and then joined into long line. I know 
that

    ''.join(text.splitlines())
    
does something similar, but

    a) it doesn't handle all whitespace munging,
    b) it just seems like an obvious functionality for 
    TextWrapper to have.

Any thoughts on it? Should I just file a bug?

Best,

Mat?j

-- 
http://www.ceplovi.cz/matej/, Jabber: mcepl<at>ceplovi.cz
GPG Finger: 89EF 4BC6 288A BF43 1BAB  25C3 E09F EF25 D964 84AC
 
[...] a superior pilot uses his superior judgment to avoid having to exercise
his superior skill.
  -- http://www.jwz.org/blog/2009/09/that-duct-tape-silliness/#comment-10653

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131226/11518279/attachment.sig>

From tjreedy at udel.edu  Thu Dec 26 23:51:42 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 26 Dec 2013 17:51:42 -0500
Subject: [Python-ideas] textwrap.TextWrapper width=None (or Inf)
In-Reply-To: <20131226221733.GA21940@wycliff.ceplovi.cz>
References: <20131226221733.GA21940@wycliff.ceplovi.cz>
Message-ID: <l9ibts$67p$1@ger.gmane.org>

On 12/26/2013 5:17 PM, Matej Cepl wrote:

> I wonder why nobody asked on bugs.python.org for rather obvious
> functionality of being able to reflow a paragraph to one line?

Because width=4000000000 should do just that?

> Any thoughts on it? Should I just file a bug?

It would be an enhancement request, not a bug report, but it seems not 
needed.

-- 
Terry Jan Reedy


From steve at pearwood.info  Fri Dec 27 00:17:44 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 27 Dec 2013 10:17:44 +1100
Subject: [Python-ideas] textwrap.TextWrapper width=None (or Inf)
In-Reply-To: <20131226221733.GA21940@wycliff.ceplovi.cz>
References: <20131226221733.GA21940@wycliff.ceplovi.cz>
Message-ID: <20131226231743.GO29356@ando>

On Thu, Dec 26, 2013 at 11:17:34PM +0100, Matej Cepl wrote:
> Hi,
> 
> I wonder why nobody asked on bugs.python.org for rather obvious 
> functionality of being able to reflow a paragraph to one line?  

Possibly because nobody needed the functionality?

Or because they didn't think that *wrapping* a long line into a 
paragraph and *unwrapping* a paragraph into a single line should be 
handled by the same function?


> Meaning, that any paragraph would be stripped of all whitespace 
> (etc. ... whatever is configured by the additional parameters of 
> the TextWrapper class) and then joined into long line. I know 
> that
> 
>    ''.join(text.splitlines())
>    
> does something similar, but

I would expect that you should use ' '.join, rather than the empty 
string. Otherwises lines will be incorrectly concatenated:

"""the cat in
the hat"""

=> "the cat inthe hat"


>    a) it doesn't handle all whitespace munging,

Can you given an example of what whitespace munging it fails to handle?


>    b) it just seems like an obvious functionality for 
>    TextWrapper to have.

py> text = """the cat in
... the hat"""
py> textwrap.wrap(text, width=len(text))
['the cat in the hat']

Is there a case that this does not handle?

Perhaps this is not obvious enough. A simple helper function may 
increase discoverability:

def unwrap(text):
    return wrap(text, width=len(text))



-- 
Steven

From amber.yust at gmail.com  Fri Dec 27 02:57:02 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Fri, 27 Dec 2013 01:57:02 +0000
Subject: [Python-ideas] "maybe import"?
Message-ID: <-7047415593530210756@gmail297201516>

It's a fairly standard pattern to see things like this:

    try:
        import foo
    except ImportError:
        foo = None

(and of course, variants with from...import et cetera). These can
potentially add a lot of clutter to the imports section of a file, given
that it requires 4 lines to do a conditional import.

It seems like it'd be useful and clean to have a syntax that looked like
this:

    maybe import foo
    from bar maybe import baz
    from qux maybe import quy as quz

Where the behavior would essentially be as above - attempt to run the
import normally, and in cases where the import fails, map the name to a
value of None instead. Users who want a different behavior are still free
to use the long-form syntax. A possibly variant might be to also only run
the import if the name isn't already bound, so that you could do something
like...

    from frobber_a maybe import frob as frobber
    from frobbler_b maybe import frobble as frobber
    from frobber_c maybe import frobit as frobber

...to potentially try different fallback options if the first choice for an
interface provider isn't available.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/3142acda/attachment.html>

From steve at pearwood.info  Fri Dec 27 05:08:09 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 27 Dec 2013 15:08:09 +1100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <-7047415593530210756@gmail297201516>
References: <-7047415593530210756@gmail297201516>
Message-ID: <20131227040808.GP29356@ando>

On Fri, Dec 27, 2013 at 01:57:02AM +0000, Amber Yust wrote:
> It's a fairly standard pattern to see things like this:
> 
>     try:
>         import foo
>     except ImportError:
>         foo = None
> 
> (and of course, variants with from...import et cetera). These can
> potentially add a lot of clutter to the imports section of a file, given
> that it requires 4 lines to do a conditional import.

You can reduce that down to two lines:

    try:  import foo
    except ImportError:  foo = None


but your point is taken.

A more common pattern in my experience is:

try:
    import this
except ImportError:
    import that as this



> It seems like it'd be useful and clean to have a syntax that looked like
> this:
> 
>     maybe import foo
>     from bar maybe import baz
>     from qux maybe import quy as quz
> 
> Where the behavior would essentially be as above - attempt to run the
> import normally, and in cases where the import fails, map the name to a
> value of None instead. Users who want a different behavior are still free
> to use the long-form syntax. 

Hmmm. The basic idea makes a certain level of sense to me, but I'm not 
sure it makes enough sense to overcome the barrier required before 
adding a new keyword.

I'm not (yet) convinced of the need for this functionality, but if 
Python did gain this, I think I would prefer the colour of this 
bike-shed to be "perhaps import" rather than "maybe import". A couple of 
reasons:

- it seems to me that "maybe" is more likely to already be used in code
  than "perhaps", e.g. in three-value logics (true, false, maybe);

- to me, "maybe" feels somewhat random, arbitrary or indeterminant,
  whereas "perhaps" feels subtly more determinant. I can't justify this
  claim by dictionary definitions, perhaps it's just me :-)


If we're entertaining changes to imports, another possibility would be 
to allow fallback module names:

    import this or that or another as this

Each of "this", "that", "another" will be attempted, the first 
successful import being bound to the name "this". The "as this" part 
would be mandatory, so as to require a consistent name regardless of 
which module was imported. This would be a syntax error, since it isn't 
clear what name would be bound at the end:

    import this or that or another


This would also be allowed:

    from this or that or another import spam


With this syntax, we could add None as a special case:

    import this or that or another or None as this

would be equivalent to:

    module_names = ("this", "that", "another", "None")
    for name in module_names:
        if name == "None":
            spam = None
        else:
            try:
                this = __import__(name)
            except ImportError:
                continue
        break
    else:
        raise ImportError


and the "from...import" case could be written as:

    from this or that or None import spam


roughly equivalent to:

    module_names = ("this", "that", "another", "None")
    for name in module_names:
        if name == "None":
            spam = None
        else:
            try:
                temp = __import__(name)
                spam = temp.spam
            except ImportError:
                continue
        break
    else:
        raise ImportError


Advantages:

- covers both use-cases where you want to try a series of
  modules, and the one where you fall back to None;

- "or" is already a keyword, no new keywords needed;

- reads more like English;

- "import None" currently gives SyntaxError, so this
  can't interfere with modules actually called "None".


Disadvantages:

- more complexity to imports;

- only saves a few lines;

- this usage of "or" is not quite the same as the usage 
  as a boolean operator, e.g. different from "x in a or b".


> A possibly variant might be to also only run
> the import if the name isn't already bound, so that you could do something
> like...
> 
>     from frobber_a maybe import frob as frobber
>     from frobbler_b maybe import frobble as frobber
>     from frobber_c maybe import frobit as frobber
> 
> ...to potentially try different fallback options if the first choice for an
> interface provider isn't available.

I dislike this form, because it requires short-circuiting execution of 
separate lines. What would you expect this to do?

from frobber_a maybe import frob as frobber
frobber = 23
from frobbler_b maybe import frobble as frobber

Is the second maybe import attempted or not? I have no idea whether it 
should be or shouldn't be.


-- 
Steven

From amber.yust at gmail.com  Fri Dec 27 05:36:21 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Fri, 27 Dec 2013 04:36:21 +0000
Subject: [Python-ideas] "maybe import"?
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
Message-ID: <2119483115886762605@gmail297201516>

On Thu Dec 26 2013 at 7:40:19 PM, Bruce Leban <bruce at leapyear.org> wrote:

> I think you mean do the import if the name is unbound *or bound to None.* Otherwise,
> it doesn't work in the example you gave.
>

Yes, that is what I meant, sorry. Another option would be "not bound to
something that is not a module" - but I think "unbound or None" is probably
the most versatile option.

On Thu Dec 26 2013 at 8:08:45 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> I'm not (yet) convinced of the need for this functionality, but if
> Python did gain this, I think I would prefer the colour of this
> bike-shed to be "perhaps import" rather than "maybe import".


Another option would be to re-use the 'or' keyword:

    from foo import bar or None

Where the bit after the 'or' simply specifies a default value to assign if
an ImportError occurs.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/a21db5f1/attachment-0001.html>

From amber.yust at gmail.com  Fri Dec 27 05:38:29 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Fri, 27 Dec 2013 04:38:29 +0000
Subject: [Python-ideas] "maybe import"?
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
Message-ID: <-9004133628104978885@gmail297201516>

In fact, reuse of the 'or' keyword also leads to these nice options:

import foo or None as bar
from foo import bar or None as baz

On Thu Dec 26 2013 at 8:36:24 PM, Amber Yust <amber.yust at gmail.com> wrote:

> On Thu Dec 26 2013 at 7:40:19 PM, Bruce Leban <bruce at leapyear.org> wrote:
>
> I think you mean do the import if the name is unbound *or bound to None.* Otherwise,
> it doesn't work in the example you gave.
>
>
> Yes, that is what I meant, sorry. Another option would be "not bound to
> something that is not a module" - but I think "unbound or None" is probably
> the most versatile option.
>
> On Thu Dec 26 2013 at 8:08:45 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>
> I'm not (yet) convinced of the need for this functionality, but if
> Python did gain this, I think I would prefer the colour of this
> bike-shed to be "perhaps import" rather than "maybe import".
>
>
> Another option would be to re-use the 'or' keyword:
>
>     from foo import bar or None
>
> Where the bit after the 'or' simply specifies a default value to assign if
> an ImportError occurs.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/b0e9d18d/attachment.html>

From greg at krypto.org  Fri Dec 27 05:57:20 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 26 Dec 2013 20:57:20 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <-7047415593530210756@gmail297201516>
References: <-7047415593530210756@gmail297201516>
Message-ID: <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>

On Thu, Dec 26, 2013 at 5:57 PM, Amber Yust <amber.yust at gmail.com> wrote:

> It's a fairly standard pattern to see things like this:
>
>     try:
>         import foo
>     except ImportError:
>         foo = None
>
> (and of course, variants with from...import et cetera). These can
> potentially add a lot of clutter to the imports section of a file, given
> that it requires 4 lines to do a conditional import.
>
> It seems like it'd be useful and clean to have a syntax that looked like
> this:
>
>     maybe import foo
>     from bar maybe import baz
>     from qux maybe import quy as quz
>
> Where the behavior would essentially be as above - attempt to run the
> import normally, and in cases where the import fails, map the name to a
> value of None instead. Users who want a different behavior are still free
> to use the long-form syntax. A possibly variant might be to also only run
> the import if the name isn't already bound, so that you could do something
> like...
>
>     from frobber_a maybe import frob as frobber
>     from frobbler_b maybe import frobble as frobber
>     from frobber_c maybe import frobit as frobber
>
> ...to potentially try different fallback options if the first choice for
> an interface provider isn't available.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>

Such idioms are common.  Though I don't think we should encourage their use.

That said, if you want to have something like this added it should not use
a new keyword ("maybe") but should use what we have. These look odd to me
but are possible ideas:

import foo else foo = None
from foo import bar else bar = None

that reuse of else on import statements would also enable the other idiom
of importing one of several things to a single name:

import foo as x else import bar as x

Just tossing those out there. I'm not convinced this is worth adding to the
language.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131226/1f532177/attachment.html>

From amber.yust at gmail.com  Fri Dec 27 06:57:13 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Fri, 27 Dec 2013 05:57:13 +0000
Subject: [Python-ideas] "maybe import"?
References: <-7047415593530210756@gmail297201516>
 <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>
 <3965602355176718462@gmail297201516>
Message-ID: <-8326952152688219870@gmail297201516>

 On Thu Dec 26 2013 at 8:57:42 PM, Gregory P. Smith <greg at krypto.org> wrote:
> Such idioms are common.  Though I don't think we should encourage their
use.

Is this a case of "we shouldn't encourage optional dependencies" or "we
shouldn't encourage this kind of idiom as a means of implementing optional
dependencies" - and if the latter, what alternative would you point people
at instead?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/ac1ac14d/attachment.html>

From guido at python.org  Fri Dec 27 09:33:11 2013
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Dec 2013 00:33:11 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <-8326952152688219870@gmail297201516>
References: <-7047415593530210756@gmail297201516>
 <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>
 <3965602355176718462@gmail297201516>
 <-8326952152688219870@gmail297201516>
Message-ID: <CAP7+vJ+ATjrUAPKfYNRfhCNVPs_M_DGm3VJLGWppuz8d5prJJw@mail.gmail.com>

We shouldn't encourage optional dependencies. They often make code harder
to reason about or harder to read.

On Thursday, December 26, 2013, Amber Yust wrote:

>  On Thu Dec 26 2013 at 8:57:42 PM, Gregory P. Smith <greg at krypto.org<javascript:_e({}, 'cvml', 'greg at krypto.org');>>
> wrote:
> > Such idioms are common.  Though I don't think we should encourage their
> use.
>
> Is this a case of "we shouldn't encourage optional dependencies" or "we
> shouldn't encourage this kind of idiom as a means of implementing optional
> dependencies" - and if the latter, what alternative would you point people
> at instead?
>


-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/4d39b66e/attachment.html>

From flying-sheep at web.de  Fri Dec 27 09:36:03 2013
From: flying-sheep at web.de (Philipp A.)
Date: Fri, 27 Dec 2013 09:36:03 +0100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <CAP7+vJ+ATjrUAPKfYNRfhCNVPs_M_DGm3VJLGWppuz8d5prJJw@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>
 <3965602355176718462@gmail297201516>
 <-8326952152688219870@gmail297201516>
 <CAP7+vJ+ATjrUAPKfYNRfhCNVPs_M_DGm3VJLGWppuz8d5prJJw@mail.gmail.com>
Message-ID: <CAN8d9gntt_Cs8b3Cvhg1z2erFt5dBZWnyCb9UGAe9koNAE=K=w@mail.gmail.com>

agreed. and the try/except pattern also allows for

try:
    import thing
except ImportError:
    import slower_thing_with_same_api as thing


2013/12/27 Guido van Rossum <guido at python.org>

> We shouldn't encourage optional dependencies. They often make code harder
> to reason about or harder to read.
>
>
> On Thursday, December 26, 2013, Amber Yust wrote:
>
>>  On Thu Dec 26 2013 at 8:57:42 PM, Gregory P. Smith <greg at krypto.org>
>> wrote:
>> > Such idioms are common.  Though I don't think we should encourage their
>> use.
>>
>> Is this a case of "we shouldn't encourage optional dependencies" or "we
>> shouldn't encourage this kind of idiom as a means of implementing optional
>> dependencies" - and if the latter, what alternative would you point people
>> at instead?
>>
>
>
> --
> --Guido van Rossum (on iPad)
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/05de5022/attachment-0001.html>

From amber.yust at gmail.com  Fri Dec 27 10:34:14 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Fri, 27 Dec 2013 09:34:14 +0000
Subject: [Python-ideas] "maybe import"?
References: <-7047415593530210756@gmail297201516>
 <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>
 <3965602355176718462@gmail297201516> <-8326952152688219870@gmail297201516>
 <CAP7+vJ+ATjrUAPKfYNRfhCNVPs_M_DGm3VJLGWppuz8d5prJJw@mail.gmail.com>
Message-ID: <876459810524566913@gmail297201516>

On Fri Dec 27 2013 at 12:33:11 AM, Guido van Rossum <guido at python.org>
wrote:

> We shouldn't encourage optional dependencies. They often make code harder
> to reason about or harder to read.
>

Is there a better alternative for utilizing functionality that may vary in
availability from platform to platform?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/5dc0f583/attachment.html>

From abarnert at yahoo.com  Fri Dec 27 13:23:39 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 27 Dec 2013 04:23:39 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <876459810524566913@gmail297201516>
References: <-7047415593530210756@gmail297201516>
 <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>
 <3965602355176718462@gmail297201516> <-8326952152688219870@gmail297201516>
 <CAP7+vJ+ATjrUAPKfYNRfhCNVPs_M_DGm3VJLGWppuz8d5prJJw@mail.gmail.com>
 <876459810524566913@gmail297201516>
Message-ID: <B601E43C-EFFB-46BF-9ABF-AA3DB6A860B0@yahoo.com>

On Dec 27, 2013, at 1:34, Amber Yust <amber.yust at gmail.com> wrote:

> On Fri Dec 27 2013 at 12:33:11 AM, Guido van Rossum <guido at python.org> wrote:
>> We shouldn't encourage optional dependencies. They often make code harder to reason about or harder to read.
> 
> Is there a better alternative for utilizing functionality that may vary in availability from platform to platform?

The only way to use this would be to then continually check "if foo:" all throughout your module, to use it if present and do something else if not. I suspect that is a bigger readability/comprehensibility problem than the extra lines of code up top.

And in many cases, once you reorganize your code to solve that problem, this one goes away as well.

>  
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/2c78ee9d/attachment.html>

From ncoghlan at gmail.com  Fri Dec 27 15:26:50 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Dec 2013 00:26:50 +1000
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <876459810524566913@gmail297201516>
References: <-7047415593530210756@gmail297201516>
 <CAGE7PNJds5+PF_TKtom=R=CQ+KDSX6QLQ-fjA1YXKAdyDVVvRg@mail.gmail.com>
 <3965602355176718462@gmail297201516>
 <-8326952152688219870@gmail297201516>
 <CAP7+vJ+ATjrUAPKfYNRfhCNVPs_M_DGm3VJLGWppuz8d5prJJw@mail.gmail.com>
 <876459810524566913@gmail297201516>
Message-ID: <CADiSq7f5JXjpkYPt_1YQTY3r9B6KpB3+wgs8zzzfiJoeGPKuTA@mail.gmail.com>

On 27 December 2013 19:34, Amber Yust <amber.yust at gmail.com> wrote:
> On Fri Dec 27 2013 at 12:33:11 AM, Guido van Rossum <guido at python.org>
> wrote:
>>
>> We shouldn't encourage optional dependencies. They often make code harder
>> to reason about or harder to read.
>
>
> Is there a better alternative for utilizing functionality that may vary in
> availability from platform to platform?

The most common approach I've seen is a compatibility module, so most
code in the application just does something like:

    from myapp.foo_compat import foo

and only foo_compat has the try/except logic. Depending on the number
of optional dependencies and the cost of importing them even when
they're not needed, this can be done on a per-dependency basis, or as
a single compatibility module that handles all the optional
dependencies.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From bruce at leapyear.org  Fri Dec 27 04:39:38 2013
From: bruce at leapyear.org (Bruce Leban)
Date: Thu, 26 Dec 2013 19:39:38 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <-7047415593530210756@gmail297201516>
References: <-7047415593530210756@gmail297201516>
Message-ID: <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>

On Thu, Dec 26, 2013 at 5:57 PM, Amber Yust <amber.yust at gmail.com> wrote:

> Where the behavior would essentially be as above - attempt to run the
> import normally, and in cases where the import fails, map the name to a
> value of None instead. Users who want a different behavior are still free
> to use the long-form syntax. A possibly variant might be to also only run
> the import if the name isn't already bound,
>

I think you mean do the import if the name is unbound *or bound to
None.* Otherwise,
it doesn't work in the example you gave.

Here's another use case. Some modules, e.g., pycrypto need to be compiled
in order to be installed. I have a stub version of pycrypto which can be
used as a substitute for testing without actually encrypting anything.
Rather than using a new keyword, the try keyword would fit here fine:

try import Crypto
try import CryptoStub as Crypto


note that there are cases where this can't be a drop in replacement:

try import Crypto.Random

try import CryptoStub.Random as Crypto.Random   # not allowed


you'd have to instead write:

try import Crypto.Random as CryptoRandom

try import CryptoStub.Random as CryptoRandom


--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131226/7d9628b6/attachment.html>

From mertz at gnosis.cx  Fri Dec 27 17:30:43 2013
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 27 Dec 2013 08:30:43 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <-7047415593530210756@gmail297201516>
References: <-7047415593530210756@gmail297201516>
Message-ID: <CAEbHw4YXPV3e5acMCSELs7gT3fkgJEC-zEnne3SmqH8uE92Mvg@mail.gmail.com>

It's a fairly standard pattern to see things like this:

    try:
        import foo
    except ImportError:
        foo = None

(and of course, variants with from...import et cetera). These can
potentially add a lot of clutter to the imports section of a file, given
that it requires 4 lines to do a conditional import.

It seems like it'd be useful and clean to have a syntax that looked like
this:

    maybe import foo
    from bar maybe import baz
    from qux maybe import quy as quz

Where the behavior would essentially be as above - attempt to run the
import normally, and in cases where the import fails, map the name to a
value of None instead. Users who want a different behavior are still free
to use the long-form syntax. A possibly variant might be to also only run
the import if the name isn't already bound, so that you could do something
like...

    from frobber_a maybe import frob as frobber
    from frobbler_b maybe import frobble as frobber
    from frobber_c maybe import frobit as frobber

...to potentially try different fallback options if the first choice for an
interface provider isn't available.

_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/e9da0c72/attachment.html>

From mertz at gnosis.cx  Fri Dec 27 18:00:41 2013
From: mertz at gnosis.cx (David Mertz)
Date: Fri, 27 Dec 2013 09:00:41 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <CAEbHw4YXPV3e5acMCSELs7gT3fkgJEC-zEnne3SmqH8uE92Mvg@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <CAEbHw4YXPV3e5acMCSELs7gT3fkgJEC-zEnne3SmqH8uE92Mvg@mail.gmail.com>
Message-ID: <CAEbHw4YGwO9m-Z6BFbHkXR6fiHPSESy2xHpYg95c4owjrb19qg@mail.gmail.com>

On Thu, Dec 26, 2013 at 5:57 PM, Amber Yust <amber.yust at gmail.com> wrote:

> It's a fairly standard pattern to see things like this:



>     try:
>         import foo
>     except ImportError:
>         foo = None
>

I would rarely, if ever, follow this pattern, although I *would* do
conditional import fairly often.  E.g. these are things I might do (well,
not really these specific ones, but just as pattern examples):

try:
    from math import pi
except ImportError:
    pi = 3.1415

Or:

try:
    import fancymodule as mod
except ImportError:
    import simpler_version as mod

Or:

try:
    from mymod import needed_func
except ImportError:
    def needed_func(a,b,c):
        "Bare bones implementation of more general function"
        return a*b + c

Or even:

try:
    import cool_feature
    FEATURE_AVAIL = True
except ImportError:
    FEATURE_AVAIL = False

What these have in common is that they are each specific to the context,
and actually have nothing much in common other than the possibility import
might fail.  The special case of defining a missing module as 'None' is
something I will probably never want to do... and therefore definitely have
no desire for special syntax to do it.


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.


On Fri, Dec 27, 2013 at 8:30 AM, David Mertz <mertz at gnosis.cx> wrote:

> It's a fairly standard pattern to see things like this:
>
>     try:
>         import foo
>     except ImportError:
>         foo = None
>
> (and of course, variants with from...import et cetera). These can
> potentially add a lot of clutter to the imports section of a file, given
> that it requires 4 lines to do a conditional import.
>
> It seems like it'd be useful and clean to have a syntax that looked like
> this:
>
>     maybe import foo
>     from bar maybe import baz
>     from qux maybe import quy as quz
>
> Where the behavior would essentially be as above - attempt to run the
> import normally, and in cases where the import fails, map the name to a
> value of None instead. Users who want a different behavior are still free
> to use the long-form syntax. A possibly variant might be to also only run
> the import if the name isn't already bound, so that you could do something
> like...
>
>     from frobber_a maybe import frob as frobber
>     from frobbler_b maybe import frobble as frobber
>     from frobber_c maybe import frobit as frobber
>
> ...to potentially try different fallback options if the first choice for
> an interface provider isn't available.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/208e6bd7/attachment-0001.html>

From grosser.meister.morti at gmx.net  Fri Dec 27 18:31:07 2013
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Fri, 27 Dec 2013 18:31:07 +0100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <CAEbHw4YGwO9m-Z6BFbHkXR6fiHPSESy2xHpYg95c4owjrb19qg@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <CAEbHw4YXPV3e5acMCSELs7gT3fkgJEC-zEnne3SmqH8uE92Mvg@mail.gmail.com>
 <CAEbHw4YGwO9m-Z6BFbHkXR6fiHPSESy2xHpYg95c4owjrb19qg@mail.gmail.com>
Message-ID: <52BDB95B.9050708@gmx.net>


I often do this:

         try:
                 import json
         except ImportError:
                 import simplejson as json

Or:

         try:
                 import optional_feature
         except ImportError:
                 HAS_OPTIONAL_FEATURE = False
         else:
                 HAS_OPTIONAL_FEATURE = True

I think these two cases are fairly common. So why not have something like this?:

         import json or simplejson as json

On 12/27/2013 06:00 PM, David Mertz wrote:
> On Thu, Dec 26, 2013 at 5:57 PM, Amber Yust <amber.yust at gmail.com <mailto:amber.yust at gmail.com>> wrote:
>
>     It's a fairly standard pattern to see things like this:
>
>          try:
>              import foo
>          except ImportError:
>              foo = None
>
>
> I would rarely, if ever, follow this pattern, although I *would* do conditional import fairly often.  E.g. these are
> things I might do (well, not really these specific ones, but just as pattern examples):
>
> try:
>      from math import pi
> except ImportError:
>      pi = 3.1415
>
> Or:
>
> try:
>      import fancymodule as mod
> except ImportError:
>      import simpler_version as mod
>
> Or:
>
> try:
>      from mymod import needed_func
> except ImportError:
>      def needed_func(a,b,c):
>          "Bare bones implementation of more general function"
>          return a*b + c
>
> Or even:
>
> try:
>      import cool_feature
>      FEATURE_AVAIL = True
> except ImportError:
>      FEATURE_AVAIL = False
>
> What these have in common is that they are each specific to the context, and actually have nothing much in common other
> than the possibility import might fail.  The special case of defining a missing module as 'None' is something I will
> probably never want to do... and therefore definitely have no desire for special syntax to do it.
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
>
> On Fri, Dec 27, 2013 at 8:30 AM, David Mertz <mertz at gnosis.cx <mailto:mertz at gnosis.cx>> wrote:
>
>     It's a fairly standard pattern to see things like this:
>
>          try:
>              import foo
>          except ImportError:
>              foo = None
>
>     (and of course, variants with from...import et cetera). These can potentially add a lot of clutter to the imports
>     section of a file, given that it requires 4 lines to do a conditional import.
>
>     It seems like it'd be useful and clean to have a syntax that looked like this:
>
>          maybe import foo
>          from bar maybe import baz
>          from qux maybe import quy as quz
>
>     Where the behavior would essentially be as above - attempt to run the import normally, and in cases where the import
>     fails, map the name to a value of None instead. Users who want a different behavior are still free to use the
>     long-form syntax. A possibly variant might be to also only run the import if the name isn't already bound, so that
>     you could do something like...
>
>          from frobber_a maybe import frob as frobber
>          from frobbler_b maybe import frobble as frobber
>          from frobber_c maybe import frobit as frobber
>
>     ...to potentially try different fallback options if the first choice for an interface provider isn't available.
>


From rymg19 at gmail.com  Fri Dec 27 18:43:42 2013
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 27 Dec 2013 11:43:42 -0600
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <2119483115886762605@gmail297201516>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
Message-ID: <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>

I like that. I usually end up doing that with ElementTree:

try:
    from xml.etree import cElementTree as etree
except:
    try:
        from lxml import etree
    except:
        from xml.etree import ElementTree as etree

It would all be:

import xml.etree.cElementTree or lxml.etree or xml.etree.ElementTree as
etree



On Thu, Dec 26, 2013 at 10:36 PM, Amber Yust <amber.yust at gmail.com> wrote:

> On Thu Dec 26 2013 at 7:40:19 PM, Bruce Leban <bruce at leapyear.org> wrote:
>
>> I think you mean do the import if the name is unbound *or bound to None.* Otherwise,
>> it doesn't work in the example you gave.
>>
>
> Yes, that is what I meant, sorry. Another option would be "not bound to
> something that is not a module" - but I think "unbound or None" is probably
> the most versatile option.
>
> On Thu Dec 26 2013 at 8:08:45 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>
>> I'm not (yet) convinced of the need for this functionality, but if
>> Python did gain this, I think I would prefer the colour of this
>> bike-shed to be "perhaps import" rather than "maybe import".
>
>
> Another option would be to re-use the 'or' keyword:
>
>     from foo import bar or None
>
> Where the bit after the 'or' simply specifies a default value to assign if
> an ImportError occurs.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
When your hammer is C++, everything begins to look like a thumb.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/b8cfb46b/attachment.html>

From g.rodola at gmail.com  Fri Dec 27 19:18:28 2013
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Fri, 27 Dec 2013 19:18:28 +0100
Subject: [Python-ideas] Determine Windows version in platform module
Message-ID: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>

Today I was looking for a way to determine whether I was on Windows >=
Vista and couldn't find anything so I came up with this [1].

That led me to think about platform module.
Honestly I can't see the usefulness of it as it doesn't give you any usable
API to figure out what platform version you're on.
As such would something like [1] be acceptable for inclusion?

[1]
http://stackoverflow.com/questions/12471772/what-is-better-way-of-getting-windows-version-in-python/20804735#20804735

--- Giampaolo
https://code.google.com/p/psutil/
https://code.google.com/p/pyftpdlib/
https://code.google.com/p/pysendfile/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/9bc10a80/attachment.html>

From mal at egenix.com  Fri Dec 27 19:38:53 2013
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Dec 2013 19:38:53 +0100
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
Message-ID: <52BDC93D.9070301@egenix.com>

On 27.12.2013 19:18, Giampaolo Rodola' wrote:
> Today I was looking for a way to determine whether I was on Windows >=
> Vista and couldn't find anything so I came up with this [1].
> 
> That led me to think about platform module.
> Honestly I can't see the usefulness of it as it doesn't give you any usable
> API to figure out what platform version you're on.

Have you had a look at the documentation ?

http://docs.python.org/2.7/library/platform.html?highlight=platform#windows-platform

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From storchaka at gmail.com  Fri Dec 27 19:58:19 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 27 Dec 2013 20:58:19 +0200
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
Message-ID: <l9kik0$t0r$1@ger.gmane.org>

27.12.13 20:18, Giampaolo Rodola' ???????(??):
> Today I was looking for a way to determine whether I was on Windows >=
> Vista and couldn't find anything so I came up with this [1].
>
> That led me to think about platform module.
> Honestly I can't see the usefulness of it as it doesn't give you any
> usable API to figure out what platform version you're on.
> As such would something like [1] be acceptable for inclusion?
>
> [1]
> http://stackoverflow.com/questions/12471772/what-is-better-way-of-getting-windows-version-in-python/20804735#20804735

See also funny issue19143 (http://bugs.python.org/issue19143).


From g.rodola at gmail.com  Fri Dec 27 19:48:52 2013
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Fri, 27 Dec 2013 19:48:52 +0100
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <52BDC93D.9070301@egenix.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
Message-ID: <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>

On Fri, Dec 27, 2013 at 7:38 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 27.12.2013 19:18, Giampaolo Rodola' wrote:
> > Today I was looking for a way to determine whether I was on Windows >=
> > Vista and couldn't find anything so I came up with this [1].
> >
> > That led me to think about platform module.
> > Honestly I can't see the usefulness of it as it doesn't give you any
> usable
> > API to figure out what platform version you're on.
>
> Have you had a look at the documentation ?
>
>
> http://docs.python.org/2.7/library/platform.html?highlight=platform#windows-platform


Yes, on Windows XP I get a tuple like this:

('XP', '5.1.2600', 'SP3', 'Uniprocessor Free')

On Windows 7:

('7', '6.1.7600', '', 'Multiprocessor Free')

Neither of those are helpful to make assumptions such as "if I'm on Windows
>= XP with SP3: do something".
The real deal would be dealing with a tuple of integers in order to use
comparison operators, similarly to sys.version_info:

if sys.version_info >= (3, 0):
     # py3-specific code


--- Giampaolo
https://code.google.com/p/psutil/
https://code.google.com/p/pyftpdlib/
https://code.google.com/p/pysendfile/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/56af69c6/attachment.html>

From abarnert at yahoo.com  Fri Dec 27 21:59:48 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 27 Dec 2013 12:59:48 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
 <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
Message-ID: <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>

On Dec 27, 2013, at 9:43, Ryan Gonzalez <rymg19 at gmail.com> wrote:

> I like that. I usually end up doing that with ElementTree:
> 
> try:
>     from xml.etree import cElementTree as etree

Python 3 has already fixed that. You just import ElementTree and you get the C-accelerated version.

And the same is true for cPickle, etc.

A new language feature that offers a different way to solve something that's already been solved better doesn't seem very useful.

Of course I do write stuff like this when I'm trying to write code that works with 2.6+/3.2+, but a new language feature that's a syntax error in 2.x wouldn't help there either.

> except:
>     try:
>         from lxml import etree
>     except:
>         from xml.etree import ElementTree as etree
> 
> It would all be:
> 
> import xml.etree.cElementTree or lxml.etree or xml.etree.ElementTree as etree
> 
> 
> 
> On Thu, Dec 26, 2013 at 10:36 PM, Amber Yust <amber.yust at gmail.com> wrote:
>> On Thu Dec 26 2013 at 7:40:19 PM, Bruce Leban <bruce at leapyear.org> wrote:
>>> I think you mean do the import if the name is unbound or bound to None. Otherwise, it doesn't work in the example you gave. 
>> 
>> Yes, that is what I meant, sorry. Another option would be "not bound to something that is not a module" - but I think "unbound or None" is probably the most versatile option.
>> 
>> On Thu Dec 26 2013 at 8:08:45 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>>> I'm not (yet) convinced of the need for this functionality, but if
>>> Python did gain this, I think I would prefer the colour of this
>>> bike-shed to be "perhaps import" rather than "maybe import".
>> 
>> Another option would be to re-use the 'or' keyword:
>> 
>>     from foo import bar or None
>> 
>> Where the bit after the 'or' simply specifies a default value to assign if an ImportError occurs. 
>> 
>> 
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> 
> -- 
> Ryan
> When your hammer is C++, everything begins to look like a thumb.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/85579fc3/attachment.html>

From abarnert at yahoo.com  Fri Dec 27 22:09:21 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 27 Dec 2013 13:09:21 -0800
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
Message-ID: <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>

On Dec 27, 2013, at 10:48, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote:

> 
> On Fri, Dec 27, 2013 at 7:38 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 27.12.2013 19:18, Giampaolo Rodola' wrote:
>> > Today I was looking for a way to determine whether I was on Windows >=
>> > Vista and couldn't find anything so I came up with this [1].
>> >
>> > That led me to think about platform module.
>> > Honestly I can't see the usefulness of it as it doesn't give you any usable
>> > API to figure out what platform version you're on.
>> 
>> Have you had a look at the documentation ?
>> 
>> http://docs.python.org/2.7/library/platform.html?highlight=platform#windows-platform
> 
> Yes, on Windows XP I get a tuple like this:
> 
> ('XP', '5.1.2600', 'SP3', 'Uniprocessor Free')
> 
> On Windows 7:
> 
> ('7', '6.1.7600', '', 'Multiprocessor Free')
> 
> Neither of those are helpful to make assumptions such as "if I'm on Windows >= XP with SP3: do something".
> The real deal would be dealing with a tuple of integers in order to use comparison operators, similarly to sys.version_info:
> 
> if sys.version_info >= (3, 0):
>      # py3-specific code
> 

So you want to write:

    if platform.win_version_info >= (5, 1, 2600):

Is that really much better than:

    if platform.win32_ver.version >= '5.1.2600':

There are a few cases with older windows versions where the third component is 3 digits, and it's always possible that could be true again, but I don't think it affects anything you're likely to be checking for in 2014.

If you want some different number... then what? It would be nice if Microsoft had a nice numbering scheme so
Windows 7 was 7 rather than 6.1, 2003 wasn't sometimes 5.1 and sometimes 5.2, etc. But they don't.

Meanwhile, Microsoft's suggested solution is that you not check for version numbers at runtime, but instead check for features. And there are good reasons for that--for example, XPSP3 N has exactly the same version numbers as XPSP3 normal, but if you try to use the new Windows Media APIs, you'll crash.

> 
> --- Giampaolo
> https://code.google.com/p/psutil/
> https://code.google.com/p/pyftpdlib/
> https://code.google.com/p/pysendfile/ 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131227/f166dbaa/attachment-0001.html>

From grosser.meister.morti at gmx.net  Fri Dec 27 22:03:09 2013
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Fri, 27 Dec 2013 22:03:09 +0100
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
Message-ID: <52BDEB0D.1040106@gmx.net>

On 12/27/2013 07:48 PM, Giampaolo Rodola' wrote:
>
> On Fri, Dec 27, 2013 at 7:38 PM, M.-A. Lemburg <mal at egenix.com <mailto:mal at egenix.com>> wrote:
>
>     On 27.12.2013 19:18, Giampaolo Rodola' wrote:
>      > Today I was looking for a way to determine whether I was on Windows >=
>      > Vista and couldn't find anything so I came up with this [1].
>      >
>      > That led me to think about platform module.
>      > Honestly I can't see the usefulness of it as it doesn't give you any usable
>      > API to figure out what platform version you're on.
>
>     Have you had a look at the documentation ?
>
>     http://docs.python.org/2.7/library/platform.html?highlight=platform#windows-platform
>
>
> Yes, on Windows XP I get a tuple like this:
>
> ('XP', '5.1.2600', 'SP3', 'Uniprocessor Free')
>
> On Windows 7:
>
> ('7', '6.1.7600', '', 'Multiprocessor Free')
>
> Neither of those are helpful to make assumptions such as "if I'm on Windows >= XP with SP3: do something".
> The real deal would be dealing with a tuple of integers in order to use comparison operators, similarly to sys.version_info:
>
> if sys.version_info >= (3, 0):
>       # py3-specific code
>
>

What about this?:

     if tuple(int(v) for v in platform.win32_ver[1].split(".")) >= (5,1,2600):
         ...

From skip at pobox.com  Fri Dec 27 22:36:56 2013
From: skip at pobox.com (Skip Montanaro)
Date: Fri, 27 Dec 2013 15:36:56 -0600
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
 <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
Message-ID: <CANc-5Uz9GztVSyfP+v4nLqsbesimvGdydc2v40C96YdZcgDPwg@mail.gmail.com>

On Fri, Dec 27, 2013 at 3:09 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> So you want to write:
>
>     if platform.win_version_info >= (5, 1, 2600):
>
> Is that really much better than:
>
>     if platform.win32_ver.version >= '5.1.2600':

Once either of the first two numbers rolls over to two digits, the
first comparison will be correct, the second will not:

>>> x = '5.9.2600'
>>> y = '5.10.2600'
>>> y > x
False

Skip

From cs at zip.com.au  Sat Dec 28 01:15:42 2013
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 28 Dec 2013 11:15:42 +1100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <-7047415593530210756@gmail297201516>
References: <-7047415593530210756@gmail297201516>
Message-ID: <20131228001542.GA35838@cskk.homeip.net>

On 27Dec2013 01:57, Amber Yust <amber.yust at gmail.com> wrote:
> It's a fairly standard pattern to see things like this:
> 
>     try:
>         import foo
>     except ImportError:
>         foo = None
> 
> (and of course, variants with from...import et cetera). These can
> potentially add a lot of clutter to the imports section of a file, given
> that it requires 4 lines to do a conditional import.
> 
> It seems like it'd be useful and clean to have a syntax that looked like
> this:
> 
>     maybe import foo
[...snip...]

The problem here is that the only reason to import as above is to
have access to the name "foo".  What is you contingency plan for
the failed import? How does the code cope without "foo".

I'd rather have an ImportError at start than an arbitrarily delayed and
harder to diagnose NameError much later.

Because of this, I'd argue you should almost always want the except:
clause if this is realistic, because you need to have a plan for
the case of failure.

I agree the try/except is wordy and possibly ugly, but I think it
is usually needed when a failed import is not to be fatal.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Every particle continues in its state of rest or uniform motion in a straight
line except insofar as it doesn't.      - Sir Arther Eddington

From abarnert at yahoo.com  Sat Dec 28 04:26:44 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 27 Dec 2013 19:26:44 -0800
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <CANc-5Uz9GztVSyfP+v4nLqsbesimvGdydc2v40C96YdZcgDPwg@mail.gmail.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
 <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
 <CANc-5Uz9GztVSyfP+v4nLqsbesimvGdydc2v40C96YdZcgDPwg@mail.gmail.com>
Message-ID: <453B4FE1-833E-4EB5-BEE6-698E78781EA2@yahoo.com>

On Dec 27, 2013, at 13:36, Skip Montanaro <skip at pobox.com> wrote:

> On Fri, Dec 27, 2013 at 3:09 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> So you want to write:
>> 
>>    if platform.win_version_info >= (5, 1, 2600):
>> 
>> Is that really much better than:
>> 
>>    if platform.win32_ver.version >= '5.1.2600':
> 
> Once either of the first two numbers rolls over to two digits, the
> first comparison will be correct, the second will not:
> 
>>>> x = '5.9.2600'
>>>> y = '5.10.2600'
>>>> y > x

Sure, but in decades of Windows versions it's never come even close to rolling over. Other than them renaming NT 3.2 as 3.5, they've never even gotten past .3. So, do you really think you need to worry about that? It seems just as plausible that they'd completely change their versioning scheme as that they'd ever go to 6.10 under the current scheme. Plus, it would be about 15 years away at their historical version numbering rates.

From guido at python.org  Sat Dec 28 04:30:41 2013
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Dec 2013 19:30:41 -0800
Subject: [Python-ideas] PEP draft - Composable futures for reactive
	programming
In-Reply-To: <CAPQPjQEB-eeDZKa0srsS1avMcMSWYB_rn+nzOSyKc5Hf3NfZNQ@mail.gmail.com>
References: <CAPQPjQFfwgUHsrxY=La8BDnGvEmcfGrx9GnUckenUiUq1SPwng@mail.gmail.com>
 <CADiSq7dMbX5YRy8QX36-0hytnMuAPDP9p7UyRtkCZTNMk9TE9w@mail.gmail.com>
 <CAP7+vJL8ydn1QhuZbLOWiH=cGvTR=mi_MZv2AB0m41mx-LZV1A@mail.gmail.com>
 <CAPQPjQHHy3hwf71Eva1fcKANQLUj8W0vk+Y5uPHQ0MKnf1AnHQ@mail.gmail.com>
 <CAP7+vJJKSrEqpJ+8fpQz+HxiCmjz=EUtvP7fFJsHDLN-54g-DA@mail.gmail.com>
 <CAFkYKJ7r3D7n_OEyFXJnkKWfg9kyn_08JWkWDsJNvqi7GjTExA@mail.gmail.com>
 <CAP7+vJ+S=mLSM7HydDJxUK5e1EKV6BL5DXndyULD0iyA+KtAHA@mail.gmail.com>
 <CAFkYKJ7wt7q3N9-RNam1XtLa9UEmakO3OvT-2xVo5mauZBsdzA@mail.gmail.com>
 <CAP7+vJJ4LpyPnejY1CjRpVoX0WKj9WOrTc+drfvanah-QW1-yQ@mail.gmail.com>
 <CAPQPjQEB-eeDZKa0srsS1avMcMSWYB_rn+nzOSyKc5Hf3NfZNQ@mail.gmail.com>
Message-ID: <CAP7+vJJAscipV8LR724B9j8ntNSZs9XKgkSf__dimG9YFB-GFQ@mail.gmail.com>

Hi Sergii,

I'm trying to give some constructive criticism here, please bear with me.

The biggest issue perhaps seems to me that the unification between
concurrent and threaded Futures still feels uncomfortable to me. A
symptom is the completely different semantics of result() -- when the
result isn't ready yet, this either raises an exception or blocks the
current thread, and that makes reasoning about what will happen
difficult.

A lesser issue is naming -- I read some earlier example code you
posted, and I couldn't understand it, because the names for the new
operations you added are pretty arbitrary. Especially grating is your
reusing some well-known names of built-in Python functions for
different purposes, the worst offender being map(), but all() isn't so
great either. In general your FutureBaseExt class (also an awkward
name IMO) introduces a bunch of new functions with a wide variety of
functionality that seems to have little logic to it. Why this set of
functions and not another?

A separate question is why the distinction between FutureBase and FutureBaseExt.

It seems you copied some phrases from the asyncio docs or PEP 3156 --
e.g. add_done_callback() references call_soon(); this seems incorrect
for threaded Futures.

The definition of an Executor seems incomplete (SynchronousExecutor is
referenced but not defined), and very vague -- I don't believe that
making it just a callable suffices for the functionality. There is
also a mention of global configuration of a default executor by
assigning to config.Default.CALLBACK_EXECUTOR, which seems a bad idea
-- I'm sure a lot of code will in practice depend on the choice of
executor.

Another issue: why the try_* methods?

Finally, I'm not sure I am convinced by your motivation section. Or,
at least, I'd like you to address how your proposal addresses each of
the bullets in your motivation, with some examples.

(I may have more, but at the current rate it would take me a day per
paragraph, so I'll get to more later.)

--Guido

On Tue, Dec 24, 2013 at 8:56 AM, Sergii Mikhtoniuk <mikhtonyuk at gmail.com> wrote:
> Thanks everyone for your feedback.
>
> Taking all your suggestions into account I have revised my proposal.
>
> In short, it?s now:
> - defines separate Future classes for cooperative and multithreaded cases in
> concurrent.futures package
> - multithreaded implementation adds thread-safety to basic implementation,
> so in cooperative concurrency case there is absolutely no overhead
> - cooperative future?s interface is identical to asyncio.future
> - asyncio.Future inherits from concurrent.futures.cooperative.Future adding
> only methods specific to `yield from`
> - adds common composition methods for futures (intended to replace and
> enhance asyncio.wait/gather and concurrent.futures.wait)
>
> There?s still some work to be done for backward compatibility of
> concurrent.futures.Future, but implementation is almost ready.
>
> Would really appreciate if you could take a look.
>
>
> Thanks,
> Sergii
>
>
>
> On Mon, Dec 23, 2013 at 12:42 AM, Guido van Rossum <guido at python.org> wrote:
>>
>> Aha. That is clever. I will have to look into the details more, but the
>> idea is promising. Sorry I didn't see tty his before.
>>
>> On Dec 22, 2013 12:30 PM, "Ben Darnell" <ben at bendarnell.com> wrote:
>>>
>>> On Sat, Dec 21, 2013 at 10:45 PM, Guido van Rossum <guido at python.org>
>>> wrote:
>>>>
>>>> There's still the issue that in the threading version, you wait for a
>>>> Future by blocking the current thread, while in the asyncio version,
>>>> you must use "yield from" to block. For interoperability you would
>>>> have to refrain from *any* blocking operations (including "yield
>>>> from") so you would only be able to use callbacks. But whether you had
>>>> to write "x = f.result()" or "x = concurrent.futures.wait_for(f)",
>>>> either way you'd implicitly be blocking the current thread.
>>>
>>>
>>> Threaded *consumers* of Futures wait for them by blocking, while
>>> asynchronous consumers wait for them by yielding.  It doesn't matter whether
>>> the *producer* of the Future is threaded or asynchronous (except that if you
>>> know you won't be using threads you can use a faster thread-unsafe Future
>>> implementation).
>>>
>>> -Ben
>>>
>>>>
>>>>
>>>> Yes, a clever scheduler could run other callbacks while blocking, but
>>>> that's not a complete solution, because another callback might do a
>>>> similar blocking operation, and whatever that waited for could hold up
>>>> the earlier blocking operation, causing an ever-deeper recursion and
>>>> of event loop invocations that might never complete. (I've had to
>>>> debug this in production code.) To cut through that you'd have to have
>>>> some kind of stack-swapping coroutine implementation like gevent, or a
>>>> syntactic preprocessor that inserts yield or yield-from operations
>>>> (I've heard from people who do this), or you'd need a clairvoyant
>>>> scheduler that would know which callbacks won't block.
>>>>
>>>> I like the C# solution, but it depends on static typing so a compiler
>>>> can know when to emit the coroutine interactions. That wouldn't work
>>>> in Python, unless you made the compiler recognizing the wait_for()
>>>> operation by name, which feels unsavory (although we do it for super()
>>>> :-).
>>>>
>>>> I guess for extreme interop, callbacks that never block is your only
>>>> option anyway, but I'd be sad if we had to to recommend this as the
>>>> preferred paradigm, or claim that it is all you need.
>>>>
>>>> --Guido (if I don't respond to this thread for the next two weeks,
>>>> it's because I'm on vacation :-)
>>>>
>>>>
>>>> On Sat, Dec 21, 2013 at 5:37 PM, Ben Darnell <ben at bendarnell.com> wrote:
>>>> > On Sat, Dec 21, 2013 at 7:26 PM, Guido van Rossum <guido at python.org>
>>>> > wrote:
>>>> >>
>>>> >> On Sat, Dec 21, 2013 at 2:48 PM, Sergii Mikhtoniuk
>>>> >> <mikhtonyuk at gmail.com>
>>>> >> wrote:
>>>> >> > Indeed there is a lot of overlap between asyncio and
>>>> >> > concurrent.futures
>>>> >> > packages, so it would be very interesting to hear your overall
>>>> >> > thoughts
>>>> >> > on
>>>> >> > role/future of concurrent package. Do you consider it rudimentary
>>>> >> > and
>>>> >> > replaceable by asyncio.Future completely?
>>>> >>
>>>> >> They don't really compare. concurrent.futures is about *threads*.
>>>> >> asyncio.Future is about *avoiding* threads in favor of more
>>>> >> lightweight "tasks" and "coroutines", which in turn are built on top
>>>> >> of lower-level callbacks.
>>>> >
>>>> >
>>>> > concurrent.futures.ThreadPoolExecutor is about threads; the Future
>>>> > class
>>>> > itself is broader.  When I integrated Futures into Tornado I used
>>>> > concurrent.futures.Future directly (when available).  asyncio.Future
>>>> > is just
>>>> > an optimized version of c.f.Future (the optimization comes from
>>>> > assuming
>>>> > single-threaded usage).  There should at least be a common ABC between
>>>> > them.
>>>> >
>>>> >>
>>>> >>
>>>> >> (While I want to get away from callbacks as a programming paradigm,
>>>> >> asyncio uses them at the lower levels both because they are a logical
>>>> >> low-level building block and for interoperability with other
>>>> >> frameworks like Tornado and Twisted.)
>>>> >>
>>>> >> > I think the question is even not much about which is your preferred
>>>> >> > implementation, but rather do we want having futures as stand-alone
>>>> >> > package
>>>> >> > or not. Do you see all these implementations converging in future?
>>>> >>
>>>> >> I see them as not even competing. They use different paradigms and
>>>> >> apply to different use cases.
>>>> >>
>>>> >> > To me Future is a very simple and self-contained primitive,
>>>> >> > independent
>>>> >> > of
>>>> >> > thread pools, processes, IO, and networking.
>>>> >>
>>>> >> (Agreed on the I/O and networking part only.)
>>>> >>
>>>> >> > One thing concurrent.futures
>>>> >> > package does a very good job at is defining an isolated namespace
>>>> >> > for
>>>> >> > futures, stressing out this clear boundary (let?s disregard here
>>>> >> > ThreadPoolExecutor and ProcessPoolExecutor classes which for some
>>>> >> > reason
>>>> >> > ended up in it too).
>>>> >>
>>>> >> Actually the executor API is an important and integral part of that
>>>> >> package, and threads underlie everything you can do with its Futures.
>>>> >>
>>>> >> > So when I think of schedulers and event loops implementations I see
>>>> >> > them
>>>> >> > as
>>>> >> > ones that build on top of the Future primitive, not providing it as
>>>> >> > part
>>>> >> > of
>>>> >> > their implementation. What I think is important is that having
>>>> >> > unified
>>>> >> > Future class simplifies interoperability between different kinds of
>>>> >> > schedulers, not necessarily associated with asyncio event loops
>>>> >> > (process
>>>> >> > pools for example).
>>>> >>
>>>> >> The interoperability is completely missing. I can't tell if you've
>>>> >> used asyncio at all, but the important operation of *waiting* for a
>>>> >> result is fundamentally different there than in concurrent.futures.
>>>> >> In
>>>> >> the latter, you just write "x = f.result()" and your thread blocks
>>>> >> until the result is available. In asyncio, you have to write "x =
>>>> >> yield from f.result()" which is a coroutine block that lets other
>>>> >> tasks run in the same thread. ("yield from" in this case is how
>>>> >> Python
>>>> >> spells the operation that C# calls "await").
>>>> >
>>>> >
>>>> > The way I see it, the fundamental operation on Futures is
>>>> > add_done_callback.
>>>> > We then have various higher-level operations that let us get away from
>>>> > using
>>>> > callbacks directly.  One of these happens to be a method on Future:
>>>> > the
>>>> > blocking mode of Future.result().  Another is implemented in asyncio
>>>> > and
>>>> > Tornado, in the ability to "yield" a Future.  The blocking mode of
>>>> > result()
>>>> > is just a convenience; if asyncio-style futures had existed first then
>>>> > we
>>>> > could instead have a function like "x =
>>>> > concurrent.futures.wait_for(f)".  In
>>>> > fact, you could write this wait_for function today in a way that works
>>>> > for
>>>> > both concurrent and asyncio futures.
>>>> >
>>>> > This is already interoperable:  Tornado's Resolver interface
>>>> >
>>>> > (https://github.com/facebook/tornado/blob/master/tornado/netutil.py#L184)
>>>> > returns a Future, which may be generated by a ThreadPoolExecutor or an
>>>> > asynchronous wrapper around pycares or twisted.  In the other
>>>> > direction I've
>>>> > worked on hybrid apps that have one Tornado thread alongside a bunch
>>>> > of
>>>> > Django threads; in these apps it would work to have a Django thread
>>>> > block on
>>>> > f.result() for a Future returned by Tornado's AsyncHTTPClient.
>>>> >
>>>> > -Ben
>>>> >
>>>> >>
>>>> >> > Futures are definitely not the final solution for concurrency
>>>> >> > problem
>>>> >> > but
>>>> >> > rather a well-established utility for representing async
>>>> >> > operations. It
>>>> >> > is a
>>>> >> > job of higher layer systems to provide more convenient ways for
>>>> >> > dealing
>>>> >> > with
>>>> >> > asyncs (yield from coroutines, async/await rewrites etc.),
>>>> >>
>>>> >> But unless you are proposing some kind of radical change to add
>>>> >> compile-time type checking/inference to Python, the rewrite option is
>>>> >> unavailable in Python.
>>>> >>
>>>> >> > so I would not
>>>> >> > say that futures encourage callback-style programming,
>>>> >>
>>>> >> The concurrent.futures.Future class does not. But unless I misread
>>>> >> your proposal, your extensions do.
>>>> >>
>>>> >> > it?s simply a
>>>> >> > lower-layer functionality. On the contrary, monadic
>>>> >>
>>>> >> (Say that word one more time and everyone tunes out. :-)
>>>> >>
>>>> >> > methods for futures
>>>> >> > composition (e.g. map(), all(), first() etc.) ensure that no errors
>>>> >> > would be
>>>> >> > lost in the process, so I think they would complement yield from
>>>> >> > model
>>>> >> > quite
>>>> >> > nicely by hiding complexity of state maintenance from user and
>>>> >> > reducing
>>>> >> > the
>>>> >> > number of back-and-forth communications between event loop and
>>>> >> > coroutines.
>>>> >>
>>>> >> I'm not sure I follow. Again, I'm not sure if you've actually written
>>>> >> any code using asyncio.
>>>> >>
>>>> >> TBH I've written a fair number of example programs for asyncio and
>>>> >> I've very rarely felt the need for these composition functions. The
>>>> >> main composition primitive I tend to use is "yield from".
>>>> >>
>>>> >> > Besides Futures, reactive programming
>>>> >>
>>>> >> What *is* reactive programming? If you're talking about
>>>> >> http://en.wikipedia.org/wiki/Reactive_programming,
>>>> >> I'm not sure that it maps well to Python.
>>>> >>
>>>> >> > has more utilities to offer, such as
>>>> >> > Observables (representing asynchronous streams of values). It is
>>>> >> > also a
>>>> >> > very
>>>> >> > useful abstraction with a rich set of composition strategies
>>>> >> > (merging,
>>>> >> > concatenation, grouping), and may deserve its place in separate
>>>> >> > package.
>>>> >>
>>>> >> It all sounds very abstract and academic. :-)
>>>> >>
>>>> >> > Hope to hear back from you to get better picture on overall design
>>>> >> > direction
>>>> >> > here before jumping to implementation details.
>>>> >> >
>>>> >> > Brief follow-up to your questions:
>>>> >> >  - Idea behind the Future/Promise separation is to draw a clean
>>>> >> > line
>>>> >> > between
>>>> >> > client-facing and scheduler-side APIs respectively. Futures should
>>>> >> > not
>>>> >> > expose any completion methods, which clients should not call
>>>> >> > anyway.
>>>> >>
>>>> >> Given that Future and Promise are often used synonymously, using them
>>>> >> to make this distinction sounds confusing. I agree that Futures have
>>>> >> two different APIs, one for the consumer and another for the
>>>> >> producer.
>>>> >> But I'm not sure that it's necessary to separate them more strictly
>>>> >> --
>>>> >> convention seems good enough to me here. (It's the same with many
>>>> >> communication primitives, like queues and even threads.)
>>>> >>
>>>> >> (The one thing that trips people up frequently is that, while
>>>> >> set_result() and set_exception() are producer APIs, cancel() is a
>>>> >> consumer API, and the cancellation signal travels from the consumer
>>>> >> to
>>>> >> the producer.)
>>>> >>
>>>> >> >  - Completely agree with you that Twisted-style callbacks are evil
>>>> >> > and
>>>> >> > it is
>>>> >> > better to have single code path for getting result or raising
>>>> >> > exception
>>>> >> >  - Sorry for bad grammar in the proposal, it?s an early draft
>>>> >> > written in
>>>> >> > 3
>>>> >> > AM, so I will definitely improve on it if we decide to move
>>>> >> > forward.
>>>> >>
>>>> >> No problem!
>>>> >>
>>>> >> > Thanks,
>>>> >> > Sergii
>>>> >>
>>>> >> --
>>>> >> --Guido van Rossum (python.org/~guido)
>>>> >> _______________________________________________
>>>> >> Python-ideas mailing list
>>>> >> Python-ideas at python.org
>>>> >> https://mail.python.org/mailman/listinfo/python-ideas
>>>> >> Code of Conduct: http://python.org/psf/codeofconduct/
>>>> >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> --Guido van Rossum (python.org/~guido)
>>>
>>>
>



-- 
--Guido van Rossum (python.org/~guido)

From abarnert at yahoo.com  Sat Dec 28 04:38:09 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 27 Dec 2013 19:38:09 -0800
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
 <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
Message-ID: <2F2A1829-56A7-4571-857C-E6010CE5C58B@yahoo.com>

On Dec 27, 2013, at 13:09, Andrew Barnert <abarnert at yahoo.com> wrote:

> Meanwhile, Microsoft's suggested solution is that you not check for version numbers at runtime, but instead check for features. And there are good reasons for that--for example, XPSP3 N has exactly the same version numbers as XPSP3 normal, but if you try to use the new Windows Media APIs, you'll crash.

Also, in forgot to mention that there are features added to Windows versions discontinuously. For example, there are features that are in XPSP3 (5.1.2600) and 2003R2SP2 (5.2.3790) but not original 2003 (5.2.3376). So, a check for >= 5, 1, 2600 would give you the wrong information anyway.

And that's not even considering the fact that some kernels are used for multiple different Windows versions with different features (e.g., that 5.2.3790 is also the kernel for 64-bit XP). To make things even more fun, 2003SP2 originally had the same 3376 kernel as stock 2003, then got bumped to the same 3790 kernel as 2003R2SP2 in a non-service-pack update.

These aren't anomalies; this is the way Windows versioning works. The kernel build number is useless as an indication of anything but kernel features that Python code isn't going to care about; the major.minor can be useful in a few cases, but usually even that doesn't tell you as much as you probably expect.

Any change to make it easier to write misleadingly broken code is probably not a good change.

From steve at pearwood.info  Sat Dec 28 13:27:00 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 28 Dec 2013 23:27:00 +1100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
 <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
 <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>
Message-ID: <20131228122700.GV29356@ando>

On Fri, Dec 27, 2013 at 12:59:48PM -0800, Andrew Barnert wrote:
> On Dec 27, 2013, at 9:43, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> 
> > I like that. I usually end up doing that with ElementTree:
> > 
> > try:
> >     from xml.etree import cElementTree as etree
> 
> Python 3 has already fixed that. You just import ElementTree and you 
> get the C-accelerated version.
>
> And the same is true for cPickle, etc.


Not if the C-accelerated version isn't available. Or perhaps there are 
two pure-Python modules with the same API, and you're not sure which is 
installed. This issue is bigger than just C-accelerators in the CPython 
standard library. It applies any time you have two or more alternatives 
with the same API, a preferred module and an fallback module. E.g. 
readline in the stdlib, versus pyreadline, a third-party Windows port.


A point against this suggestion: the best syntactic sugar opens up new 
and successful idioms. List comps and decorators have been great 
successes. Being able to embed a list comp as an expression, instead of 
a for-loop as a statement, is a great win. This new syntax doesn't open 
up new idioms for writing code. It is purely sugar, and as such, it 
isn't such a compelling feature.

On the other hand, I think it does simplify a very common use-case. I 
often use fallback modules with the same API, and if only this new 
syntax had been introduced back in Python 2.4 or 2.5 I would be using it 
all the time now. Unfortunately, most (but not all) of my code has to 
support 2.4 or 2.5, or at least 3.3, so the usefulness of new syntax is 
severely limited *right now*. But if this syntax were introduced into 
Python 3.5 (its surely too late for 3.4) I reckon I would be able to use 
it around Python 3.6 or 3.7. And of course those who won't need to 
support versions of Python prior to 3.5 could use it straight away.

So don't think of this as something you are going to use. Think of this 
as an investment for future Python users who are starting with 3.5 and 
don't care about backwards compatibility with Python 3.3 and 3.4 (let 
alone older versions).

On that basis, I'm going to vote:

+1 on "import preferred or fallback as name".

+0 on allowing "import module or None", to cover the idiom:

    try:
        import module
    except ImportError:
        module = None

which may not be encouraged, but it is occasionally useful.

-0 on "from preferred or fallback import name", simply because I'm not 
sure how to extend it to multiple names, so I'd rather leave it out.

-1 on Amber's original suggestion "maybe import name", as the 
benefit is too little to justify introducing a new keyword.


-- 
Steven

From ncoghlan at gmail.com  Sat Dec 28 14:30:25 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Dec 2013 23:30:25 +1000
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <20131228122700.GV29356@ando>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
 <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
 <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>
 <20131228122700.GV29356@ando>
Message-ID: <CADiSq7ckKSr_Gs06KsN=addpJcr6PG_ib-ZtUitrAv6HnAO=fg@mail.gmail.com>

On 28 December 2013 22:27, Steven D'Aprano <steve at pearwood.info> wrote:
> So don't think of this as something you are going to use. Think of this
> as an investment for future Python users who are starting with 3.5 and
> don't care about backwards compatibility with Python 3.3 and 3.4 (let
> alone older versions).
>
> On that basis, I'm going to vote:
>
> +1 on "import preferred or fallback as name".
>
> +0 on allowing "import module or None", to cover the idiom:
>
>     try:
>         import module
>     except ImportError:
>         module = None
>
> which may not be encouraged, but it is occasionally useful.
>
> -0 on "from preferred or fallback import name", simply because I'm not
> sure how to extend it to multiple names, so I'd rather leave it out.
>
> -1 on Amber's original suggestion "maybe import name", as the
> benefit is too little to justify introducing a new keyword.

I used to be a fan of the "or" based variants, but I eventually went
off the idea. My reasoning went as follows:

1. For a single file script, you're only doing the import once, so
there's no repetition, and hence little to be gained from the more
concise syntax (and the cost of a new thing for readers to learn)

2. For a multi file package or application, it's trivial to factor out
the compatibility checking to a helper module, so there's *still* no
need for repetition. A compatibility module also lets you build an
adaptation layer if the APIs aren't *quite* the same (for example,
RHEL5 has the ancient pre-simplejson JSON module, so naive "import
json" code will mistakenly think it has the 2.6+ stdlib version - a
helper module can detect oddball discrepancies like that one and work
around them, but naive syntax would do the wrong thing if a similar
situation arose again in the future).

Essentially, the niche that the syntax variants can cover is too small
to be worth making the already horrendously complex import statement
syntax and semantics even *more* complicated, when you can either
spell it out explicitly in the single file case, or use a helper
module to centralise the selection logic in the multiple file case.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From denis.spir at gmail.com  Sat Dec 28 15:59:52 2013
From: denis.spir at gmail.com (spir)
Date: Sat, 28 Dec 2013 15:59:52 +0100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <20131227040808.GP29356@ando>
References: <-7047415593530210756@gmail297201516> <20131227040808.GP29356@ando>
Message-ID: <52BEE768.7060401@gmail.com>

On 12/27/2013 05:08 AM, Steven D'Aprano wrote:
> Hmmm. The basic idea makes a certain level of sense to me, but I'm not
> sure it makes enough sense to overcome the barrier required before
> adding a new keyword.

Same for me. In fact, while the use case is clear and common, I'm not sure 
syntax it's worth dedicated syntax at all, especially because this happens once 
in a module and at a definite place (import section).

> [...]
>
> If we're entertaining changes to imports, another possibility would be
> to allow fallback module names:
>
>      import this or that or another as this

I like that form.

> Each of "this", "that", "another" will be attempted, the first
> successful import being bound to the name "this". The "as this" part
> would be mandatory, so as to require a consistent name regardless of
> which module was imported. This would be a syntax error, since it isn't
> clear what name would be bound at the end:
>
>      import this or that or another

Right.

> This would also be allowed:
>
>      from this or that or another import spam

We'd also, or rather, need:

	from this import foo or that import bar or another import baz as spam

which starts to be a bit complicated... Maybe (or perhaps) with parens:

	from ((this import foo) or (that import bar) or (another import baz)) as spam

?

> With this syntax, we could add None as a special case:
>
>      import this or that or another or None as this

Yep!

> would be equivalent to:
>
>      module_names = ("this", "that", "another", "None")
>      for name in module_names:
>          if name == "None":
>              spam = None
>          else:
>              try:
>                  this = __import__(name)
>              except ImportError:
>                  continue
>          break
>      else:
>          raise ImportError

Seems correct, expected semantics.

> and the "from...import" case could be written as:
>
>      from this or that or None import spam
>
>
> roughly equivalent to:
>
>      module_names = ("this", "that", "another", "None")
>      for name in module_names:
>          if name == "None":
>              spam = None
>          else:
>              try:
>                  temp = __import__(name)
>                  spam = temp.spam
>              except ImportError:
>                  continue
>          break
>      else:
>          raise ImportError

Right.

> Advantages:
>
> - covers both use-cases where you want to try a series of
>    modules, and the one where you fall back to None;

Plus, as shown above we need to cover the case were a imported element have 
various names in original module. (Either all cases are covered, or it's 
definitely not worth bothering, imo.)

> - "or" is already a keyword, no new keywords needed;
>
> - reads more like English;

Not clearer than "maybe", in my ears (sic!) but I'm not english native speaker.

> - "import None" currently gives SyntaxError, so this
>    can't interfere with modules actually called "None".
>
>
> Disadvantages:
>
> - more complexity to imports;
>
> - only saves a few lines;
>
> - this usage of "or" is not quite the same as the usage
>    as a boolean operator, e.g. different from "x in a or b".

Yes, it's more like prioritized choice in parsing (an operand in either a symbol 
or a real or an int), which is at times termed 'or' as well.

Denis

From rosuav at gmail.com  Sat Dec 28 17:10:10 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 29 Dec 2013 03:10:10 +1100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <52BEE768.7060401@gmail.com>
References: <-7047415593530210756@gmail297201516> <20131227040808.GP29356@ando>
 <52BEE768.7060401@gmail.com>
Message-ID: <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>

On Sun, Dec 29, 2013 at 1:59 AM, spir <denis.spir at gmail.com> wrote:
>> This would also be allowed:
>>
>>      from this or that or another import spam
>
>
> We'd also, or rather, need:
>
>         from this import foo or that import bar or another import baz as
> spam
>
> which starts to be a bit complicated... Maybe (or perhaps) with parens:
>
>         from ((this import foo) or (that import bar) or (another import
> baz)) as spam

Can't say I like that variant - getting complicated and hard to read,
even with parens. If you're importing different names from different
modules, it's going to get very long, and then people are going to
want to split it across lines, and then the obvious question is:
what's been gained over the try/except variant?

There is one thing, though, that I'm seeing of all this. Exception
throwing is asymmetrical: you can attempt a series of statements until
one fails, but there's no convenient syntax to attempt a series of
statements until one succeeds. I wonder, could the more general case
be solved? Is there a way to, without stupid stuff like eval, wrap up
a few statements so they can be executed in a loop:

def import_any(statement_list):
    for try_me in statement_list:
        try:
            # uhh, this is the bit I'm not sure about...
            try_me() # this would work if they're functions instead!
            return
        except ImportError:
            pass
    raise ImportError

This works for a set of functions, but not for a bunch of "from this
import that" statements. Would it be worth mangling the top of your
script until it can be done with importlib or __import__? Doesn't seem
nearly as clean, somehow.

ChrisA

From g.rodola at gmail.com  Sat Dec 28 17:12:57 2013
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 28 Dec 2013 17:12:57 +0100
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
 <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
Message-ID: <CAFYqXL8HFJs6txLFaGxEakNsOpTtQyU3CH5J-5eooxbMJjq80A@mail.gmail.com>

> Meanwhile, Microsoft's suggested solution is that you not check for
> version numbers at runtime, but instead check for features. And there are
> good reasons for that--for example, XPSP3 N has exactly the same version
> numbers as XPSP3 normal, but if you try to use the new Windows Media APIs,
> you'll crash.
>

Mmm. What's weird is that if you use MSDN as a reference every Windows API
has a "minimum supported client" section near the bottom of the page.
See for example:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms684874(v=vs.85).aspx
To me that suggests that (most of the times at least) you are encouraged to
use the windows version to decide whether to support a feature or not.


--- Giampaolo
https://code.google.com/p/psutil/
https://code.google.com/p/pyftpdlib/
https://code.google.com/p/pysendfile/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131228/b0b063d0/attachment-0001.html>

From amber.yust at gmail.com  Sat Dec 28 20:14:28 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Sat, 28 Dec 2013 19:14:28 +0000
Subject: [Python-ideas] "maybe import"?
References: <-7047415593530210756@gmail297201516> <20131227040808.GP29356@ando>
 <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
Message-ID: <360207118627343726@gmail297201516>

I'm fairly convinced by the "too small" reasoning (summed up well by Nick).
I could potentially see a small and not incredibly intrusive improvement of
allowing imports from a tuple-style group of alternatives being useful for
things like c-accelerators, e.g...

    import (accelerated_foo, alternative_foo) as foo
    from (accelerated_foo, alternative_foo) import bar

But that would be a "nice to have" thing rather than a big difference.

On Sat Dec 28 2013 at 8:10:47 AM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Dec 29, 2013 at 1:59 AM, spir <denis.spir at gmail.com> wrote:
> >> This would also be allowed:
> >>
> >>      from this or that or another import spam
> >
> >
> > We'd also, or rather, need:
> >
> >         from this import foo or that import bar or another import baz as
> > spam
> >
> > which starts to be a bit complicated... Maybe (or perhaps) with parens:
> >
> >         from ((this import foo) or (that import bar) or (another import
> > baz)) as spam
>
> Can't say I like that variant - getting complicated and hard to read,
> even with parens. If you're importing different names from different
> modules, it's going to get very long, and then people are going to
> want to split it across lines, and then the obvious question is:
> what's been gained over the try/except variant?
>
> There is one thing, though, that I'm seeing of all this. Exception
> throwing is asymmetrical: you can attempt a series of statements until
> one fails, but there's no convenient syntax to attempt a series of
> statements until one succeeds. I wonder, could the more general case
> be solved? Is there a way to, without stupid stuff like eval, wrap up
> a few statements so they can be executed in a loop:
>
> def import_any(statement_list):
>     for try_me in statement_list:
>         try:
>             # uhh, this is the bit I'm not sure about...
>             try_me() # this would work if they're functions instead!
>             return
>         except ImportError:
>             pass
>     raise ImportError
>
> This works for a set of functions, but not for a bunch of "from this
> import that" statements. Would it be worth mangling the top of your
> script until it can be done with importlib or __import__? Doesn't seem
> nearly as clean, somehow.
>
> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131228/03df5e41/attachment.html>

From abarnert at yahoo.com  Sat Dec 28 20:46:56 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 28 Dec 2013 11:46:56 -0800
Subject: [Python-ideas] Determine Windows version in platform module
In-Reply-To: <CAFYqXL8HFJs6txLFaGxEakNsOpTtQyU3CH5J-5eooxbMJjq80A@mail.gmail.com>
References: <CAFYqXL9qSW-swM0abKL=D6nqfXJfMfcnRQickHH3LA5oBBX+3g@mail.gmail.com>
 <52BDC93D.9070301@egenix.com>
 <CAFYqXL_1uxjtF2HQS7X4ZWbZ9ATvFkQCWhjsmfcrbkMb4jni7A@mail.gmail.com>
 <A3043147-82C6-4A25-992D-FEEAD5735455@yahoo.com>
 <CAFYqXL8HFJs6txLFaGxEakNsOpTtQyU3CH5J-5eooxbMJjq80A@mail.gmail.com>
Message-ID: <B4A8177A-17CD-4222-972C-0BC6EFAA9E02@yahoo.com>

On Dec 28, 2013, at 8:12, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote:

> 
>> Meanwhile, Microsoft's suggested solution is that you not check for version numbers at runtime, but instead check for features. And there are good reasons for that--for example, XPSP3 N has exactly the same version numbers as XPSP3 normal, but if you try to use the new Windows Media APIs, you'll crash.
> 
> Mmm. What's weird is that if you use MSDN as a reference every Windows API has a "minimum supported client" section near the bottom of the page.
> See for example:
> http://msdn.microsoft.com/en-us/library/windows/desktop/ms684874(v=vs.85).aspx
> To me that suggests that (most of the times at least) you are encouraged to use the windows version to decide whether to support a feature or not.

Those versions don't correspond to any unique kernel version number, so you _can't_ check for it that way.

What you _can_ do with that information is write your app's version requirements, so your end users know not to buy your all if they have xpsp1.

http://msdn.microsoft.com/en-us/library/windows/desktop/ms724832(v=vs.85).aspx is one of the places they recommend against using version numbers as a substitute for feature test. it says:


way to determine whether a particular operating system feature is present. This is because the operating system may have had new features added in a redistributable DLL. Rather than using the Version API Helper functions to determine the operating system platform or version number, test for the presence of the feature itself.
To determine the best way to test for a feature, refer to the documentation for the feature of interest. The following list discusses some common techniques for feature detection:
You can test for the presence of the functions associated with a feature. To test for the presence of a function in a system DLL, call the LoadLibrary function to load the DLL. Then call the GetProcAddress function to determine whether the function of interest is present in the DLL. Use the pointer returned by GetProcAddress to call the function. Note that even if the function is present, it may be a stub that just returns an error code such as ERROR_CALL_NOT_IMPLEMENTED.
You can determine the presence of some features by using the GetSystemMetrics function. For example, you can detect multiple display monitors by callingGetSystemMetrics(SM_CMONITORS).
There are several versions of the redistributable DLLs that implement shell and common control features. For information about determining which versions are present on the system your application is running on, see the topicShell and Common Controls Versions.
If you must require a particular operating system, be sure to use it as a minimum supported version, rather than design the test for the one operating system. This way, your detection code will continue to work on future versions of Windows.
Note that a 32-bit application can detect whether it is running under WOW64 by calling the IsWow64Process function. It can obtain additional processor information by calling theGetNativeSystemInfo function.
> 
> 
> --- Giampaolo
> https://code.google.com/p/psutil/
> https://code.google.com/p/pyftpdlib/
> https://code.google.com/p/pysendfile/ 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131228/b2dba14d/attachment.html>

From abarnert at yahoo.com  Sat Dec 28 20:55:19 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 28 Dec 2013 11:55:19 -0800
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <20131228122700.GV29356@ando>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
 <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
 <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>
 <20131228122700.GV29356@ando>
Message-ID: <D73F561C-933E-4708-A5A5-6FABF604B2D9@yahoo.com>

On Dec 28, 2013, at 4:27, Steven D'Aprano <steve at pearwood.info> wrote:

> On Fri, Dec 27, 2013 at 12:59:48PM -0800, Andrew Barnert wrote:
>> On Dec 27, 2013, at 9:43, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> 
>>> I like that. I usually end up doing that with ElementTree:
>>> 
>>> try:
>>>    from xml.etree import cElementTree as etree
>> 
>> Python 3 has already fixed that. You just import ElementTree and you 
>> get the C-accelerated version.
>> 
>> And the same is true for cPickle, etc.
> 
> 
> Not if the C-accelerated version isn't available.

Right, in which case you get the pure Python version--exactly as if you'd done the try/import/except/import, or used this proposed new feature. So there is no advantage to doing the more complicated thing, except 2.x compatibility--which you obviously wouldn't get with a new language feature.

> Or perhaps there are 
> two pure-Python modules with the same API, and you're not sure which is 
> installed. This issue is bigger than just C-accelerators in the CPython 
> standard library. It applies any time you have two or more alternatives 
> with the same API, a preferred module and an fallback module. E.g. 
> readline in the stdlib, versus pyreadline, a third-party Windows port.

There are maybe three such examples that people use regularly. Is that enough to warrant a change in the language?

Well, that's not quite true. It's very common for modules that were renamed in 3.0, or that were recently added but have backward compat ports on PyPI. But again, a language change won't help those cases anyway. Unless you're expecting another mass renaming in 4.0?

From abarnert at yahoo.com  Sat Dec 28 21:05:28 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 28 Dec 2013 12:05:28 -0800
Subject: [Python-ideas] chained try (was Re:  "maybe import"?)
In-Reply-To: <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <20131227040808.GP29356@ando> <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
Message-ID: <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>

On Dec 28, 2013, at 8:10, Chris Angelico <rosuav at gmail.com> wrote:

> There is one thing, though, that I'm seeing of all this. Exception
> throwing is asymmetrical: you can attempt a series of statements until
> one fails, but there's no convenient syntax to attempt a series of
> statements until one succeeds. I wonder, could the more general case
> be solved? Is there a way to, without stupid stuff like eval, wrap up
> a few statements so they can be executed in a loop:
> 
> def import_any(statement_list):
>    for try_me in statement_list:
>        try:
>            # uhh, this is the bit I'm not sure about...
>            try_me() # this would work if they're functions instead!
>            return
>        except ImportError:
>            pass
>    raise ImportError
> 
> This works for a set of functions, but not for a bunch of "from this
> import that" statements. Would it be worth mangling the top of your
> script until it can be done with importlib or __import__? Doesn't seem
> nearly as clean, somehow.

This is a clever idea. 

If you could work out a clean syntax for the "try until one works" syntax, it would solve a wider range of problems than just this import issue.

It strikes me that there's some similarity with if/elif/else. Maybe just:

    try:
        stuff
    except try:
        other stuff
    except try:
        different stuff
    except Exception as e:
        deal with it

But there are a lot of open questions. First, "except try" looks horrible, but I can't think of anything better. If they all fail, do you only get the last exception, or are they all chained together? Would there be a use for a sequence like this without except on the end (so if they all fail, it just raises)? Can you put an exception type or tuple between the "except try"? What about an as clause (so the block could use it)?

From steve at pearwood.info  Sun Dec 29 00:43:23 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 29 Dec 2013 10:43:23 +1100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <D73F561C-933E-4708-A5A5-6FABF604B2D9@yahoo.com>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
 <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
 <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>
 <20131228122700.GV29356@ando>
 <D73F561C-933E-4708-A5A5-6FABF604B2D9@yahoo.com>
Message-ID: <20131228234322.GW29356@ando>

On Sat, Dec 28, 2013 at 11:55:19AM -0800, Andrew Barnert wrote:
> On Dec 28, 2013, at 4:27, Steven D'Aprano <steve at pearwood.info> wrote:

> > Or perhaps there are 
> > two pure-Python modules with the same API, and you're not sure which is 
> > installed. This issue is bigger than just C-accelerators in the CPython 
> > standard library. It applies any time you have two or more alternatives 
> > with the same API, a preferred module and an fallback module. E.g. 
> > readline in the stdlib, versus pyreadline, a third-party Windows port.
> 
> There are maybe three such examples that people use regularly. Is that 
> enough to warrant a change in the language?

There's only one *common* example of augmented assignment, x += 1, but 
that was enough to justify a whole raft of new syntax, += -= *= |= etc
because the *idiom* of "x = x op arg" is common.

What matters is not the specific examples of which modules get used in 
this way, but that the idiom of

try:
    import this
except ImportError:
    import that as this

is common, not the specific values of this/that. I do not expect that 
the idiom of having a fallback module when the preferred module isn't 
available will ever disappear, simply because there are too many 
circumstances in which it comes up.

Having said that, Nick's comments strike me as critical. "import this or 
that" doesn't buy us anything that can't already be solved in other 
ways. If it were easy to add, then perhaps it would be justified. But 
the implementation of import is already gnarly and hard to maintain (so 
I'm told), so adding more complexity to it for something which is purely 
syntactic sugar is probably not the best thing to do.

Maybe in Python 4000 :-)


-- 
Steven

From bruce at leapyear.org  Sun Dec 29 05:39:50 2013
From: bruce at leapyear.org (Bruce Leban)
Date: Sat, 28 Dec 2013 20:39:50 -0800
Subject: [Python-ideas] chained try (was Re: "maybe import"?)
In-Reply-To: <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>
References: <-7047415593530210756@gmail297201516> <20131227040808.GP29356@ando>
 <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
 <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>
Message-ID: <CAGu0AnuLATmA0B5EtP6mka+rW-6D8XeA8y=f68Nz6GZaYWzFBg@mail.gmail.com>

On Sat, Dec 28, 2013 at 12:05 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> Would there be a use for a sequence like this without except on the end
> (so if they all fail, it just raises)?
>

I think what you're talking about is just leaving the try off the last
clause (as in the example below).


> Can you put an exception type or tuple between the "except try"? What
> about an as clause (so the block could use it)?
>

I started writing a mail with exactly this subject line and discarded it
because of the latter two issues. In code I've recently written I have a
chain of three and while the indentation is ugly, it's not so ugly that it
cries out as a must fix language feature. And the problem is that I really
want to limit to specific exceptions. In one case I have something like
this:

try:
    ....
except KeyError:
    try:
        ....
    except AttributeError:

        try:
            ....

        except IndexError:
            ....


and if I can't specify the specific exception, this is not helpful. On the
other hand

try:
    ....
except KeyError try:
    ....
except AttributeError try:
    ....

except IndexError:
    ....


only really saves me two colons and some whitespace. If the language had
this feature, I'd use it, but that doesn't make it worth adding.

--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131228/b6f3c9b1/attachment.html>

From abarnert at yahoo.com  Sun Dec 29 10:07:10 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 29 Dec 2013 01:07:10 -0800
Subject: [Python-ideas] chained try (was Re: "maybe import"?)
In-Reply-To: <CAGu0AnuLATmA0B5EtP6mka+rW-6D8XeA8y=f68Nz6GZaYWzFBg@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <20131227040808.GP29356@ando> <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
 <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>
 <CAGu0AnuLATmA0B5EtP6mka+rW-6D8XeA8y=f68Nz6GZaYWzFBg@mail.gmail.com>
Message-ID: <2C9DDD38-F5E1-44E5-91C0-949A1FC0752A@yahoo.com>

On Dec 28, 2013, at 20:39, Bruce Leban <bruce at leapyear.org> wrote:

> 
> On Sat, Dec 28, 2013 at 12:05 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> Can you put an exception type or tuple between the "except try"? What about an as clause (so the block could use it)?
> 
> I started writing a mail with exactly this subject line and discarded it because of the latter two issues.

The second one is pretty minor; it's just a matter of making the syntax clean, and I'm sure someone would come up with something if it were worth doing.

The part that worries me about the first issue is that there seem to be two reasonable ways to read it. In your example below, if the first attempt raises an AttributeError, should it go to the "except AttributeError try:" block, or should that only happen if the second attempt raises one? The first variation means it's _not_ just syntactic sugar for the overly indented form you gave before it. But that also means (I think) it requires a change to the way exceptions are handled. (I'd have to sit down and work out the bytecode to be sure about that.) And, more importantly, I don't think the meaning would be obvious to a reader--the fact that you assumed the other meaning was the only one possible is pretty strong evidence for that.

> In code I've recently written I have a chain of three and while the indentation is ugly, it's not so ugly that it cries out as a must fix language feature. And the problem is that I really want to limit to specific exceptions. In one case I have something like this:
> 
> try:
>     ....
> except KeyError:
>     try:
>         ....
>     except AttributeError:
>         try:
>             ....
>         except IndexError:
>             ....
> 
> and if I can't specify the specific exception, this is not helpful. On the other hand 
> 
> try:
>     ....
> except KeyError try:
>     ....
> except AttributeError try:
>     ....
> except IndexError:
>     ....
> 
> only really saves me two colons and some whitespace. If the language had this feature, I'd use it, but that doesn't make it worth adding.
> 
> --- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131229/ba0f2342/attachment-0001.html>

From denis.spir at gmail.com  Sun Dec 29 10:17:00 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 29 Dec 2013 10:17:00 +0100
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>
 <20131227040808.GP29356@ando> <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
Message-ID: <52BFE88C.7080407@gmail.com>

On 12/28/2013 05:10 PM, Chris Angelico wrote:
> On Sun, Dec 29, 2013 at 1:59 AM, spir <denis.spir at gmail.com> wrote:
>>> This would also be allowed:
>>>
>>>       from this or that or another import spam
>>
>>
>> We'd also, or rather, need:
>>
>>          from this import foo or that import bar or another import baz as
>> spam
>>
>> which starts to be a bit complicated... Maybe (or perhaps) with parens:
>>
>>          from ((this import foo) or (that import bar) or (another import
>> baz)) as spam
>
> Can't say I like that variant - getting complicated and hard to read,
> even with parens. If you're importing different names from different
> modules, it's going to get very long, and then people are going to
> want to split it across lines, and then the obvious question is:
> what's been gained over the try/except variant?
>
> There is one thing, though, that I'm seeing of all this. Exception
> throwing is asymmetrical: you can attempt a series of statements until
> one fails, but there's no convenient syntax to attempt a series of
> statements until one succeeds.

This is the logic of ordered choice (in parsing) I evoked in previous post, 
isn't it? Also same logic as 'find_first' funcs or the like (or of python's 
'any' I guess).

>  I wonder, could the more general case
> be solved? Is there a way to, without stupid stuff like eval, wrap up
> a few statements so they can be executed in a loop:
>
> def import_any(statement_list):
>      for try_me in statement_list:
>          try:
>              # uhh, this is the bit I'm not sure about...
>              try_me() # this would work if they're functions instead!
>              return
>          except ImportError:
>              pass
>      raise ImportError
>
> This works for a set of functions, but not for a bunch of "from this
> import that" statements. Would it be worth mangling the top of your
> script until it can be done with importlib or __import__? Doesn't seem
> nearly as clean, somehow.

You seem to be asking for kinds of ruby blocks (or procs?), aren't you? Very 
practicle at times (while I'm not a fan of them, abuse leads to less readable 
code, as in FP). I'd enjoy them in python to encode conditional actions, like 
game events: a kind of pair (cause_state, effect_action) with a generic logic. 
In Python, one has to encode every cause and every action as separate functions, 
because they are distinct pieces of (named) data, while typically they're both 
one-liners (eg: if a unit steps on this very hex, add 15% to its health points).

Denis

From denis.spir at gmail.com  Sun Dec 29 10:41:19 2013
From: denis.spir at gmail.com (spir)
Date: Sun, 29 Dec 2013 10:41:19 +0100
Subject: [Python-ideas] chained try (was Re:  "maybe import"?)
In-Reply-To: <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>
References: <-7047415593530210756@gmail297201516>
 <20131227040808.GP29356@ando> <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
 <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>
Message-ID: <52BFEE3F.8050901@gmail.com>

On 12/28/2013 09:05 PM, Andrew Barnert wrote:
> On Dec 28, 2013, at 8:10, Chris Angelico <rosuav at gmail.com> wrote:
>
>> There is one thing, though, that I'm seeing of all this. Exception
>> throwing is asymmetrical: you can attempt a series of statements until
>> one fails, but there's no convenient syntax to attempt a series of
>> statements until one succeeds. I wonder, could the more general case
>> be solved? Is there a way to, without stupid stuff like eval, wrap up
>> a few statements so they can be executed in a loop:
>>
>> def import_any(statement_list):
>>     for try_me in statement_list:
>>         try:
>>             # uhh, this is the bit I'm not sure about...
>>             try_me() # this would work if they're functions instead!
>>             return
>>         except ImportError:
>>             pass
>>     raise ImportError
>>
>> This works for a set of functions, but not for a bunch of "from this
>> import that" statements. Would it be worth mangling the top of your
>> script until it can be done with importlib or __import__? Doesn't seem
>> nearly as clean, somehow.
>
> This is a clever idea.
>
> If you could work out a clean syntax for the "try until one works" syntax, it would solve a wider range of problems than just this import issue.
>
> It strikes me that there's some similarity with if/elif/else. Maybe just:
>
>      try:
>          stuff
>      except try:
>          other stuff
>      except try:
>          different stuff
>      except Exception as e:
>          deal with it
>
> But there are a lot of open questions. First, "except try" looks horrible, but I can't think of anything better.

I would buy (if cheap!):

     try:
         stuff
     else try:
         other stuff
     else try:
         different stuff
     except Exception as e:
         deal with it

> If they all fail, do you only get the last exception, or are they all chained together?

Same issue in parsing in front of a choice (ordered or not): when match fails, 
what helpful error message? The parser cannot know which branch should have 
matched, so the only solution is to list failure causes of every branch; but 
branches may hold choices themselves, etc...

expected: p = a | b| c
at position: ....
found: ...

     expected: a = ...
     at position: ....
     found: ...

     expected: b = x | y
     at position: ....
     found: ...

         expected: x = ...
         at position: ....
         found: ...

         expected: y = ...
         at position: ....
         found: ...

     expected: c = ...
     at position: ....
     found: ...

This suggests that a general pattern for such an (ordered) choice of potential 
actions should not be modeled after exception catching; except (sic) for the 
failure branch in case none of them runs. My solution in parsing is only giving 
a general error (the first block above), but providing a solution (a test check 
tool func, the building block of test suites) for the user to try each potential 
branch individually (easily, quickly, and with helpful feedback).
	pat.check(str)

> Would there be a use for a sequence like this without except on the end (so if they all fail, it just raises)? Can you put an exception type or tuple between the "except try"? What about an as clause (so the block could use it)?

I would require the general failure branch in syntax, for safety, like ordinary; 
with the same contents contents beeing the same: "pass" or "assert False". [1]

Denis

[1] I take the opportunity to ask: is there a builtin exception or other means 
to say "this block should never be reached"?


From techtonik at gmail.com  Sun Dec 29 12:28:11 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 29 Dec 2013 14:28:11 +0300
Subject: [Python-ideas] os.architecture
Message-ID: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>

if os.architecture == 32:
  download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
else:
  download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')


?

From rosuav at gmail.com  Sun Dec 29 13:15:10 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 29 Dec 2013 23:15:10 +1100
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
Message-ID: <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>

On Sun, Dec 29, 2013 at 10:28 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> if os.architecture == 32:
>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
> else:
>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')
>

Could you get that info from platform.uname()[4]?

ChrisA

From p.f.moore at gmail.com  Sun Dec 29 13:39:52 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 29 Dec 2013 12:39:52 +0000
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
Message-ID: <CACac1F-=je+gLCPRn=dUs1WkhBrVQeBndWUkSON2=vt=jEk0AA@mail.gmail.com>

On 29 December 2013 12:15, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Dec 29, 2013 at 10:28 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> if os.architecture == 32:
>>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
>> else:
>>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')
>>
>
> Could you get that info from platform.uname()[4]?

I would have thought so, and more accurately too. An AMD64 executable
won't be much use on SPARC64, no matter that they are both 64-bit -
the OP doesn't have this problem as his example is for Windows (at
least as long as he's happy to ignore old Itanium systems).

I hadn't realised that platform.uname() gave this information on
Windows, but given that it does, it looks like a better solution for
the OP's problem.
Paul

From ncoghlan at gmail.com  Sun Dec 29 16:12:36 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 30 Dec 2013 01:12:36 +1000
Subject: [Python-ideas] "maybe import"?
In-Reply-To: <20131228234322.GW29356@ando>
References: <-7047415593530210756@gmail297201516>
 <CAGu0AnuFruvP-+CGW0HqXhquifo=s+cxMOWg_Q7DbLP2YetpNw@mail.gmail.com>
 <2119483115886762605@gmail297201516>
 <CAO41-mNpjwOvrrk=mLC3WX=17k+nrzm3E5+PBFVvtXfmaTwPkA@mail.gmail.com>
 <C8AEEA1B-4A3D-4C3C-BF3B-E4C59C6B42B3@yahoo.com>
 <20131228122700.GV29356@ando>
 <D73F561C-933E-4708-A5A5-6FABF604B2D9@yahoo.com>
 <20131228234322.GW29356@ando>
Message-ID: <CADiSq7fH8VVB7wLsiodgRQiXmv7Kd3kKx4+sJQPGeq=u2VDmNA@mail.gmail.com>

On 29 Dec 2013 09:44, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Sat, Dec 28, 2013 at 11:55:19AM -0800, Andrew Barnert wrote:
> > On Dec 28, 2013, at 4:27, Steven D'Aprano <steve at pearwood.info> wrote:
>
> > > Or perhaps there are
> > > two pure-Python modules with the same API, and you're not sure which
is
> > > installed. This issue is bigger than just C-accelerators in the
CPython
> > > standard library. It applies any time you have two or more
alternatives
> > > with the same API, a preferred module and an fallback module. E.g.
> > > readline in the stdlib, versus pyreadline, a third-party Windows port.
> >
> > There are maybe three such examples that people use regularly. Is that
> > enough to warrant a change in the language?
>
> There's only one *common* example of augmented assignment, x += 1, but
> that was enough to justify a whole raft of new syntax, += -= *= |= etc
> because the *idiom* of "x = x op arg" is common.
>
> What matters is not the specific examples of which modules get used in
> this way, but that the idiom of
>
> try:
>     import this
> except ImportError:
>     import that as this
>
> is common, not the specific values of this/that. I do not expect that
> the idiom of having a fallback module when the preferred module isn't
> available will ever disappear, simply because there are too many
> circumstances in which it comes up.
>
> Having said that, Nick's comments strike me as critical. "import this or
> that" doesn't buy us anything that can't already be solved in other
> ways. If it were easy to add, then perhaps it would be justified. But
> the implementation of import is already gnarly and hard to maintain (so
> I'm told), so adding more complexity to it for something which is purely
> syntactic sugar is probably not the best thing to do.

I suspect this would actually be pretty easy to implement (for a given
definition of easy, since we'd likely handle it in the bytecode generation
phase). However, it would still make the import statement even harder to
fully comprehend than it already is, which is actually quite a notable
achievement :)

Cheers,
Nick.
>
> Maybe in Python 4000 :-)
>
>
> --
> Steven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131230/953f7261/attachment.html>

From techtonik at gmail.com  Sun Dec 29 20:38:39 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 29 Dec 2013 22:38:39 +0300
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
Message-ID: <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>

On Sun, Dec 29, 2013 at 3:15 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Dec 29, 2013 at 10:28 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> if os.architecture == 32:
>>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
>> else:
>>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')
>>
>
> Could you get that info from platform.uname()[4]?

If Python is about readability then looking at your example, I have
two questions:
1. What is platform - hardware, CPU, OS, Python flavor?
2. What is uname()[4]?

For os.architecture I have only one question - what is the returned
value? And it is easy to remember once you get it. It is also easy to
debug.
--
anatoly t.

From amber.yust at gmail.com  Sun Dec 29 21:53:15 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Sun, 29 Dec 2013 20:53:15 +0000
Subject: [Python-ideas] os.architecture
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
Message-ID: <2403736338056934130@gmail297201516>

Until an architecture that differs in more than just number of bits comes
out, at which point 'os.architecture == 32' starts becoming awkward.

On Sun Dec 29 2013 at 12:34:09 PM, anatoly techtonik <techtonik at gmail.com>
wrote:

> On Sun, Dec 29, 2013 at 3:15 PM, Chris Angelico <rosuav at gmail.com> wrote:
> > On Sun, Dec 29, 2013 at 10:28 PM, anatoly techtonik <techtonik at gmail.com>
> wrote:
> >> if os.architecture == 32:
> >>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
> >> else:
> >>   download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')
> >>
> >
> > Could you get that info from platform.uname()[4]?
>
> If Python is about readability then looking at your example, I have
> two questions:
> 1. What is platform - hardware, CPU, OS, Python flavor?
> 2. What is uname()[4]?
>
> For os.architecture I have only one question - what is the returned
> value? And it is easy to remember once you get it. It is also easy to
> debug.
> --
> anatoly t.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131229/f5312057/attachment-0001.html>

From rosuav at gmail.com  Sun Dec 29 22:17:06 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 30 Dec 2013 08:17:06 +1100
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
Message-ID: <CAPTjJmrj96SeBgHLB7e+LSL_7prg1uhp3+qwphFFbTcJ5bu17A@mail.gmail.com>

On Mon, Dec 30, 2013 at 6:38 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> If Python is about readability then looking at your example, I have
> two questions:
> 1. What is platform - hardware, CPU, OS, Python flavor?
> 2. What is uname()[4]?
>
> For os.architecture I have only one question - what is the returned
> value? And it is easy to remember once you get it. It is also easy to
> debug.

You can also use platform.uname().machine if you prefer readability,
but that doesn't work on Python 2.

Number of bits is sufficient only if you can always assume you're on
the same architecture. It's much better to simply quote the uname
value. And hey, you can even cut your code down from four lines to
one:

download('http://www.libsdl.org/release/SDL2-2.0.1-win32-'+uname()[4])+'.zip')

If you get back a 404, obviously this platform's not supported. Bingo!
You've just protected yourself against trying to download and run
Intel code on an ARM chip.

ChrisA

From abarnert at yahoo.com  Sun Dec 29 23:21:08 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 29 Dec 2013 14:21:08 -0800
Subject: [Python-ideas] chained try (was Re:  "maybe import"?)
In-Reply-To: <52BFEE3F.8050901@gmail.com>
References: <-7047415593530210756@gmail297201516>
 <20131227040808.GP29356@ando> <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
 <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com> <52BFEE3F.8050901@gmail.com>
Message-ID: <9E44C300-08A6-4883-BC71-DC8BD4F0DC41@yahoo.com>

On Dec 29, 2013, at 1:41, spir <denis.spir at gmail.com> wrote:

>> "except try" looks horrible, but I can't think of anything better.
> 
> I would buy (if cheap!):
> 
>    try:
>        stuff
>    else try:
>        other stuff
>    else try:
>        different stuff
>    except Exception as e:
>        deal with it

The advantage of "except try" is that it leaves the door open for specifying only certain exceptions instead of all, which would be useful in some cases.

But the advantage of "else try" is that it closes that door, eliminating most of the other questions I and others raised. It's clearly pure syntactic sugar with no ambiguity. Maybe that's a better idea.

From abarnert at yahoo.com  Sun Dec 29 23:31:43 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 29 Dec 2013 14:31:43 -0800
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
Message-ID: <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>

On Dec 29, 2013, at 3:28, anatoly techtonik <techtonik at gmail.com> wrote:

> if os.architecture == 32:
>  download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
> else:
>  download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')

Love it! A win32-x86 build is exactly what you want on 32-bit ARM9 linux, a win32-x64 build is perfect for ppc64 OS X. And if you're running a 32-bit python on 64-bit Windows you almost certainly want to download 64-bit libraries, right?

I also love hard coding SDL 2.0.1 into your portable application, because there's no way they'll ever release a newer version without telling you in advance so you can forcibly update all your users, which makes their "latest" links useless.

From rosuav at gmail.com  Sun Dec 29 23:38:34 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 30 Dec 2013 09:38:34 +1100
Subject: [Python-ideas] os.architecture
In-Reply-To: <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>
Message-ID: <CAPTjJmomkz9WD0dzHh6cXmHBUB6UZxUSBXcNGifWXXQRzQKhOw@mail.gmail.com>

On Mon, Dec 30, 2013 at 9:31 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
> And if you're running a 32-bit python on 64-bit Windows you almost certainly want to download 64-bit libraries, right?

Actually, that one's a bit harder to work out. I'd say uname() would
say that it's a 64-bit system (of whatever architecture), so all you
have is the platform module and its methods of looking at
sys.executable - and the docs comments kinda scare me off using that
anywhere other than Linux (and even there, it looks inefficient). Is
there a best-practice way of finding out what binary libraries you
should be using for this process? Surely that should be a simple
thing?

ChrisA

From abarnert at yahoo.com  Mon Dec 30 00:14:33 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 29 Dec 2013 15:14:33 -0800
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPTjJmomkz9WD0dzHh6cXmHBUB6UZxUSBXcNGifWXXQRzQKhOw@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>
 <CAPTjJmomkz9WD0dzHh6cXmHBUB6UZxUSBXcNGifWXXQRzQKhOw@mail.gmail.com>
Message-ID: <4A8405B6-586A-43C2-B653-B4F30137C97B@yahoo.com>

On Dec 29, 2013, at 14:38, Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, Dec 30, 2013 at 9:31 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> And if you're running a 32-bit python on 64-bit Windows you almost certainly want to download 64-bit libraries, right?
> 
> Actually, that one's a bit harder to work out. I'd say uname() would
> say that it's a 64-bit system (of whatever architecture), so all you
> have is the platform module and its methods of looking at
> sys.executable - and the docs comments kinda scare me off using that
> anywhere other than Linux (and even there, it looks inefficient). Is
> there a best-practice way of finding out what binary libraries you
> should be using for this process? Surely that should be a simple
> thing?

Sure. And this is all documented in the first place I would have thought to look--right near the top of the platform module, the first thing that comes up when I google "python platform" or "python architecture". 

There are actually 4 different things you could be asking for, not just 2 (even ignoring the fact that you can have different architectures that aren't just 32- and 64-bit versions of the same thing). 

What you really want here is probably not whether you're on a 64-bit OS, or whether Python was built 64-bit--if you ever care about any platform besides Windows, you may be running under a fat/universal binary that could be built for both 32- and 64-bit, and you want to know which one it's being
_run_ as. So, as the docs say:

To get at the "64-bitness" of the current interpreter, it is more reliable to query the sys.maxsize attribute.

Maybe that could be given a nicer name, but adding an "os.architecture" that gives you a different of the four values than "platform.architecture" and probably isn't the one you want as Anatoly suggested doesn't seem like an improvement.

Maybe we could have different functions in platform for hardware_architecture, os_architecture, executable_architecture, and interpreter_architecture? But I don't think they would get much use. Besides, historically I've been as interested in knowing whether I was on, say, real ARM7 or ARM9 emulating ARM7 as in real i386 or x86_64 emulating i386, where there's no simple 32 vs. 64 cross-platform answer that has any meaning, and I don't see any reason to believe that the future will be different, just because the present happens to be different on one platform.

From amber.yust at gmail.com  Mon Dec 30 00:17:30 2013
From: amber.yust at gmail.com (Amber Yust)
Date: Sun, 29 Dec 2013 23:17:30 +0000
Subject: [Python-ideas] os.architecture
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>
Message-ID: <-4609820619587005742@gmail297201516>

On Sun Dec 29 2013 at 2:35:07 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> I also love hard coding SDL 2.0.1 into your portable application, because
> there's no way they'll ever release a newer version without telling you in
> advance so you can forcibly update all your users, which makes their
> "latest" links useless.
>

Using a fixed version isn't necessarily a terrible idea. While lots of
libraries strive for backwards compatibility, that doesn't mean they don't
sometimes break it (intentionally or accidentally). Always pulling down the
latest version of a library can lead to trickier debugging.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131229/c1b0572c/attachment-0001.html>

From rosuav at gmail.com  Mon Dec 30 00:42:42 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 30 Dec 2013 10:42:42 +1100
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8xKhidz3hj2Ow8YZmGT1g17ghp2F13dyyvCrjhNnwA4dCQ@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
 <CAPTjJmrj96SeBgHLB7e+LSL_7prg1uhp3+qwphFFbTcJ5bu17A@mail.gmail.com>
 <CAPkN8xKhidz3hj2Ow8YZmGT1g17ghp2F13dyyvCrjhNnwA4dCQ@mail.gmail.com>
Message-ID: <CAPTjJmoJ_5bd0BtTLTsxt+tGm4OpQbx-RK2SjAhNNVs5wBOuMw@mail.gmail.com>

On Mon, Dec 30, 2013 at 10:39 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>> download('http://www.libsdl.org/release/SDL2-2.0.1-win32-'+uname()[4])+'.zip')
>
> Nothing personal, but that code is awful.

You're right, it's buggy because I didn't test it. Superfluous close
parenthesis there. But my point is that the architecture is right
there in the name, so you can save yourself the trouble and just use
what comes from uname directly.

ChrisA

From stephen at xemacs.org  Mon Dec 30 00:30:26 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 30 Dec 2013 08:30:26 +0900
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
Message-ID: <87a9fjkzfx.fsf@uwakimon.sk.tsukuba.ac.jp>

anatoly techtonik writes:
 > On Sun, Dec 29, 2013 at 3:15 PM, Chris Angelico <rosuav at gmail.com> wrote:

 > If Python is about readability then looking at your example, I have
 > two questions:
 > 1. What is platform - hardware, CPU, OS, Python flavor?

Everything you might want in this area.  (Note: 'os' is the wrong
namespace for this.  'os' provides wrappers for OS services, not
platform information.)  In particular

sum-65-1 08:16$ python
Python 2.6.1 (r261:67515, Aug  2 2010, 20:10:18) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.architecture()
('64bit', '')
>>> ^D

Duh.  Watch out for that time machine, Anatoly.

For more information, RTFM or try python-list.


From rosuav at gmail.com  Mon Dec 30 01:08:16 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 30 Dec 2013 11:08:16 +1100
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8x+J-rQ7X6+XntcSDRO3LeSp3iqi+LOg3Qb6BNRE_4tK=Q@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
 <CAPTjJmrj96SeBgHLB7e+LSL_7prg1uhp3+qwphFFbTcJ5bu17A@mail.gmail.com>
 <CAPkN8xKhidz3hj2Ow8YZmGT1g17ghp2F13dyyvCrjhNnwA4dCQ@mail.gmail.com>
 <CAPTjJmoJ_5bd0BtTLTsxt+tGm4OpQbx-RK2SjAhNNVs5wBOuMw@mail.gmail.com>
 <CAPkN8x+J-rQ7X6+XntcSDRO3LeSp3iqi+LOg3Qb6BNRE_4tK=Q@mail.gmail.com>
Message-ID: <CAPTjJmrCXe+HRtv1HhERJ_5EprcxhbhR5VkAsOmVkHeo+1MJXg@mail.gmail.com>

On Mon, Dec 30, 2013 at 10:57 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> On Mon, Dec 30, 2013 at 2:42 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Mon, Dec 30, 2013 at 10:39 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>>>> download('http://www.libsdl.org/release/SDL2-2.0.1-win32-'+uname()[4])+'.zip')
>>>
>>> Nothing personal, but that code is awful.
>>
>> You're right, it's buggy because I didn't test it. Superfluous close
>> parenthesis there. But my point is that the architecture is right
>> there in the name, so you can save yourself the trouble and just use
>> what comes from uname directly.
>
> I write code for aliens. They might be confused that x86 < x64.

Who cares? You're building up a URL and going and downloading it. The
user needn't even be aware of it. You're actually writing code for
code, here.

ChrisA

From ncoghlan at gmail.com  Mon Dec 30 01:24:13 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 30 Dec 2013 10:24:13 +1000
Subject: [Python-ideas] chained try (was Re: "maybe import"?)
In-Reply-To: <9E44C300-08A6-4883-BC71-DC8BD4F0DC41@yahoo.com>
References: <-7047415593530210756@gmail297201516> <20131227040808.GP29356@ando>
 <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
 <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>
 <52BFEE3F.8050901@gmail.com>
 <9E44C300-08A6-4883-BC71-DC8BD4F0DC41@yahoo.com>
Message-ID: <CADiSq7emR5jBqbfDnAwLSKq_K2-874sF4Aby=tG9vGKxWbnQOw@mail.gmail.com>

On 30 Dec 2013 08:24, "Andrew Barnert" <abarnert at yahoo.com> wrote:
>
> On Dec 29, 2013, at 1:41, spir <denis.spir at gmail.com> wrote:
>
> >> "except try" looks horrible, but I can't think of anything better.
> >
> > I would buy (if cheap!):
> >
> >    try:
> >        stuff
> >    else try:
> >        other stuff
> >    else try:
> >        different stuff
> >    except Exception as e:
> >        deal with it
>
> The advantage of "except try" is that it leaves the door open for
specifying only certain exceptions instead of all, which would be useful in
some cases.
>
> But the advantage of "else try" is that it closes that door, eliminating
most of the other questions I and others raised. It's clearly pure
syntactic sugar with no ambiguity. Maybe that's a better idea.

The else block triggers when no exception was thrown, so even if this idea
was to be accepted at all (which seems unlikely), that particular spelling
definitely wouldn't be accepted for something that only triggers when an
exception is thrown.

In practice, the existing more general solution of extracting an
appropriately named function is likely to improve readability more than
adding a new variation on the already complex try/except/else/finally
construct, which is why I consider it unlikely that a change along these
lines would be accepted.

Cheers,
Nick.

> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131230/e492a4e5/attachment.html>

From stephen at xemacs.org  Mon Dec 30 01:27:17 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 30 Dec 2013 09:27:17 +0900
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPTjJmoJ_5bd0BtTLTsxt+tGm4OpQbx-RK2SjAhNNVs5wBOuMw@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
 <CAPTjJmrj96SeBgHLB7e+LSL_7prg1uhp3+qwphFFbTcJ5bu17A@mail.gmail.com>
 <CAPkN8xKhidz3hj2Ow8YZmGT1g17ghp2F13dyyvCrjhNnwA4dCQ@mail.gmail.com>
 <CAPTjJmoJ_5bd0BtTLTsxt+tGm4OpQbx-RK2SjAhNNVs5wBOuMw@mail.gmail.com>
Message-ID: <8761q7kwt6.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > But my point is that the architecture is right there in the name,
 > so you can save yourself the trouble and just use what comes from
 > uname directly.

Except you probably can't.  "x64" != "x86_64".  (The latter is what my
Intel Mac says; don't have a 64-bit machine with win32 on it handy.)


From denis.spir at gmail.com  Mon Dec 30 09:24:16 2013
From: denis.spir at gmail.com (spir)
Date: Mon, 30 Dec 2013 09:24:16 +0100
Subject: [Python-ideas] chained try (was Re: "maybe import"?)
In-Reply-To: <CADiSq7emR5jBqbfDnAwLSKq_K2-874sF4Aby=tG9vGKxWbnQOw@mail.gmail.com>
References: <-7047415593530210756@gmail297201516>	<20131227040808.GP29356@ando>	<52BEE768.7060401@gmail.com>	<CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>	<DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com>	<52BFEE3F.8050901@gmail.com>	<9E44C300-08A6-4883-BC71-DC8BD4F0DC41@yahoo.com>
 <CADiSq7emR5jBqbfDnAwLSKq_K2-874sF4Aby=tG9vGKxWbnQOw@mail.gmail.com>
Message-ID: <52C12DB0.40101@gmail.com>

On 12/30/2013 01:24 AM, Nick Coghlan wrote:
> On 30 Dec 2013 08:24, "Andrew Barnert" <abarnert at yahoo.com> wrote:
>>
>> On Dec 29, 2013, at 1:41, spir <denis.spir at gmail.com> wrote:
>>
>>>> "except try" looks horrible, but I can't think of anything better.
>>>
>>> I would buy (if cheap!):
>>>
>>>     try:
>>>         stuff
>>>     else try:
>>>         other stuff
>>>     else try:
>>>         different stuff
>>>     except Exception as e:
>>>         deal with it
>>
>> The advantage of "except try" is that it leaves the door open for
> specifying only certain exceptions instead of all, which would be useful in
> some cases.
>>
>> But the advantage of "else try" is that it closes that door, eliminating
> most of the other questions I and others raised. It's clearly pure
> syntactic sugar with no ambiguity. Maybe that's a better idea.
>
> The else block triggers when no exception was thrown, so even if this idea
> was to be accepted at all (which seems unlikely), that particular spelling
> definitely wouldn't be accepted for something that only triggers when an
> exception is thrown.

I guess mean the else block in a 'normal' "try...except...else" construct? In 
this different form, there "else try", not just "else". I understood it as a 
single keyword (maybe read it elsetry, somewhat like elif) that denotes 
different semantics.

> In practice, the existing more general solution of extracting an
> appropriately named function is likely to improve readability more than
> adding a new variation on the already complex try/except/else/finally
> construct, which is why I consider it unlikely that a change along these
> lines would be accepted.

After some more reflexion, I find this general pattern of "chained try" (new 
thread title) really meaningful and useful, but maybe too rare to deserve proper 
syntax. However, what is the proper way to express it using current Python features?

Clearly, a series of nested try/except (compare: a series of nested if/else if 
python had no elif) is not the right tool, in my view, because try/except is to 
be used whenever the except branch is rare (semantically and practically, 
because itis clstly). What else?

Try/elsetry would not need to trigger actual exceptions with full error objects, 
exception catching, and stack unwinding (the whole machinary), but only set a 
plain flag meaning "this branch fails to execute normally [1].

Denis

[1] Could be a global bit or other var in the implementation, a CPU register, or 
even the carry flag in the CPU. See also: 
[https://www.lyngvig.org/%2FTeknik%2FA-Proposal-for-Exception-Handling-in-C]

From jeanpierreda at gmail.com  Mon Dec 30 11:23:12 2013
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Mon, 30 Dec 2013 02:23:12 -0800
Subject: [Python-ideas] chained try (was Re: "maybe import"?)
In-Reply-To: <52C12DB0.40101@gmail.com>
References: <-7047415593530210756@gmail297201516> <20131227040808.GP29356@ando>
 <52BEE768.7060401@gmail.com>
 <CAPTjJmrpQkhoz2bJVSp_70zxN_T3Tis8O3cReLXgDHDYOi71gw@mail.gmail.com>
 <DE50720B-D521-48D6-AB56-14BE5BC386AE@yahoo.com> <52BFEE3F.8050901@gmail.com>
 <9E44C300-08A6-4883-BC71-DC8BD4F0DC41@yahoo.com>
 <CADiSq7emR5jBqbfDnAwLSKq_K2-874sF4Aby=tG9vGKxWbnQOw@mail.gmail.com>
 <52C12DB0.40101@gmail.com>
Message-ID: <CABicbJJnGNb1Cn_rjQD6E8+EsYe2MsTe4nbkP3vRDAPUccFrGQ@mail.gmail.com>

On Mon, Dec 30, 2013 at 12:24 AM, spir <denis.spir at gmail.com> wrote:
>
> After some more reflexion, I find this general pattern of "chained try" (new
> thread title) really meaningful and useful, but maybe too rare to deserve
> proper syntax. However, what is the proper way to express it using current
> Python features?
>
> Clearly, a series of nested try/except (compare: a series of nested if/else
> if python had no elif) is not the right tool, in my view, because try/except
> is to be used whenever the except branch is rare (semantically and
> practically, because itis clstly). What else?
>
> Try/elsetry would not need to trigger actual exceptions with full error
> objects, exception catching, and stack unwinding (the whole machinary), but
> only set a plain flag meaning "this branch fails to execute normally [1].

If you, instead of raising an exception, return None or that thing,
then you can do things as so:

return foo() or bar() or baz()

This can't work if None is a reasonable value rather than just an
error value (which is often why one switches to exceptions to begin
with), but it works OK much of the time, and works OK all of the time
in some other programming languages that are built around this idiom.

-- Devin

From techtonik at gmail.com  Mon Dec 30 00:45:01 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 30 Dec 2013 02:45:01 +0300
Subject: [Python-ideas] os.architecture
In-Reply-To: <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <363CF3DE-58AF-40AA-96DA-3CFB62D50BF0@yahoo.com>
Message-ID: <CAPkN8xK4ePwJZokk18EO+0norX9ngNQZc_qBV+e+=qzFrmwjTg@mail.gmail.com>

On Mon, Dec 30, 2013 at 1:31 AM, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Dec 29, 2013, at 3:28, anatoly techtonik <techtonik at gmail.com> wrote:
>
>> if os.architecture == 32:
>>  download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x86.zip')
>> else:
>>  download('http://www.libsdl.org/release/SDL2-2.0.1-win32-x64.zip')
>
> Love it! A win32-x86 build is exactly what you want on 32-bit ARM9 linux, a win32-x64 build is perfect for ppc64 OS X. And if you're running a 32-bit python on 64-bit Windows you almost certainly want to download 64-bit libraries, right?

I am glad you've noticed this piece of code is for Windows. It is also
nice that you understand the difference between OS bitness and Python
bitness.

> I also love hard coding SDL 2.0.1 into your portable application, because there's no way they'll ever release a newer version without telling you in advance so you can forcibly update all your users, which makes their "latest" links useless.

You'll like it even more if I tell you there is also a hash and size
info hard coded.
https://bitbucket.org/techtonik/locally/src/8367fad824c111367b99baf443c5d38b525111a5/03.fetch-sdl2.py?at=default#cl-85

From techtonik at gmail.com  Mon Dec 30 00:54:29 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 30 Dec 2013 02:54:29 +0300
Subject: [Python-ideas] os.architecture
In-Reply-To: <87a9fjkzfx.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
 <87a9fjkzfx.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPkN8xJJuDm6nh0QQ5_-f-uMp5A4KO3btPk3NuwSTbsYusX2Tw@mail.gmail.com>

On Mon, Dec 30, 2013 at 2:30 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> anatoly techtonik writes:
>  > On Sun, Dec 29, 2013 at 3:15 PM, Chris Angelico <rosuav at gmail.com> wrote:
>
>  > If Python is about readability then looking at your example, I have
>  > two questions:
>  > 1. What is platform - hardware, CPU, OS, Python flavor?
>
> Everything you might want in this area.

I want to know if I am running 32-bit or 64-bit Windows on my 64-bit CPU.
So, how to resolve the ambiguity?

> (Note: 'os' is the wrong
> namespace for this.  'os' provides wrappers for OS services, not
> platform information.)  In particular

For OS information, 'os' is a logical namespace.
I don't know about CPU info though. Would be nice to have 'hardware'
or 'hwinfo' module in stdlib.
I don't know where info about Python bitness should be.

>>>> import platform
>>>> platform.architecture()
> ('64bit', '')
>>>> ^D

From techtonik at gmail.com  Mon Dec 30 09:56:33 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 30 Dec 2013 11:56:33 +0300
Subject: [Python-ideas] os.architecture
In-Reply-To: <8761q7kwt6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
 <CAPTjJmrj96SeBgHLB7e+LSL_7prg1uhp3+qwphFFbTcJ5bu17A@mail.gmail.com>
 <CAPkN8xKhidz3hj2Ow8YZmGT1g17ghp2F13dyyvCrjhNnwA4dCQ@mail.gmail.com>
 <CAPTjJmoJ_5bd0BtTLTsxt+tGm4OpQbx-RK2SjAhNNVs5wBOuMw@mail.gmail.com>
 <8761q7kwt6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPkN8xKSokR4ySXz-4rG-eA+sMAL1JHJ1_4P0ixH1SzhtrU2Zg@mail.gmail.com>

Ok. Architecture is a fail in terminology. The word "OS architecture"
can mean many things, and it will be the same design flaw as os.name.

How about os.bitness instead?

From abarnert at yahoo.com  Mon Dec 30 13:13:45 2013
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 30 Dec 2013 04:13:45 -0800
Subject: [Python-ideas] os.architecture
In-Reply-To: <CAPkN8xKSokR4ySXz-4rG-eA+sMAL1JHJ1_4P0ixH1SzhtrU2Zg@mail.gmail.com>
References: <CAPkN8x+UcP4_7HXmMCLZ8Q=3_Zg3d-oRbEN0QYwJ5d4A0E5E1w@mail.gmail.com>
 <CAPTjJmpY_S_hYgt5HcB2LZbvYPwhUYo-8ZgPQiEzQBMcOhMtXQ@mail.gmail.com>
 <CAPkN8xLGTn86He8aLqvgcPLRGbtcKYFzGDkCDeja55LziFUxPA@mail.gmail.com>
 <CAPTjJmrj96SeBgHLB7e+LSL_7prg1uhp3+qwphFFbTcJ5bu17A@mail.gmail.com>
 <CAPkN8xKhidz3hj2Ow8YZmGT1g17ghp2F13dyyvCrjhNnwA4dCQ@mail.gmail.com>
 <CAPTjJmoJ_5bd0BtTLTsxt+tGm4OpQbx-RK2SjAhNNVs5wBOuMw@mail.gmail.com>
 <8761q7kwt6.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAPkN8xKSokR4ySXz-4rG-eA+sMAL1JHJ1_4P0ixH1SzhtrU2Zg@mail.gmail.com>
Message-ID: <F5C6E13D-C132-4111-9066-28221DC08E38@yahoo.com>

On Dec 30, 2013, at 0:56, anatoly techtonik <techtonik at gmail.com> wrote:

> Ok. Architecture is a fail in terminology. The word "OS architecture"
> can mean many things, and it will be the same design flaw as os.name.
> 
> How about os.bitness instead?

You missed the part where you were told that os is for OS services, not platform (including hardware, interpreter, and OS) information. That, as you might guess, goes in platform. And it's already there, as you've been told at least three times, so there is no need to add it.

Anyway, "bitness" by itself doesn't tell you whether it will return 32 or 64 when running a 32-bit Python on 64-bit Windows (much less when running a universal 64-bit-capable Python on 64-bit Mac in 32-bit mode). It's just as potentially ambiguous as the functions that already exist, and therefore no better than them. And of course they have the advantage of already existing, and working in older versions of Python, and so on.

Instead of being bewildered by "what could platform possibly mean", just skim the docs, or use the builtin help.