From Nikolaus at rath.org  Thu Oct  1 00:42:10 2015
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 30 Sep 2015 15:42:10 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <653DC9A1-3C9B-4E66-AA97-4CA24F21F9E9@yahoo.com> (Andrew Barnert
 via Python-ideas's message of "Wed, 30 Sep 2015 14:40:15 -0700")
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <CAP7h-xbA7vQX9D_roXpWTYfH-2gXjeADEvpiSVuV3apk2UXabA@mail.gmail.com>
 <CAF7AXFEX6aYzjSvjTFWhe4_VLc43To1iyFQaDbcSw6U3gcveZg@mail.gmail.com>
 <560C12AD.90305@trueblade.com>
 <CAJ1Wxn1h7GR4V0_D06dtfoSGwEqPMm2usLXyUSgwBr7Vvo80qw@mail.gmail.com>
 <653DC9A1-3C9B-4E66-AA97-4CA24F21F9E9@yahoo.com>
Message-ID: <877fn78r7h.fsf@thinkpad.rath.org>

On Sep 30 2015, Andrew Barnert via Python-ideas <python-ideas-+ZN9ApsXKcEdnm+yROfE0A at public.gmane.org> wrote:
> Also, explicit "it" would be usable in other situations:
>
>     z = dangerous_thing(arg) if it.value() > 3 else DummyValue(3)

Really? I'm not sure if "it" is "dangerous_thing" or
"dangerous_thing(arg)".


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From srkunze at mail.de  Thu Oct  1 00:57:52 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 01 Oct 2015 00:57:52 +0200
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560BC0C3.4040502@btinternet.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <D91E3BCA-A368-4F9E-8C5A-11ABE744F391@gmail.com>
 <560BC0C3.4040502@btinternet.com>
Message-ID: <560C68F0.4080600@mail.de>

On 30.09.2015 13:00, Rob Cliffe wrote:
> Or:
>     x = a orelse b        # Visual Basic has a short-circuiting OrElse 
> operator for boolean operands
>     x = a orifNone b
>
>
> On 30/09/2015 01:39, Ryan Gonzalez wrote:
>> What about 'otherwise'?
>>
>> x = a otherwise b
>>


The only reason why I would prefer "else" over the proposed 
alternatives: it's already a reserved keyword and it's not really 
necessary to waste another one. Otherwise, I don't care too much.

Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/cb3099c6/attachment.html>

From random832 at fastmail.com  Thu Oct  1 01:07:27 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 30 Sep 2015 19:07:27 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <877fn78r7h.fsf@thinkpad.rath.org>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <CAP7h-xbA7vQX9D_roXpWTYfH-2gXjeADEvpiSVuV3apk2UXabA@mail.gmail.com>
 <CAF7AXFEX6aYzjSvjTFWhe4_VLc43To1iyFQaDbcSw6U3gcveZg@mail.gmail.com>
 <560C12AD.90305@trueblade.com>
 <CAJ1Wxn1h7GR4V0_D06dtfoSGwEqPMm2usLXyUSgwBr7Vvo80qw@mail.gmail.com>
 <653DC9A1-3C9B-4E66-AA97-4CA24F21F9E9@yahoo.com>
 <877fn78r7h.fsf@thinkpad.rath.org>
Message-ID: <1443654447.723246.398079841.55A7B687@webmail.messagingengine.com>

On Wed, Sep 30, 2015, at 18:42, Nikolaus Rath wrote:
> On Sep 30 2015, Andrew Barnert via Python-ideas
> <python-ideas-+ZN9ApsXKcEdnm+yROfE0A at public.gmane.org> wrote:
> > Also, explicit "it" would be usable in other situations:
> >
> >     z = dangerous_thing(arg) if it.value() > 3 else DummyValue(3)
> 
> Really? I'm not sure if "it" is "dangerous_thing" or
> "dangerous_thing(arg)".

I thought it was arg, though it's a bit silly for it to be.

Why not go full Scheme and add a way to do lexical expressions that
capture values for multiple use:

z = (let it=arg: dangerous_thing(it) if it.value() > 3 else
DummyValue(3))

Would be equivalent to

z = (lambda it: dangerous_thing(it) if it.value() > 3 else
DummyValue(3))(arg)

Only without necessarily actually instantiating a lambda (i.e. it could
push the result of arg [which may be any expression] onto the stack or
in an anonymous or obscurely-named local variable slot to refer whenever
it is referenced in the inner expression.

From srkunze at mail.de  Thu Oct  1 01:15:03 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 01 Oct 2015 01:15:03 +0200
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
Message-ID: <560C6CF7.10609@mail.de>

On 30.09.2015 23:33, Andrew Barnert via Python-ideas wrote:
> But you're conflating the concept of "lazy" with the concept of "iterator". While generators, and iterators in general, are always technically lazy and nearly-always practically lazy, lazy things are not always iterators. Range, dict views, memoryview/buffer objects, NumPy slices, third-party lazy-list types, etc. are not generators, nor are they like generators in any way, except for being lazy. They're lazy sequences (well, except for the ones that aren't sequences, but they're still lazy containers, or lazy non-iterator iterables if you want to stick to terms in the glossary).
>
> And I think experienced developers conflating the two orthogonal concepts is part of what leads to novices getting confused. They think that if they want laziness, they need a generator. That makes them unable to even form the notion that what they really want is a view/lazy container/virtual container even when that's what they want.
>
> And it makes it hard to discuss issues like this thread clearly.
>
> (The fact that we don't have a term for "non-iterator iterable", and that experienced users and even the documentation sometimes use the term "sequence" for that, only makes things worse. For example, a dict_keys is not a sequence in any useful sense, but the glossary says it is, because there is no word for what it wants to say.)

I have absolutely no idea what you are talking about here. ;)

I have to admit I try to avoid thinking too much about such tiny little 
details by using 
generators/lists/sequences/iterables/iterators/did-I-miss-one? directly 
in for loops only. Thus, the differences between all of them go away 
pretty fast.

But honestly, does it really need to be that complicated?


Best,
Sven

From tjreedy at udel.edu  Thu Oct  1 02:04:26 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 30 Sep 2015 20:04:26 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
Message-ID: <muhtaf$uca$1@ger.gmane.org>

On 9/30/2015 5:33 PM, Andrew Barnert via Python-ideas wrote:

> (The fact that we don't have a term for "non-iterator iterable",

'collection'
Some are concrete: they contain reference to actual Python objects.
Some are virtual (lazy): they contain the information need to create 
Python objects as needed.
Strings are a bit in between.

-- 
Terry Jan Reedy


From 4kir4.1i at gmail.com  Thu Oct  1 02:28:13 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Thu, 01 Oct 2015 03:28:13 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
Message-ID: <87vbar300y.fsf@gmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:
...
> (The fact that we don't have a term for "non-iterator iterable", and

All iterators are iterable but some iterables are not iterators.

If your code accepts only iterators then use the term *iterator*.
Otherwise the term *iterable* could be used.

It is misleading to use *iterable* if your code only accepts iterators.

If an iterable is an iterator; It is called *iterator*. The term
*iterable* implies that some instances are not iterators.





From steve at pearwood.info  Thu Oct  1 02:33:58 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 1 Oct 2015 10:33:58 +1000
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <CAF7AXFEX6aYzjSvjTFWhe4_VLc43To1iyFQaDbcSw6U3gcveZg@mail.gmail.com>
References: <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <CAP7h-xbA7vQX9D_roXpWTYfH-2gXjeADEvpiSVuV3apk2UXabA@mail.gmail.com>
 <CAF7AXFEX6aYzjSvjTFWhe4_VLc43To1iyFQaDbcSw6U3gcveZg@mail.gmail.com>
Message-ID: <20151001003358.GA23642@ando.pearwood.info>

On Wed, Sep 30, 2015 at 09:41:53AM -0700, Jeff Hardy wrote:

> 'def' is currently short for 'define', which would be too confusing.
> Spelling out 'default' isn't so bad, though:
> 
>     self.x = x default []

Heh, people jest :-)

Oh, you're serious? That just goes to show how different we all are. I 
think that using "default" as an infix operator is hideous.

"default" as a function/verb? Sure, that works fine:

if len(missed_payments) > 5:
    loan.default()

"default" as a variable/noun? Absolutely fine:

default = "something or other"

But "x default z" just doesn't read well: in English, it sounds like you 
are ordering x to cause z to default. Not very many words read naturally 
as a binary infix operator.

Besides, I would expect that making "default" a keyword will break a 
*huge* number of programs. I'm sure that many, many people use it as a 
variable or parameter name.


> And if it's going to be that long anyway, we might as well just put a
> `default` function in the builtins:
> 
>     self.x = default(x, [])

That's useless, as it breaks the short-circuiting property, which is one 
of the critical motives for introducing the null coalescing operator. 
Without the short-circuiting property, there's no point in making it a 
built-in. If all you want is a default function, then just add it as a 
helper function to the top of your code:

def default(obj, alternative):
    if obj is None:
        return alternative
    return obj



> I actually really like 'otherwise', but it's certainly not brief:
> 
>     self.x = x if x is not None else []
>     self.x = x otherwise []

That's almost half the length of the alternative. And if you have 
something more realistic, you save proportionally even less:

    document if document is not None else Document()
    document otherwise Document()


But personally, I have absolutely zero interest in the null coalescing 
operator on its own. I don't object to it specifically, but what really 
interests me are the two (pseudo)operators, null-aware indexing and 
null-aware attribute lookup. How do you extend "otherwise" to work with 
the [] and . operators?



-- 
Steve

From random832 at fastmail.com  Thu Oct  1 02:46:48 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 30 Sep 2015 20:46:48 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com>
Message-ID: <m2mvw38lfr.fsf@fastmail.com>

Akira Li <4kir4.1i at gmail.com> writes:

> Andrew Barnert via Python-ideas
> <python-ideas at python.org> writes:
> ...
>> (The fact that we don't have a term for "non-iterator iterable", and
>
> All iterators are iterable but some iterables are not iterators.
>
> If your code accepts only iterators then use the term *iterator*.
> Otherwise the term *iterable* could be used.
>
> It is misleading to use *iterable* if your code only accepts iterators.
>
> If an iterable is an iterator; It is called *iterator*. The term
> *iterable* implies that some instances are not iterators.

There are three (well, three and a half) kinds of code that consume
iterables, how would you describe each simply?

1. Does not call iter, simply calls next. Therefore cannot consume a
   non-iterator iterable.
2. Calls iter, but can accept an iterator (e.g. only goes through it once)
3. Cannot accept an iterator (goes through it twice, or permanently
   stores a reference to it, etc)
4. Can accept either, but behaves differently in each case (e.g. zip
   when passed two of the same iterator) - this can be regarded as a
   special case of #2.


From python at mrabarnett.plus.com  Thu Oct  1 03:08:51 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 1 Oct 2015 02:08:51 +0100
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
Message-ID: <560C87A3.1020507@mrabarnett.plus.com>

On 2015-09-29 21:43, Jeff Hardy wrote:
> On Tue, Sep 29, 2015 at 10:35 AM, Barry Warsaw <barry at python.org
> <mailto:barry at python.org>> wrote:
>
>     On Sep 28, 2015, at 03:04 PM, Carl Meyer wrote:
>
>     >But even if they are rejected, I think a simple `??` or `or?` (or
>     >however it's spelled) operator to reduce the repetition of "x if x is
>     >not None else y" is worth consideration on its own merits. This operator
>     >is entirely unambiguous, and I think would be useful and frequently
>     >used, whether or not ?. and ?[ are added along with it.
>
>     But why is it an improvement?  The ternary operator is entirely
>     obvious and
>     readable, and at least in my experience, is rare enough that the
>     repetition
>     doesn't hurt my fingers that much.  It seems like such a radical,
>     ugly new
>     syntax unjustified by the frequency of use and readability improvement.
>
>
> I use it all over the place in C# code, where it makes null checks much
> cleaner, and the punctuation choice makes sense:
>
>      var x = foo != null ? foo : "";
>      var y = foo ?? "";
>
>   (it also has other uses in C# relating to nullable types that aren't
> relevant in Python.)
>
> I'd argue the same is true in Python, if a decent way to spell it can be
> found:
>
>      x = foo if foo is not None else ""
>      y = foo or? ""
>
> It's pure syntactic sugar, but it *is* pretty sweet.
>
> (It would also make get-with-default unnecessary, but since it already
> exists that's not a useful argument.)
>
It's only just occurred to me that there's a small inconsistency here.
The "?.", "?[" and "?(" will short-circuit on None, whereas the "??"
will short-circuit on non-None.

Would that cause any confusion in practice?


From steve at pearwood.info  Thu Oct  1 03:15:06 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 1 Oct 2015 11:15:06 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
Message-ID: <20151001011504.GB23642@ando.pearwood.info>

On Wed, Sep 30, 2015 at 05:19:53PM +0000, Neil Girdhar wrote:

> I guess, I'm just asking for enumerate to go through the same change that
> range went through.  Why wasn't it a problem for range?

There is a pernicious myth that (x)range is an iterator. It is not. It 
is a sequence, but one where the items are calculated on demand rather 
than pre-populated into some large data structure (a list or array).

This is not just a matter of labels. It is a matter of the actual 
behaviour. (x)range objects don't behave like iterators except in the 
simplest sense that you can iterate over them.

So your question is based on false assumptions -- range didn't go 
through any such change. In Python 2, range was eager and xrange lazy, 
but both are sequences, and in Python 3 the eager version is gone and 
the lazy version renamed without the "x" prefix.




-- 
Steve

From steve at pearwood.info  Thu Oct  1 03:31:06 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 1 Oct 2015 11:31:06 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
References: <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
Message-ID: <20151001013106.GC23642@ando.pearwood.info>

On Wed, Sep 30, 2015 at 12:19:05PM -0700, Andrew Barnert via Python-ideas wrote:
[...]
> > FWIW: I don't think many people use the lazy sequence features
> > of range(), e.g. the slicing or index support. By far most
> > uses are in for-loops.
> 
> I've used range as a sequence (or at least a reusable iterable, a 
> sized object, and a container). I've answered questions from people on 
> StackOverflow who are doing so, and seen the highest-rep Python 
> answerer on SO suggest such uses to other people.
> 
> I don't think I'd ever use the index method (although I did see one SO 
> user who was doing so, to wrap up some arithmetic in a way that avoids 
> a possibly off-by-one error, and wanted to know why it was so slow in 
> 3.1 but worked fine in 3.2...), but there's no reason range should be 
> a defective "not-quite-sequence" instead of a sequence. What would be 
> the point of that?

There's also __contains__.

Personally, I don't like it, but using "n in range(a, b+1)" for testing 
whether integer n falls within a particular range seems to be popular. I 
don't know why they don't just write a <= n <= b, but it seems to be a 
popular idiom for some weird reason.


-- 
Steve

From 4kir4.1i at gmail.com  Thu Oct  1 04:04:07 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Thu, 01 Oct 2015 05:04:07 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
Message-ID: <87r3lf2vl4.fsf@gmail.com>

Random832 <random832 at fastmail.com> writes:

> Akira Li <4kir4.1i at gmail.com> writes:
>
>> Andrew Barnert via Python-ideas
>> <python-ideas at python.org> writes:
>> ...
>>> (The fact that we don't have a term for "non-iterator iterable", and
>>
>> All iterators are iterable but some iterables are not iterators.
>>
>> If your code accepts only iterators then use the term *iterator*.
>> Otherwise the term *iterable* could be used.
>>
>> It is misleading to use *iterable* if your code only accepts iterators.
>>
>> If an iterable is an iterator; It is called *iterator*. The term
>> *iterable* implies that some instances are not iterators.
>
> There are three (well, three and a half) kinds of code that consume
> iterables, how would you describe each simply?
>
> 1. Does not call iter, simply calls next. Therefore cannot consume a
>    non-iterator iterable.
iterator

> 2. Calls iter, but can accept an iterator (e.g. only goes through it
>    once)
iterable

> 3. Cannot accept an iterator (goes through it twice, or permanently
>    stores a reference to it, etc)
neither

*iterable* is an object that you can pass to iter() to get *iterator*.
An iterable does not guarantee that it yields the same items twice.

> 4. Can accept either, but behaves differently in each case (e.g. zip
>    when passed two of the same iterator) - this can be regarded as a
>    special case of #2.
iterable


From abarnert at yahoo.com  Thu Oct  1 04:24:03 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 19:24:03 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <muhtaf$uca$1@ger.gmane.org>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <muhtaf$uca$1@ger.gmane.org>
Message-ID: <27D77A4D-7CDE-470D-BD55-AC25CE9F782B@yahoo.com>

On Sep 30, 2015, at 17:04, Terry Reedy <tjreedy at udel.edu> wrote:
> 
>> On 9/30/2015 5:33 PM, Andrew Barnert via Python-ideas wrote:
>> 
>> (The fact that we don't have a term for "non-iterator iterable",
> 
> 'collection'

That's a perfectly good term, but it's not used that way in the docs, nor is anyone else using it in the discussions so far. Are you suggesting that we should start doing so?

There are definitely parts of the docs that could be clarified or simplified with this term, such as the glossary entry and definitions for dict views, which inaccurately use the term "sequence". (And similarly, although not quite as badly, someone in this thread referred to "sequences and sequence-like things", which may be a little more intuitive than my "non-iterator iterables", but still isn't all that clear.)

Also, the tutorial uses the phrases "data structures" or "data type" a few zillion times, apparently to avoid having to come up with a term that includes sequences, sets, dicts, and strings without being inaccurate. I've seen novices have no idea what "data structure" means, or get confused by what the difference between a "data type" and a "regular type" is.

> Some are concrete: they contain reference to actual Python objects.
> Some are virtual (lazy): they contain the information need to create Python objects as needed.

I think the docs used to use the word "virtual" as a more specific term than "lazy": a view onto an object that conceptually exists but doesn't actually exist is "virtual" (like range, which is a view into the infinite set of integers), but a view into a real object isn't (like dict_keys, which has a reference to an actual dict), nor is something that isn't conceptually view-like at all (like a RNG iterator), even though they're all "lazy". It looks like the word "virtual" in this context doesn't appear anywhere in the docs anymore, so I suppose it could be repurposed, but if it's just a synonym for "lazy", what's wrong with "lazy"?

From rob.cliffe at btinternet.com  Thu Oct  1 04:08:47 2015
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 1 Oct 2015 03:08:47 +0100
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560C68F0.4080600@mail.de>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <D91E3BCA-A368-4F9E-8C5A-11ABE744F391@gmail.com>
 <560BC0C3.4040502@btinternet.com> <560C68F0.4080600@mail.de>
Message-ID: <560C95AF.1020501@btinternet.com>



On 30/09/2015 23:57, Sven R. Kunze wrote:
> On 30.09.2015 13:00, Rob Cliffe wrote:
>> Or:
>>     x = a orelse b        # Visual Basic has a short-circuiting 
>> OrElse operator for boolean operands
>>     x = a orifNone b
>>
>>
>> On 30/09/2015 01:39, Ryan Gonzalez wrote:
>>> What about 'otherwise'?
>>>
>>> x = a otherwise b
>>>
>
>
> The only reason why I would prefer "else" over the proposed 
> alternatives: it's already a reserved keyword and it's not really 
> necessary to waste another one. Otherwise, I don't care too much.
>
> Best,
> Sven
Are you suggesting that "a else b" is a possibility?
That's a no-no because
     a if b else c else d
is ambiguous; it could mean either of
     a if b else (c else d)
     (a if b else c) else d
Rob

>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/67c361ef/attachment.html>

From rymg19 at gmail.com  Thu Oct  1 04:30:02 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 30 Sep 2015 21:30:02 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560C95AF.1020501@btinternet.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <D91E3BCA-A368-4F9E-8C5A-11ABE744F391@gmail.com>
 <560BC0C3.4040502@btinternet.com> <560C68F0.4080600@mail.de>
 <560C95AF.1020501@btinternet.com>
Message-ID: <BD1768A9-478A-401A-9E2A-3CBC7678EA57@gmail.com>



On September 30, 2015 9:08:47 PM CDT, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
>
>
>On 30/09/2015 23:57, Sven R. Kunze wrote:
>> On 30.09.2015 13:00, Rob Cliffe wrote:
>>> Or:
>>>     x = a orelse b        # Visual Basic has a short-circuiting 
>>> OrElse operator for boolean operands
>>>     x = a orifNone b
>>>
>>>
>>> On 30/09/2015 01:39, Ryan Gonzalez wrote:
>>>> What about 'otherwise'?
>>>>
>>>> x = a otherwise b
>>>>
>>
>>
>> The only reason why I would prefer "else" over the proposed 
>> alternatives: it's already a reserved keyword and it's not really 
>> necessary to waste another one. Otherwise, I don't care too much.
>>
>> Best,
>> Sven
>Are you suggesting that "a else b" is a possibility?
>That's a no-no because
>     a if b else c else d
>is ambiguous; it could mean either of
>     a if b else (c else d)
>     (a if b else c) else d
>Rob
>

I already said that...

>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From abarnert at yahoo.com  Thu Oct  1 04:31:23 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 19:31:23 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <87r3lf2vl4.fsf@gmail.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
Message-ID: <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>

On Sep 30, 2015, at 19:04, Akira Li <4kir4.1i at gmail.com> wrote:
> 
> Random832 <random832 at fastmail.com> writes:
> 
>> Akira Li <4kir4.1i at gmail.com> writes:
>> 
>>> Andrew Barnert via Python-ideas
>>> <python-ideas at python.org> writes:
>>> ...
>>>> (The fact that we don't have a term for "non-iterator iterable", and
>>> 
>>> All iterators are iterable but some iterables are not iterators.
>>> 
>>> If your code accepts only iterators then use the term *iterator*.
>>> Otherwise the term *iterable* could be used.
>>> 
>>> It is misleading to use *iterable* if your code only accepts iterators.
>>> 
>>> If an iterable is an iterator; It is called *iterator*. The term
>>> *iterable* implies that some instances are not iterators.
>> 
>> There are three (well, three and a half) kinds of code that consume
>> iterables, how would you describe each simply?
>> 
>> 1. Does not call iter, simply calls next. Therefore cannot consume a
>>   non-iterator iterable.
> iterator
> 
>> 2. Calls iter, but can accept an iterator (e.g. only goes through it
>>   once)
> iterable
> 
>> 3. Cannot accept an iterator (goes through it twice, or permanently
>>   stores a reference to it, etc)
> neither
> 
> *iterable* is an object that you can pass to iter() to get *iterator*.
> An iterable does not guarantee that it yields the same items twice.

And this is exactly the problem. We don't have any way to simply describe this thing. Hence all the confusion in this thread, and in similar discussions elsewhere, and even in the docs (e.g., describing dict views as sequences and then going on to explain that they're not actually sequences).

The fact that it took your previous message four paragraphs without inventing a new term, to say what I said in one sentence with a new term, demonstrates the problem. As does the fact that my attempted new term, "non-iterator iterable", is sufficiently ugly and not intuitively helpful enough that you felt the need to expand on it for four paragraphs.

From abarnert at yahoo.com  Thu Oct  1 04:39:13 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 19:39:13 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560C87A3.1020507@mrabarnett.plus.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
Message-ID: <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>

On Sep 30, 2015, at 18:08, MRAB <python at mrabarnett.plus.com> wrote:
> 
> It's only just occurred to me that there's a small inconsistency here.
> The "?.", "?[" and "?(" will short-circuit on None, whereas the "??"
> will short-circuit on non-None.
> 
> Would that cause any confusion in practice?

I noticed this when I was trying to write out grammar, sample ASTs, and sample bytecode for these things. I went searching the thread and saw no one had pointed it out. I went through docs and blogs for other languages, and didn't see anyone pointing out, complaining about, or offering to clear up any confusion. So I figured I wouldn't mention it, and see if anyone else even noticed.

The fact that an experienced programmer like you didn't even notice it until after a zillion messages over a span of weeks, and apparently nobody else did at all, seems to imply that there's no dangerously misleading intuitive parallel here.

(By the way, I have a feeling that including ?= will increase the risk of confusion, but I have no idea where that feeling comes from, so it may just be random noise in my head, maybe because I like ?. but don't particularly like ?= or something...)

From bruce at leban.us  Thu Oct  1 04:53:26 2015
From: bruce at leban.us (Bruce Leban)
Date: Wed, 30 Sep 2015 19:53:26 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560C87A3.1020507@mrabarnett.plus.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
Message-ID: <CAGu0AntFVOoHJQeYbapNFm6BXhWqZyssZHvMRgfsrn7tf7SAkA@mail.gmail.com>

I don't understand the resistance to using ?? ?. ?[] ?() and the quest to
contort some English words into this. The concepts "or" and "and" are
natural language concepts and "if null" is not. The goal here should be
readability and sometimes that readability is enhanced by being slightly
different than other programming languages and sometimes it's

For example, Java using -> for lambda is confusing to someone coming from
C++ where -> means de-reference. C# avoided that confound by using =>. I'm
sure someone thought that -> was more Javalike.

Very few people writing/reading Python only use Python. There is a strong
benefit to being consistent with other languages when there's not "one
obvious" way to do it in "more Pythonic" syntax. So far, *none* of the
alternatives are easier to read (for my taste) than the ? operators.

As to the people that are questioning the need for any of this, I challenge
you to look through your projects for code like this:

Python:  \bif\b.*\bis *(not *)?none *else
C-like:  ([=!]= *null\b|\bnull *[=!]=) *\?

and see how many hits you find. If you want to avoid using None (or null)
in your code, that's fine but the reality is that None is used extensively
*and* there are *many* cases where we do exactly what these operators
simplify.


On Wed, Sep 30, 2015 at 6:08 PM, MRAB <python at mrabarnett.plus.com> wrote:

>
>> It's only just occurred to me that there's a small inconsistency here.
> The "?.", "?[" and "?(" will short-circuit on None, whereas the "??"
> will short-circuit on non-None.
>
> Would that cause any confusion in practice?


Personally, I don't think so and I never thought about this as an area of
confusion. In the ?? case, it's obvious


Two more notes:

(1) If we have ?. ?() ?[] then spelling the ?? operator with a ? makes a
lot of sense but mixing a word and an operator (like or?) just looks ugly
to me. That said, if Guido declares absolutely no on ?. ?() ?[] and yes on
?? but it has to be spelled with words, my preference would be "*or else*".
It doesn't really say "if is None" to me but (i) it's a syntax error right
now; and (ii) the else in there implies some if-like logic is going on
rather than just or-like logic:

(foo or else bar) == (foo if foo is not None else bar).

(2) As to ?= or ??=, I am less enamored of that but not because that case
never occurs. I often find that the initialization code after if foo is
null: is more than a single expression and therefore not amenable to ?=.
That is, I don't just want to create an object and assign to the variable,
I want to make initialization calls to it.


--- Bruce
Check out my new puzzle book: http://J.mp/ingToConclusions
<http://j.mp/ingToConclusions>
Get it free here: http://J.mp/ingToConclusionsFree
<http://j.mp/ingToConclusionsFree> (available on iOS)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150930/9520d66e/attachment.html>

From random832 at fastmail.com  Thu Oct  1 06:03:50 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 00:03:50 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
Message-ID: <m2bncj8cbd.fsf@fastmail.com>

Akira Li <4kir4.1i at gmail.com> writes:
>> 3. Cannot accept an iterator (goes through it twice, or permanently
>>    stores a reference to it, etc)
> neither
>
> *iterable* is an object that you can pass to iter() to get *iterator*.
> An iterable does not guarantee that it yields the same items twice.

True or false?: It is reasonable to write algorithms that iterate twice
over a passed-in iterable, with the expectation that said iterable will
typically be an object (or a view of such an object) which will not be
concurrently modified (e.g. by a different thread or by a side-effect of
a callback) during the execution of the algorithm, but which does not
behave in a useful way when given an iterator, a generator, or any other
kind of iterable which exhibits similar behavior whereby the second and
further attempts to iterate will yield no items.


From emile at fenx.com  Thu Oct  1 06:05:51 2015
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 30 Sep 2015 21:05:51 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <27D77A4D-7CDE-470D-BD55-AC25CE9F782B@yahoo.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <muhtaf$uca$1@ger.gmane.org>
 <27D77A4D-7CDE-470D-BD55-AC25CE9F782B@yahoo.com>
Message-ID: <muibf8$4t0$1@ger.gmane.org>

On 9/30/2015 7:24 PM, Andrew Barnert via Python-ideas wrote:
> Also, the tutorial uses the phrases "data structures" or "data type" a few zillion times, apparently to avoid having to come up with a term that includes sequences, sets, dicts, and strings without being inaccurate. I've seen novices have no idea what "data structure" means, or get confused by what the difference between a "data type" and a "regular type" is.

container?

https://docs.python.org/3/library/collections.html

Emile


From random832 at fastmail.com  Thu Oct  1 06:13:50 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 00:13:50 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <muhtaf$uca$1@ger.gmane.org>
 <27D77A4D-7CDE-470D-BD55-AC25CE9F782B@yahoo.com>
Message-ID: <m237xv8bup.fsf@fastmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:
> I think the docs used to use the word "virtual" as a more specific
> term than "lazy": a view onto an object that conceptually exists but
> doesn't actually exist is "virtual" (like range, which is a view into
> the infinite set of integers), but a view into a real object isn't
> (like dict_keys, which has a reference to an actual dict), nor is
> something that isn't conceptually view-like at all (like a RNG
> iterator), even though they're all "lazy".

If either "lazy" or "virtual" means that the contained objects don't
exist as python objects until they are accessed, doesn't this extend to
strings and arrays (and byte strings, byte arrays, and memory views)?


From random832 at fastmail.com  Thu Oct  1 06:19:08 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 00:19:08 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
Message-ID: <m2vbar6x1f.fsf@fastmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:

> On Sep 30, 2015, at 18:08, MRAB <python at mrabarnett.plus.com> wrote:
>> 
>> It's only just occurred to me that there's a small inconsistency here.
>> The "?.", "?[" and "?(" will short-circuit on None, whereas the "??"
>> will short-circuit on non-None.
>> 
>> Would that cause any confusion in practice?
>
> I noticed this when I was trying to write out grammar, sample ASTs,
> and sample bytecode for these things. I went searching the thread and
> saw no one had pointed it out. I went through docs and blogs for other
> languages, and didn't see anyone pointing out, complaining about, or
> offering to clear up any confusion. So I figured I wouldn't mention
> it, and see if anyone else even noticed.

How is it worse than the fact that and short-circuits on true whereas or
short-circuits on false? Short-circuiting logically applies to the case
that *can* be short-circuited.

For the AST issue, I'm curious as to what you ended up doing about the
whole-atom_expr nature of the short-circuiting and the fact that ASTs
don't currently represent an atom_expr as a single object containing a
list of subscript/attribute/call items?


From 4kir4.1i at gmail.com  Thu Oct  1 07:15:25 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Thu, 01 Oct 2015 08:15:25 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
Message-ID: <87bncj2mqa.fsf@gmail.com>

Random832 <random832 at fastmail.com> writes:

> Akira Li <4kir4.1i at gmail.com> writes:
>>> 3. Cannot accept an iterator (goes through it twice, or permanently
>>>    stores a reference to it, etc)
>> neither
>>
>> *iterable* is an object that you can pass to iter() to get *iterator*.
>> An iterable does not guarantee that it yields the same items twice.
>
> True or false?: It is reasonable to write algorithms that iterate twice
> over a passed-in iterable, with the expectation that said iterable will
> typically be an object (or a view of such an object) which will not be
> concurrently modified (e.g. by a different thread or by a side-effect of
> a callback) during the execution of the algorithm, but which does not
> behave in a useful way when given an iterator, a generator, or any other
> kind of iterable which exhibits similar behavior whereby the second and
> further attempts to iterate will yield no items.
>
True or false?: do all iterables return the same items twice?
  http://www.fallacyfiles.org/loadques.html

Specific application may use more specific requirements e.g.:

list(iterable):

- does it mean that all iterables must be finite?
- do we need a special word to describe what list() accepts?

set(iterable):

- does it mean that all iterables must yield hashable items?
- do we need a special word to describe what set() accepts?

dict(iterable):

- does it mean that all iterables must yield pairs?
- do we need a special word to describe what dict() accepts?


You've got the idea: the word *iterable* may be used in the context when
not all iterables are accepted.


From rosuav at gmail.com  Thu Oct  1 07:30:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 1 Oct 2015 15:30:14 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151001013106.GC23642@ando.pearwood.info>
References: <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <20151001013106.GC23642@ando.pearwood.info>
Message-ID: <CAPTjJmr-r-tNiyjMZzJ2LOAoj=RN2NHb1drsfXwOywfkkU5vTQ@mail.gmail.com>

On Thu, Oct 1, 2015 at 11:31 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Personally, I don't like it, but using "n in range(a, b+1)" for testing
> whether integer n falls within a particular range seems to be popular. I
> don't know why they don't just write a <= n <= b, but it seems to be a
> popular idiom for some weird reason.

You could have a third arg to range, in which case it also adds in a
modulo check.

ChrisA

From 4kir4.1i at gmail.com  Thu Oct  1 07:39:51 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Thu, 01 Oct 2015 08:39:51 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
Message-ID: <87a8s32llk.fsf@gmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:

> On Sep 30, 2015, at 19:04, Akira Li <4kir4.1i at gmail.com> wrote:
>>
>> Random832 <random832 at fastmail.com> writes:
>>
>>> Akira Li <4kir4.1i at gmail.com> writes:
>>>
>>>> Andrew Barnert via Python-ideas
>>>> <python-ideas at python.org> writes:
>>>> ...
>>>>> (The fact that we don't have a term for "non-iterator iterable", and
>>>>
>>>> All iterators are iterable but some iterables are not iterators.
>>>>
>>>> If your code accepts only iterators then use the term *iterator*.
>>>> Otherwise the term *iterable* could be used.
>>>>
>>>> It is misleading to use *iterable* if your code only accepts iterators.
>>>>
>>>> If an iterable is an iterator; It is called *iterator*. The term
>>>> *iterable* implies that some instances are not iterators.
>>>
>>> There are three (well, three and a half) kinds of code that consume
>>> iterables, how would you describe each simply?
>>>
>>> 1. Does not call iter, simply calls next. Therefore cannot consume a
>>>   non-iterator iterable.
>> iterator
>>
>>> 2. Calls iter, but can accept an iterator (e.g. only goes through it
>>>   once)
>> iterable
>>
>>> 3. Cannot accept an iterator (goes through it twice, or permanently
>>>   stores a reference to it, etc)
>> neither
>>
>> *iterable* is an object that you can pass to iter() to get *iterator*.
>> An iterable does not guarantee that it yields the same items twice.
>
> And this is exactly the problem. We don't have any way to simply
> describe this thing. Hence all the confusion in this thread, and in
> similar discussions elsewhere, and even in the docs (e.g., describing
> dict views as sequences and then going on to explain that they're not
> actually sequences).

Use *iterable* instead of "non-iterator iterable" -- it is that simple.

"dict views" seems a pretty good term for dict views.
Are you suggesting to call dict views "non-iterator iterable"?
I don't see that it says more than that all dict views are iterable.

It seems there is a bug in the glossary: the entry name should be "dict
views", not just "view" that is too generic for the description.

I've submitted a patch http://bugs.python.org/issue25286

> The fact that it took your previous message four paragraphs without
> inventing a new term, to say what I said in one sentence with a new
> term, demonstrates the problem. As does the fact that my attempted new
> term, "non-iterator iterable", is sufficiently ugly and not
> intuitively helpful enough that you felt the need to expand on it for
> four paragraphs.

I don't need 4 paragraphs to describe it:

  if you need an iterator; use the term *iterator* --
  otherwise use *iterable* unless you need something more specific e.g.,
  *seq* name is common for generic sequences

I don't remember ever using "non-iterator iterable". "non-iterator
iterable" does not qualify as more specific. You need to introduce new
requirements to the type for that.


From random832 at fastmail.com  Thu Oct  1 07:45:11 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 01:45:11 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
 <87a8s32llk.fsf@gmail.com>
Message-ID: <m2r3lf6t20.fsf@fastmail.com>

Akira Li <4kir4.1i at gmail.com> writes:
> I don't remember ever using "non-iterator iterable". "non-iterator
> iterable" does not qualify as more specific. You need to introduce new
> requirements to the type for that.

The question is, how do you *simply* state the very common requirement
for an iterable to not behave in a specific undesirable way that all
iterators do, and that it is very uncommon for any iterable other than
an iterator to do? Are you opposed to having a word for this concept at
all, or do you just not like the terms other people are suggesting?


From 4kir4.1i at gmail.com  Thu Oct  1 07:59:24 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Thu, 01 Oct 2015 08:59:24 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
 <87a8s32llk.fsf@gmail.com> <m2r3lf6t20.fsf@fastmail.com>
Message-ID: <87612r2koz.fsf@gmail.com>

Random832 <random832 at fastmail.com> writes:

> Akira Li <4kir4.1i at gmail.com> writes:
>> I don't remember ever using "non-iterator iterable". "non-iterator
>> iterable" does not qualify as more specific. You need to introduce new
>> requirements to the type for that.
>
> The question is, how do you *simply* state the very common requirement
> for an iterable to not behave in a specific undesirable way that all
> iterators do, and that it is very uncommon for any iterable other than
> an iterator to do? Are you opposed to having a word for this concept at
> all, or do you just not like the terms other people are suggesting?

That term is **iterable**. As I already said:

Specific application may use more specific requirements e.g.:

list(iterable):

- does it mean that all iterables must be finite?
- do we need a special word to describe what list() accepts?

set(iterable):

- does it mean that all iterables must yield hashable items?
- do we need a special word to describe what set() accepts?

dict(iterable):

- does it mean that all iterables must yield pairs?
- do we need a special word to describe what dict() accepts?


You've got the idea: the word *iterable* may be used in the context when
not all iterables are accepted.

  https://mail.python.org/pipermail/python-ideas/2015-October/036692.html


From abarnert at yahoo.com  Thu Oct  1 07:59:32 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 22:59:32 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <m2vbar6x1f.fsf@fastmail.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
Message-ID: <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>

On Sep 30, 2015, at 21:19, Random832 <random832 at fastmail.com> wrote:
> 
> Andrew Barnert via Python-ideas
> <python-ideas at python.org> writes:
> 
>>> On Sep 30, 2015, at 18:08, MRAB <python at mrabarnett.plus.com> wrote:
>>> 
>>> It's only just occurred to me that there's a small inconsistency here.
>>> The "?.", "?[" and "?(" will short-circuit on None, whereas the "??"
>>> will short-circuit on non-None.
>>> 
>>> Would that cause any confusion in practice?
>> 
>> I noticed this when I was trying to write out grammar, sample ASTs,
>> and sample bytecode for these things. I went searching the thread and
>> saw no one had pointed it out. I went through docs and blogs for other
>> languages, and didn't see anyone pointing out, complaining about, or
>> offering to clear up any confusion. So I figured I wouldn't mention
>> it, and see if anyone else even noticed.
> 
> How is it worse than the fact that and short-circuits on true whereas or
> short-circuits on false?

Why are you asking me how it's worse when my conclusion was that it's fine, in the sentence right after the part you quoted?

> For the AST issue, I'm curious as to what you ended up doing about the
> whole-atom_expr nature of the short-circuiting and the fact that ASTs
> don't currently represent an atom_expr as a single object containing a
> list of subscript/attribute/call items?

I posted ?. examples earlier; I don't want to repeat the whole thing (especially after Guido pointed out that it probably wasn't helping anyone who didn't already get it). But briefly, the AST doesn't have to represent the short-circuiting here, any more than it does anywhere else that short-circuits; it just adds an uptalk flag to each Attribute node. At code generation time, any Attribute node that has uptalk=True has a JUMP_IF_NONE to after the primary-or-call (leaving the None attrib value on the stack) after the LOAD_ATTR. (Or, if you don't want to add a new bytecode, it has a string of three ops that do the equivalent.)

The same works for ?[]. For ?(), I'm not sure what the desired semantics are, and for ?? it seemed obviously trivial so I didn't bother.


From abarnert at yahoo.com  Thu Oct  1 08:07:47 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 23:07:47 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <m237xv8bup.fsf@fastmail.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <muhtaf$uca$1@ger.gmane.org>
 <27D77A4D-7CDE-470D-BD55-AC25CE9F782B@yahoo.com>
 <m237xv8bup.fsf@fastmail.com>
Message-ID: <AE4A7EA5-9AAC-494B-AA4A-12543F05EB91@yahoo.com>

On Sep 30, 2015, at 21:13, Random832 <random832 at fastmail.com> wrote:
> 
> Andrew Barnert via Python-ideas
> <python-ideas at python.org> writes:
>> I think the docs used to use the word "virtual" as a more specific
>> term than "lazy": a view onto an object that conceptually exists but
>> doesn't actually exist is "virtual" (like range, which is a view into
>> the infinite set of integers), but a view into a real object isn't
>> (like dict_keys, which has a reference to an actual dict), nor is
>> something that isn't conceptually view-like at all (like a RNG
>> iterator), even though they're all "lazy".
> 
> If either "lazy" or "virtual" means that the contained objects don't
> exist as python objects until they are accessed, doesn't this extend to
> strings and arrays (and byte strings, byte arrays, and memory views)?

There's a sense in which that's true, and in some discussions (e.g., intimately involving the GC or object sharing or optimization of array algorithms) that would be the most relevant sense, but there's also a sense in which they concretely hold all the values in memory, and in most discussions (e.g., talking about generic sequence algorithms) that would be more relevant. I don't think there's a major problem here?we don't need to eliminate all ambiguity from our speech, only the ambiguity that actually gets in the way.

From abarnert at yahoo.com  Thu Oct  1 08:24:24 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 23:24:24 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <87a8s32llk.fsf@gmail.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com> <87a8s32llk.fsf@gmail.com>
Message-ID: <EECDF35E-7A42-406E-ABB7-2BB8226B3114@yahoo.com>

On Sep 30, 2015, at 22:39, Akira Li <4kir4.1i at gmail.com> wrote:
> 
> Andrew Barnert via Python-ideas
> <python-ideas at python.org> writes:
> 
>>> On Sep 30, 2015, at 19:04, Akira Li <4kir4.1i at gmail.com> wrote:
>>> 
>>> Random832 <random832 at fastmail.com> writes:
>>> 
>>>> Akira Li <4kir4.1i at gmail.com> writes:
>>>> 
>>>>> Andrew Barnert via Python-ideas
>>>>> <python-ideas at python.org> writes:
>>>>> ...
>>>>>> (The fact that we don't have a term for "non-iterator iterable", and
>>>>> 
>>>>> All iterators are iterable but some iterables are not iterators.
>>>>> 
>>>>> If your code accepts only iterators then use the term *iterator*.
>>>>> Otherwise the term *iterable* could be used.
>>>>> 
>>>>> It is misleading to use *iterable* if your code only accepts iterators.
>>>>> 
>>>>> If an iterable is an iterator; It is called *iterator*. The term
>>>>> *iterable* implies that some instances are not iterators.
>>>> 
>>>> There are three (well, three and a half) kinds of code that consume
>>>> iterables, how would you describe each simply?
>>>> 
>>>> 1. Does not call iter, simply calls next. Therefore cannot consume a
>>>>  non-iterator iterable.
>>> iterator
>>> 
>>>> 2. Calls iter, but can accept an iterator (e.g. only goes through it
>>>>  once)
>>> iterable
>>> 
>>>> 3. Cannot accept an iterator (goes through it twice, or permanently
>>>>  stores a reference to it, etc)
>>> neither
>>> 
>>> *iterable* is an object that you can pass to iter() to get *iterator*.
>>> An iterable does not guarantee that it yields the same items twice.
>> 
>> And this is exactly the problem. We don't have any way to simply
>> describe this thing. Hence all the confusion in this thread, and in
>> similar discussions elsewhere, and even in the docs (e.g., describing
>> dict views as sequences and then going on to explain that they're not
>> actually sequences).
> 
> Use *iterable* instead of "non-iterator iterable" -- it is that simple.

No it isn't. The word "iterable" just means "iterable". When you want to talk about sequences?a subtype of iterables?you don't just say "iterable", you say "sequence". And likewise, when you want to talk about iterables that aren't iterators, or iterables that are repeatable, or any other subtype of iterables, you have to use a word (or phrase) that actually means what you're saying.

I don't know how to explain this any better. Everyone else seems to get it, but you just post the same reply to each of them that you posted to me when they try to explain further. What am I not getting across here?

> "dict views" seems a pretty good term for dict views.
> Are you suggesting to call dict views "non-iterator iterable"?

Why would you think that?

> I don't see that it says more than that all dict views are iterable.
> 
> It seems there is a bug in the glossary: the entry name should be "dict
> views", not just "view" that is too generic for the description.

There's a much larger problem. The glossary says that dict views are sequences. They aren't. The actual documentation for dict views is a little better, because it explains that they're not actually sequences. But the problem is still there: what the docs are trying to say is that dict views are some kind of non-iterator iterable, but, because we don't have a term form that, they use the incorrect term "sequence".

> I've submitted a patch http://bugs.python.org/issue25286
> 
>> The fact that it took your previous message four paragraphs without
>> inventing a new term, to say what I said in one sentence with a new
>> term, demonstrates the problem. As does the fact that my attempted new
>> term, "non-iterator iterable", is sufficiently ugly and not
>> intuitively helpful enough that you felt the need to expand on it for
>> four paragraphs.
> 
> I don't need 4 paragraphs to describe it:
> 
>  if you need an iterator; use the term *iterator* --
>  otherwise use *iterable* unless you need something more specific e.g.,
>  *seq* name is common for generic sequences

Why would you use "seq" instead of "sequence" for the name of the abstract sequence type?

And, more importantly, what name do you use when you need something more specific than "iterable", but less specific than "sequence"?as in the glossary entry for dict views, for example?

> I don't remember ever using "non-iterator iterable".

Why would you expect to remember using it, when you're replying to a message where I invented it for lack of an established better name (and in hopes that someone would come up with one)?

> "non-iterator
> iterable" does not qualify as more specific. You need to introduce new
> requirements to the type for that.

There are things that are iterables, that are not non-iterator iterables, but the reverse is not true. It's a name for a strict subset. Which means it's more specific.

As for a new requirement: an iterable is a non-iterator iterable if its __iter__ method does not return self.

(If you're going to argue that this requirement can't be checked by, e.g., a structural type checker, remember that neither is the distinction between sequence and mapping, and that doesn't mean they're the same type.)

From abarnert at yahoo.com  Thu Oct  1 08:34:34 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 23:34:34 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <87612r2koz.fsf@gmail.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com> <87a8s32llk.fsf@gmail.com>
 <m2r3lf6t20.fsf@fastmail.com> <87612r2koz.fsf@gmail.com>
Message-ID: <D05C8C1A-4BE0-49E0-BD38-B0B6CB150BD8@yahoo.com>

On Sep 30, 2015, at 22:59, Akira Li <4kir4.1i at gmail.com> wrote:
> 
> Random832 <random832 at fastmail.com> writes:
> 
>> Akira Li <4kir4.1i at gmail.com> writes:
>>> I don't remember ever using "non-iterator iterable". "non-iterator
>>> iterable" does not qualify as more specific. You need to introduce new
>>> requirements to the type for that.
>> 
>> The question is, how do you *simply* state the very common requirement
>> for an iterable to not behave in a specific undesirable way that all
>> iterators do, and that it is very uncommon for any iterable other than
>> an iterator to do? Are you opposed to having a word for this concept at
>> all, or do you just not like the terms other people are suggesting?
> 
> That term is **iterable**. As I already said:
> 
> Specific application may use more specific requirements e.g.:
> 
> list(iterable):
> 
> - does it mean that all iterables must be finite?
> - do we need a special word to describe what list() accepts?
> 
> set(iterable):
> 
> - does it mean that all iterables must yield hashable items?
> - do we need a special word to describe what set() accepts?
> 
> dict(iterable):
> 
> - does it mean that all iterables must yield pairs?
> - do we need a special word to describe what dict() accepts?
> 
> 
> You've got the idea: the word *iterable* may be used in the context when
> not all iterables are accepted.
> 
>  https://mail.python.org/pipermail/python-ideas/2015-October/036692.html

This is a link to a reply where you pasted exactly the same text as in this reply?and in a third one. What is that supposed to mean?

I feel like you must be trying to get across something really important here, and it's my fault for not getting it, but I still can't get it. Can you try rewording it instead of just pasting the same text again and/or a link to the same text?

If it helps, let me try to ask specific questions:

Are you arguing one of the following:

 * there is no such thing as an iterable that isn't an iterator, or an iterable that is repeatable, or an iterable that provides a new iterator each time iter is called?
 * there are such things, but no corresponding property that can be used to characterize a set?
 * that such sets do exist, but are never useful to discuss?
 * that such sets may be useful to discuss, but the names I (and Terry and others) came up with are unhelpful?

More concretely: the documentation for dict views goes out of its way to point out that these are not iterators, but a different kind of iterable that's more like a sequence (presumably meaning at least one of the three things above). But it does so inaccurately, by saying they are sequences, which is not true. How could it be rewritten to get that point across accurately, but still concisely and readably?

From abarnert at yahoo.com  Thu Oct  1 08:39:04 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 30 Sep 2015 23:39:04 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <muibf8$4t0$1@ger.gmane.org>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <muhtaf$uca$1@ger.gmane.org>
 <27D77A4D-7CDE-470D-BD55-AC25CE9F782B@yahoo.com> <muibf8$4t0$1@ger.gmane.org>
Message-ID: <9F8724E9-9B86-4789-AD5C-491F6AC864F0@yahoo.com>

On Sep 30, 2015, at 21:05, Emile van Sebille <emile at fenx.com> wrote:
> 
>> On 9/30/2015 7:24 PM, Andrew Barnert via Python-ideas wrote:
>> Also, the tutorial uses the phrases "data structures" or "data type" a few zillion times, apparently to avoid having to come up with a term that includes sequences, sets, dicts, and strings without being inaccurate. I've seen novices have no idea what "data structure" means, or get confused by what the difference between a "data type" and a "regular type" is.
> 
> container?

But that means something with a __contains__ test. Containers don't even have to be iterables. It's true that all of the types discussed in the tutorial are containers, but is that actually the meaning we're looking for, or just something that's coincidentally true?

At any rate, even if that does work for the tutorial, I don't think it solves the more general problem. When I want to talk about iterables that give you a different, independent iterator each time you call __iter__, "container" is not the right word for that. Terry's "collection" seems like a better choice, because it doesn't already have a conflicting meaning.

From brenbarn at brenbarn.net  Thu Oct  1 08:44:04 2015
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Wed, 30 Sep 2015 23:44:04 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <EECDF35E-7A42-406E-ABB7-2BB8226B3114@yahoo.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com> <87a8s32llk.fsf@gmail.com>
 <EECDF35E-7A42-406E-ABB7-2BB8226B3114@yaho o.com>
Message-ID: <560CD634.6080509@brenbarn.net>

On 2015-09-30 23:24, Andrew Barnert via Python-ideas wrote:
> There are things that are iterables, that are not non-iterator
> iterables, but the reverse is not true. It's a name for a strict
> subset. Which means it's more specific.
>
> As for a new requirement: an iterable is a non-iterator iterable if
> its __iter__ method does not return self.

	Not sure I followed all the discussion of these terms, but is your main 
reason for wanting this term to describe the behavior that non-iterator 
iterables can be "restarted" (and are so restarted if reused in a 
different context)?

	Personally I prefer to take a duck-typing view and focus on what 
operations you can or can't do on these various things.  Whether you 
call it a view or a virtual indexer or whatever is, to me, less 
important than what you can do with the object.  I agree there are a 
number of relevant subcategories of objects here, some of which we have 
a name for and some that we don't.  But I think it gets easier if we 
move from generic nouns like "view" to specific adjectives describing 
the behaviors the object support.  Something like "re-entrant iterable" 
(meaning if you use it in two for loops right after each other you get 
the whole thing both times) would focus on that aspect of the behavior. 
  Something like "random-accessible" or "sliceable" if we want to talk 
about iterables where we can "jump ahead" or slice if needed.

	It's an interesting idea to think about what kinds of operations (map, 
filter, etc.) could return iterables supporting what other kinds of 
operations.  That is, can we make sure the result of map/filter can be 
sliced/indexed/reentered if the source can.  To me the interesting 
question is which of these actual behaviors can usefully and 
non-mind-bendingly be preserved through map/filter/etc. manipulations.

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no
path, and leave a trail."
    --author unknown

From stephen at xemacs.org  Thu Oct  1 09:13:06 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 1 Oct 2015 16:13:06 +0900
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <muheqk$ute$1@ger.gmane.org>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <CALGmxEJk94+4CgKK3tyOUxD3kmAWZzNnXsX2qsjDzj8eoffdpw@mail.gmail.com>
 <muheqk$ute$1@ger.gmane.org>
Message-ID: <22028.56578.901600.144640@turnbull.sk.tsukuba.ac.jp>

Terry Reedy writes:
 > On 9/30/2015 1:28 PM, Chris Barker wrote:
 > 
 > > But again, we could add indexing to enumerate, and have it do the ugly
 > > inefficient thing when it's using an underlying non-indexable iterator,
 > 
 > If the ugly inefficient thing is to call list(iterable), then that does 
 > not work with unbounded iterables.

I think he means

    from itertools import islice
    a = list(islice(iterable, 0, 99))[42]

 > Or the input iterable might produce inputs at various times in the
 > future.

Horrors!  We'll have to add a "block=False" parameter to next().  (We
can bikeshed on the default later.)

Seriously, I think that one we just have to live with, just as we
already live with it in any context where we access an iterable.

Regards,



From ncoghlan at gmail.com  Thu Oct  1 09:59:11 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 1 Oct 2015 17:59:11 +1000
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
Message-ID: <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>

On 1 October 2015 at 12:39, Andrew Barnert via Python-ideas
<python-ideas at python.org> wrote:
> On Sep 30, 2015, at 18:08, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> It's only just occurred to me that there's a small inconsistency here.
>> The "?.", "?[" and "?(" will short-circuit on None, whereas the "??"
>> will short-circuit on non-None.
>>
>> Would that cause any confusion in practice?
>
> I noticed this when I was trying to write out grammar, sample ASTs, and sample bytecode for these things. I went searching the thread and saw no one had pointed it out. I went through docs and blogs for other languages, and didn't see anyone pointing out, complaining about, or offering to clear up any confusion. So I figured I wouldn't mention it, and see if anyone else even noticed.
>
> The fact that an experienced programmer like you didn't even notice it until after a zillion messages over a span of weeks, and apparently nobody else did at all, seems to imply that there's no dangerously misleading intuitive parallel here.
>
> (By the way, I have a feeling that including ?= will increase the risk of confusion, but I have no idea where that feeling comes from, so it may just be random noise in my head, maybe because I like ?. but don't particularly like ?= or something...)

Because unlike "?.", "?[" and "?(", "?=" would also shortcircuit on
"if not None" if expanded as a ternary expression:

    target = target if target is not None else default

However, it's possible to make it consistent by instead expanding it as:

    if target is None: target = default

As a result, one interesting way of looking at this problem is to ask:
what if we *only* offered the conditional operations, *without*
offering a binary null-coalescing operator?

That is, we could allow conditional assignment:

    data ?= []
    headers ?= {}
    arg ?= make_default()

With the meaning:

    if data is None: data = []
    if headers is None: headers = {}
    if arg is None: arg = make_default()

That would deal directly with the idiomatic case of handling a default
argument of "None" and replacing it with a mutable container or
expensive to calculate default value.

Multiple levels of coalescence would need to be spelled out as
multiple statements rather than as chained binary operations (in this
case, I think putting it all on one line helps make the "first
non-None value wins" semantics clearer):

    title ?= user_title; title ?= local_default_title;  title ?=
global_default_title

The semantics of dict.setdefault() could also potentially be made
clearer, as "d[key] ?= make_default()" could become a short circuiting
equivalent of "dict.setdefault(k, make_default())":

    try:
        d[key]
    except KeyError:
        d[key] = make_default()

Custom sentinels would still need to be spelled out with an if statement:

    if arg is _sentinel:
        arg = make_default()

If they were offered, conditional attribute access, conditional item
lookup and conditional calls would use the same "LHS is None" check,
but in a ternary expression rather than an if statement, with the
following:

   title?.upper()
   person?['name']
   func?(arg)

being equivalent to:

    None if title is None else title.upper()
    None if person is None else person['name']
    None if func is None else func(arg)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Thu Oct  1 17:10:28 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 01:10:28 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <87bncj2mqa.fsf@gmail.com>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
Message-ID: <20151001151028.GD23642@ando.pearwood.info>

On Thu, Oct 01, 2015 at 08:15:25AM +0300, Akira Li wrote:

> True or false?: do all iterables return the same items twice?
>   http://www.fallacyfiles.org/loadques.html

[Aside: I have no idea what point you are making with the above link.]

Of course they don't necessarily do so, but those that don't are not 
necessarily well-behaved. 

In the case of sequences and collections, the concept is that (absent 
any explicit mutation operation), iterating over it twice *should* give 
the same results, that is the normal expectation. But that isn't 
enforced, we can write something that breaks that rule:

class WeirdIterable:
    def __getitem__(self, i):
        if random.random() > 0.9: raise IndexError
        return random.choice(["fe", "fi", "fo", "fum"])

but most people would consider that to be a pathological case. Yes, you 
can do it, and maybe you have a reason to do so, but you can't expect 
other people's code to deal with it gracefully.

In the case of iterators, the answer is *certainly not*.

Iterators are designed for the express purpose of handling not just the 
"lazy sequence" case where you choose to calculate results on demand as 
an optimization, but the case where you *have no choice* because the 
results are coming from some source which may change from run to run, 
e.g. an external data source. An iterator *may* repeat if run twice, but 
there is no expectation that it will do so. It's not just that the rule 
about repeatability is not enforced, but that there is no such rule in 
the first place.

(By the way, when I talk about running an iterator twice, I'm completely 
aware that technically you cannot ever do so. What I mean is to iterate 
over the object, then *recreate the object* in some sense, then iterate 
over it again.)


> Specific application may use more specific requirements e.g.:
> 
> list(iterable):
> 
> - does it mean that all iterables must be finite?
> - do we need a special word to describe what list() accepts?

No, and no.

In principle, list() will quite happily create an infinite list for you, 
if you have infinite memory :-) The fact that in practice lists are 
probably limited to something of the order of 2**64 items or less is a 
mere quality of implementation issue :-)

But to be more serious, no, in context we should understand that lists 
have actual physical limits, and even finite iterables may not be 
capable of being turned into lists:

def gen():
    for i in range(10**10000):
        yield i

Perfectly finite in size, but you cannot have a list that big. It's not 
just *infinite iterables* which are prohibited, that's just a special 
case of iterables that will provide more items than you have memory to 
store. And that's not a fixed limit, it will differ from machine to 
machine.


[...]
> You've got the idea: the word *iterable* may be used in the context when
> not all iterables are accepted.

Sure. But the distinction is that while there are a whole lot of 
different iterables:

- iterables with a sufficiently small number of items
- iterables of hashable items
- iterables of (hashable key, item) pairs
- iterables of prime numbers less than one million
- iterables of strings containing exactly 1 vowel

etc they are special cases and don't need specialised names. But there 
is a *general* distinction between two cases:

- iterables which are iterators
- iterables which are not iterators

We have a name for the first set: "iterators". But we don't have a name 
for the second set. Andrew suggested "non-iterator iterables" is too 
clumsy for general use, and suggests we need a better name. You 
suggested "iterables", but that clearly cannot work, since iterators are 
a kind of iterable.




-- 
Steve

From christoph at grothesque.org  Thu Oct  1 18:27:01 2015
From: christoph at grothesque.org (Christoph Groth)
Date: Thu, 01 Oct 2015 18:27:01 +0200
Subject: [Python-ideas] asyncio: factoring-out of the general mechanism
Message-ID: <877fn61rmy.fsf@grothesque.org>

Hello,

I have a question/idea about the support for asynchronous programming in
Python.

With Python 3.5, support for asynchronous programming has been added to
the core Python language.  The new language features, however, are only
useful with an event loop, as provided (only?) by the asyncio stdlib
module.

I notice that already the class asyncio.AbstractEventLoop contains many
things that to me seem very specific to particular applications.  For
examples that class contains placeholders for methods that expect IP
host names and IP ports as arguments.

In a world where networking is done in other ways than through IP,
Python would be still useful.  One could keep using most of the standard
library that is not specific to IP networking.

Asynchronous programming is a concept that is way more universal than
many of the things in asyncio (IP ports, unix pipes, etc.).  Has there
been a discussion about separating the abstract bits needed for
asynchronous programming in Python from the application-specific ones?
Or would this be a bad idea for some reason?

Christoph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 810 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/2b3e1a62/attachment.sig>

From guido at python.org  Thu Oct  1 19:07:43 2015
From: guido at python.org (Guido van Rossum)
Date: Thu, 1 Oct 2015 10:07:43 -0700
Subject: [Python-ideas] asyncio: factoring-out of the general mechanism
In-Reply-To: <877fn61rmy.fsf@grothesque.org>
References: <877fn61rmy.fsf@grothesque.org>
Message-ID: <CAP7+vJLOMF7_3DqvCiFkug9=-AztCPHzFL6oWiwoAVLXrYRLTQ@mail.gmail.com>

Well, what else did you have in mind? Remember that until we fix the GIL
(which won't happen until Python 4) you won't get benefit from async
programming unless you are overlapping with I/O, and (due to the
limitations of select/poll/etc.) that is pretty much limited to network and
IPC (disk I/O in particular cannot be overlapped in this way).

The main other area of interest would be UI events (as in a browser or GUI)
and for that I think an adaptation of the asyncio event loop will be
suitable. (If you're not going to use them it's fine not to implement the
network-specific methods on an event loop.)

Note that the async/await keywords themselves have no direct link to an
event loop -- they are just a different way to spell generators. It's just
that they are only useful with asyncio or some other event loop library.

On Thu, Oct 1, 2015 at 9:27 AM, Christoph Groth <christoph at grothesque.org>
wrote:

> Hello,
>
> I have a question/idea about the support for asynchronous programming in
> Python.
>
> With Python 3.5, support for asynchronous programming has been added to
> the core Python language.  The new language features, however, are only
> useful with an event loop, as provided (only?) by the asyncio stdlib
> module.
>
> I notice that already the class asyncio.AbstractEventLoop contains many
> things that to me seem very specific to particular applications.  For
> examples that class contains placeholders for methods that expect IP
> host names and IP ports as arguments.
>
> In a world where networking is done in other ways than through IP,
> Python would be still useful.  One could keep using most of the standard
> library that is not specific to IP networking.
>
> Asynchronous programming is a concept that is way more universal than
> many of the things in asyncio (IP ports, unix pipes, etc.).  Has there
> been a discussion about separating the abstract bits needed for
> asynchronous programming in Python from the application-specific ones?
> Or would this be a bad idea for some reason?
>
> Christoph
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/f96402d0/attachment.html>

From steve at pearwood.info  Thu Oct  1 19:12:38 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 03:12:38 +1000
Subject: [Python-ideas] secrets module -- secret.keeper?
In-Reply-To: <CA+OGgf6bbMcfP=UmkKTN93Bkg+2UyTuLKz==GZeZZ0yxowyV4g@mail.gmail.com>
References: <CA+OGgf6bbMcfP=UmkKTN93Bkg+2UyTuLKz==GZeZZ0yxowyV4g@mail.gmail.com>
Message-ID: <20151001171238.GE23642@ando.pearwood.info>

On Wed, Sep 30, 2015 at 01:55:49PM -0400, Jim J. Jewett wrote:

> Will the secrets module offer any building blocks to actually protect a secret?
> 
> e.g.,
> 
> an easy way to encrypt a file with a given password?
> an encrypted datastore?

I don't know. I think probably not -- they sound a bit too high level, 
at least for a first version. But if people want to propose functions to 
do so, they can be considered.


> a getpass that works even in IDLE?

I would expect that if getpass doesn't work in IDLE, that's a bug (in 
IDLE or getpass, I'm not sure :-)


-- 
Steve

From srkunze at mail.de  Thu Oct  1 19:37:04 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 01 Oct 2015 19:37:04 +0200
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <BD1768A9-478A-401A-9E2A-3CBC7678EA57@gmail.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <muetrf$lh$1@ger.gmane.org> <560B1E49.7050102@canterbury.ac.nz>
 <CAA0BC24-7591-4CCF-ADA4-E6D547E1715F@yahoo.com>
 <D91E3BCA-A368-4F9E-8C5A-11ABE744F391@gmail.com>
 <560BC0C3.4040502@btinternet.com> <560C68F0.4080600@mail.de>
 <560C95AF.1020501@btinternet.com>
 <BD1768A9-478A-401A-9E2A-3CBC7678EA57@gmail.com>
Message-ID: <560D6F40.8070802@mail.de>

On 01.10.2015 04:30, Ryan Gonzalez wrote:
> On September 30, 2015 9:08:47 PM CDT, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
>> On 30/09/2015 23:57, Sven R. Kunze wrote:
>>>
>>> The only reason why I would prefer "else" over the proposed
>>> alternatives: it's already a reserved keyword and it's not really
>>> necessary to waste another one. Otherwise, I don't care too much.
>>>
>>> Best,
>>> Sven
>> Are you suggesting that "a else b" is a possibility?
>> That's a no-no because
>>      a if b else c else d
>> is ambiguous; it could mean either of
>>      a if b else (c else d)
>>      (a if b else c) else d
>> Rob
>>
> I already said that...

And others already said "use parentheses" if in doubt.

From srkunze at mail.de  Thu Oct  1 20:15:12 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 01 Oct 2015 20:15:12 +0200
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <CAGu0AntFVOoHJQeYbapNFm6BXhWqZyssZHvMRgfsrn7tf7SAkA@mail.gmail.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <CAGu0AntFVOoHJQeYbapNFm6BXhWqZyssZHvMRgfsrn7tf7SAkA@mail.gmail.com>
Message-ID: <560D7830.7030808@mail.de>

On 01.10.2015 04:53, Bruce Leban wrote:
> If you want to avoid using None (or null) in your code, that's fine 
> but the reality is that None is used extensively *and* there are 
> *many* cases where we do exactly what these operators simplify.

Nothing what you said is wrong. However, it lacks the deeper 
understanding of the None issue.


Just because everybody tells you the world is flat doesn't make it flat. 
Tell others it's round instead.


Best,
Sven

From random832 at fastmail.com  Thu Oct  1 20:16:22 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 14:16:22 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
Message-ID: <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>

(Towards the bottom of this post I ask the question of *why* it's a bad
thing for uptalk to "escape parentheses" - mentioning it up here so as
to not bury the lead.)

On Thu, Oct 1, 2015, at 01:59, Andrew Barnert wrote:
> I posted ?. examples earlier; I don't want to repeat the whole thing
> (especially after Guido pointed out that it probably wasn't helping
> anyone who didn't already get it). But briefly, the AST doesn't have to
> represent the short-circuiting here, any more than it does anywhere else
> that short-circuits; it just adds an uptalk flag to each Attribute node.
> At code generation time, any Attribute node that has uptalk=True has a
> JUMP_IF_NONE to after the primary-or-call (leaving the None attrib value
> on the stack) after the LOAD_ATTR. (Or, if you don't want to add a new
> bytecode, it has a string of three ops that do the equivalent.)

Four ops, actually - DUP_TOP LOAD_CONST(None) COMPARE_OP(is)
POP_JUMP_IF_TRUE - unless I'm missing a more efficient way to do it. And
it has to jump an arbitrary distance forward (however many calls,
attributes, or subscripts are in the expression), not just "to after the
one after".

And the problem is, the AST can't differentiate (a.b).c from a.b.c,
whereas (a?.b).c is *semantically different* from a?.b.c - I think a new
AST structure is therefore necessary. At the very least you'd need
something in the node corresponding to the uptalk to say _where_ to jump
(i.e. how many levels to escape from).

If designing an AST structure from scratch for such a language I think
a?.b.c would absolutely not be represented as any kind of Y(X('a', 'b'),
'c') because it doesn't *make sense*. I was just wondering how you
tackled this problem.

> The same works for ?[]. For ?(), I'm not sure what the desired semantics
> are

The same as for the others, AIUI.

>, and for ?? it seemed obviously trivial so I didn't bother.
> 

My understanding of how the bytecode would work:

a?.b.c.d?.e.f.g =

LOAD_[whatever] a
JUMP_IF_NONE *
LOAD_ATTR b
LOAD_ATTR c
LOAD_ATTR d
JUMP_IF_NONE *
LOAD_ATTR e
LOAD_ATTR f
LOAD_ATTR g
* this position is the target of both jumps

(a?.b.c).d?.e.f.g

LOAD_[whatever] a
JUMP_IF_NONE *
LOAD_ATTR b
LOAD_ATTR c
* target of first jump
LOAD_ATTR d
JUMP_IF_NONE **
LOAD_ATTR e
LOAD_ATTR f
LOAD_ATTR g
** target of second jump

P.S.

Now that I spell it out like that, it occurs to me that this second case
is *not actually that useful*. You're guaranteeing an exception if a is
None, which defeats the purpose of using uptalk on a?... at all. (It
does change whether or not the side effects of a subscript/call
arguments get evaluated, but it's not clear that this justifies the
added complexity.)

Maybe the uptalk should instead be apply to _any_ chain (by left
operands) of AST Attribute/Subscript/Call nodes regardless of whether or
not there are parentheses around them. It's been stated repeatedly that
uptalk shouldn't "escape parentheses", but no-one's clearly stated *why*
that should be the case.

From chris.barker at noaa.gov  Thu Oct  1 20:29:51 2015
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 1 Oct 2015 11:29:51 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151001151028.GD23642@ando.pearwood.info>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com> <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com> <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
Message-ID: <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>

On Thu, Oct 1, 2015 at 8:10 AM, Steven D'Aprano <steve at pearwood.info> wrote:

>  But there
> is a *general* distinction between two cases:
>
> - iterables which are iterators
> - iterables which are not iterators
>
> We have a name for the first set: "iterators". But we don't have a name
> for the second set. Andrew suggested "non-iterator iterables" is too
> clumsy for general use, and suggests we need a better name. You
> suggested "iterables", but that clearly cannot work, since iterators are
> a kind of iterable.
>

sure -- but I've lost track of why it matters. "iterator" is well defined.
And so is "iterable" -- why do we need to care whether the iterable returns
itself when asked for an iterator?

the term "sequence" is useful -- it defines certain behavior. So is the
term "iterable", for the same reason.

And it would be useful to say that given object is both a sequence and an
iterable (are sequences iterable by definition?)

But if why do you need to know that something is an iterable, but NOT an
iterator? isn't that an implementation detail?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/02b84b69/attachment.html>

From random832 at fastmail.com  Thu Oct  1 20:33:26 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 14:33:26 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151001151028.GD23642@ando.pearwood.info>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
Message-ID: <1443724406.464714.398903593.33D0C348@webmail.messagingengine.com>

On Thu, Oct 1, 2015, at 11:10, Steven D'Aprano wrote:
> On Thu, Oct 01, 2015 at 08:15:25AM +0300, Akira Li wrote:
> 
> > True or false?: do all iterables return the same items twice?
> >   http://www.fallacyfiles.org/loadques.html
> 
> [Aside: I have no idea what point you are making with the above link.]
> 
> Of course they don't necessarily do so, but those that don't are not 
> necessarily well-behaved. 

I think what he is claiming, more or less, is that there is not a
universal notion of "well-behaved" (this is true), or indeed *any*
broadly-applicable notions of "well-behaved" (this is false).

From python at mrabarnett.plus.com  Thu Oct  1 20:36:20 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 1 Oct 2015 19:36:20 +0100
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
Message-ID: <560D7D24.9030700@mrabarnett.plus.com>

On 2015-10-01 19:16, Random832 wrote:
> (Towards the bottom of this post I ask the question of *why* it's a bad
> thing for uptalk to "escape parentheses" - mentioning it up here so as
> to not bury the lead.)
>
> On Thu, Oct 1, 2015, at 01:59, Andrew Barnert wrote:
>> I posted ?. examples earlier; I don't want to repeat the whole thing
>> (especially after Guido pointed out that it probably wasn't helping
>> anyone who didn't already get it). But briefly, the AST doesn't have to
>> represent the short-circuiting here, any more than it does anywhere else
>> that short-circuits; it just adds an uptalk flag to each Attribute node.
>> At code generation time, any Attribute node that has uptalk=True has a
>> JUMP_IF_NONE to after the primary-or-call (leaving the None attrib value
>> on the stack) after the LOAD_ATTR. (Or, if you don't want to add a new
>> bytecode, it has a string of three ops that do the equivalent.)
>
> Four ops, actually - DUP_TOP LOAD_CONST(None) COMPARE_OP(is)
> POP_JUMP_IF_TRUE - unless I'm missing a more efficient way to do it. And
> it has to jump an arbitrary distance forward (however many calls,
> attributes, or subscripts are in the expression), not just "to after the
> one after".
>
> And the problem is, the AST can't differentiate (a.b).c from a.b.c,
> whereas (a?.b).c is *semantically different* from a?.b.c - I think a new
> AST structure is therefore necessary. At the very least you'd need
> something in the node corresponding to the uptalk to say _where_ to jump
> (i.e. how many levels to escape from).
>
> If designing an AST structure from scratch for such a language I think
> a?.b.c would absolutely not be represented as any kind of Y(X('a', 'b'),
> 'c') because it doesn't *make sense*. I was just wondering how you
> tackled this problem.
>
>> The same works for ?[]. For ?(), I'm not sure what the desired semantics
>> are
>
> The same as for the others, AIUI.
>
>>, and for ?? it seemed obviously trivial so I didn't bother.
>>
>
> My understanding of how the bytecode would work:
>
> a?.b.c.d?.e.f.g =
>
> LOAD_[whatever] a
> JUMP_IF_NONE *
> LOAD_ATTR b
> LOAD_ATTR c
> LOAD_ATTR d
> JUMP_IF_NONE *
> LOAD_ATTR e
> LOAD_ATTR f
> LOAD_ATTR g
> * this position is the target of both jumps
>
> (a?.b.c).d?.e.f.g
>
> LOAD_[whatever] a
> JUMP_IF_NONE *
> LOAD_ATTR b
> LOAD_ATTR c
> * target of first jump
> LOAD_ATTR d
> JUMP_IF_NONE **
> LOAD_ATTR e
> LOAD_ATTR f
> LOAD_ATTR g
> ** target of second jump
>
> P.S.
>
> Now that I spell it out like that, it occurs to me that this second case
> is *not actually that useful*. You're guaranteeing an exception if a is
> None, which defeats the purpose of using uptalk on a?... at all.
 >
It's _not_ guaranteed that there'll be an exception if a is None; None
does have _some_ attributes.

 > (It does change whether or not the side effects of a subscript/call
> arguments get evaluated, but it's not clear that this justifies the
> added complexity.)
>
> Maybe the uptalk should instead be apply to _any_ chain (by left
> operands) of AST Attribute/Subscript/Call nodes regardless of whether or
> not there are parentheses around them. It's been stated repeatedly that
> uptalk shouldn't "escape parentheses", but no-one's clearly stated *why*
> that should be the case.
>
'and' and 'or' don't escape parentheses. I'd take it as a general rule
that short-circuiting doesn't escape parentheses.


From srkunze at mail.de  Thu Oct  1 20:38:45 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 01 Oct 2015 20:38:45 +0200
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
Message-ID: <560D7DB5.1060802@mail.de>

On 01.10.2015 20:29, Chris Barker wrote:
> On Thu, Oct 1, 2015 at 8:10 AM, Steven D'Aprano <steve at pearwood.info 
> <mailto:steve at pearwood.info>> wrote:
>
>      But there
>     is a *general* distinction between two cases:
>
>     - iterables which are iterators
>     - iterables which are not iterators
>
>     We have a name for the first set: "iterators". But we don't have a
>     name
>     for the second set. Andrew suggested "non-iterator iterables" is too
>     clumsy for general use, and suggests we need a better name. You
>     suggested "iterables", but that clearly cannot work, since
>     iterators are
>     a kind of iterable.
>
>
> sure -- but I've lost track of why it matters. "iterator" is well 
> defined. And so is "iterable" -- why do we need to care whether the 
> iterable returns itself when asked for an iterator?
>
> the term "sequence" is useful -- it defines certain behavior. So is 
> the term "iterable", for the same reason.
>
> And it would be useful to say that given object is both a sequence and 
> an iterable (are sequences iterable by definition?)
>
> But if why do you need to know that something is an iterable, but NOT 
> an iterator? isn't that an implementation detail?

You say some terms are useful because they define certain behavior.

I don't question this, but what I find questionable is the proliferation 
of all this equally-sounding and -feeling concepts. Your reaction 
supports that observation.


Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/931b2bbd/attachment.html>

From random832 at fastmail.com  Thu Oct  1 20:41:15 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 14:41:15 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
Message-ID: <1443724875.466486.398907305.46E7BFE5@webmail.messagingengine.com>

On Thu, Oct 1, 2015, at 14:29, Chris Barker wrote:
> But if why do you need to know that something is an iterable, but NOT an
> iterator? isn't that an implementation detail?

Because an iterator *cannot possibly* allow you to loop through the
contents twice [either one after the other or in parallel], whereas
*most* non-iterator iterables do allow this. This (among other things
such as representing a well-defined finite bag of values) is the
property we're really chasing, "non-iterator iterable" is just a clumsy
and inaccurate way of saying it.

(I'm actually moderately disappointed, incidentally, that there's no
easy way to create e.g. an iterable that will spin up a fresh copy of
the same generator each time it's called. But it's easy enough to make a
decorator for that.)

From random832 at fastmail.com  Thu Oct  1 20:50:07 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 14:50:07 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560D7D24.9030700@mrabarnett.plus.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
Message-ID: <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>

On Thu, Oct 1, 2015, at 14:36, MRAB wrote:
> It's _not_ guaranteed that there'll be an exception if a is None; None
> does have _some_ attributes.

Well, it hasn't got a "d" attribute, but point taken. However, wanting
to build expressions like this that will sometimes operate on the
attributes of None and other times operate on the attributes of the
non-None object normally expected to be present seems like an
*extremely* obscure thing to want.

> 'and' and 'or' don't escape parentheses. I'd take it as a general rule
> that short-circuiting doesn't escape parentheses.

Sure they do. In what way is (a and b) and c different from a and b and
c? Heck, (a and b) and (c and d) even compiles to the same bytecode as
other groupings [demonstrating it has the same actual semantics] *even
though it is a different AST*. But even if it didn't, it would just be
jumping to another jump opcode.

It doesn't escape being mixed with a different kind of expression, which
is sometimes accomplished with parentheses, but that's not really the
same thing.

From steve at pearwood.info  Thu Oct  1 21:37:09 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 05:37:09 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
References: <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
Message-ID: <20151001193709.GF23642@ando.pearwood.info>

On Thu, Oct 01, 2015 at 11:29:51AM -0700, Chris Barker wrote:
> On Thu, Oct 1, 2015 at 8:10 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> >  But there
> > is a *general* distinction between two cases:
> >
> > - iterables which are iterators
> > - iterables which are not iterators
> >
> > We have a name for the first set: "iterators". But we don't have a name
> > for the second set. Andrew suggested "non-iterator iterables" is too
> > clumsy for general use, and suggests we need a better name. You
> > suggested "iterables", but that clearly cannot work, since iterators are
> > a kind of iterable.
> >
> 
> sure -- but I've lost track of why it matters. "iterator" is well defined.
> And so is "iterable" -- why do we need to care whether the iterable returns
> itself when asked for an iterator?

In and of itself, it probably isn't, except as a short-cut for deciding 
whether something is an iterator. 

[...]
> But if why do you need to know that something is an iterable, but NOT an
> iterator? isn't that an implementation detail?

I forget the original context -- I think it was Andrew who first 
mentioned this. Possibly over confusion about (x)range.

But in general, it's important because:

- iterators are not random access, other iterables typically are;

- iterators are one-shot (cannot be restarted), other iterables are 
typically re-runnable.

This makes a difference. Just a few days ago, somebody mis-reported a 
supposed "bug" in all() and any(). For example:

values = (x%5 == 3 for x in range(8))
print(list(values))
print(all(values))  # should return False

Obvious error is obvious: having printed out the values from the 
generator expression, values is now exhausted, and all() of the empty 
set is True (vacuous truth). The difference between general iterables 
which may or may not be one-shot iterators, and those which are 
definitely not iterators, is not always just an implementation detail.


-- 
Steve

From steve at pearwood.info  Thu Oct  1 21:48:43 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 05:48:43 +1000
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
References: <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
 <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
Message-ID: <20151001194843.GG23642@ando.pearwood.info>

On Thu, Oct 01, 2015 at 02:50:07PM -0400, Random832 wrote:
> On Thu, Oct 1, 2015, at 14:36, MRAB wrote:
[...]
> > 'and' and 'or' don't escape parentheses. I'd take it as a general rule
> > that short-circuiting doesn't escape parentheses.
> 
> Sure they do. In what way is (a and b) and c different from a and b and
> c? Heck, (a and b) and (c and d) even compiles to the same bytecode as
> other groupings [demonstrating it has the same actual semantics] *even
> though it is a different AST*. But even if it didn't, it would just be
> jumping to another jump opcode.

I don't see what this has to do with null aware attribute access.

If I wrote:

(spam?.eggs.cheese).__class__

the obvious intention is that it should evaluate like:

temp = spam?.eggs.cheese
temp.__class__


only without the use of a temporary name.



-- 
Steve

From python at mrabarnett.plus.com  Thu Oct  1 22:07:26 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 1 Oct 2015 21:07:26 +0100
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
 <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
Message-ID: <560D927E.5030800@mrabarnett.plus.com>

On 2015-10-01 19:50, Random832 wrote:
> On Thu, Oct 1, 2015, at 14:36, MRAB wrote:
>> It's _not_ guaranteed that there'll be an exception if a is None; None
>> does have _some_ attributes.
>
> Well, it hasn't got a "d" attribute, but point taken. However, wanting
> to build expressions like this that will sometimes operate on the
> attributes of None and other times operate on the attributes of the
> non-None object normally expected to be present seems like an
> *extremely* obscure thing to want.
>
>> 'and' and 'or' don't escape parentheses. I'd take it as a general rule
>> that short-circuiting doesn't escape parentheses.
>
> Sure they do. In what way is (a and b) and c different from a and b and
> c? Heck, (a and b) and (c and d) even compiles to the same bytecode as
> other groupings [demonstrating it has the same actual semantics] *even
> though it is a different AST*. But even if it didn't, it would just be
> jumping to another jump opcode.
>
If a is falsy, it short-circuits "a and b".

The parenthesised expression returns a falsy result.

That falsy result then short-circuits "(a and b) and c".

It happens to show the same behaviour as "a and b and c" and can be
optimised to that.

Well, that's my opinion, anyway!

> It doesn't escape being mixed with a different kind of expression, which
> is sometimes accomplished with parentheses, but that's not really the
> same thing.
>


From p.f.moore at gmail.com  Thu Oct  1 22:11:58 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 1 Oct 2015 21:11:58 +0100
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <1443724875.466486.398907305.46E7BFE5@webmail.messagingengine.com>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <1443724875.466486.398907305.46E7BFE5@webmail.messagingengine.com>
Message-ID: <CACac1F8koJ0EjVnB07Q4Yd6YiNtsYP2qJRPd_O4LtBrzaBjZMA@mail.gmail.com>

On 1 October 2015 at 19:41, Random832 <random832 at fastmail.com> wrote:
> On Thu, Oct 1, 2015, at 14:29, Chris Barker wrote:
>> But if why do you need to know that something is an iterable, but NOT an
>> iterator? isn't that an implementation detail?
>
> Because an iterator *cannot possibly* allow you to loop through the
> contents twice [either one after the other or in parallel], whereas
> *most* non-iterator iterables do allow this. This (among other things
> such as representing a well-defined finite bag of values) is the
> property we're really chasing, "non-iterator iterable" is just a clumsy
> and inaccurate way of saying it.

If I understand what you mean by "non-iterator iterable", then a long
time ago, there was a similar discussion and the term "reiterable" was
used (Google will probably find references). Nothing ever came of the
discussion - if I recall, there was a lot of theoretical debate, but
few practical use cases.

Anyone wanting to avoid a long, inconclusive discussion should
probably chase up that old thread and see if anything new has been
added this time around :-)
Paul

From random832 at fastmail.com  Thu Oct  1 22:36:53 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 16:36:53 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560D927E.5030800@mrabarnett.plus.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
 <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
 <560D927E.5030800@mrabarnett.plus.com>
Message-ID: <1443731813.493510.399004009.2A460957@webmail.messagingengine.com>

On Thu, Oct 1, 2015, at 16:07, MRAB wrote:
> If a is falsy, it short-circuits "a and b".
> 
> The parenthesised expression returns a falsy result.
> 
> That falsy result then short-circuits "(a and b) and c".
> 
> It happens to show the same behaviour as "a and b and c" and can be
> optimised to that.

Er... that's what a and b and c *is*. The 'and' operator is a
left-associative binary operator.

((a and b) and c) is literally the same AST as (a and b and c). Being
optimized to "short-circuit the whole thing" is an optimization for both
of them. The naive way you describe of evaluating each one in turn is
also the same for both of them. They are in fact both being optimized to
(a and (b and c)) [rather, to the same byte code that a 'naive'
implementation would still generate for that expression].

From random832 at fastmail.com  Thu Oct  1 22:37:28 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 16:37:28 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <20151001194843.GG23642@ando.pearwood.info>
References: <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
 <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
 <20151001194843.GG23642@ando.pearwood.info>
Message-ID: <1443731848.493597.398999825.4FD7CC16@webmail.messagingengine.com>

On Thu, Oct 1, 2015, at 15:48, Steven D'Aprano wrote:
> I don't see what this has to do with null aware attribute access.
> 
> If I wrote:
> 
> (spam?.eggs.cheese).__class__
> 
> the obvious intention is that it should evaluate like:
> 
> temp = spam?.eggs.cheese
> temp.__class__

Why would you need to do that, though? What's the practical use case for
this, that you need to do it frequently enough to want a way to do it
without using a temporary?

Or do you think that, as the thing that it short-circuits to the end of,
"same atom_expr unbroken by parentheses" [a concept which, I remind you,
*does not exist* in AST] is somehow simpler or easier to explain than
"same chain of Attribute/Call/Subscript operations"?

From ron3200 at gmail.com  Thu Oct  1 22:57:18 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Thu, 1 Oct 2015 15:57:18 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
Message-ID: <muk6nf$a9n$1@ger.gmane.org>

On 10/01/2015 02:59 AM, Nick Coghlan wrote:

> As a result, one interesting way of looking at this problem is to
> ask: what if we *only* offered the conditional operations, *without*
> offering a binary null-coalescing operator?

How languages treat None, null, and nil, are very fundamental aspects of 
how they work.  For example I noticed that if we add an None specific 
'or' operator, we may want a None specific 'and' operator too.  And in 
order to use those in conditional statements, it makes sense to have 
None specific 'if' and 'while'.  So it may be a slippery slope.  (Or 
they may be good to have.  I'm very undecided still.)


> That is, we could allow conditional assignment:
>
> data ?= [] headers ?= {} arg ?= make_default()
>
> With the meaning:
>
> if data is None: data = [] if headers is None: headers = {} if arg is
> None: arg = make_default()

Is the above three statements, three expressions, or one long expression?

I was contemplating the possibility of having a variation of "is" as a 
way to test for not-null, but it leads down the same path.  Once you 
have one None specific bool operator, the rest are needed to make full 
use of it.



An Observation...

In many discussions here, the same shortcut problem keeps coming up when 
ever we don't want to evaluate a functions arguments too early.

So maybe what we really need is a syntax for a conditional partial where 
some arguments aren't evaluated until asked for.

The ?( is almost that, but some way to short cut the arguments would 
still be needed.  One possibility is to have them all as optional only, 
and just not ask for them.



And to go out on a limb... ;-)

Another possibility is to have a *special magic callable* that when 
called skips the argument evaluation and returns None.

     NoneCall(these, args, are, never, evaluated)

In this case the Magic is underwhelming as it does absolutely nothing 
except return None.  I think I can live with that.

Then a special method on objects, obj __cond__call__() can return a 
NoneCall to skip the arguments on the right side of ?(, or return 
another callable to use them.

This is a bit like decorators in that to use the arguments you need to 
return a callable.  So I think maybe the @ symbol would be better.  I 
can see this being used in decorators to apply decorators conditionally.


The case of a Null default might be done by just having a conditional 
call on None, unpack the arguments. So it's __cond_call__ method if it 
had one, would be an identity function.

    value = obj@(default)   # if obj is None: value = default
                            # if obj has __cond_call__ method
                            #    it calls it with the arguments.
                            # else TypeError


So the '@(', or '?(' if preferred, may be the only one we need.  The 
rest might be implementable with it by defining __cond_call__ methods on 
the objects.


Cheers,
    Ron



















From jbvsmo at gmail.com  Thu Oct  1 23:00:44 2015
From: jbvsmo at gmail.com (=?UTF-8?Q?Jo=C3=A3o_Bernardo?=)
Date: Thu, 1 Oct 2015 18:00:44 -0300
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <560D927E.5030800@mrabarnett.plus.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com> <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
 <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
 <560D927E.5030800@mrabarnett.plus.com>
Message-ID: <CAOyAWgjnL1ZrEVXMhczq6U_K7iTS1Mn6yVGj35yjevcCht7V4g@mail.gmail.com>

I've been thinking of practical uses for this new None aware operator and
although I really like the idea, I believe this may become too weird to use
if some things are not taken care of.
Are all of those things meant to work? (or am I just crazy?) -- assuming
C#-like syntax:


for el in re.match(foo, bar)?.groups() ?? []:
    print(el.upper())

------

# Assuming this function for readability
def is_useful(x):
    return x is not None

if not foo?: # Should read like "not is_useful(foo)"
    print('foo is None')
    # With this syntax it is easier to correct all those pieces of code "if
not x:" where it should be "if x is None:"

if foo??:  # A shorter version possibly meaning "foo ?? True" or "foo is
None"
    print('foo is still None')

-----
# Are operators other than [] and () allowed?

bar = foo? + 1  # For "foo?.__add__(1)" it would work, but for the realistic

                # "type(foo)?.__add__(foo, 1)" it does not translate
correctly
bar = foo ?+ 1  # Another spelling. Is this any better??

bar = foo? / baz? / 2  # How would this work? '(foo / baz) / 2' or 'foo /
(bar / 2)'

-----

Also for correctness, the shouldn't the '?=' operator be '??='
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/e0d22da0/attachment.html>

From random832 at fastmail.com  Thu Oct  1 23:12:26 2015
From: random832 at fastmail.com (Random832)
Date: Thu, 01 Oct 2015 17:12:26 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <muk6nf$a9n$1@ger.gmane.org>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
Message-ID: <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>

On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
> And to go out on a limb... ;-)
> 
> Another possibility is to have a *special magic callable* that when 
> called skips the argument evaluation and returns None.

That's dangerous talk indeed. Special magic callables are Lisp
territory. ;)

And I don't even know how you'd implement it efficiently without them
being known at compile-time.

I guess at *every* callsite you could test if the callable is magic, and
if it is evaluate the arguments, and if it's not just pass in a lambda
that will return the arguments. But you've got to generate those lambdas
for *all* callsites, even the vast majority that will never be a magic
callable.

What if the callable only wants *some* of the arguments? Hey, if this
had existed back then the ternary operator could have been a normal
function - instead of (b() if a else c()) just do iif(a, b(), c()).

And is this going to be fully general? I.e. should this be supported for
regular operators? If __add__ is magic does + do this, for example?

From abarnert at yahoo.com  Thu Oct  1 23:45:22 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 1 Oct 2015 14:45:22 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <CAOyAWgjnL1ZrEVXMhczq6U_K7iTS1Mn6yVGj35yjevcCht7V4g@mail.gmail.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <m2vbar6x1f.fsf@fastmail.com>
 <063100D7-C750-4318-BE76-29C6D14C446A@yahoo.com>
 <1443723382.460965.398871433.5A10F489@webmail.messagingengine.com>
 <560D7D24.9030700@mrabarnett.plus.com>
 <1443725407.467972.398914841.50CDD4B3@webmail.messagingengine.com>
 <560D927E.5030800@mrabarnett.plus.com>
 <CAOyAWgjnL1ZrEVXMhczq6U_K7iTS1Mn6yVGj35yjevcCht7V4g@mail.gmail.com>
Message-ID: <FFF9A6E4-05D7-4018-A0C0-5CE53860AE36@yahoo.com>

On Oct 1, 2015, at 14:00, Jo?o Bernardo <jbvsmo at gmail.com> wrote:
> 
> I've been thinking of practical uses for this new None aware operator and although I really like the idea, I believe this may become too weird to use if some things are not taken care of.
> Are all of those things meant to work? (or am I just crazy?) -- assuming C#-like syntax:
> 
> 
> for el in re.match(foo, bar)?.groups() ?? []:
>     print(el.upper())

Yes.

> 
> ------
> 
> # Assuming this function for readability
> def is_useful(x):
>     return x is not None
> 
> if not foo?: # Should read like "not is_useful(foo)"
>     print('foo is None')
>     # With this syntax it is easier to correct all those pieces of code "if not x:" where it should be "if x is None:"

No. Neither this, nor any of your other examples below, are meant to work.

The proposal adds a binary operator ??, three primary operations ?., ?[], and ?(), and possibly a new assignment ?=.

It does not add a unary postfix operator ?, and you can't infer one from any of those forms any more than you can infer a unary postfix + from +=.

While there was some discussion very early in the thread about whether a unary postfix ? operator could be used instead of all these separate things, but it was quickly realized that it doesn't express short-circuiting right for null-conditional access, doesn't make any sense at all for binary null coalescing, and does the wrong thing for null-conditional assignment, so Guido soundly rejected it and nobody disagreed.

Guido also raised the question of whether we should uptalk most operators, skipping only the ones that make no sense, or only update a few that have compelling use cases. People had a bit of fun exploring the former, but I think everyone agreed that the latter makes a lot more sense, and that's what's in the preliminary PEP.

So, under the proposal under discussions, this example, and your other examples, are all syntax errors, just like "if foo+:" and "if foo++:" are syntax errors.

> if foo??:  # A shorter version possibly meaning "foo ?? True" or "foo is None"
>     print('foo is still None')
> 
> -----
> # Are operators other than [] and () allowed?

> bar = foo? + 1  # For "foo?.__add__(1)" it would work, but for the realistic 
>                 # "type(foo)?.__add__(foo, 1)" it does not translate correctly
> bar = foo ?+ 1  # Another spelling. Is this any better??
> 
> bar = foo? / baz? / 2  # How would this work? '(foo / baz) / 2' or 'foo / (bar / 2)'
> 
> -----
> 
> Also for correctness, the shouldn't the '?=' operator be '??='
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/d779e207/attachment-0001.html>

From abarnert at yahoo.com  Fri Oct  2 00:11:23 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 1 Oct 2015 15:11:23 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
Message-ID: <3865BD76-6CCB-4920-81E8-F0001D3B7A8D@yahoo.com>

On Oct 1, 2015, at 14:12, Random832 <random832 at fastmail.com> wrote:
> 
>> On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
>> And to go out on a limb... ;-)
>> 
>> Another possibility is to have a *special magic callable* that when 
>> called skips the argument evaluation and returns None.
> 
> That's dangerous talk indeed. Special magic callables are Lisp
> territory. ;)

Despite the smiley at the end, I think that second sentence is exactly the issue here. This is basically fexprs, with the same pros and cons.

The language doesn't have to provide new short-circuiting operators like if-else or ?? or other kinds of flow-control expressions because the end user can write them himself easily. On the other hand, because the end user can write flow-control expressions himself, every project ends up being written in a similar but semantically different language.

(Of course hardcore Lisp advocates claim that con is also a pro: it means you can spend the first 80% of a project building up a language that makes the application trivial, and the last 20% coding and debugging the application. Sure, anyone who wasn't involved in that first 80% is never going to be able to understand the code you wrote in the last 20%, but really, who else in the world is smart enough to grok your brilliant code anyway, so where's the problem? Fortunately, no one has outlawed Lisp, so those people can continue to use it and don't have to use horribly restrictive languages like Python or Scala like all of us dumb sheeple, so we can ignore that.)

> And I don't even know how you'd implement it efficiently without them
> being known at compile-time.
> 
> I guess at *every* callsite you could test if the callable is magic, and
> if it is evaluate the arguments, and if it's not just pass in a lambda
> that will return the arguments. But you've got to generate those lambdas
> for *all* callsites, even the vast majority that will never be a magic
> callable.
> 
> What if the callable only wants *some* of the arguments? Hey, if this
> had existed back then the ternary operator could have been a normal
> function - instead of (b() if a else c()) just do iif(a, b(), c()).
> 
> And is this going to be fully general? I.e. should this be supported for
> regular operators? If __add__ is magic does + do this, for example?

From chris.barker at noaa.gov  Fri Oct  2 01:08:54 2015
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 1 Oct 2015 16:08:54 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <1443724875.466486.398907305.46E7BFE5@webmail.messagingengine.com>
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com> <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com> <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <1443724875.466486.398907305.46E7BFE5@webmail.messagingengine.com>
Message-ID: <CALGmxEJZqhyp0CkCKJr89EOAmq7G65Gh+2VRPpRdNGNv-=3EEQ@mail.gmail.com>

On Thu, Oct 1, 2015 at 11:41 AM, Random832 <random832 at fastmail.com> wrote:

> > But if why do you need to know that something is an iterable, but NOT an
> > iterator? isn't that an implementation detail?
>
> Because an iterator *cannot possibly* allow you to loop through the
> contents twice [either one after the other or in parallel], whereas
> *most* non-iterator iterables do allow this. This (among other things
> such as representing a well-defined finite bag of values) is the
> property we're really chasing, "non-iterator iterable" is just a clumsy
> and inaccurate way of saying it.
>

um, then shod;nt you simply describe the iterator as an iterator?

so any "iterable" would be assumed to be a non-iterator iterable.

I guess this all comes about because we don't want to have to write this:

for i in iter(an_iterable):
    .....

i.e have a different interface for interable and an iterable

but I'm still lost on when tha all has to be spelled out...

And back the original question, for enumerate:

OK, you can't really have it e both an iterator AND a sequence, but
couldn't it be an iterator and support indexing?

Though I'm starting to wonder about the use case:

enumerate() is a way to get the items in an iterable and an index at the
same time -- so if you want to pass in a sequence, and index the result,
why not just index into the sequence in the first place??

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/48b40f70/attachment.html>

From chris.barker at noaa.gov  Fri Oct  2 01:13:37 2015
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 1 Oct 2015 16:13:37 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151001193709.GF23642@ando.pearwood.info>
References: <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com> <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
Message-ID: <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>

On Thu, Oct 1, 2015 at 12:37 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> This makes a difference. Just a few days ago, somebody mis-reported a
> supposed "bug" in all() and any(). For example:
>
> values = (x%5 == 3 for x in range(8))
> print(list(values))
> print(all(values))  # should return False
>
> Obvious error is obvious: having printed out the values from the
> generator expression, values is now exhausted, and all() of the empty
> set is True (vacuous truth). The difference between general iterables
> which may or may not be one-shot iterators, and those which are
> definitely not iterators, is not always just an implementation detail.


they used a generator expression, when they clearly wanted a list
comprehension -- so yes, it matters what they were getting, I don't know
that adding more vocabulary would help prevent people from making that
mistake...

if they had been smart enough to call the list() again, before claiming
there was a bug in all -- it may have explained itself.

-Chris







>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151001/8391c129/attachment.html>

From 4kir4.1i at gmail.com  Fri Oct  2 02:26:31 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Fri, 02 Oct 2015 03:26:31 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
 <87a8s32llk.fsf@gmail.com> <m2r3lf6t20.fsf@fastmail.com>
 <87612r2koz.fsf@gmail.com>
 <D05C8C1A-4BE0-49E0-BD38-B0B6CB150BD8@yahoo.com>
Message-ID: <87y4fm15fs.fsf@gmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:

> On Sep 30, 2015, at 22:59, Akira Li <4kir4.1i at gmail.com> wrote:
...

>> You've got the idea: the word *iterable* may be used in the context when
>> not all iterables are accepted.
>> 
>>  https://mail.python.org/pipermail/python-ideas/2015-October/036692.html
>
> This is a link to a reply where you pasted exactly the same text as in
> this reply?and in a third one. What is that supposed to mean?

It means exactly what it says -- literally: "the word *iterable* may be
used in the context when not all iterables are accepted." and
list(iterable), set(iterable), dict(iterable) are the specific examples.

It is a statement of "how it *is*" and that it is acceptable in my view
and there is no need to change it.

Obviously, list/set/dict docs describe what subset of iterables they accept.

If you agree on that then there is no disagreement. And it should answer
the questions from your post.

If you disagree then to ground the discussion what _specific_ places in
the documentation would you like to change?

...
> More concretely: the documentation for dict views goes out of its way
> to point out that these are not iterators, but a different kind of
> iterable that's more like a sequence (presumably meaning at least one
> of the three things above). But it does so inaccurately, by saying
> they are sequences, which is not true. How could it be rewritten to
> get that point across accurately, but still concisely and readably?

"How could it be rewritten": I remember posting the link to Python issue
already http://bugs.python.org/issue25286


From tjreedy at udel.edu  Fri Oct  2 02:46:54 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 1 Oct 2015 20:46:54 -0400
Subject: [Python-ideas] secrets module -- secret.keeper?
In-Reply-To: <20151001171238.GE23642@ando.pearwood.info>
References: <CA+OGgf6bbMcfP=UmkKTN93Bkg+2UyTuLKz==GZeZZ0yxowyV4g@mail.gmail.com>
 <20151001171238.GE23642@ando.pearwood.info>
Message-ID: <mukk66$gbk$1@ger.gmane.org>

On 10/1/2015 1:12 PM, Steven D'Aprano wrote:
> On Wed, Sep 30, 2015 at 01:55:49PM -0400, Jim J. Jewett wrote:
>
>> Will the secrets module offer any
...
>> a getpass that works even in IDLE?

> I would expect that if getpass doesn't work in IDLE, that's a bug (in
> IDLE or getpass, I'm not sure :-)

Since Jim did not explain the implied 'does not work' I will guess that 
'works' means replacing all typed or pasted chars with '*' before they 
are echoed to the screen.  One can do this with tk(inter) and, I 
presume, any or all gui frameworks in use with Python.  Text mode is 
tricker as system-specific commands are needed, and they may not work 
when tk (or another gui framework?) has control of the keyboard and 
screen.

-- 
Terry Jan Reedy


From njs at pobox.com  Fri Oct  2 02:52:34 2015
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 1 Oct 2015 17:52:34 -0700
Subject: [Python-ideas] secrets module -- secret.keeper?
In-Reply-To: <CA+OGgf6bbMcfP=UmkKTN93Bkg+2UyTuLKz==GZeZZ0yxowyV4g@mail.gmail.com>
References: <CA+OGgf6bbMcfP=UmkKTN93Bkg+2UyTuLKz==GZeZZ0yxowyV4g@mail.gmail.com>
Message-ID: <CAPJVwBkx8Oj3qDF+Xt5XP6ady+FNNXenhn85UbDsaRCvtHhWAg@mail.gmail.com>

On Wed, Sep 30, 2015 at 10:55 AM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
> Will the secrets module offer any building blocks to actually protect a secret?
>
> e.g.,
>
> an easy way to encrypt a file with a given password?
> an encrypted datastore?

-100

These are *waayyyyy* too complicated and subtle problems to try to
handle in the stdlib. We cannot possibly do an acceptably good job
here.

> a getpass that works even in IDLE?

This sounds like a bug in getpass and/or IDLE that should just be fixed.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org

From tjreedy at udel.edu  Fri Oct  2 02:55:51 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 1 Oct 2015 20:55:51 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
Message-ID: <mukkmu$oiv$1@ger.gmane.org>

On 9/30/2015 10:31 PM, Andrew Barnert via Python-ideas wrote:
> On Sep 30, 2015, at 19:04, Akira Li <4kir4.1i at gmail.com> wrote:
>>
>> Random832 <random832 at fastmail.com> writes:
>>
>>> Akira Li <4kir4.1i at gmail.com> writes:
>>>
>>>> Andrew Barnert via Python-ideas
>>>> <python-ideas at python.org> writes:
>>>> ...
>>>>> (The fact that we don't have a term for "non-iterator iterable", and
>>>>
>>>> All iterators are iterable but some iterables are not iterators.
>>>>
>>>> If your code accepts only iterators then use the term *iterator*.
>>>> Otherwise the term *iterable* could be used.
>>>>
>>>> It is misleading to use *iterable* if your code only accepts iterators.
>>>>
>>>> If an iterable is an iterator; It is called *iterator*. The term
>>>> *iterable* implies that some instances are not iterators.
>>>
>>> There are three (well, three and a half) kinds of code that consume
>>> iterables, how would you describe each simply?
>>>
>>> 1. Does not call iter, simply calls next. Therefore cannot consume a
>>>    non-iterator iterable.
>> iterator
>>
>>> 2. Calls iter, but can accept an iterator (e.g. only goes through it
>>>    once)
>> iterable
>>
>>> 3. Cannot accept an iterator (goes through it twice, or permanently
>>>    stores a reference to it, etc)
>> neither

re-iterable (with implication of same sequence of yields)
I have used this for years on python list and do not think I am unique.

>> *iterable* is an object that you can pass to iter() to get *iterator*.
>> An iterable does not guarantee that it yields the same items twice.
>
> We don't have any way to simply describe this thing.

Depends of the 'we'.

-- 
Terry Jan Reedy


From 4kir4.1i at gmail.com  Fri Oct  2 03:47:45 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Fri, 02 Oct 2015 04:47:45 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <1443724406.464714.398903593.33D0C348@webmail.messagingengine.com>
Message-ID: <87lhbm11oe.fsf@gmail.com>

Random832 <random832 at fastmail.com> writes:

> On Thu, Oct 1, 2015, at 11:10, Steven D'Aprano wrote:
>> On Thu, Oct 01, 2015 at 08:15:25AM +0300, Akira Li wrote:
>> 
... 
>> Of course they don't necessarily do so, but those that don't are not 
>> necessarily well-behaved. 
>
> I think what he is claiming, more or less, is that there is not a
> universal notion of "well-behaved" (this is true), or indeed *any*
> broadly-applicable notions of "well-behaved" (this is false).

I would say that "well-behaved" "non-iterator iterable" is a strict
subset of "non-iterator iterable".

You probably want "reiterable" word that I see mentioned in the thread.

I don't know whether *reiterable* implies that it produces the same
items the second time but it certainly implies that
next(iter(reiterable)) _may produce something if_ list(reiterable) call
is successful. Perhaps *rerunnable* (that I also see mentioned in the
thread) more strongly implies that the same items should be produced.


From 4kir4.1i at gmail.com  Fri Oct  2 03:59:15 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Fri, 02 Oct 2015 04:59:15 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAP1=2W7SmOrJVey6ywCRJ+ifU1FxXh--q8uT5CoP-XscVeR__g@mail.gmail.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com>
 <87a8s32llk.fsf@gmail.com>
 <EECDF35E-7A42-406E-ABB7-2BB8226B3114@yahoo.com>
Message-ID: <87k2r61158.fsf@gmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:

> On Sep 30, 2015, at 22:39, Akira Li <4kir4.1i at gmail.com> wrote:
... 
>> Use *iterable* instead of "non-iterator iterable" -- it is that simple.
>
> No it isn't. The word "iterable" just means "iterable". When you want
> to talk about sequences?a subtype of iterables?you don't just say
> "iterable", you say "sequence"...

"iterables that aren't iterators" unlike sequences do not introduce new
requirements -- __iter__ returning non-self doesn't count -- the return
value is still an _arbitrary_ iterator e.g., it may return the same
iterator each time. Sequences on the other hand do
introduce new requirements (__len__, __getitem__ and their specific
semantics that distinguishes them from Mappings).

> ... And likewise, when you want to talk about iterables that aren't
> iterators, or iterables that are repeatable, or any other subtype of
> iterables, you have to use a word (or phrase) that actually means what
> you're saying.

I have no objection to the phrase "repeatable iterable" because it does
introduce a useful distinction.

...
>>  if you need an iterator; use the term *iterator* --
>>  otherwise use *iterable* unless you need something more specific e.g.,
>>  *seq* name is common for generic sequences
>
> Why would you use "seq" instead of "sequence" for the name of the abstract sequence type?

I meant in the code e.g., random.choice(seq), to communicate that an
arbitrary iterable is not enough.

> And, more importantly, what name do you use when you need something
> more specific than "iterable", but less specific than "sequence"?as in
> the glossary entry for dict views, for example?

As I said, I would use the term "dict views". If you mean how "dict
views" could be defined then I've already linked more than once to the
corresponding Python docs issue with the patch
http://bugs.python.org/issue25286

>> I don't remember ever using "non-iterator iterable".
>
> Why would you expect to remember using it, when you're replying to a
> message where I invented it for lack of an established better name
> (and in hopes that someone would come up with one)?
>
>> "non-iterator
>> iterable" does not qualify as more specific. You need to introduce new
>> requirements to the type for that.
>
> There are things that are iterables, that are not non-iterator
> iterables, but the reverse is not true. It's a name for a strict
> subset. Which means it's more specific.

Could you provide a non-hypothetical practical example from existing
code of a function that accepts arbitrary iterables but rejects
iterators?

Perhaps discussing a specific example would help to iron out the
terminology. 


From 4kir4.1i at gmail.com  Fri Oct  2 04:13:39 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Fri, 02 Oct 2015 05:13:39 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
Message-ID: <87io6q10h8.fsf@gmail.com>

Steven D'Aprano <steve at pearwood.info> writes:
...
>> You've got the idea: the word *iterable* may be used in the context when
>> not all iterables are accepted.
>
> Sure. But the distinction is that while there are a whole lot of 
> different iterables:
>
> - iterables with a sufficiently small number of items
> - iterables of hashable items
> - iterables of (hashable key, item) pairs
> - iterables of prime numbers less than one million
> - iterables of strings containing exactly 1 vowel
>
> etc they are special cases and don't need specialised names. But there 
> is a *general* distinction between two cases:
>
> - iterables which are iterators
> - iterables which are not iterators
>
> We have a name for the first set: "iterators". But we don't have a name 
> for the second set. Andrew suggested "non-iterator iterables" is too 
> clumsy for general use, and suggests we need a better name. You 
> suggested "iterables", but that clearly cannot work, since iterators are 
> a kind of iterable.

I meant that you won't use the word *iterable* if your function accepts
_only_ iterators and therefore if I see *iterable" I expect that the
function can handle arbitrary iterables, not just iterators.

If your function rejects iterators then in practice it means that you
might want a re-iterable/re-runnable (that I see mentioned in the
thread) iterable (not an arbitrary "non-iterator iterable").

If there is no restriction that the same items should be produced the
second time ("rerunnable iterable"?)  then *collection" (introduced at
the top of the thread) may work. Though *collection* implies that an
iterable can't return the same (non-self) iterator -- otherwise an
iterator is also a collection.

I still don't see a practical need to avoid the word "iterable" unless
new requirements (in addition to being non-iterator) are present.


From steve at pearwood.info  Fri Oct  2 04:57:12 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 12:57:12 +1000
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <muk6nf$a9n$1@ger.gmane.org>
References: <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
Message-ID: <20151002025712.GH23642@ando.pearwood.info>

On Thu, Oct 01, 2015 at 03:57:18PM -0500, Ron Adam wrote:
> On 10/01/2015 02:59 AM, Nick Coghlan wrote:
> 
> >As a result, one interesting way of looking at this problem is to
> >ask: what if we *only* offered the conditional operations, *without*
> >offering a binary null-coalescing operator?
> 
> How languages treat None, null, and nil, are very fundamental aspects of 
> how they work.  For example I noticed that if we add an None specific 
> 'or' operator, we may want a None specific 'and' operator too.

Why would we do that? What would it do and which other languages have 
it?


> And in order to use those in conditional statements, it makes sense to 
> have None specific 'if' and 'while'.

Which other languages have these?


> So it may be a slippery slope.  (Or 
> they may be good to have.  I'm very undecided still.)

Most slippery-slope arguments aren't actually that slippery. Sure, we 
could invent a new short-cut for "if spam is None" statement, but it 
doesn't buy us much benefit, so why would we bother?

    if spam.eggs[cheese] is None: ...
    ifn spam.eggs[cheese]: ...

You save a few characters typing, big deal. But with an expression, you 
can save complexity, by avoiding temporary variables:

    # before
    expensive = spam.eggs[cheese]  # don't calculate this twice
    value = expensive.attribute if expensive is not None else None

    # after
    value = spam.eggs[cheese]?.attribute



[...] 
> Another possibility is to have a *special magic callable* that when 
> called skips the argument evaluation and returns None.
> 
>     NoneCall(these, args, are, never, evaluated)

That cannot work with standard Python semantics. In Python, the function 
doesn't evaluate the arguments, they are evaluated before the function 
even gets called.

For this to work, the parser would have to recognise this NoneCall 
function *at compile time*, so it can decide whether or not to evaluate 
the arguments. That is, in general, an impossible task:

func = random.choice([NoneCall, lambda *args: None])
func(a+1, b*2, c**3)  # to evaluate or not to evaluate?


Python has only one thing even remotely like this, the special handling 
of "super" inside functions, and that's a special case which I think 
Guido has ruled won't be repeated. But even there, it isn't a change in 
what gets evaluated when.


> Then a special method on objects, obj __cond__call__() can return a 
> NoneCall to skip the arguments on the right side of ?(, or return 
> another callable to use them.

No, it can't work anything like that. It can't be a regular method call, 
because the arguments are already evaluated before the method call is 
made. It has to be a change in what code gets compiled.


-- 
Steve

From steve at pearwood.info  Fri Oct  2 05:49:36 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 13:49:36 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <87io6q10h8.fsf@gmail.com>
References: <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info> <87io6q10h8.fsf@gmail.com>
Message-ID: <20151002034936.GI23642@ando.pearwood.info>

On Fri, Oct 02, 2015 at 05:13:39AM +0300, Akira Li wrote:

> I still don't see a practical need to avoid the word "iterable" unless
> new requirements (in addition to being non-iterator) are present.

I don't think anyone has suggested that we should avoid the word 
iterable. At most, some have suggested that we don't have a good 
word for those iterables which are not iterators.

-- 
Steve

From steve at pearwood.info  Fri Oct  2 05:58:33 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 2 Oct 2015 13:58:33 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
References: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
 <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
Message-ID: <20151002035833.GJ23642@ando.pearwood.info>

On Thu, Oct 01, 2015 at 04:13:37PM -0700, Chris Barker wrote:
> On Thu, Oct 1, 2015 at 12:37 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
> 
> > This makes a difference. Just a few days ago, somebody mis-reported a
> > supposed "bug" in all() and any(). For example:
> >
> > values = (x%5 == 3 for x in range(8))
> > print(list(values))
> > print(all(values))  # should return False
> >
> > Obvious error is obvious: having printed out the values from the
> > generator expression, values is now exhausted, and all() of the empty
> > set is True (vacuous truth). The difference between general iterables
> > which may or may not be one-shot iterators, and those which are
> > definitely not iterators, is not always just an implementation detail.
> 
> 
> they used a generator expression, when they clearly wanted a list
> comprehension -- so yes, it matters what they were getting, I don't know
> that adding more vocabulary would help prevent people from making that
> mistake...
> 
> if they had been smart enough to call the list() again, before claiming
> there was a bug in all -- it may have explained itself.

You're missing the point. Don't focus on the fact that the bug was in 
their understanding of what their code did. Let's just pretend that 
their *intentional* algorithm was:

def alg(iterable):
    print(list(iterable))
    print(all(iterable))

and for the sake of bring this never-ending thread to an end, let's 
agree for the sake of argument that it cannot be re-written in any other 
way.

Since the semantics of the function are intentional and correct, the 
parameter is named misleadingly. *iterable* is not sufficiently precise, 
because the function does not accept any old iterable -- it fails to 
work correctly on *iterators*, are a sub-kind of iterable.

If you want a more practical example, any algorithm which needs to 
iterate over an interable two or more times needs to specify "iterable 
which is not an iterator".


-- 
Steve

From rosuav at gmail.com  Fri Oct  2 06:02:09 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 2 Oct 2015 14:02:09 +1000
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151002035833.GJ23642@ando.pearwood.info>
References: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
 <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
 <20151002035833.GJ23642@ando.pearwood.info>
Message-ID: <CAPTjJmr7jCAuWdp6b+zDDmJ5EWfujGezdg67s=pJ8Fuud3bMSQ@mail.gmail.com>

On Fri, Oct 2, 2015 at 1:58 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> If you want a more practical example, any algorithm which needs to
> iterate over an interable two or more times needs to specify "iterable
> which is not an iterator".

For that particular case, I'd reiterate what others have suggested,
and use the term "reiterable" for something you can iterate over more
than once and get the same results. Sequences are normally reiterable.
Any object whose __iter__ is a generator function with stable results
will be reiterable. An iterator is not; nor is an open file object, or
any other object where iteration consumes an external resource.

This seems reasonable.

ChrisA

From brenbarn at brenbarn.net  Fri Oct  2 06:05:59 2015
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Thu, 01 Oct 2015 21:05:59 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151002035833.GJ23642@ando.pearwood.info>
References: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
 <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
 <20151002035833.GJ23642@ando.pearwood.info>
Message-ID: <560E02A7.6050604@brenbarn.net>

On 2015-10-01 20:58, Steven D'Aprano wrote:
> Since the semantics of the function are intentional and correct, the
> parameter is named misleadingly. *iterable* is not sufficiently precise,
> because the function does not accept any old iterable -- it fails to
> work correctly on *iterators*, are a sub-kind of iterable.
>
> If you want a more practical example, any algorithm which needs to
> iterate over an interable two or more times needs to specify "iterable
> which is not an iterator".

	I would disagree with this, because this terminology is both too 
technical and not technical enough.  Just because something isn't an 
iterator doesn't mean you can iterate it multiple times.  You *could* 
write an iterable which is not an iterator but still can't be iterated 
over multiple times (because, say, it returns a reference to some stored 
iterator that can't be restarted, or because it creates a custom 
iterator that references some persistent state of the iterable).  If 
what you want is an iterable that can be iterated multiple times, then 
just say that.  (Or say "reiterable" or "reentrant iterable" or 
whatever.)  There's no need to bring iterators into it at all.

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown

From brenbarn at brenbarn.net  Fri Oct  2 06:06:51 2015
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Thu, 01 Oct 2015 21:06:51 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151002035833.GJ23642@ando.pearwood.info>
References: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
 <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
 <20151002035833.GJ23642@ando.pearwood.info>
Message-ID: <560E02DB.50305@brenbarn.net>

On 2015-10-01 20:58, Steven D'Aprano wrote:
 > > Since the semantics of the function are intentional and correct, the
 > > parameter is named misleadingly. *iterable* is not sufficiently 
precise,
 > > because the function does not accept any old iterable -- it fails to
 > > work correctly on *iterators*, are a sub-kind of iterable.
 > >
 > > If you want a more practical example, any algorithm which needs to
 > > iterate over an interable two or more times needs to specify "iterable
 > > which is not an iterator".

	I would disagree with this, because this terminology is both too
technical and not technical enough.  Just because something isn't an
iterator doesn't mean you can iterate it multiple times.  You *could*
write an iterable which is not an iterator but still can't be iterated
over multiple times (because, say, it returns a reference to some stored
iterator that can't be restarted, or because it creates a custom
iterator that references some persistent state of the iterable).  If
what you want is an iterable that can be iterated multiple times, then
just say that.  (Or say "reiterable" or "reentrant iterable" or
whatever.)  There's no need to bring iterators into it at all.

-- Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no 
path, and leave a trail." --author unknown


From 4kir4.1i at gmail.com  Fri Oct  2 08:30:23 2015
From: 4kir4.1i at gmail.com (Akira Li)
Date: Fri, 02 Oct 2015 09:30:23 +0300
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
References: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
 <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
 <20151002035833.GJ23642@ando.pearwood.info>
Message-ID: <87bnch235s.fsf@gmail.com>

Steven D'Aprano <steve at pearwood.info> writes:

> On Thu, Oct 01, 2015 at 04:13:37PM -0700, Chris Barker wrote:
>> On Thu, Oct 1, 2015 at 12:37 PM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>> 
>> > This makes a difference. Just a few days ago, somebody mis-reported a
>> > supposed "bug" in all() and any(). For example:
>> >
>> > values = (x%5 == 3 for x in range(8))
>> > print(list(values))
>> > print(all(values))  # should return False
>> >
>> > Obvious error is obvious: having printed out the values from the
>> > generator expression, values is now exhausted, and all() of the empty
>> > set is True (vacuous truth). The difference between general iterables
>> > which may or may not be one-shot iterators, and those which are
>> > definitely not iterators, is not always just an implementation detail.
>> 
>> 
>> they used a generator expression, when they clearly wanted a list
>> comprehension -- so yes, it matters what they were getting, I don't know
>> that adding more vocabulary would help prevent people from making that
>> mistake...
>> 
>> if they had been smart enough to call the list() again, before claiming
>> there was a bug in all -- it may have explained itself.
>
> You're missing the point. Don't focus on the fact that the bug was in 
> their understanding of what their code did. Let's just pretend that 
> their *intentional* algorithm was:
>
> def alg(iterable):
>     print(list(iterable))
>     print(all(iterable))
>
> and for the sake of bring this never-ending thread to an end, let's 
> agree for the sake of argument that it cannot be re-written in any other 
> way.
>
> Since the semantics of the function are intentional and correct, the 
> parameter is named misleadingly. *iterable* is not sufficiently precise, 
> because the function does not accept any old iterable -- it fails to 
> work correctly on *iterators*, are a sub-kind of iterable.

list(iterable) does not work for infinite iterables.
set(iterable) does not work for iterables that yield non-hashable items.
dict(iterable) does not work for iterables that do not yield key,value
pairs.

Each builtin specifies what type of iterable it accepts but the
parameter IS called *iterable*. Are you suggesting to rename the
parameters?

> If you want a more practical example, any algorithm which needs to 
> iterate over an interable two or more times needs to specify "iterable 
> which is not an iterator".

If the intent is to write:

  def alg(iterable):
      seq = list(iterable)
      print(seq)
      print(all(seq))

You could say that the 1st alg() accepts a "repeatable deterministic iterable".


From tjreedy at udel.edu  Fri Oct  2 09:19:24 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 2 Oct 2015 03:19:24 -0400
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <87612r2koz.fsf@gmail.com>
References: <bc0a8c1e-50fa-43fd-baf8-c80464be958f@googlegroups.com>
 <CAA68w_k42bEES3xNfjX3bU6Gq2QXPZG4r0+n657T-mvrB_jy2A@mail.gmail.com>
 <CAP1=2W5rCK-yu6V=JTWF7dRBQ7W9mP3ZsbJ5zLPwb-TKnJo+eg@mail.gmail.com>
 <CAA68w_kymM3yPz6tedKrDfxyKgEig4_gkWNPAxqMSii8dBd8Tg@mail.gmail.com>
 <CAP7h-xZDLtm3319jafO_kpojgV5=TCbpTeiHCfsDur3wyi0iDg@mail.gmail.com>
 <CAA68w_k5+z=_eC+eFH8c+BeBmfd-euRjEWT6KdkgrXN4YvrQhQ@mail.gmail.com>
 <CAA68w_mLWADmnX-H9JMwt7S3d4TCpwz4buFgbpP0MxG6a8z8KA@mail.gmail.com>
 <560C25B4.5050000@egenix.com>
 <147B2DEC-310B-498A-B468-03F0053F55B7@yahoo.com>
 <560C2D52.3080809@egenix.com>
 <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <FA850014-E738-4A5D-A848-1BE2868E041D@yahoo.com> <87a8s32llk.fsf@gmail.com>
 <m2r3lf6t20.fsf@fastmail.com> <87612r2koz.fsf@gmail.com>
Message-ID: <mulb63$fi7$1@ger.gmane.org>

On 10/1/2015 1:59 AM, Akira Li wrote:
> Random832 <random832 at fastmail.com> writes:
>
>> Akira Li <4kir4.1i at gmail.com> writes:
>>> I don't remember ever using "non-iterator iterable". "non-iterator
>>> iterable" does not qualify as more specific. You need to introduce new
>>> requirements to the type for that.
>>
>> The question is, how do you *simply* state the very common requirement
>> for an iterable to not behave in a specific undesirable way that all
>> iterators do, and that it is very uncommon for any iterable other than
>> an iterator to do? Are you opposed to having a word for this concept at
>> all, or do you just not like the terms other people are suggesting?
>
> That term is **iterable**. As I already said:
>
> Specific application may use more specific requirements e.g.:
>
> list(iterable):

finite_iterable

> - does it mean that all iterables must be finite?
> - do we need a special word to describe what list() accepts?
>
> set(iterable):

iterable_of_hashables

> - does it mean that all iterables must yield hashable items?
> - do we need a special word to describe what set() accepts?
>
> dict(iterable):

iterable_of_pairs (whose first member is hashable)

> - does it mean that all iterables must yield pairs?
> - do we need a special word to describe what dict() accepts?

Whatever word is used in the signature, the description should be in the 
doc and docstring, preferable in the first line.

Return a list with members from a finite iterable.
Return a set with members from an iterable of hashable objects.*
Return a string joining items from an iterable of strings.

*Also, equality between object should be transitive


> You've got the idea: the word *iterable* may be used in the context when
> not all iterables are accepted.

Right. Each one should be documented properly.

-- 
Terry Jan Reedy


From ron3200 at gmail.com  Fri Oct  2 12:01:12 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 2 Oct 2015 05:01:12 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
Message-ID: <mulkl9$6do$1@ger.gmane.org>

On 10/01/2015 04:12 PM, Random832 wrote:
> On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
>> >And to go out on a limb...;-)
>> >
>> >Another possibility is to have a*special magic callable*  that when
>> >called skips the argument evaluation and returns None.

> That's dangerous talk indeed. Special magic callables are Lisp
> territory.;)

It's also lambda calculus territory.

> And I don't even know how you'd implement it efficiently without them
> being known at compile-time.

> I guess at*every*  callsite you could test if the callable is magic, and
> if it is evaluate the arguments, and if it's not just pass in a lambda
> that will return the arguments. But you've got to generate those lambdas
> for*all*  callsites, even the vast majority that will never be a magic
> callable.

I realized this evening the parser doesn't need to know at parse time, 
and the object doesn't need to be special.  It's the syntax that gives 
it the specialness, not the object.  So what I was thinking is still 
possible, but it would work more like the other suggestions.

If you look at byte code generated for a function call...

 >>> def bar(x, y):
...     return x + y
...
 >>> def foo(x, y):
...     return bar(x+1, y+1)
...
 >>> dis(foo)
   2           0 LOAD_GLOBAL              0 (bar)
               3 LOAD_FAST                0 (x)
               6 LOAD_CONST               1 (1)
               9 BINARY_ADD
              10 LOAD_FAST                1 (y)
              13 LOAD_CONST               1 (1)
              16 BINARY_ADD
              17 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
              20 RETURN_VALUE

You will notice the function is loaded on the stack *before* the 
argument expressions are evaluated.

It won't require generating lamba expressions, just a conditional jump 
where '?(' is used.  So no, it won't be required at every call site.

Something roughly like ...  (presume the index's are adjusted correctly)

2  0  LOAD_GLOBAL 0 (bar)

    #-- inserted only if '?(' is used --------------
       POP_JUMP_IF_NOT_NOCALL 3
       LOAD_CONST 0 (None)
       JUMP_FORWARD  8 (to 20)
    #-----------------------------------------------

    3  LOAD_FAST 0 (x)
    6  LOAD_CONST 1 (1)
    9  BINARY_ADD
    10 LOAD_FAST 1 (y)
    13 LOAD_CONST 1 (1)
    16 BINARY_ADD
    17 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
    20 RETURN_VALUE

So the parser doesn't need to know if it's the NoneCall object at parse 
time, it just needs to know to check for it, and the ?( syntax can tell 
it to do that only where it's needed.

There really isn't anything special about the NoneCall object, it's just 
a reserved builtin object.  And adding the check for it isn't all the 
special either.  It's the behavior that is special because of the ?( sytnax.

I was thinking it could call a __cond_call__ method on the object.  That 
extra indirect call could allow objects to specify how they respond to 
?( conditional calls.  It's just requires an extra LOAD_ATTR and call it 
with no arguments in the bytecode.

The case of unpacking the args if the object is None would require an 
additional check for None and to load an identity function in it's place 
or could be done by adding a __cond_call__ method (if that is how ?( 
will work) to None that is an identity function.

     value = obj?(default)  #None?(default) --> identity(default)

Think of these as moving conditions that would otherwise be elsewhere to 
a more convenient location.  So I don't expect them to be much slower 
than the equivalent code that does the same thing.

A __cond_call__ method works because it can't conflict with __call__. 
So it avoids any conflicts that might occur by calling __call__ by mistake.


The point is that a conditional call syntax could also fill the need of 
the or None operator.  The specific details may be a bit different than 
what I describe here, but I think it's doable.



> What if the callable only wants*some*  of the arguments? Hey, if this
> had existed back then the ternary operator could have been a normal
> function - instead of (b() if a else c()) just do iif(a, b(), c()).

Only wanting some arguments would be much harder.  It might be done by 
splitting the arguments into nested ?( calls similar to how we apply 
multiple decorators.  I don't expect it to be pretty or practical, so 
lets just not go there for now.

> And is this going to be fully general? I.e. should this be supported for
> regular operators? If __add__ is magic does + do this, for example?

No, it would only work with ?( syntax.


Cheers,
    Ron



From abarnert at yahoo.com  Fri Oct  2 12:21:43 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 2 Oct 2015 03:21:43 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <mulkl9$6do$1@ger.gmane.org>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
Message-ID: <2A88A715-4547-4166-9F52-5B467FF9B174@yahoo.com>

On Oct 2, 2015, at 03:01, Ron Adam <ron3200 at gmail.com> wrote:
> 
>> On 10/01/2015 04:12 PM, Random832 wrote:
>>> On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
>>> >And to go out on a limb...;-)
>>> >
>>> >Another possibility is to have a*special magic callable*  that when
>>> >called skips the argument evaluation and returns None.
> 
>> That's dangerous talk indeed. Special magic callables are Lisp
>> territory.;)
> 
> It's also lambda calculus territory.
> 
>> And I don't even know how you'd implement it efficiently without them
>> being known at compile-time.
> 
>> I guess at*every*  callsite you could test if the callable is magic, and
>> if it is evaluate the arguments, and if it's not just pass in a lambda
>> that will return the arguments. But you've got to generate those lambdas
>> for*all*  callsites, even the vast majority that will never be a magic
>> callable.
> 
> I realized this evening the parser doesn't need to know at parse time, and the object doesn't need to be special.  It's the syntax that gives it the specialness, not the object.  So what I was thinking is still possible, but it would work more like the other suggestions.

But if you've already got special handling for ?(), you don't need this NoCall object at all. If bar is None, ?() just skips the call. Compare the bytecode for your version:
> 
> 2  0  LOAD_GLOBAL 0 (bar)
> 
>   #-- inserted only if '?(' is used --------------
>      POP_JUMP_IF_NOT_NOCALL 3
>      LOAD_CONST 0 (None)
>      JUMP_FORWARD  8 (to 20)
>   #-----------------------------------------------

... to the simple ?() check version:

    2  0  LOAD_GLOBAL 0 (bar)
    #-- inserted only if '?(' is used
    JUMP_IF_NONE 8 (to 20)
    #--------------------------------

That's it. (And if you don't want to add a JUMP_IF_NONE opcode, you just DUP, compare to None, and POP_JUMP_IF_TRUE.) Just like ?. and ?[], the LOAD_* is followed by a JUMP_IF_NONE to the end of the first call (or the end of the entire primary if there is no call, but that case can't come up for ?() because it is a call), and nothing else has to be done.

> The case of unpacking the args if the object is None would require an additional check for None and to load an identity function in it's place or could be done by adding a __cond_call__ method (if that is how ?( will work) to None that is an identity function.

Why do you want to do this? And what does it even mean to "unpack the args" here? Does the identity function return a tuple and a dict for the positional and keyword functions respectively? When would you ever want that?

Also, to call the identity function (or anything else) on the args requires evaluating them, which defeats the entire purpose of short-circuiting evaluation of the args.

And, even if the idea of a universal identity function made sense here, why even call it instead of just not calling anything? What's the benefit from that extra work?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151002/18db6486/attachment.html>

From random832 at fastmail.com  Fri Oct  2 14:54:46 2015
From: random832 at fastmail.com (Random832)
Date: Fri, 02 Oct 2015 08:54:46 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <mulkl9$6do$1@ger.gmane.org>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJLF3FzwpAvVXRYudNmCMyvjh_8nQrJQrwB8zYWQ_Npf3A@mail.gmail.com>
 <56097AFB.1040906@oddbird.net>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net>
 <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net>
 <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
Message-ID: <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>

On Fri, Oct 2, 2015, at 06:01, Ron Adam wrote:
> On 10/01/2015 04:12 PM, Random832 wrote:
> > On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
> >> >And to go out on a limb...;-)
> >> >
> >> >Another possibility is to have a*special magic callable*  that when
> >> >called skips the argument evaluation and returns None.
> 
> > That's dangerous talk indeed. Special magic callables are Lisp
> > territory.;)
> 
> It's also lambda calculus territory.

Does lambda calculus really have a notion of two types of function that
are called with the same syntax but one receives the values as they have
been evaluated and the other receives some abstract representation of
how to get them if it needs them? That doesn't even make any sense. Does
it even have a notion of side effects and therefore when/whether things
are evaluated if they are not needed?

Not everything in Lisp is in lambda calculus.

> I realized this evening the parser doesn't need to know at parse time, 
> and the object doesn't need to be special.  It's the syntax that gives 
> it the specialness, not the object.  So what I was thinking is still 
> possible, but it would work more like the other suggestions.
> 
> If you look at byte code generated for a function call...
> 
> You will notice the function is loaded on the stack *before* the 
> argument expressions are evaluated.
> 
> It won't require generating lamba expressions, just a conditional jump 
> where '?(' is used.  So no, it won't be required at every call site.

What does this have to do with the idea of a magic callable? If this is
implemented with a magic callable then A) *any* callable might be a
magic callable and B) the idea original idea strongly implied a
generalized notion of magic callables, of which NoneCall would be only
one example. How do you pass the arguments to the *other* magic
callables that *do* [maybe] evaluate them?

You've basically just explained how the bytecode works for ?( if it is
*not* implemented with a magic callable. So how does making it NoneCall
instead of just None improve anything?

From srkunze at mail.de  Fri Oct  2 16:07:13 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 02 Oct 2015 16:07:13 +0200
Subject: [Python-ideas] Consider making enumerate a sequence if its
 argument is a sequence
In-Reply-To: <20151002034936.GI23642@ando.pearwood.info>
References: <08E63279-5DFF-4F7A-9B7B-B927D34BC4FA@yahoo.com>
 <560C3C45.1070107@egenix.com>
 <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com> <87vbar300y.fsf@gmail.com>
 <m2mvw38lfr.fsf@fastmail.com> <87r3lf2vl4.fsf@gmail.com>
 <m2bncj8cbd.fsf@fastmail.com> <87bncj2mqa.fsf@gmail.com>
 <20151001151028.GD23642@ando.pearwood.info> <87io6q10h8.fsf@gmail.com>
 <20151002034936.GI23642@ando.pearwood.info>
Message-ID: <560E8F91.4000209@mail.de>

On 02.10.2015 05:49, Steven D'Aprano wrote:
> At most, some have suggested that we don't have a good
> word for those iterables which are not iterators.

I sometimes use re-usable iterables.

From rymg19 at gmail.com  Fri Oct  2 16:13:19 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 02 Oct 2015 09:13:19 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
 <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
Message-ID: <D26076B1-B8A0-49A5-ACE0-BEEB89639666@gmail.com>



On October 2, 2015 7:54:46 AM CDT, Random832 <random832 at fastmail.com> wrote:
>On Fri, Oct 2, 2015, at 06:01, Ron Adam wrote:
>> On 10/01/2015 04:12 PM, Random832 wrote:
>> > On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
>> >> >And to go out on a limb...;-)
>> >> >
>> >> >Another possibility is to have a*special magic callable*  that
>when
>> >> >called skips the argument evaluation and returns None.
>> 
>> > That's dangerous talk indeed. Special magic callables are Lisp
>> > territory.;)
>> 
>> It's also lambda calculus territory.
>
>Does lambda calculus really have a notion of two types of function that
>are called with the same syntax but one receives the values as they
>have
>been evaluated and the other receives some abstract representation of
>how to get them if it needs them? That doesn't even make any sense.
>Does
>it even have a notion of side effects and therefore when/whether things
>are evaluated if they are not needed?
>

Indeed. Lambda calculus is untyped...

>Not everything in Lisp is in lambda calculus.
>
>> I realized this evening the parser doesn't need to know at parse
>time, 
>> and the object doesn't need to be special.  It's the syntax that
>gives 
>> it the specialness, not the object.  So what I was thinking is still 
>> possible, but it would work more like the other suggestions.
>> 
>> If you look at byte code generated for a function call...
>> 
>> You will notice the function is loaded on the stack *before* the 
>> argument expressions are evaluated.
>> 
>> It won't require generating lamba expressions, just a conditional
>jump 
>> where '?(' is used.  So no, it won't be required at every call site.
>
>What does this have to do with the idea of a magic callable? If this is
>implemented with a magic callable then A) *any* callable might be a
>magic callable and B) the idea original idea strongly implied a
>generalized notion of magic callables, of which NoneCall would be only
>one example. How do you pass the arguments to the *other* magic
>callables that *do* [maybe] evaluate them?
>
>You've basically just explained how the bytecode works for ?( if it is
>*not* implemented with a magic callable. So how does making it NoneCall
>instead of just None improve anything?
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From srkunze at mail.de  Fri Oct  2 16:20:02 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 02 Oct 2015 16:20:02 +0200
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <56089692.1080303@trueblade.com>
References: <56089692.1080303@trueblade.com>
Message-ID: <560E9292.6040304@mail.de>

On 28.09.2015 03:23, Eric V. Smith wrote:
> The only real question is: what encoding to use for the second parameter
> to bytes()? Since an object must return unicode from __format__(), I
> need to convert that to bytes in order to join everything together. But how?

Cf. https://www.python.org/dev/peps/pep-0461/#interpolation


It says:

b"%x" % val

is equivalent to:

("%x" % val).encode("ascii")


So, ASCII would make a lot of sense to me as well.


> Here I suggest 'ascii'. Unfortunately, this would give an error if
> __format__ returned anything with a char greater than 127. I think we've
> learned that an API that only raises an exception with certain specific
> inputs is fragile.

Could you be more specific here?


Best,
Sven

From eric at trueblade.com  Fri Oct  2 16:26:49 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 02 Oct 2015 10:26:49 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <560E9292.6040304@mail.de>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
Message-ID: <560E9429.9040205@trueblade.com>

On 10/02/2015 10:20 AM, Sven R. Kunze wrote:
> On 28.09.2015 03:23, Eric V. Smith wrote:
>> Here I suggest 'ascii'. Unfortunately, this would give an error if
>> __format__ returned anything with a char greater than 127. I think we've
>> learned that an API that only raises an exception with certain specific
>> inputs is fragile.
> 
> Could you be more specific here?

bf'{foo}'

Might succeed or fail, depending on what foo returns for __format__. If
foo is 'bar', it succeeds. If it's '\u1234', it fails.

But some of the other arguments are making me think bf'' is a bad idea,
so now I'm leaning towards not implementing it.

Eric.


From jbvsmo at gmail.com  Fri Oct  2 16:48:07 2015
From: jbvsmo at gmail.com (=?UTF-8?Q?Jo=C3=A3o_Bernardo?=)
Date: Fri, 2 Oct 2015 11:48:07 -0300
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <560E9429.9040205@trueblade.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com>
Message-ID: <CAOyAWgis7St7GpO+FPTGWVw=B9MtrjiDOUz9zF=uib6dJBnifg@mail.gmail.com>

>
>
> But some of the other arguments are making me think bf'' is a bad idea,
> so now I'm leaning towards not implementing it.
>
>
What about rf''? (sorry for being off topic here)

Regex could benefit from it:

my_regex = rf"?\w+\s*({'|'.join(expected_words)})$"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151002/faa3edf8/attachment.html>

From rosuav at gmail.com  Fri Oct  2 16:52:23 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 3 Oct 2015 00:52:23 +1000
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAOyAWgis7St7GpO+FPTGWVw=B9MtrjiDOUz9zF=uib6dJBnifg@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com>
 <CAOyAWgis7St7GpO+FPTGWVw=B9MtrjiDOUz9zF=uib6dJBnifg@mail.gmail.com>
Message-ID: <CAPTjJmqhewCzKGXj8sU9dVjdCJoyor5oAa4sDeXzWB7XCoh78g@mail.gmail.com>

On Sat, Oct 3, 2015 at 12:48 AM, Jo?o Bernardo <jbvsmo at gmail.com> wrote:
>> But some of the other arguments are making me think bf'' is a bad idea,
>> so now I'm leaning towards not implementing it.
>>
>
> What about rf''? (sorry for being off topic here)
>
> Regex could benefit from it:
>
> my_regex = rf"?\w+\s*({'|'.join(expected_words)})$"
>

Works fine:

rosuav at sikorsky:~$ python3
Python 3.6.0a0 (default:48943533965e, Sep 28 2015, 11:27:38)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> expected_words = ["foo", "bar"]
>>> my_regex = rf"?\w+\s*({'|'.join(expected_words)})$"
>>> print(my_regex)
?\w+\s*(foo|bar)$
>>>

ChrisA

From eric at trueblade.com  Fri Oct  2 16:52:14 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Fri, 02 Oct 2015 10:52:14 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAOyAWgis7St7GpO+FPTGWVw=B9MtrjiDOUz9zF=uib6dJBnifg@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com>
 <CAOyAWgis7St7GpO+FPTGWVw=B9MtrjiDOUz9zF=uib6dJBnifg@mail.gmail.com>
Message-ID: <560E9A1E.5020409@trueblade.com>

On 10/02/2015 10:48 AM, Jo?o Bernardo wrote:
> 
>     But some of the other arguments are making me think bf'' is a bad idea,
>     so now I'm leaning towards not implementing it.
> 
> 
> What about rf''? (sorry for being off topic here)
> 
> Regex could benefit from it:
> 
> my_regex = rf"?\w+\s*({'|'.join(expected_words)})$"
>  

That's already implemented. Its use in regular expressions is mentioned
in the PEP.


From srkunze at mail.de  Fri Oct  2 17:43:46 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 02 Oct 2015 17:43:46 +0200
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <560E9429.9040205@trueblade.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com>
Message-ID: <560EA632.80805@mail.de>

On 02.10.2015 16:26, Eric V. Smith wrote:
> bf'{foo}'
>
> Might succeed or fail, depending on what foo returns for __format__. If
> foo is 'bar', it succeeds. If it's '\u1234', it fails.

I know a lot of functions that fail when passing the wrong kind of 
arguments. What's so wrong with it?

> But some of the other arguments are making me think bf'' is a bad idea,
> so now I'm leaning towards not implementing it.

I see. Do you think of an alternative solution?


I was digging deeper into the matter of binary/byte strings formatting 
in order to sympathise why {} is not usable for binary protocols. Let's 
look at this practical example 
https://blog.tox.chat/2015/09/fuzzing-the-new-groupchats/ I hope the tox 
protocol fully qualifies as a wireframe protocol. What he's trying to do 
it is to fuzzing his new groupchat implementation by creating 
more-or-less random packages and feed them into tox core to see if it 
breaks.

He conveniently uses this type of syntax to describe the structure of 
the first header:

Header 1:  [  Packet ID  (1 b)  |  Chat ID hash  (4 b)  | Sender PK (32 
b)  |  nonce (24 b)  ]


Interested in writing a fuzzer, I would find the following really 
helpful as it mirrors the description within his blog post:

header1 = bf'{packet_id}{chat_id}{sender_pk}{nonce}'

# which should be the same as

header1 = b'%b%b%b%b' % (packet_id, chat_id, sender_pk, nonce)

I wouldn't mind specifying the encoding for all non-byte-string arguments.
Why? Because I would be working with bytes anyway, so no formatting (as 
in format()) would be necessary in the first place. However, I like the 
syntax for specifying the structure of (byte) strings.


Does this makes sense?


Best,
Sven

From guido at python.org  Fri Oct  2 18:00:56 2015
From: guido at python.org (Guido van Rossum)
Date: Fri, 2 Oct 2015 09:00:56 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <560EA632.80805@mail.de>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
Message-ID: <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>

Bingo. IMO the exact same arguments that show why f'{x} {y}' is better than
'%s %s' % (x, y) applies to byte strings. It would be totally acceptable if
it only took bytes (and bytearray, and memoryview) and numbers (which we
can guarantee are rendered in ASCII only).

On Fri, Oct 2, 2015 at 8:43 AM, Sven R. Kunze <srkunze at mail.de> wrote:

> On 02.10.2015 16:26, Eric V. Smith wrote:
>
>> bf'{foo}'
>>
>> Might succeed or fail, depending on what foo returns for __format__. If
>> foo is 'bar', it succeeds. If it's '\u1234', it fails.
>>
>
> I know a lot of functions that fail when passing the wrong kind of
> arguments. What's so wrong with it?
>
> But some of the other arguments are making me think bf'' is a bad idea,
>> so now I'm leaning towards not implementing it.
>>
>
> I see. Do you think of an alternative solution?
>
>
> I was digging deeper into the matter of binary/byte strings formatting in
> order to sympathise why {} is not usable for binary protocols. Let's look
> at this practical example
> https://blog.tox.chat/2015/09/fuzzing-the-new-groupchats/ I hope the tox
> protocol fully qualifies as a wireframe protocol. What he's trying to do it
> is to fuzzing his new groupchat implementation by creating more-or-less
> random packages and feed them into tox core to see if it breaks.
>
> He conveniently uses this type of syntax to describe the structure of the
> first header:
>
> Header 1:  [  Packet ID  (1 b)  |  Chat ID hash  (4 b)  | Sender PK (32
> b)  |  nonce (24 b)  ]
>
>
> Interested in writing a fuzzer, I would find the following really helpful
> as it mirrors the description within his blog post:
>
> header1 = bf'{packet_id}{chat_id}{sender_pk}{nonce}'
>
> # which should be the same as
>
> header1 = b'%b%b%b%b' % (packet_id, chat_id, sender_pk, nonce)
>
> I wouldn't mind specifying the encoding for all non-byte-string arguments.
> Why? Because I would be working with bytes anyway, so no formatting (as in
> format()) would be necessary in the first place. However, I like the syntax
> for specifying the structure of (byte) strings.
>
>
> Does this makes sense?
>
>
> Best,
> Sven
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151002/33147b2e/attachment.html>

From ron3200 at gmail.com  Fri Oct  2 19:47:53 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Fri, 2 Oct 2015 12:47:53 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
 <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
Message-ID: <mumg0a$h9f$1@ger.gmane.org>

On 10/02/2015 07:54 AM, Random832 wrote:
> On Fri, Oct 2, 2015, at 06:01, Ron Adam wrote:
>> On 10/01/2015 04:12 PM, Random832 wrote:
>>> On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
>>>>> And to go out on a limb...;-)
>>>>>
>>>>> Another possibility is to have a*special magic callable*  that when
>>>>> called skips the argument evaluation and returns None.
>>
>>> That's dangerous talk indeed. Special magic callables are Lisp
>>> territory.;)
>>
>> It's also lambda calculus territory.
>
> Does lambda calculus really have a notion of two types of function that
> are called with the same syntax but one receives the values as they have
> been evaluated and the other receives some abstract representation of
> how to get them if it needs them? That doesn't even make any sense. Does
> it even have a notion of side effects and therefore when/whether things
> are evaluated if they are not needed?
>
> Not everything in Lisp is in lambda calculus.

I wasn't really comparing it to lisp, but I understood your reference to 
it.  The comparison to lambda calculus is very loose also.

BTW, in lisp (and scheme) there are a handful of special functions.  One 
of those is the if function which avoids early evaluation of the 
expressions till after the condition is evaluated.  It can be 
implemented as a normal lisp expression but not without quoting the 
arguments.

>> I realized this evening the parser doesn't need to know at parse time,
>> and the object doesn't need to be special.  It's the syntax that gives
>> it the specialness, not the object.  So what I was thinking is still
>> possible, but it would work more like the other suggestions.
>>
>> If you look at byte code generated for a function call...
>>
>> You will notice the function is loaded on the stack *before* the
>> argument expressions are evaluated.
>>
>> It won't require generating lamba expressions, just a conditional jump
>> where '?(' is used.  So no, it won't be required at every call site.
>
> What does this have to do with the idea of a magic callable?

As I said above it doesn't need to magic after all.

> If this is
> implemented with a magic callable then A) *any* callable might be a
> magic callable and B) the idea original idea strongly implied a
> generalized notion of magic callables, of which NoneCall would be only
> one example. How do you pass the arguments to the *other* magic
> callables that *do* [maybe] evaluate them?
>
> You've basically just explained how the bytecode works for ?( if it is
> *not* implemented with a magic callable. So how does making it NoneCall
> instead of just None improve anything?

It's not quite the same.  The proposed ?( skips the arguments if the 
right hand side is None.

          None?(...)   ---> None

So you can't use that to supply a default directly.

         value = None?(default)    # returns None (not useful here)



To make that work, think of applying None to the arguments.  Which in 
turn does nothing to them, they just get returned.

         value = None?(default)   --> default

But to get the inverse of that which is the original preposed ?( 
behavior, you need a different way to trigger that.  We could use False 
I suppose.

        value = False?(default)   --> None

And of course....

        value = other?(default)   -->  other(default)



This is just based on the thought that a conditional ?( calling syntax 
could be useful in a broader scope and not just used as a None 
coalescing operator.

So in the above, None, False, and other can be replaced by expressions.

      def when(x):
          return None if x else False

      value = when(cond)?(expr)       # expr or None



With the original proposed ?( operator the set default example is still 
doable, but it needs a helper function.

        None?(expr) --> None

        other?(expr) --> other(expr)    # The True case


      def is_none(cond):
          return None if cond is None else lambda x:x

      value = is_none(value)?(default_value)


Compared to

      value = default_value if expr is None else None



I was trying to shorten it to..

     value = expr?(default)

But maybe that just won't work in a nice enough way.

Well it's just a thought/suggestion.  (shrug)


Cheers,
    Ron




















From abarnert at yahoo.com  Fri Oct  2 21:56:47 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 2 Oct 2015 12:56:47 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <mumg0a$h9f$1@ger.gmane.org>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <5609985C.40603@oddbird.net>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
 <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
 <mumg0a$h9f$1@ger.gmane.org>
Message-ID: <A0318928-F31C-4257-AEA3-55478E0C10EF@yahoo.com>

On Oct 2, 2015, at 10:47, Ron Adam <ron3200 at gmail.com> wrote:
> 
>> On 10/02/2015 07:54 AM, Random832 wrote:
>>> On Fri, Oct 2, 2015, at 06:01, Ron Adam wrote:
>>>> On 10/01/2015 04:12 PM, Random832 wrote:
>>>> On Thu, Oct 1, 2015, at 16:57, Ron Adam wrote:
>>>>>> And to go out on a limb...;-)
>>>>>> 
>>>>>> Another possibility is to have a*special magic callable*  that when
>>>>>> called skips the argument evaluation and returns None.
>>> 
>>>> That's dangerous talk indeed. Special magic callables are Lisp
>>>> territory.;)
>>> 
>>> It's also lambda calculus territory.
>> 
>> Does lambda calculus really have a notion of two types of function that
>> are called with the same syntax but one receives the values as they have
>> been evaluated and the other receives some abstract representation of
>> how to get them if it needs them? That doesn't even make any sense. Does
>> it even have a notion of side effects and therefore when/whether things
>> are evaluated if they are not needed?
>> 
>> Not everything in Lisp is in lambda calculus.
> 
> I wasn't really comparing it to lisp, but I understood your reference to it.  The comparison to lambda calculus is very loose also.

The comparison to Lisp is very good. Lisp has the exact concept you're asking for: a way to define a function that receives the expressions used as its arguments rather than their evaluated values.

The comparison to lambda calculus doesn't even make sense. It does even have the concept of the string of lambda calculus symbols (or any other parallel) as a thing that can be passed around.

> BTW, in lisp (and scheme) there are a handful of special functions.  One of those is the if function which avoids early evaluation of the expressions till after the condition is evaluated.  It can be implemented as a normal lisp expression but not without quoting the arguments.

Lisp and scheme also let you define new special functions that automatically get the quoted form of their arguments. (Modern Lisps don't usually have fexprs, but they have macros and special-forms, which are close enough for this discussion, I think.) And that's exactly what you're asking for here. The fact that you can define an if special form that doesn't require the caller to quote the arguments means the language doesn't need flow control expressions; you can define them yourself. There are both pros and cons to that, but the cons make it highly unsuitable for Python.

>>> I realized this evening the parser doesn't need to know at parse time,
>>> and the object doesn't need to be special.  It's the syntax that gives
>>> it the specialness, not the object.  So what I was thinking is still
>>> possible, but it would work more like the other suggestions.
>>> 
>>> If you look at byte code generated for a function call...
>>> 
>>> You will notice the function is loaded on the stack *before* the
>>> argument expressions are evaluated.
>>> 
>>> It won't require generating lamba expressions, just a conditional jump
>>> where '?(' is used.  So no, it won't be required at every call site.
>> 
>> What does this have to do with the idea of a magic callable?
> 
> As I said above it doesn't need to magic after all.
> 
>> If this is
>> implemented with a magic callable then A) *any* callable might be a
>> magic callable and B) the idea original idea strongly implied a
>> generalized notion of magic callables, of which NoneCall would be only
>> one example. How do you pass the arguments to the *other* magic
>> callables that *do* [maybe] evaluate them?
>> 
>> You've basically just explained how the bytecode works for ?( if it is
>> *not* implemented with a magic callable. So how does making it NoneCall
>> instead of just None improve anything?
> 
> It's not quite the same.  The proposed ?( skips the arguments if the right hand side is None.
> 
>         None?(...)   ---> None
> 
> So you can't use that to supply a default directly.
> 
>        value = None?(default)    # returns None (not useful here)

Again, why would you want that to return default? 

Imagine you write spam = eggs?(default), hoping to get default if eggs is None. What are you expecting if spam is not None then?

> To make that work, think of applying None to the arguments.  Which in turn does nothing to them, they just get returned.
> 
>        value = None?(default)   --> default

Functions don't take one argument; they take zero or more positional arguments and zero or more keyword arguments. And there are probably more calls with 0 or 2 arguments than 1 in an average program (at least if you don't count the implicit self in method calls). So, what does it mean to return "the argument"? The most reasonable answer I can imagine is this:

    def ident(*args, **kw):
        return args, kw

But that means that to actually use it, you have to write something like this:

    eggs = spam?(default)
    try:
        args, kw = default
        eggs = args[0]
    except (IndexError, ValueError):
        pass

And even that isn't correct if the normal return value of non-None spam(default) happens to be an iterable of two elements whose first element is a non-empty sequence. And, even if it were correct, it seems a lot less readable than:

    eggs = spam(default) if spam is not None else default

By comparison, returning None no matter what the arguments are makes sense in a way that returning the arguments doesn't: because None is a potentially usable value (especially with further chaining), unlike the arguments pair of collections.

> But to get the inverse of that which is the original preposed ?( behavior, you need a different way to trigger that.  We could use False I suppose.
> 
>       value = False?(default)   --> None
> 
> And of course....
> 
>       value = other?(default)   -->  other(default)
> 
> 
> 
> This is just based on the thought that a conditional ?( calling syntax could be useful in a broader scope and not just used as a None coalescing operator.
> 
> So in the above, None, False, and other can be replaced by expressions.
> 
>     def when(x):
>         return None if x else False
> 
>     value = when(cond)?(expr)       # expr or None
> 
> 
> 
> With the original proposed ?( operator the set default example is still doable, but it needs a helper function.
> 
>       None?(expr) --> None
> 
>       other?(expr) --> other(expr)    # The True case
> 
> 
>     def is_none(cond):
>         return None if cond is None else lambda x:x
> 
>     value = is_none(value)?(default_value)
> 
> 
> Compared to
> 
>     value = default_value if expr is None else None
> 
> 
> 
> I was trying to shorten it to..
> 
>    value = expr?(default)
> 
> But maybe that just won't work in a nice enough way.
> 
> Well it's just a thought/suggestion.  (shrug)
> 
> 
> Cheers,
>   Ron

From abarnert at yahoo.com  Fri Oct  2 22:10:30 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 2 Oct 2015 13:10:30 -0700
Subject: [Python-ideas] Consider making enumerate a sequence if its
	argument is a sequence
In-Reply-To: <87bnch235s.fsf@gmail.com>
References: <9E2C1054-551B-41B1-A0D1-2E54A1FA8BF3@yahoo.com>
 <87vbar300y.fsf@gmail.com> <m2mvw38lfr.fsf@fastmail.com>
 <87r3lf2vl4.fsf@gmail.com> <m2bncj8cbd.fsf@fastmail.com>
 <87bncj2mqa.fsf@gmail.com> <20151001151028.GD23642@ando.pearwood.info>
 <CALGmxELd8KLpyCQzAnxam90vysMRb-bP7g8VdBTMqUvACNHHsQ@mail.gmail.com>
 <20151001193709.GF23642@ando.pearwood.info>
 <CALGmxEKvk59B6cYQ3KgDwYVxcREQMo4LF4EETt+FEGogu2Lz8g@mail.gmail.com>
 <20151002035833.GJ23642@ando.pearwood.info> <87bnch235s.fsf@gmail.com>
Message-ID: <58A4DFE1-0ED5-4ED0-90E2-2F3346F64E40@yahoo.com>

On Oct 1, 2015, at 23:30, Akira Li <4kir4.1i at gmail.com> wrote:
> 
> Each builtin specifies what type of iterable it accepts but the
> parameter IS called *iterable*. Are you suggesting to rename the
> parameters?

I think this is where the confusion arises. No one but you is talking about parameter names. The issue is just about having consistent, well-understood names that can be used in documentation, discussions like this thread, answers on the main Python list or StackOverflow, etc.

The question of exactly which things we need to name (non-iterator iterable, repeatable iterables, repeatable iterables that always iterate the same elements, iterables that don't return self when iterates, iterables that always return a new object each time they're iterafed, ?) may be an open question (because you're right that they're not all the same thing, even if they usually overlap), but some subset of those things is relevant often enough that some of them need names.

For example, you could tell the user who had the any/all "bug" that "The way you've coded that function, it only works for re-iterables/collections/whatever, but you're passing it the result of a generator expression, which is an iterator." That (together with a place the user can look up re-iterable/collection/whatever--whether that place is the glossary or the collective mind of the community or whatever) explains the problem without the need for a long explanation on the fact that not all iterables are reusable in the way he's trying to reuse them, and in particular iterators never are, and so on.

The user doesn't then need to change his parameter name from "tests" to "reiterable" or anything like that--he's already got a perfectly good name.

Of course it's not impossible that some of these concepts might also be useful as ABCs and/or static types, in which case it's possible he may want to add an annotation. But that's a separate issue, one which, again, nobody else has raised. 

From christoph at grothesque.org  Fri Oct  2 23:41:30 2015
From: christoph at grothesque.org (Christoph Groth)
Date: Fri, 02 Oct 2015 23:41:30 +0200
Subject: [Python-ideas] asyncio: factoring-out of the general mechanism
References: <877fn61rmy.fsf@grothesque.org>
 <CAP7+vJLOMF7_3DqvCiFkug9=-AztCPHzFL6oWiwoAVLXrYRLTQ@mail.gmail.com>
Message-ID: <87twq9uewl.fsf@grothesque.org>

Guido van Rossum wrote:

> Well, what else did you have in mind? Remember that until we fix the GIL
> (which won't happen until Python 4) you won't get benefit from async
> programming unless you are overlapping with I/O, and (due to the
> limitations of select/poll/etc.) that is pretty much limited to network and
> IPC (disk I/O in particular cannot be overlapped in this way).

I recently realized that asynchronous programming is also useful for
scientific computation: It allows to quite naturally express a complex
algorithm that at various stages launches background computation.

A good application of this technique is adaptive numerical integration.
See [1] for an example script that uses asyncio.  There, the
asynchronous master Python process performs the integration algorithm
(that is computationally cheap but has quite complex control flow),
while the integrand function is evaluated by worker processes.

Now, this particular example uses concurrent.futures, but one could also
use MPI, scoop, or 0mq.  This is what made me wonder about the
monolithic design of asyncio.  We do not want to add support for all
these libraries to the asyncio package?  Already now importing asyncio
triggers a considerable avalanche of imports that do not all seem
necessary for most problems.

> Note that the async/await keywords themselves have no direct link to an
> event loop -- they are just a different way to spell generators. It's just
> that they are only useful with asyncio or some other event loop library.

That's clear.  I was just wondering whether it would make sense to split
up asyncio into a basic package that only provides the event loop itself
(and does not depend on any particular protocol), and support for
various "applications".

The parts that relate to various "applications" could then be added to
other modules.  For example, the subprocess package could be extended by
some coroutines for launching processes.

But perhaps coroutines (or at least their Python flavor) only make sense
within the context of a specific event loop and the language does not
want to mandate the use of a specific one?  Then indeed it seems to be a
good idea to structure async functionality around specific event loops.

Christoph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 810 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151002/82faae46/attachment.sig>

From christoph at grothesque.org  Fri Oct  2 23:47:22 2015
From: christoph at grothesque.org (Christoph Groth)
Date: Fri, 02 Oct 2015 23:47:22 +0200
Subject: [Python-ideas] asyncio: factoring-out of the general mechanism
References: <877fn61rmy.fsf@grothesque.org>
 <CAP7+vJLOMF7_3DqvCiFkug9=-AztCPHzFL6oWiwoAVLXrYRLTQ@mail.gmail.com>
 <87twq9uewl.fsf@grothesque.org>
Message-ID: <87oaghuemt.fsf@grothesque.org>

Here is the missing link [1]:
https://mail.scipy.org/pipermail/scipy-dev/2015-September/020932.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 810 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151002/e421b698/attachment-0001.sig>

From abarnert at yahoo.com  Sat Oct  3 00:23:48 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 2 Oct 2015 15:23:48 -0700
Subject: [Python-ideas] asyncio: factoring-out of the general mechanism
In-Reply-To: <87twq9uewl.fsf@grothesque.org>
References: <877fn61rmy.fsf@grothesque.org>
 <CAP7+vJLOMF7_3DqvCiFkug9=-AztCPHzFL6oWiwoAVLXrYRLTQ@mail.gmail.com>
 <87twq9uewl.fsf@grothesque.org>
Message-ID: <F9BCAE3F-7E55-4D9A-B345-1CA007BC34B2@yahoo.com>

On Oct 2, 2015, at 14:41, Christoph Groth <christoph at grothesque.org> wrote:
> 
> Guido van Rossum wrote:
> 
>> Well, what else did you have in mind? Remember that until we fix the GIL
>> (which won't happen until Python 4) you won't get benefit from async
>> programming unless you are overlapping with I/O, and (due to the
>> limitations of select/poll/etc.) that is pretty much limited to network and
>> IPC (disk I/O in particular cannot be overlapped in this way).
> 
> I recently realized that asynchronous programming is also useful for
> scientific computation: It allows to quite naturally express a complex
> algorithm that at various stages launches background computation.
> 
> A good application of this technique is adaptive numerical integration.
> See [1] for an example script that uses asyncio.  There, the
> asynchronous master Python process performs the integration algorithm
> (that is computationally cheap but has quite complex control flow),
> while the integrand function is evaluated by worker processes.
> 
> Now, this particular example uses concurrent.futures, but one could also
> use MPI, scoop, or 0mq.  This is what made me wonder about the
> monolithic design of asyncio.  

Why would you want asyncio here? You explicitly need parallelism, and independent threads/processes rather than coroutines (with or without an event loop). And it seems like composable futures are exactly the abstraction you want.

It seems like what you really need here is to extend (or make it easier for end users and third party libs to extend) concurrent.futures to work with 0mq, etc., so you can leave the controller code alone while swapping out the dispatch mechanism.


From steve at pearwood.info  Sat Oct  3 05:27:26 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 3 Oct 2015 13:27:26 +1000
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
Message-ID: <20151003032725.GL23642@ando.pearwood.info>

On Fri, Oct 02, 2015 at 09:00:56AM -0700, Guido van Rossum wrote:
> Bingo. IMO the exact same arguments that show why f'{x} {y}' is better than
> '%s %s' % (x, y) applies to byte strings. It would be totally acceptable if
> it only took bytes (and bytearray, and memoryview) and numbers (which we
> can guarantee are rendered in ASCII only).

As Chris A pointed out earlier, identifiers are not ASCII only. What are 
we to make of something like this?

    bf'{??????? + 1}'

And don't say "re-write your code to only use ASCII variable names" :-)


-- 
Steve

From rosuav at gmail.com  Sat Oct  3 05:33:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 3 Oct 2015 13:33:12 +1000
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <20151003032725.GL23642@ando.pearwood.info>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <20151003032725.GL23642@ando.pearwood.info>
Message-ID: <CAPTjJmqW3pbGOQPSmiN3Zq-QmyV0_vPy6ET_YKJhQ9QYvQx14g@mail.gmail.com>

On Sat, Oct 3, 2015 at 1:27 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Oct 02, 2015 at 09:00:56AM -0700, Guido van Rossum wrote:
>> Bingo. IMO the exact same arguments that show why f'{x} {y}' is better than
>> '%s %s' % (x, y) applies to byte strings. It would be totally acceptable if
>> it only took bytes (and bytearray, and memoryview) and numbers (which we
>> can guarantee are rendered in ASCII only).
>
> As Chris A pointed out earlier, identifiers are not ASCII only. What are
> we to make of something like this?
>
>     bf'{??????? + 1}'
>
> And don't say "re-write your code to only use ASCII variable names" :-)

It should be technically legal, btw; it's just going to look very odd.
The check for ASCII-only has to be done _after_ the fracturing into
strings and expressions. But I don't like how that reads.

ChrisA

From guido at python.org  Sat Oct  3 05:44:23 2015
From: guido at python.org (Guido van Rossum)
Date: Fri, 2 Oct 2015 20:44:23 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAPTjJmqW3pbGOQPSmiN3Zq-QmyV0_vPy6ET_YKJhQ9QYvQx14g@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <20151003032725.GL23642@ando.pearwood.info>
 <CAPTjJmqW3pbGOQPSmiN3Zq-QmyV0_vPy6ET_YKJhQ9QYvQx14g@mail.gmail.com>
Message-ID: <CAP7+vJLO=pE0v=OgPWW=hbSo3g_O10LaoCePy41WaOYso8532A@mail.gmail.com>

On Fri, Oct 2, 2015 at 8:33 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, Oct 3, 2015 at 1:27 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
> > On Fri, Oct 02, 2015 at 09:00:56AM -0700, Guido van Rossum wrote:
> >> Bingo. IMO the exact same arguments that show why f'{x} {y}' is better
> than
> >> '%s %s' % (x, y) applies to byte strings. It would be totally
> acceptable if
> >> it only took bytes (and bytearray, and memoryview) and numbers (which we
> >> can guarantee are rendered in ASCII only).
> >
> > As Chris A pointed out earlier, identifiers are not ASCII only. What are
> > we to make of something like this?
> >
> >     bf'{??????? + 1}'
> >
> > And don't say "re-write your code to only use ASCII variable names" :-)
>
> It should be technically legal, btw; it's just going to look very odd.
> The check for ASCII-only has to be done _after_ the fracturing into
> strings and expressions. But I don't like how that reads.
>

I don't think this concern should be a showstopper. Honestly either
approach sounds fine to me. :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151002/cc087cd7/attachment.html>

From ncoghlan at gmail.com  Sat Oct  3 16:05:19 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Oct 2015 00:05:19 +1000
Subject: [Python-ideas] secrets module -- secret.keeper?
In-Reply-To: <560C263A.1010600@mgmiller.net>
References: <CA+OGgf6bbMcfP=UmkKTN93Bkg+2UyTuLKz==GZeZZ0yxowyV4g@mail.gmail.com>
 <560C263A.1010600@mgmiller.net>
Message-ID: <CADiSq7dr=3QG31dHETvBiE6a_0UkysPC5pLf7DARrUG3Eaqn6g@mail.gmail.com>

On 1 October 2015 at 04:13, Mike Miller <python-ideas at mgmiller.net> wrote:
> Somewhat related, there is a keyring module, the functionality of which I've
> sometimes wished were part of the stdlib:
>
>     https://pypi.python.org/pypi/keyring
>
> It supports the big three OSs.

We've learned from experience that these kinds of module are better
coupled to operating system update cycles than they are to Python
language definition update cycles.

However, a case could potentially be made for a pip-style bundling
arrangement that provides the other benefits of stdlib inclusion
(python-dev's technical endorsement, PSF's legal blessing), without
incurring the costs (i.e. coupling to the language update cycle for
feature additions).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Oct  3 16:37:04 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Oct 2015 00:37:04 +1000
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
Message-ID: <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>

On 3 October 2015 at 02:00, Guido van Rossum <guido at python.org> wrote:
> Bingo. IMO the exact same arguments that show why f'{x} {y}' is better than
> '%s %s' % (x, y) applies to byte strings. It would be totally acceptable if
> it only took bytes (and bytearray, and memoryview) and numbers (which we can
> guarantee are rendered in ASCII only).

Given that restriction, what if we dropped the format() call in the
bytestring case, and instead always used printf-style formatting?

That is:

    bf'{packet_id}{chat_id}{sender_pk}{nonce}'

could be roughly equivalent to (with parens to help make the pieces clearer):

    (b'%b' % packet_id) + (b'%b' % chat_id) + (b'%b' % sender_pk) +
(b'%b' % nonce)

If a ":fmt" section is provided for the substitution field, it would
replace the mod-format sequence for that section:

    bf'{number:0.2d} ===> b'%0.2d' % number

With that approach, over time, printf-style formatting (aka
mod-formatting) may come to just be known as bytes formatting (even
though text strings also support it).

Something else that's neat with this: you could use the struct module
for more complex subsections of a binary protocol, while doing the
higher level composition with bf-strings*:

    bf'{header}{struct.pack('<10sHHb', record)}{footer}'

Cheers,
Nick.

* which I am now tempted to call Big Friendly Strings**, since I read
a lot of Roald Dahl books as a kid :)
** this would further mean that normal f-strings are friendly strings
in addition to being format strings ;)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stephen at xemacs.org  Sat Oct  3 16:53:56 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 3 Oct 2015 23:53:56 +0900
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <20151003032725.GL23642@ando.pearwood.info>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <20151003032725.GL23642@ando.pearwood.info>
Message-ID: <22031.60420.627606.122630@turnbull.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > What are we to make of something like this?
 > 
 >     bf'{??????? + 1}'

Greek authorship.  What would you make of

    bf'{junban + 1}'

and why is that better than

    bf'{?? + 1}'

?  (I would guess that is a sort of Japanese approximation to your
Greek.)

I think it was Alex Martelli who argued very strongly at the time of
PEP 263 that using native identifiers and even comments in your native
language is a very risky practice (at least from management's POV
<wink/>).  I think that's still true, but it's clearly in consenting
adults territory.






From bondarenkoedik at gmail.com  Sat Oct  3 17:05:26 2015
From: bondarenkoedik at gmail.com (Eduard Bondarenko)
Date: Sat, 3 Oct 2015 18:05:26 +0300
Subject: [Python-ideas] Proposal how to remove all occurrences of a value
	from a Python list
Message-ID: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>

Hello everyone,

the main idea of this proposal is to create convenient, maybe more Pythonic
way to remove all occurrences of a value from a Python list.

Suppose we have list 'arr':

arr = [1, 2, 3, 1]


and we want to remove all 1 from this list.

The most Pythonic way to do it is:


arr[:] = (x for x in arr if x != 1)


Looks good, at least for experienced developer, but for Python's newcomers
solution will look like this:

>>> while True:

...     try:

...         a.remove(1)

...     except:

...         break


I was surprised that Python doesn't have easy way to remove all occurrences.
Currently I am reading "Effective Python" book and I have encounter good
idea that it's also important to have readable code for new or
another-language developers. And to my mind current Pythonic 'remove all
occurrences' is not readable code and does not give insight (at least at
first glance) into what happens in the code.

This way looks a little bit nicer. I think so..

a.remove(1, all = True)


So, that's idea in brief.
Bellow you can find description how to install and test patch.
Patch is attached.

Note: this is test patch, so some things may not be done in accordance with
PEP 007.

How to install patch:

hg update 3.5

cd Objects

patch < remove_all.patch

make

How to use patch:


>>> arr = [1, 2, 3, 1]

>>> arr.remove(1)

>>> arr

[2, 3, 1]

>>> arr = [1, 2, 3, 1]

>>> arr.remove(1, all = True)

>>> arr

[2, 3]


>>> arr = [1, 2, 3, 1]

>>> arr.remove(1, True)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: remove() takes one argument


>>> arr.remove(1, al = True)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: 'al' is an invalid keyword argument for this function



Many thanks for your attention!
- Eduard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151003/8cce30b8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: remove_all.patch
Type: application/octet-stream
Size: 2176 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151003/8cce30b8/attachment.obj>

From steve.dower at python.org  Sat Oct  3 18:20:20 2015
From: steve.dower at python.org (Steve Dower)
Date: Sat, 3 Oct 2015 09:20:20 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
Message-ID: <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>

"Something else that's neat with this: you could use the struct module
for more complex subsections of a binary protocol"

Personally, if binary f-strings did struct packing by default, I'd want to use them all the time.

bf'{header}{record:<10sHHb}{footer}'

Practically, if they aren't equivalent to removing the b and encoding the resulting f-string, I expect we'll regularly hit confusion and misunderstanding.

$0.02

Cheers,
Steve

Top-posted from my Windows Phone

-----Original Message-----
From: "Nick Coghlan" <ncoghlan at gmail.com>
Sent: ?10/?3/?2015 7:37
To: "Guido van Rossum" <guido at python.org>
Cc: "Python-Ideas" <python-ideas at python.org>
Subject: Re: [Python-ideas] Binary f-strings

On 3 October 2015 at 02:00, Guido van Rossum <guido at python.org> wrote:
> Bingo. IMO the exact same arguments that show why f'{x} {y}' is better than
> '%s %s' % (x, y) applies to byte strings. It would be totally acceptable if
> it only took bytes (and bytearray, and memoryview) and numbers (which we can
> guarantee are rendered in ASCII only).

Given that restriction, what if we dropped the format() call in the
bytestring case, and instead always used printf-style formatting?

That is:

    bf'{packet_id}{chat_id}{sender_pk}{nonce}'

could be roughly equivalent to (with parens to help make the pieces clearer):

    (b'%b' % packet_id) + (b'%b' % chat_id) + (b'%b' % sender_pk) +
(b'%b' % nonce)

If a ":fmt" section is provided for the substitution field, it would
replace the mod-format sequence for that section:

    bf'{number:0.2d} ===> b'%0.2d' % number

With that approach, over time, printf-style formatting (aka
mod-formatting) may come to just be known as bytes formatting (even
though text strings also support it).

Something else that's neat with this: you could use the struct module
for more complex subsections of a binary protocol, while doing the
higher level composition with bf-strings*:

    bf'{header}{struct.pack('<10sHHb', record)}{footer}'

Cheers,
Nick.

* which I am now tempted to call Big Friendly Strings**, since I read
a lot of Roald Dahl books as a kid :)
** this would further mean that normal f-strings are friendly strings
in addition to being format strings ;)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151003/d4e96c83/attachment-0001.html>

From eric at trueblade.com  Sat Oct  3 21:18:06 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Sat, 03 Oct 2015 15:18:06 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
Message-ID: <561029EE.8050006@trueblade.com>

On 10/03/2015 12:20 PM, Steve Dower wrote:
> "Something else that's neat with this: you could use the struct module
> for more complex subsections of a binary protocol"
> 
> Personally, if binary f-strings did struct packing by default, I'd want
> to use them all the time.

That appeals to me, too. There are a number of practical problems that
would need to be worked out. We can argue those later :)

I guess it comes down to: what would the commonest use case for
fb-strings be?

> Practically, if they aren't equivalent to removing the b and encoding
> the resulting f-string, I expect we'll regularly hit confusion and
> misunderstanding.

This is one of my two big concerns. If we do something other than remove
'b' and encode, then we've got two similar looking things that have
vastly different implementations. But maybe struct.pack or %-formatting
are so compelling that it's worth breaking the equivalence.

My other concern is non-ascii chars inside the braces in an fb-string.

Eric.




From ron3200 at gmail.com  Sat Oct  3 22:27:20 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 3 Oct 2015 15:27:20 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <A0318928-F31C-4257-AEA3-55478E0C10EF@yahoo.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
 <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
 <mumg0a$h9f$1@ger.gmane.org> <A0318928-F31C-4257-AEA3-55478E0C10EF@yahoo.com>
Message-ID: <mupdn9$mq0$1@ger.gmane.org>

On 10/02/2015 02:56 PM, Andrew Barnert via Python-ideas wrote:
>> It's not quite the same.  The proposed ?( skips the arguments if
>> the right hand side is None.
>>>
>>> None?(...)   ---> None
>>>
>>> So you can't use that to supply a default directly.
>>>
>>> value = None?(default)    # returns None (not useful here)
> Again, why would you want that to return default?

I guess it depends on what you think it should do.  None is a type, 
NoneType to be exact, then if like int, or float, what kind of argument 
should it have?

'' and empty containers come to mind.  Just like float and int can take 
'1', and error if you give it something that can't be converted to a 
float or int.

The None?(...) syntax looks like it should do a call.  Just like it 
would do if you did int?(...).

So both the return None and the return everything are special behaviors. 
  Possibly a None result is a bit more correct.

To tell the truth I think looking at some real examples from the library 
would be good.  How can those be improved both in how they read and how 
they look, and how frequent are they?

This whole line of discussion started out as a very tenuous 
suggestion/possibility.  I even said it was going out on a limb, so I 
really wasn't expecting to defend it as if it was a complete idea.

The concept I am thing of is doable, but not in the way I was thinking 
at first, but I also think it doesn't fit with python very well.

Cheers,
    Ron







From tjreedy at udel.edu  Sat Oct  3 23:54:40 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 3 Oct 2015 17:54:40 -0400
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
Message-ID: <mupira$tua$1@ger.gmane.org>

On 10/3/2015 11:05 AM, Eduard Bondarenko wrote:
> Hello everyone,
>
> the main idea of this proposal is to create convenient, maybe more
> Pythonic way to remove all occurrences of a value from a Python list.

This is a special case of in-place filtering.

> Suppose we have list 'arr':
>
>     arr = [1, 2, 3, 1]
>
>
> and we want to remove all 1 from this list.
>
> The most Pythonic way to do it is:
>
>
>     arr[:] = (x for x in arr if x != 1)

arr[:] = filter(lambda x: x != 1, x)

> Looks good, at least for experienced developer, but for Python's
> newcomers solution will look like this:
>
>      >>> while True:
>     ...     try:
>     ...         a.remove(1)
>     ...     except:
>     ...         break
>

Python programmers really must learn either comprehensions or map and 
filter.  They must also learn that repeatedly scanning a sequence should 
be avoided when possible.

> I was surprised that Python doesn't have easy way to remove all occurrences.

The two ways above are pretty easy.

> Currently I am reading "Effective Python" book and I have encounter good
> idea that it's also important to have readable code for new or
> another-language developers. And to my mind current Pythonic 'remove all
> occurrences' is not readable code and does not give insight (at least at
> first glance) into what happens in the code.
>
> This way looks a little bit nicer. I think so..
>
>     a.remove(1, all = True)

I would not mind this, but I don't know if there are enough use cases to 
justify an addition.

-- 
Terry Jan Reedy


From abarnert at yahoo.com  Sun Oct  4 00:25:47 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 3 Oct 2015 15:25:47 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
Message-ID: <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>

On Oct 3, 2015, at 09:20, Steve Dower <steve.dower at python.org> wrote:
> 
> "Something else that's neat with this: you could use the struct module
> for more complex subsections of a binary protocol"
> 
> Personally, if binary f-strings did struct packing by default, I'd want to use them all the time.
> 
> bf'{header}{record:<10sHHb}{footer}'

I love that at first glance. But if the point of bf-strings (like the point of bytes.__mod__ and the other str-like stuff added back to bytes since 3.0) is for things like ascii-based, partly-human-readable protocols and formats, it's obviously important to do things like hex and octal, space- and zero-padding, etc., and if the format specifier always means struct, there's no way to do that.

> Practically, if they aren't equivalent to removing the b and encoding the resulting f-string, I expect we'll regularly hit confusion and misunderstanding.

But removing the b and encoding the resulting f-string is useless. For example:

    header = b'Spam'
    value = 42
    lines.append(bf'{header}: {value}\r\n')

This gives you b"b'Spam': 42\r\n". Can you imagine ever wanting that?

The only way the feature makes sense is if it does something different.

Nick's suggestion of having it do %-formatting makes sense. Yes, it means that {count:03} is an error and you need '{count:03d}', which is inconsistent with f-strings. But that seems like a much less serious problem than bytes formatting not being able to handle bytes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151003/d78cb056/attachment.html>

From tritium-list at sdamon.com  Sun Oct  4 00:31:26 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Sat, 03 Oct 2015 18:31:26 -0400
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <mupira$tua$1@ger.gmane.org>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org>
Message-ID: <5610573E.3070607@sdamon.com>



On 10/3/2015 17:54, Terry Reedy wrote:
> On 10/3/2015 11:05 AM, Eduard Bondarenko wrote:
>> Hello everyone,
>>
>> the main idea of this proposal is to create convenient, maybe more
>> Pythonic way to remove all occurrences of a value from a Python list.
>
> This is a special case of in-place filtering.
>
>> Suppose we have list 'arr':
>>
>>     arr = [1, 2, 3, 1]
>>
>>
>> and we want to remove all 1 from this list.
>>
>> The most Pythonic way to do it is:
>>
>>
>>     arr[:] = (x for x in arr if x != 1)
>
> arr[:] = filter(lambda x: x != 1, x)
>
>> Looks good, at least for experienced developer, but for Python's
>> newcomers solution will look like this:
>>
>>      >>> while True:
>>     ...     try:
>>     ...         a.remove(1)
>>     ...     except:
>>     ...         break
>>
>
> Python programmers really must learn either comprehensions or map and 
> filter.  They must also learn that repeatedly scanning a sequence 
> should be avoided when possible.
>
>> I was surprised that Python doesn't have easy way to remove all 
>> occurrences.
>
> The two ways above are pretty easy.
>
>> Currently I am reading "Effective Python" book and I have encounter good
>> idea that it's also important to have readable code for new or
>> another-language developers. And to my mind current Pythonic 'remove all
>> occurrences' is not readable code and does not give insight (at least at
>> first glance) into what happens in the code.
>>
>> This way looks a little bit nicer. I think so..
>>
>>     a.remove(1, all = True)
>
> I would not mind this, but I don't know if there are enough use cases 
> to justify an addition.
>

I am generally opposed to this.  This is sugar for filtering on a 
predicate, with the predicate simply being predefined as `bar == foo`.  
There are a number of other common filtering use cases, should we 
include them all?  Does this make python an easier language to use?

In the new user case, does this teach, or encourage the new user to 
intuit, how any other filtering operations should be written?  Would it 
not benefit the new user more to just have them learn listcomps, map, 
filter, and even while loops?

From abarnert at yahoo.com  Sun Oct  4 00:59:06 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 3 Oct 2015 15:59:06 -0700
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <mupdn9$mq0$1@ger.gmane.org>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <CAP7+vJKKBS4NudBDQ7Z34Dv4x33m09y3T6R3HL5J3O98i+qrMw@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
 <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
 <mumg0a$h9f$1@ger.gmane.org> <A0318928-F31C-4257-AEA3-55478E0C10EF@yahoo.com>
 <mupdn9$mq0$1@ger.gmane.org>
Message-ID: <F73A772A-C167-425C-915D-A37CFF626700@yahoo.com>

On Oct 3, 2015, at 13:27, Ron Adam <ron3200 at gmail.com> wrote:
> 
> On 10/02/2015 02:56 PM, Andrew Barnert via Python-ideas wrote:
>>> It's not quite the same.  The proposed ?( skips the arguments if
>>> the right hand side is None.
>>>> 
>>>> None?(...)   ---> None
>>>> 
>>>> So you can't use that to supply a default directly.
>>>> 
>>>> value = None?(default)    # returns None (not useful here)
>> Again, why would you want that to return default?
> 
> I guess it depends on what you think it should do.  None is a type, NoneType to be exact, then if like int, or float, what kind of argument should it have?

None isn't a type. None _has_ a type, just like 42 does. And, given that, it does exactly what I'd expect:

    >>> 42()
    TypeError: 'int' object is not callable
    >>> None()
    TypeError: 'NoneType' object is not callable
    >>> type(42)()
    0
    >>> type(None)()
    None
    >>> 42('23')
    TypeError: 'int' object is not callable
    >>> None('23')
    TypeError: 'NoneType' object is not callable
    >>> type(42)('23')
    23
    >>> type(None)('23')
    TypeError: NoneType takes no arguments

Calling int('23') makes sense as a conversion constructor; calling NoneType('23') doesn't.

But, even if you insisted that it should mean something, it would make far more sense for it to return None than '23'. After all, it's a constructor, it ought to construct an object of its type. But that still wouldn't mean that None should be callable just because NoneType is, any more than it means that 42 should be callable just because int is.

Meanwhile, you still haven't answered what "return the arguments" should actually mean, given that in Python, "the arguments" are a tuple of positional arguments plus a dict of keyword arguments. What would be the point in returning that? In a language where a 1-tuple is the same thing as its element, like the ML family or Haskell, and where there are no keyword arguments, something like this might be sensible. But in Python, it's not. Unless you have a way to make sense of it that I haven't thought of--which is why I'm asking.

> '' and empty containers come to mind.  Just like float and int can take '1', and error if you give it something that can't be converted to a float or int.
> 
> The None?(...) syntax looks like it should do a call.  Just like it would do if you did int?(...).

Again, int is a type, and None isn't. Callability is a special property that only certain types--functions, methods, types, and things that want to act like one of the above--have. None is not a function, method, or type, any more than 42 is.

Second, the whole reason spam?(?) has been proposed in the first place is that None(?) obviously doesn't make sense, so we want a way to avoid evaluating it if spam is None. In a language where everything is callable, there's no reason to even talk about ?().

> So both the return None and the return everything are special behaviors.  Possibly a None result is a bit more correct.

Making it return None isn't more correct, at least not for Python. It's definitely more correct for a language like SmallTalk with an "active null", which should propagate through almost all expressions by making them evaluate to None rather than raising. In such a language, for None(?) to do anything but return None would be inconsistent. But in a language where None doesn't propagate through any expressions, and instead usually raises a TypeError, for None(?) to return None would be highly inconsistent.

And if you're suggesting that we should replace None with a SmallTalk-style active null, that s a much bigger change, which would have to change a lot more than NoneType.__call__, in a way that would radically affect the semantics of millions of APIs, lines of code, etc.

> To tell the truth I think looking at some real examples from the library would be good.  How can those be improved both in how they read and how they look, and how frequent are they?

People have given real examples (from their own code, not the stdlib) for ?() as defined in the proposal. If you want examples for your different proposal, I think you're the one who needs to give them. (And I agree they would be worthwhile.)

> This whole line of discussion started out as a very tenuous suggestion/possibility.  I even said it was going out on a limb, so I really wasn't expecting to defend it as if it was a complete idea.
> 
> The concept I am thing of is doable, but not in the way I was thinking at first, but I also think it doesn't fit with python very well.

I think either it isn't doable, or it fits into Python even more poorly than you think. But I'm just one guy, who could easily be wrong. If you think otherwise, I would be interested in seeing what you can come up with. Even if it ends up not being a proposal you'd want to back, it would probably give insights to half the people involved in the larger discussion on null coalescing and null conditionals. And if it turns out that I'm right and you can't make it work, convincing yourself of that might lead to useful insights into the larger discussion that you could share here. After all, it seems like we're kind of stuck on conflicting incomplete intuitions here, and just arguing those back and forth isn't getting us very far.

From abarnert at yahoo.com  Sun Oct  4 01:32:04 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sat, 3 Oct 2015 16:32:04 -0700
Subject: [Python-ideas] Proposal how to remove all occurrences of a
	value from a Python list
In-Reply-To: <mupira$tua$1@ger.gmane.org>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org>
Message-ID: <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>

On Oct 3, 2015, at 14:54, Terry Reedy <tjreedy at udel.edu> wrote:
> 
>> On 10/3/2015 11:05 AM, Eduard Bondarenko wrote:
>> Hello everyone,
>> 
>> the main idea of this proposal is to create convenient, maybe more
>> Pythonic way to remove all occurrences of a value from a Python list.
> 
> This is a special case of in-place filtering.
> 
>> Suppose we have list 'arr':
>> 
>>    arr = [1, 2, 3, 1]
>> 
>> 
>> and we want to remove all 1 from this list.
>> 
>> The most Pythonic way to do it is:
>> 
>> 
>>    arr[:] = (x for x in arr if x != 1)
> 
> arr[:] = filter(lambda x: x != 1, x)

I don't think this is more Pythonic. You've wrapped a perfectly good expression in an unnecessary function just so you can unnecessarily use a higher-order function. (Of course, by the same token, if you already have a perfectly good function lying around, using filter makes more sense than wrapping it up in an unnecessary call expression just so you can avoid a higher-order function. And there are cases where is unclear which is more appropriate--e.g., if what you have lying around is type or instance with a bound or unbound method, or something you can call partial on, is that really better than wrapping it in an expression? But this isn't either of those cases.) I'd have no problem with a novice who didn't yet know comprehensions writing this, but I wouldn't want to teach it as a better alternative to comprehensions in cases where it's not actually better.

>> Looks good, at least for experienced developer, but for Python's
>> newcomers solution will look like this:
>> 
>>     >>> while True:
>>    ...     try:
>>    ...         a.remove(1)
>>    ...     except:
>>    ...         break
> 
> Python programmers really must learn either comprehensions or map and filter.  They must also learn that repeatedly scanning a sequence should be avoided when possible.

Agreed.

In my experience, most novices who ask about code like this, once you explain to them that remove() has to keep repeatedly scanning from the start, immediately see why that's bad and ask how you can avoid doing that. That's a perfect opportunity to teach them about comprehensions while they're looking for exactly what comprehensions can do.

There are some exceptions, but those aren't novices, they're intermediate-experienced C devs who insist that the following is the "simplest" code and must be the fastest or Python is broken (even though it's not necessarily fastest even in C, largely because memmove is so much faster than element by element move):

    idx = skip = 0
    length = len(arr)
    while idx+skip < length:
        if arr[idx+skip] == element_to_remove:
            skip += 1
        else:
            arr[idx] = arr[idx+skip]
            idx += 1
    del arr[idx+1:]

>> Currently I am reading "Effective Python" book and I have encounter good
>> idea that it's also important to have readable code for new or
>> another-language developers. And to my mind current Pythonic 'remove all
>> occurrences' is not readable code and does not give insight (at least at
>> first glance) into what happens in the code.

It's readable to anyone who understands comprehensions and slice assignment. And both of those are such fundamental concepts that, if you don't understand either of them yet at all, your intuitions aren't very good yet.

But notice that there's nothing stopping you from wrapping this up in a function, or adding whitespace or comments to help yourself work through it:

    def remove_all(arr, value):
        """Remove all instances of value from arr"""
        arr[:] = (x for x in arr # keep every element
                  if x != value) # that doesn't equal value

And now, everywhere you use it looks like this:

    remove_all(arr, 1)

And it's hard to imagine anything more readable.

And, even if remove_all isn't the kind of function an experienced developer would write, learning how to factor out the tricky bits into documentable and testable functions is one of the most useful skills for any developer in any language.


From rob.cliffe at btinternet.com  Sun Oct  4 01:14:15 2015
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Sun, 4 Oct 2015 00:14:15 +0100
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <mupira$tua$1@ger.gmane.org>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org>
Message-ID: <56106147.8000704@btinternet.com>



On 03/10/2015 22:54, Terry Reedy wrote:
> On 10/3/2015 11:05 AM, Eduard Bondarenko wrote:
>> Hello everyone,
>>
>> the main idea of this proposal is to create convenient, maybe more
>> Pythonic way to remove all occurrences of a value from a Python list.
>
> This is a special case of in-place filtering.
>
>> Suppose we have list 'arr':
>>
>>     arr = [1, 2, 3, 1]
>>
>>
>> and we want to remove all 1 from this list.
>>
>> The most Pythonic way to do it is:
>>
>>
>>     arr[:] = (x for x in arr if x != 1)
>
> arr[:] = filter(lambda x: x != 1, x)
I think you mean
     arr[:] = filter(lambda x: x!=1, arr)
which suggests that using filter/map is not quite as easy as its 
advocates believe. :-)
>
>>
>> This way looks a little bit nicer. I think so..
>>
>>     a.remove(1, all = True)
>
Other things that might be useful (sorry, I have no use cases in mind):
     a.remove(1, count)    # maximum number of removals, analogous to 
aString.replace(old, new, count)
     a remove-like function that does not raise an error if the item is 
not present
     a remove-like function that returns the mutated list (like 
sorted(), as opposed to list.sort() )
It's not obvious to me how to design good API(s) to do some/all of this.
Rob Cliffe

From ron3200 at gmail.com  Sun Oct  4 03:58:19 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 3 Oct 2015 20:58:19 -0500
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <F73A772A-C167-425C-915D-A37CFF626700@yahoo.com>
References: <CAF7AXFE4M=1Gv_RONgH1Y9qoR0nw4JkSgan9GCboMZu-CfcW9g@mail.gmail.com>
 <56099C6F.90700@oddbird.net> <36AB4531-96BD-4D22-A957-B2199BA7912E@stufft.io>
 <CAP7+vJJFciihZwtFoEpvpfFEHGUoo0oL_nLMGJepor972VFRvg@mail.gmail.com>
 <85oagm2saa.fsf@benfinney.id.au>
 <CAP7+vJKiyen75dmUEqRa29EfkZkWmwy-P3qYDcB_UdmuWG30Jw@mail.gmail.com>
 <5609AB62.5040503@oddbird.net> <20150929133542.4d04f6dd@anarchist.wooz.org>
 <CAF7AXFGPB1CXdHLce5T-1OOPieJduVgOKPk4t9HMBvyA1QgAKg@mail.gmail.com>
 <560C87A3.1020507@mrabarnett.plus.com>
 <5CF3DF17-D9FD-46B6-B74B-A161855D8B04@yahoo.com>
 <CADiSq7eN-Nv1tYcpLMGoAQLMgtz7Rv2WUFmdqjGkoThGpENymw@mail.gmail.com>
 <muk6nf$a9n$1@ger.gmane.org>
 <1443733946.502049.399030513.749E8ACE@webmail.messagingengine.com>
 <mulkl9$6do$1@ger.gmane.org>
 <1443790486.723247.399552329.2D51881A@webmail.messagingengine.com>
 <mumg0a$h9f$1@ger.gmane.org> <A0318928-F31C-4257-AEA3-55478E0C10EF@yahoo.com>
 <mupdn9$mq0$1@ger.gmane.org> <F73A772A-C167-425C-915D-A37CFF626700@yahoo.com>
Message-ID: <muq13r$t8n$1@ger.gmane.org>

On 10/03/2015 05:59 PM, Andrew Barnert via Python-ideas wrote:


> Meanwhile, you still haven't answered what "return the arguments"
> should actually mean, given that in Python, "the arguments" are a
> tuple of positional arguments plus a dict of keyword arguments.

The simplest example is if a function has 5 arguments, then it will 
return those in the order they are in the signature.

 >>> def foo(a, b, c, x=1, y=2):
...     return a, b, c, x, y

 >>> (a, b, c, x, y) = foo(1, 2, 3, 4, 5)
 >>> a, b, c, x, y
(1, 2, 3, 4, 5)

 >>> a = foo(1, 2, 3, 4, 5)
 >>> a
(1, 2, 3, 4, 5)

But I think that may not be as useful as I was thinking.  Only a few 
functions return the same number of arguments as they take.

It would actually be more applicable to filters, where a function is 
applied to each item, and None might be the default to do nothing with 
that item.


>>> This whole line of discussion started out as a very tenuous
>>> suggestion/possibility.  I even said it was going out on a limb,
>>> so I really wasn't expecting to defend it as if it was a
>>> complete idea.
>>>
>>> The concept I am thing of is doable, but not in the way I was
>>> thinking at first, but I also think it doesn't fit with python
>>> very well.

> I think either it isn't doable, or it fits into Python even more
> poorly than you think. But I'm just one guy, who could easily be
> wrong. If you think otherwise, I would be interested in seeing what
> you can come up with. Even if it ends up not being a proposal you'd
> want to back, it would probably give insights to half the people
> involved in the larger discussion on null coalescing and null
> conditionals. And if it turns out that I'm right and you can't make
> it work, convincing yourself of that might lead to useful insights
> into the larger discussion that you could share here. After all, it
> seems like we're kind of stuck on conflicting incomplete intuitions
> here, and just arguing those back and forth isn't getting us very
> far.

That's exactly what we are doing. ;-)

I doubt I can do better than what's already suggested at this point.

Cheers,
    Ron
























From steve at pearwood.info  Sun Oct  4 15:22:03 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 5 Oct 2015 00:22:03 +1100
Subject: [Python-ideas] Proposal how to remove all occurrences of a
	value from a Python list
In-Reply-To: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
Message-ID: <20151004132203.GN23642@ando.pearwood.info>

On Sat, Oct 03, 2015 at 06:05:26PM +0300, Eduard Bondarenko wrote:
> Hello everyone,
> 
> the main idea of this proposal is to create convenient, maybe more Pythonic
> way to remove all occurrences of a value from a Python list.

What's so special about removing all occurances from a list? How often 
do you find yourself doing such a thing?

These aren't rhetorical questions. They're questions that need to be 
answered before your proposal can be turned into a new feature. If the 
answer is "not very special, and only rarely" that won't necessarily 
rule out the change, but it will make it harder to convince that it is 
desirable.


> Suppose we have list 'arr':
[...]
> Looks good, at least for experienced developer, but for Python's newcomers
> solution will look like this:
> 
> >>> while True:
> ...     try:
> ...         a.remove(1)
> ...     except:
> ...         break

Part of the process of learning to be a programmer is learning to avoid 
awful code like the above and instead learn general purpose processing 
techniques like the list comp. In my experience, e beginner is more 
likely to come up with this:

while 1 in arr:
    arr.remove(1)


since it is far more straight-forward than your version, and involves 
fewer concepts ("what's try...except do?").


> I was surprised that Python doesn't have easy way to remove all 
> occurrences. 

That depends on what you mean by "easy". Or "obvious". I would consider 
the list comp to be both. But of course, I don't expect beginners to see 
things the same way.


> Currently I am reading "Effective Python" book and I have 
> encounter good idea that it's also important to have readable code for 
> new or another-language developers. And to my mind current Pythonic 
> 'remove all occurrences' is not readable code and does not give 
> insight (at least at first glance) into what happens in the code.

I disagree. The list comp *does* give insight into what happens in the 
code: you iterate over the existing elements of the list, collecting 
the ones which don't equal 1, and then save them back into the list. 
Whereas your suggestion:

> This way looks a little bit nicer. I think so..
> 
> a.remove(1, all = True)

is just a mysterious method call. What insight does it give? How does it 
work? There is no hint, no clue. It might as well be magic.

On the other hand, once the programmer can reason about the task well 
enough to write the list comprehension, they can easily extend it to 
slightly different tasks:

# Skip the first 8 items, then remove numbers less than 5:
arr[9:] = [x for x in arr[9:] if x >= 5]

Your remove(all=True) method cannot be extended and teaches the 
programmer nothing except how to solve this one problem.


-- 
Steve

From steve.dower at python.org  Sun Oct  4 17:34:22 2015
From: steve.dower at python.org (Steve Dower)
Date: Sun, 4 Oct 2015 08:34:22 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
Message-ID: <E1ZilJi-0005Uh-7Q@se2-syd.hostedmail.net.au>

Maybe we could spell it {spam!p:<10sHHb}?

Top-posted from my Windows Phone

-----Original Message-----
From: "Andrew Barnert" <abarnert at yahoo.com>
Sent: ?10/?3/?2015 15:31
To: "Steve Dower" <steve.dower at python.org>
Cc: "Nick Coghlan" <ncoghlan at gmail.com>; "Guido van Rossum" <guido at python.org>; "Python-Ideas" <python-ideas at python.org>
Subject: Re: [Python-ideas] Binary f-strings

On Oct 3, 2015, at 09:20, Steve Dower <steve.dower at python.org> wrote:


"Something else that's neat with this: you could use the struct module
for more complex subsections of a binary protocol"

Personally, if binary f-strings did struct packing by default, I'd want to use them all the time.

bf'{header}{record:<10sHHb}{footer}'



I love that at first glance. But if the point of bf-strings (like the point of bytes.__mod__ and the other str-like stuff added back to bytes since 3.0) is for things like ascii-based, partly-human-readable protocols and formats, it's obviously important to do things like hex and octal, space- and zero-padding, etc., and if the format specifier always means struct, there's no way to do that.


Practically, if they aren't equivalent to removing the b and encoding the resulting f-string, I expect we'll regularly hit confusion and misunderstanding.



But removing the b and encoding the resulting f-string is useless. For example:


    header = b'Spam'
    value = 42
    lines.append(bf'{header}: {value}\r\n')


This gives you b"b'Spam': 42\r\n". Can you imagine ever wanting that?


The only way the feature makes sense is if it does something different.


Nick's suggestion of having it do %-formatting makes sense. Yes, it means that {count:03} is an error and you need '{count:03d}', which is inconsistent with f-strings. But that seems like a much less serious problem than bytes formatting not being able to handle bytes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151004/236fbe78/attachment.html>

From srkunze at mail.de  Mon Oct  5 08:28:49 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Mon, 05 Oct 2015 08:28:49 +0200
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org> <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
Message-ID: <561218A1.6030100@mail.de>

On 04.10.2015 01:32, Andrew Barnert via Python-ideas wrote:
> And now, everywhere you use it looks like this:
>
>      remove_all(arr, 1)
>
> And it's hard to imagine anything more readable.

arr.remove_all(1)

> And, even if remove_all isn't the kind of function an experienced developer would write, learning how to factor out the tricky bits into documentable and testable functions is one of the most useful skills for any developer in any language.#

True.


Btw. the same is true for Python core devs. This said, I would 
appreciate the method 'remove_all' provided by the stdlib. ;-)

Best,
Sven

From srkunze at mail.de  Mon Oct  5 08:59:39 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Mon, 05 Oct 2015 08:59:39 +0200
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <561029EE.8050006@trueblade.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <561029EE.8050006@trueblade.com>
Message-ID: <56121FDB.3050401@mail.de>

On 03.10.2015 21:18, Eric V. Smith wrote:
> On 10/03/2015 12:20 PM, Steve Dower wrote:
>> "Something else that's neat with this: you could use the struct module
>> for more complex subsections of a binary protocol"
>>
>> Personally, if binary f-strings did struct packing by default, I'd want
>> to use them all the time.
> That appeals to me, too. There are a number of practical problems that
> would need to be worked out. We can argue those later :)

I think that's were I reach the limit of my "binary" experience in 
Python. But if people (here Steve) found it useful, why not? If there 
are problems that cannot be solved easily, we can do a V1 and later a V2 
that includes the struct packing.

> I guess it comes down to: what would the commonest use case for
> fb-strings be?

To me, it's the same as for all f-strings: the bloody easy string 
concatenation of easily distinguishable parts.

I even have to admit I thought they were called *format strings* because 
they *give format/structure* to the resulting string. Well, now I know 
better (format refers to the formatting of the input expressions) but 
the analogy is still in my mind.

> My other concern is non-ascii chars inside the braces in an fb-string.

What's wrong with them?


Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151005/38dc545e/attachment-0001.html>

From eric at trueblade.com  Mon Oct  5 10:46:40 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Mon, 5 Oct 2015 04:46:40 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <56121FDB.3050401@mail.de>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <561029EE.8050006@trueblade.com> <56121FDB.3050401@mail.de>
Message-ID: <561238F0.20200@trueblade.com>

>> My other concern is non-ascii chars inside the braces in an fb-string.
> 
> What's wrong with them?

It has to do with the order of processing we defined for regular
f-strings. I'm still working through it to see what the implications are.

Eric.


From random832 at fastmail.com  Mon Oct  5 16:06:07 2015
From: random832 at fastmail.com (Random832)
Date: Mon, 05 Oct 2015 10:06:07 -0400
Subject: [Python-ideas] PEP 505 (None coalescing operators) thoughts
In-Reply-To: <1082717728.2280152.1443469633455.JavaMail.yahoo@mail.yahoo.com>
References: <CAP7+vJJdvUy-qaLFi2m3==XCDaZEGtqOGgCP7RVegMbk-nsH_w@mail.gmail.com>
 <1082717728.2280152.1443469633455.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <1444053967.1647566.401641825.08F2130B@webmail.messagingengine.com>

On Mon, Sep 28, 2015, at 15:47, Andrew Barnert via Python-ideas wrote:
> spam?.eggs.cheese becomes this pseudo-AST (I've skipped the loads and
> maybe some other stuff):
> 
>     Expr(
>         value=Attribute(
>             value=Attribute(
>                 value=Name(id='spam'), attr='eggs', uptalk=True),
>             attr='cheese', uptalk=False))

> ? which is then compiled as this pseudo-bytecode:
> 
>     LOAD_NAME 'spam'
>     DUP_TOP
>     POP_JUMP_IF_NONE :label
>     LOAD_ATTR 'eggs'
>     LOAD_ATTR 'cheese'
>     :label

To put this in more concrete terms... What pseudo-AST does
(spam?.eggs).cheese end up as, if the notion that uptalk must not escape
parentheses is accepted?

The pseudo-bytecode is obvious:
LOAD_NAME 'spam'
JUMP_IF_NONE :label
LOAD_ATTR 'eggs'
:label
LOAD_ATTR 'cheese'

[If we're going to define a new opcode, might as well be one that
doesn't require a dup]

The AST, though, not so much. spam.eggs.cheese and (spam.eggs).cheese
are identical:

Expr(Attribute(Attribute(Name('spam'), 'eggs'), 'cheese'))

And you gave this for the non-parenthesized spam?.eggs.cheese
Expr(Attribute(Attribute(Name('spam'), 'eggs', True), 'cheese', False))

> I suppose the reference documentation wording is also important here, to
> explain that an uptalked attributeref or subscription short-circuits the
> whole primary.

Primary's not the right term here, because a parenthesized expression is
also a primary. And so is the first three (first two, first four) terms
of a five-dot expression. In the grammar the thing we want is called
atom_expr. Why *is* the BNF in the syntax documentation different from
the BNF in the grammar anyway?

From tjreedy at udel.edu  Mon Oct  5 22:18:46 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 5 Oct 2015 16:18:46 -0400
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <561218A1.6030100@mail.de>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org> <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
 <561218A1.6030100@mail.de>
Message-ID: <muulvi$qro$1@ger.gmane.org>

On 10/5/2015 2:28 AM, Sven R. Kunze wrote:
> On 04.10.2015 01:32, Andrew Barnert via Python-ideas wrote:
>> And now, everywhere you use it looks like this:
>>
>>      remove_all(arr, 1)
>>
>> And it's hard to imagine anything more readable.
>
> arr.remove_all(1)
>
>> And, even if remove_all isn't the kind of function an experienced
>> developer would write, learning how to factor out the tricky bits into
>> documentable and testable functions is one of the most useful skills
>> for any developer in any language.#
>
> True.
>
>
> Btw. the same is true for Python core devs. This said, I would
> appreciate the method 'remove_all' provided by the stdlib. ;-)

The problem with methods is that they only work with one class.  A 
list.removeall would only remove things equal to a specific item from a 
list (and presumably in place).  We already have both a generic filter 
function and syntax that will remove all items from any iterable that 
meet any condition.  The stream can be fed into any other function that 
accept an iterable.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Mon Oct  5 23:25:24 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 5 Oct 2015 17:25:24 -0400
Subject: [Python-ideas] Non-English names in the turtle module (with
	Spanish example)
In-Reply-To: <CAPyZGSn5woL=jde7abXh4oJmhZpgy2bRux6gdVojKM5QqtRPtg@mail.gmail.com>
References: <CAPyZGSn5woL=jde7abXh4oJmhZpgy2bRux6gdVojKM5QqtRPtg@mail.gmail.com>
Message-ID: <muupsg$stk$1@ger.gmane.org>

On 9/16/2015 6:28 PM, Al Sweigart wrote:
> I've created a prototype for how we could add foreign language names to
> the turtle.py module

Before going very far with turtle.py, you should be aware that the 
current turtle.py, writen by Gregor Lingl,  is a replacement for the 
first version. It has two problems.

First, it was never properly contributed to Python.  Lingl did not sign 
the contributor agreement before Martin von Loewis committed it, and has 
not done so subsequently when requested.  It has its own copyright and 
proprietary license, which pydev has violated.

Second, the original was written in tkinter, while Gregor's version is 
writen in his custom graphics language, which in turn is implemented in 
tkinter.  Gregor planned to also implement his new intermediate layer in 
some other framework, though he never did.

I personally will not touch the current turtle.py.  I would rather 
revert to the original tkinter version, fix that, and move forward. But 
I am currently busy enough with IDLE.

> and erase the language barrier for non-English schoolkids.

The third problem is that this is not a a priority for any core 
developer.  Turtle itself is not anyone's priority.

> The Tortuga module has the same functionality as

turtle, I presume

> You can test it out by running "pip install tortuga"
>
> https://pypi.python.org/pypi/Tortuga

You should keep this as a pypi project.  Future turtle.py could be 
changed to load external dictionaries without being monkey patched, but 
even that would do nothing for existing installations.  I feel strongly 
that language packs not be put in the stdlib.  Each dictionary has to be 
checked for accuracy and suitability for kids.  This is not a job for 
pydev.

> Since Python 2 doesn't have simpledialog,

Since python-ideas is for possible new features, and new features only 
go in future 3.x releases, code discussed here should be Python 3.

> Check out the diff between Tortuga and turtle.py here:
> https://www.diffchecker.com/2xmbrkhk
>
> This file can be easily adapted to support multiple programming languages.

I think only one language should be loaded at a time.  With 50 
languages, it is possible that the same 'word' (sequence of letters) 
might be used by different languages to translate different turtle 
words.  Then the last language loaded would win.

> Thoughts? Suggestions?

Good idea. Go ahead and do it -- on pypi.

-- 
Terry Jan Reedy


From mal at egenix.com  Tue Oct  6 13:24:08 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 6 Oct 2015 13:24:08 +0200
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <muulvi$qro$1@ger.gmane.org>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org> <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
 <561218A1.6030100@mail.de> <muulvi$qro$1@ger.gmane.org>
Message-ID: <5613AF58.6070802@egenix.com>

If you have more than a few values to remove, it's often
faster to create a new list, since each removal will
require a copy operation of all trailing items and (every
now and then) a realloc of the list object to free
up the unused space:

new_arr = [x
           for x in arr
           if x != 1]


On 05.10.2015 22:18, Terry Reedy wrote:
> On 10/5/2015 2:28 AM, Sven R. Kunze wrote:
>> On 04.10.2015 01:32, Andrew Barnert via Python-ideas wrote:
>>> And now, everywhere you use it looks like this:
>>>
>>>      remove_all(arr, 1)
>>>
>>> And it's hard to imagine anything more readable.
>>
>> arr.remove_all(1)
>>
>>> And, even if remove_all isn't the kind of function an experienced
>>> developer would write, learning how to factor out the tricky bits into
>>> documentable and testable functions is one of the most useful skills
>>> for any developer in any language.#
>>
>> True.
>>
>>
>> Btw. the same is true for Python core devs. This said, I would
>> appreciate the method 'remove_all' provided by the stdlib. ;-)
> 
> The problem with methods is that they only work with one class.  A list.removeall would only remove
> things equal to a specific item from a list (and presumably in place).  We already have both a
> generic filter function and syntax that will remove all items from any iterable that meet any
> condition.  The stream can be fed into any other function that accept an iterable.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From steve at pearwood.info  Tue Oct  6 18:00:24 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 7 Oct 2015 03:00:24 +1100
Subject: [Python-ideas] PEP 506 (secrets module) and token functions
In-Reply-To: <20150926130715.GG23642@ando.pearwood.info>
References: <20150919181612.GT31152@ando.pearwood.info>
 <20150926130715.GG23642@ando.pearwood.info>
Message-ID: <20151006160024.GA28222@ando.pearwood.info>

Hi all,

An updated version of PEP 506 is now available:

https://www.python.org/dev/peps/pep-0506/


If there are no major objections, I intend to take it to python-dev 
in a day or two for discussion and a ruling.

Thank you to everyone who contributed to the discussion.


-- 
Steve

From emilrosendahlpetersen at outlook.com  Tue Oct  6 21:25:00 2015
From: emilrosendahlpetersen at outlook.com (Emil Rosendahl Petersen)
Date: Tue, 6 Oct 2015 12:25:00 -0700
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
Message-ID: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>

I think string.replace should be changed accept a list as a first
argument.

That way, if I had this string:

"There are a lot of undesirable people in this filthy world"

Then I could do this, replace(['undesirable', 'filthy'], ''), in case
that's what I wanted to do.

Now, string.replace doesn't accept a list as its first argument, and
complains about implicit conversion.

Is there any great obstacle to just having the function loop over that
list, calling itself in case we get a list argument instead of a str?

Doesn't that seem like the more obvious behaviour? To me the results of
running the above code should be unsurprising, if this change was
implemented: "there are a lot of people in this world".

/ Emil Petersen


From srkunze at mail.de  Tue Oct  6 18:29:54 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 06 Oct 2015 18:29:54 +0200
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <5613AF58.6070802@egenix.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org> <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
 <561218A1.6030100@mail.de> <muulvi$qro$1@ger.gmane.org>
 <5613AF58.6070802@egenix.com>
Message-ID: <5613F702.2000502@mail.de>

On 06.10.2015 13:24, M.-A. Lemburg wrote:
> If you have more than a few values to remove, it's often
> faster to create a new list, since each removal will
> require a copy operation of all trailing items and (every
> now and then) a realloc of the list object to free
> up the unused space:
>
> new_arr = [x
>             for x in arr
>             if x != 1]
>

I think that's a technical detail which can be sorted out in C somehow 
more efficiently.

But the proposal is not about performance rather than readability and 
maintainability.

> On 05.10.2015 22:18, Terry Reedy wrote:
>> The problem with methods is that they only work with one class.  A list.removeall would only remove
>> things equal to a specific item from a list (and presumably in place).  We already have both a
>> generic filter function and syntax that will remove all items from any iterable that meet any
>> condition.  The stream can be fed into any other function that accept an iterable.

remove also removes in-place but it's not overly useful when you know or 
unsure about whether you need to remove an item multiple times.

remove_all would basically complement remove as a more generic alternative.

Btw. one could also think of an additional generalization for this that 
basically takes n arguments which are then removed altogether from the 
list in question:

a = [1,2,3,5,4,2,3,4,1,2]
a.remove_all(1,4,2)
a == [3,5,3]

Just thinking, and I might remember some time where I would have found 
it useful. Not sure.

Best,
Sven

From mal at egenix.com  Tue Oct  6 18:34:16 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 6 Oct 2015 18:34:16 +0200
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
Message-ID: <5613F808.1050702@egenix.com>

On 06.10.2015 21:25, Emil Rosendahl Petersen wrote:
> I think string.replace should be changed accept a list as a first
> argument.
> 
> That way, if I had this string:
> 
> "There are a lot of undesirable people in this filthy world"
> 
> Then I could do this, replace(['undesirable', 'filthy'], ''), in case
> that's what I wanted to do.
> 
> Now, string.replace doesn't accept a list as its first argument, and
> complains about implicit conversion.
> 
> Is there any great obstacle to just having the function loop over that
> list, calling itself in case we get a list argument instead of a str?
> 
> Doesn't that seem like the more obvious behaviour? To me the results of
> running the above code should be unsurprising, if this change was
> implemented: "there are a lot of people in this world".

I think the "one obvious way" of doing a multi-replace is to
use the re module, since implementing this efficiently is
non-trivial.

String methods are meant to be basic (high performance)
operations.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From rymg19 at gmail.com  Tue Oct  6 18:37:20 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 6 Oct 2015 11:37:20 -0500
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
Message-ID: <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>

import re

result = re.sub('undesirable|filthy', '', 'There are a lot of undesirable
people in this filthy world')
print(result) # There are a lot of people in this world


On Tue, Oct 6, 2015 at 2:25 PM, Emil Rosendahl Petersen <
emilrosendahlpetersen at outlook.com> wrote:

> I think string.replace should be changed accept a list as a first
> argument.
>
> That way, if I had this string:
>
> "There are a lot of undesirable people in this filthy world"
>
> Then I could do this, replace(['undesirable', 'filthy'], ''), in case
> that's what I wanted to do.
>
> Now, string.replace doesn't accept a list as its first argument, and
> complains about implicit conversion.
>
> Is there any great obstacle to just having the function loop over that
> list, calling itself in case we get a list argument instead of a str?
>
> Doesn't that seem like the more obvious behaviour? To me the results of
> running the above code should be unsurprising, if this change was
> implemented: "there are a lot of people in this world".
>
> / Emil Petersen
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something?s wrong.
http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151006/2a8eff0d/attachment-0001.html>

From python at mrabarnett.plus.com  Tue Oct  6 19:03:38 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 6 Oct 2015 18:03:38 +0100
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
Message-ID: <5613FEEA.1040703@mrabarnett.plus.com>

On 2015-10-06 20:25, Emil Rosendahl Petersen wrote:
> I think string.replace should be changed accept a list as a first
> argument.
>
> That way, if I had this string:
>
> "There are a lot of undesirable people in this filthy world"
>
> Then I could do this, replace(['undesirable', 'filthy'], ''), in case
> that's what I wanted to do.
>
> Now, string.replace doesn't accept a list as its first argument, and
> complains about implicit conversion.
>
> Is there any great obstacle to just having the function loop over that
> list, calling itself in case we get a list argument instead of a str?
>
> Doesn't that seem like the more obvious behaviour? To me the results of
> running the above code should be unsurprising, if this change was
> implemented: "there are a lot of people in this world".
>
Looping over the list is the wrong way to do it because each pass might
produce a string that leads to different matches for the subsequent
passes.

I think that the replacements should be done based on the earliest
longest match.


From srkunze at mail.de  Tue Oct  6 19:35:03 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 06 Oct 2015 19:35:03 +0200
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
Message-ID: <56140647.4030501@mail.de>

On 06.10.2015 21:25, Emil Rosendahl Petersen wrote:
> I think string.replace should be changed accept a list as a first
> argument.

I think I can understand the sentiment here.

However, I would prefer a str.replace_all method that would accept it. I 
cannot think of any place, where I would want to use either a str or a 
list of str. So, this could hide bugs.


Speaking of "replace", sometimes, I would love to pass an "replace all 
of these with all of those" dict, which is then processed internally. I 
can remember two times where we needed to write some kind of for-loop; 
which actually might produce wrong results:

convert_dict = {
     '1': '2',
     '2': '3',
}
original = '12345'
for from_, to in convert_dict.items():
     original = original.replace(from_, to)

Real-world examples don't really have this issue so we accepted this 
kind of workaround for smaller scripts. But I would prefer a stdlib 
solution for this.


Best,
Sven

From srkunze at mail.de  Tue Oct  6 19:47:30 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 06 Oct 2015 19:47:30 +0200
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
Message-ID: <56140932.1090700@mail.de>

On 06.10.2015 18:37, Ryan Gonzalez wrote:
> import re
>
> result = re.sub('undesirable|filthy', '', 'There are a lot of 
> undesirable people in this filthy world')
> print(result) # There are a lot of people in this world

Yes, but(TM):

The OP has a list, not a string. Yes, he could create one by 
'|'.join(['undesirable', 'filthy']) but that's like:

"Hey, I have some structured data, let's create some applesauce and have 
a secondary parser re-create structured data from it." IMHO that feels 
wrong.

Best,
Sven

From guido at python.org  Tue Oct  6 20:26:11 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Oct 2015 11:26:11 -0700
Subject: [Python-ideas] PEP 506 (secrets module) and token functions
In-Reply-To: <20151006160024.GA28222@ando.pearwood.info>
References: <20150919181612.GT31152@ando.pearwood.info>
 <20150926130715.GG23642@ando.pearwood.info>
 <20151006160024.GA28222@ando.pearwood.info>
Message-ID: <CAP7+vJLrvNoCZX6xRe076tYGw_D8gwjK7sfTGApqOcwnPj=Zxg@mail.gmail.com>

This is already looking good. An additional advantage to having a new
module (as opposed to changing random) is that it could easily be
backported as a PyPI package, all the way back to Python 2.7.

I do still think that having a concrete proposal for what should
(initially) go into secrets.py would make for a more compelling PEP.

On Tue, Oct 6, 2015 at 9:00 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> Hi all,
>
> An updated version of PEP 506 is now available:
>
> https://www.python.org/dev/peps/pep-0506/
>
>
> If there are no major objections, I intend to take it to python-dev
> in a day or two for discussion and a ruling.
>
> Thank you to everyone who contributed to the discussion.
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151006/5b3eef45/attachment.html>

From moiein2000 at gmail.com  Tue Oct  6 21:40:55 2015
From: moiein2000 at gmail.com (Matthew Einhorn)
Date: Tue, 6 Oct 2015 15:40:55 -0400
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <56140647.4030501@mail.de>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <56140647.4030501@mail.de>
Message-ID: <CALCauYfbYoZ7gQRxvj5k1yseepe_8HD_ivGKe42nqCi5bV0CEg@mail.gmail.com>

On Tue, Oct 6, 2015 at 1:35 PM, Sven R. Kunze <srkunze at mail.de> wrote:

> ...
> Speaking of "replace", sometimes, I would love to pass an "replace all of
> these with all of those" dict, which is then processed internally. I can
> remember two times where we needed to write some kind of for-loop; which
> actually might produce wrong results:
>
> convert_dict = {
>     '1': '2',
>     '2': '3',
> }
> original = '12345'
> for from_, to in convert_dict.items():
>     original = original.replace(from_, to)
>

Also, you can do:

  re.sub('|'.join(convert_dict.keys()), lambda x: convert_dict[x.group(0)],
original)

Which is probably better as it performs the match at once. But of course
you have to know ahead of time what the keys of the dict may be as they
need to be escaped etc.

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151006/ba3f46c8/attachment.html>

From abarnert at yahoo.com  Tue Oct  6 22:59:59 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Tue, 6 Oct 2015 13:59:59 -0700
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
Message-ID: <BC2FECF0-0C68-43C2-9EE6-DFE1E9E1C861@yahoo.com>

On Oct 6, 2015, at 12:25, Emil Rosendahl Petersen <emilrosendahlpetersen at outlook.com> wrote:
> 
> I think string.replace should be changed accept a list as a first
> argument.

Just a list? If I call it with a tuple, or some other kind of iterable, should I get a TypeError telling me to pass a str or list?

But of course str is also an iterable of str, which makes it ambiguous if you do otherwise. Python does have a few functions they can take either one Foo or multiple Food (and Foo may itself be iterable), but everywhere else it uses tuple as the special type, not list.

> That way, if I had this string:
> 
> "There are a lot of undesirable people in this filthy world"
> 
> Then I could do this, replace(['undesirable', 'filthy'], ''), in case
> that's what I wanted to do.

Even without the multiple arguments, this is a very strange use of replace. For one thing, it leaves extra spaces behind. Also, I suspect your be tempted to use it similarly if you wanted to remove "filth", and the complain that it turned "filthy" into "y".

I suspect that, in your actual motivating code, you want to either split the string, filter it, and rejoin it, or use regular expressions or a more complicated parser.

> Now, string.replace doesn't accept a list as its first argument, and
> complains about implicit conversion.
> 
> Is there any great obstacle to just having the function loop over that
> list, calling itself in case we get a list argument instead of a str?

There are various inconsistent things that replace could do with multiple arguments, but I think this is the one you'd be least likely to want when they differ, and that would surprise people the most.

Consider cases like removing "fily" and "th": should removing the "th" from "filthy" make it eligible to have the remaining "fily" replaced? Does it matter which order they're passed in the list? What if the replacement string isn't "" but "c"; does that mean "filcy" can be replaced? Of course as long as you think of a string as a sequence of words rather than a sequence of characters, you don't think of these issues?which is exactly why I think you should probably be splitting the string into words, so you don't have to.

Or, if you really do want to do this character by character, you need to think through what you mean by how it affects order, greediness, etc. Often, the simplest way to express that is a regular expression, in which case, just do that. If you explicitly want something that's hard to express in a regexp, it's probably uncommon enough that you don't want it as a method on str, and want to have the logic in clear Python code for later reading.

> Doesn't that seem like the more obvious behaviour? To me the results of
> running the above code should be unsurprising, if this change was
> implemented: "there are a lot of people in this world".
> 
> / Emil Petersen
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From rosuav at gmail.com  Wed Oct  7 00:50:15 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 7 Oct 2015 09:50:15 +1100
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <5613F702.2000502@mail.de>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org>
 <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
 <561218A1.6030100@mail.de> <muulvi$qro$1@ger.gmane.org>
 <5613AF58.6070802@egenix.com> <5613F702.2000502@mail.de>
Message-ID: <CAPTjJmqB=oj2xrw8=pce4Z4UB65hq7Qkc9TaX2pEbvRxjh=w_w@mail.gmail.com>

On Wed, Oct 7, 2015 at 3:29 AM, Sven R. Kunze <srkunze at mail.de> wrote:
> Btw. one could also think of an additional generalization for this that
> basically takes n arguments which are then removed altogether from the list
> in question:
>
> a = [1,2,3,5,4,2,3,4,1,2]
> a.remove_all(1,4,2)
> a == [3,5,3]
>
> Just thinking, and I might remember some time where I would have found it
> useful. Not sure.
>

The more generalizations you offer, the more tempting the
comprehension looks. It already allows as many generalizations as you
like, because comprehensions are a fully-supported part of the
language.

ChrisA

From srkunze at mail.de  Wed Oct  7 01:21:10 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 07 Oct 2015 01:21:10 +0200
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <CAPTjJmqB=oj2xrw8=pce4Z4UB65hq7Qkc9TaX2pEbvRxjh=w_w@mail.gmail.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org> <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
 <561218A1.6030100@mail.de> <muulvi$qro$1@ger.gmane.org>
 <5613AF58.6070802@egenix.com> <5613F702.2000502@mail.de>
 <CAPTjJmqB=oj2xrw8=pce4Z4UB65hq7Qkc9TaX2pEbvRxjh=w_w@mail.gmail.com>
Message-ID: <56145766.8000103@mail.de>

On 07.10.2015 00:50, Chris Angelico wrote:
> On Wed, Oct 7, 2015 at 3:29 AM, Sven R. Kunze <srkunze at mail.de> wrote:
>> Btw. one could also think of an additional generalization for this that
>> basically takes n arguments which are then removed altogether from the list
>> in question:
>>
>> a = [1,2,3,5,4,2,3,4,1,2]
>> a.remove_all(1,4,2)
>> a == [3,5,3]
>>
>> Just thinking, and I might remember some time where I would have found it
>> useful. Not sure.
>>
> The more generalizations you offer, the more tempting the
> comprehension looks.

To you, not to everybody.

> It already allows as many generalizations as you
> like, because comprehensions are a fully-supported part of the
> language.

You miss the point. Common use-cases deserve methods on their own. 
People will never stop asking for meaningfully named methods, even when 
pretending comprehensions are the ultimate answer to all questions 
concerning lists/sets/dicts, They simply are not.

Which variant conveys the intent of the developer more clearly?

a.remove_all(1,4,2)
a[:] = [x for x in a if x not in {1,4,2}]

I have to admit, the latter variant has certain appeal if you love 
special characters.

Best,
Sven

From rosuav at gmail.com  Wed Oct  7 02:01:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 7 Oct 2015 11:01:10 +1100
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <56145766.8000103@mail.de>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org>
 <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com>
 <561218A1.6030100@mail.de> <muulvi$qro$1@ger.gmane.org>
 <5613AF58.6070802@egenix.com> <5613F702.2000502@mail.de>
 <CAPTjJmqB=oj2xrw8=pce4Z4UB65hq7Qkc9TaX2pEbvRxjh=w_w@mail.gmail.com>
 <56145766.8000103@mail.de>
Message-ID: <CAPTjJmpf=QFNb=5tXDf7LWtpd1=rG5rpFQ1g3=dxVAw04MGhtg@mail.gmail.com>

On Wed, Oct 7, 2015 at 10:21 AM, Sven R. Kunze <srkunze at mail.de> wrote:
> You miss the point. Common use-cases deserve methods on their own. People
> will never stop asking for meaningfully named methods, even when pretending
> comprehensions are the ultimate answer to all questions concerning
> lists/sets/dicts, They simply are not.
>
> Which variant conveys the intent of the developer more clearly?
>
> a.remove_all(1,4,2)
> a[:] = [x for x in a if x not in {1,4,2}]
>

Maybe, but how many other variants do you need? "remove all elements
2.7<x<7.5"? "remove all strings that begin with 'a'"? Every new method
you create adds cognitive load to everyone who reads the docs for the
list object, and every new feature of a method adds cognitive load to
understanding that method. Is it worth it? How common *is* this case?
Is it really worth having the method? Comprehensions already exist,
and are already general enough to handle all the variants.

ChrisA

From random832 at fastmail.com  Wed Oct  7 02:49:21 2015
From: random832 at fastmail.com (Random832)
Date: Tue, 06 Oct 2015 20:49:21 -0400
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <56140932.1090700@mail.de> (Sven R. Kunze's message of "Tue, 06
 Oct 2015 19:47:30 +0200")
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de>
Message-ID: <m28u7fo63y.fsf@fastmail.com>

"Sven R. Kunze" <srkunze at mail.de> writes:

> On 06.10.2015 18:37, Ryan Gonzalez wrote:
> The OP has a list, not a string. Yes, he could create one by
> '|'.join(['undesirable', 'filthy']) but that's like:
>
> "Hey, I have some structured data, let's create some applesauce and
> have a secondary parser re-create structured data from it." IMHO that
> feels wrong.

Not sure how to get around that, other than by creating a general
"structured regex" module, to build a compiled regex from an abstract
regex syntax tree rather than a string. Which actually might not be the
worst thing in the world. I think there are some Lisp dialects that have
something like that.

From rosuav at gmail.com  Wed Oct  7 02:59:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 7 Oct 2015 11:59:31 +1100
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <m28u7fo63y.fsf@fastmail.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de> <m28u7fo63y.fsf@fastmail.com>
Message-ID: <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>

On Wed, Oct 7, 2015 at 11:49 AM, Random832 <random832 at fastmail.com> wrote:
> Not sure how to get around that, other than by creating a general
> "structured regex" module, to build a compiled regex from an abstract
> regex syntax tree rather than a string. Which actually might not be the
> worst thing in the world. I think there are some Lisp dialects that have
> something like that.

Ooh, I like this idea. Kinda like the difference between building up
an SQL statement by escaping all your input, and using a parameterized
statement.

ChrisA

From python at mrabarnett.plus.com  Wed Oct  7 03:05:42 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 7 Oct 2015 02:05:42 +0100
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de> <m28u7fo63y.fsf@fastmail.com>
 <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
Message-ID: <56146FE6.2050309@mrabarnett.plus.com>

On 2015-10-07 01:59, Chris Angelico wrote:
> On Wed, Oct 7, 2015 at 11:49 AM, Random832 <random832 at fastmail.com> wrote:
>> Not sure how to get around that, other than by creating a general
>> "structured regex" module, to build a compiled regex from an abstract
>> regex syntax tree rather than a string. Which actually might not be the
>> worst thing in the world. I think there are some Lisp dialects that have
>> something like that.
>
> Ooh, I like this idea. Kinda like the difference between building up
> an SQL statement by escaping all your input, and using a parameterized
> statement.
>
There's always this:

https://pypi.python.org/pypi/regex

Look at "Named lists". ;-)


From emilrosendahlpetersen at outlook.com  Wed Oct  7 06:06:40 2015
From: emilrosendahlpetersen at outlook.com (Emil Rosendahl Petersen)
Date: Tue, 6 Oct 2015 21:06:40 -0700
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <CAPTjJmpf=QFNb=5tXDf7LWtpd1=rG5rpFQ1g3=dxVAw04MGhtg@mail.gmail.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org>
 <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com> <561218A1.6030100@mail.de>
 <muulvi$qro$1@ger.gmane.org> <5613AF58.6070802@egenix.com>
 <5613F702.2000502@mail.de>
 <CAPTjJmqB=oj2xrw8=pce4Z4UB65hq7Qkc9TaX2pEbvRxjh=w_w@mail.gmail.com>
 <56145766.8000103@mail.de>
 <CAPTjJmpf=QFNb=5tXDf7LWtpd1=rG5rpFQ1g3=dxVAw04MGhtg@mail.gmail.com>
Message-ID: <BLU437-SMTP384A13CDCEA61677ECCDFBBB360@phx.gbl>

On Wed, 2015-10-07 at 11:01 +1100, Chris Angelico wrote:
> On Wed, Oct 7, 2015 at 10:21 AM, Sven R. Kunze <srkunze at mail.de> wrote:
> > You miss the point. Common use-cases deserve methods on their own. People
> > will never stop asking for meaningfully named methods, even when pretending
> > comprehensions are the ultimate answer to all questions concerning
> > lists/sets/dicts, They simply are not.
> >
> > Which variant conveys the intent of the developer more clearly?
> >
> > a.remove_all(1,4,2)
> > a[:] = [x for x in a if x not in {1,4,2}]
> >
> 
> Maybe, but how many other variants do you need? "remove all elements
> 2.7<x<7.5"? "remove all strings that begin with 'a'"? Every new method
> you create adds cognitive load to everyone who reads the docs for the
> list object, and every new feature of a method adds cognitive load to
> understanding that method. Is it worth it? How common *is* this case?
> Is it really worth having the method? Comprehensions already exist,
> and are already general enough to handle all the variants.
> 
> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

Python is supposed to make use of duck-typing. Well... I say the
standard library should apply that philosophy.

If we are supposed to be a dynamic language, why can't we be like perl,
and do list operations if we see lists, and scalar/single operations if
we see single value types?

It really comes down to "quack like a duck", in this case - Is it
possible to loop over each thing in the given list, treat it like a
string, and replace it with the string we apply to each iteration?

I believe it should be.

print(['One thing', 'another thing', 'each of these', 'on their own
line'])

Looking at that... isn't the intent very obvious? Why should python not
respect that?

"A long thing".replace("long ", '') 
>>> a thing

"A plyable soft tube an undescribable color.".replace(['soft',
'plyable'], '').replace('undescribable', 'indescribable')

Isn't the behaviour of this code - as it would be, if this worked -
fairly obvious?

Frankly to me it seems much more readable and step-by-step to me than a
comparable comprehension. What actually physically happens in the code
is easily apparently, unlike in a comprehension, at least until you take
in its local scope variables. 

And lets face it, nested comprehensions can be rather messy. Since
Python does not encourage tail calls, there should be a good and
efficient way to apply operations everywhere.

But maybe map is best? I haven't seen anyone in this thread present an
alternative to changing the function - Perhaps some trick of a decorator
or similar.

But there is no obvious syntax, in my opinion, for accomplishing what I
did with the nested .replace calls.


From rosuav at gmail.com  Wed Oct  7 03:11:40 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 7 Oct 2015 12:11:40 +1100
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <56146FE6.2050309@mrabarnett.plus.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de> <m28u7fo63y.fsf@fastmail.com>
 <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
 <56146FE6.2050309@mrabarnett.plus.com>
Message-ID: <CAPTjJmp6gcN+vAzKpmzRFbOX+a9rq8x4oBH3AhhmcdxixU4etw@mail.gmail.com>

On Wed, Oct 7, 2015 at 12:05 PM, MRAB <python at mrabarnett.plus.com> wrote:
>> Ooh, I like this idea. Kinda like the difference between building up
>> an SQL statement by escaping all your input, and using a parameterized
>> statement.
>>
> There's always this:
>
> https://pypi.python.org/pypi/regex
>
> Look at "Named lists". ;-)

The time machine strikes again!

ChrisA

From steve at pearwood.info  Wed Oct  7 03:30:07 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 7 Oct 2015 12:30:07 +1100
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <5613F808.1050702@egenix.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <5613F808.1050702@egenix.com>
Message-ID: <20151007013007.GC28222@ando.pearwood.info>

On Tue, Oct 06, 2015 at 06:34:16PM +0200, M.-A. Lemburg wrote:
> On 06.10.2015 21:25, Emil Rosendahl Petersen wrote:
> > I think string.replace should be changed accept a list as a first
> > argument.
> > 
> > That way, if I had this string:
> > 
> > "There are a lot of undesirable people in this filthy world"
> > 
> > Then I could do this, replace(['undesirable', 'filthy'], ''), in case
> > that's what I wanted to do.
> > 
> > Now, string.replace doesn't accept a list as its first argument, and
> > complains about implicit conversion.

[Emil]
> > Is there any great obstacle to just having the function loop over that
> > list, calling itself in case we get a list argument instead of a str?

Looping over each replacement item is the wrong solution. Think of 
the result when one of the search strings is a substring of the 
replacement:

py> source = "I ate a chicken salad, and she had a ham sandwich."
py> for term in ["ham", "turkey", "chicken", "spam"]:
...     source = source.replace(term, "spam and cheese")
...
py> print(source)
I ate a spam and cheese and cheese salad, and she had a spam and cheese 
and cheese sandwich.


You need to be a bit more careful about how to do the replacements.


> > Doesn't that seem like the more obvious behaviour? To me the results of
> > running the above code should be unsurprising, if this change was
> > implemented: "there are a lot of people in this world".

[MAL] 
> I think the "one obvious way" of doing a multi-replace is to
> use the re module, since implementing this efficiently is
> non-trivial.
> 
> String methods are meant to be basic (high performance)
> operations.

A similar issue was discussed last month, in the context of str.split 
rather than replace, and I talked about the pitfalls of using the re 
module:

https://mail.python.org/pipermail/python-ideas/2015-September/036586.html


The implementation isn't hard, but it's just tricky enough that some 
people will get it wrong, and just useful enough that a helper function 
will be a good idea. The question is, should that helper function be a 
string method, in the standard library, or merely something that you add 
to your own projects?

def replace_all(source, old, new, count=None):
    if isinstance(old, str):
        return source.replace(old, new, count)
    elif isinstance(old, tuple)
        regex = '|'.join(re.escape(s) for s in old)
        return new.join(re.split(regex, source, count))


-- 
Steve

From steve at pearwood.info  Wed Oct  7 04:02:04 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 7 Oct 2015 13:02:04 +1100
Subject: [Python-ideas] Proposal how to remove all occurrences of a
	value from a Python list
In-Reply-To: <BLU437-SMTP384A13CDCEA61677ECCDFBBB360@phx.gbl>
References: <mupira$tua$1@ger.gmane.org>
 <47AF8D14-EA4D-460C-B487-39DD222A8C09@yahoo.com> <561218A1.6030100@mail.de>
 <muulvi$qro$1@ger.gmane.org> <5613AF58.6070802@egenix.com>
 <5613F702.2000502@mail.de>
 <CAPTjJmqB=oj2xrw8=pce4Z4UB65hq7Qkc9TaX2pEbvRxjh=w_w@mail.gmail.com>
 <56145766.8000103@mail.de>
 <CAPTjJmpf=QFNb=5tXDf7LWtpd1=rG5rpFQ1g3=dxVAw04MGhtg@mail.gmail.com>
 <BLU437-SMTP384A13CDCEA61677ECCDFBBB360@phx.gbl>
Message-ID: <20151007020204.GD28222@ando.pearwood.info>

On Tue, Oct 06, 2015 at 09:06:40PM -0700, Emil Rosendahl Petersen wrote:

> print(['One thing', 'another thing', 'each of these', 'on their own
> line'])
> 
> Looking at that... isn't the intent very obvious? Why should python not
> respect that?

Yes, the intent is obvious, because I am a sentient human who can read 
the English text and guess -- I emphasis that is it just a guess -- what 
you want. Do you believe that Python should do the same thing that I 
did? Read the individual strings, analyse them as English text, and 
understand that because the final item says "on their own line" that is 
your intent.

But what if I wrote this instead:

print(['one string per line', 'three strings per line', 
       'everything on one line', 'nah just kidding',
       'print using two equal-spaced columns'])

What is my intention now? If *you* can't guess what I want, how can the 
interpreter guess?


Do I want the following printed one number per line or all numbers on 
one line? Should I see the list delimiters? Should the output be 
formatted into multiple columns? How many columns? Should each number be 
left-justified, right-justified? Centered?

print([1, 2, 3, 4, 5, 6, 7, 8, 998, 999])


Trying to have a programming language intuit the programmers *intent* is 
a fool's errand: it cannot be done successfully. Computers cannot do 
what we want, they can only do what we tell them to do.


> "A long thing".replace("long ", '') 
> >>> a thing
> 
> "A plyable soft tube an undescribable color.".replace(['soft',
> 'plyable'], '').replace('undescribable', 'indescribable')
> 
> Isn't the behaviour of this code - as it would be, if this worked -
> fairly obvious?

No, of course not. If you think it is obvious, you haven't thought about 
it in enough detail.

What happens if the replacement strings overlap?

What happens if the new string contains one or more of the old strings 
as substrings?

Should the order of the old strings make a difference to the final 
result?

Why a list? Isn't that likely to indicate a programming error?


If replace() took multiple target substrings to be replaced, I have 
answers to those questions. But I don't know if those answers are the 
same as your answers. Maybe they are, maybe they're not. Who knows?

- overlapping target strings shouldn't make a difference;

- neither should the replacement string containing one or more of the 
targets;

- or the order of the targets;

- but a list probably means a programming error, I would prefer to 
require a tuple of substrings to be consistent with other string 
methods, and to avoid any questions of what happens with arbitrary 
iterables.



-- 
Steve

From steve at pearwood.info  Wed Oct  7 04:07:45 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 7 Oct 2015 13:07:45 +1100
Subject: [Python-ideas] PEP 506 (secrets module) and token functions
In-Reply-To: <CAP7+vJLrvNoCZX6xRe076tYGw_D8gwjK7sfTGApqOcwnPj=Zxg@mail.gmail.com>
References: <20150919181612.GT31152@ando.pearwood.info>
 <20150926130715.GG23642@ando.pearwood.info>
 <20151006160024.GA28222@ando.pearwood.info>
 <CAP7+vJLrvNoCZX6xRe076tYGw_D8gwjK7sfTGApqOcwnPj=Zxg@mail.gmail.com>
Message-ID: <20151007020745.GE28222@ando.pearwood.info>

On Tue, Oct 06, 2015 at 11:26:11AM -0700, Guido van Rossum wrote:
> This is already looking good. An additional advantage to having a new
> module (as opposed to changing random) is that it could easily be
> backported as a PyPI package, all the way back to Python 2.7.
> 
> I do still think that having a concrete proposal for what should
> (initially) go into secrets.py would make for a more compelling PEP.


Thanks Guido.

I'm not sure how much more concrete a proposal you are looking for. The 
PEP now lists a sample implementation. I've described it as 
"pseudo-code" only to indicate that it may be incomplete (e.g. missing 
some imports to make it work, lacking in error checking). It also lacks 
docstrings and tests, but otherwise I think it is most of the module.

It's actually not very large, because most of the implementation is 
elsewhere (e.g. the random module). 

Is there something else the PEP should include? Any other requested 
functions? There have been a few functions suggested that were 
requested, e.g. password generation.

https://www.python.org/dev/peps/pep-0506/




-- 
Steve

From random832 at fastmail.com  Wed Oct  7 04:14:54 2015
From: random832 at fastmail.com (Random832)
Date: Tue, 06 Oct 2015 22:14:54 -0400
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
 (Chris Angelico's message of "Wed, 7 Oct 2015 11:59:31 +1100")
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de> <m28u7fo63y.fsf@fastmail.com>
 <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
Message-ID: <m24mi3o25d.fsf@fastmail.com>

Chris Angelico <rosuav at gmail.com> writes:

> On Wed, Oct 7, 2015 at 11:49 AM, Random832 <random832 at fastmail.com> wrote:
>> Not sure how to get around that, other than by creating a general
>> "structured regex" module, to build a compiled regex from an abstract
>> regex syntax tree rather than a string. Which actually might not be the
>> worst thing in the world. I think there are some Lisp dialects that have
>> something like that.
>
> Ooh, I like this idea. Kinda like the difference between building up
> an SQL statement by escaping all your input, and using a parameterized
> statement.

Not really. My idea was that no string in regex syntax ever exists, but
instead something like "(?a|b|c)d*" becomes more like this:
Sequence(Alternate("a","b","c"), Star("d")), which the regex engine
could then use instead of parsing a string to build a state machine.
All strings would be literal strings - "a\.b" would become "a.b" and
"a.b" would become Sequence("a", Dot(), "b")

Then you could just use Alternate(*lst).

From rymg19 at gmail.com  Wed Oct  7 04:19:35 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 06 Oct 2015 21:19:35 -0500
Subject: [Python-ideas] string.replace should accept a list as a
	first	argument
In-Reply-To: <m24mi3o25d.fsf@fastmail.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de> <m28u7fo63y.fsf@fastmail.com>
 <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
 <m24mi3o25d.fsf@fastmail.com>
Message-ID: <3B4C7819-0C14-4737-8D43-DFFD6A7A666E@gmail.com>



On October 6, 2015 9:14:54 PM CDT, Random832 <random832 at fastmail.com> wrote:
>Chris Angelico <rosuav at gmail.com> writes:
>
>> On Wed, Oct 7, 2015 at 11:49 AM, Random832 <random832 at fastmail.com>
>wrote:
>>> Not sure how to get around that, other than by creating a general
>>> "structured regex" module, to build a compiled regex from an
>abstract
>>> regex syntax tree rather than a string. Which actually might not be
>the
>>> worst thing in the world. I think there are some Lisp dialects that
>have
>>> something like that.
>>
>> Ooh, I like this idea. Kinda like the difference between building up
>> an SQL statement by escaping all your input, and using a
>parameterized
>> statement.
>
>Not really. My idea was that no string in regex syntax ever exists, but
>instead something like "(?a|b|c)d*" becomes more like this:
>Sequence(Alternate("a","b","c"), Star("d")), which the regex engine
>could then use instead of parsing a string to build a state machine.
>All strings would be literal strings - "a\.b" would become "a.b" and
>"a.b" would become Sequence("a", Dot(), "b")
>
>Then you could just use Alternate(*lst).

Isn't that kind of like PyParsing with a DFA?

...and that's a really cool idea. Completely coincidentally, I've been working on a JIT-ted regex library in C. There's no parser yet, and I was planning on de-exposing the internal structures, but, after reading this, I'll probably just leave it exposed.

>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From python at mrabarnett.plus.com  Wed Oct  7 04:24:40 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 7 Oct 2015 03:24:40 +0100
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <20151007013007.GC28222@ando.pearwood.info>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <5613F808.1050702@egenix.com> <20151007013007.GC28222@ando.pearwood.info>
Message-ID: <56148268.3030908@mrabarnett.plus.com>

On 2015-10-07 02:30, Steven D'Aprano wrote:
> On Tue, Oct 06, 2015 at 06:34:16PM +0200, M.-A. Lemburg wrote:
>> On 06.10.2015 21:25, Emil Rosendahl Petersen wrote:
>> > I think string.replace should be changed accept a list as a first
>> > argument.
>> >
>> > That way, if I had this string:
>> >
>> > "There are a lot of undesirable people in this filthy world"
>> >
>> > Then I could do this, replace(['undesirable', 'filthy'], ''), in case
>> > that's what I wanted to do.
>> >
>> > Now, string.replace doesn't accept a list as its first argument, and
>> > complains about implicit conversion.
>
> [Emil]
>> > Is there any great obstacle to just having the function loop over that
>> > list, calling itself in case we get a list argument instead of a str?
>
> Looping over each replacement item is the wrong solution. Think of
> the result when one of the search strings is a substring of the
> replacement:
>
> py> source = "I ate a chicken salad, and she had a ham sandwich."
> py> for term in ["ham", "turkey", "chicken", "spam"]:
> ...     source = source.replace(term, "spam and cheese")
> ...
> py> print(source)
> I ate a spam and cheese and cheese salad, and she had a spam and cheese
> and cheese sandwich.
>
>
> You need to be a bit more careful about how to do the replacements.
>
>
>> > Doesn't that seem like the more obvious behaviour? To me the results of
>> > running the above code should be unsurprising, if this change was
>> > implemented: "there are a lot of people in this world".
>
> [MAL]
>> I think the "one obvious way" of doing a multi-replace is to
>> use the re module, since implementing this efficiently is
>> non-trivial.
>>
>> String methods are meant to be basic (high performance)
>> operations.
>
> A similar issue was discussed last month, in the context of str.split
> rather than replace, and I talked about the pitfalls of using the re
> module:
>
> https://mail.python.org/pipermail/python-ideas/2015-September/036586.html
>
>
> The implementation isn't hard, but it's just tricky enough that some
> people will get it wrong, and just useful enough that a helper function
> will be a good idea. The question is, should that helper function be a
> string method, in the standard library, or merely something that you add
> to your own projects?
>
> def replace_all(source, old, new, count=None):
>      if isinstance(old, str):
>          return source.replace(old, new, count)
>      elif isinstance(old, tuple)
>          regex = '|'.join(re.escape(s) for s in old)
>          return new.join(re.split(regex, source, count))
>
>
Again, using the regex module, you can split on a named list, without
having to worry about sorting or escaping the items. :-)


From rymg19 at gmail.com  Wed Oct  7 04:36:00 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 06 Oct 2015 21:36:00 -0500
Subject: [Python-ideas] string.replace should accept a list as a first
	argument
In-Reply-To: <56148268.3030908@mrabarnett.plus.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <5613F808.1050702@egenix.com> <20151007013007.GC28222@ando.pearwood.info>
 <56148268.3030908@mrabarnett.plus.com>
Message-ID: <76387C69-5DF0-4913-B969-5EEE5D6EFC50@gmail.com>

Would you happen to have anything to do with the creation of the regex module?

Just curious. ;)

On October 6, 2015 9:24:40 PM CDT, MRAB <python at mrabarnett.plus.com> wrote:
>On 2015-10-07 02:30, Steven D'Aprano wrote:
>> On Tue, Oct 06, 2015 at 06:34:16PM +0200, M.-A. Lemburg wrote:
>>> On 06.10.2015 21:25, Emil Rosendahl Petersen wrote:
>>> > I think string.replace should be changed accept a list as a first
>>> > argument.
>>> >
>>> > That way, if I had this string:
>>> >
>>> > "There are a lot of undesirable people in this filthy world"
>>> >
>>> > Then I could do this, replace(['undesirable', 'filthy'], ''), in
>case
>>> > that's what I wanted to do.
>>> >
>>> > Now, string.replace doesn't accept a list as its first argument,
>and
>>> > complains about implicit conversion.
>>
>> [Emil]
>>> > Is there any great obstacle to just having the function loop over
>that
>>> > list, calling itself in case we get a list argument instead of a
>str?
>>
>> Looping over each replacement item is the wrong solution. Think of
>> the result when one of the search strings is a substring of the
>> replacement:
>>
>> py> source = "I ate a chicken salad, and she had a ham sandwich."
>> py> for term in ["ham", "turkey", "chicken", "spam"]:
>> ...     source = source.replace(term, "spam and cheese")
>> ...
>> py> print(source)
>> I ate a spam and cheese and cheese salad, and she had a spam and
>cheese
>> and cheese sandwich.
>>
>>
>> You need to be a bit more careful about how to do the replacements.
>>
>>
>>> > Doesn't that seem like the more obvious behaviour? To me the
>results of
>>> > running the above code should be unsurprising, if this change was
>>> > implemented: "there are a lot of people in this world".
>>
>> [MAL]
>>> I think the "one obvious way" of doing a multi-replace is to
>>> use the re module, since implementing this efficiently is
>>> non-trivial.
>>>
>>> String methods are meant to be basic (high performance)
>>> operations.
>>
>> A similar issue was discussed last month, in the context of str.split
>> rather than replace, and I talked about the pitfalls of using the re
>> module:
>>
>>
>https://mail.python.org/pipermail/python-ideas/2015-September/036586.html
>>
>>
>> The implementation isn't hard, but it's just tricky enough that some
>> people will get it wrong, and just useful enough that a helper
>function
>> will be a good idea. The question is, should that helper function be
>a
>> string method, in the standard library, or merely something that you
>add
>> to your own projects?
>>
>> def replace_all(source, old, new, count=None):
>>      if isinstance(old, str):
>>          return source.replace(old, new, count)
>>      elif isinstance(old, tuple)
>>          regex = '|'.join(re.escape(s) for s in old)
>>          return new.join(re.split(regex, source, count))
>>
>>
>Again, using the regex module, you can split on a named list, without
>having to worry about sorting or escaping the items. :-)
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
CURRENTLY LISTENING TO: Prison of Beauty (Kirby Triple Deluxe) by Jun Ishikawa, Hirokazu Ando

From guido at python.org  Wed Oct  7 06:05:12 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Oct 2015 21:05:12 -0700
Subject: [Python-ideas] PEP 506 (secrets module) and token functions
In-Reply-To: <20151007020745.GE28222@ando.pearwood.info>
References: <20150919181612.GT31152@ando.pearwood.info>
 <20150926130715.GG23642@ando.pearwood.info>
 <20151006160024.GA28222@ando.pearwood.info>
 <CAP7+vJLrvNoCZX6xRe076tYGw_D8gwjK7sfTGApqOcwnPj=Zxg@mail.gmail.com>
 <20151007020745.GE28222@ando.pearwood.info>
Message-ID: <CAP7+vJKZQtkq1TodASUoX7Xb6JqhcWh0sN84X0m2js_-G-K+cQ@mail.gmail.com>

On Tue, Oct 6, 2015 at 7:07 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Tue, Oct 06, 2015 at 11:26:11AM -0700, Guido van Rossum wrote:
> > This is already looking good. An additional advantage to having a new
> > module (as opposed to changing random) is that it could easily be
> > backported as a PyPI package, all the way back to Python 2.7.
> >
> > I do still think that having a concrete proposal for what should
> > (initially) go into secrets.py would make for a more compelling PEP.
>
>
> Thanks Guido.
>
> I'm not sure how much more concrete a proposal you are looking for. The
> PEP now lists a sample implementation. I've described it as
> "pseudo-code" only to indicate that it may be incomplete (e.g. missing
> some imports to make it work, lacking in error checking). It also lacks
> docstrings and tests, but otherwise I think it is most of the module.
>

Hm... I totally did not find that when I read the PEP and even just now I
almost missed it!

What I had expected was something that could be directly committed into the
stlib, similar to the statistics.py module you contributed. But apparently
it's so small it could be inlined and overlooked! :-)

Maybe you could clean it up, write some tests, and publish it somewhere?
(Not sure if you do GitHub. :-)

One bikeshed: maybe we should keep only randrange() and drop the confusing
randint()?


> It's actually not very large, because most of the implementation is
> elsewhere (e.g. the random module).
>
> Is there something else the PEP should include? Any other requested
> functions? There have been a few functions suggested that were
> requested, e.g. password generation.
>
> https://www.python.org/dev/peps/pep-0506/
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151006/e55d0a8c/attachment.html>

From random832 at fastmail.com  Wed Oct  7 08:34:58 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 07 Oct 2015 02:34:58 -0400
Subject: [Python-ideas] string.replace should accept a list as a first
 argument
In-Reply-To: <3B4C7819-0C14-4737-8D43-DFFD6A7A666E@gmail.com>
References: <BLU437-SMTP214ADF50BCFC302388E2B8BB370@phx.gbl>
 <CAO41-mNSFh=YpNYYxxaG3g78gFHYgutvBfnMeH9+KQtzChuFJA@mail.gmail.com>
 <56140932.1090700@mail.de> <m28u7fo63y.fsf@fastmail.com>
 <CAPTjJmoN=FRHsuZTPyDjZTLHbWSZvDoR53P7R_mnKOPSJn7oxQ@mail.gmail.com>
 <m24mi3o25d.fsf@fastmail.com>
 <3B4C7819-0C14-4737-8D43-DFFD6A7A666E@gmail.com>
Message-ID: <1444199698.676568.403463313.347A9A6E@webmail.messagingengine.com>

On Tue, Oct 6, 2015, at 22:19, Ryan Gonzalez wrote:
> Isn't that kind of like PyParsing with a DFA?
> 
> ...and that's a really cool idea. Completely coincidentally, I've been
> working on a JIT-ted regex library in C. There's no parser yet, and I was
> planning on de-exposing the internal structures, but, after reading this,
> I'll probably just leave it exposed.

Be careful about validating the resulting structure, though. Circular
references could be dangerous.

I did manage to find the lisp stuff I mentioned:

http://scsh.net/docu/html/man-Z-H-7.html
http://www.ccs.neu.edu/home/shivers/papers/sre.txt
http://srfi.schemers.org/srfi-115/srfi-115.html
https://common-lisp.net/~loliveira/ediware/cl-ppcre/doc/

I think the scsh one was the one I'd actually seen before posting the
idea.

From paddy3118 at gmail.com  Wed Oct  7 08:31:42 2015
From: paddy3118 at gmail.com (Paddy3118)
Date: Tue, 6 Oct 2015 23:31:42 -0700 (PDT)
Subject: [Python-ideas] Rosetta Code on Python.org site?
Message-ID: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>

Just saw what the Perl 6 site does  to punt the curious to their entries on 
the Rosetta Code site here: http://perl6.org/community/rosettacode and I 
thought maybe we could do something similar on Python.org?

I know that the core development team of Perl 6 have put a lot of effort 
into their RC entries, and over the years I have seen evidence of tasks 
being used to highlight areas for subsequent improvement in Perl 6. 

I don't *recognise *such a contribution from core development Pythoneers on 
RC but members of the Python community have made great efforts in making 
Python solutions available for most tasks on RC in a mixture of Python 2 
and 3 and also showing greater use of our interactive command line 
interface/REPL for solutions which other languages may have, but rarely 
show unless the task asks specifically for an answer using a REPL.

*In short:* Python on RC <http://rosettacode.org/wiki/Category:Python> has 
good code examples - why not advertise that fact on Python.org in a similar 
way to the Perl 6 site?

*Disclosure: *I should add that I am an administrator, task writer and one 
of the Python code contributors to the Rosetta Code site.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151006/1f9515d5/attachment.html>

From ncoghlan at gmail.com  Wed Oct  7 13:35:05 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 7 Oct 2015 21:35:05 +1000
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
Message-ID: <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>

On 4 October 2015 at 08:25, Andrew Barnert <abarnert at yahoo.com> wrote:
> Nick's suggestion of having it do %-formatting makes sense. Yes, it means
> that {count:03} is an error and you need '{count:03d}', which is
> inconsistent with f-strings. But that seems like a much less serious problem
> than bytes formatting not being able to handle bytes.

Exactly, if someone is mistakenly thinking
bf"{header}{content}{footer}" is equivalent to
f"{header}{content}{footer}".encode(), they're likely to get immediate
noisy errors when they start trying to format fields.

The parallel I'd attempt to draw is that:

    f"{header}{content}{footer}" is to "{}{}{}".format(header, content, footer)

as:

    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
(header, content, footer)

To make the behaviour clearer in the latter case, it may be reasonable
to *require* an explicit field format code, since that corresponds
more closely to the mandatory field format codes in mod-formatting.

I'm not sold on the idea of a struct.pack conversion specifier - if we
added binary format strings, I think it would be better to start with
explicit "pack(value, format)" expressions, and see how that goes for
a release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From abarnert at yahoo.com  Wed Oct  7 14:34:41 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 7 Oct 2015 05:34:41 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
Message-ID: <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>

On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
>> On 4 October 2015 at 08:25, Andrew Barnert <abarnert at yahoo.com> wrote:
>> Nick's suggestion of having it do %-formatting makes sense. Yes, it means
>> that {count:03} is an error and you need '{count:03d}', which is
>> inconsistent with f-strings. But that seems like a much less serious problem
>> than bytes formatting not being able to handle bytes.
> 
> Exactly, if someone is mistakenly thinking
> bf"{header}{content}{footer}" is equivalent to
> f"{header}{content}{footer}".encode(), they're likely to get immediate
> noisy errors when they start trying to format fields.

Except that multiple people in this thread are saying that'd exactly what it should mean (which I think is a very bad idea).

> The parallel I'd attempt to draw is that:
> 
>    f"{header}{content}{footer}" is to "{}{}{}".format(header, content, footer)
> 
> as:
> 
>    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
> (header, content, footer)
> 
> To make the behaviour clearer in the latter case, it may be reasonable
> to *require* an explicit field format code, since that corresponds
> more closely to the mandatory field format codes in mod-formatting.

Are you suggestive that if a format specifier is given, it must include the format code (which seems perfectly reasonable to me--guessing that :3 means %3b is likely to be wrong more often than it's right?), or that a format specifier must always be given, with no default to :b (which seems more obtrusive and solves less of a problem).


From guido at python.org  Wed Oct  7 18:25:44 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Oct 2015 09:25:44 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
 <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
Message-ID: <CAP7+vJKJ_gqm6nFCv+iKGAts2_FVVJDc7oUg9p4WZ9_B3FZf3A@mail.gmail.com>

I think bf'...' should be compared to b'...' % rather than to f'...'. IOW
bf'...' is to f'...' as b'...'% is to '...'%.

On Wed, Oct 7, 2015 at 5:34 AM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> >> On 4 October 2015 at 08:25, Andrew Barnert <abarnert at yahoo.com> wrote:
> >> Nick's suggestion of having it do %-formatting makes sense. Yes, it
> means
> >> that {count:03} is an error and you need '{count:03d}', which is
> >> inconsistent with f-strings. But that seems like a much less serious
> problem
> >> than bytes formatting not being able to handle bytes.
> >
> > Exactly, if someone is mistakenly thinking
> > bf"{header}{content}{footer}" is equivalent to
> > f"{header}{content}{footer}".encode(), they're likely to get immediate
> > noisy errors when they start trying to format fields.
>
> Except that multiple people in this thread are saying that'd exactly what
> it should mean (which I think is a very bad idea).
>
> > The parallel I'd attempt to draw is that:
> >
> >    f"{header}{content}{footer}" is to "{}{}{}".format(header, content,
> footer)
> >
> > as:
> >
> >    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
> > (header, content, footer)
> >
> > To make the behaviour clearer in the latter case, it may be reasonable
> > to *require* an explicit field format code, since that corresponds
> > more closely to the mandatory field format codes in mod-formatting.
>
> Are you suggestive that if a format specifier is given, it must include
> the format code (which seems perfectly reasonable to me--guessing that :3
> means %3b is likely to be wrong more often than it's right?), or that a
> format specifier must always be given, with no default to :b (which seems
> more obtrusive and solves less of a problem).
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151007/c55cdd4c/attachment.html>

From eric at trueblade.com  Wed Oct  7 19:53:08 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 07 Oct 2015 13:53:08 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAP7+vJKJ_gqm6nFCv+iKGAts2_FVVJDc7oUg9p4WZ9_B3FZf3A@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
 <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
 <CAP7+vJKJ_gqm6nFCv+iKGAts2_FVVJDc7oUg9p4WZ9_B3FZf3A@mail.gmail.com>
Message-ID: <56155C04.3000908@trueblade.com>

On 10/07/2015 12:25 PM, Guido van Rossum wrote:
> I think bf'...' should be compared to b'...' % rather than to f'...'.
> IOW bf'...' is to f'...' as b'...'% is to '...'%.

I'm leaning this way, at least in the sense of "there's a fixed number
of known types supported, and there's no extensible protocol involved.

Eric.

> 
> On Wed, Oct 7, 2015 at 5:34 AM, Andrew Barnert <abarnert at yahoo.com
> <mailto:abarnert at yahoo.com>> wrote:
> 
>     On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com
>     <mailto:ncoghlan at gmail.com>> wrote:
>     >
>     >> On 4 October 2015 at 08:25, Andrew Barnert <abarnert at yahoo.com <mailto:abarnert at yahoo.com>> wrote:
>     >> Nick's suggestion of having it do %-formatting makes sense. Yes, it means
>     >> that {count:03} is an error and you need '{count:03d}', which is
>     >> inconsistent with f-strings. But that seems like a much less serious problem
>     >> than bytes formatting not being able to handle bytes.
>     >
>     > Exactly, if someone is mistakenly thinking
>     > bf"{header}{content}{footer}" is equivalent to
>     > f"{header}{content}{footer}".encode(), they're likely to get immediate
>     > noisy errors when they start trying to format fields.
> 
>     Except that multiple people in this thread are saying that'd exactly
>     what it should mean (which I think is a very bad idea).
> 
>     > The parallel I'd attempt to draw is that:
>     >
>     >    f"{header}{content}{footer}" is to "{}{}{}".format(header, content, footer)
>     >
>     > as:
>     >
>     >    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
>     > (header, content, footer)
>     >
>     > To make the behaviour clearer in the latter case, it may be reasonable
>     > to *require* an explicit field format code, since that corresponds
>     > more closely to the mandatory field format codes in mod-formatting.
> 
>     Are you suggestive that if a format specifier is given, it must
>     include the format code (which seems perfectly reasonable to
>     me--guessing that :3 means %3b is likely to be wrong more often than
>     it's right?), or that a format specifier must always be given, with
>     no default to :b (which seems more obtrusive and solves less of a
>     problem).
> 
> 
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> 
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 


From guido at python.org  Wed Oct  7 19:58:37 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Oct 2015 10:58:37 -0700
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <56155C04.3000908@trueblade.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
 <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
 <CAP7+vJKJ_gqm6nFCv+iKGAts2_FVVJDc7oUg9p4WZ9_B3FZf3A@mail.gmail.com>
 <56155C04.3000908@trueblade.com>
Message-ID: <CAP7+vJKh2DEt0O=VLar81Mrqoj7M6KTKU040fSHx_gpWNem5VQ@mail.gmail.com>

Of course that would still leave the door open for struct.pack support
(maybe recognized by having the string start with <,=, > or @). Pro:
everybody who currently uses struct.pack will love it. Con: the struct.pack
mini-language is pretty inscrutable if you don't already know it. (And no,
I don't propose to invent a different mini-language -- it's just easier to
figure out where to find docs for this when the code explicitly imports the
struct module.)

On Wed, Oct 7, 2015 at 10:53 AM, Eric V. Smith <eric at trueblade.com> wrote:

> On 10/07/2015 12:25 PM, Guido van Rossum wrote:
> > I think bf'...' should be compared to b'...' % rather than to f'...'.
> > IOW bf'...' is to f'...' as b'...'% is to '...'%.
>
> I'm leaning this way, at least in the sense of "there's a fixed number
> of known types supported, and there's no extensible protocol involved.
>
> Eric.
>
> >
> > On Wed, Oct 7, 2015 at 5:34 AM, Andrew Barnert <abarnert at yahoo.com
> > <mailto:abarnert at yahoo.com>> wrote:
> >
> >     On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com
> >     <mailto:ncoghlan at gmail.com>> wrote:
> >     >
> >     >> On 4 October 2015 at 08:25, Andrew Barnert <abarnert at yahoo.com
> <mailto:abarnert at yahoo.com>> wrote:
> >     >> Nick's suggestion of having it do %-formatting makes sense. Yes,
> it means
> >     >> that {count:03} is an error and you need '{count:03d}', which is
> >     >> inconsistent with f-strings. But that seems like a much less
> serious problem
> >     >> than bytes formatting not being able to handle bytes.
> >     >
> >     > Exactly, if someone is mistakenly thinking
> >     > bf"{header}{content}{footer}" is equivalent to
> >     > f"{header}{content}{footer}".encode(), they're likely to get
> immediate
> >     > noisy errors when they start trying to format fields.
> >
> >     Except that multiple people in this thread are saying that'd exactly
> >     what it should mean (which I think is a very bad idea).
> >
> >     > The parallel I'd attempt to draw is that:
> >     >
> >     >    f"{header}{content}{footer}" is to "{}{}{}".format(header,
> content, footer)
> >     >
> >     > as:
> >     >
> >     >    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
> >     > (header, content, footer)
> >     >
> >     > To make the behaviour clearer in the latter case, it may be
> reasonable
> >     > to *require* an explicit field format code, since that corresponds
> >     > more closely to the mandatory field format codes in mod-formatting.
> >
> >     Are you suggestive that if a format specifier is given, it must
> >     include the format code (which seems perfectly reasonable to
> >     me--guessing that :3 means %3b is likely to be wrong more often than
> >     it's right?), or that a format specifier must always be given, with
> >     no default to :b (which seems more obtrusive and solves less of a
> >     problem).
> >
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> >
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151007/f33de49f/attachment.html>

From eric at trueblade.com  Wed Oct  7 20:01:36 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 07 Oct 2015 14:01:36 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <CAP7+vJKh2DEt0O=VLar81Mrqoj7M6KTKU040fSHx_gpWNem5VQ@mail.gmail.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
 <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
 <CAP7+vJKJ_gqm6nFCv+iKGAts2_FVVJDc7oUg9p4WZ9_B3FZf3A@mail.gmail.com>
 <56155C04.3000908@trueblade.com>
 <CAP7+vJKh2DEt0O=VLar81Mrqoj7M6KTKU040fSHx_gpWNem5VQ@mail.gmail.com>
Message-ID: <56155E00.9090804@trueblade.com>

On 10/07/2015 01:58 PM, Guido van Rossum wrote:
> Of course that would still leave the door open for struct.pack support
> (maybe recognized by having the string start with <,=, > or @). Pro:
> everybody who currently uses struct.pack will love it. Con: the
> struct.pack mini-language is pretty inscrutable if you don't already
> know it. (And no, I don't propose to invent a different mini-language --
> it's just easier to figure out where to find docs for this when the code
> explicitly imports the struct module.)

Right. I think Steve Dower's idea of :p switching to struct.pack mode is
reasonable. But as Nick says, we don't need to add it on day 1.

Eric.

> 
> On Wed, Oct 7, 2015 at 10:53 AM, Eric V. Smith <eric at trueblade.com
> <mailto:eric at trueblade.com>> wrote:
> 
>     On 10/07/2015 12:25 PM, Guido van Rossum wrote:
>     > I think bf'...' should be compared to b'...' % rather than to f'...'.
>     > IOW bf'...' is to f'...' as b'...'% is to '...'%.
> 
>     I'm leaning this way, at least in the sense of "there's a fixed number
>     of known types supported, and there's no extensible protocol involved.
> 
>     Eric.
> 
>     >
>     > On Wed, Oct 7, 2015 at 5:34 AM, Andrew Barnert <abarnert at yahoo.com <mailto:abarnert at yahoo.com>
>     > <mailto:abarnert at yahoo.com <mailto:abarnert at yahoo.com>>> wrote:
>     >
>     >     On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>
>     >     <mailto:ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>>> wrote:
>     >     >
>     >     >> On 4 October 2015 at 08:25, Andrew Barnert
>     <abarnert at yahoo.com <mailto:abarnert at yahoo.com>
>     <mailto:abarnert at yahoo.com <mailto:abarnert at yahoo.com>>> wrote:
>     >     >> Nick's suggestion of having it do %-formatting makes sense.
>     Yes, it means
>     >     >> that {count:03} is an error and you need '{count:03d}',
>     which is
>     >     >> inconsistent with f-strings. But that seems like a much
>     less serious problem
>     >     >> than bytes formatting not being able to handle bytes.
>     >     >
>     >     > Exactly, if someone is mistakenly thinking
>     >     > bf"{header}{content}{footer}" is equivalent to
>     >     > f"{header}{content}{footer}".encode(), they're likely to get
>     immediate
>     >     > noisy errors when they start trying to format fields.
>     >
>     >     Except that multiple people in this thread are saying that'd
>     exactly
>     >     what it should mean (which I think is a very bad idea).
>     >
>     >     > The parallel I'd attempt to draw is that:
>     >     >
>     >     >    f"{header}{content}{footer}" is to
>     "{}{}{}".format(header, content, footer)
>     >     >
>     >     > as:
>     >     >
>     >     >    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
>     >     > (header, content, footer)
>     >     >
>     >     > To make the behaviour clearer in the latter case, it may be
>     reasonable
>     >     > to *require* an explicit field format code, since that
>     corresponds
>     >     > more closely to the mandatory field format codes in
>     mod-formatting.
>     >
>     >     Are you suggestive that if a format specifier is given, it must
>     >     include the format code (which seems perfectly reasonable to
>     >     me--guessing that :3 means %3b is likely to be wrong more
>     often than
>     >     it's right?), or that a format specifier must always be given,
>     with
>     >     no default to :b (which seems more obtrusive and solves less of a
>     >     problem).
>     >
>     >
>     >
>     >
>     > --
>     > --Guido van Rossum (python.org/~guido <http://python.org/~guido>
>     <http://python.org/~guido>)
>     >
>     >
>     > _______________________________________________
>     > Python-ideas mailing list
>     > Python-ideas at python.org <mailto:Python-ideas at python.org>
>     > https://mail.python.org/mailman/listinfo/python-ideas
>     > Code of Conduct: http://python.org/psf/codeofconduct/
>     >
> 
> 
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)


From eric at trueblade.com  Wed Oct  7 23:27:11 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 7 Oct 2015 17:27:11 -0400
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <56155E00.9090804@trueblade.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
 <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
 <CAP7+vJKJ_gqm6nFCv+iKGAts2_FVVJDc7oUg9p4WZ9_B3FZf3A@mail.gmail.com>
 <56155C04.3000908@trueblade.com>
 <CAP7+vJKh2DEt0O=VLar81Mrqoj7M6KTKU040fSHx_gpWNem5VQ@mail.gmail.com>
 <56155E00.9090804@trueblade.com>
Message-ID: <5022E7D0-6783-45B7-89BE-C58B166C3FFC@trueblade.com>


> On Oct 7, 2015, at 2:01 PM, Eric V. Smith <eric at trueblade.com> wrote:
> 
>> On 10/07/2015 01:58 PM, Guido van Rossum wrote:
>> Of course that would still leave the door open for struct.pack support
>> (maybe recognized by having the string start with <,=, > or @). Pro:
>> everybody who currently uses struct.pack will love it. Con: the
>> struct.pack mini-language is pretty inscrutable if you don't already
>> know it. (And no, I don't propose to invent a different mini-language --
>> it's just easier to figure out where to find docs for this when the code
>> explicitly imports the struct module.)
> 
> Right. I think Steve Dower's idea of :p switching to struct.pack mode is
> reasonable. But as Nick says, we don't need to add it on day 1.

Make that "!p". 

Eric. 

> 
> Eric.
> 
>> 
>> On Wed, Oct 7, 2015 at 10:53 AM, Eric V. Smith <eric at trueblade.com
>> <mailto:eric at trueblade.com>> wrote:
>> 
>>>    On 10/07/2015 12:25 PM, Guido van Rossum wrote:
>>> I think bf'...' should be compared to b'...' % rather than to f'...'.
>>> IOW bf'...' is to f'...' as b'...'% is to '...'%.
>> 
>>    I'm leaning this way, at least in the sense of "there's a fixed number
>>    of known types supported, and there's no extensible protocol involved.
>> 
>>    Eric.
>> 
>>> 
>>> On Wed, Oct 7, 2015 at 5:34 AM, Andrew Barnert <abarnert at yahoo.com <mailto:abarnert at yahoo.com>
>>> <mailto:abarnert at yahoo.com <mailto:abarnert at yahoo.com>>> wrote:
>>> 
>>>    On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>
>>>    <mailto:ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>>> wrote:
>>>> 
>>>>> On 4 October 2015 at 08:25, Andrew Barnert
>>    <abarnert at yahoo.com <mailto:abarnert at yahoo.com>
>>    <mailto:abarnert at yahoo.com <mailto:abarnert at yahoo.com>>> wrote:
>>>>> Nick's suggestion of having it do %-formatting makes sense.
>>    Yes, it means
>>>>> that {count:03} is an error and you need '{count:03d}',
>>    which is
>>>>> inconsistent with f-strings. But that seems like a much
>>    less serious problem
>>>>> than bytes formatting not being able to handle bytes.
>>>> 
>>>> Exactly, if someone is mistakenly thinking
>>>> bf"{header}{content}{footer}" is equivalent to
>>>> f"{header}{content}{footer}".encode(), they're likely to get
>>    immediate
>>>> noisy errors when they start trying to format fields.
>>> 
>>>    Except that multiple people in this thread are saying that'd
>>    exactly
>>>    what it should mean (which I think is a very bad idea).
>>> 
>>>> The parallel I'd attempt to draw is that:
>>>> 
>>>>   f"{header}{content}{footer}" is to
>>    "{}{}{}".format(header, content, footer)
>>>> 
>>>> as:
>>>> 
>>>>   bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
>>>> (header, content, footer)
>>>> 
>>>> To make the behaviour clearer in the latter case, it may be
>>    reasonable
>>>> to *require* an explicit field format code, since that
>>    corresponds
>>>> more closely to the mandatory field format codes in
>>    mod-formatting.
>>> 
>>>    Are you suggestive that if a format specifier is given, it must
>>>    include the format code (which seems perfectly reasonable to
>>>    me--guessing that :3 means %3b is likely to be wrong more
>>    often than
>>>    it's right?), or that a format specifier must always be given,
>>    with
>>>    no default to :b (which seems more obtrusive and solves less of a
>>>    problem).
>>> 
>>> 
>>> 
>>> 
>>> --
>>> --Guido van Rossum (python.org/~guido <http://python.org/~guido>
>>    <http://python.org/~guido>)
>>> 
>>> 
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org <mailto:Python-ideas at python.org>
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>> 
>> 
>> 
>> 
>> -- 
>> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From ncoghlan at gmail.com  Thu Oct  8 09:39:19 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Oct 2015 17:39:19 +1000
Subject: [Python-ideas] Binary f-strings
In-Reply-To: <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
References: <56089692.1080303@trueblade.com> <560E9292.6040304@mail.de>
 <560E9429.9040205@trueblade.com> <560EA632.80805@mail.de>
 <CAP7+vJK6wk=xU+Zgh44Of+q+TRcxb-EwKrCt-yRTKj--gTGzzA@mail.gmail.com>
 <CADiSq7cvHUnFH7yhR9ELmw+JxOdL+xNuC+VZZrStki8J8ojY6Q@mail.gmail.com>
 <E1ZiPYg-0005Wg-Om@se2-syd.hostedmail.net.au>
 <D8A9F41E-317E-4760-BBD5-8DB03F645DAE@yahoo.com>
 <CADiSq7cdmSjaLS5ddMRkMsppFYKuGoFJehzs7OdL8Obdoy1iVA@mail.gmail.com>
 <FF894DCC-7EFA-444A-8010-E19E220579DA@yahoo.com>
Message-ID: <CADiSq7cawTmsYX0z=8upfdMPo_pjM7w9MACShiXGv7XH-NQZLA@mail.gmail.com>

On 7 October 2015 at 22:34, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Oct 7, 2015, at 04:35, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> The parallel I'd attempt to draw is that:
>>
>>    f"{header}{content}{footer}" is to "{}{}{}".format(header, content, footer)
>>
>> as:
>>
>>    bf"{header:b}{content:b}{footer:b}" would be to b"%b%b%b" %
>> (header, content, footer)
>>
>> To make the behaviour clearer in the latter case, it may be reasonable
>> to *require* an explicit field format code, since that corresponds
>> more closely to the mandatory field format codes in mod-formatting.
>
> Are you suggestive that if a format specifier is given, it must include the format code (which seems perfectly reasonable to me--guessing that :3 means %3b is likely to be wrong more often than it's right?), or that a format specifier must always be given, with no default to :b (which seems more obtrusive and solves less of a problem).

I was thinking the latter, but your idea of ":b" being implied only if
there's no format specifier at all (and otherwise requiring an
explicit "b" or other format code) might be better.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From demianbrecht at gmail.com  Fri Oct  9 22:21:12 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Fri, 9 Oct 2015 13:21:12 -0700
Subject: [Python-ideas] Utility to override custom signal handlers
Message-ID: <CAE+T62a_dtB9_t-oRLmFO5ZE+2RnpvvucQQ6xJ+hN1gPeJZXSw@mail.gmail.com>

Sometimes it's useful to be able to reset a select number of
overridden signal handlers temporarily to allow for default handling.
While trivial, might there be use for such a utility existing in the
signal module? Something to the effect of:


@contextmanager
def default_sighandlers(*signums):
    handlers = []
    for signum in signums:
        handlers.append((signum, signal.getsignal(signum)))
        signal.signal(
            signum,
            signal.SIG_DFL if signum != signal.SIGINT
            else signal.default_int_handler)

    try:
        yield
    finally:
        for signum, handler in handlers:
            signal.signal(signum, handler)


Thoughts?

From andre.roberge at gmail.com  Fri Oct  9 22:22:43 2015
From: andre.roberge at gmail.com (Andre Roberge)
Date: Fri, 9 Oct 2015 17:22:43 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
Message-ID: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>

Summary:  Visual environments based on using
simple functions without using variables have been found
to be useful to introduce programming concepts to young
learners.  While Python is an ideal language for beginners,
the Python construct for iterations, namely

for var in range(n):
    # block
    # of
    # code

does require a variable and is needlessly complicated for
beginners.  A simpler construct like:

repeat n:    # n is a specific integer
    # block
    # of
    # code

would be useful to have in such contexts.  Other keywords could
be chosen instead of "repeat" used above.

====

For young (and perhaps not so young) students
visual environments are an ideal choice to learn programming.
A well-known example of a visual environment for beginners
is Karel the Robot, invented by Richard Pattis in 1981.
In this environment, a robot can perform a limited number
of basic functions which can be combined to create
complex tasks.

In the original Karel the Robot, variables were not
allowed.  However, new procedures could be created.

One essential concept that can be demonstrated in such worlds
is that of iterations.  In the original Karel, this was
done using the following syntax:

ITERATE n TIMES
    single_instruction

or

ITERATE n TIMES
    BEGIN
        single_instruction;
        other_instruction;
    END

where "n" is a specific integer.  In the above, indentation
has been introduced only to make the code easier to read.

A popular Python-like implementation of Pattis' idea
is Guido van Robot [1], also known as GvR.
In GvR, iteration is done as follows:

do n:
    # block
    # of
    # code indented like Python

To do something equivalent in Python requires to use:

for var in range(n):
    # block
    # of
    # code

which requires to use variables, which are not part of
the original Karel the Robot philosophy, as well as an
additional builtin function.  I would argue that,
for beginners, this is needlessly complicated.
A better alternative would be to use a similar
syntax to that used by Guido van Robot for this case.
This would require to either introduce a new keyword
(such as "do", "repeat", "iterate", or "loop")

repeat n:
    # block
    # of
    # code

An advantage of using one of these keywords would
be for easier readability (for English speakers).
The disadvantage is that some existing Python programs
might use variables with the names suggested
above.

An alternative would be to reuse the "for" keyword by
adding this new construct to the grammar:

for n:
    # block
    # of
    # code

This last choice, while perhaps not as immediately readable
to English speakers as some of the previous ones, would
have the advantage of not requiring any change to
any existing Python programs; it could thus be implemented
rather quickly without needing some transition
time and the use of the __future__ module.

There exists at least 4 different pure Python implementations of
Karel the Robot where this new construct could, in principle,
be immediately put to use:

RUR-PLE [2], Rurple NG [3], both of which are desktop versions,
Reeborg's World [4], a web version using Brython as a Python
interpreter, and Monty Karel [5], an Eclipse plugin which
is targeted at older students as it uses a not so simple
OOP syntax even for the simplest programs.
Young learners using any of the first three such implementations
would likely benefit if they could learn programming first
using this simpler construct.

Andr? Roberge

[1] Guido van Robot:
http://gvr.sourceforge.net/documents/languageRef/gvr.html

[2] RUR-PLE: https://code.google.com/p/rur-ple/

[3] Rurple NG:
http://www.lshift.net/downloads/dev.lshift.net/paul/rurple/manual.html

[4] Reeborg's World  documentation: http://reeborg.ca/docs/en/index.html
    World itself: http://reeborg.ca/world.html
    Python tutorial: http://reeborg.ca/docs/begin_py_en/

[5] Monty Karel:
http://www.csis.pace.edu/~bergin/MontyKarel/MontyInEclipse.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151009/6d4dbff2/attachment.html>

From rymg19 at gmail.com  Fri Oct  9 22:27:53 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 09 Oct 2015 15:27:53 -0500
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
Message-ID: <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>

Just use:


for _ in range(n):
    # magic here


and tell them they'll figure out what the _ is later on. That's how I learned it...

BTW, that Guido van Robot thing is kind of funny... :D

On October 9, 2015 3:22:43 PM CDT, Andre Roberge <andre.roberge at gmail.com> wrote:
>Summary:  Visual environments based on using
>simple functions without using variables have been found
>to be useful to introduce programming concepts to young
>learners.  While Python is an ideal language for beginners,
>the Python construct for iterations, namely
>
>for var in range(n):
>    # block
>    # of
>    # code
>
>does require a variable and is needlessly complicated for
>beginners.  A simpler construct like:
>
>repeat n:    # n is a specific integer
>    # block
>    # of
>    # code
>
>would be useful to have in such contexts.  Other keywords could
>be chosen instead of "repeat" used above.
>
>====
>
>For young (and perhaps not so young) students
>visual environments are an ideal choice to learn programming.
>A well-known example of a visual environment for beginners
>is Karel the Robot, invented by Richard Pattis in 1981.
>In this environment, a robot can perform a limited number
>of basic functions which can be combined to create
>complex tasks.
>
>In the original Karel the Robot, variables were not
>allowed.  However, new procedures could be created.
>
>One essential concept that can be demonstrated in such worlds
>is that of iterations.  In the original Karel, this was
>done using the following syntax:
>
>ITERATE n TIMES
>    single_instruction
>
>or
>
>ITERATE n TIMES
>    BEGIN
>        single_instruction;
>        other_instruction;
>    END
>
>where "n" is a specific integer.  In the above, indentation
>has been introduced only to make the code easier to read.
>
>A popular Python-like implementation of Pattis' idea
>is Guido van Robot [1], also known as GvR.
>In GvR, iteration is done as follows:
>
>do n:
>    # block
>    # of
>    # code indented like Python
>
>To do something equivalent in Python requires to use:
>
>for var in range(n):
>    # block
>    # of
>    # code
>
>which requires to use variables, which are not part of
>the original Karel the Robot philosophy, as well as an
>additional builtin function.  I would argue that,
>for beginners, this is needlessly complicated.
>A better alternative would be to use a similar
>syntax to that used by Guido van Robot for this case.
>This would require to either introduce a new keyword
>(such as "do", "repeat", "iterate", or "loop")
>
>repeat n:
>    # block
>    # of
>    # code
>
>An advantage of using one of these keywords would
>be for easier readability (for English speakers).
>The disadvantage is that some existing Python programs
>might use variables with the names suggested
>above.
>
>An alternative would be to reuse the "for" keyword by
>adding this new construct to the grammar:
>
>for n:
>    # block
>    # of
>    # code
>
>This last choice, while perhaps not as immediately readable
>to English speakers as some of the previous ones, would
>have the advantage of not requiring any change to
>any existing Python programs; it could thus be implemented
>rather quickly without needing some transition
>time and the use of the __future__ module.
>
>There exists at least 4 different pure Python implementations of
>Karel the Robot where this new construct could, in principle,
>be immediately put to use:
>
>RUR-PLE [2], Rurple NG [3], both of which are desktop versions,
>Reeborg's World [4], a web version using Brython as a Python
>interpreter, and Monty Karel [5], an Eclipse plugin which
>is targeted at older students as it uses a not so simple
>OOP syntax even for the simplest programs.
>Young learners using any of the first three such implementations
>would likely benefit if they could learn programming first
>using this simpler construct.
>
>Andr? Roberge
>
>[1] Guido van Robot:
>http://gvr.sourceforge.net/documents/languageRef/gvr.html
>
>[2] RUR-PLE: https://code.google.com/p/rur-ple/
>
>[3] Rurple NG:
>http://www.lshift.net/downloads/dev.lshift.net/paul/rurple/manual.html
>
>[4] Reeborg's World  documentation:
>http://reeborg.ca/docs/en/index.html
>    World itself: http://reeborg.ca/world.html
>    Python tutorial: http://reeborg.ca/docs/begin_py_en/
>
>[5] Monty Karel:
>http://www.csis.pace.edu/~bergin/MontyKarel/MontyInEclipse.html
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From python at lucidity.plus.com  Fri Oct  9 23:27:11 2015
From: python at lucidity.plus.com (Erik)
Date: Fri, 9 Oct 2015 22:27:11 +0100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>
Message-ID: <5618312F.8010403@lucidity.plus.com>

On 09/10/15 21:27, Ryan Gonzalez wrote:
> Just use:
>
> for _ in range(n):
>      # magic here
>
> and tell them they'll figure out what the _ is later on. That's how I learned it...

Or, to make it look that little bit more "natural", spell it:

for each in range(n):
   # magic

... and then one day, move on to explain that "each" is actually 
something you can USE rather than just being a fixed part of the syntax 
- at which point it can be spelled better according to what it actually is.

An alternative suggestion to remove the need for the variable (rather 
than a new keyword) would be something like allowing the "var in" part 
to be optioinal:

for <iterator>:
   # magic

... but the problem with that is that pretty much the ONLY use would be 
for this sort of "iterate n times" construct or something where you're 
relying on the iterator to have side-effects (which is pretty nasty):

foo = myiter()
for foo:
   bar(foo.getcurrentvalue())

It seems like a lot of effort just to hide a bit of syntax for a very 
specific use-case when you can just temporarily make that syntax look a 
bit nicer for your audience.

E.

From andre.roberge at gmail.com  Fri Oct  9 23:51:23 2015
From: andre.roberge at gmail.com (Andre Roberge)
Date: Fri, 9 Oct 2015 18:51:23 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <5618312F.8010403@lucidity.plus.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>
 <5618312F.8010403@lucidity.plus.com>
Message-ID: <CAGMu_=qcaJA2CRoNrSoPNdD16_S98xNYvHZGSFCDG=0S1K6P3g@mail.gmail.com>

On Fri, Oct 9, 2015 at 6:27 PM, Erik <python at lucidity.plus.com> wrote:

> On 09/10/15 21:27, Ryan Gonzalez wrote:
>
>> Just use:
>>
>> for _ in range(n):
>>      # magic here
>>
>> and tell them they'll figure out what the _ is later on. That's how I
>> learned it...
>>
>
> Or, to make it look that little bit more "natural", spell it:
>
> for each in range(n):
>   # magic
>
> ... and then one day, move on to explain that "each" is actually something
> you can USE rather than just being a fixed part of the syntax - at which
> point it can be spelled better according to what it actually is.
>
> An alternative suggestion to remove the need for the variable (rather than
> a new keyword) would be something like allowing the "var in" part to be
> optioinal:
>
> for <iterator>:
>   # magic
>
> ... but the problem with that is that pretty much the ONLY use would be
> for this sort of "iterate n times" construct or something where you're
> relying on the iterator to have side-effects (which is pretty nasty):
>
> foo = myiter()
> for foo:
>   bar(foo.getcurrentvalue())
>
> It seems like a lot of effort just to hide a bit of syntax for a very
> specific use-case when you can just temporarily make that syntax look a bit
> nicer for your audience.


I have been using a special builtin alternative (which can be translated
into various human languages, as Reeborg's World is meant to support
various languages) as

def repeat(f, n):
    for i in range(n):
        f()

and used as

repeat(turn_left, 3)

as seen here: http://reeborg.ca/docs/begin_py_en/repeat.html
(and for the French version: http://reeborg.ca/docs/begin_py_fr/repeat.html
).

In smaller tutorials, I sometimes use the standard "for .." construct
instead of this repeat() function, but it always feels clumsy for beginners.

However, repeat()  has its own complexity in that it uses function
arguments, which are inexistant in the original Karel the Robot approach
and do not need to be used for simple worlds in Reeborg's World.

And, just to put things in perspective: I first wrote RUR-PLE in 2004 and
have been working on tutorials for it, discussing with many teachers using
it during all these years.  So, yes, I know I (and others) can make do
without this special construct I am suggesting ... but at the cost of a
larger hurdle for the students than would be required if the alternative
construct existed.

Andr?




>
>
> E.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151009/f32c7911/attachment.html>

From gokoproject at gmail.com  Sat Oct 10 00:36:20 2015
From: gokoproject at gmail.com (John Wong)
Date: Fri, 9 Oct 2015 18:36:20 -0400
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAGMu_=qcaJA2CRoNrSoPNdD16_S98xNYvHZGSFCDG=0S1K6P3g@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>
 <5618312F.8010403@lucidity.plus.com>
 <CAGMu_=qcaJA2CRoNrSoPNdD16_S98xNYvHZGSFCDG=0S1K6P3g@mail.gmail.com>
Message-ID: <CACCLA57dMuTmja6TDEqft-KHX05+mVk5qOPW_6r5viqa5gtUgw@mail.gmail.com>

On Friday, October 9, 2015, Andre Roberge <andre.roberge at gmail.com> wrote:
>
>
> And, just to put things in perspective: I first wrote RUR-PLE in 2004 and
> have been working on tutorials for it, discussing with many teachers using
> it during all these years.  So, yes, I know I (and others) can make do
> without this special construct I am suggesting ... but at the cost of a
> larger hurdle for the students than would be required if the alternative
> construct existed.
>
>
But also at the cost of introducting new syntaxatic sugar which is
literally a few chars fewer,  at the expense of teaching and explain using
one or another, "longer standard" version and the shortcut, is not going to
make students more productive. I think in general people are smart enough
to make decision, recognize tricks. Time should spend on educating students
on writing code. This feels like micro optimization.

I would rather see
for key, value in mydict.items():
And
for index, element in enumerate(mylist)

to be sugar-coated. This is more annoying to type than the problem being
discussed. Everyone's mileage is different but I bet the use of items and
enumerate is too frequent and yet there is no briefer version.

And I also dont think repeat(..) is helpful for a few chars either. Nice to
show how to abstract user from detail implementation (e.g complex repeated
task).




-- 
Sent from Jeff Dean's printf() mobile console
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151009/b41c95f2/attachment.html>

From chris.barker at noaa.gov  Sat Oct 10 01:33:54 2015
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 9 Oct 2015 16:33:54 -0700
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <5618312F.8010403@lucidity.plus.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>
 <5618312F.8010403@lucidity.plus.com>
Message-ID: <-5740670168466924456@unknownmsgid>

> Or, to make it look that little bit more "natural", spell it:
>
> for each in range(n):
>  # magic

Indeed -- while python's for loop can be (and often is) used to repeat
something a certain number of times, what it really is is a way to
apply a block of code to each item in an iterable.

Making a slightly shorter/easier way to do something n times will just
make it harder to "get" what for loops really are later on.

And aside from using it as a teaching tool, does one want to loop
without using the index all that often to get special syntax?

The Karel-style simplified languages may be a great teaching tool --
but that's not what Python is.

-CHB



> ... and then one day, move on to explain that "each" is actually something you can USE rather than just being a fixed part of the syntax - at which point it can be spelled better according to what it actually is.
>
> An alternative suggestion to remove the need for the variable (rather than a new keyword) would be something like allowing the "var in" part to be optioinal:
>
> for <iterator>:
>  # magic
>
> ... but the problem with that is that pretty much the ONLY use would be for this sort of "iterate n times" construct or something where you're relying on the iterator to have side-effects (which is pretty nasty):
>
> foo = myiter()
> for foo:
>  bar(foo.getcurrentvalue())
>
> It seems like a lot of effort just to hide a bit of syntax for a very specific use-case when you can just temporarily make that syntax look a bit nicer for your audience.
>
> E.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From tjreedy at udel.edu  Sat Oct 10 04:30:18 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 9 Oct 2015 22:30:18 -0400
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
Message-ID: <mv9t8d$r0e$2@ger.gmane.org>

On 10/9/2015 4:22 PM, Andre Roberge wrote:
> Summary:  Visual environments based on using
> simple functions without using variables have been found
> to be useful to introduce programming concepts to young
> learners.  While Python is an ideal language for beginners,
> the Python construct for iterations, namely
>
> for var in range(n):
>      # block
>      # of
>      # code
>
> does require a variable and is needlessly complicated for
> beginners.  A simpler construct like:
>
> repeat n:    # n is a specific integer
>      # block
>      # of
>      # code
>
> would be useful to have in such contexts.  Other keywords could
> be chosen instead of "repeat" used above.

...

> do n:
>      # block
>      # of
>      # code indented like Python
>
> To do something equivalent in Python requires to use:
>
> for var in range(n):
>      # block
>      # of
>      # code

...

> An alternative would be to reuse the "for" keyword by
> adding this new construct to the grammar:
>
> for n:
>      # block
>      # of
>      # code

It was once proposed that int n should be an abbreviation for range(n), 
at least in the context of a for statement, partly on the basis that 
counts can be defined in set theory as the set of previous counts, or a 
similar variation, but really for convenience.  Guido vetoed the idea.
---

I think trying to persuade core developers to mutilate Python a bit, at 
couple of years in the future, is futile.  And unneeded.  Instead, 
translate, say 'repeat n:' into 'for _ in range(n):' with something like

def robotrans(codetext):
     out = []
     for i, line in enumerate(codetext.splitlines()):
         if line.startswith('repeat'):
             <check rest of line and either append translation
              or raise SyntaxError with proper details>
         else:
             out.append(line)
     return '\n'.join(outlines)

You could, for instance, patch IDLE two places (for shell and editor 
code) to run robotran on user code before compiling.  You could also add 
'repeat' to the list of keywords for colorizing.  I would be willing to 
help (without any guarantee of code stability).

Guido recently approved on idle-dev that we rethink IDLE for beginners. 
  I have already thought about adding the necessary hooks so that 
education developers like you could define beginner Python dialects, 
such as you suggest here.  This would open a world of possibilities, 
such as 'move 3' versus 'move(3)', and foreign language keywords.  I 
just started a thread 'Running beginner Pythons'.

-- 
Terry Jan Reedy


From breamoreboy at yahoo.co.uk  Sat Oct 10 22:11:55 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 10 Oct 2015 21:11:55 +0100
Subject: [Python-ideas] Proposal how to remove all occurrences of a
 value from a Python list
In-Reply-To: <56106147.8000704@btinternet.com>
References: <CAK=jXxkFvE44CSsU1OovSB7svZC4zMZuJB1_jpO-2DyFbOAmKg@mail.gmail.com>
 <mupira$tua$1@ger.gmane.org> <56106147.8000704@btinternet.com>
Message-ID: <mvbrf9$cqk$1@ger.gmane.org>

On 04/10/2015 00:14, Rob Cliffe wrote:
>
> Other things that might be useful (sorry, I have no use cases in mind):
>      a.remove(1, count)    # maximum number of removals, analogous to
> aString.replace(old, new, count)

Make the default count=1 to preserve the current behaviour and wouldn't 
everybody be happy?

>      a remove-like function that does not raise an error if the item is
> not present
>      a remove-like function that returns the mutated list (like
> sorted(), as opposed to list.sort() )
> It's not obvious to me how to design good API(s) to do some/all of this.
> Rob Cliffe

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From andre.roberge at gmail.com  Sun Oct 11 03:44:13 2015
From: andre.roberge at gmail.com (Andre Roberge)
Date: Sat, 10 Oct 2015 22:44:13 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mv9t8d$r0e$2@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
Message-ID: <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>

On Fri, Oct 9, 2015 at 11:30 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 10/9/2015 4:22 PM, Andre Roberge wrote:
>
>> Summary:  Visual environments based on using
>> simple functions without using variables have been found
>> to be useful to introduce programming concepts to young
>> learners.  While Python is an ideal language for beginners,
>> the Python construct for iterations, namely
>>
>> for var in range(n):
>>      # block
>>      # of
>>      # code
>>
>> does require a variable and is needlessly complicated for
>> beginners.  A simpler construct like:
>>
>> repeat n:    # n is a specific integer
>>      # block
>>      # of
>>      # code
>>
>> would be useful to have in such contexts.  Other keywords could
>> be chosen instead of "repeat" used above.
>>
>
> ...
>
> do n:
>>      # block
>>      # of
>>      # code indented like Python
>>
>> To do something equivalent in Python requires to use:
>>
>> for var in range(n):
>>      # block
>>      # of
>>      # code
>>
>
> ...
>
> An alternative would be to reuse the "for" keyword by
>> adding this new construct to the grammar:
>>
>> for n:
>>      # block
>>      # of
>>      # code
>>
>
>
> ---
>
> I think trying to persuade core developers to mutilate Python a bit, at
> couple of years in the future, is futile.


It is unfortunate, given the original CP4E goal, but not altogether
unexpected -- although, I do find the word "mutilate" to be a bit strong.



> And unneeded.


I respectfully disagree (however see below).
The reason is that I don't believe in students having to unlearn something
(like a special repeat syntax available only in a specific environment)
when they might have been under the impression that it was standard Python
syntax.  Furthermore, it might instill doubt in their mind about other
construct they have learn  (are they standard? ... or were they too
something only available in the specific environment?...)


>   Instead, translate, say 'repeat n:' into 'for _ in range(n):' with
> something like
>
> def robotrans(codetext):
>     out = []
>     for i, line in enumerate(codetext.splitlines()):
>         if line.startswith('repeat'):
>             <check rest of line and either append translation
>              or raise SyntaxError with proper details>
>         else:
>             out.append(line)
>     return '\n'.join(outlines)
>
> You could, for instance, patch IDLE two places (for shell and editor code)
> to run robotran on user code before compiling.  You could also add 'repeat'
> to the list of keywords for colorizing.  I would be willing to help
> (without any guarantee of code stability).
>

I don't use IDLE.  To make things as easy as possible for beginners, I have
moved away from desktop programs (RUR-PLE) which require a download, to a
purely web version, accessible via a single URL.    However, prompted by
your suggestion, I did implement a simplified syntax for iteration
(tentatively using "repeat").  If you click on the link below, it will
bring up a concrete example, ready to run.   (Note: the syntax used in the
example may not work in the future ... say, in a few days from now, after I
discuss with other users of my site about this new possible syntax.)  If I
do end up keeping something like this, I'll have to make sure it is flagged
very clearly as being non-standard, so that the students are properly
warned.

http://reeborg.ca/world_dev.html?proglang=python-en&world=%7B%0A%20%20%22robots%22%3A%20%5B%0A%20%20%20%20%7B%0A%20%20%20%20%20%20%22x%22%3A%201%2C%0A%20%20%20%20%20%20%22y%22%3A%201%2C%0A%20%20%20%20%20%20%22orientation%22%3A%200%2C%0A%20%20%20%20%20%20%22_prev_x%22%3A%201%2C%0A%20%20%20%20%20%20%22_prev_y%22%3A%201%2C%0A%20%20%20%20%20%20%22_prev_orientation%22%3A%200%2C%0A%20%20%20%20%20%20%22objects%22%3A%20%7B%7D%0A%20%20%20%20%7D%0A%20%20%5D%2C%0A%20%20%22walls%22%3A%20%7B%7D%2C%0A%20%20%22description%22%3A%20%22%22%2C%0A%20%20%22small_tiles%22%3A%20false%2C%0A%20%20%22rows%22%3A%2012%2C%0A%20%20%22cols%22%3A%2014%0A%7D&editor=from%20library%20import%20left%2C%20right%0A%0Arepeat%204%3A%0A%20%20%20%20move()%0A%20%20%20%20left()%0A%0Aright()%0A&library=left%20%3D%20turn_left%0A%20%20%20%20%0Adef%20right()%3A%0A%20%20%20%20repeat%203%3A%0A%20%20%20%20%20%20%20%20turn_left()%0A




> Guido recently approved on idle-dev that we rethink IDLE for beginners.  I
> have already thought about adding the necessary hooks so that education
> developers like you could define beginner Python dialects, such as you
> suggest here.  This would open a world of possibilities, such as 'move 3'
> versus 'move(3)', and foreign language keywords.  I just started a thread
> 'Running beginner Pythons'.


I'll have a look.  However, as I mentioned above, I'm not keen on
introducing what amounts to a potentially completely different language.
Python is a fantastic language for beginners (and advanced) programmers ...
I was just hoping to make it a tiny bit easier to learn for complete
beginners who are introduced to Python in visual environments (like the
turtle module or a Karel the Robot clone).

Andr? Roberge

>
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151010/f5c18609/attachment.html>

From ian.team.python at gmail.com  Sun Oct 11 04:55:58 2015
From: ian.team.python at gmail.com (Ian)
Date: Sun, 11 Oct 2015 13:55:58 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <-5740670168466924456@unknownmsgid>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <90392E0C-EFB5-4DEB-BB69-24B44FC72052@gmail.com>
 <5618312F.8010403@lucidity.plus.com> <-5740670168466924456@unknownmsgid>
Message-ID: <5619CFBE.3050700@gmail.com>

if anything is changed at all, then the

for <iterator>:
  # magic


seems to have the least impact and best addresses the original goal.


repeat n:  # saves 4 characters
     vs
for range(n):  # keeps syntax consistent

Which (consistency vs save 4 characters) would you favour for a beginner?
Not to mention that introducing a new keyword introduces many challenges.

>> Or, to make it look that little bit more "natural", spell it:
>>
>> for each in range(n):
>>   # magic
> Indeed -- while python's for loop can be (and often is) used to repeat
> something a certain number of times, what it really is is a way to
> apply a block of code to each item in an iterable.
>
> Making a slightly shorter/easier way to do something n times will just
> make it harder to "get" what for loops really are later on.
>
> And aside from using it as a teaching tool, does one want to loop
> without using the index all that often to get special syntax?
>
> The Karel-style simplified languages may be a great teaching tool --
> but that's not what Python is.
>
> -CHB
>
>
>
>> ... and then one day, move on to explain that "each" is actually something you can USE rather than just being a fixed part of the syntax - at which point it can be spelled better according to what it actually is.
>>
>> An alternative suggestion to remove the need for the variable (rather than a new keyword) would be something like allowing the "var in" part to be optioinal:
>>
>> for <iterator>:
>>   # magic
>>
>> ... but the problem with that is that pretty much the ONLY use would be for this sort of "iterate n times" construct or something where you're relying on the iterator to have side-effects (which is pretty nasty):
>>
>> foo = myiter()
>> for foo:
>>   bar(foo.getcurrentvalue())
>>
>> It seems like a lot of effort just to hide a bit of syntax for a very specific use-case when you can just temporarily make that syntax look a bit nicer for your audience.
>>
>> E.
>> _______________________________________________
>>


From tjreedy at udel.edu  Sun Oct 11 06:37:06 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 11 Oct 2015 00:37:06 -0400
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
Message-ID: <mvcp27$n3m$1@ger.gmane.org>

On 10/10/2015 9:44 PM, Andre Roberge wrote

> On Fri, Oct 9, 2015 at 11:30 PM, Terry Reedy
> <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:

> The reason is that I don't believe in students having to unlearn
> something (like a special repeat syntax available only in a specific
> environment) when they might have been under the impression that it was
> standard Python syntax.

To avoid mis-impressions, non-standard syntax might get a special syntax 
coloring, such as an orange 'warning color' background. Or it could be 
translated line-by-line on entry: enter 'for n' and the line is 
immediately replaced by 'for _ in range(n):'.  (I might like that for my 
own use ;-). Or if the beginner language is really not Python, don't 
call it Python.

I know I don't know how to teach beginning programming, especially to 
children, and I doubt anyone really knows the *best* way.  So I am 
interested in facilitating experiments.

> Furthermore, it might instill doubt in their
> mind about other construct they have learn  (are they standard? ... or
> were they too something only available in the specific environment?...)

I think I understood at a pretty early age that training wheels were a 
special part of a beginner bike for little kids, only present for learning.

>    Python is a fantastic language for beginners (and advanced)
> programmers ... I was just hoping to make it a tiny bit easier to learn
> for complete beginners who are introduced to Python in visual
> environments (like the turtle module or a Karel the Robot clone).

Your are asking for a special case of a special case of something 
already present. In my view, it is like a training wheel - very useful 
when needed, but not so thereafter.  Since it is only needed for a short 
time for young beginners, it only need be present, in my view, in the 
environments used by such beginners.

-- 
Terry Jan Reedy


From stephen at xemacs.org  Sun Oct 11 09:48:12 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 11 Oct 2015 16:48:12 +0900
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
Message-ID: <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>

Andre Roberge writes:

 > >> While Python is an ideal language for beginners, the Python
 > >> construct for iterations, namely
 > >>
 > >> for var in range(n):
 > >>      # suite
 > >>
 > >> does require a variable and is needlessly complicated for
 > >> beginners.  A simpler construct like:
 > >>
 > >> repeat n:    # n is a specific integer
 > >>      # suite
 > >>
 > >> would be useful to have in such contexts.  Other keywords could
 > >> be chosen instead of "repeat" used above.

I have no objection to such a language, but it ain't Python.  It's an
important principle of Python that "There's one- and preferably only
one -obvious way to do it."  While the parenthetical "only one" is
violated on occasion, it's still considered important.

 > >> for var in range(n):
 > >>      # suite

I just don't have a problem with this.

It seems to me that on the occasions when I've taught programming[2],
I've always had the most success when the student has a concrete
problem in mind (sometimes they need to write programs to complete
another project, but also when they take programming as a required
course).  Take the traditional make a square in turtle graphics:

    right = -90
    length = 10
    def make_square (turtle):
        for side in ('top', 'right', 'bottom', 'left'):
            turtle.forward(length)
            turtle.turn(right)

Do you really have students writing code like

    for _ in range(20):
        print('yes')

where the repetition is exact?

 > It is unfortunate, given the original CP4E goal, but not altogether
 > unexpected -- although, I do find the word "mutilate" to be a bit
 > strong.

The goal called "CP4E" as I understood it was to give the *full* power
of "computer programming" to "everyone", not to just provide a taste.
The *power* of computer programming comes from small but expressive
vocabularies, it seems to me.

 > The reason is that I don't believe in students having to unlearn
 > something (like a special repeat syntax available only in a
 > specific environment) when they might have been under the
 > impression that it was standard Python syntax.

So call the new language "Childrens-Python"[1] if you like.<wink/>  Or
"Woma" (which is a bit bigger than Children's Python) if that's too
close to "Python".

 > Furthermore, it might instill doubt in their mind about other
 > construct they have learn (are they standard? ... or were they too
 > something only available in the specific environment?...)

So what?  If they're going to do computer programming after they leave
your class, they're surely going to encounter "specific environments".
So many times I've typed an example into the interpreter only to
discover that the author didn't explicitly import a function or
identifier from a module, but it looks like it could be a builtin I
hadn't used before.

 > I'll have a look.  However, as I mentioned above, I'm not keen on
 > introducing what amounts to a potentially completely different
 > language.

It doesn't look to me like that's what Terry is talking about, though.
I suppose that as usual there will be a "consenting adults" convention
that "you could do that with this hook, but please, *not* in front of
the children!"  Rather, for cases where you don't need the full power
of Python's expression syntax, you could eliminate parentheses from
function calls and that kind of thing.

 > Python is a fantastic language for beginners (and advanced)
 > programmers ...  I was just hoping to make it a tiny bit easier to
 > learn for complete beginners who are introduced to Python in visual
 > environments (like the turtle module or a Karel the Robot clone).

At the cost of requiring every Python programmer to beware of a new
and infrequently syntax.


Footnotes: 
[1]  http://www.snakeranch.com.au/snake-profiles/childrens-python/

[2]  Always at the college level or above, though.



From luciano at ramalho.org  Sun Oct 11 23:47:10 2015
From: luciano at ramalho.org (Luciano Ramalho)
Date: Sun, 11 Oct 2015 18:47:10 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>

I've felt the same need Andre described, and so far I have not seen a
convincing argument against his proposal to accept this syntax:

for n:  # repeat block n times
     fd(100)
     lt(360/n)

Where n is any expression that evaluates to an integer.

The goal is to repeat some operation N times without creating a
spurious variable. I prefer this simpler alternative than Ian's later
proposal where n must be an iterable, involving a lot of machinery
just to produce a value to be discarded.

While we are at it, we could borrow Go's "forever" loop:

for:  # repeat forever
    spam()

Although I am quite happy with `while True:` for that purpose.

Cheers,

Luciano




On Sun, Oct 11, 2015 at 4:48 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Andre Roberge writes:
>
>  > >> While Python is an ideal language for beginners, the Python
>  > >> construct for iterations, namely
>  > >>
>  > >> for var in range(n):
>  > >>      # suite
>  > >>
>  > >> does require a variable and is needlessly complicated for
>  > >> beginners.  A simpler construct like:
>  > >>
>  > >> repeat n:    # n is a specific integer
>  > >>      # suite
>  > >>
>  > >> would be useful to have in such contexts.  Other keywords could
>  > >> be chosen instead of "repeat" used above.
>
> I have no objection to such a language, but it ain't Python.  It's an
> important principle of Python that "There's one- and preferably only
> one -obvious way to do it."  While the parenthetical "only one" is
> violated on occasion, it's still considered important.
>
>  > >> for var in range(n):
>  > >>      # suite
>
> I just don't have a problem with this.
>
> It seems to me that on the occasions when I've taught programming[2],
> I've always had the most success when the student has a concrete
> problem in mind (sometimes they need to write programs to complete
> another project, but also when they take programming as a required
> course).  Take the traditional make a square in turtle graphics:
>
>     right = -90
>     length = 10
>     def make_square (turtle):
>         for side in ('top', 'right', 'bottom', 'left'):
>             turtle.forward(length)
>             turtle.turn(right)
>
> Do you really have students writing code like
>
>     for _ in range(20):
>         print('yes')
>
> where the repetition is exact?
>
>  > It is unfortunate, given the original CP4E goal, but not altogether
>  > unexpected -- although, I do find the word "mutilate" to be a bit
>  > strong.
>
> The goal called "CP4E" as I understood it was to give the *full* power
> of "computer programming" to "everyone", not to just provide a taste.
> The *power* of computer programming comes from small but expressive
> vocabularies, it seems to me.
>
>  > The reason is that I don't believe in students having to unlearn
>  > something (like a special repeat syntax available only in a
>  > specific environment) when they might have been under the
>  > impression that it was standard Python syntax.
>
> So call the new language "Childrens-Python"[1] if you like.<wink/>  Or
> "Woma" (which is a bit bigger than Children's Python) if that's too
> close to "Python".
>
>  > Furthermore, it might instill doubt in their mind about other
>  > construct they have learn (are they standard? ... or were they too
>  > something only available in the specific environment?...)
>
> So what?  If they're going to do computer programming after they leave
> your class, they're surely going to encounter "specific environments".
> So many times I've typed an example into the interpreter only to
> discover that the author didn't explicitly import a function or
> identifier from a module, but it looks like it could be a builtin I
> hadn't used before.
>
>  > I'll have a look.  However, as I mentioned above, I'm not keen on
>  > introducing what amounts to a potentially completely different
>  > language.
>
> It doesn't look to me like that's what Terry is talking about, though.
> I suppose that as usual there will be a "consenting adults" convention
> that "you could do that with this hook, but please, *not* in front of
> the children!"  Rather, for cases where you don't need the full power
> of Python's expression syntax, you could eliminate parentheses from
> function calls and that kind of thing.
>
>  > Python is a fantastic language for beginners (and advanced)
>  > programmers ...  I was just hoping to make it a tiny bit easier to
>  > learn for complete beginners who are introduced to Python in visual
>  > environments (like the turtle module or a Karel the Robot clone).
>
> At the cost of requiring every Python programmer to beware of a new
> and infrequently syntax.
>
>
> Footnotes:
> [1]  http://www.snakeranch.com.au/snake-profiles/childrens-python/
>
> [2]  Always at the college level or above, though.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
|     http://shop.oreilly.com/product/0636920032519.do
|  Professor em: http://python.pro.br
|  Twitter: @ramalhoorg

From ben+python at benfinney.id.au  Mon Oct 12 00:03:27 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 12 Oct 2015 09:03:27 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
Message-ID: <858u79ukpc.fsf@benfinney.id.au>

Luciano Ramalho <luciano at ramalho.org>
writes:

> I've felt the same need Andre described, and so far I have not seen a
> convincing argument against his proposal to accept this syntax:
>
> for n:  # repeat block n times
>      fd(100)
>      lt(360/n)
>
> Where n is any expression that evaluates to an integer.

Huh? If ?n? is an expression that evaluates to one integer, then why are
you also dividing by that one integer ?n? every time through?


I don't claim it will convince you, but I find compelling the fact that
this::

    for n in range(FOO):
         fd(100)
         lt(360/n)

already works fine and is quite clear.

-- 
 \         ?Science is a way of trying not to fool yourself. The first |
  `\     principle is that you must not fool yourself, and you are the |
_o__)               easiest person to fool.? ?Richard P. Feynman, 1964 |
Ben Finney


From ben+python at benfinney.id.au  Mon Oct 12 00:08:13 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 12 Oct 2015 09:08:13 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <858u79ukpc.fsf@benfinney.id.au>
Message-ID: <854mhxukhe.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> I don't claim it will convince you, but I find compelling the fact that
> this::
>
>     for n in range(FOO):
>          fd(100)
>          lt(360/n)
>
> already works fine and is quite clear.

And, of course, in my haste I divided by zero :-)

-- 
 \        ?We cannot solve our problems with the same thinking we used |
  `\                           when we created them.? ?Albert Einstein |
_o__)                                                                  |
Ben Finney


From luciano at ramalho.org  Mon Oct 12 00:13:27 2015
From: luciano at ramalho.org (Luciano Ramalho)
Date: Sun, 11 Oct 2015 19:13:27 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <858u79ukpc.fsf@benfinney.id.au>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <858u79ukpc.fsf@benfinney.id.au>
Message-ID: <CALxg4FXdQFR+GB16aY3bCVDEG1aHr9ZzgBpuPAe38=1iSLf8Og@mail.gmail.com>

On Sun, Oct 11, 2015 at 7:03 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Luciano Ramalho <luciano at ramalho.org>
> writes:
>> for n:  # repeat block n times
>>      fd(100)
>>      lt(360/n)
>>
>> Where n is any expression that evaluates to an integer.
>
> Huh? If ?n? is an expression that evaluates to one integer, then why are
> you also dividing by that one integer ?n? every time through?

That's the classic formula for drawing an N-sided polygon in Logo. N
can be any positive integer. Maybe this is clearer:

for 3:  # draw a triangle
    fd(100)
    lt(120)

> I don't claim it will convince you, but I find compelling the fact that
> this::
>
>     for n in range(FOO):
>          fd(100)
>          lt(360/n)
>
> already works fine and is quite clear.

It is quite clear for us. When teaching the first steps of programming
to kids with the Turtle module, `for x in range(n):` leaves us in the
unfortunate position of having to say: "we won't use that x for
anything, and I can't explain what range this until much later..."

Cheers,

Luciano


>
> --
>  \         ?Science is a way of trying not to fool yourself. The first |
>   `\     principle is that you must not fool yourself, and you are the |
> _o__)               easiest person to fool.? ?Richard P. Feynman, 1964 |
> Ben Finney
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
|     http://shop.oreilly.com/product/0636920032519.do
|  Professor em: http://python.pro.br
|  Twitter: @ramalhoorg

From ben+python at benfinney.id.au  Mon Oct 12 00:28:22 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 12 Oct 2015 09:28:22 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <858u79ukpc.fsf@benfinney.id.au>
 <CALxg4FXdQFR+GB16aY3bCVDEG1aHr9ZzgBpuPAe38=1iSLf8Og@mail.gmail.com>
Message-ID: <85zizpt4zd.fsf@benfinney.id.au>

Luciano Ramalho <luciano at ramalho.org>
writes:

> That's the classic formula for drawing an N-sided polygon in Logo. N
> can be any positive integer. Maybe this is clearer:
>
> for 3:  # draw a triangle
>     fd(100)
>     lt(120)

Thank you, yes it's much clearer because you're not doing exactly the
same division multiple times.


So, to follow your clarification, this::

    for __ in range(3):
        forward(100)
        turn_left(120)

already works and is quite clear (though, again, I don't claim it will
convince you).

> It is quite clear for us. When teaching the first steps of programming
> to kids with the Turtle module, `for x in range(n):` leaves us in the
> unfortunate position of having to say: "we won't use that x for
> anything, and I can't explain what range this until much later..."

Yes, the fact you weren't using it confused me too :-)

Which is why we advocate a name, ?__?, that stands out and is used only
for dummy assignment.

-- 
 \         ?How wonderful that we have met with a paradox. Now we have |
  `\                        some hope of making progress.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From guido at python.org  Mon Oct 12 00:59:00 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2015 15:59:00 -0700
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <85zizpt4zd.fsf@benfinney.id.au>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <858u79ukpc.fsf@benfinney.id.au>
 <CALxg4FXdQFR+GB16aY3bCVDEG1aHr9ZzgBpuPAe38=1iSLf8Og@mail.gmail.com>
 <85zizpt4zd.fsf@benfinney.id.au>
Message-ID: <CAP7+vJ+uS2mXPtoGSWng4sd+rk0iJNg+FoNnmwJ15Wg8mQJCRw@mail.gmail.com>

Argh!

Here's why I don't like adding special syntax to do something N times. I
didn't leave this out by accident 25 years ago. Python takes a stance
against a common approach to beginners' programming education focused on
counting and indexing arrays of data, rooted in machine language and
FORTRAN.

Standard problems and examples like "print 'hello world' 10 times" or "draw
an N-sided polygon" are designed to teach students those counting and
indexing skills. It's backwards to request that the language support
counting better so those examples will look simpler.

Python has so much richer data structures. Create problems to teach those!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151011/cc71c033/attachment.html>

From luciano at ramalho.org  Mon Oct 12 01:22:28 2015
From: luciano at ramalho.org (Luciano Ramalho)
Date: Sun, 11 Oct 2015 20:22:28 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAP7+vJ+uS2mXPtoGSWng4sd+rk0iJNg+FoNnmwJ15Wg8mQJCRw@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <858u79ukpc.fsf@benfinney.id.au>
 <CALxg4FXdQFR+GB16aY3bCVDEG1aHr9ZzgBpuPAe38=1iSLf8Og@mail.gmail.com>
 <85zizpt4zd.fsf@benfinney.id.au>
 <CAP7+vJ+uS2mXPtoGSWng4sd+rk0iJNg+FoNnmwJ15Wg8mQJCRw@mail.gmail.com>
Message-ID: <CALxg4FUZmpzZFBqhWdzaLcUkhPPVipZ8v58QUUDdvQFKt8sQ7A@mail.gmail.com>

On Sun, Oct 11, 2015 at 7:59 PM, Guido van Rossum <guido at python.org> wrote:
> Standard problems and examples like "print 'hello world' 10 times" or "draw
> an N-sided polygon" are designed to teach students those counting and
> indexing skills. It's backwards to request that the language support
> counting better so those examples will look simpler.

I agree that counting and indexing dominate many stupid introductory
programming exercises. And I praise the design of Python's `for`
command every time I teach the language or speak about it.

However, in Logo, using a command like the one below, there is
absolutely no indexing or counting in sight.

REPEAT 3 [FD 100 RT 120]

That's why we were advocating for a similar construct in Python.

But obviously you're not going to entertain the idea, so I'll drop it.

Cheers,

Luciano



>
> Python has so much richer data structures. Create problems to teach those!
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
|     http://shop.oreilly.com/product/0636920032519.do
|  Professor em: http://python.pro.br
|  Twitter: @ramalhoorg

From emile at fenx.com  Mon Oct 12 03:59:13 2015
From: emile at fenx.com (Emile van Sebille)
Date: Sun, 11 Oct 2015 18:59:13 -0700
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
Message-ID: <mvf45n$s2s$1@ger.gmane.org>

On 10/11/2015 2:47 PM, Luciano Ramalho wrote:
> I've felt the same need Andre described, and so far I have not seen a
> convincing argument against his proposal to accept this syntax:
>
> for n:  # repeat block n times
>       fd(100)
>       lt(360/n)
>
> Where n is any expression that evaluates to an integer.

Umm... https://www.python.org/dev/peps/pep-0284/

Emile


From guido at python.org  Mon Oct 12 04:12:03 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2015 19:12:03 -0700
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mvf45n$s2s$1@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
Message-ID: <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>

On Sun, Oct 11, 2015 at 6:59 PM, Emile van Sebille <emile at fenx.com> wrote:

> On 10/11/2015 2:47 PM, Luciano Ramalho wrote:
>
>> I've felt the same need Andre described, and so far I have not seen a
>> convincing argument against his proposal to accept this syntax:
>>
>> for n:  # repeat block n times
>>       fd(100)
>>       lt(360/n)
>>
>> Where n is any expression that evaluates to an integer.
>>
>
> Umm... https://www.python.org/dev/peps/pep-0284/
>

Heh. "The whole point (15 years ago) of range() was to *avoid* needing
syntax to specify a loop over numbers." I guess I'm nothing if not
consistent. :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151011/b501847e/attachment.html>

From luciano at ramalho.org  Mon Oct 12 05:03:48 2015
From: luciano at ramalho.org (Luciano Ramalho)
Date: Mon, 12 Oct 2015 00:03:48 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
Message-ID: <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>

On Sun, Oct 11, 2015 at 11:12 PM, Guido van Rossum <guido at python.org> wrote:
> Heh. "The whole point (15 years ago) of range() was to *avoid* needing
> syntax to specify a loop over numbers." I guess I'm nothing if not
> consistent. :-)

Andre and I are not arguing for a special syntax to loop over numbers.
We are arguing for special syntax to repeat a block a number of times.
No indexing at all, and no variables need to be involved.

for 3:
     fd(100)
     lt(120)


Best,

Luciano

-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
|     http://shop.oreilly.com/product/0636920032519.do
|  Professor em: http://python.pro.br
|  Twitter: @ramalhoorg

From guido at python.org  Mon Oct 12 06:43:52 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2015 21:43:52 -0700
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
Message-ID: <CAP7+vJLpAD-AOLxUXG8TdirZfnHDAfaS9NnkTVc=6VM3YMLyNg@mail.gmail.com>

Well, that's even more limited.

On Sunday, October 11, 2015, Luciano Ramalho <luciano at ramalho.org> wrote:

> On Sun, Oct 11, 2015 at 11:12 PM, Guido van Rossum <guido at python.org
> <javascript:;>> wrote:
> > Heh. "The whole point (15 years ago) of range() was to *avoid* needing
> > syntax to specify a loop over numbers." I guess I'm nothing if not
> > consistent. :-)
>
> Andre and I are not arguing for a special syntax to loop over numbers.
> We are arguing for special syntax to repeat a block a number of times.
> No indexing at all, and no variables need to be involved.
>
> for 3:
>      fd(100)
>      lt(120)
>
>
> Best,
>
> Luciano
>
> --
> Luciano Ramalho
> |  Author of Fluent Python (O'Reilly, 2015)
> |     http://shop.oreilly.com/product/0636920032519.do
> |  Professor em: http://python.pro.br
> |  Twitter: @ramalhoorg
>


-- 
--Guido (mobile)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151011/cf65e280/attachment-0001.html>

From luciano at ramalho.org  Mon Oct 12 06:58:45 2015
From: luciano at ramalho.org (Luciano Ramalho)
Date: Mon, 12 Oct 2015 01:58:45 -0300
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CAP7+vJLpAD-AOLxUXG8TdirZfnHDAfaS9NnkTVc=6VM3YMLyNg@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <CAP7+vJLpAD-AOLxUXG8TdirZfnHDAfaS9NnkTVc=6VM3YMLyNg@mail.gmail.com>
Message-ID: <CALxg4FVanEg75YgYhzr7KWNZMyG6Jb8aH3WPTD9Ec9oS_hF_PQ@mail.gmail.com>

On Mon, Oct 12, 2015 at 1:43 AM, Guido van Rossum <guido at python.org> wrote:
> Well, that's even more limited.

Yep. That's the whole point.

>
>
> On Sunday, October 11, 2015, Luciano Ramalho <luciano at ramalho.org> wrote:
>>
>> On Sun, Oct 11, 2015 at 11:12 PM, Guido van Rossum <guido at python.org>
>> wrote:
>> > Heh. "The whole point (15 years ago) of range() was to *avoid* needing
>> > syntax to specify a loop over numbers." I guess I'm nothing if not
>> > consistent. :-)
>>
>> Andre and I are not arguing for a special syntax to loop over numbers.
>> We are arguing for special syntax to repeat a block a number of times.
>> No indexing at all, and no variables need to be involved.
>>
>> for 3:
>>      fd(100)
>>      lt(120)
>>
>>
>> Best,
>>
>> Luciano
>>
>> --
>> Luciano Ramalho
>> |  Author of Fluent Python (O'Reilly, 2015)
>> |     http://shop.oreilly.com/product/0636920032519.do
>> |  Professor em: http://python.pro.br
>> |  Twitter: @ramalhoorg
>
>
>
> --
> --Guido (mobile)



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
|     http://shop.oreilly.com/product/0636920032519.do
|  Professor em: http://python.pro.br
|  Twitter: @ramalhoorg

From guido at python.org  Mon Oct 12 07:05:18 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Oct 2015 22:05:18 -0700
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CALxg4FVanEg75YgYhzr7KWNZMyG6Jb8aH3WPTD9Ec9oS_hF_PQ@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <CAP7+vJLpAD-AOLxUXG8TdirZfnHDAfaS9NnkTVc=6VM3YMLyNg@mail.gmail.com>
 <CALxg4FVanEg75YgYhzr7KWNZMyG6Jb8aH3WPTD9Ec9oS_hF_PQ@mail.gmail.com>
Message-ID: <CAP7+vJL56cpkABkwMy8rm0cg+xh9EP8GAk5xe028T6zyMzY8Dw@mail.gmail.com>

It's not very compelling. So let's drop it.

On Sunday, October 11, 2015, Luciano Ramalho <luciano at ramalho.org> wrote:

> On Mon, Oct 12, 2015 at 1:43 AM, Guido van Rossum <guido at python.org
> <javascript:;>> wrote:
> > Well, that's even more limited.
>
> Yep. That's the whole point.
>
> >
> >
> > On Sunday, October 11, 2015, Luciano Ramalho <luciano at ramalho.org
> <javascript:;>> wrote:
> >>
> >> On Sun, Oct 11, 2015 at 11:12 PM, Guido van Rossum <guido at python.org
> <javascript:;>>
> >> wrote:
> >> > Heh. "The whole point (15 years ago) of range() was to *avoid* needing
> >> > syntax to specify a loop over numbers." I guess I'm nothing if not
> >> > consistent. :-)
> >>
> >> Andre and I are not arguing for a special syntax to loop over numbers.
> >> We are arguing for special syntax to repeat a block a number of times.
> >> No indexing at all, and no variables need to be involved.
> >>
> >> for 3:
> >>      fd(100)
> >>      lt(120)
> >>
> >>
> >> Best,
> >>
> >> Luciano
> >>
> >> --
> >> Luciano Ramalho
> >> |  Author of Fluent Python (O'Reilly, 2015)
> >> |     http://shop.oreilly.com/product/0636920032519.do
> >> |  Professor em: http://python.pro.br
> >> |  Twitter: @ramalhoorg
> >
> >
> >
> > --
> > --Guido (mobile)
>
>
>
> --
> Luciano Ramalho
> |  Author of Fluent Python (O'Reilly, 2015)
> |     http://shop.oreilly.com/product/0636920032519.do
> |  Professor em: http://python.pro.br
> |  Twitter: @ramalhoorg
>


-- 
--Guido (mobile)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151011/5192d2fe/attachment.html>

From rustompmody at gmail.com  Mon Oct 12 07:37:43 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 12 Oct 2015 11:07:43 +0530
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CALxg4FVanEg75YgYhzr7KWNZMyG6Jb8aH3WPTD9Ec9oS_hF_PQ@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <CAP7+vJLpAD-AOLxUXG8TdirZfnHDAfaS9NnkTVc=6VM3YMLyNg@mail.gmail.com>
 <CALxg4FVanEg75YgYhzr7KWNZMyG6Jb8aH3WPTD9Ec9oS_hF_PQ@mail.gmail.com>
Message-ID: <CAJ+TeocDp-wr4TDJXmiG2G3pHpnVs8M0o+qHBti0xFrWsSempg@mail.gmail.com>

On Mon, Oct 12, 2015 at 10:28 AM, Luciano Ramalho <luciano at ramalho.org>
wrote:

> On Mon, Oct 12, 2015 at 1:43 AM, Guido van Rossum <guido at python.org>
> wrote:
> > Well, that's even more limited.
>
> Yep. That's the whole point.
>
>

In discussions like this there are 3 ? at least ? interlocking parties with
their conflicting interests.

   1. Professional programmers
   2. Language Designers/implementers
   3. Programming/CS teachers

It is unlikely to find people equal to all these but at least let us be
clear that these 3 exist!

[For those who find this too long, tl;dr at bottom ]

To see that these viewpoints are inherently conflicting consider that a
programmer may wish that his language precludes errors ? a natural wish.
Translated to the designer camp this means an automatic detection of errors
? also known as the Halting Problem. Which is to show that the programmer's
and the language-designer's needs are not just conflicting, they are
inherently conflicting

And so, in like manner the programming-teachers' needs are in inherent
conflict with the needs of professional programmers and language designers.

3-4 decades ago this was widely understood and some sort of compromise was
worked out: Beginners studied Pascal (or Basic); professionals lived inside
Fortran, Cobol, PL-1 etc.

With the coming of C, people began to imagine that C was sufficiently close
to Pascal that a one-fit-all solution was possible. The result has
been terribly
detrimental
<http://blog.languager.org/2013/02/c-in-education-and-software-engineering.html>
to CS-edcuation

The increasing widespreadness of languages like Python and Haskell may seem
to improve this situation. IMHO it only worsens it: Sure Python is better
to teach than C++ but its not better enough that its suitable as a
teaching-language. [And as Haskell gallops towards Real-World
<http://book.realworldhaskell.org/> status it also gallops towards
C++-dom].

Specifically coming to the suggestion at hand: Pascal got one thing right
that all its successors, starting C, got terribly wrong: the need to
separate and *do equal justice to*
Platonic world Mundane World Values Effects Expressions Statements functions
procedures

More about this 50 year-old error and a beginning gleam of light:  a  ACM
CS Curriculum 2013
<http://blog.languager.org/2015/06/functional-programming-moving-target.html>
and its prequel Timeline
<http://blog.languager.org/2015/04/cs-history-1.html>

The error of imagining imperative programming is easy to learn comes from
people not seeing that imperative languages *as they exist* contain a
subset functional language which is ineluctable. Also known as the rhs of
assignment statements ? with the '=' being the visible indicator of
the infamous
von Neumann bottleneck
<https://www.cs.cmu.edu/%7Ecrary/819-f09/Backus78.pdf>.

The functional programming folks seeing this error try to expand that
expression world to cover all of programming needs. [Whether the results of
these good-intentions eg Haskell are more teachable is another matter!]

The alternative (eg Logo and I am guessing Luciano and Andre ) that for the
'atoms' of the statement world, instead of having assignment statements to
have turtle-left/right etc. This renders the language useless for general
purpose programming but can be a neat intro to algorithmic thinking


tl;dr

   1. Language designer/implementers may wish to ask themselves whether
   having a family of easily learnable but slightly inconsistent languages
   helps or hinders their language.
   2. Teachers may wish to re-evaluate their fixed-cost/variable-cost
   (equivalently batch/interactive cost) outlay in the languages they teach
   and check whether implementing a cut-down subset of their chosen language
   doesn't speed up their students and themselves
   3. General programmers need to remember ? not only were they noobs at
   some stage, there is probably something they are learning right now to
   which noob-ness applies even to them.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151012/a2e209ff/attachment-0001.html>

From tjreedy at udel.edu  Mon Oct 12 08:10:24 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 12 Oct 2015 02:10:24 -0400
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
Message-ID: <mvfish$9e9$1@ger.gmane.org>

On 10/11/2015 5:47 PM, Luciano Ramalho wrote:
> I've felt the same need Andre described, and so far I have not seen a
> convincing argument against his proposal to accept this syntax:

Your logic is backwards.  The default answer to 'add this' is 'No'.  The 
burden of persuasion is on you.  The people who have to be persuaded, by 
their own standards of goodness for Python, are Guido and core developers

>
> for n:  # repeat block n times
>       fd(100)
>       lt(360/n)
>
> Where n is any expression that evaluates to an integer.

Andre wanted, I believe, 'for <literal number>:' to avoid naming 
numbers, or at least variable quantities.  But I am 'not convinced'. 
The above should be

turn = 360/n
for n:
   fd(100)
   lt(turn)

Anyway, very young kids get the concept of 'variable quantity': number 
of days left to Christmas, number of cookies handed out, etc.

> The goal is to repeat some operation N times without creating a
> spurious variable.

The fraction of for loops using range(n) is small, especially after 
enumerate was introduced, and especially if n is a constant.  The 
fraction of such loops where the iteration count can never have any use 
is also small, because if the code has a bug, you will likely want the 
iteration count.  Adding 'print(i, a, k)' to a loop, where a and k 
change with each loop is, for me, a very common debugging tool.

Even without bugs, I believe that displaying iteration counts can help 
learning and program understanding.  Right now, one could step through

n = 17
for i in range(n):  # i = number of chord already drawn
   fd(100)
   lt(360/n)

with IDLE's debugger and see the step number increase just before a 
chord is drawn.  Or one might want to add 'label(i)' at the top, where 
label(i) puts a small circled number near the current position. The more 
I think about this, the less I am convinced that the fraction of truly, 
permanently spurious loop counts is enough to support a new syntax.

-- 
Terry Jan Reedy


From rustompmody at gmail.com  Mon Oct 12 10:01:15 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 12 Oct 2015 01:01:15 -0700 (PDT)
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mvfish$9e9$1@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvfish$9e9$1@ger.gmane.org>
Message-ID: <2f77b95c-ffd1-460d-bd72-9991068780ca@googlegroups.com>

On Monday, October 12, 2015 at 11:41:06 AM UTC+5:30, Terry Reedy wrote:
>
> On 10/11/2015 5:47 PM, Luciano Ramalho wrote: 
> > I've felt the same need Andre described, and so far I have not seen a 
> > convincing argument against his proposal to accept this syntax: 
>
> Your logic is backwards.  The default answer to 'add this' is 'No'.  The 
> burden of persuasion is on you.  The people who have to be persuaded, by 
> their own standards of goodness for Python, are Guido and core developers 
>

Just to clarify my earlier post:  I dont regard this suggestion as a 
serious contender for python.

I would however like it that if someone forked and added this to (a 
probably heavily cut-down) python, that fork be regarded as a 
friendly-by-default fork
In my personal estimate: of such talk 1% will go from being vaporware to 
something others can actually try out
Of that 1%, 0.1% can become a serious competitor for (C)Python 
 

>
> > 
> > for n:  # repeat block n times 
> >       fd(100) 
> >       lt(360/n) 
> > 
> > Where n is any expression that evaluates to an integer. 
>
> Andre wanted, I believe, 'for <literal number>:' to avoid naming 
> numbers, or at least variable quantities.  But I am 'not convinced'. 
> The above should be 
>
> turn = 360/n 
> for n: 
>    fd(100) 
>    lt(turn) 
>
> Anyway, very young kids get the concept of 'variable quantity': number 
> of days left to Christmas, number of cookies handed out, etc. 
>
>
I just attended some lectures in which a gifted teacher used Jupyter 
(ipython) to give abstruse linear algebra intuitions to kids.
So yes a good teacher can convey variables to kids.

Is it a good idea to do so?

I remember as a 10-11 year old having a hard time going from arithmetic to 
algebra.

Current day imperative languages makes this (in my experience inherent) 
hardness harder on 2 more counts:
- the variable is time-varying
- the assignment is denoted with '=' which is a confusing synonym/homonym 
to the math equality

And so the teacher trying to disentangle all this confusingness is to be 
supported (in my book)

Its another matter that python is not a 'for-kids' language so it need not 
change to incorporate this.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151012/7be30e94/attachment.html>

From ben+python at benfinney.id.au  Tue Oct 13 00:21:31 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 13 Oct 2015 09:21:31 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
Message-ID: <85mvvnu3ro.fsf@benfinney.id.au>

Luciano Ramalho <luciano at ramalho.org>
writes:

> Andre and I are not arguing for a special syntax to loop over numbers.
> We are arguing for special syntax to repeat a block a number of times.

That's been clear, at least to my reading of your messages.

What you haven't argued is any extraordinary utility of this new syntax,
sufficient to overcome the high barrier that is rightly imposed on new
syntax.

> for 3:
>      fd(100)
>      lt(120)

Since this is quite clearly expressed in existing Python syntax with::

    for __ in range(3):
        forward(100)
        turn_left(120)

the single use case doesn't IMO argue for benefit sufficient to overcome
the enormous burden of new syntax.

-- 
 \         ?I'm not a bad guy! I work hard, and I love my kids. So why |
  `\      should I spend half my Sunday hearing about how I'm going to |
_o__)                                            Hell?? ?Homer Simpson |
Ben Finney


From srkunze at mail.de  Tue Oct 13 19:59:55 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 13 Oct 2015 19:59:55 +0200
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <85mvvnu3ro.fsf@benfinney.id.au>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au>
Message-ID: <561D469B.9020305@mail.de>

On 13.10.2015 00:21, Ben Finney wrote:
> Since this is quite clearly expressed in existing Python syntax with::
>
>      for __ in range(3):
>          forward(100)
>          turn_left(120)
>
> the single use case doesn't IMO argue for benefit sufficient to overcome
> the enormous burden of new syntax.

I still like instead of the cryptic __ or _

for each in range(3):
     dosomething

To me, that reads like a normal English statement.

Best,
Sven

From ben+python at benfinney.id.au  Tue Oct 13 21:49:39 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 14 Oct 2015 06:49:39 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
Message-ID: <85eggytup8.fsf@benfinney.id.au>

"Sven R. Kunze" <srkunze at mail.de> writes:

> On 13.10.2015 00:21, Ben Finney wrote:
> > Since this is quite clearly expressed in existing Python syntax with::
> >
> >      for __ in range(3):
> >          forward(100)
> >          turn_left(120)
> >
> > the single use case doesn't IMO argue for benefit sufficient to overcome
> > the enormous burden of new syntax.
>
> I still like instead of the cryptic __ or _
>
> for each in range(3):
>     dosomething
>
> To me, that reads like a normal English statement.

What you cite as an advantage is exactly why I'd advise against it. Your
example would lead me to ask ?Is this a mistake? ?each? is a normal
name, why isn't it being used??

Whereas ?__? is not a normal name, and like other unusual names in
Python should lead the reader to say ?Okay, obviously the person writing
this had something odd in mind. I'd better find out what.? That is more
likely to teach about the ?__? convention, than to be seen as a mistake
by experienced programmers.

-- 
 \        ?You don't change the world by placidly finding your bliss ? |
  `\        you do it by focusing your discontent in productive ways.? |
_o__)                                       ?Paul Z. Myers, 2011-08-31 |
Ben Finney


From srkunze at mail.de  Tue Oct 13 22:20:55 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 13 Oct 2015 22:20:55 +0200
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <85eggytup8.fsf@benfinney.id.au>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <85eggytup8.fsf@benfinney.id.au>
Message-ID: <561D67A7.5000406@mail.de>

On 13.10.2015 21:49, Ben Finney wrote:
> What you cite as an advantage is exactly why I'd advise against it.

Feel free to do so as do I.

> Your example would lead me to ask ?Is this a mistake? ?each? is a normal
> name, why isn't it being used??

You as an experienced Python developer.

That thread is about students learning programming. Not learning Python.

> Whereas ?__? is not a normal name, and like other unusual names in
> Python should lead the reader to say ?Okay, obviously the person writing
> this had something odd in mind. I'd better find out what.? That is more
> likely to teach about the ?__? convention, than to be seen as a mistake
> by experienced programmers.

Python convention necessary when the student need to write real Python.

Best,
Sven

From breamoreboy at yahoo.co.uk  Tue Oct 13 22:21:12 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 13 Oct 2015 21:21:12 +0100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <561D469B.9020305@mail.de>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
Message-ID: <mvjp4n$irg$1@ger.gmane.org>

On 13/10/2015 18:59, Sven R. Kunze wrote:
> On 13.10.2015 00:21, Ben Finney wrote:
>> Since this is quite clearly expressed in existing Python syntax with::
>>
>>      for __ in range(3):
>>          forward(100)
>>          turn_left(120)
>>
>> the single use case doesn't IMO argue for benefit sufficient to overcome
>> the enormous burden of new syntax.
>
> I still like instead of the cryptic __ or _
>
> for each in range(3):
>      dosomething
>
> To me, that reads like a normal English statement.
>
> Best,
> Sven

No thanks.  The current situation has suited the Python community for 25 
years give or take, so if it ain't broke, there's no way it's going to 
get fixed, thank goodness.  Can we move on please?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From srkunze at mail.de  Tue Oct 13 22:35:25 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 13 Oct 2015 22:35:25 +0200
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mvjp4n$irg$1@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org>
Message-ID: <561D6B0D.8050502@mail.de>

On 13.10.2015 22:21, Mark Lawrence wrote:
> On 13/10/2015 18:59, Sven R. Kunze wrote:
>> I still like instead of the cryptic __ or _
>>
>> for each in range(3):
>>      dosomething
>>
>> To me, that reads like a normal English statement.
>>
>> Best,
>> Sven
>
> No thanks.  The current situation has suited the Python community for 
> 25 years give or take, so if it ain't broke, there's no way it's going 
> to get fixed, thank goodness.  Can we move on please?
>

What exactly do you mean?

From breamoreboy at yahoo.co.uk  Wed Oct 14 00:19:46 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 13 Oct 2015 23:19:46 +0100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <561D6B0D.8050502@mail.de>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
Message-ID: <mvk031$6k4$1@ger.gmane.org>

On 13/10/2015 21:35, Sven R. Kunze wrote:
> On 13.10.2015 22:21, Mark Lawrence wrote:
>> On 13/10/2015 18:59, Sven R. Kunze wrote:
>>> I still like instead of the cryptic __ or _
>>>
>>> for each in range(3):
>>>      dosomething
>>>
>>> To me, that reads like a normal English statement.
>>>
>>> Best,
>>> Sven
>>
>> No thanks.  The current situation has suited the Python community for
>> 25 years give or take, so if it ain't broke, there's no way it's going
>> to get fixed, thank goodness.  Can we move on please?
>>
>
> What exactly do you mean?
>

Stop pointlessly discussing something that simply is not going to happen.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From rymg19 at gmail.com  Wed Oct 14 00:24:17 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 13 Oct 2015 17:24:17 -0500
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mvk031$6k4$1@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
 <mvk031$6k4$1@ger.gmane.org>
Message-ID: <272CC295-56A4-42BA-A81D-D6829803C9A4@gmail.com>

Like multi-line lambdas, multi-line with statements, null coalescing operators, and all the other topics that come up every other week?

On October 13, 2015 5:19:46 PM CDT, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>On 13/10/2015 21:35, Sven R. Kunze wrote:
>> On 13.10.2015 22:21, Mark Lawrence wrote:
>>> On 13/10/2015 18:59, Sven R. Kunze wrote:
>>>> I still like instead of the cryptic __ or _
>>>>
>>>> for each in range(3):
>>>>      dosomething
>>>>
>>>> To me, that reads like a normal English statement.
>>>>
>>>> Best,
>>>> Sven
>>>
>>> No thanks.  The current situation has suited the Python community
>for
>>> 25 years give or take, so if it ain't broke, there's no way it's
>going
>>> to get fixed, thank goodness.  Can we move on please?
>>>
>>
>> What exactly do you mean?
>>
>
>Stop pointlessly discussing something that simply is not going to
>happen.
>
>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151013/856d1e2a/attachment.html>

From steve at pearwood.info  Wed Oct 14 02:01:12 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 14 Oct 2015 11:01:12 +1100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <85eggytup8.fsf@benfinney.id.au>
References: <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <85eggytup8.fsf@benfinney.id.au>
Message-ID: <20151014000111.GG13813@ando.pearwood.info>

On Wed, Oct 14, 2015 at 06:49:39AM +1100, Ben Finney wrote:
[...]
> Whereas ?__? is not a normal name, and like other unusual names in
> Python should lead the reader to say ?Okay, obviously the person writing
> this had something odd in mind. I'd better find out what.? That is more
> likely to teach about the ?__? convention, than to be seen as a mistake
> by experienced programmers.

I'd just like to point out that using __ as a loop variable (or any 
other variable) is incompatible with the IPython convention for __. It 
is just wrong to talk about "the" __ convention, since there are at 
least two in common use, and using one necessarily conflicts with using 
the other.

If anyone wants to argue the pros and cons of one convention versus the 
other, or what unused loop variables ought to be called (x, unused, 
each, _, __, dontcare, whatever...) please take it to 
python-list at python.org or comp.lang.python, as it is off-topic here.

Thanks,


-- 
Steve

From python at lucidity.plus.com  Wed Oct 14 02:01:50 2015
From: python at lucidity.plus.com (Erik)
Date: Wed, 14 Oct 2015 01:01:50 +0100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mvk031$6k4$1@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
 <mvk031$6k4$1@ger.gmane.org>
Message-ID: <561D9B6E.9020601@lucidity.plus.com>

On 13/10/15 23:19, Mark Lawrence wrote:
> On 13/10/2015 21:35, Sven R. Kunze wrote:
>> On 13.10.2015 22:21, Mark Lawrence wrote:
>>> On 13/10/2015 18:59, Sven R. Kunze wrote:
>>>> I still like instead of the cryptic __ or _
>>>>
>>>> for each in range(3):
>>>>      dosomething
>>>>
>>>> To me, that reads like a normal English statement.
>>>>
>>>> Best,
>>>> Sven
>>>
>>> No thanks.  The current situation has suited the Python community for
>>> 25 years give or take, so if it ain't broke, there's no way it's going
>>> to get fixed, thank goodness.  Can we move on please?
>>>
>>
>> What exactly do you mean?
>
> Stop pointlessly discussing something that simply is not going to happen.

Huh?

for each in range(3):
   foo()

... is already valid Python. In what way is that "not going to happen"?

This spelling was just suggested as an alternative to the original 
poster who did not like having to explain the cryptic "___" spelling to 
students.

Same construct, different spelling.

E.

From python at lucidity.plus.com  Wed Oct 14 02:12:05 2015
From: python at lucidity.plus.com (Erik)
Date: Wed, 14 Oct 2015 01:12:05 +0100
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <20151014000111.GG13813@ando.pearwood.info>
References: <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <85eggytup8.fsf@benfinney.id.au> <20151014000111.GG13813@ando.pearwood.info>
Message-ID: <561D9DD5.90406@lucidity.plus.com>

On 14/10/15 01:01, Steven D'Aprano wrote:
> If anyone wants to argue the pros and cons of one convention versus the
> other, or what unused loop variables ought to be called (x, unused,
> each, _, __, dontcare, whatever...) please take it to
> python-list at python.org or comp.lang.python, as it is off-topic here.

The discussion of "what unused loop variables ought to be called" is in 
direct response to someone suggesting a syntax change to make things 
easier to teach.

To suggest that they spell the loop differently to make it easier to 
teach whilst not requiring any syntax changes to the language is, I 
would argue, on-topic as a rebuttal to the change request.

However, I agree with you that this particular discussion has probably 
run its course.

E.

From tjreedy at udel.edu  Wed Oct 14 06:10:41 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 14 Oct 2015 00:10:41 -0400
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <561D9B6E.9020601@lucidity.plus.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
 <mvk031$6k4$1@ger.gmane.org> <561D9B6E.9020601@lucidity.plus.com>
Message-ID: <mvkkk2$pao$1@ger.gmane.org>

On 10/13/2015 8:01 PM, Erik wrote:
> On 13/10/15 23:19, Mark Lawrence wrote:

>> Stop pointlessly discussing something that simply is not going to happen.
>
> Huh?
>
> for each in range(3):
>    foo()
>
> ... is already valid Python. In what way is that "not going to happen"?

What is not going to happen is that Python will not be changed to 
actually make a particular identifier a dummy, by not, for instance, 
letting it be used within the loop.  I know you were not proposing that.

> This spelling was just suggested as an alternative to the original
> poster who did not like having to explain the cryptic "___" spelling to
> students.

What I think will not and should not happen is that Guido and/or core 
devs 'bless' in PEP 8 any particular work for teaching beginners.  Mark 
gets a bit over-anxious sometimes, but insofar as he meant 'its futile 
for us to try to decide on one spelling here', I think he is correct.

> Same construct, different spelling.

Right, and I think 'each' if better than '_' or '__' for beginners.  I 
recommend 'count' for range() and 'item' for a generic iterable.  I also 
recommend that teachers of beginners experiment with various choices, 
and share their experiences with each other. Who knows, maybe kids would 
prefer 'for kangaroo in range(3):', or other personal choices.

-- 
Terry Jan Reedy


From stephen at xemacs.org  Wed Oct 14 07:28:07 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 14 Oct 2015 14:28:07 +0900
Subject: [Python-ideas] What is Python-ideas for? [meta] [Inspired by:
 Simpler syntax for basic iterations]
In-Reply-To: <272CC295-56A4-42BA-A81D-D6829803C9A4@gmail.com>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
 <mvk031$6k4$1@ger.gmane.org>
 <272CC295-56A4-42BA-A81D-D6829803C9A4@gmail.com>
Message-ID: <22045.59367.919094.178853@turnbull.sk.tsukuba.ac.jp>

Ryan Gonzalez writes:

Avoid top-posting.  It helps prevent posting merely to be
contrary.<wink />  Reordering....

 > Mark Lawrence writes:

 >> Stop pointlessly discussing something that simply is not going to
 >> happen.

 > Like multi-line lambdas, multi-line with statements, null
 > coalescing operators, and all the other topics that come up every
 > other week?

There is a clear qualitative difference here.  Mark is right about the
"repeat n:" proposal, and your comparison is specious.

				*****

The rest of my post is an attempt to present my understanding of some
aspects of Pythonic design thinking as relevant to discussions on
Python-Ideas.  As a running example, I try to argue why this thread
should have ended with at most two posts in it.  (Some might say it
shouldn't have started because of PEP 284, but figuring out relevance
of several hundred PEPs is a task more suited to scripture scholars
than language designers.)

Guido admits the benefits of all of the features Ryan mentions, and
entertains *new* proposals of syntax to invoke them.  That's not
because he's a nice guy handing out warm fuzzies to would-be
contributors.  We currently don't have a good way to do them, where
"good" involves both "concise" and "esthetically pleasing, especially
to Guido".  He really would like to serve the constituencies for those
features (and might even warm up to them himself, given suitable
syntax).

Guido also values the educational use of Python, and when introducing
new features he works hard to find syntax (including choice of
keyword) that is "intuitive" to both new users of Python and new
programmers.  If anybody "gets" CP4E, Guido does.  Of course there are
different ways to get there, but arguing that lack of "repeat n:" is
somehow contrary to CP4E is disrespectful.  It's a different path to
that end.

*This proposal* to introduce a special syntax for pure repetition,
however, obviously falls afoul of several principles *of Pythonic
language design*.  On the contrary, the arguments *for* the proposal
are "non-Pythonic", which doesn't make them unimportant, much less
invalid.  It does, however, make them low-value on *this* list, which
is for proposed improvements to the Python language.  We're not here
to discuss language improvements in general.[1]  Here's my list:

1.  The bar for new keywords is *very* high.  I don't think the
    educational purpose would be well-served by twisting "for" into a
    knot with "for 4:"; that just doesn't correspond to valid English
    usage.  "while 4:" is a little closer (IMO YMMV), but already has
    a meaning so it's out anyway due to backward compatibility.  I
    think this proposal needs a new keyword, and "repeat" is excellent
    for the purpose.  That's already a deal-killer for such a
    specialized construct.

2.  We already have a good way to write "repeat 4:".  The proposed
    syntax is just syntactic sugar, and even in its educational
    context, it's not much sweeter than (proper) use of the existing
    syntax.  Watch one user's lips move as she draws a square:

        First I draw the top, then I go down the right, then I draw
        across the bottom, and then I connect the left up to the top.

    Now compare:

        repeat 4:
            forward(10)
            right(90)

    versus:

        for side in ('across top',
                     'down right',
                     'across bottom',
                     'up left'):
            forward(10)
            right(90)

    Which expresses her intent better?  Is it really educational to
    have *this* user to think of a square as an abstract regular
    polygon that happens to have 4 sides?  True, it's easy to cargo-
    cult the "repeat 4:" idiom the next time she needs a square, but
    in learning to use "repeat n:", has she learned anything about
    implementing her own procedures?  (This is a variation on the
    individuality issue that Terry expressed with "kangaroo".)

    Note: this is a plausible example user, and I am asking whether
    *she* is served by the new syntax.  I don't maintain that she is
    *the* typical user.  I do think that educators should consider her
    needs as well as the question of how common her type is.  While
    there probably is educational benefit to this change, it's far
    from clear to me what it is, or whether in fact the student's 
    benefit is anywhere near as large as that to teachers who want to
    present one "simple" programming construct and move on quickly to
    another one.

3.  The "repeat 4" idiom clearly involves counting (otherwise the
    program doesn't know when to stop), but it completely conceals
    the process.  That violates "explicit is better than implicit",
    and furthermore makes the idiom very specialized, dramatically
    reducing the benefits it can have.

    It seems to me that the "for ... in" construct gets the balance
    just right.  It exposes both counting as an isomorphism between
    concrete sets (above, repetitions of a subprocedure corresponding
    to named sides of a square), and abstractly using numbers merely
    to count (via "range(n)").  (Of course "range()" also provides a
    flexible way to denote various concrete sets of numbers.)

4.  True, when you really do have an algorithm that simply cares about
    the number and repeats the same suite independently of the
    iteration count, exposing an iteration count variable is arguably
    a wart.  But it's very small compared to the power of the "for
    ... in" idiom.  Proposing to address that wart runs into "Special
    cases aren't special enough to break the rules."

Now, all of these principles ("Read my lips: no new keywords!" and the
three from the Zen) are pragmatic guidelines, not absolutes.  Not even
"no keywords".  And they're Pythonic principles: they don't always
generalize to other languages.[2]  But when a proposal infringes so
many Pythonic principles so flagrantly (IMHO, each is sufficient
reason to deny the proposal by itself, though YMMV), it is indeed a
non-starter on this list.

Regards,


Footnotes: 
[1]  Note that several core Python developers have unsubscribed or
muted this list completely, partly because of these off-target
threads.

[2]  I have to admit that I'm thoroughly drunk on the Kool-Aid
though, and tend to apply them to other languages, which inevitably
fall short.


From rymg19 at gmail.com  Wed Oct 14 16:14:33 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 14 Oct 2015 09:14:33 -0500
Subject: [Python-ideas] What is Python-ideas for? [meta] [Inspired by:
	Simpler syntax for basic iterations]
In-Reply-To: <22045.59367.919094.178853@turnbull.sk.tsukuba.ac.jp>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
 <mvk031$6k4$1@ger.gmane.org> <272CC295-56A4-42BA-A81D-D6829803C9A4@gmail.com>
 <22045.59367.919094.178853@turnbull.sk.tsukuba.ac.jp>
Message-ID: <30332C3E-2363-4D1E-86EB-1B05E60D5B9F@gmail.com>



On October 14, 2015 12:28:07 AM CDT, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
>Ryan Gonzalez writes:
>
>Avoid top-posting.  It helps prevent posting merely to be
>contrary.<wink />  Reordering....
>
> > Mark Lawrence writes:
>
> >> Stop pointlessly discussing something that simply is not going to
> >> happen.
>
> > Like multi-line lambdas, multi-line with statements, null
> > coalescing operators, and all the other topics that come up every
> > other week?
>
>There is a clear qualitative difference here.  Mark is right about the
>"repeat n:" proposal, and your comparison is specious.
>
>				*****
>
>The rest of my post is an attempt to present my understanding of some
>aspects of Pythonic design thinking as relevant to discussions on
>Python-Ideas.  As a running example, I try to argue why this thread
>should have ended with at most two posts in it.  (Some might say it
>shouldn't have started because of PEP 284, but figuring out relevance
>of several hundred PEPs is a task more suited to scripture scholars
>than language designers.)
>


...

That was supposed to be a joke. :O

>Guido admits the benefits of all of the features Ryan mentions, and
>entertains *new* proposals of syntax to invoke them.  That's not
>because he's a nice guy handing out warm fuzzies to would-be
>contributors.  We currently don't have a good way to do them, where
>"good" involves both "concise" and "esthetically pleasing, especially
>to Guido".  He really would like to serve the constituencies for those
>features (and might even warm up to them himself, given suitable
>syntax).
>
>Guido also values the educational use of Python, and when introducing
>new features he works hard to find syntax (including choice of
>keyword) that is "intuitive" to both new users of Python and new
>programmers.  If anybody "gets" CP4E, Guido does.  Of course there are
>different ways to get there, but arguing that lack of "repeat n:" is
>somehow contrary to CP4E is disrespectful.  It's a different path to
>that end.
>
>*This proposal* to introduce a special syntax for pure repetition,
>however, obviously falls afoul of several principles *of Pythonic
>language design*.  On the contrary, the arguments *for* the proposal
>are "non-Pythonic", which doesn't make them unimportant, much less
>invalid.  It does, however, make them low-value on *this* list, which
>is for proposed improvements to the Python language.  We're not here
>to discuss language improvements in general.[1]  Here's my list:
>
>1.  The bar for new keywords is *very* high.  I don't think the
>    educational purpose would be well-served by twisting "for" into a
>    knot with "for 4:"; that just doesn't correspond to valid English
>    usage.  "while 4:" is a little closer (IMO YMMV), but already has
>    a meaning so it's out anyway due to backward compatibility.  I
>    think this proposal needs a new keyword, and "repeat" is excellent
>    for the purpose.  That's already a deal-killer for such a
>    specialized construct.
>
>2.  We already have a good way to write "repeat 4:".  The proposed
>    syntax is just syntactic sugar, and even in its educational
>    context, it's not much sweeter than (proper) use of the existing
>    syntax.  Watch one user's lips move as she draws a square:
>
>        First I draw the top, then I go down the right, then I draw
>        across the bottom, and then I connect the left up to the top.
>
>    Now compare:
>
>        repeat 4:
>            forward(10)
>            right(90)
>
>    versus:
>
>        for side in ('across top',
>                     'down right',
>                     'across bottom',
>                     'up left'):
>            forward(10)
>            right(90)
>
>    Which expresses her intent better?  Is it really educational to
>    have *this* user to think of a square as an abstract regular
>    polygon that happens to have 4 sides?  True, it's easy to cargo-
>    cult the "repeat 4:" idiom the next time she needs a square, but
>    in learning to use "repeat n:", has she learned anything about
>    implementing her own procedures?  (This is a variation on the
>    individuality issue that Terry expressed with "kangaroo".)
>
>    Note: this is a plausible example user, and I am asking whether
>    *she* is served by the new syntax.  I don't maintain that she is
>    *the* typical user.  I do think that educators should consider her
>    needs as well as the question of how common her type is.  While
>    there probably is educational benefit to this change, it's far
>    from clear to me what it is, or whether in fact the student's 
>    benefit is anywhere near as large as that to teachers who want to
>    present one "simple" programming construct and move on quickly to
>    another one.
>
>3.  The "repeat 4" idiom clearly involves counting (otherwise the
>    program doesn't know when to stop), but it completely conceals
>    the process.  That violates "explicit is better than implicit",
>    and furthermore makes the idiom very specialized, dramatically
>    reducing the benefits it can have.
>
>    It seems to me that the "for ... in" construct gets the balance
>    just right.  It exposes both counting as an isomorphism between
>    concrete sets (above, repetitions of a subprocedure corresponding
>    to named sides of a square), and abstractly using numbers merely
>    to count (via "range(n)").  (Of course "range()" also provides a
>    flexible way to denote various concrete sets of numbers.)
>
>4.  True, when you really do have an algorithm that simply cares about
>    the number and repeats the same suite independently of the
>    iteration count, exposing an iteration count variable is arguably
>    a wart.  But it's very small compared to the power of the "for
>    ... in" idiom.  Proposing to address that wart runs into "Special
>    cases aren't special enough to break the rules."
>
>Now, all of these principles ("Read my lips: no new keywords!" and the
>three from the Zen) are pragmatic guidelines, not absolutes.  Not even
>"no keywords".  And they're Pythonic principles: they don't always
>generalize to other languages.[2]  But when a proposal infringes so
>many Pythonic principles so flagrantly (IMHO, each is sufficient
>reason to deny the proposal by itself, though YMMV), it is indeed a
>non-starter on this list.
>
>Regards,
>
>
>Footnotes: 
>[1]  Note that several core Python developers have unsubscribed or
>muted this list completely, partly because of these off-target
>threads.
>
>[2]  I have to admit that I'm thoroughly drunk on the Kool-Aid
>though, and tend to apply them to other languages, which inevitably
>fall short.

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From tjreedy at udel.edu  Thu Oct 15 03:03:22 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 14 Oct 2015 21:03:22 -0400
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
Message-ID: <561EFB5A.1080703@udel.edu>

On 10/7/2015 2:31 AM, Paddy3118 wrote:
 > Just saw what the Perl 6 site does  to punt the curious to their entries
 > on the Rosetta Code site here: http://perl6.org/community/rosettacode

Perl6 is apparently different from Perl 5 and before to justify adding 
Perl 6 sections after the Perl sections, with Perl entries re-written. 
Given the slow development and use of Perl 6, I can understand the Perl 
6 people wanting to promote Perl 6 that way. Python 3 is not quite in 
the same position.

 > and I thought maybe we could do something similar on Python.org
 > <http://Python.org>?

python-ideas is about future version of the language.
The pydotorg list discusses the site
https://mail.python.org/mailman/listinfo/pydotorg
This would be the place to suggest an addition under 'Community'.

But if modern Python 3 code is not welcome on the site (see below), I 
would not want it promoted as containing examples of 'Python' code.

 > I know that the core development team of Perl 6 have put a lot of effort
 > into their RC entries, and over the years I have seen evidence of tasks
 > being used to highlight areas for subsequent improvement in Perl 6.
 >
 > I don't /recognise /such a contribution from core development Pythoneers
 > on RC but members of the Python community have made great efforts in
 > making Python solutions available for most tasks on RC in a mixture of
 > Python 2 and 3

The page you reference seems a bit hostile to Python 3. It equates 
'Python' with 'Python 2' and calls code that also works with Python 3 
'unidiomatic Python', and makes no mention of the possibility of 
submitting Python 3.

 > and also showing greater use of our interactive command
 > line interface/REPL for solutions which other languages may have,

Since one cannot do anything in the REPL that one cannot do in a file, 
except leave out 'print', I do not see the point. Leaving '>>>' and 
'...' in examples only makes it hard to copy and paste.

 > but
 > rarely show unless the task asks specifically for an answer using a REPL.
 >
 > *In short:* Python on RC <http://rosettacode.org/wiki/Category:Python>

Reading this page, I am not sure how welcome I would be if I came and 
tried to contribute.

Aside from that, I do not see any obvious way to get of list of tasks 
without a Python solution, or is that an empty set?

 > has good code examples - why not advertise that fact on Python.org in a
 > similar way to the Perl 6 site?

Because I do not consider code with 'print x' instead of 'print(x)'. 
especially when that is the only thing keeping the code from also 
running on Python 3, to be a 'good code example'. Ditto for 2.x-only 
code (or 3.x-only code, for that matter) that is not labelled as such. 
On Python list and elsewhere, newbies frequently post 'I copied this 
code form ... and it does not run.' because they are running it on the 
wrong interpreter.

 > *Disclosure: *I should add that I am an administrator, task writer and
 > one of the Python code contributors to the Rosetta Code site.

Please consider making it more Python 3 friendly than it strikes me as 
being. Or if my impression is wrong, edit the Python page to at least 
treat Python 3 on an equal footing.

Out of curiosity, and to make a point: Does the current tasks include 
any that test the quality and completeness of unicode implementation? 
Something simple like the following, involving astral chars.

"Construct a string consisting of the unicode characters
U+33, U+333, U+11111, U+334, U+11112, U+11113. U+34. Print the decimal 
codepoint of the third character before the first occurrence of U+11113. 
(Answer: 69905.) The code should work on any platform the code could run 
on."

3.x code:
s = "\u0033\u0333\U00011111\u0334\U00011112\U00011113"
print(ord(s[s.index('\U00011113')-3]))

2.x code: several lines more to accommodate Windows and narrow unicode 
builds on some linuxes.


-- 
Terry Jan Reedy


From abarnert at yahoo.com  Thu Oct 15 04:00:15 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 14 Oct 2015 19:00:15 -0700
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <561EFB5A.1080703@udel.edu>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
 <561EFB5A.1080703@udel.edu>
Message-ID: <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>

On Oct 14, 2015, at 18:03, Terry Reedy <tjreedy at udel.edu> wrote:
> 
> The page you reference seems a bit hostile to Python 3. It equates 'Python' with 'Python 2' and calls code that also works with Python 3 'unidiomatic Python', and makes no mention of the possibility of submitting Python 3.

It also seems to imply that widespread use of reduce is pythonic, apologizing for the unpythonic need to check for and import it in case of Python 3, which makes me wonder how pythonic the code is even for a decade ago?

From steve.dower at python.org  Thu Oct 15 15:19:27 2015
From: steve.dower at python.org (Steve Dower)
Date: Thu, 15 Oct 2015 06:19:27 -0700
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
 <561EFB5A.1080703@udel.edu> <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>
Message-ID: <E1ZmiSD-0006QY-RD@se2-syd.hostedmail.net.au>

FWIW, that comment is from five years ago and was written by someone who claims to have been using Python far longer than that (and has enough contributions for that to be believable). It reads more to me like a preemptive apology to "The Haters" than a dismissal of Python 3.

I'm sure any contributor to the site can easily reword it to better reflect that not all examples are as terse as they could be if they avoided the 2/3 straddle.

Cheers,
Steve

Top-posted from my Windows Phone

-----Original Message-----
From: "Andrew Barnert via Python-ideas" <python-ideas at python.org>
Sent: ?10/?14/?2015 19:03
To: "Terry Reedy" <tjreedy at udel.edu>
Cc: "python-ideas at python.org" <python-ideas at python.org>
Subject: Re: [Python-ideas] Rosetta Code on Python.org site?

On Oct 14, 2015, at 18:03, Terry Reedy <tjreedy at udel.edu> wrote:
> 
> The page you reference seems a bit hostile to Python 3. It equates 'Python' with 'Python 2' and calls code that also works with Python 3 'unidiomatic Python', and makes no mention of the possibility of submitting Python 3.

It also seems to imply that widespread use of reduce is pythonic, apologizing for the unpythonic need to check for and import it in case of Python 3, which makes me wonder how pythonic the code is even for a decade ago?
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151015/30402c61/attachment.html>

From jason at schwerberg.com  Thu Oct 15 19:37:47 2015
From: jason at schwerberg.com (Jason Schwerberg)
Date: Thu, 15 Oct 2015 10:37:47 -0700
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <E1ZmiSD-0006QY-RD@se2-syd.hostedmail.net.au>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
 <561EFB5A.1080703@udel.edu> <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>
 <E1ZmiSD-0006QY-RD@se2-syd.hostedmail.net.au>
Message-ID: <561FE46B.2070906@schwerberg.com>

At risk of completely going off topic from the mission of Python-Ideas, 
I'd almost rather see a delineation between Python 2 and Python 3, like 
RC has done with Perl 5 (http://rosettacode.org/wiki/Category:Perl) and 
Perl 6 (http://rosettacode.org/wiki/Category:Perl_6).

 From RC:

 > Though it resembles previous versions of Perl to no small degree, 
Perl 6 is substantially a new language; by design, it isn't 
backwards-compatible with Perl 5.

And from https://wiki.python.org/moin/Python2orPython3

 > Guido van Rossum (the original creator of the Python language) 
decided to clean up Python 2.x properly, with less regard for backwards 
compatibility than is the case for new releases in the 2.x range.

While there are many ways to make 'good' code, compatible with both 
Python 2 and 3, I think that the most terse, efficient code would be 
better demonstrated by segregating Python 2.x from Python 3.x.

On 10/15/2015 6:19 AM, Steve Dower wrote:
> FWIW, that comment is from five years ago and was written by someone 
> who claims to have been using Python far longer than that (and has 
> enough contributions for that to be believable). It reads more to me 
> like a preemptive apology to "The Haters" than a dismissal of Python 3.
>
> I'm sure any contributor to the site can easily reword it to better 
> reflect that not all examples are as terse as they could be if they 
> avoided the 2/3 straddle.
>
> Cheers,
> Steve
>
> Top-posted from my Windows Phone
> ------------------------------------------------------------------------
> From: Andrew Barnert via Python-ideas <mailto:python-ideas at python.org>
> Sent: ?10/?14/?2015 19:03
> To: Terry Reedy <mailto:tjreedy at udel.edu>
> Cc: python-ideas at python.org <mailto:python-ideas at python.org>
> Subject: Re: [Python-ideas] Rosetta Code on Python.org site?
>
> On Oct 14, 2015, at 18:03, Terry Reedy <tjreedy at udel.edu> wrote:
> >
> > The page you reference seems a bit hostile to Python 3. It equates 
> 'Python' with 'Python 2' and calls code that also works with Python 3 
> 'unidiomatic Python', and makes no mention of the possibility of 
> submitting Python 3.
>
> It also seems to imply that widespread use of reduce is pythonic, 
> apologizing for the unpythonic need to check for and import it in case 
> of Python 3, which makes me wonder how pythonic the code is even for a 
> decade ago?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151015/c05624df/attachment.html>

From chris.barker at noaa.gov  Thu Oct 15 23:14:16 2015
From: chris.barker at noaa.gov (Chris Barker)
Date: Thu, 15 Oct 2015 14:14:16 -0700
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <561FE46B.2070906@schwerberg.com>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
 <561EFB5A.1080703@udel.edu> <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>
 <E1ZmiSD-0006QY-RD@se2-syd.hostedmail.net.au>
 <561FE46B.2070906@schwerberg.com>
Message-ID: <CALGmxELH9Yg0-evLw8ZDMEVBXg-MufXw9Zz0RaCD6LMVOUudbw@mail.gmail.com>

On Thu, Oct 15, 2015 at 10:37 AM, Jason Schwerberg <jason at schwerberg.com>
wrote:

> At risk of completely going off topic from the mission of Python-Ideas,
> I'd almost rather see a delineation between Python 2 and Python 3
>


> While there are many ways to make 'good' code, compatible with both Python
> 2 and 3, I think that the most terse, efficient code would be better
> demonstrated by segregating Python 2.x from Python 3.x.
>

makes sense to me. RC is a wiki, yes? So it looks like this:

http://rosettacode.org/wiki/Rosetta_Code:Add_a_Language

Is what you need to do...

Or would there really  be strong objections from the community to having a
Python3 section on the site?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151015/a3fd9eec/attachment-0001.html>

From gokoproject at gmail.com  Thu Oct 15 23:25:16 2015
From: gokoproject at gmail.com (John Wong)
Date: Thu, 15 Oct 2015 17:25:16 -0400
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
Message-ID: <CACCLA55LbyGKY24k6cnXK+C_WFQsayS+wyemNoaT+R=i70diPg@mail.gmail.com>

On Wed, Oct 7, 2015 at 2:31 AM, Paddy3118 <paddy3118 at gmail.com> wrote:

> Just saw what the Perl 6 site does  to punt the curious to their entries
> on the Rosetta Code site here: http://perl6.org/community/rosettacode and
> I thought maybe we could do something similar on Python.org
> <http://python.org/>?
>
>
Problem with rosettacode is they can end up like stackoverflow answers that
some are good and some are bad. Some will view that as Bible and some will
constantly make "suggestions" why X is better. But in the spirit of
playground and show case, why not make this a personal project and have
people commit to a public repo such as GitHub? Everyone will come up with a
different solution. Take BFS and DFS as examples, there are at least two
kinds of popular ways to implement that in Python.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151015/df9aa344/attachment.html>

From gokoproject at gmail.com  Thu Oct 15 23:29:57 2015
From: gokoproject at gmail.com (John Wong)
Date: Thu, 15 Oct 2015 17:29:57 -0400
Subject: [Python-ideas] Simpler syntax for basic iterations
In-Reply-To: <mvkkk2$pao$1@ger.gmane.org>
References: <CAGMu_=rJLT+vr+cqmucOS5onkAZT4grVwpSguZ1fy3kDEK9PbA@mail.gmail.com>
 <mv9t8d$r0e$2@ger.gmane.org>
 <CAGMu_=r_6Nsbu+jN_RrMLQ32fw=ZbXu9ti8cSg2e3r38wjri6Q@mail.gmail.com>
 <22042.5180.435279.647018@turnbull.sk.tsukuba.ac.jp>
 <CALxg4FUGdB=UqfhvNGOAfrDR6QOoe3XzPH43fE8Z-fKtmnMoPw@mail.gmail.com>
 <mvf45n$s2s$1@ger.gmane.org>
 <CAP7+vJLPDiaMZTVRD3Oo8kuXg4TKhcQc8FhFLXx==riBV_YNbg@mail.gmail.com>
 <CALxg4FUOf4uRBZwUK_h6yCaSSTuuqE0bq16JEbA-9Gw-tFVNDg@mail.gmail.com>
 <85mvvnu3ro.fsf@benfinney.id.au> <561D469B.9020305@mail.de>
 <mvjp4n$irg$1@ger.gmane.org> <561D6B0D.8050502@mail.de>
 <mvk031$6k4$1@ger.gmane.org> <561D9B6E.9020601@lucidity.plus.com>
 <mvkkk2$pao$1@ger.gmane.org>
Message-ID: <CACCLA56_PpQG794hKeX8yRhtAmkDhuiacBC7YBfqaOxT6TTGoA@mail.gmail.com>

On Wed, Oct 14, 2015 at 12:10 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>
> This spelling was just suggested as an alternative to the original
>> poster who did not like having to explain the cryptic "___" spelling to
>> students.
>>
>
> What I think will not and should not happen is that Guido and/or core devs
> 'bless' in PEP 8 any particular work for teaching beginners.  Mark gets a
> bit over-anxious sometimes, but insofar as he meant 'its futile for us to
> try to decide on one spelling here', I think he is correct.
>
> Same construct, different spelling.
>>
>
> Right, and I think 'each' if better than '_' or '__' for beginners.  I
> recommend 'count' for range() and 'item' for a generic iterable.  I also
> recommend that teachers of beginners experiment with various choices, and
> share their experiences with each other. Who knows, maybe kids would prefer
> 'for kangaroo in range(3):', or other personal choices.
>
>
If a student is asking _ then it should be explained. Soon or later the
students will probably see _ in other source code. Whether it is a 4-year
old child or a 18 year-old college freshman. I wouldn't expect my nephew to
understand everything I write, but they will grow up and have the "ah"
moment.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151015/27832ac7/attachment.html>

From steve at pearwood.info  Fri Oct 16 02:02:42 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 16 Oct 2015 11:02:42 +1100
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <CALGmxELH9Yg0-evLw8ZDMEVBXg-MufXw9Zz0RaCD6LMVOUudbw@mail.gmail.com>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
 <561EFB5A.1080703@udel.edu> <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>
 <E1ZmiSD-0006QY-RD@se2-syd.hostedmail.net.au>
 <561FE46B.2070906@schwerberg.com>
 <CALGmxELH9Yg0-evLw8ZDMEVBXg-MufXw9Zz0RaCD6LMVOUudbw@mail.gmail.com>
Message-ID: <20151016000241.GA11980@ando.pearwood.info>

On Thu, Oct 15, 2015 at 02:14:16PM -0700, Chris Barker wrote:
> On Thu, Oct 15, 2015 at 10:37 AM, Jason Schwerberg <jason at schwerberg.com>
> wrote:
> 
> > At risk of completely going off topic from the mission of Python-Ideas,
> > I'd almost rather see a delineation between Python 2 and Python 3
> 
> > While there are many ways to make 'good' code, compatible with both Python
> > 2 and 3, I think that the most terse, efficient code would be better
> > demonstrated by segregating Python 2.x from Python 3.x.
> 
> makes sense to me. RC is a wiki, yes? So it looks like this:
> 
> http://rosettacode.org/wiki/Rosetta_Code:Add_a_Language
> 
> Is what you need to do...
> 
> Or would there really  be strong objections from the community to having a
> Python3 section on the site?

I would object to adding Python 3 as a different language. It is not, 
and we should be emphasising the similarities, not the differences. 
Migration to Python 3 is slow enough without giving more ammunition 
("it's a completely different language, like Perl 6 is to Perl 5") to 
the foot-draggers and haters.

Adding Python 3 as a different language would be a strategic error, 
because most code snippets will be identical except for possibly 
print and (raw_)input. That's going to discourage people from adding a 
Python 3 version that is all but identical to the Python 2 version, 
which will increase the perception that nobody is using Python 3. 
("There are hundreds of Python 2 examples, and only a dozen Python 3 
examples.")

In my opinion, as far as Rosetta Code and similar sites go, all it takes 
to manage the Python 2/3 transition is a single comment at the start of 
the code snippet listing the version(s) targeted. I would consider that 
the minimum needed. Actually providing hybrid 2+3 code, or side-by-side 
versions, would be acceptible too, but that's more work.

I just looked at a small sample of arbitrary examples, and found that 
three out of the four already support Python 3, and the one that didn't 
just needed parens added to print to work.

So I don't think there's actually a problem here that needs fixing.



-- 
Steven

From tjreedy at udel.edu  Fri Oct 16 02:59:41 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 15 Oct 2015 20:59:41 -0400
Subject: [Python-ideas] Rosetta Code on Python.org site?
In-Reply-To: <20151016000241.GA11980@ando.pearwood.info>
References: <cc983e63-0633-470c-a17d-d48171193111@googlegroups.com>
 <561EFB5A.1080703@udel.edu> <392862D4-5C29-429B-80D2-EA2B0E4A2BC3@yahoo.com>
 <E1ZmiSD-0006QY-RD@se2-syd.hostedmail.net.au>
 <561FE46B.2070906@schwerberg.com>
 <CALGmxELH9Yg0-evLw8ZDMEVBXg-MufXw9Zz0RaCD6LMVOUudbw@mail.gmail.com>
 <20151016000241.GA11980@ando.pearwood.info>
Message-ID: <mvpi5u$efs$1@ger.gmane.org>

On 10/15/2015 8:02 PM, Steven D'Aprano wrote:
> On Thu, Oct 15, 2015 at 02:14:16PM -0700, Chris Barker wrote:
>> On Thu, Oct 15, 2015 at 10:37 AM, Jason Schwerberg <jason at schwerberg.com>
>> wrote:
>>
>>> At risk of completely going off topic from the mission of Python-Ideas,
>>> I'd almost rather see a delineation between Python 2 and Python 3
>>
>>> While there are many ways to make 'good' code, compatible with both Python
>>> 2 and 3, I think that the most terse, efficient code would be better
>>> demonstrated by segregating Python 2.x from Python 3.x.
>>
>> makes sense to me. RC is a wiki, yes? So it looks like this:
>>
>> http://rosettacode.org/wiki/Rosetta_Code:Add_a_Language
>>
>> Is what you need to do...
>>
>> Or would there really  be strong objections from the community to having a
>> Python3 section on the site?

Yes, Python 3 *is* Python. Python has always been dynamic in its 
definition, as well as its execution. 'Python 2' is a sequence of older, 
frozen dialects.  Note that 2.7 code may not run in older versions.  And 
that 2.0 code, for instance, may not work in some 2.x and after.  Python 
2 sections could be added for 2.x-only code, but for most of the tasks, 
solutions that run on 3.x and some subset of 2.x are easily possible. 
When not, the format allow multiple solutions within a language section. 
  For example, FizzBuzz/Python has a both 'print 100 lines' for-loop 
solution and and infinite itertools-based iterator solution.

> I would object to adding Python 3 as a different language. It is not,
> and we should be emphasising the similarities, not the differences.
> Migration to Python 3 is slow enough without giving more ammunition
> ("it's a completely different language, like Perl 6 is to Perl 5") to
> the foot-draggers and haters.

This is why I said the Perl6/5- was not the example to follow.

> Adding Python 3 as a different language would be a strategic error,
> because most code snippets will be identical except for possibly
> print and (raw_)input. That's going to discourage people from adding a
> Python 3 version that is all but identical to the Python 2 version,
> which will increase the perception that nobody is using Python 3.
> ("There are hundreds of Python 2 examples, and only a dozen Python 3
> examples.")
>
> In my opinion, as far as Rosetta Code and similar sites go, all it takes
> to manage the Python 2/3 transition is a single comment at the start of
> the code snippet listing the version(s) targeted. I would consider that
> the minimum needed. Actually providing hybrid 2+3 code, or side-by-side
> versions, would be acceptible too, but that's more work.
>
> I just looked at a small sample of arbitrary examples, and found that
> three out of the four already support Python 3, and the one that didn't
> just needed parens added to print to work.
>
> So I don't think there's actually a problem here that needs fixing.

Except for version labeling and blessing rather than dissing 3.x or 2&3 
solutions.

-- 
Terry Jan Reedy


From ian at feete.org  Fri Oct 16 12:36:42 2015
From: ian at feete.org (Ian Foote)
Date: Fri, 16 Oct 2015 11:36:42 +0100
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
Message-ID: <5620D33A.3050709@feete.org>

I recently wanted to use an OrderedCounter and an OrderedDefaultDict. 
After a bit of googling I discovered that OrderedCounter is quite easy 
to implement:

from collections import Counter, OrderedDict

class OrderedCounter(Counter, OrderedDict):
      'Counter that remembers the order elements are first seen'
      def __repr__(self):
          return '%s(%r)' % (self.__class__.__name__,
                             OrderedDict(self))
      def __reduce__(self):
          return self.__class__, (OrderedDict(self),)


from https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

Unfortunately an OrderedDefaultDict did not seem so easy. I did find 
http://stackoverflow.com/questions/6190331/can-i-do-an-ordered-default-dict-in-python 
which suggests:

|fromcollections 
importOrderedDict,CallableclassDefaultOrderedDict(OrderedDict):# Source: 
http://stackoverflow.com/a/6190500/562769def__init__(self,default_factory=None,*a,**kw):if(default_factory 
isnotNoneandnotisinstance(default_factory,Callable)):raiseTypeError('first 
argument must be 
callable')OrderedDict.__init__(self,*a,**kw)self.default_factory 
=default_factory 
def__getitem__(self,key):try:returnOrderedDict.__getitem__(self,key)exceptKeyError:returnself.__missing__(key)def__missing__(self,key):ifself.default_factory 
isNone:raiseKeyError(key)self[key]=value 
=self.default_factory()returnvalue 
def__reduce__(self):ifself.default_factory isNone:args =tuple()else:args 
=self.default_factory,returntype(self),args,None,None,self.items()defcopy(self):returnself.__copy__()def__copy__(self):returntype(self)(self.default_factory,self)def__deepcopy__(self,memo):importcopy 
returntype(self)(self.default_factory,copy.deepcopy(self.items()))def__repr__(self):return'OrderedDefaultDict(%s, 
%s)'%(self.default_factory,OrderedDict.__repr__(self))|


This seems to me both easy to get wrong and hard to discover. It also 
risks getting out of sync with updates to collections.
I'm wondering if this is sufficient justification to add OrderedCounter 
and OrderedDict to collections, either directly or as recipes.

Thanks,
Ian F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151016/ba1a8d67/attachment.html>

From abarnert at yahoo.com  Sat Oct 17 03:39:02 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 16 Oct 2015 18:39:02 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <5620D33A.3050709@feete.org>
References: <5620D33A.3050709@feete.org>
Message-ID: <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>

On Oct 16, 2015, at 03:36, Ian Foote <ian at feete.org> wrote:
> 
> I recently wanted to use an OrderedCounter and an OrderedDefaultDict. After a bit of googling I discovered that OrderedCounter is quite easy to implement:

This is already in the docs for OrderedDict, so it shouldn't have taken any googling.
> from collections import Counter, OrderedDict
> 
> class OrderedCounter(Counter, OrderedDict):
>      'Counter that remembers the order elements are first seen'
>      def __repr__(self):
>          return '%s(%r)' % (self.__class__.__name__,
>                             OrderedDict(self))
>      def __reduce__(self):
>          return self.__class__, (OrderedDict(self),)
> 
> from https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
> 
> Unfortunately an OrderedDefaultDict did not seem so easy. I did find http://stackoverflow.com/questions/6190331/can-i-do-an-ordered-default-dict-in-python which suggests:

Most of the trickiness here is handling None as a factory, which is only useful for subclasses that implement a custom __missing__. To make something that works like you'd expect except for that part is a lot simpler. And I'm not sure a docs recipe that's mostly code that's unnecessary for most uses, hiding the important and simple part, would be a great idea.
> from collections import OrderedDict, Callable
> 
> class DefaultOrderedDict(OrderedDict):
>     # Source: http://stackoverflow.com/a/6190500/562769
>     def __init__(self, default_factory=None, *a, **kw):
>         if (default_factory is not None and
>            not isinstance(default_factory, Callable)):
>             raise TypeError('first argument must be callable')
>         OrderedDict.__init__(self, *a, **kw)
>         self.default_factory = default_factory
> 
>     def __getitem__(self, key):
>         try:
>             return OrderedDict.__getitem__(self, key)
>         except KeyError:
>             return self.__missing__(key)
> 
>     def __missing__(self, key):
>         if self.default_factory is None:
>             raise KeyError(key)
>         self[key] = value = self.default_factory()
>         return value
> 
>     def __reduce__(self):
>         if self.default_factory is None:
>             args = tuple()
>         else:
>             args = self.default_factory,
>         return type(self), args, None, None, self.items()
> 
>     def copy(self):
>         return self.__copy__()
> 
>     def __copy__(self):
>         return type(self)(self.default_factory, self)
> 
>     def __deepcopy__(self, memo):
>         import copy
>         return type(self)(self.default_factory,
>                           copy.deepcopy(self.items()))
> 
>     def __repr__(self):
>         return 'OrderedDefaultDict(%s, %s)' % (self.default_factory,
>                                                OrderedDict.__repr__(self))
> 
> This seems to me both easy to get wrong and hard to discover. It also risks getting out of sync with updates to collections.
> I'm wondering if this is sufficient justification to add OrderedCounter and OrderedDict to collections, either directly or as recipes.

Since one of them is already a recipe and the other one would probably not be useful as one, I don't think that's a great idea.

Adding OrderedDefaultDict to the module itself might be useful. Or, alternatively, put it in PyPI or ActiveState and add a link to it from the docs? (I'm pretty sure there are already implementations in both places, actually. Also, it would be nice to be able to point at a third-party implementation that works in 2.6+/3.2+ or whatever even if it only appears in the 3.6 docs.)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151016/088f0314/attachment-0001.html>

From abarnert at yahoo.com  Sat Oct 17 03:57:21 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 16 Oct 2015 18:57:21 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
Message-ID: <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>

Sorry, sent too soon?

Sent from my iPhone

> On Oct 16, 2015, at 18:39, Andrew Barnert via Python-ideas <python-ideas at python.org> wrote:
> 
>> On Oct 16, 2015, at 03:36, Ian Foote <ian at feete.org> wrote:
>> 
>> I recently wanted to use an OrderedCounter and an OrderedDefaultDict. After a bit of googling I discovered that OrderedCounter is quite easy to implement:
> 
> This is already in the docs for OrderedDict, so it shouldn't have taken any googling.
>> from collections import Counter, OrderedDict
>> 
>> class OrderedCounter(Counter, OrderedDict):
>>      'Counter that remembers the order elements are first seen'
>>      def __repr__(self):
>>          return '%s(%r)' % (self.__class__.__name__,
>>                             OrderedDict(self))
>>      def __reduce__(self):
>>          return self.__class__, (OrderedDict(self),)
>> 
>> from https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
>> 
>> Unfortunately an OrderedDefaultDict did not seem so easy. I did find http://stackoverflow.com/questions/6190331/can-i-do-an-ordered-default-dict-in-python which suggests:
> 
> Most of the trickiness here is handling None as a factory, which is only useful for subclasses that implement a custom __missing__

? or that requires two-stage initialization, where it can initialize the base with None and then assign the factory later. Still not all that common, but it is a separate reason for the extra complexity besides a custom __missing__ that doesn't need the factory.

> To make something that works like you'd expect except for that part is a lot simpler.

Off the top of my head (untested, since I'm on my phone):

class DefaultOrderedDict(OrderedDict):
    def __init__(self, default_factory, *a, **kw):
        super().__init__(*a, **kw)
        self.default_factory = default_factory
    def __getitem__(self, key):
        try:
            return super().__getitem__(key)
        except KeyError:
            return self.__missing__(key)
    def __missing__(self, key):
        self[key] = value = self.default_factory()
        return value
    def __repr__(self):
        return '{}({}, {})'.format(type(self).__name__, self.default_factory, dict.__repr__(self))

In fact, if you aren't worried about subclassing, or about being able to directly call or pass around a bound__missing__, you can make this even simpler.

I may be wrong about this being picklable/copyable without any extra help; if so, it still doesn't need the whole complicated "treat a None factory as if it doesn't exist" part.

Also, I think the SO recipe works in Python 2.4+, or whichever first added OrderedDict, while this requires 3.1 (although it should be obvious how to backport it, make sure that OrderedDict isn't an old-style class?).

Anyway, other than those issues, it seems pretty obvious, and I don't think there's any part of that functionality that's easy to get wrong. Handling a None factory in __missing__ and pickle/copy is easy to get wrong, but if you aren't going to subclass this and therefore aren't going to try to build that part, you're not going to build it wrong.

> And I'm not sure a docs recipe that's mostly code that's unnecessary for most uses, hiding the important and simple part, would be a great idea.
>> from collections import OrderedDict, Callable
>> 
>> class DefaultOrderedDict(OrderedDict):
>>     # Source: http://stackoverflow.com/a/6190500/562769
>>     def __init__(self, default_factory=None, *a, **kw):
>>         if (default_factory is not None and
>>            not isinstance(default_factory, Callable)):
>>             raise TypeError('first argument must be callable')
>>         OrderedDict.__init__(self, *a, **kw)
>>         self.default_factory = default_factory
>> 
>>     def __getitem__(self, key):
>>         try:
>>             return OrderedDict.__getitem__(self, key)
>>         except KeyError:
>>             return self.__missing__(key)
>> 
>>     def __missing__(self, key):
>>         if self.default_factory is None:
>>             raise KeyError(key)
>>         self[key] = value = self.default_factory()
>>         return value
>> 
>>     def __reduce__(self):
>>         if self.default_factory is None:
>>             args = tuple()
>>         else:
>>             args = self.default_factory,
>>         return type(self), args, None, None, self.items()
>> 
>>     def copy(self):
>>         return self.__copy__()
>> 
>>     def __copy__(self):
>>         return type(self)(self.default_factory, self)
>> 
>>     def __deepcopy__(self, memo):
>>         import copy
>>         return type(self)(self.default_factory,
>>                           copy.deepcopy(self.items()))
>> 
>>     def __repr__(self):
>>         return 'OrderedDefaultDict(%s, %s)' % (self.default_factory,
>>                                                OrderedDict.__repr__(self))
>> 
>> This seems to me both easy to get wrong and hard to discover. It also risks getting out of sync with updates to collections.
>> I'm wondering if this is sufficient justification to add OrderedCounter and OrderedDict to collections, either directly or as recipes.
> 
> Since one of them is already a recipe and the other one would probably not be useful as one, I don't think that's a great idea.
> 
> Adding OrderedDefaultDict to the module itself might be useful. Or, alternatively, put it in PyPI or ActiveState and add a link to it from the docs? (I'm pretty sure there are already implementations in both places, actually. Also, it would be nice to be able to point at a third-party implementation that works in 2.6+/3.2+ or whatever even if it only appears in the 3.6 docs.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151016/7cc66631/attachment.html>

From abarnert at yahoo.com  Sat Oct 17 04:08:15 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Fri, 16 Oct 2015 19:08:15 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
Message-ID: <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>

Actually, forget all that; it's even simpler.

At least in recent 3.x, the only thing wrong with inheriting from both types, assuming you put OrderedDict first, is the __init__ signature. So:

    class OrderedDefaultDict(OrderedDict, defaultdict):
        def __init__(self, default_factory=None, *a, **kw):
            OrderedDict.__init__(self, *a, **kw)
            self.default_factory = default_factory

More importantly, because __missing__ support is built into dict, despite the confusing docs for defaultdict, you don't really need defaultdict at all here:

    class OrderedDefaultDict(OrderedDict):
        def __init__(self, default_factory=None, *a, **kw):
            OrderedDict.__init__(self, *a, **kw)
            self.default_factory = default_factory
        def __missing__(self, key):
            self[key] = value = default_factory()
            return value

And either of these should work with 2.5+ (according to https://docs.python.org/2/library/stdtypes.html#dict that's when dict.__missing__ was added).

Sent from my iPhone

> On Oct 16, 2015, at 18:57, Andrew Barnert <abarnert at yahoo.com> wrote:
> 
> Sorry, sent too soon?
> 
> Sent from my iPhone
> 
>> On Oct 16, 2015, at 18:39, Andrew Barnert via Python-ideas <python-ideas at python.org> wrote:
>> 
>>> On Oct 16, 2015, at 03:36, Ian Foote <ian at feete.org> wrote:
>>> 
>>> I recently wanted to use an OrderedCounter and an OrderedDefaultDict. After a bit of googling I discovered that OrderedCounter is quite easy to implement:
>> 
>> This is already in the docs for OrderedDict, so it shouldn't have taken any googling.
>>> from collections import Counter, OrderedDict
>>> 
>>> class OrderedCounter(Counter, OrderedDict):
>>>      'Counter that remembers the order elements are first seen'
>>>      def __repr__(self):
>>>          return '%s(%r)' % (self.__class__.__name__,
>>>                             OrderedDict(self))
>>>      def __reduce__(self):
>>>          return self.__class__, (OrderedDict(self),)
>>> 
>>> from https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
>>> 
>>> Unfortunately an OrderedDefaultDict did not seem so easy. I did find http://stackoverflow.com/questions/6190331/can-i-do-an-ordered-default-dict-in-python which suggests:
>> 
>> Most of the trickiness here is handling None as a factory, which is only useful for subclasses that implement a custom __missing__
> 
> ? or that requires two-stage initialization, where it can initialize the base with None and then assign the factory later. Still not all that common, but it is a separate reason for the extra complexity besides a custom __missing__ that doesn't need the factory.
> 
>> To make something that works like you'd expect except for that part is a lot simpler.
> 
> Off the top of my head (untested, since I'm on my phone):
> 
> class DefaultOrderedDict(OrderedDict):
>     def __init__(self, default_factory, *a, **kw):
>         super().__init__(*a, **kw)
>         self.default_factory = default_factory
>     def __getitem__(self, key):
>         try:
>             return super().__getitem__(key)
>         except KeyError:
>             return self.__missing__(key)
>     def __missing__(self, key):
>         self[key] = value = self.default_factory()
>         return value
>     def __repr__(self):
>         return '{}({}, {})'.format(type(self).__name__, self.default_factory, dict.__repr__(self))
> 
> In fact, if you aren't worried about subclassing, or about being able to directly call or pass around a bound__missing__, you can make this even simpler.
> 
> I may be wrong about this being picklable/copyable without any extra help; if so, it still doesn't need the whole complicated "treat a None factory as if it doesn't exist" part.
> 
> Also, I think the SO recipe works in Python 2.4+, or whichever first added OrderedDict, while this requires 3.1 (although it should be obvious how to backport it, make sure that OrderedDict isn't an old-style class?).
> 
> Anyway, other than those issues, it seems pretty obvious, and I don't think there's any part of that functionality that's easy to get wrong. Handling a None factory in __missing__ and pickle/copy is easy to get wrong, but if you aren't going to subclass this and therefore aren't going to try to build that part, you're not going to build it wrong.
> 
>> And I'm not sure a docs recipe that's mostly code that's unnecessary for most uses, hiding the important and simple part, would be a great idea.
>>> from collections import OrderedDict, Callable
>>> 
>>> class DefaultOrderedDict(OrderedDict):
>>>     # Source: http://stackoverflow.com/a/6190500/562769
>>>     def __init__(self, default_factory=None, *a, **kw):
>>>         if (default_factory is not None and
>>>            not isinstance(default_factory, Callable)):
>>>             raise TypeError('first argument must be callable')
>>>         OrderedDict.__init__(self, *a, **kw)
>>>         self.default_factory = default_factory
>>> 
>>>     def __getitem__(self, key):
>>>         try:
>>>             return OrderedDict.__getitem__(self, key)
>>>         except KeyError:
>>>             return self.__missing__(key)
>>> 
>>>     def __missing__(self, key):
>>>         if self.default_factory is None:
>>>             raise KeyError(key)
>>>         self[key] = value = self.default_factory()
>>>         return value
>>> 
>>>     def __reduce__(self):
>>>         if self.default_factory is None:
>>>             args = tuple()
>>>         else:
>>>             args = self.default_factory,
>>>         return type(self), args, None, None, self.items()
>>> 
>>>     def copy(self):
>>>         return self.__copy__()
>>> 
>>>     def __copy__(self):
>>>         return type(self)(self.default_factory, self)
>>> 
>>>     def __deepcopy__(self, memo):
>>>         import copy
>>>         return type(self)(self.default_factory,
>>>                           copy.deepcopy(self.items()))
>>> 
>>>     def __repr__(self):
>>>         return 'OrderedDefaultDict(%s, %s)' % (self.default_factory,
>>>                                                OrderedDict.__repr__(self))
>>> 
>>> This seems to me both easy to get wrong and hard to discover. It also risks getting out of sync with updates to collections.
>>> I'm wondering if this is sufficient justification to add OrderedCounter and OrderedDict to collections, either directly or as recipes.
>> 
>> Since one of them is already a recipe and the other one would probably not be useful as one, I don't think that's a great idea.
>> 
>> Adding OrderedDefaultDict to the module itself might be useful. Or, alternatively, put it in PyPI or ActiveState and add a link to it from the docs? (I'm pretty sure there are already implementations in both places, actually. Also, it would be nice to be able to point at a third-party implementation that works in 2.6+/3.2+ or whatever even if it only appears in the 3.6 docs.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151016/b1602a47/attachment-0001.html>

From vgr255 at live.ca  Sun Oct 18 00:17:47 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sat, 17 Oct 2015 18:17:47 -0400
Subject: [Python-ideas] Discussion about a 'xor' keyword?
Message-ID: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>

You've probably read this title with skepticism. I know I would.
As the title says. I am curious as to engage in the discussion, and see what everyone's thoughts are on it. I know this has been discussed in the past several years ago ( https://mail.python.org/pipermail/python-3000/2007-December/011560.html ) and on various websites already. I've found myself writing code that would be simpler with a xor keyword or similar syntax, and this is why I'm suggesting this.
Here are my thoughts:
Since 'a or b' and 'a and b' return either a or b, what should 'a xor b' return? I have a few ideas, and no particular preference:
To be consistent with other binary operators, 'xor' should return one of its operands, IMO. So 'a xor b' should be a if a is True and b is False, and b if b is True and a is False. That much makes sense so far (to me). What I'm a bit less sure about is what to return in the case that both are True, or both False.
We already have a per-type definition of xor, and if we want to have only the boolean xor, we can easily do 'bool(a) ^ bool(b)' and so I feel this discussion should disregard the possibility of returning only a boolean value; we can already do that in a way that a keyword wouldn't help.
My thoughts on this would be to return either a or b if only one of them is True in a boolean context, else return None. None is a fairly common name already, and we could use it to say that the exclusive-or check returned no value (just like a function call). I don't think adding a new builtin name would be the way to go - adding a new keyword is already a big breaking change. If one of the two values is None already... well then I guess that's something to be fixed in the code? That's one of the things that should probably be discussed, should the people on this list be favorable to the idea.
I'm expecting there'll be some sort of BDFL pronouncement at some point, but I still want to have the discussion. If it gets rejected, I would at least be happy if some discussion took place to talk about it.
-Emanuel Barry 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151017/73cc5bf0/attachment.html>

From storchaka at gmail.com  Sun Oct 18 00:49:42 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 18 Oct 2015 01:49:42 +0300
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
Message-ID: <mvuja6$e28$1@ger.gmane.org>

On 18.10.15 01:17, Emanuel Barry wrote:
> My thoughts on this would be to return either a or b if only one of them
> is True in a boolean context, else return None. None is a fairly common
> name already, and we could use it to say that the exclusive-or check
> returned no value (just like a function call). I don't think adding a
> new builtin name would be the way to go - adding a new keyword is
> already a big breaking change. If one of the two values is None
> already... well then I guess that's something to be fixed in the code?
> That's one of the things that should probably be discussed, should the
> people on this list be favorable to the idea.

'and' and 'or' are short-circuit operators. But 'xor' needs to know 
values of both operands. Therefore no need in the xor operator, it can 
be a function.

     def xor(a, b):
         if a:
             if b:
                 return None
             else:
                 return a
         else:
             if b:
                 return b
             else:
                 return None

But I suppose that in most cases boolean result is enough, and it can be 
written as bool(a) != bool(b).



From emile at fenx.com  Sun Oct 18 00:50:35 2015
From: emile at fenx.com (Emile van Sebille)
Date: Sat, 17 Oct 2015 15:50:35 -0700
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
Message-ID: <mvujc1$eoj$1@ger.gmane.org>

Do you have additional arguments that weren't covered in the PEPs 
discussing the addition of an XOR operator?

Emile


On 10/17/2015 3:17 PM, Emanuel Barry wrote:
> You've probably read this title with skepticism. I know I would.
>
> As the title says. I am curious as to engage in the discussion, and see
> what everyone's thoughts are on it. I know this has been discussed in
> the past several years ago (
> https://mail.python.org/pipermail/python-3000/2007-December/011560.html ) and
> on various websites already. I've found myself writing code that would
> be simpler with a xor keyword or similar syntax, and this is why I'm
> suggesting this.
>
> Here are my thoughts:
>
> Since 'a or b' and 'a and b' return either a or b, what should 'a xor b'
> return? I have a few ideas, and no particular preference:
>
> To be consistent with other binary operators, 'xor' should return one of
> its operands, IMO. So 'a xor b' should be a if a is True and b is False,
> and b if b is True and a is False. That much makes sense so far (to me).
> What I'm a bit less sure about is what to return in the case that both
> are True, or both False.
>
> We already have a per-type definition of xor, and if we want to have
> only the boolean xor, we can easily do 'bool(a) ^ bool(b)' and so I feel
> this discussion should disregard the possibility of returning only a
> boolean value; we can already do that in a way that a keyword wouldn't help.
>
> My thoughts on this would be to return either a or b if only one of them
> is True in a boolean context, else return None. None is a fairly common
> name already, and we could use it to say that the exclusive-or check
> returned no value (just like a function call). I don't think adding a
> new builtin name would be the way to go - adding a new keyword is
> already a big breaking change. If one of the two values is None
> already... well then I guess that's something to be fixed in the code?
> That's one of the things that should probably be discussed, should the
> people on this list be favorable to the idea.
>
> I'm expecting there'll be some sort of BDFL pronouncement at some point,
> but I still want to have the discussion. If it gets rejected, I would at
> least be happy if some discussion took place to talk about it.
>
> -Emanuel Barry
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



From vgr255 at live.ca  Sun Oct 18 00:59:23 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sat, 17 Oct 2015 18:59:23 -0400
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <mvuja6$e28$1@ger.gmane.org>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>,
 <mvuja6$e28$1@ger.gmane.org>
Message-ID: <BLU172-W50F8E4330435AC6E1B29E2913C0@phx.gbl>




> To: python-ideas at python.org
> From: storchaka at gmail.com
> Date: Sun, 18 Oct 2015 01:49:42 +0300
> Subject: Re: [Python-ideas] Discussion about a 'xor' keyword?
> 
> On 18.10.15 01:17, Emanuel Barry wrote:
> > My thoughts on this would be to return either a or b if only one of them
> > is True in a boolean context, else return None. None is a fairly common
> > name already, and we could use it to say that the exclusive-or check
> > returned no value (just like a function call). I don't think adding a
> > new builtin name would be the way to go - adding a new keyword is
> > already a big breaking change. If one of the two values is None
> > already... well then I guess that's something to be fixed in the code?
> > That's one of the things that should probably be discussed, should the
> > people on this list be favorable to the idea.
> 
> 'and' and 'or' are short-circuit operators. But 'xor' needs to know 
> values of both operands. Therefore no need in the xor operator, it can 
> be a function.
> 
>      def xor(a, b):
>          if a:
>              if b:
>                  return None
>              else:
>                  return a
>          else:
>              if b:
>                  return b
>              else:
>                  return None
> 
> But I suppose that in most cases boolean result is enough, and it can be 
> written as bool(a) != bool(b).

I must admit I forgot about the short-circuit feature of the 'and' and 'or' operands while I was writing this. I agree it could be a function (I already use one for most cases), but was mostly suggesting the idea (rather than stating it as a hard necessity).
>Do you have additional arguments that weren't covered in the PEPs >discussing the addition of an XOR operator?>Emile
I must admit I did not see such PEP (I still don't, unless I'm not looking right). My bad in that case.
-Emanuel Barry
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151017/9baf2d65/attachment.html>

From breamoreboy at yahoo.co.uk  Sun Oct 18 01:28:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 18 Oct 2015 00:28:01 +0100
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <mvujc1$eoj$1@ger.gmane.org>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
 <mvujc1$eoj$1@ger.gmane.org>
Message-ID: <mvulj3$ch3$1@ger.gmane.org>

On 17/10/2015 23:50, Emile van Sebille wrote:
> Do you have additional arguments that weren't covered in the PEPs
> discussing the addition of an XOR operator?
>
> Emile
>

The only one I can find that refers specifically to xor is PEP 225 -- 
Elementwise/Objectwise Operators[1].  Which others have I missed?

[1] https://www.python.org/dev/peps/pep-0225/

>
> On 10/17/2015 3:17 PM, Emanuel Barry wrote:
>> You've probably read this title with skepticism. I know I would.
>>
>> As the title says. I am curious as to engage in the discussion, and see
>> what everyone's thoughts are on it. I know this has been discussed in
>> the past several years ago (
>> https://mail.python.org/pipermail/python-3000/2007-December/011560.html )
>> and
>> on various websites already. I've found myself writing code that would
>> be simpler with a xor keyword or similar syntax, and this is why I'm
>> suggesting this.
>>
>> Here are my thoughts:
>>
>> Since 'a or b' and 'a and b' return either a or b, what should 'a xor b'
>> return? I have a few ideas, and no particular preference:
>>
>> To be consistent with other binary operators, 'xor' should return one of
>> its operands, IMO. So 'a xor b' should be a if a is True and b is False,
>> and b if b is True and a is False. That much makes sense so far (to me).
>> What I'm a bit less sure about is what to return in the case that both
>> are True, or both False.
>>
>> We already have a per-type definition of xor, and if we want to have
>> only the boolean xor, we can easily do 'bool(a) ^ bool(b)' and so I feel
>> this discussion should disregard the possibility of returning only a
>> boolean value; we can already do that in a way that a keyword wouldn't
>> help.
>>
>> My thoughts on this would be to return either a or b if only one of them
>> is True in a boolean context, else return None. None is a fairly common
>> name already, and we could use it to say that the exclusive-or check
>> returned no value (just like a function call). I don't think adding a
>> new builtin name would be the way to go - adding a new keyword is
>> already a big breaking change. If one of the two values is None
>> already... well then I guess that's something to be fixed in the code?
>> That's one of the things that should probably be discussed, should the
>> people on this list be favorable to the idea.
>>
>> I'm expecting there'll be some sort of BDFL pronouncement at some point,
>> but I still want to have the discussion. If it gets rejected, I would at
>> least be happy if some discussion took place to talk about it.
>>
>> -Emanuel Barry

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From vgr255 at live.ca  Sun Oct 18 01:53:22 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sat, 17 Oct 2015 19:53:22 -0400
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <mvulj3$ch3$1@ger.gmane.org>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>,
 <mvujc1$eoj$1@ger.gmane.org>, <mvulj3$ch3$1@ger.gmane.org>
Message-ID: <BLU172-W21D1BB836B4F3DB8E8C24913C0@phx.gbl>



> To: python-ideas at python.org
> From: breamoreboy at yahoo.co.uk
> Date: Sun, 18 Oct 2015 00:28:01 +0100
> Subject: Re: [Python-ideas] Discussion about a 'xor' keyword?
> 
> On 17/10/2015 23:50, Emile van Sebille wrote:
> > Do you have additional arguments that weren't covered in the PEPs
> > discussing the addition of an XOR operator?
> >
> > Emile
> >
> 
> The only one I can find that refers specifically to xor is PEP 225 -- 
> Elementwise/Objectwise Operators[1].  Which others have I missed?
> 
> [1] https://www.python.org/dev/peps/pep-0225/
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

That's also what I found, but it very clearly refers to the various operators which are type-overridable, so I think it doesn't really matter here (as I stated in my original post).
-Emanuel Barry 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151017/9ee1b515/attachment.html>

From pyideas at rebertia.com  Sun Oct 18 02:40:54 2015
From: pyideas at rebertia.com (Chris Rebert)
Date: Sat, 17 Oct 2015 17:40:54 -0700
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
Message-ID: <CAMZYqRSpF8ty3DANKD-nE1cAScLZmBOU-A0ma1FA9hPhq_eBRg@mail.gmail.com>

On Sat, Oct 17, 2015 at 3:17 PM, Emanuel Barry <vgr255 at live.ca> wrote:
> You've probably read this title with skepticism. I know I would.
>
> As the title says. I am curious as to engage in the discussion, and see what
> everyone's thoughts are on it. I know this has been discussed in the past
> several years ago (
> https://mail.python.org/pipermail/python-3000/2007-December/011560.html )
> and on various websites already. I've found myself writing code that would
> be simpler with a xor keyword or similar syntax, and this is why I'm
> suggesting this.

There were a couple times in the past when I found myself wanting a
"xor" operator for clarity/aesthetics. In the most recent case, it was
to express a validity condition for rows in an SQL DB table which was
acting like a tagged union, so only 1 of a subset of the columns in a
given row could be non-NULL. That case would have been served just
fine by a simple version of xor that always gave a bool result.

However, it ultimately turned out that what I really wanted was an
"exactly 1 of these N subconditions must be true" operation, which
doesn't correspond to conventional definitions of XOR (they instead
calculate a parity bit; see http://mathworld.wolfram.com/XOR.html ).
So xor would only truly have been useful in cases when N
just-so-happens to be 2, and therefore I no longer hunger for a "xor"
keyword.

I would be interested to hear what your concrete use cases for xor are.

Cheers,
Chris
--
http://chrisrebert.com

From pyideas at rebertia.com  Sun Oct 18 02:50:36 2015
From: pyideas at rebertia.com (Chris Rebert)
Date: Sat, 17 Oct 2015 17:50:36 -0700
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
Message-ID: <CAMZYqRSm2wYUUxcodyvJzf76223rrcG7U1tOgsEjX-gnw7vkNg@mail.gmail.com>

On Sat, Oct 17, 2015 at 3:17 PM, Emanuel Barry <vgr255 at live.ca> wrote:
> You've probably read this title with skepticism. I know I would.
>
> As the title says. I am curious as to engage in the discussion, and see what
> everyone's thoughts are on it. I know this has been discussed in the past
> several years ago (
> https://mail.python.org/pipermail/python-3000/2007-December/011560.html )
> and on various websites already.

This 2009 discussion was more vigorous:
https://mail.python.org/pipermail/python-list/2009-July/543956.html

Cheers,
Chris

From emile at fenx.com  Sun Oct 18 03:05:09 2015
From: emile at fenx.com (Emile van Sebille)
Date: Sat, 17 Oct 2015 18:05:09 -0700
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <mvulj3$ch3$1@ger.gmane.org>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>
 <mvujc1$eoj$1@ger.gmane.org> <mvulj3$ch3$1@ger.gmane.org>
Message-ID: <mvur8a$m9o$1@ger.gmane.org>

On 10/17/2015 4:28 PM, Mark Lawrence wrote:
> On 17/10/2015 23:50, Emile van Sebille wrote:
>> Do you have additional arguments that weren't covered in the PEPs
>> discussing the addition of an XOR operator?
>>
>> Emile
>>
>
> The only one I can find that refers specifically to xor is PEP 225 --
> Elementwise/Objectwise Operators[1].  Which others have I missed?
>
> [1] https://www.python.org/dev/peps/pep-0225/
>

It's also touched upon in 207.  I thought I saw it mentioned in at least 
one other, but that may well have been referring to the bitwise operator.

Emile




From vgr255 at live.ca  Sun Oct 18 03:54:54 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sat, 17 Oct 2015 21:54:54 -0400
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <CAMZYqRSm2wYUUxcodyvJzf76223rrcG7U1tOgsEjX-gnw7vkNg@mail.gmail.com>
References: <BLU172-W5CA995CD0CFA1649B18FE913C0@phx.gbl>,
 <CAMZYqRSm2wYUUxcodyvJzf76223rrcG7U1tOgsEjX-gnw7vkNg@mail.gmail.com>
Message-ID: <BLU172-W23B11B9B95111AA0D0FBFD913B0@phx.gbl>



> Date: Sat, 17 Oct 2015 17:50:36 -0700
> Subject: Re: [Python-ideas] Discussion about a 'xor' keyword?
> From: pyideas at rebertia.com
> To: vgr255 at live.ca
> CC: python-ideas at python.org
> 
> 
> This 2009 discussion was more vigorous:
> https://mail.python.org/pipermail/python-list/2009-July/543956.html

Having a look right now. Looks interesting.

> I would be interested to hear what your concrete use cases for xor are.

I have a few from the top of my head (not backed by code right now, will post some when I feel like digging into it).
For an IRC game bot I work on, we have a special message if someone has two targets available, but a different one if they have only one - although it's the same message, and we made a complicated conditional just for that. A xor keyword would have saved us that trouble.
In a library I dug in some time ago (I didn't make it), an exception was raised if two parameters had a True (or False) value - one of them had to be True. I (probably naively) think that some more libraries do that.
My friend in college is learning programming, and for his projects he says he really wants to see a 'xor' keyword in, as he has to do some mangling to get what he wants. I'm unaware what his work consists of, however.
------------------
'xor' would by no means be as common as any existing operator, but I believe it is a good addition to consider. Another upside is that keywords only evaluate the value once, so chaining two wouldn't re-evaluate the value (in the case of a function/property).
>From that 2009 discussion, most of the rejections focused on "You can coerce to bool before then perform the xor operations", but I'm considering not coercing to bool here, so I think it is different enough to the other discussion to (at least in my opinion) bring it up one more time. If I wanted boolean coercion, I'd just do it myself.
Also, the fact that other people have considered the idea before makes me think that it's not as rare a case as some may think.
-Emanuel Barry 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151017/337f0d3e/attachment.html>

From abarnert at yahoo.com  Sun Oct 18 07:18:21 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 18 Oct 2015 05:18:21 +0000 (UTC)
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <BLU172-W23B11B9B95111AA0D0FBFD913B0@phx.gbl>
References: <BLU172-W23B11B9B95111AA0D0FBFD913B0@phx.gbl>
Message-ID: <506401735.2106125.1445145501455.JavaMail.yahoo@mail.yahoo.com>

First, it seems like if this operator were to exist, it should be defined in terms of "and", "or", and "not" in one of the usual ways: "a and not b or b and not a", or "(a or b) and not (a and b)".

Of course this means it returns False, not None, if both values are falsey or both truthy, but I think that's what you'd want anyway. While you sometimes do want the value of "a and b" or "a or b", it's far more common to just want the truthiness of that value (as in "if a or b:"), and, while None is falsey, it's not the canonical falsey value.

But anyway, as my replies below explain, I don't think it should exist.

On Saturday, October 17, 2015 6:54 PM, Emanuel Barry <vgr255 at live.ca> wrote:

>I have a few from the top of my head (not backed by code right now, will post some when I feel like digging into it).


But all of these ideas are just as well served by a function as by a new keyword.

There are two reasons for "and" and "or" to be operators.

First, functions can't short-circuit. That's obviously not relevant to "xor", because by definition, it has to evaluate both operands.

Second, function calls chained in a complicated expression can look ugly compared to operators?compare "or(and(a, b), and(c, d))" to "a and b or c and d". But that isn't relevant to "xor" either. I think most readers would find "a xor b xor c xor d xor e" confusing and have to work through its meaning?and, in fact, they'd find xor(a, xor(b, xor(c, xor(d, e))))" (or the reverse associativity) easier to understand. Worse, I think some readers would _think_ they understand what it means and be wrong (exactly one, or some but not all, or exactly half). And, more importantly, I don't think they'd ever want what it does. It's pretty common to want to check that one or all of three values are true, but when would you ever want to check that an odd number of those three values are true? (You might occasionally want to check that an odd number of the possibly-only-known-at-runtime members of some iterable are true, but for that, you'd use a function akin to "any" and "all", maybe called "parity".)

>'xor' would by no means be as common as any existing operator, but I believe it is a good addition to consider. Another upside is that keywords only evaluate the value once, so chaining two wouldn't re-evaluate the value (in the case of a function/property).


Again, I don't think you'd want to chain them. But, even if you did, functions also only evaluate their arguments once, so there's no advantage here anyway.

For example, try this:


    def xor(a, b):
        return a and not b or b and not a

    xor((print('spam'), 2)[1], (print('eggs'), 3)[1])

Of course "spam" and "eggs" only get printed once, because what gets passed to "xor" isn't the expression "(print('spam'), 2)[1]", it's the value of that expression, "2".

And likewise, xor(a, xor(b, c)) isn't going to cause b or c to get evaluated twice. And of course neither would "parity((a, b, c))" or "some((a, b, c))", etc.

And I'm not sure what you mean by "property" here.

>From that 2009 discussion, most of the rejections focused on "You can coerce to bool before then perform the xor operations", but I'm considering not coercing to bool here, so I think it is different enough to the other discussion to (at least in my opinion) bring it up one more time. If I wanted boolean coercion, I'd just do it myself.


But you can also just do it yourself _without_ boolean coercion, so how is this any different?

From j.wielicki at sotecware.net  Sun Oct 18 13:32:35 2015
From: j.wielicki at sotecware.net (Jonas Wielicki)
Date: Sun, 18 Oct 2015 13:32:35 +0200
Subject: [Python-ideas] Add appdirs module to stdlib
In-Reply-To: <CAN8d9gmN3J+G+jetardW5EsyDqyFfJEopHx8Zs7d2mZhzjKH8w@mail.gmail.com>
References: <CAN8d9gmN3J+G+jetardW5EsyDqyFfJEopHx8Zs7d2mZhzjKH8w@mail.gmail.com>
Message-ID: <56238353.5020807@sotecware.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 01.09.2015 10:00, Philipp A. wrote:
> When defining a place for config files, cache files, and so on,
> people usually hack around in a OS-dependent, misinformed, and
> therefore wrong way.
> 
> Thanks to the tempfile API we at least don?t see people hardcoding
> /tmp/ too much.
> 
> There is a beautiful little module that does things right and is
> easy to use: appdirs <https://pypi.python.org/pypi/appdirs>
> 
> TI think this is a *really* good candidate for the stdlib since
> this functionality is useful for everything that needs a cache or
> config (so not only GUI and CLI applications, but also scripts that
> download and cache stuff from the internet for faster re-running)
> 
> probably we should build the API around pathlib, since i found
> myself not touching os.path with a barge pole since pathlib
> exists.
> 
> i?ll write a PEP about this soon :)

Is anyone working on this?

I would really like to see the functionality discussed in this thread.
I have no deep knowledge about Windows, MacOS or FreeBSD though, which
would make it hard for me to take this on my own.

If we could get together a team with knowledgeable people from each
platform, that would be great.

best regards,
jwi

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJWI4NSAAoJEMBiAyWXYliKfdwP/Rgi2w2ZyZA2tfFcjZMxh6ox
niJYz3Uq2Jcci7VJlvPIObu9ioS1pdc/KdLIqMCO9BZzkm0+V/d9/6TLecBtE73s
MDFBkVbBooK/km2xx1HCnfj3vT+Eh8+blPDwNABRKzVdGnuGLfZdQHGvw16NDYPI
zKcBboS8Xp1F7Njwdm3ozlW34A5ApLMZV0D2WrtrhZk3PBTbjNVBmKqVdNrBzsyO
F04F/8kPLoANBj4tBuI9O1KjnMbfwCN7snbu6LlDwhT8pAGW6kpTzOjxRZZdbjD6
NH6NyLbTjIWgVP8JgQ766vGXAwiatrd0ECQWoYHXgAhuP//KVgfmTBpIqoPPME7Y
eGmM8XclzI4FS5ud87bgf9Zgr4xZcycX5J/lcXIp/ZPAajHfyWxVbJPHZDocNw7V
ZDlEEfIfkbU4aT5faQutPF95eZgwXOxloOSNZI+u7a8Ke8iaTjeoawtksBlVzYbb
6MOLP6xdVXc+vg/9FFwbiDrLuOQiGv2YYENY2Lj3lMw6jvj6GL31XDKhVcyQQeJn
yE1ylKwrBQzq2hw57vLwCJD9Tdn0qgOjA74pgl7SHPvkFjNpPXre5YkMTUxI13jI
MGof+oR32WHLaA+Kt0dByzNSJPSdzubc4/zKKAT49G96znXua2MlM1nDdrRmhbHo
uihjjtEQzIZ1O/uNMV2n
=UtH/
-----END PGP SIGNATURE-----

From vgr255 at live.ca  Sun Oct 18 14:45:35 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sun, 18 Oct 2015 08:45:35 -0400
Subject: [Python-ideas] Add appdirs module to stdlib
In-Reply-To: <56238353.5020807@sotecware.net>
References: <CAN8d9gmN3J+G+jetardW5EsyDqyFfJEopHx8Zs7d2mZhzjKH8w@mail.gmail.com>,
 <56238353.5020807@sotecware.net>
Message-ID: <BLU172-W232E9F197CEEDA3197F5F1913B0@phx.gbl>

> To: python-ideas at python.org
> From: j.wielicki at sotecware.net
> Date: Sun, 18 Oct 2015 13:32:35 +0200
> Subject: Re: [Python-ideas] Add appdirs module to stdlib
> > [snip]> > Is anyone working on this?
> 
> I would really like to see the functionality discussed in this thread.
> I have no deep knowledge about Windows, MacOS or FreeBSD though, which
> would make it hard for me to take this on my own.
> 
> If we could get together a team with knowledgeable people from each
> platform, that would be great.
> 
> best regards,
> jwi
> > [snip]

As someone who had to work with temporary files in the past, this appeals to me. I develop on Windows, and the program was not meant to be used on non-Windows platforms, so I used a lot of platform-dependant hacks I wish I could not use. I am by no means a pro, but I would still like to help with this.
Cheers,-Emanuel Barry 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151018/718533a5/attachment.html>

From guido at python.org  Sun Oct 18 17:00:51 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 18 Oct 2015 08:00:51 -0700
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <506401735.2106125.1445145501455.JavaMail.yahoo@mail.yahoo.com>
References: <BLU172-W23B11B9B95111AA0D0FBFD913B0@phx.gbl>
 <506401735.2106125.1445145501455.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAP7+vJ+h3rsZdxJhHbynUH60YVSVVytNH3NyMANACszbCCzdXw@mail.gmail.com>

This just isn't going to happen. The combination of the expense of a new
keyword, the lack of use cases that can't be done just as easily with a
function, and the murky definition make it a non-started.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151018/0b4a3455/attachment.html>

From vgr255 at live.ca  Sun Oct 18 17:03:43 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sun, 18 Oct 2015 11:03:43 -0400
Subject: [Python-ideas] Discussion about a 'xor' keyword?
In-Reply-To: <CAP7+vJ+h3rsZdxJhHbynUH60YVSVVytNH3NyMANACszbCCzdXw@mail.gmail.com>
References: <BLU172-W23B11B9B95111AA0D0FBFD913B0@phx.gbl>
 <506401735.2106125.1445145501455.JavaMail.yahoo@mail.yahoo.com>,
 <CAP7+vJ+h3rsZdxJhHbynUH60YVSVVytNH3NyMANACszbCCzdXw@mail.gmail.com>
Message-ID: <BLU172-W44F60AB8F9B7A073392634913B0@phx.gbl>

I figured as much, thanks anyway.
Cheers,-Emanuel Barry

From: guido at python.org
Date: Sun, 18 Oct 2015 08:00:51 -0700
Subject: Re: [Python-ideas] Discussion about a 'xor' keyword?
To: abarnert at yahoo.com
CC: vgr255 at live.ca; python-ideas at python.org

This just isn't going to happen. The combination of the expense of a new keyword, the lack of use cases that can't be done just as easily with a function, and the murky definition make it a non-started.
-- 
--Guido van Rossum (python.org/~guido)
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151018/f213f88e/attachment.html>

From srkunze at mail.de  Tue Oct 20 20:14:31 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 20 Oct 2015 20:14:31 +0200
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
Message-ID: <56268487.9050405@mail.de>

On 17.10.2015 04:08, Andrew Barnert via Python-ideas wrote:
> Actually, forget all that; it's even simpler.
>
> At least in recent 3.x, the only thing wrong with inheriting from both 
> types, assuming you put OrderedDict first, is the __init__ signature. So:
>
>     class OrderedDefaultDict(OrderedDict, defaultdict):
>         def __init__(self, default_factory=None, *a, **kw):
>             OrderedDict.__init__(self, *a, **kw)
>             self.default_factory = default_factory

Not saying that inheritance is a bad thing but to me It seems to me that 
ordering and default values should be orthogonal aspects of the standard 
dict.

Just as Sandi described it here: https://www.youtube.com/watch?v=29MAL8pJImQ

Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151020/44c5cb35/attachment.html>

From rosuav at gmail.com  Tue Oct 20 23:40:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 21 Oct 2015 08:40:16 +1100
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <56268487.9050405@mail.de>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
 <56268487.9050405@mail.de>
Message-ID: <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>

On Wed, Oct 21, 2015 at 5:14 AM, Sven R. Kunze <srkunze at mail.de> wrote:
> Not saying that inheritance is a bad thing but to me It seems to me that
> ordering and default values should be orthogonal aspects of the standard
> dict.
>
> Just as Sandi described it here: https://www.youtube.com/watch?v=29MAL8pJImQ

Yeah... that video is absolutely correct... for Ruby. In Python, you
can use multiple inheritance to do that in the exactly obvious way.

class EchoRandomHouse(EchoHouse, RandomHouse): pass

When I got to the bit in that video where she says that inheritance
paints you into a corner, I went and reimplemented her example code in
Python, and it's flawless. It doesn't even matter which order you put
the two superclasses, as there's no conflict.

ChrisA

From carl at oddbird.net  Tue Oct 20 23:56:59 2015
From: carl at oddbird.net (Carl Meyer)
Date: Tue, 20 Oct 2015 15:56:59 -0600
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
Message-ID: <5626B8AB.5040806@oddbird.net>

On 10/20/2015 03:40 PM, Chris Angelico wrote:
> On Wed, Oct 21, 2015 at 5:14 AM, Sven R. Kunze <srkunze at mail.de> wrote:
>> Not saying that inheritance is a bad thing but to me It seems to me that
>> ordering and default values should be orthogonal aspects of the standard
>> dict.
>>
>> Just as Sandi described it here: https://www.youtube.com/watch?v=29MAL8pJImQ
> 
> Yeah... that video is absolutely correct... for Ruby. In Python, you
> can use multiple inheritance to do that in the exactly obvious way.
> 
> class EchoRandomHouse(EchoHouse, RandomHouse): pass
> 
> When I got to the bit in that video where she says that inheritance
> paints you into a corner, I went and reimplemented her example code in
> Python, and it's flawless. It doesn't even matter which order you put
> the two superclasses, as there's no conflict.

I think you missed her real point, which applies both to Python and
Ruby. In her presentation, it's obscured a bit by too much discussion of
"one side or the other" code duplication (which can be "solved" with
multiple inheritance). It takes her a couple minutes more to get to the
real point, which starts at the slide "inheritance is for
specialization, not for sharing code."

One symptom of the problem is that using multiple inheritance this way
doesn't scale: the number of leaf subclasses you need grows
geometrically with the number of behavior knobs. Composition with
strategy objects doesn't have this issue.

Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151020/29a12c33/attachment.sig>

From rosuav at gmail.com  Wed Oct 21 00:50:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 21 Oct 2015 09:50:21 +1100
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <5626B8AB.5040806@oddbird.net>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
 <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
Message-ID: <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>

On Wed, Oct 21, 2015 at 8:56 AM, Carl Meyer <carl at oddbird.net> wrote:
> I think you missed her real point, which applies both to Python and
> Ruby. In her presentation, it's obscured a bit by too much discussion of
> "one side or the other" code duplication (which can be "solved" with
> multiple inheritance). It takes her a couple minutes more to get to the
> real point, which starts at the slide "inheritance is for
> specialization, not for sharing code."

Sure, but a dictionary with a default handler _is_ a form of
specialization - as is a dictionary that preserves order. Both of them
behave absolutely identically to a regular dict when you set
something, retrieve it, iterate over them, etc, etc, etc. She
recommends a massive superclass that's capable of any form of
injection; using inheritance allows the dict type to be broadly
unaware of the modified dictionaries that exist.

Suppose I write a SortedDict that behaves exactly the way any other
dictionary does, but when you iterate over its keys, you get them in
order. I should be able to do this without tampering with the original
dict type. In the composition model, I would need to appeal for an
"IterationOrder" feature to be added to the base dict; using
inheritance, it's as simple as:

class SortedDict(dict):
    def __iter__(self):
        yield from sorted(self.keys())

To me, this is the biggest benefit of inheritance: you do NOT have to
predict what someone might want to do. I can subclass someone else's
object and change how it works.

> One symptom of the problem is that using multiple inheritance this way
> doesn't scale: the number of leaf subclasses you need grows
> geometrically with the number of behavior knobs. Composition with
> strategy objects doesn't have this issue.

Sure it does... but nobody needs to know about _all_ the leaf
subclasses. How many subclasses of object are there in Python?

>>> len(object.__subclasses__())
122

And that in a bare interactive Py3, without importing anything fancy.
('import decimal' raises that to 159, for instance.)

Composition has its place, don't get me wrong. But inheritance isn't
the attractive nuisance she makes it out to be. It works just fine.

ChrisA

From chris.barker at noaa.gov  Wed Oct 21 01:19:53 2015
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Tue, 20 Oct 2015 16:19:53 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
 <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
Message-ID: <2900444672847875025@unknownmsgid>

It seems to me that
>> ordering and default values should be orthogonal aspects of the standard
>> dict.

Exactly -- which makes it a perfect candidate for multiple
inheritance, I.e. "Mixins"

-CHB

From steve at pearwood.info  Wed Oct 21 03:50:26 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 21 Oct 2015 12:50:26 +1100
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
Message-ID: <20151021015025.GM3813@ando.pearwood.info>

On Wed, Oct 21, 2015 at 09:50:21AM +1100, Chris Angelico wrote:

> To me, this is the biggest benefit of inheritance: you do NOT have to
> predict what someone might want to do. I can subclass someone else's
> object and change how it works.

I think this is wrong. I think that in general you need to have a fairly 
good idea of the internal workings of the class before you can inherit 
from it. Otherwise, how do you know what methods need to be overwritten?

Take dict. One nuisance with inheriting from dict is that it isn't 
sufficient to override __setitem__, you also have to override update and 
clear as well. And possibly others -- the exact set of which methods 
depend on which other methods are not documented in the dict API. Given 
an arbitrary class, how can you possibly tell which methods you need to 
override, or even which methods are *safe* to override?


[...]
> Composition has its place, don't get me wrong. But inheritance isn't
> the attractive nuisance she makes it out to be. It works just fine.

It doesn't work "just fine", it has risks and annoyances, and multiple 
inheritance even more so. Classes have problems.

See Jack Diederich's talk "Stop Writing Classes", or at least stop 
writing *stupid* classes:

http://eev.ee/blog/2013/03/03/the-controller-pattern-is-awful-and-other-oo-heresy/

As far as multiple inheritance, there are real issues with MI that 
aren't solved by coming up with a nifty solution to the diamond problem. 
See Michele Simionato's series of essays on super, multiple inheritance, 
mixins and traits:

http://www.artima.com/weblogs/viewpost.jsp?thread=246488


Clearly classes are useful, but they aren't an unalloyed good thing, and 
inheriting from them has its issues.


-- 
Steve

From mehaase at gmail.com  Wed Oct 21 04:17:24 2015
From: mehaase at gmail.com (Mark E. Haase)
Date: Tue, 20 Oct 2015 22:17:24 -0400
Subject: [Python-ideas] PEP-505: Draft (Coalescing Operators)
Message-ID: <CALb0Rk6k+Sh-63SByqAWJ7Hb9NYNzOuoO8S2rBATbf0GSEKTtQ@mail.gmail.com>

I've finally finished a reasonably complete draft of PEP-505. It's taken me
longer than I expected due to work and also a new addition to our family :)
I tried to keep the PEP humorous; but please don't mistake silliness for a
lack of seriousness.

Let's try to keep separate two different debates: (1) the utility of the
proposed operators (2) how to spell them. Arguing about #2 is a waste of
time if #1 is a non-starter. If we can achieve some consensus on #1, then
with Guido's consent, I think the next logical step would be to conduct a
survey of opinions about proposed spellings, just as the authors of PEP-308
did. The survey is a data-gathering exercise, of course, not a democratic
vote. The PEP has more details on that.

I envision the PEP as being severable: it might be accepted in whole, in
part, or not at all. For this reason, there are quite a few alternatives
proposed, and some of them are mutually conflicting. The conflicts aren't a
problem unless and until any parts of the proposal get the green light.
(Again, similar to PEP-308.)

https://www.python.org/dev/peps/pep-0505/

-- 
Mark E. Haase
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151020/2dd3c03a/attachment.html>

From random832 at fastmail.com  Wed Oct 21 04:58:32 2015
From: random832 at fastmail.com (Random832)
Date: Tue, 20 Oct 2015 22:58:32 -0400
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
 <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
Message-ID: <87eggporl3.fsf@fastmail.com>

Chris Angelico <rosuav at gmail.com> writes:

> On Wed, Oct 21, 2015 at 8:56 AM, Carl Meyer <carl at oddbird.net> wrote:
> Sure, but a dictionary with a default handler _is_ a form of
> specialization - as is a dictionary that preserves order. Both of them
> behave absolutely identically to a regular dict when you set
> something, retrieve it, iterate over them, etc, etc, etc. She
> recommends a massive superclass that's capable of any form of
> injection; using inheritance allows the dict type to be broadly
> unaware of the modified dictionaries that exist.

Even considering that OrderedDict (or a _proper_ SortedDict - other
languages' equivalent class doesn't require hashable keys because it
stores items as a sorted list) requires a complete overhaul of how the
dictionary stores items?

Really, from an implementation standpoint, it seems like a bad idea to
even inherit in this case.


From guido at python.org  Wed Oct 21 05:36:24 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Oct 2015 20:36:24 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <20151021015025.GM3813@ando.pearwood.info>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
 <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <20151021015025.GM3813@ando.pearwood.info>
Message-ID: <CAP7+vJJBLV8Bpaoq-Hc+_65M0NSdvNQdCayO8q7vHwGpuVqJUQ@mail.gmail.com>

On Tue, Oct 20, 2015 at 6:50 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Wed, Oct 21, 2015 at 09:50:21AM +1100, Chris Angelico wrote:
>
> > To me, this is the biggest benefit of inheritance: you do NOT have to
> > predict what someone might want to do. I can subclass someone else's
> > object and change how it works.
>
> I think this is wrong. I think that in general you need to have a fairly
> good idea of the internal workings of the class before you can inherit
> from it. Otherwise, how do you know what methods need to be overwritten?
>

Right. Chris's thinking recalls the reason why inheritance at some became
popular (I guess it was in the '90s). Steven explains (in the part that
I've cut) why many experts have soured on it quite a bit.

Personally, I happen to think that inheritance is often useful when a
number of classes are designed together (typically all at once and
belonging to the same library) and also occasionally when a base class is
explicitly and carefully designed to be inherited (there are beautiful
things you can do with the template pattern, for example). But inheriting
from an implementation that wasn't designed with your use case in mind is
often asking for trouble -- if not now, then in the future when a
refactored implementation is released.

You might interject, that's the fault of the implementation refactoring --
they didn't properly think about interface compatibility. But while it's
usually easy enough to keep an interface compatible where it's just the
user calling methods on the implementation, the "interface" presented by
subclassing is much more complex -- you would have to specify exactly which
method's implementation calls which other method, and you'd also have to
ensure that the object is in a sane state when it calls that other method,
because it *could* be the case that the latter is overridden by a subclass.
It's terribly fragile, and better avoided.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151020/6baa9b94/attachment.html>

From brenbarn at brenbarn.net  Wed Oct 21 07:54:25 2015
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Tue, 20 Oct 2015 22:54:25 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <20151021015025.GM3813@ando.pearwood.info>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <20151021015025.GM3813@ando.pearwood.info>
Message-ID: <56272891.2050500@brenbarn.net>

On 2015-10-20 18:50, Steven D'Aprano wrote:
> Take dict. One nuisance with inheriting from dict is that it isn't
> sufficient to override __setitem__, you also have to override update and
> clear as well. And possibly others -- the exact set of which methods
> depend on which other methods are not documented in the dict API. Given
> an arbitrary class, how can you possibly tell which methods you need to
> override, or even which methods are*safe*  to override?

	I've always considered that (and related problems) to be one of 
Python's warts.  And, as you say, it's a documentation flaw.

	It's true that it's easy to write classes with poorly documented APIs, 
which makes them hard to extend because you don't know how they work. 
But I don't think that means subclassing is not a good idea.  It means 
writing classes with clearly-specified APIs is a good idea.

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown

From victor.stinner at gmail.com  Wed Oct 21 17:43:57 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 21 Oct 2015 17:43:57 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
Message-ID: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>

Hi,

I would like to share with you an idea to try to optimize CPython.

My previous attempts to optimize CPython failed because they changed the
language semantic. For example, it's not possible to replace len('abc')
with 3 because it is technically possible to override the builtin len()
function.

I propose to add a new "FAT" mode to Python to allow to add specialized
bytecode to function with guards. Example:

    >>> import builtins
    >>> def f(): return len("abc")
    ...
    >>> f()
    3

    >>> def g(): return 3
    ...
    >>> i=f.add(g)   # i is the index of the new specialized bytecode
    >>> f.add_dict_key_guard(i, builtins.__dict__, 'len')
    >>> f()
    3

    >>> builtins.len = lambda obj: "len"
    >>> f()
    'len'

In this example, the f() function gets a fast version (g()) returning
directly 3 instead of calling len("abc"). The optimization is disabled
when the builtin len() function is modified (when
builtins.__dict__['len'] is modified). (The example is not complete, we
need a second guard on the current global namespace, but I wanted to
write a short example.)

With such machinery, it becomes possible to implement interesting
optimizations. Some examples:

* inline a function instead of calling it: calling a Python function is
  "expensive". For example, if you inline the _get_sep() call in
  posixpath.isabs(), the function becomes 20% faster.

* move invariant out of the loop: classic micro-optimization done
  manually. For example, set "append = obj.append" before the loop and
  then call "append(item)" instead of "obj.append(item)" in the loop
  body. On a microbenchmark, it's between 5% (1 item) and 30% faster (10
  items or more)

* call pure functions at the compilation. A pure function has no side
  effect. Example: len(str).


Technical Challenges
====================

The problem is the cost of "guards". A guard is a check done before
calling a specialized function. Example of guards:

* Type of a function parameter
* Watch a dictionary key
* Watch a function (especially it's __code__ attribute)

Watching a dictionary key is a very important guard to disable an
optimization when a builtin function is modified, when a function is
replaced in a namespace, etc. The problem is that a dictionary lookup is
not cheap: we have to get the hash value of the key, browse the hash
table to find the bucket which may require multiple iterations, then we
have to compare the key value, etc.

To have faster guard, I propose to create a subtype of dict which
provides a global "version" of the dictionary, incremented at each
modification (create a new key, modify a key, delete a key) and a
"version" per key=>value mapping, incremented each time
that the mapping is modified. The lookup can be avoided when the
dictionary is not modified. If a different key is modified, we need a
lookup, but only once (we store the new global version to avoid a lookup
at the next guard check).


Limitations
===========

Specialized bytecode is build at compilation, *not* at runtime: it's not a
just-in-time (JIT) compiler. A JIT can implement even more efficient
optimizations and can use better guards. Please continue to use PyPy for
best performances! ;-)

The name "FAT" comes from the fact that multiple bytecode versions of a
function are stored in the memory, so a larger memory footprint
and larger .pyc files on disk can be expected.

Checking guards can also be more expensive than the optimization of the
specialized bytecode. The optimizer should use an heuristic to decide if
it's worth to use a specialized bytecode or not depending on the theoric
speeup and the cost of guards.

My motivation to write FAT Python is that CPython remains the reference
implementation where new features are implemented, and other Python
implementations still have issues with C extensions. JIT also has some
issues, like longer startup time, slow warmup and memory footprint (this
one may also be an issue of FAT Python!).


Implementation
==============

I wrote a proof-of-concept of my idea:

    https://hg.python.org/sandbox/fatpython/

It's a fork of CPython 3.6. Try it with:

    hg clone https://hg.python.org/sandbox/fatpython/
    cd fatpython
    ./configure && make
    ./python -F

See bench.py, Lib/posixpath.py (isabs) and Lib/test/test_fat.py
for examples of optimizations.

The command line -F flag enables the FAT mode. By default, *all*
optimizations are disabled, there is a negligible overhead when the
FAT mode is not used.

In the FAT mode, the dictionary for modules, classes and instances
becomes "versionned". Functions gets new methods to support adding
specialized bytecode with guards.

You can add manually a specialized bytecode for microbenchmarks, but
functions are *not* optimized automatically. (See the Roadmap below.)


How to write an optimizer?
==========================

Currently, my proof-of-oncept is only the machinery to support adding
specialized bytecodes with guards. The next step is to generate
automatically these specialized versions.

IMHO an optimizer should be implemented in Python and not in C, it's
easier to write Python code. We can begin with simple optimizations on
AST. See for example my astoptimizer which implements basic
optimizations:

    https://bitbucket.org/haypo/astoptimizer/

At the beginning, we may use type hints (PEP 484) and other manual
hints to help the optimizer. Later, a profiler learning the most common
types of function parameters can be written for automatic optimizations.
Example of hints: a list of constants which should not be modified in the
application. A global "DEBUG" flag is common in applications, relying on
DEBUG=False helps to remove dead code.


Roadmap
=======

* Finish the proof-of-concept: implement new guards, optimize method
  calls. For example, guards on module imports are required. Checking
  if a module is the expected mode can be tricky.
* Write an optimizer based on manual hints
* Write a profiler to generate hints

Right now, my main question is if it will be possible and efficient to
optimize classes, not only simple functions, especially classes defined
in two different modules. I wrote a proof-of-concept of optimized
method (with inlining), but I'm not sure yet that it's safe (don't
change Python semantic).

For more information on FAT Python, read:

   https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst


So, what do you think? Do you expect real speedup on applications, not
only on microbechmarks? Do you know similar optimizations in other
scripting languages?

Victor Stinner

From rosuav at gmail.com  Wed Oct 21 17:48:51 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 22 Oct 2015 02:48:51 +1100
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
Message-ID: <CAPTjJmq5187DChFZaf+LFWFyEWJHmO159X6JvjwiLuU9MePCSQ@mail.gmail.com>

On Thu, Oct 22, 2015 at 2:43 AM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Roadmap
> =======
>
> * Finish the proof-of-concept: implement new guards, optimize method
>   calls. For example, guards on module imports are required. Checking
>   if a module is the expected mode can be tricky.
> * Write an optimizer based on manual hints
> * Write a profiler to generate hints

* Compare against PyPy.

For optimized Python execution, my brain immediately jumps to PyPy.
How does FAT Python compare in terms of functionality, performance,
CPython compatibility, etc? What can it brag as its advantage?

ChrisA

From ncoghlan at gmail.com  Wed Oct 21 18:11:04 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Oct 2015 18:11:04 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAPTjJmq5187DChFZaf+LFWFyEWJHmO159X6JvjwiLuU9MePCSQ@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAPTjJmq5187DChFZaf+LFWFyEWJHmO159X6JvjwiLuU9MePCSQ@mail.gmail.com>
Message-ID: <CADiSq7cfeURAzxCWjUASoJSLFFe1Se9uMhBGSy1TCC-LJsdqfw@mail.gmail.com>

On 21 October 2015 at 17:48, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Oct 22, 2015 at 2:43 AM, Victor Stinner
> <victor.stinner at gmail.com> wrote:
>> Roadmap
>> =======
>>
>> * Finish the proof-of-concept: implement new guards, optimize method
>>   calls. For example, guards on module imports are required. Checking
>>   if a module is the expected mode can be tricky.
>> * Write an optimizer based on manual hints
>> * Write a profiler to generate hints
>
> * Compare against PyPy.
>
> For optimized Python execution, my brain immediately jumps to PyPy.
> How does FAT Python compare in terms of functionality, performance,
> CPython compatibility, etc? What can it brag as its advantage?

* Full CPython C API compatibility
* Reliance on ahead of time compilation means it works for command
line scripts, not just long running server processes that allow the
PyPy tracing JIT to work its magic

As an ahead-of-time optimisation, it's also possible something like
this could speed up applications running on PyPy during the JIT
warm-up phase.

An existing example of this kind of guard-based optimisation exists in
the ABC machinery - we globally version the ABC registration graph in
order to automatically invalidate caches when a new explicit
registration is added.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rosuav at gmail.com  Wed Oct 21 18:12:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 22 Oct 2015 03:12:56 +1100
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CADiSq7cfeURAzxCWjUASoJSLFFe1Se9uMhBGSy1TCC-LJsdqfw@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAPTjJmq5187DChFZaf+LFWFyEWJHmO159X6JvjwiLuU9MePCSQ@mail.gmail.com>
 <CADiSq7cfeURAzxCWjUASoJSLFFe1Se9uMhBGSy1TCC-LJsdqfw@mail.gmail.com>
Message-ID: <CAPTjJmpuk1QyssD5fo0d-d+cBZ9c_gA_PvEYvJSha8_CCgNa2w@mail.gmail.com>

On Thu, Oct 22, 2015 at 3:11 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> For optimized Python execution, my brain immediately jumps to PyPy.
>> How does FAT Python compare in terms of functionality, performance,
>> CPython compatibility, etc? What can it brag as its advantage?
>
> * Full CPython C API compatibility
> * Reliance on ahead of time compilation means it works for command
> line scripts, not just long running server processes that allow the
> PyPy tracing JIT to work its magic

Cool! Thanks for the summary.

ChrisA

From guido at python.org  Wed Oct 21 18:20:02 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Oct 2015 09:20:02 -0700
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
Message-ID: <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>

 I haven't tried the prototype yet, but I wonder if this mightn't be a
useful addition to an existing optimizing Python system, e.g. PyPy or (plug
:-) Pyston?

On Wed, Oct 21, 2015 at 8:43 AM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> Hi,
>
> I would like to share with you an idea to try to optimize CPython.
>
> My previous attempts to optimize CPython failed because they changed the
> language semantic. For example, it's not possible to replace len('abc')
> with 3 because it is technically possible to override the builtin len()
> function.
>
> I propose to add a new "FAT" mode to Python to allow to add specialized
> bytecode to function with guards. Example:
>
>     >>> import builtins
>     >>> def f(): return len("abc")
>     ...
>     >>> f()
>     3
>
>     >>> def g(): return 3
>     ...
>     >>> i=f.add(g)   # i is the index of the new specialized bytecode
>     >>> f.add_dict_key_guard(i, builtins.__dict__, 'len')
>     >>> f()
>     3
>
>     >>> builtins.len = lambda obj: "len"
>     >>> f()
>     'len'
>
> In this example, the f() function gets a fast version (g()) returning
> directly 3 instead of calling len("abc"). The optimization is disabled
> when the builtin len() function is modified (when
> builtins.__dict__['len'] is modified). (The example is not complete, we
> need a second guard on the current global namespace, but I wanted to
> write a short example.)
>
> With such machinery, it becomes possible to implement interesting
> optimizations. Some examples:
>
> * inline a function instead of calling it: calling a Python function is
>   "expensive". For example, if you inline the _get_sep() call in
>   posixpath.isabs(), the function becomes 20% faster.
>
> * move invariant out of the loop: classic micro-optimization done
>   manually. For example, set "append = obj.append" before the loop and
>   then call "append(item)" instead of "obj.append(item)" in the loop
>   body. On a microbenchmark, it's between 5% (1 item) and 30% faster (10
>   items or more)
>
> * call pure functions at the compilation. A pure function has no side
>   effect. Example: len(str).
>
>
> Technical Challenges
> ====================
>
> The problem is the cost of "guards". A guard is a check done before
> calling a specialized function. Example of guards:
>
> * Type of a function parameter
> * Watch a dictionary key
> * Watch a function (especially it's __code__ attribute)
>
> Watching a dictionary key is a very important guard to disable an
> optimization when a builtin function is modified, when a function is
> replaced in a namespace, etc. The problem is that a dictionary lookup is
> not cheap: we have to get the hash value of the key, browse the hash
> table to find the bucket which may require multiple iterations, then we
> have to compare the key value, etc.
>
> To have faster guard, I propose to create a subtype of dict which
> provides a global "version" of the dictionary, incremented at each
> modification (create a new key, modify a key, delete a key) and a
> "version" per key=>value mapping, incremented each time
> that the mapping is modified. The lookup can be avoided when the
> dictionary is not modified. If a different key is modified, we need a
> lookup, but only once (we store the new global version to avoid a lookup
> at the next guard check).
>
>
> Limitations
> ===========
>
> Specialized bytecode is build at compilation, *not* at runtime: it's not a
> just-in-time (JIT) compiler. A JIT can implement even more efficient
> optimizations and can use better guards. Please continue to use PyPy for
> best performances! ;-)
>
> The name "FAT" comes from the fact that multiple bytecode versions of a
> function are stored in the memory, so a larger memory footprint
> and larger .pyc files on disk can be expected.
>
> Checking guards can also be more expensive than the optimization of the
> specialized bytecode. The optimizer should use an heuristic to decide if
> it's worth to use a specialized bytecode or not depending on the theoric
> speeup and the cost of guards.
>
> My motivation to write FAT Python is that CPython remains the reference
> implementation where new features are implemented, and other Python
> implementations still have issues with C extensions. JIT also has some
> issues, like longer startup time, slow warmup and memory footprint (this
> one may also be an issue of FAT Python!).
>
>
> Implementation
> ==============
>
> I wrote a proof-of-concept of my idea:
>
>     https://hg.python.org/sandbox/fatpython/
>
> It's a fork of CPython 3.6. Try it with:
>
>     hg clone https://hg.python.org/sandbox/fatpython/
>     cd fatpython
>     ./configure && make
>     ./python -F
>
> See bench.py, Lib/posixpath.py (isabs) and Lib/test/test_fat.py
> for examples of optimizations.
>
> The command line -F flag enables the FAT mode. By default, *all*
> optimizations are disabled, there is a negligible overhead when the
> FAT mode is not used.
>
> In the FAT mode, the dictionary for modules, classes and instances
> becomes "versionned". Functions gets new methods to support adding
> specialized bytecode with guards.
>
> You can add manually a specialized bytecode for microbenchmarks, but
> functions are *not* optimized automatically. (See the Roadmap below.)
>
>
> How to write an optimizer?
> ==========================
>
> Currently, my proof-of-oncept is only the machinery to support adding
> specialized bytecodes with guards. The next step is to generate
> automatically these specialized versions.
>
> IMHO an optimizer should be implemented in Python and not in C, it's
> easier to write Python code. We can begin with simple optimizations on
> AST. See for example my astoptimizer which implements basic
> optimizations:
>
>     https://bitbucket.org/haypo/astoptimizer/
>
> At the beginning, we may use type hints (PEP 484) and other manual
> hints to help the optimizer. Later, a profiler learning the most common
> types of function parameters can be written for automatic optimizations.
> Example of hints: a list of constants which should not be modified in the
> application. A global "DEBUG" flag is common in applications, relying on
> DEBUG=False helps to remove dead code.
>
>
> Roadmap
> =======
>
> * Finish the proof-of-concept: implement new guards, optimize method
>   calls. For example, guards on module imports are required. Checking
>   if a module is the expected mode can be tricky.
> * Write an optimizer based on manual hints
> * Write a profiler to generate hints
>
> Right now, my main question is if it will be possible and efficient to
> optimize classes, not only simple functions, especially classes defined
> in two different modules. I wrote a proof-of-concept of optimized
> method (with inlining), but I'm not sure yet that it's safe (don't
> change Python semantic).
>
> For more information on FAT Python, read:
>
>    https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst
>
>
> So, what do you think? Do you expect real speedup on applications, not
> only on microbechmarks? Do you know similar optimizations in other
> scripting languages?
>
> Victor Stinner
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/d10d5181/attachment-0001.html>

From victor.stinner at gmail.com  Wed Oct 21 18:35:38 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 21 Oct 2015 18:35:38 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CADiSq7cfeURAzxCWjUASoJSLFFe1Se9uMhBGSy1TCC-LJsdqfw@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAPTjJmq5187DChFZaf+LFWFyEWJHmO159X6JvjwiLuU9MePCSQ@mail.gmail.com>
 <CADiSq7cfeURAzxCWjUASoJSLFFe1Se9uMhBGSy1TCC-LJsdqfw@mail.gmail.com>
Message-ID: <CAMpsgwZNENy3Afvo_zx_RqpAqdzNx0OJuKB=zmGgOe2eLij4Rg@mail.gmail.com>

On 21 October 2015 at 17:48, Chris Angelico <rosuav at gmail.com> wrote:
> * Compare against PyPy.

As written in my email, don't expect better performances than PyPy.
PyPy must remain the fatest implementation of Python ;-)

The final goal is to see real speedup on non-trivial applications like
Django Mercurial, or OpenStack.

2015-10-21 18:11 GMT+02:00 Nick Coghlan <ncoghlan at gmail.com>:
> * Full CPython C API compatibility

FYI the FAT mode uses fat.verdict and fat.SpecializedFunction types
which are subtypes of dict and types.FunctionTypes, so you keep a full
compatibility of the C API.

I noticed one issue with the C API:

#define PyFunction_Check(op) (Py_TYPE(op) == &PyFunction_Type)

This test fails with fat.SpecializedFunction. It means that C
extensions must be recompiled to with the FAT mode. I modified the
macro to:

#define PyFunction_Check(op) \
    (Py_TYPE(op) == &PyFunction_Type \
     || PyType_IsSubtype(Py_TYPE(op), &PyFunction_Type))

> * Reliance on ahead of time compilation means it works for command
> line scripts, not just long running server processes that allow the
> PyPy tracing JIT to work its magic

Yes, a I wrote in my email, CPython still beats PyPy for fast command
line scripts like many Mercurial commands. PyPy JIT takes time to
warmup. (Hum, in fact I only read time, I didn't check, sorry.)

So yes, I also expect faster performances with static compilation on
such use case.

> An existing example of this kind of guard-based optimisation exists in
> the ABC machinery - we globally version the ABC registration graph in
> order to automatically invalidate caches when a new explicit
> registration is added.

Hum, this is a common cache. CPython has many similar caches, like
cache for methods on types, but it's not guard for function calls. I
would like to know if someone else tried to write specialized
functions with guards in other scripting languages (or even Python?).

Victor

From victor.stinner at gmail.com  Wed Oct 21 18:50:47 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 21 Oct 2015 18:50:47 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
Message-ID: <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>

2015-10-21 18:20 GMT+02:00 Guido van Rossum <guido at python.org>:
>  I haven't tried the prototype yet, but I wonder if this mightn't be a
> useful addition to an existing optimizing Python system, e.g. PyPy or (plug
> :-) Pyston?

I'm not sure that specialized functions with guards would benefit to
JIT compilers.

A best match is with Cython, pythran and other projects generating a
specialized version of a function (in native code), maybe releasing
the GIL. It would be cool to be able to use a specialized version
under some conditions. For example, if parameters are all floats, use
the specialized version optimized by Cython, other fallback to the
(slow) regular bytecode. The challenge is to automate everything and
make it easy to use.

I don't know if Numba would match with this design.

Victor

From random832 at fastmail.com  Wed Oct 21 18:52:01 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 21 Oct 2015 12:52:01 -0400
Subject: [Python-ideas] Add specialized bytecode with guards to functions
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
Message-ID: <87pp08w4em.fsf@fastmail.com>

Victor Stinner <victor.stinner at gmail.com>
writes:

> Hi,
>
> I would like to share with you an idea to try to optimize CPython.
>
> My previous attempts to optimize CPython failed because they changed the
> language semantic. For example, it's not possible to replace len('abc')
> with 3 because it is technically possible to override the builtin len()
> function.

I think this is solving the wrong problem...

What about adding an explicit way to "bake in" the values of globals?

freeze_globals(func, values)

values is a dict with the names of globals and their values, so, you
could for example:

def foo():
    return len('abc')

foo = freeze_globals(foo, {'len': len})
# or
foo = freeze_globals(foo, __builtins__.__dict__)

It would replace any LOAD_GLOBAL for an item that is in the given
dictionary (maybe excluding anything that is also stored in the same
function) with a LOAD_CONST with a reference to the value from the
dictionary. As a second pass, now that the bytecode looks like this:

LOAD_CONST ('abc')
LOAD_CONST (<built-in function len>)
CALL_FUNCTION 1

This is no longer dependent on anything that can be overridden, and can
safely be replaced (given the optimizer knows len is a pure function)
with the constant 3. This second pass could also take care of e.g. empty
set and frozenset constructors, etc.

Decorators could be included to freeze only built-ins, to freeze all
globals and built-ins, or to have a list of names to exclude. So you
could have @freeze_builtins or @freeze_all_globals for each function
that needs heavy optimization.


From guido at python.org  Wed Oct 21 18:59:50 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Oct 2015 09:59:50 -0700
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
Message-ID: <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>

On Wed, Oct 21, 2015 at 9:50 AM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> 2015-10-21 18:20 GMT+02:00 Guido van Rossum <guido at python.org>:
> >  I haven't tried the prototype yet, but I wonder if this mightn't be a
> > useful addition to an existing optimizing Python system, e.g. PyPy or
> (plug
> > :-) Pyston?
>
> I'm not sure that specialized functions with guards would benefit to
> JIT compilers.
>

Pyston is not primarily a JIT compiler (at least that's my understanding).

And my point was the opposite: Pyston (or PyPy, less sure) might benefit
from your idea.


> A best match is with Cython, pythran and other projects generating a
> specialized version of a function (in native code), maybe releasing
> the GIL. It would be cool to be able to use a specialized version
> under some conditions. For example, if parameters are all floats, use
> the specialized version optimized by Cython, other fallback to the
> (slow) regular bytecode. The challenge is to automate everything and
> make it easy to use.
>
> I don't know if Numba would match with this design.
>

Admittedly these are also very attractive-looking pairings!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/40fbe246/attachment.html>

From victor.stinner at gmail.com  Wed Oct 21 18:59:54 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 21 Oct 2015 18:59:54 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <87pp08w4em.fsf@fastmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <87pp08w4em.fsf@fastmail.com>
Message-ID: <CAMpsgwZ2qB1S9j1tJusXEsLPdxgUUBpT0eROfD-pXOa6wasTjg@mail.gmail.com>

Hi,

2015-10-21 18:52 GMT+02:00 Random832 <random832 at fastmail.com>:
> so, you could for example:
>
> def foo():
>     return len('abc')
>
> foo = freeze_globals(foo, {'len': len})
> # or
> foo = freeze_globals(foo, __builtins__.__dict__)

You are changing the semantic of Python. In Python, you should be able
to override builtin functions.

Binding a global name to a local name is a known trick to optimize a
function, but you have to write it explicitly, and again you loose the
ability to override len.

Example:

def f(len=len):
   return len("abc")

Here len becomes a fast local variable.

> Decorators could be included to freeze only built-ins, to freeze all
> globals and built-ins, or to have a list of names to exclude. So you
> could have @freeze_builtins or @freeze_all_globals for each function
> that needs heavy optimization.

I cited "previous attempts" which failed.

What you describe is close to my "readonly" attempt which tried to
make module and type namespaces readonly:
https://hg.python.org/sandbox/readonly/file/tip/READONLY.txt

Making namespaces read-only simply doesn't work. It's very common to
modify namespaces. I added callbacks to disable optimizations when a
namespace is modified. But this design doesn't scale: you add a
complexity of O(n) when a namespace is modified, even if optimized
functions are never called.

My new design (guards) avoids this performance issue.

Victor

From brett at python.org  Wed Oct 21 19:12:34 2015
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Oct 2015 17:12:34 +0000
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
Message-ID: <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>

On Wed, 21 Oct 2015 at 10:00 Guido van Rossum <guido at python.org> wrote:

> On Wed, Oct 21, 2015 at 9:50 AM, Victor Stinner <victor.stinner at gmail.com>
> wrote:
>
>> 2015-10-21 18:20 GMT+02:00 Guido van Rossum <guido at python.org>:
>> >  I haven't tried the prototype yet, but I wonder if this mightn't be a
>> > useful addition to an existing optimizing Python system, e.g. PyPy or
>> (plug
>> > :-) Pyston?
>>
>> I'm not sure that specialized functions with guards would benefit to
>> JIT compilers.
>>
>
> Pyston is not primarily a JIT compiler (at least that's my understanding).
>

There GitHub project seems to suggest otherwise based on them saying they
are leveraging LLVM's JIT: https://github.com/dropbox/pyston#pyston-- .


>
> And my point was the opposite: Pyston (or PyPy, less sure) might benefit
> from your idea.
>

As would Pyjion (which is really early work and being played with in Dino
and my spare time at work): https://github.com/microsoft/pyjion . We would
probably love to see at least the dict watcher get into CPython. I know
this was something that Unladen Swallow really wanted as well for easy
optimizations of built-in functions. Extend dict watchers to a module's or
class' __dict__ and you start to get more inlining opportunities as Victor
is hoping for. But as a starting point, getting some way to cheaply detect
when the built-ins have not been modified from what they are set to by
default would be nice to have.

-Brett


>
>
>> A best match is with Cython, pythran and other projects generating a
>> specialized version of a function (in native code), maybe releasing
>> the GIL. It would be cool to be able to use a specialized version
>> under some conditions. For example, if parameters are all floats, use
>> the specialized version optimized by Cython, other fallback to the
>> (slow) regular bytecode. The challenge is to automate everything and
>> make it easy to use.
>>
>> I don't know if Numba would match with this design.
>>
>
> Admittedly these are also very attractive-looking pairings!
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/e6194332/attachment.html>

From srkunze at mail.de  Wed Oct 21 19:23:41 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 21 Oct 2015 19:23:41 +0200
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <CAP7+vJJBLV8Bpaoq-Hc+_65M0NSdvNQdCayO8q7vHwGpuVqJUQ@mail.gmail.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <20151021015025.GM3813@ando.pearwood.info>
 <CAP7+vJJBLV8Bpaoq-Hc+_65M0NSdvNQdCayO8q7vHwGpuVqJUQ@mail.gmail.com>
Message-ID: <5627CA1D.9040302@mail.de>

On 21.10.2015 05:36, Guido van Rossum wrote:
> Right. Chris's thinking recalls the reason why inheritance at some 
> became popular (I guess it was in the '90s). Steven explains (in the 
> part that I've cut) why many experts have soured on it quite a bit.
>

Yep. It's because experience shows that the usefulness is limited when 
it comes to orthogonal aspects.

Furthermore, I can remember changing an algorithm's design **because** 
of the lack of such built-in datastructure. That's always a bad sign.

> Personally, I happen to think that inheritance is often useful when a 
> number of classes are designed together (typically all at once and 
> belonging to the same library) and also occasionally when a base class 
> is explicitly and carefully designed to be inherited (there are 
> beautiful things you can do with the template pattern, for example). 
> But inheriting from an implementation that wasn't designed with your 
> use case in mind is often asking for trouble -- if not now, then in 
> the future when a refactored implementation is released.

That's a pretty good observation.

> You might interject, that's the fault of the implementation 
> refactoring -- they didn't properly think about interface 
> compatibility. But while it's usually easy enough to keep an interface 
> compatible where it's just the user calling methods on the 
> implementation, the "interface" presented by subclassing is much more 
> complex -- you would have to specify exactly which method's 
> implementation calls which other method, and you'd also have to ensure 
> that the object is in a sane state when it calls that other method, 
> because it *could* be the case that the latter is overridden by a 
> subclass. It's terribly fragile, and better avoided.

Maybe, that's one reason why people hesitate to write their own 
OrderDefaultDict or DefaultOrderedDict. It's just not their responsibility.

Best,
Sven

From p.f.moore at gmail.com  Wed Oct 21 19:31:35 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 21 Oct 2015 18:31:35 +0100
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
 <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
Message-ID: <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>

On 21 October 2015 at 18:12, Brett Cannon <brett at python.org> wrote:
> But as a starting point, getting some way to cheaply detect when the
> built-ins have not been modified from what they are set to by default would
> be nice to have.

Would it be worth focusing on mechanisms like this, which would be of
general use to various projects, and treating Victor's specific
optimisation use case as simply that, a motivating example? At least
in the first instance - as Victor pointed out, previous attempts to
introduce optimisations have always failed because it's not possible
to switch off Python's dynamic nature. So maybe if we had tools which
allowed code to more easily *detect* when Python's dynamic features
have been used (or rather, when they have *not* been used) we'd be in
a better position to develop optimisation strategies that know when
they are appropriate?

(Another example - Victor mentioned that CPython uses
PyFunction_Check, which isn't friendly to function subclasses. A fix
to allow function subclasses to work more seamlessly would be another
useful general mechanism).

Paul

From random832 at fastmail.com  Wed Oct 21 19:35:32 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 21 Oct 2015 13:35:32 -0400
Subject: [Python-ideas] Add specialized bytecode with guards to functions
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <87pp08w4em.fsf@fastmail.com>
 <CAMpsgwZ2qB1S9j1tJusXEsLPdxgUUBpT0eROfD-pXOa6wasTjg@mail.gmail.com>
Message-ID: <878u6ww2e3.fsf@fastmail.com>

Victor Stinner <victor.stinner at gmail.com>
writes:
> You are changing the semantic of Python. In Python, you should be able
> to override builtin functions.

What I am proposing is a generalized way to add objects that do not have
a literal to a function's constants pool. The global name within the
first version of the function is simply used as a placeholder for
loading the new constant.

It could just as easily be something like

def foo():
   return _1(abc)

foo = add_consts(foo, {'_1':len})

> Binding a global name to a local name is a known trick to optimize a
> function, but you have to write it explicitly, and again you loose the
> ability to override len.

This would also have to be written explicitly, and the whole point is to
lose the ability to override things. Explicitly.

> Example:
>
> def f(len=len):
>    return len("abc")
>
> Here len becomes a fast local variable.

It'd be even faster as a constant, and that way people couldn't call it
as f(len=whatever) [so an optimizer doesn't have to deal with the
possibility of people doing that]

>> Decorators could be included to freeze only built-ins, to freeze all
>> globals and built-ins, or to have a list of names to exclude. So you
>> could have @freeze_builtins or @freeze_all_globals for each function
>> that needs heavy optimization.
>
> I cited "previous attempts" which failed.
>
> What you describe is close to my "readonly" attempt which tried to
> make module and type namespaces readonly:
> https://hg.python.org/sandbox/readonly/file/tip/READONLY.txt

This would apply to the function, not the namespace.

> Making namespaces read-only simply doesn't work. It's very common to
> modify namespaces. I added callbacks to disable optimizations when a
> namespace is modified. But this design doesn't scale: you add a
> complexity of O(n) when a namespace is modified, even if optimized
> functions are never called.

Nothing happens when a namespace is modified, because the function is no
longer using the namespace in any form. This adds to the cost of
defining a function, but in most cases (I'm not 100% sure of the
semantics for nested functions, let alone the implementation) this could
be done at compile time, so it would be a one-time cost.


From srkunze at mail.de  Wed Oct 21 19:41:55 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 21 Oct 2015 19:41:55 +0200
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
Message-ID: <5627CE63.9030704@mail.de>

On 21.10.2015 00:50, Chris Angelico wrote:
> She recommends a massive superclass that's capable of any form of injection

Nope. She does not.

The "superclass" is not "massive" at all. It is even slimmer as 
orthogonal aspects are refactored out into separate entities. In fact, 
it makes it even easier to test and maintain these separate aspects (the 
core dev should be interested in that). Furthermore, it's, of course, up 
to debate which aspects should be injectable and which are not.


Just imagine, I would need to use several orderings and/or a default 
value for missing keys:

normal_dict = dict()
ordered_dict = dict(order=dict.order_by_insert)
sorted_dict = dict(order=sorted)
sorted_default_dict = dict(order=sorted, default=int)


How many subclasses am I supposed to write, maintain and upgrade (in 
case Guido rewrites his precious dict implementation)? I would even 
allege that for sufficiently large teams and projects, there are 
*multiple* implementations with the same intent.


Please, no.


Best,
Sven


PS: the instances above are real-world examples, I remember requiring 
during the course of the last year.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/2f1a8e02/attachment.html>

From ian.g.kelly at gmail.com  Wed Oct 21 19:57:36 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 21 Oct 2015 11:57:36 -0600
Subject: [Python-ideas] "else if" as equivalent for "elif"
Message-ID: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>

Parse the sequence "else if" equivalently to "elif".

This would be 100% backward compatible. It wouldn't require adding any
new keywords to the language. It shouldn't be burdensome to maintain,
since "elif" only appears in one place in the grammar. And it solves a
minor readability complaint.

Thoughts?

From rymg19 at gmail.com  Wed Oct 21 20:05:01 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 21 Oct 2015 13:05:01 -0500
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
Message-ID: <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>

The Python Zen says:

There should be one-- and preferably only one --obvious way to do it.

elif uses less typing, anyway. Why type 3 more characters when you really don't need to? :D

On October 21, 2015 12:57:36 PM CDT, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>Parse the sequence "else if" equivalently to "elif".
>
>This would be 100% backward compatible. It wouldn't require adding any
>new keywords to the language. It shouldn't be burdensome to maintain,
>since "elif" only appears in one place in the grammar. And it solves a
>minor readability complaint.
>
>Thoughts?
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/5da5fc6c/attachment.html>

From srkunze at mail.de  Wed Oct 21 20:16:07 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 21 Oct 2015 20:16:07 +0200
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
Message-ID: <5627D667.8020301@mail.de>

On 21.10.2015 20:05, Ryan Gonzalez wrote:
> The Python Zen says:
>
> There should be one-- and preferably only one --obvious way to do it.
>
> elif uses less typing, anyway. Why type 3 more characters when you 
> really don't need to? :D

Because it's one of those "Python looks strange"? ;-)

Best,
Sven

From jonathan at slenders.be  Wed Oct 21 20:25:58 2015
From: jonathan at slenders.be (Jonathan Slenders)
Date: Wed, 21 Oct 2015 20:25:58 +0200
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <5627CE63.9030704@mail.de>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com>
 <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <5627CE63.9030704@mail.de>
Message-ID: <CAKfyG3zU+8138_hvAwbFLLp9mzC1v10+8ae3t6QpPKvXzyNX2A@mail.gmail.com>

Just want to say that I'm happy to see that lately the disadvantages of
inheritance (which are already known for a very long time) are getting more
attention.
It's not bad by definition, but there's so much truth in Sandy her talk and
I think for many Python projects, we went way too far into "abusing"
inheritance.
Actually, it's a bit unfortunate that we made inheritance so user friendly
and powerful in Python that for many people it became the logical way to
extend or reuse some code.

Jonathan







2015-10-21 19:41 GMT+02:00 Sven R. Kunze <srkunze at mail.de>:

> On 21.10.2015 00:50, Chris Angelico wrote:
>
> She recommends a massive superclass that's capable of any form of injection
>
>
> Nope. She does not.
>
> The "superclass" is not "massive" at all. It is even slimmer as orthogonal
> aspects are refactored out into separate entities. In fact, it makes it
> even easier to test and maintain these separate aspects (the core dev
> should be interested in that). Furthermore, it's, of course, up to debate
> which aspects should be injectable and which are not.
>
>
> Just imagine, I would need to use several orderings and/or a default value
> for missing keys:
>
> normal_dict = dict()
> ordered_dict = dict(order=dict.order_by_insert)
> sorted_dict = dict(order=sorted)
> sorted_default_dict = dict(order=sorted, default=int)
>
>
> How many subclasses am I supposed to write, maintain and upgrade (in case
> Guido rewrites his precious dict implementation)? I would even allege that
> for sufficiently large teams and projects, there are *multiple*
> implementations with the same intent.
>
>
> Please, no.
>
>
> Best,
> Sven
>
>
> PS: the instances above are real-world examples, I remember requiring
> during the course of the last year.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/eed77672/attachment-0001.html>

From brett at python.org  Wed Oct 21 20:43:07 2015
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Oct 2015 18:43:07 +0000
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
 <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
 <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>
Message-ID: <CAP1=2W74-nkHvVjyE6sf_cJqpEMhjj3zmKxm-ViP=B-G7D67Eg@mail.gmail.com>

On Wed, 21 Oct 2015 at 10:31 Paul Moore <p.f.moore at gmail.com> wrote:

> On 21 October 2015 at 18:12, Brett Cannon <brett at python.org> wrote:
> > But as a starting point, getting some way to cheaply detect when the
> > built-ins have not been modified from what they are set to by default
> would
> > be nice to have.
>
> Would it be worth focusing on mechanisms like this, which would be of
> general use to various projects, and treating Victor's specific
> optimisation use case as simply that, a motivating example?


I think so.


> At least
> in the first instance - as Victor pointed out, previous attempts to
> introduce optimisations have always failed because it's not possible
> to switch off Python's dynamic nature. So maybe if we had tools which
> allowed code to more easily *detect* when Python's dynamic features
> have been used (or rather, when they have *not* been used) we'd be in
> a better position to develop optimisation strategies that know when
> they are appropriate?
>

Exactly, and especially if the check is cheap. After that you start to
potentially get into granularity, e.g. do you care if *any* built-in
changed or do you only care if a *specific* built-in changed, and how do
either approach affect performance? The trick is not breaking compatibility
and not hurting performance in the general case. But yes, knowing when the
uncommon case of overriding a built-in occurs would allow for a bunch of
optimization techniques from inlining to using a faster implementation that
is compatible with the standard built-in.

-Brett


>
> (Another example - Victor mentioned that CPython uses
> PyFunction_Check, which isn't friendly to function subclasses. A fix
> to allow function subclasses to work more seamlessly would be another
> useful general mechanism).
>
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/e75aae5e/attachment.html>

From abarnert at yahoo.com  Wed Oct 21 21:10:03 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 21 Oct 2015 12:10:03 -0700
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <5627D667.8020301@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
Message-ID: <C064F7FA-3499-433C-B14B-9727EF2C1FC9@yahoo.com>

On Oct 21, 2015, at 11:16, Sven R. Kunze <srkunze at mail.de> wrote:
> 
>> On 21.10.2015 20:05, Ryan Gonzalez wrote:
>> The Python Zen says:
>> 
>> There should be one-- and preferably only one --obvious way to do it.
>> 
>> elif uses less typing, anyway. Why type 3 more characters when you really don't need to? :D
> 
> Because it's one of those "Python looks strange"? ;-)

Unless you also deprecate "elif", Python will still look strange?it'll just look inconsistently strange, which is even worse than consistently strange.

Besides, it's not as if "else if" reads like English any more than "elif" does. In English, you'd use a semicolon and an "if" for all but the last, and "or if" for the last (or, if there's only one, maybe "otherwise if" or "but if", depending on the connotations).

Also, the number of people coming to Python from C and its descendants (especially Java and JavaScript) is high enough that the false impression given by "else if" (that it's just an if statement controlled by an else clause) could be harmful. Yes, once you think about it, the fact that there's no colon after the "else" and only one level of indentation instead of two makes sense, but you shouldn't have to think about it.


From random832 at fastmail.com  Wed Oct 21 21:21:18 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 21 Oct 2015 15:21:18 -0400
Subject: [Python-ideas] "else if" as equivalent for "elif"
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de>
 <C064F7FA-3499-433C-B14B-9727EF2C1FC9@yahoo.com>
Message-ID: <87twpkuixd.fsf@fastmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:

> Also, the number of people coming to Python from C and its descendants
> (especially Java and JavaScript) is high enough that the false
> impression given by "else if" (that it's just an if statement
> controlled by an else clause) could be harmful. Yes, once you think
> about it, the fact that there's no colon after the "else" and only one
> level of indentation instead of two makes sense, but you shouldn't
> have to think about it.

Could go all the way and actually allow "else: if ...:" or any other
combination such as "for ...: if...:" with a single level of
indentation.

I bet that's been proposed and rejected before.


From python at mrabarnett.plus.com  Wed Oct 21 21:26:47 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 21 Oct 2015 20:26:47 +0100
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <5627D667.8020301@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
Message-ID: <5627E6F7.50200@mrabarnett.plus.com>

On 2015-10-21 19:16, Sven R. Kunze wrote:
> On 21.10.2015 20:05, Ryan Gonzalez wrote:
>> The Python Zen says:
>>
>> There should be one-- and preferably only one --obvious way to do it.
>>
>> elif uses less typing, anyway. Why type 3 more characters when you
>> really don't need to? :D
>
> Because it's one of those "Python looks strange"? ;-)
>
The C preprocessor has #elif. Is that strange too? :-)

(It could be argued that it _is_ strange that the C preprocessor has an
explicitly-terminated "#if ... #endif", whereas the C language itself
doesn't have an explicitly-terminated "if".)


From rymg19 at gmail.com  Wed Oct 21 21:29:32 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 21 Oct 2015 14:29:32 -0500
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <5627E6F7.50200@mrabarnett.plus.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
Message-ID: <A681948F-AAB8-4032-86F0-E1202A6E1237@gmail.com>



On October 21, 2015 2:26:47 PM CDT, MRAB <python at mrabarnett.plus.com> wrote:
>On 2015-10-21 19:16, Sven R. Kunze wrote:
>> On 21.10.2015 20:05, Ryan Gonzalez wrote:
>>> The Python Zen says:
>>>
>>> There should be one-- and preferably only one --obvious way to do
>it.
>>>
>>> elif uses less typing, anyway. Why type 3 more characters when you
>>> really don't need to? :D
>>
>> Because it's one of those "Python looks strange"? ;-)
>>
>The C preprocessor has #elif. Is that strange too? :-)
>
>(It could be argued that it _is_ strange that the C preprocessor has an
>explicitly-terminated "#if ... #endif", whereas the C language itself
>doesn't have an explicitly-terminated "if".)
>

It's the C preprocessor. How is it NOT strange? :)

>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From abarnert at yahoo.com  Wed Oct 21 21:27:29 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 21 Oct 2015 12:27:29 -0700
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <CAKfyG3zU+8138_hvAwbFLLp9mzC1v10+8ae3t6QpPKvXzyNX2A@mail.gmail.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <5627CE63.9030704@mail.de>
 <CAKfyG3zU+8138_hvAwbFLLp9mzC1v10+8ae3t6QpPKvXzyNX2A@mail.gmail.com>
Message-ID: <3840798D-28DD-4AF3-A6A2-4BAA1D89442A@yahoo.com>

On Oct 21, 2015, at 11:25, Jonathan Slenders <jonathan at slenders.be> wrote:
> 
> Just want to say that I'm happy to see that lately the disadvantages of inheritance (which are already known for a very long time) are getting more attention.
> It's not bad by definition, but there's so much truth in Sandy her talk and I think for many Python projects, we went way too far into "abusing" inheritance.
> Actually, it's a bit unfortunate that we made inheritance so user friendly and powerful in Python that for many people it became the logical way to extend or reuse some code.

One of the most important things people have learned about OO over the past two decades is that subtyping, implementation extension, and implementation mixins, and interface-extending mixins (think collections.abc.Sequence adding count for you) are all different things. Compared to the other languages in existence at the time of Python 1.x or even the 2.2/2.3 changeover, it's hard to fault Python. The fact that it can easily be used to write bad code is a little unfortunate, but the fact that it can also easily be used to write good code, when other languages either don't allow some things to be expressed, force them to be expressed in clumsy or limited ways, or force you to misuse inappropriate features instead more than makes up for it. In particular, any language that has fixed structure layouts and vtables makes is much more unfriendly to both kinds of mixins, and makes interface subtyping clumsy; Python has no such problems. Yeah, maybe we could design something better in 2015, but based on the knowledge people had at the time?

From abarnert at yahoo.com  Wed Oct 21 21:45:50 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 21 Oct 2015 12:45:50 -0700
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <878u6ww2e3.fsf@fastmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <87pp08w4em.fsf@fastmail.com>
 <CAMpsgwZ2qB1S9j1tJusXEsLPdxgUUBpT0eROfD-pXOa6wasTjg@mail.gmail.com>
 <878u6ww2e3.fsf@fastmail.com>
Message-ID: <1C6ACCBD-EB5F-49EB-85E2-1FE2B1003CBC@yahoo.com>

On Oct 21, 2015, at 10:35, Random832 <random832 at fastmail.com> wrote:
> 
> Victor Stinner <victor.stinner at gmail.com>
> writes:
>> You are changing the semantic of Python. In Python, you should be able
>> to override builtin functions.
> 
> What I am proposing is a generalized way to add objects that do not have
> a literal to a function's constants pool. The global name within the
> first version of the function is simply used as a placeholder for
> loading the new constant.
> 
> It could just as easily be something like
> 
> def foo():
>   return _1(abc)
> 
> foo = add_consts(foo, {'_1':len})
> 
>> Binding a global name to a local name is a known trick to optimize a
>> function, but you have to write it explicitly, and again you loose the
>> ability to override len.
> 
> This would also have to be written explicitly, and the whole point is to
> lose the ability to override things. Explicitly.
> 
>> Example:
>> 
>> def f(len=len):
>>   return len("abc")
>> 
>> Here len becomes a fast local variable.
> 
> It'd be even faster as a constant, and that way people couldn't call it
> as f(len=whatever) [so an optimizer doesn't have to deal with the
> possibility of people doing that]
> 
>>> Decorators could be included to freeze only built-ins, to freeze all
>>> globals and built-ins, or to have a list of names to exclude. So you
>>> could have @freeze_builtins or @freeze_all_globals for each function
>>> that needs heavy optimization.
>> 
>> I cited "previous attempts" which failed.
>> 
>> What you describe is close to my "readonly" attempt which tried to
>> make module and type namespaces readonly:
>> https://hg.python.org/sandbox/readonly/file/tip/READONLY.txt
> 
> This would apply to the function, not the namespace.
> 
>> Making namespaces read-only simply doesn't work. It's very common to
>> modify namespaces. I added callbacks to disable optimizations when a
>> namespace is modified. But this design doesn't scale: you add a
>> complexity of O(n) when a namespace is modified, even if optimized
>> functions are never called.
> 
> Nothing happens when a namespace is modified, because the function is no
> longer using the namespace in any form. This adds to the cost of
> defining a function, but in most cases (I'm not 100% sure of the
> semantics for nested functions, let alone the implementation) this could
> be done at compile time, so it would be a one-time cost.

Nested functions are just like top-level functions: at compile time, the function body is compiled to a code object (bytecode, a list of free variables, etc.) that gets stored as a hidden constant; at runtime, all that happens is building a function object out of it (and, if def rather than lambda, binding it to a name). The only way nested functions are more costly at runtime is that they're more likely to have free variables that have to get bound to cells, but even that isn't a big deal.

Anyway, if you just assume that free variables can never be optimized out (which seems fair for a first implementation), you should be able to treat nested functions the same as global functions.

If you want to later optimize out free variables bound to constants, you need to add some way to guard the local namespace?which isn't really a dict, but an array in each frame (although you only have to guard the frames that have cells bound to some closure). That seems doable too, but probably worth skipping in the first version.

From geoffspear at gmail.com  Wed Oct 21 21:49:36 2015
From: geoffspear at gmail.com (Geoffrey Spear)
Date: Wed, 21 Oct 2015 15:49:36 -0400
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <5627E6F7.50200@mrabarnett.plus.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
Message-ID: <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>

On Wed, Oct 21, 2015 at 3:26 PM, MRAB <python at mrabarnett.plus.com> wrote:
>
> The C preprocessor has #elif. Is that strange too? :-)
>
> (It could be argued that it _is_ strange that the C preprocessor has an
> explicitly-terminated "#if ... #endif", whereas the C language itself
> doesn't have an explicitly-terminated "if".)
>
>
"We stole this short spelling from C" shouldn't be an argument in favor of
any spelling, in my opinion.

(Not creating a second way to do something or deprecating a language
keyword is a much stronger argument, of course; I'm as opposed to this
proposal as I'd be to a propose to rename strptime and strftime, but I'd
also think "but that's how C spells them!" would be a bad argument in that
case. Don't even get me started about Popen objects...)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/018d377e/attachment.html>

From random832 at fastmail.com  Wed Oct 21 22:00:41 2015
From: random832 at fastmail.com (Random832)
Date: Wed, 21 Oct 2015 16:00:41 -0400
Subject: [Python-ideas] Add specialized bytecode with guards to functions
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <87pp08w4em.fsf@fastmail.com>
 <CAMpsgwZ2qB1S9j1tJusXEsLPdxgUUBpT0eROfD-pXOa6wasTjg@mail.gmail.com>
 <878u6ww2e3.fsf@fastmail.com>
 <1C6ACCBD-EB5F-49EB-85E2-1FE2B1003CBC@yahoo.com>
Message-ID: <87pp08htzq.fsf@fastmail.com>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:
> Nested functions are just like top-level functions:

The point is that my suggested feature is a decorator (or decorator-like
function) attached to the function, to modify the function to replace
free variables with constants. This means if the decorator is attached
to the inner function it gets executed every time the outer function is
executed, and if it is attached to the outer function we have to decide
whether it will walk through that function's inner functions and change
them too.


From abarnert at yahoo.com  Wed Oct 21 22:01:04 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Wed, 21 Oct 2015 13:01:04 -0700
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <5627E6F7.50200@mrabarnett.plus.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
Message-ID: <1E164613-20A4-4616-B392-3CB19957D74B@yahoo.com>

On Oct 21, 2015, at 12:26, MRAB <python at mrabarnett.plus.com> wrote:
> 
>> On 2015-10-21 19:16, Sven R. Kunze wrote:
>>> On 21.10.2015 20:05, Ryan Gonzalez wrote:
>>> The Python Zen says:
>>> 
>>> There should be one-- and preferably only one --obvious way to do it.
>>> 
>>> elif uses less typing, anyway. Why type 3 more characters when you
>>> really don't need to? :D
>> 
>> Because it's one of those "Python looks strange"? ;-)
> The C preprocessor has #elif. Is that strange too? :-)
> 
> (It could be argued that it _is_ strange that the C preprocessor has an
> explicitly-terminated "#if ... #endif", whereas the C language itself
> doesn't have an explicitly-terminated "if".)

C++ of course solved that by embedding a whole compile-time language that's even less like C than the preprocessor, where the clauses have to be bounded in angle brackets, removing all ambiguity and leading to nice readable things like this:

    typedef typename if_c<spam, SpamType,
        typename if_c<eggs, EggsType, CheeseType>::type>
        ::type type;

Compare to:

    if (spam) type = SpamType;
    else if (eggs) type = EggsType;
    else type = CheeseType;

The second one leaves it unclear to the reader that the final else is actually part of the second if which is itself controlled by the first if's else. If you indebted this "properly" it would be a lot uglier. Which is the whole problem elif is meant to solve. But the first version makes the tree structure the clearest thing in the whole construction, without needing elif, so it's clearly better. ;)


From alexander.belopolsky at gmail.com  Wed Oct 21 22:09:06 2015
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 21 Oct 2015 16:09:06 -0400
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de> <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
Message-ID: <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>

On Wed, Oct 21, 2015 at 3:49 PM, Geoffrey Spear <geoffspear at gmail.com>
wrote:

> "We stole this short spelling from C" shouldn't be an argument in favor of
> any spelling, in my opinion.


Can you explain your opinion?  C is by far the most influential language in
the history of computing.  As a source of keywords it is second only to
English.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/00dd31e7/attachment-0001.html>

From srkunze at mail.de  Wed Oct 21 22:24:04 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 21 Oct 2015 22:24:04 +0200
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <A681948F-AAB8-4032-86F0-E1202A6E1237@gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <A681948F-AAB8-4032-86F0-E1202A6E1237@gmail.com>
Message-ID: <5627F464.5030700@mail.de>

On 21.10.2015 21:29, Ryan Gonzalez wrote:
> O October 21, 2015 2:26:47 PM CDT, MRAB <python at mrabarnett.plus.com> wrote:
>> On 2015-10-21 19:16, Sven R. Kunze wrote:
>>> Because it's one of those "Python looks strange"? ;-)
>> The C preprocessor has #elif. Is that strange too? :-)
>>
>> (It could be argued that it _is_ strange that the C preprocessor has an
>> explicitly-terminated "#if ... #endif", whereas the C language itself
>> doesn't have an explicitly-terminated "if".)
>>
> It's the C preprocessor. How is it NOT strange? :)

:D You made my day!

Best,
Sven

From python at mrabarnett.plus.com  Wed Oct 21 22:30:53 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 21 Oct 2015 21:30:53 +0100
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
Message-ID: <5627F5FD.8080707@mrabarnett.plus.com>

On 2015-10-21 20:49, Geoffrey Spear wrote:
>
>
> On Wed, Oct 21, 2015 at 3:26 PM, MRAB <python at mrabarnett.plus.com
> <mailto:python at mrabarnett.plus.com>> wrote:
>
>     The C preprocessor has #elif. Is that strange too? :-)
>
>     (It could be argued that it _is_ strange that the C preprocessor has an
>     explicitly-terminated "#if ... #endif", whereas the C language itself
>     doesn't have an explicitly-terminated "if".)
>
>
> "We stole this short spelling from C" shouldn't be an argument in favor
> of any spelling, in my opinion.
>
> (Not creating a second way to do something or deprecating a language
> keyword is a much stronger argument, of course; I'm as opposed to this
> proposal as I'd be to a propose to rename strptime and strftime, but I'd
> also think "but that's how C spells them!" would be a bad argument in
> that case. Don't even get me started about Popen objects...)
>
Modula-2 spells it "elsif". The problem there is that you then have to 
be very
careful when talking about it to someone else: "Did you say 'elsif' or 
'else if'?"


From kmod at dropbox.com  Wed Oct 21 22:55:27 2015
From: kmod at dropbox.com (Kevin Modzelewski)
Date: Wed, 21 Oct 2015 13:55:27 -0700
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
Message-ID: <CAO=oM6vB-OH-e8RgyPh22yBDQ3cHp7jGFxqEHot++rDX4hp9XA@mail.gmail.com>

I don't think there will be that many cases that len() gets called on a
constant object, especially in performance-sensitive code since that would
be easy to spot and get rid of.  And without some good knowledge of the
argument, I would guess that it's only marginally helpful to know that
"len" is the builtin len.  I'm not saying that CPython couldn't add
techniques like this, but I think it might need to go decently far into JIT
territory to really make use of it.  For example, which functions would
have multiple versions generated?  There could be startup+memory overheads
if it gets applied to all functions, so maybe there needs to be some sort
of light profiling to determine when to produce the optimized version.  I'm
biased but I think this should be left to the JITs :)

On the other hand, I think it would be pretty interesting for the core
python community to come up with source-level constructs that could help
with this sort of thing.  For example, one idea is to make it canonical to
do something like:
def f():
  from __builtin__ import len
  return len("abc")

and then let implementations pattern-match this to statically resolve the
import.  This doesn't quite work since you can mutate __builtin__ (or
override __import__); maybe a new module name could be used, or maybe there
could be some sort of "bind_globals()" decorator.  I think these are
interesting since they could improve name-resolution in a targeted way, and
also give more information to the JITs.

kmod

On Wed, Oct 21, 2015 at 8:43 AM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> Hi,
>
> I would like to share with you an idea to try to optimize CPython.
>
> My previous attempts to optimize CPython failed because they changed the
> language semantic. For example, it's not possible to replace len('abc')
> with 3 because it is technically possible to override the builtin len()
> function.
>
> I propose to add a new "FAT" mode to Python to allow to add specialized
> bytecode to function with guards. Example:
>
>     >>> import builtins
>     >>> def f(): return len("abc")
>     ...
>     >>> f()
>     3
>
>     >>> def g(): return 3
>     ...
>     >>> i=f.add(g)   # i is the index of the new specialized bytecode
>     >>> f.add_dict_key_guard(i, builtins.__dict__, 'len')
>     >>> f()
>     3
>
>     >>> builtins.len = lambda obj: "len"
>     >>> f()
>     'len'
>
> In this example, the f() function gets a fast version (g()) returning
> directly 3 instead of calling len("abc"). The optimization is disabled
> when the builtin len() function is modified (when
> builtins.__dict__['len'] is modified). (The example is not complete, we
> need a second guard on the current global namespace, but I wanted to
> write a short example.)
>
> With such machinery, it becomes possible to implement interesting
> optimizations. Some examples:
>
> * inline a function instead of calling it: calling a Python function is
>   "expensive". For example, if you inline the _get_sep() call in
>   posixpath.isabs(), the function becomes 20% faster.
>
> * move invariant out of the loop: classic micro-optimization done
>   manually. For example, set "append = obj.append" before the loop and
>   then call "append(item)" instead of "obj.append(item)" in the loop
>   body. On a microbenchmark, it's between 5% (1 item) and 30% faster (10
>   items or more)
>
> * call pure functions at the compilation. A pure function has no side
>   effect. Example: len(str).
>
>
> Technical Challenges
> ====================
>
> The problem is the cost of "guards". A guard is a check done before
> calling a specialized function. Example of guards:
>
> * Type of a function parameter
> * Watch a dictionary key
> * Watch a function (especially it's __code__ attribute)
>
> Watching a dictionary key is a very important guard to disable an
> optimization when a builtin function is modified, when a function is
> replaced in a namespace, etc. The problem is that a dictionary lookup is
> not cheap: we have to get the hash value of the key, browse the hash
> table to find the bucket which may require multiple iterations, then we
> have to compare the key value, etc.
>
> To have faster guard, I propose to create a subtype of dict which
> provides a global "version" of the dictionary, incremented at each
> modification (create a new key, modify a key, delete a key) and a
> "version" per key=>value mapping, incremented each time
> that the mapping is modified. The lookup can be avoided when the
> dictionary is not modified. If a different key is modified, we need a
> lookup, but only once (we store the new global version to avoid a lookup
> at the next guard check).
>
>
> Limitations
> ===========
>
> Specialized bytecode is build at compilation, *not* at runtime: it's not a
> just-in-time (JIT) compiler. A JIT can implement even more efficient
> optimizations and can use better guards. Please continue to use PyPy for
> best performances! ;-)
>
> The name "FAT" comes from the fact that multiple bytecode versions of a
> function are stored in the memory, so a larger memory footprint
> and larger .pyc files on disk can be expected.
>
> Checking guards can also be more expensive than the optimization of the
> specialized bytecode. The optimizer should use an heuristic to decide if
> it's worth to use a specialized bytecode or not depending on the theoric
> speeup and the cost of guards.
>
> My motivation to write FAT Python is that CPython remains the reference
> implementation where new features are implemented, and other Python
> implementations still have issues with C extensions. JIT also has some
> issues, like longer startup time, slow warmup and memory footprint (this
> one may also be an issue of FAT Python!).
>
>
> Implementation
> ==============
>
> I wrote a proof-of-concept of my idea:
>
>     https://hg.python.org/sandbox/fatpython/
>
> It's a fork of CPython 3.6. Try it with:
>
>     hg clone https://hg.python.org/sandbox/fatpython/
>     cd fatpython
>     ./configure && make
>     ./python -F
>
> See bench.py, Lib/posixpath.py (isabs) and Lib/test/test_fat.py
> for examples of optimizations.
>
> The command line -F flag enables the FAT mode. By default, *all*
> optimizations are disabled, there is a negligible overhead when the
> FAT mode is not used.
>
> In the FAT mode, the dictionary for modules, classes and instances
> becomes "versionned". Functions gets new methods to support adding
> specialized bytecode with guards.
>
> You can add manually a specialized bytecode for microbenchmarks, but
> functions are *not* optimized automatically. (See the Roadmap below.)
>
>
> How to write an optimizer?
> ==========================
>
> Currently, my proof-of-oncept is only the machinery to support adding
> specialized bytecodes with guards. The next step is to generate
> automatically these specialized versions.
>
> IMHO an optimizer should be implemented in Python and not in C, it's
> easier to write Python code. We can begin with simple optimizations on
> AST. See for example my astoptimizer which implements basic
> optimizations:
>
>     https://bitbucket.org/haypo/astoptimizer/
>
> At the beginning, we may use type hints (PEP 484) and other manual
> hints to help the optimizer. Later, a profiler learning the most common
> types of function parameters can be written for automatic optimizations.
> Example of hints: a list of constants which should not be modified in the
> application. A global "DEBUG" flag is common in applications, relying on
> DEBUG=False helps to remove dead code.
>
>
> Roadmap
> =======
>
> * Finish the proof-of-concept: implement new guards, optimize method
>   calls. For example, guards on module imports are required. Checking
>   if a module is the expected mode can be tricky.
> * Write an optimizer based on manual hints
> * Write a profiler to generate hints
>
> Right now, my main question is if it will be possible and efficient to
> optimize classes, not only simple functions, especially classes defined
> in two different modules. I wrote a proof-of-concept of optimized
> method (with inlining), but I'm not sure yet that it's safe (don't
> change Python semantic).
>
> For more information on FAT Python, read:
>
>    https://hg.python.org/sandbox/fatpython/file/tip/FATPYTHON.rst
>
>
> So, what do you think? Do you expect real speedup on applications, not
> only on microbechmarks? Do you know similar optimizations in other
> scripting languages?
>
> Victor Stinner
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/810dc5b4/attachment-0001.html>

From srkunze at mail.de  Wed Oct 21 22:56:41 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 21 Oct 2015 22:56:41 +0200
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <3840798D-28DD-4AF3-A6A2-4BAA1D89442A@yahoo.com>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <5627CE63.9030704@mail.de>
 <CAKfyG3zU+8138_hvAwbFLLp9mzC1v10+8ae3t6QpPKvXzyNX2A@mail.gmail.com>
 <3840798D-28DD-4AF3-A6A2-4BAA1D89442A@yahoo.com>
Message-ID: <5627FC09.7060106@mail.de>

On 21.10.2015 21:27, Andrew Barnert wrote:
> On Oct 21, 2015, at 11:25, Jonathan Slenders <jonathan at slenders.be> wrote:
>> Just want to say that I'm happy to see that lately the disadvantages of inheritance (which are already known for a very long time) are getting more attention.
>> It's not bad by definition, but there's so much truth in Sandy her talk and I think for many Python projects, we went way too far into "abusing" inheritance.
>> Actually, it's a bit unfortunate that we made inheritance so user friendly and powerful in Python that for many people it became the logical way to extend or reuse some code.
> One of the most important things people have learned about OO over the past two decades is that subtyping, implementation extension, and implementation mixins, and interface-extending mixins (think collections.abc.Sequence adding count for you) are all different things. Compared to the other languages in existence at the time of Python 1.x or even the 2.2/2.3 changeover, it's hard to fault Python. The fact that it can easily be used to write bad code is a little unfortunate, but the fact that it can also easily be used to write good code, when other languages either don't allow some things to be expressed, force them to be expressed in clumsy or limited ways, or force you to misuse inappropriate features instead more than makes up for it. In particular, any language that has fixed structure layouts and vtables makes is much more unfriendly to both kinds of mixins, and makes interface subtyping clumsy; Python has no such problems. Yeah, maybe we could design something better in 2015, but based on the knowledge people had at the time?

True. Just because everything is possible does not necessarily mean one 
should do it.


@Jonathan
Inheritance has its disadvantages. The bigger the class hierarchy, the 
uglier it becomes. (We have this with the django view class system.)

I see the same coming for the dict discussion if people insist on having 
separate leaf classes for all possible orthogonal aspects. That does not 
scale and it's hard to test ( --> @core-devs ).


Something that's become clearer and clearer to me is that *nobody could 
have known this in advance*. It seemed like a good idea to start with 
several different base classes for each aspect (not knowing that they 
could be combine later). But now, people start to realize they might 
need these different aspects in a single implementation.

So, it's the time to think about how to solve it in a scalable and 
maintainable way. Multiple inheritance might be a solution, composition 
is another. This time I think composition is the better one.


Best,
Sven


PS: Funny coincidence, Sandi uses the aspect "order" for the house. The 
same as a dict could have an aspect "order".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/1f8606a9/attachment.html>

From antoine at python.org  Wed Oct 21 23:03:00 2015
From: antoine at python.org (Antoine Pitrou)
Date: Wed, 21 Oct 2015 21:03:00 +0000 (UTC)
Subject: [Python-ideas] Add specialized bytecode with guards to functions
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
 <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
 <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>
 <CAP1=2W74-nkHvVjyE6sf_cJqpEMhjj3zmKxm-ViP=B-G7D67Eg@mail.gmail.com>
Message-ID: <loom.20151021T225918-737@post.gmane.org>

Brett Cannon <brett at ...> writes:
> 
> Exactly, and especially if the check is cheap. After that you start to
potentially get into granularity, e.g. do you care if any?built-in changed
or do you only care if a specific?built-in changed, and how do either
approach affect performance? The trick is not breaking compatibility and not
hurting performance in the general case. But yes, knowing when the uncommon
case of overriding a built-in occurs would allow for a bunch of optimization
techniques from inlining to using a faster implementation that is compatible
with the standard built-in.

I think adding a version number to dicts would be a very cheap change
and would potentially help several CPython-based projects (including
perhaps Numba some day, though nowadays it wouldn't benefit).

However, I would also like to challenge the idea that an accelerator
has to be 100% compatible. If your API is opt-in (which Numba's is),
then you can happily relax some constraints (provided it is documented)
in exchange for much faster code. Our experience with Numba is that
nobody cares if you can't monkeypatch the len() builtin, for example.

(Numba actually goes quite far in that direction since e.g. its
integers are fixed-width; the speed of generated code, for supported
Python constructs, usually approaches or reaches C, Fortran or even
hand-tuned assembly speed)

Regards

Antoine.

From antoine at python.org  Wed Oct 21 23:09:07 2015
From: antoine at python.org (Antoine Pitrou)
Date: Wed, 21 Oct 2015 21:09:07 +0000 (UTC)
Subject: [Python-ideas] Add specialized bytecode with guards to functions
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
Message-ID: <loom.20151021T230356-723@post.gmane.org>

Victor Stinner <victor.stinner at ...> writes:
> 
> 2015-10-21 18:20 GMT+02:00 Guido van Rossum <guido at ...>:
> >  I haven't tried the prototype yet, but I wonder if this mightn't be a
> > useful addition to an existing optimizing Python system, e.g. PyPy or (plug
> >  Pyston?
> 
> I'm not sure that specialized functions with guards would benefit to
> JIT compilers.
> 
> A best match is with Cython, pythran and other projects generating a
> specialized version of a function (in native code), maybe releasing
> the GIL. It would be cool to be able to use a specialized version
> under some conditions. For example, if parameters are all floats, use
> the specialized version optimized by Cython, other fallback to the
> (slow) regular bytecode. The challenge is to automate everything and
> make it easy to use.
> 
> I don't know if Numba would match with this design.

Numba is primarily a JIT compiler but next release will also feature
a limited AOT compilation mode (meaning you can compile extension
modules for given explicit signatures of whatever functions you want
to compile, and don't need Numba to import or run those functions).
Documentation preview here:
http://numba.pydata.org/numba-doc/dev/user/pycc.html

As a JIT compiler, however, Numba is special as it's opt-in and doesn't
claim to support the whole range of Python semantics (far from it).
Numba takes your Python code and pretty much assumes it has the same
dynamicity as Fortran and C++ code, runs type inference and compiles
a very fast version out of it, for the CPU (by default) or for the
GPU (with a specific programming model).

Regards

Antoine.



From victor.stinner at gmail.com  Wed Oct 21 23:11:01 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 21 Oct 2015 23:11:01 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAO=oM6vB-OH-e8RgyPh22yBDQ3cHp7jGFxqEHot++rDX4hp9XA@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAO=oM6vB-OH-e8RgyPh22yBDQ3cHp7jGFxqEHot++rDX4hp9XA@mail.gmail.com>
Message-ID: <CAMpsgwbMN-Pab9JKpBzgHzyq3dNJdg7w_ONX8nCknVf2N+garg@mail.gmail.com>

Hi,

2015-10-21 22:55 GMT+02:00 Kevin Modzelewski <kmod at dropbox.com>:
> I don't think there will be that many cases that len() gets called on a
> constant object, especially in performance-sensitive code since that would
> be easy to spot and get rid of.

Hum, I guess that almost no code calls len() on a constant string :-)
In my experience astoptimizer, I noticed that it becomes more common
when you combine it with constant folding optimization. Many strings
are "constants" in Python and commonly tested. Example: "if
sys.platform.startswith('freebsd'): ...". If sys.platform is "linux",
it's dead code and the whole if can be removed.

>  And without some good knowledge of the
> argument, I would guess that it's only marginally helpful to know that "len"
> is the builtin len.

I took the len("abc") example just because it's very easy to
understand :-) It's just to explain the principle. Later, if you
combine multiple optimizations, I bet that it will become really
interesting on real applications. But first we need a framework to
make these optimizations legit in Python (don't change Python
semantic).

The goal is to optimize classes, not only builtin functions.

>  I'm not saying that CPython couldn't add techniques
> like this, but I think it might need to go decently far into JIT territory
> to really make use of it.  For example, which functions would have multiple
> versions generated?  There could be startup+memory overheads if it gets
> applied to all functions, so maybe there needs to be some sort of light
> profiling to determine when to produce the optimized version.  I'm biased
> but I think this should be left to the JITs :)

We will need heuristic to estimate the theoric performance speedup
with specialized bytecode, to then decide if it's worth to keep it or
not.

In the long term, we can design a profiler for profile guided
optimizations (PGO).

In the short term, we can use hints as type hints or manual hints, to
decide which functions should be optimized or not.

> On the other hand, I think it would be pretty interesting for the core
> python community to come up with source-level constructs that could help
> with this sort of thing.  For example, one idea is to make it canonical to
> do something like:
> def f():
>   from __builtin__ import len
>   return len("abc")
>
> and then let implementations pattern-match this to statically resolve the
> import. (...)

As written in other messages, they are already many ways to reduce the
"overhead" of the Python language. But I'm trying to write an generic
optimizer which would not require to modify the source code.

Maybe we can modify Python to make some of these optimizations easy to
use, but I'm not sure that it's worth it.

Victor

From victor.stinner at gmail.com  Wed Oct 21 23:19:41 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 21 Oct 2015 23:19:41 +0200
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <loom.20151021T225918-737@post.gmane.org>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
 <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
 <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>
 <CAP1=2W74-nkHvVjyE6sf_cJqpEMhjj3zmKxm-ViP=B-G7D67Eg@mail.gmail.com>
 <loom.20151021T225918-737@post.gmane.org>
Message-ID: <CAMpsgwZdPFKt2Qzg-eox5e7j1Oj-ta5teRBGV-f7OqZXNCLLkA@mail.gmail.com>

2015-10-21 23:03 GMT+02:00 Antoine Pitrou <antoine at python.org>:
> I think adding a version number to dicts would be a very cheap change
> and would potentially help several CPython-based projects (including
> perhaps Numba some day, though nowadays it wouldn't benefit).

I tried to leave the dict type unchanged (it's not true in the PoC
right now, but I'm working on fixing this). Currently, the versionned
dictionary type (fat.verdict) is only used when Python starts in FAT
mode and only in a few places: module, class and instance
dictionaries. I'm not sure that it's worth to add the versionning
feature to the base dict type. It uses a little bit more memory, and
IMHO the dict type is already heavy in term of memory usage (compared
to a compact C array).

I'm trying to keep all FAT features optional to be able to easily
compare performances, but also ensure that the FAT mode can be added
to CPython without hurting performances (CPU and memory) when it's
disabled.

> However, I would also like to challenge the idea that an accelerator
> has to be 100% compatible. If your API is opt-in (which Numba's is),
> then you can happily relax some constraints (provided it is documented)
> in exchange for much faster code. Our experience with Numba is that
> nobody cares if you can't monkeypatch the len() builtin, for example.

Sure, it will be possible to skip some guards if you know that
builtins are never monkey-patched, that a module variable is constant,
that a class will never be modified, etc. This will reduce even more
the cost of guards (especially when all guards are skipped :-)).

But I prefer to start with something that doesn't change the Python
semantic at all, and then provide opt-in options to give more freedom
to the optimizer.

Victor

From tjreedy at udel.edu  Thu Oct 22 03:11:33 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 21 Oct 2015 21:11:33 -0400
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <5627CE63.9030704@mail.de>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <5627CE63.9030704@mail.de>
Message-ID: <n09d4f$4u9$1@ger.gmane.org>

On 10/21/2015 1:41 PM, Sven R. Kunze wrote:
> On 21.10.2015 00:50, Chris Angelico wrote:
>> She recommends a massive superclass that's capable of any form of injection
>
> Nope. She does not.
>
> The "superclass" is not "massive" at all. It is even slimmer as
> orthogonal aspects are refactored out into separate entities. In fact,
> it makes it even easier to test and maintain these separate aspects (the
> core dev should be interested in that). Furthermore, it's, of course, up
> to debate which aspects should be injectable and which are not.

The dict class itself is, in a sense, a poor example for this 
discussion.  It is a critical part of Python's infrastructure, involved 
in a large fraction of executed statements.  It therefore needs to be as 
fast as possible.  For CPython, this means a heavily optimized C 
implementation that disallows injection and that takes shortcuts like 
bypassing method lookups.  This makes the usefulness of subclassing limited.

Of course, before 2.2, dict (and other built-in types) could not even be 
subclassed.  UserDict is the 'user base dict class', meant to be 
subclassed by users.  A solution to UserDict being too slow could be a C 
accelerator that did not bypass method lookups.  A revised UserDict 
could be designed for injection in the way being discussed.

-- 
Terry Jan Reedy


From yselivanov.ml at gmail.com  Thu Oct 22 04:24:02 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Wed, 21 Oct 2015 22:24:02 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
Message-ID: <562848C2.4090509@gmail.com>

Hi,

I was a little bit frustrated that Sublime Text and Atom didn't support
all Python 3 features (mainly annotations & async/await syntax), so I
decided to write a new highlighter:

     https://github.com/MagicStack/MagicPython

In the process, we had to make a decision on how to highlight raw string
literals -- r''.  Many existing highlighters assume that all raw strings
are regexps, and highlight them as such, i.e. '\s' and '\n' will be
highlighted.

I think that it might be a good idea to state the following in PEP 8:

- use r'...' strings for raw strings that describe regular expressions;
these strings might be highlighted specially in some editors.

- use R'...' strings for raw strings; editors *should not* highlight
any escaped characters in them.

What do you think?

Yury


From ben+python at benfinney.id.au  Thu Oct 22 04:44:14 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 22 Oct 2015 13:44:14 +1100
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
References: <562848C2.4090509@gmail.com>
Message-ID: <858u6vpqpt.fsf@benfinney.id.au>

Yury Selivanov <yselivanov.ml at gmail.com>
writes:

> I was a little bit frustrated that Sublime Text and Atom didn't
> support all Python 3 features (mainly annotations & async/await
> syntax), so I decided to write a new highlighter:

Thanks for scratching your itch and releasing the result as free software!

> In the process, we had to make a decision on how to highlight raw
> string literals -- r''. Many existing highlighters assume that all raw
> strings are regexps, and highlight them as such, i.e. '\s' and '\n'
> will be highlighted.

That is evidently a simple mistake. Merely knowing that a token is a raw
string does not justify the assumption that the string is a regular
expression, or a filesystem entry name, or a line in a network protocol,
or anything except plain text.

Perhaps some more explicit context could be used to signal what the
intent of a raw string is, but you'd need to find a strong consensus
that programmers actually intend that. ?It's a raw string? doesn't
justify any of those assumptions.

> I think that it might be a good idea to state the following in PEP 8:

No, I don't think the mistaken assumptions you've described should be
enshrined in a style guide. Instead, the mistaken assumptions should be
changed.

-- 
 \           ?We have clumsy, sputtering, inefficient brains?. It is a |
  `\     *struggle* to be rational and objective, and failures are not |
_o__) evidence for an alternative reality.? ?Paul Z. Myers, 2010-10-14 |
Ben Finney


From yselivanov.ml at gmail.com  Thu Oct 22 04:53:05 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Wed, 21 Oct 2015 22:53:05 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <858u6vpqpt.fsf@benfinney.id.au>
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
Message-ID: <56284F91.8020600@gmail.com>

On 2015-10-21 10:44 PM, Ben Finney wrote:
>> >In the process, we had to make a decision on how to highlight raw
>> >string literals -- r''. Many existing highlighters assume that all raw
>> >strings are regexps, and highlight them as such, i.e. '\s' and '\n'
>> >will be highlighted.
> That is evidently a simple mistake. Merely knowing that a token is a raw
> string does not justify the assumption that the string is a regular
> expression, or a filesystem entry name, or a line in a network protocol,
> or anything except plain text.

I agree 100%.

But: github, gitlab, Atom, Sublime Text, and many other tools
assume that raw strings (with lowercase r) are regexps.  If you
don't highlight them as such, people think that it's a bug.

Since I wanted MagicPython to be a drop-in replacement for
standard highlighters, I simply *could not* change this
behavior.  It's already a standard of some sorts, whether we
like it or not.

> Perhaps some more explicit context could be used to signal what the
> intent of a raw string is, but you'd need to find a strong consensus
> that programmers actually intend that. ?It's a raw string? doesn't
> justify any of those assumptions.
>

If we want to design some special marker for highlighters to
hint what language is in the string, I'd strongly suggest that
it should be before the string literal.  For instance, it
*won't* be possible for most highlighters to detect this:

    my_re = '''...
    ...''' # regex

Yury

From rymg19 at gmail.com  Thu Oct 22 04:58:10 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 21 Oct 2015 21:58:10 -0500
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <56284F91.8020600@gmail.com>
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
 <56284F91.8020600@gmail.com>
Message-ID: <15E3326A-68CA-42DA-BF2E-F17FDF797721@gmail.com>



On October 21, 2015 9:53:05 PM CDT, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>On 2015-10-21 10:44 PM, Ben Finney wrote:
>>> >In the process, we had to make a decision on how to highlight raw
>>> >string literals -- r''. Many existing highlighters assume that all
>raw
>>> >strings are regexps, and highlight them as such, i.e. '\s' and '\n'
>>> >will be highlighted.
>> That is evidently a simple mistake. Merely knowing that a token is a
>raw
>> string does not justify the assumption that the string is a regular
>> expression, or a filesystem entry name, or a line in a network
>protocol,
>> or anything except plain text.
>
>I agree 100%.
>
>But: github, gitlab, Atom, Sublime Text, and many other tools

GitHub and Atom both use language-python.

>assume that raw strings (with lowercase r) are regexps.  If you
>don't highlight them as such, people think that it's a bug.
>
>Since I wanted MagicPython to be a drop-in replacement for
>standard highlighters, I simply *could not* change this
>behavior.  It's already a standard of some sorts, whether we
>like it or not.
>
>> Perhaps some more explicit context could be used to signal what the
>> intent of a raw string is, but you'd need to find a strong consensus
>> that programmers actually intend that. ?It's a raw string? doesn't
>> justify any of those assumptions.
>>
>
>If we want to design some special marker for highlighters to
>hint what language is in the string, I'd strongly suggest that
>it should be before the string literal.  For instance, it
>*won't* be possible for most highlighters to detect this:
>
>    my_re = '''...
>    ...''' # regex
>
>Yury
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From rymg19 at gmail.com  Thu Oct 22 04:52:42 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 21 Oct 2015 21:52:42 -0500
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562848C2.4090509@gmail.com>
References: <562848C2.4090509@gmail.com>
Message-ID: <1F751EE2-ED99-4F21-BE0F-DE1CA0D0E3C4@gmail.com>



On October 21, 2015 9:24:02 PM CDT, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>Hi,
>
>I was a little bit frustrated that Sublime Text and Atom didn't support
>all Python 3 features (mainly annotations & async/await syntax), so I
>decided to write a new highlighter:
>
>     https://github.com/MagicStack/MagicPython
>

Now see if you can get the Linguist guys to use it! :) Every time I look at the Fbuild source online, my eyes sting due to the lack of annotation support in language-python, which screws up highlighting for a bit:

https://github.com/felix-lang/fbuild/blob/master/lib/fbuild/builders/c/gcc/__init__.py#L436 - Keywords aren't highlighted for several lines.

Or:

https://github.com/felix-lang/fbuild/blob/master/lib/fbuild/builders/bison.py#L15

Where almost (but not quiet) everything is un-highlighted! :O

>In the process, we had to make a decision on how to highlight raw
>string
>literals -- r''.  Many existing highlighters assume that all raw
>strings
>are regexps, and highlight them as such, i.e. '\s' and '\n' will be
>highlighted.
>
>I think that it might be a good idea to state the following in PEP 8:
>
>- use r'...' strings for raw strings that describe regular expressions;
>these strings might be highlighted specially in some editors.
>
>- use R'...' strings for raw strings; editors *should not* highlight
>any escaped characters in them.
>
>What do you think?
>
>Yury
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

From yselivanov.ml at gmail.com  Thu Oct 22 05:22:05 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Wed, 21 Oct 2015 23:22:05 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <15E3326A-68CA-42DA-BF2E-F17FDF797721@gmail.com>
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
 <56284F91.8020600@gmail.com> <15E3326A-68CA-42DA-BF2E-F17FDF797721@gmail.com>
Message-ID: <5628565D.1010509@gmail.com>



On 2015-10-21 10:58 PM, Ryan Gonzalez wrote:
> On October 21, 2015 9:53:05 PM CDT, Yury Selivanov<yselivanov.ml at gmail.com>  wrote:
>> >On 2015-10-21 10:44 PM, Ben Finney wrote:
>>>>> >>> >In the process, we had to make a decision on how to highlight raw
>>>>> >>> >string literals -- r''. Many existing highlighters assume that all
>> >raw
>>>>> >>> >strings are regexps, and highlight them as such, i.e. '\s' and '\n'
>>>>> >>> >will be highlighted.
>>> >>That is evidently a simple mistake. Merely knowing that a token is a
>> >raw
>>> >>string does not justify the assumption that the string is a regular
>>> >>expression, or a filesystem entry name, or a line in a network
>> >protocol,
>>> >>or anything except plain text.
>> >
>> >I agree 100%.
>> >
>> >But: github, gitlab, Atom, Sublime Text, and many other tools
> GitHub and Atom both use language-python.
>

Yeah.  I even created a PR to use MagicPython in language-python
a few days ago.

BTW, here's a link to github to show how raw strings are highlighted:
https://github.com/python/cpython/blob/master/Lib/_pydecimal.py#L6087

Yury

From njs at pobox.com  Thu Oct 22 05:31:07 2015
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 21 Oct 2015 20:31:07 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <858u6vpqpt.fsf@benfinney.id.au>
References: <562848C2.4090509@gmail.com>
	<858u6vpqpt.fsf@benfinney.id.au>
Message-ID: <CAPJVwB=e0jPFkFOFpFztqoPP6GSJJwE18pjgdfsEMWo9NVAp8A@mail.gmail.com>

On Wed, Oct 21, 2015 at 7:44 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Yury Selivanov <yselivanov.ml at gmail.com> writes:
>> In the process, we had to make a decision on how to highlight raw
>> string literals -- r''. Many existing highlighters assume that all raw
>> strings are regexps, and highlight them as such, i.e. '\s' and '\n'
>> will be highlighted.
>
> That is evidently a simple mistake. Merely knowing that a token is a raw
> string does not justify the assumption that the string is a regular
> expression, or a filesystem entry name, or a line in a network protocol,
> or anything except plain text.

This isn't necessarily true, just as a matter of like... epistemology.
For example, if hypothetically it turned out that 99% of raw strings
are in fact regular expressions, then knowing something is a raw
string would give you quite a bit of evidence that it's a regular
expression -- quite possibly enough to justify treating it as such for
something like code highlighting.

I haven't actually gathered any data to find out how strong the
association between raw strings and regexen is, but it'd be pretty
easy for someone to do. (Parse a large corpus of python code to
extract all raw strings, randomly subsample 100 of them, review
manually to decide if each is a regex.)

-n

From yselivanov.ml at gmail.com  Thu Oct 22 05:35:18 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Wed, 21 Oct 2015 23:35:18 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <CAPJVwB=e0jPFkFOFpFztqoPP6GSJJwE18pjgdfsEMWo9NVAp8A@mail.gmail.com>
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
 <CAPJVwB=e0jPFkFOFpFztqoPP6GSJJwE18pjgdfsEMWo9NVAp8A@mail.gmail.com>
Message-ID: <56285976.9090609@gmail.com>

On 2015-10-21 11:31 PM, Nathaniel Smith wrote:
> I haven't actually gathered any data to find out how strong the
> association between raw strings and regexen is, but it'd be pretty
> easy for someone to do. (Parse a large corpus of python code to
> extract all raw strings, randomly subsample 100 of them, review
> manually to decide if each is a regex.)

I haven't done this in a scientific way you suggest, but
I did glance over the stdlib when I was testing MP.

Most of the raw strings I saw were either docstrings or
regexps.  Docstrings are possible to detect by highlighters,
so MagicPython does not highlight r'' as regexp if it's
a docstring.

Yury

From scoutoss at gmail.com  Wed Oct 21 20:36:30 2015
From: scoutoss at gmail.com (Scott Sanderson)
Date: Wed, 21 Oct 2015 11:36:30 -0700 (PDT)
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <878u6ww2e3.fsf@fastmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <87pp08w4em.fsf@fastmail.com>
 <CAMpsgwZ2qB1S9j1tJusXEsLPdxgUUBpT0eROfD-pXOa6wasTjg@mail.gmail.com>
 <878u6ww2e3.fsf@fastmail.com>
Message-ID: <ce6ef7bb-e239-427a-aa88-2ee5e465b1e1@googlegroups.com>


>
> It could just as easily be something like 
>
> def foo(): 
>    return _1(abc) 
>
> foo = add_consts(foo, {'_1':len}) 
>

You can actually do this already in pure Python.  We implemented almost 
exactly what's proposed here as the asconstants 
<https://github.com/llllllllll/codetransformer#asconstants> decorator in 
codetransformer <https://github.com/llllllllll/codetransformer>:

>>> from codetransformer.transformers import asconstants
>>> @asconstants(a=1)
>>> def f():
...     return a
...
>>> f()
1
>>> a = 5
>>> f()
1


This works by making two changes to the function's __code__:

1. We add an entry to foo.__code__.co_consts.
2. We rewrite all LOAD_{GLOBAL|NAME} bytecode instructions of that name to 
be LOAD_CONSTs instead.

This makes the function itself slightly faster, since LOAD_CONST is 
generally faster than other loads.  But more importantly, it statically 
freezes the value of a particular name in the function, which makes it 
possible/sane to implement other more interesting 
transformations/optimizations.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151021/a9e4b0e9/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Thu Oct 22 00:30:03 2015
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 22 Oct 2015 11:30:03 +1300
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <5626B8AB.5040806@oddbird.net>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
Message-ID: <562811EB.8020103@canterbury.ac.nz>

Carl Meyer wrote:
> It takes her a couple minutes more to get to the
> real point, which starts at the slide "inheritance is for
> specialization, not for sharing code."

I'm not sure there's any content in that statement.
In a duck-typed language, sharing code is the *only*
reason to use inheritance.

Also, what is "specialisation" anyway? Any time
you have two objects with the same interface, and
mostly-similar but slightly different behaviours,
you could regard one as being a specialisation of
the other. Whether they share any code or not is
an implementation detail.

I think the real issue is that if you specialise
something by inheriting and overriding random
methods, you make your class fragile with respect
to changes in the base class. If the author of
the base class changes something undocumented about
the way its methods interact with each other, your
class may break.

With pluggable behaviours, there is a well-defined
interface between the main class and the classes
representing the behaviours, so that sort of thing
is much less likely to happen.

You could get the same result with inheritance by
designating some methods as "designed to be
overridden", and documenting how they interact with
the rest of the class. Effectively you are then
defining an interface *within* the class and
between the class and its subclasses. This is
really just another implementation of pluggable
behaviours where the plugging mechanism consists
of overriding one or more methods.

So to summarise, I would say that the issue here
isn't really composition vs. inheritance, but
structured vs. ad-hoc behaviour modification.

I wouldn't say that one is always better than the
other. Sometimes you need to do things that the
author of a class didn't anticipate, and then the
ability to override things in an ad-hoc manner is
very useful. But I think the speaker is right to
point out that doing *everything* that way can
lead to a lot of trouble.

-- 
Greg

From ben+python at benfinney.id.au  Thu Oct 22 07:44:00 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 22 Oct 2015 16:44:00 +1100
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
 <CAPJVwB=e0jPFkFOFpFztqoPP6GSJJwE18pjgdfsEMWo9NVAp8A@mail.gmail.com>
Message-ID: <854mhjpie7.fsf@benfinney.id.au>

Nathaniel Smith <njs at pobox.com> writes:

> On Wed, Oct 21, 2015 at 7:44 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > [Automatically interpreting every raw string as a regex pattern] is
> > evidently a simple mistake. Merely knowing that a token is a raw
> > string does not justify the assumption that the string is a regular
> > expression, or a filesystem entry name, or a line in a network
> > protocol, or anything except plain text.
>
> This isn't necessarily true, just as a matter of like... epistemology.

Well, yes, if you like. Epistemically, a sytntax highlighter cannot know
that a raw string is, merely because it's a raw string, definitely a
regular expression pattern.

We have a definition of the language which allows syntax highlighters to
know with certainty what is and is not a particular element of the
language. So if the highlighter shows a sequence of characters as being
what the Python language definition says it is, then it will not be
wrong in any case.

We do not have a definition which allows syntax highlighters to decide
that a raw string is or is not a regular expression, merely because it's
a raw string. So if a highlighter shows a sequence of characters in a
Python program as being a regular expression, it will be wrong for some
cases.

> For example, if hypothetically it turned out that 99% of raw strings
> are in fact regular expressions, then knowing something is a raw
> string would give you quite a bit of evidence that it's a regular
> expression -- quite possibly enough to justify treating it as such for
> something like code highlighting.

Presenting the code highlighted to show particular semantics is a binary
state: it either is shown as (for example) a regular expression, or it
is not. The reader only gets to see what the highlighter decided, not
how certain the epistemic decision was.

How is the person viewing it to know whether the highlighter is wrong
about the intention of the code in any particular case, or if the
highlighter is right and the code doesn't match the author's intention?

If the reader has to second-guess the highlighter (am I wrong here, or
is the highlighter wrong, or both?) every time it doesn't match
expectations, that's a poor syntax highlighter which should never have
made such a binary decision on uncertain data.

-- 
 \           ?Never express yourself more clearly than you are able to |
  `\                                               think.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From tjreedy at udel.edu  Thu Oct 22 09:44:58 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 22 Oct 2015 03:44:58 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562848C2.4090509@gmail.com>
References: <562848C2.4090509@gmail.com>
Message-ID: <n0a465$kdh$1@ger.gmane.org>

On 10/21/2015 10:24 PM, Yury Selivanov wrote:
> Hi,
>
> I was a little bit frustrated that Sublime Text and Atom didn't support
> all Python 3 features (mainly annotations & async/await syntax), so I
> decided to write a new highlighter:
>
>      https://github.com/MagicStack/MagicPython
>
> In the process, we had to make a decision on how to highlight raw string
> literals -- r''.  Many existing highlighters assume that all raw strings
> are regexps, and highlight them as such, i.e. '\s' and '\n' will be
> highlighted.

What 3rd parth editors do is their business.

> I think that it might be a good idea to state the following in PEP 8:
>
> - use r'...' strings for raw strings that describe regular expressions;
> these strings might be highlighted specially in some editors.
>
> - use R'...' strings for raw strings; editors *should not* highlight
> any escaped characters in them.
>
> What do you think?

I think it a bad idea.  For beginners on Windows, 
r'windows\path\file.py' might be more common than r're'.  I have never 
seen R used.

If you wanted to promote the use of the currently rare R for REs, and 
have editors specially mark raw literals with this special prefix, I 
would not mind.

-- 
Terry Jan Reedy



From mal at egenix.com  Thu Oct 22 10:32:43 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 22 Oct 2015 10:32:43 +0200
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <858u6vpqpt.fsf@benfinney.id.au>
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
Message-ID: <56289F2B.6010808@egenix.com>

On 22.10.2015 04:44, Ben Finney wrote:
> Yury Selivanov <yselivanov.ml at gmail.com>
> writes:
> 
>> I was a little bit frustrated that Sublime Text and Atom didn't
>> support all Python 3 features (mainly annotations & async/await
>> syntax), so I decided to write a new highlighter:
> 
> Thanks for scratching your itch and releasing the result as free software!
> 
>> In the process, we had to make a decision on how to highlight raw
>> string literals -- r''. Many existing highlighters assume that all raw
>> strings are regexps, and highlight them as such, i.e. '\s' and '\n'
>> will be highlighted.
> 
> That is evidently a simple mistake. Merely knowing that a token is a raw
> string does not justify the assumption that the string is a regular
> expression, or a filesystem entry name, or a line in a network protocol,
> or anything except plain text.
> 
> Perhaps some more explicit context could be used to signal what the
> intent of a raw string is, but you'd need to find a strong consensus
> that programmers actually intend that. ?It's a raw string? doesn't
> justify any of those assumptions.
> 
>> I think that it might be a good idea to state the following in PEP 8:
> 
> No, I don't think the mistaken assumptions you've described should be
> enshrined in a style guide. Instead, the mistaken assumptions should be
> changed.

Agreed. Highlighters should follow language definitions,
not the other way around ;-)

Yuri: Perhaps you could make the behavior optional in your
highlighter and allow people to turn off the highlighting
as regular expression, if they find they don't like the
highlighting.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Oct 22 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From rob.cliffe at btinternet.com  Thu Oct 22 11:41:44 2015
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 22 Oct 2015 10:41:44 +0100
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <n0a465$kdh$1@ger.gmane.org>
References: <562848C2.4090509@gmail.com> <n0a465$kdh$1@ger.gmane.org>
Message-ID: <5628AF58.1010001@btinternet.com>



On 22/10/2015 08:44, Terry Reedy wrote:
> On 10/21/2015 10:24 PM, Yury Selivanov wrote:
>> Hi,
>>
>> I was a little bit frustrated that Sublime Text and Atom didn't support
>> all Python 3 features (mainly annotations & async/await syntax), so I
>> decided to write a new highlighter:
>>
>>      https://github.com/MagicStack/MagicPython
>>
>> In the process, we had to make a decision on how to highlight raw string
>> literals -- r''.  Many existing highlighters assume that all raw strings
>> are regexps, and highlight them as such, i.e. '\s' and '\n' will be
>> highlighted.
>
> What 3rd parth editors do is their business.
>
>> I think that it might be a good idea to state the following in PEP 8:
>>
>> - use r'...' strings for raw strings that describe regular expressions;
>> these strings might be highlighted specially in some editors.
>>
>> - use R'...' strings for raw strings; editors *should not* highlight
>> any escaped characters in them.
>>
>> What do you think?
>
> I think it a bad idea.  For beginners on Windows, 
> r'windows\path\file.py' might be more common than r're'.  I have never 
> seen R used.
>
> If you wanted to promote the use of the currently rare R for REs, and 
> have editors specially mark raw literals with this special prefix, I 
> would not mind.
>
+1.  I may or may not be typical, but I use regular expressions very rarely.
Rob Cliffe.

From abarnert at yahoo.com  Thu Oct 22 12:00:20 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Thu, 22 Oct 2015 03:00:20 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <n0a465$kdh$1@ger.gmane.org>
References: <562848C2.4090509@gmail.com> <n0a465$kdh$1@ger.gmane.org>
Message-ID: <2302BDF8-74D6-4752-9182-7856F07F1A24@yahoo.com>

On Oct 22, 2015, at 00:44, Terry Reedy <tjreedy at udel.edu> wrote:
> 
>> On 10/21/2015 10:24 PM, Yury Selivanov wrote:
> 
>> I think that it might be a good idea to state the following in PEP 8:
>> 
>> - use r'...' strings for raw strings that describe regular expressions;
>> these strings might be highlighted specially in some editors.
>> 
>> - use R'...' strings for raw strings; editors *should not* highlight
>> any escaped characters in them.
>> 
>> What do you think?
> 
> I think it a bad idea.  For beginners on Windows, r'windows\path\file.py' might be more common than r're'.  

It's also worth noting that an awful lot of code that uses Windows pathnames is either beginner code, local scripts, or closed-source commercial code, which means a typical code search is probably going to vastly underrepresent how common they are in raw strings.

(Of course that same fact means it may be perfectly reasonable for GitHub to assume raw strings are regexps rather than Windows pathnames, even if it isn't reasonable for Python itself, or general-purpose tools like IDLE?)

> I have never seen R used.
> 
> If you wanted to promote the use of the currently rare R for REs, and have editors specially mark raw literals with this special prefix, I would not mind.

That doesn't sound as bad.

But I still don't like it. Where else does Python provide two equivalent ways to do something, specifically to support external semantic connotations? It's like having <> and != both mean the same thing to support people coming up with some language-external difference between the spellings.

(Yes, I realize there are a few cases like this?e.g., someone could use the fact that int and 'int' annotate the same type to give them different connotations?but those are accidental effects of some other language feature; PEP 8 certainly isn't going to suggest using 'int' to mean one thing and int another.)

If we really want there to be a difference, we should have a regex literal syntax?maybe an x or s prefix or something?in place of re.compile(r'?'). 

From storchaka at gmail.com  Thu Oct 22 12:14:51 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 22 Oct 2015 13:14:51 +0300
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562848C2.4090509@gmail.com>
References: <562848C2.4090509@gmail.com>
Message-ID: <n0acur$2hr$1@ger.gmane.org>

On 22.10.15 05:24, Yury Selivanov wrote:
> I was a little bit frustrated that Sublime Text and Atom didn't support
> all Python 3 features (mainly annotations & async/await syntax), so I
> decided to write a new highlighter:
>
>      https://github.com/MagicStack/MagicPython
>
> In the process, we had to make a decision on how to highlight raw string
> literals -- r''.  Many existing highlighters assume that all raw strings
> are regexps, and highlight them as such, i.e. '\s' and '\n' will be
> highlighted.
>
> I think that it might be a good idea to state the following in PEP 8:
>
> - use r'...' strings for raw strings that describe regular expressions;
> these strings might be highlighted specially in some editors.
>
> - use R'...' strings for raw strings; editors *should not* highlight
> any escaped characters in them.
>
> What do you think?

Do highlighters highlight formatting strings ("%*.2f" or 
"{0.addr:0{}x}") specially?


From rosuav at gmail.com  Thu Oct 22 12:52:52 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 22 Oct 2015 21:52:52 +1100
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <2302BDF8-74D6-4752-9182-7856F07F1A24@yahoo.com>
References: <562848C2.4090509@gmail.com> <n0a465$kdh$1@ger.gmane.org>
 <2302BDF8-74D6-4752-9182-7856F07F1A24@yahoo.com>
Message-ID: <CAPTjJmrOYm+VFU6BJvxOYbbXWSfOAdOU=+V_84tbr5FS+BCpNw@mail.gmail.com>

On Thu, Oct 22, 2015 at 9:00 PM, Andrew Barnert via Python-ideas
<python-ideas at python.org> wrote:
> But I still don't like it. Where else does Python provide two equivalent ways to do something, specifically to support external semantic connotations? It's like having <> and != both mean the same thing to support people coming up with some language-external difference between the spellings.
>

I don't know, but since PEP 8 recommends using """ for docstrings and
not ''', it would be entirely possible for someone to assign special
meaning to the use of ''' for a docstring.

ChrisA

From geoffspear at gmail.com  Thu Oct 22 14:19:46 2015
From: geoffspear at gmail.com (Geoffrey Spear)
Date: Thu, 22 Oct 2015 08:19:46 -0400
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
Message-ID: <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>

On Wed, Oct 21, 2015 at 4:09 PM, Alexander Belopolsky <
alexander.belopolsky at gmail.com> wrote:

>
> On Wed, Oct 21, 2015 at 3:49 PM, Geoffrey Spear <geoffspear at gmail.com>
> wrote:
>
>> "We stole this short spelling from C" shouldn't be an argument in favor
>> of any spelling, in my opinion.
>
>
> Can you explain your opinion?  C is by far the most influential language
> in the history of computing.  As a source of keywords it is second only to
> English.
>

Lots of stuff in C favors terseness over readability. Python generally
takes the opposite approach to naming things.

Obviously this is sometimes a cause of confusion to people new to the
language, when people think "oh, Python is just like English, so I should
use 'is' instead of '=='" or "why doesn't `if foo or bar == 'baz'` do what
I mean?", but for the most part having easy-to-read actual English words
for keywords and functions with names that actually say what they do
instead of making some obscure reference to a badly-named C library
function written at a time when limiting identifiers to 6 letters seemed
like a good idea to save precious disk space (or room on a punchcard), in
my opinion, has greatly contributed to Python's reputation as a readable
language. "elif" is a wart, but again, one I think we're stuck with.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151022/0b8e2e29/attachment.html>

From yselivanov.ml at gmail.com  Thu Oct 22 16:04:15 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Thu, 22 Oct 2015 10:04:15 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <n0acur$2hr$1@ger.gmane.org>
References: <562848C2.4090509@gmail.com> <n0acur$2hr$1@ger.gmane.org>
Message-ID: <5628ECDF.7000702@gmail.com>



On 2015-10-22 6:14 AM, Serhiy Storchaka wrote:
> On 22.10.15 05:24, Yury Selivanov wrote:
>> I was a little bit frustrated that Sublime Text and Atom didn't support
>> all Python 3 features (mainly annotations & async/await syntax), so I
>> decided to write a new highlighter:
>>
>>      https://github.com/MagicStack/MagicPython
>>
>> In the process, we had to make a decision on how to highlight raw string
>> literals -- r''.  Many existing highlighters assume that all raw strings
>> are regexps, and highlight them as such, i.e. '\s' and '\n' will be
>> highlighted.
>>
>> I think that it might be a good idea to state the following in PEP 8:
>>
>> - use r'...' strings for raw strings that describe regular expressions;
>> these strings might be highlighted specially in some editors.
>>
>> - use R'...' strings for raw strings; editors *should not* highlight
>> any escaped characters in them.
>>
>> What do you think?
>
> Do highlighters highlight formatting strings ("%*.2f" or 
> "{0.addr:0{}x}") specially?

Many highlight old-style formatting.  Less highlight the new-style.

Github, for example, highlights both.

Yury


From storchaka at gmail.com  Thu Oct 22 18:03:05 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 22 Oct 2015 19:03:05 +0300
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <5628ECDF.7000702@gmail.com>
References: <562848C2.4090509@gmail.com> <n0acur$2hr$1@ger.gmane.org>
 <5628ECDF.7000702@gmail.com>
Message-ID: <n0b1bq$ags$1@ger.gmane.org>

On 22.10.15 17:04, Yury Selivanov wrote:
> On 2015-10-22 6:14 AM, Serhiy Storchaka wrote:
>> Do highlighters highlight formatting strings ("%*.2f" or
>> "{0.addr:0{}x}") specially?
>
> Many highlight old-style formatting.  Less highlight the new-style.
>
> Github, for example, highlights both.

If the fact that not all strings are formatting strings is ignored, you 
are free to ignore the fact that not all raw strings are regular 
expression patterns.



From srkunze at mail.de  Thu Oct 22 18:09:04 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 22 Oct 2015 18:09:04 +0200
Subject: [Python-ideas] OrderedCounter and OrderedDefaultDict
In-Reply-To: <n09d4f$4u9$1@ger.gmane.org>
References: <5620D33A.3050709@feete.org>
 <83EDFBB7-679E-4193-8765-7A2B46F0E2F8@yahoo.com>
 <C94FB06F-6919-4D78-9E53-F94A6DB2DFC7@yahoo.com>
 <9BD02B4D-EB84-4D21-B232-DA023882503E@yahoo.com> <56268487.9050405@mail.de>
 <CAPTjJmpvpCFEkdQunT5-iu7VMX_4UzHVmYoC_p+opkMcJ1+0MQ@mail.gmail.com>
 <5626B8AB.5040806@oddbird.net>
 <CAPTjJmriJ+HHMfJRgO+vYq4Ury652GpUhizvoUQZ0QOsUH-YLA@mail.gmail.com>
 <5627CE63.9030704@mail.de> <n09d4f$4u9$1@ger.gmane.org>
Message-ID: <56290A20.2090602@mail.de>

On 22.10.2015 03:11, Terry Reedy wrote:
> On 10/21/2015 1:41 PM, Sven R. Kunze wrote:
>> The "superclass" is not "massive" at all. It is even slimmer as
>> orthogonal aspects are refactored out into separate entities. In fact,
>> it makes it even easier to test and maintain these separate aspects (the
>> core dev should be interested in that). Furthermore, it's, of course, up
>> to debate which aspects should be injectable and which are not.
>
> The dict class itself is, in a sense, a poor example for this 
> discussion.  It is a critical part of Python's infrastructure, 
> involved in a large fraction of executed statements.  It therefore 
> needs to be as fast as possible.  For CPython, this means a heavily 
> optimized C implementation that disallows injection and that takes 
> shortcuts like bypassing method lookups.  This makes the usefulness of 
> subclassing limited.

The discussion *is* about dict:

normal_dict = dict()
ordered_dict = dict(order=dict.order_by_insert)
sorted_dict = dict(order=sorted)
sorted_default_dict = dict(order=sorted, default=int)

Why couldn't dict() or{} redirect to the super-fast built-in 
C-implementation whereas dict(order=sorted, default=int) redirects to 
some more feature rich one?

As you see, I for one don't see a contradiction between performance and 
features.

Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151022/b1f99eec/attachment.html>

From guido at python.org  Thu Oct 22 18:13:38 2015
From: guido at python.org (Guido van Rossum)
Date: Thu, 22 Oct 2015 09:13:38 -0700
Subject: [Python-ideas] PEP-505: Draft (Coalescing Operators)
In-Reply-To: <CALb0Rk6k+Sh-63SByqAWJ7Hb9NYNzOuoO8S2rBATbf0GSEKTtQ@mail.gmail.com>
References: <CALb0Rk6k+Sh-63SByqAWJ7Hb9NYNzOuoO8S2rBATbf0GSEKTtQ@mail.gmail.com>
Message-ID: <CAP7+vJ+8Yvgd+sFHTo5tnt6-UMf3XcMJYUS7P_ifb2k4Va5a0w@mail.gmail.com>

Hi Mark,

I'm confused by the example

>>> 2 or None ?? err()

The PEP states this raises, but since 2 is already True, and you state that
the operator has higher precedence than 'or', this example shouldn't
evaluate anything to the right of 'or', so it shouldn't raise -- just like
"2 or err()" doesn't raise.

Also, unrelated (but in the same section) I think this operator should have
a precedence higher than 'not'. That is,

>>> not x ?? y

should be parsed as

>>> not (x ?? y)

After all, if we parsed it as

>>> (not x) ?? y

it wouldn't be very useful, since (not x) never returns None, hence y would
never be evaluated here -- this is a redundant way to spell (not x).

Other notes:

"The idea of a None -aware function invocation syntax was discussed on
python- ideas, but the idea was rejected by BDFL." -- I don't think it's
that strong. It's merely that the use cases for x?.attr and x?[key] are
stronger, and various people suggested to focus on the most important.

"[...] except it returns its left operand if that operand is None and
otherwise returns the right operand." -- the part about returning the left
operand if it is None is silly -- there is only one None, so it should just
say "returns None if the left operand is None and otherwise returns the
right operand."

I'll have to think more about the rest.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151022/5b5d81b0/attachment.html>

From srkunze at mail.de  Thu Oct 22 18:19:20 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 22 Oct 2015 18:19:20 +0200
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562848C2.4090509@gmail.com>
References: <562848C2.4090509@gmail.com>
Message-ID: <56290C88.9040302@mail.de>

On 22.10.2015 04:24, Yury Selivanov wrote:
> Hi,
>
> I was a little bit frustrated that Sublime Text and Atom didn't support
> all Python 3 features (mainly annotations & async/await syntax), so I
> decided to write a new highlighter:
>
>     https://github.com/MagicStack/MagicPython
>
> In the process, we had to make a decision on how to highlight raw string
> literals -- r''.  Many existing highlighters assume that all raw strings
> are regexps, and highlight them as such, i.e. '\s' and '\n' will be
> highlighted.
>
> I think that it might be a good idea to state the following in PEP 8:
>
> - use r'...' strings for raw strings that describe regular expressions;
> these strings might be highlighted specially in some editors.
>
> - use R'...' strings for raw strings; editors *should not* highlight
> any escaped characters in them.
>
> What do you think?
>
> Yury

Thanks releasing your package.

I think: leave it as is.

Why?
1) It's not really something that needs a standard.
2) r and R are equivalent.

Best,
Sven

From srkunze at mail.de  Thu Oct 22 18:22:31 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 22 Oct 2015 18:22:31 +0200
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
Message-ID: <56290D47.9030800@mail.de>

On 22.10.2015 14:19, Geoffrey Spear wrote:
> Obviously this is sometimes a cause of confusion to people new to the 
> language, when people think "oh, Python is just like English, so I 
> should use 'is' instead of '=='" or "why doesn't `if foo or bar == 
> 'baz'` do what I mean?", but for the most part having easy-to-read 
> actual English words for keywords and functions with names that 
> actually say what they do instead of making some obscure reference to 
> a badly-named C library function written at a time when limiting 
> identifiers to 6 letters seemed like a good idea to save precious disk 
> space (or room on a punchcard), in my opinion, has greatly contributed 
> to Python's reputation as a readable language.

So, why stop there? ;)

> "elif" is a wart, but again, one I think we're stuck with.

Dangerous thinking.

I don't regard changing it to be a big deal given 10 years or so of 
parallel usage.

Best,
Sven

From mehaase at gmail.com  Thu Oct 22 18:36:07 2015
From: mehaase at gmail.com (Mark E. Haase)
Date: Thu, 22 Oct 2015 12:36:07 -0400
Subject: [Python-ideas] PEP-505: Draft (Coalescing Operators)
In-Reply-To: <CAP7+vJ+8Yvgd+sFHTo5tnt6-UMf3XcMJYUS7P_ifb2k4Va5a0w@mail.gmail.com>
References: <CALb0Rk6k+Sh-63SByqAWJ7Hb9NYNzOuoO8S2rBATbf0GSEKTtQ@mail.gmail.com>
 <CAP7+vJ+8Yvgd+sFHTo5tnt6-UMf3XcMJYUS7P_ifb2k4Va5a0w@mail.gmail.com>
Message-ID: <CALb0Rk7ohDjBa0ib5-awojHkmmhiA2Hb3QPpgBNdUSjuawn1zw@mail.gmail.com>

Yes, that `or` example is incorrect. I have short circuiting and precedence
jumbled up in my head, but thinking about that example has improved my
understanding.

I will fix that and incorporate your other notes shortly.

Thanks!

On Thu, Oct 22, 2015 at 12:13 PM, Guido van Rossum <guido at python.org> wrote:

> Hi Mark,
>
> I'm confused by the example
>
> >>> 2 or None ?? err()
>
> The PEP states this raises, but since 2 is already True, and you state
> that the operator has higher precedence than 'or', this example shouldn't
> evaluate anything to the right of 'or', so it shouldn't raise -- just like
> "2 or err()" doesn't raise.
>
> Also, unrelated (but in the same section) I think this operator should
> have a precedence higher than 'not'. That is,
>
> >>> not x ?? y
>
> should be parsed as
>
> >>> not (x ?? y)
>
> After all, if we parsed it as
>
> >>> (not x) ?? y
>
> it wouldn't be very useful, since (not x) never returns None, hence y
> would never be evaluated here -- this is a redundant way to spell (not x).
>
> Other notes:
>
> "The idea of a None -aware function invocation syntax was discussed on
> python- ideas, but the idea was rejected by BDFL." -- I don't think it's
> that strong. It's merely that the use cases for x?.attr and x?[key] are
> stronger, and various people suggested to focus on the most important.
>
> "[...] except it returns its left operand if that operand is None and
> otherwise returns the right operand." -- the part about returning the left
> operand if it is None is silly -- there is only one None, so it should just
> say "returns None if the left operand is None and otherwise returns the
> right operand."
>
> I'll have to think more about the rest.
>
> --
> --Guido van Rossum (python.org/~guido)
>



-- 
Mark E. Haase
202-815-0201
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151022/24232130/attachment-0001.html>

From brett at python.org  Thu Oct 22 19:24:34 2015
From: brett at python.org (Brett Cannon)
Date: Thu, 22 Oct 2015 17:24:34 +0000
Subject: [Python-ideas] Add specialized bytecode with guards to functions
In-Reply-To: <CAMpsgwZdPFKt2Qzg-eox5e7j1Oj-ta5teRBGV-f7OqZXNCLLkA@mail.gmail.com>
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
 <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
 <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>
 <CAP1=2W74-nkHvVjyE6sf_cJqpEMhjj3zmKxm-ViP=B-G7D67Eg@mail.gmail.com>
 <loom.20151021T225918-737@post.gmane.org>
 <CAMpsgwZdPFKt2Qzg-eox5e7j1Oj-ta5teRBGV-f7OqZXNCLLkA@mail.gmail.com>
Message-ID: <CAP1=2W5JqBJ1Uixi_8=L3F18njx+eC2QRQyKo08s6eLApKJc-A@mail.gmail.com>

On Wed, 21 Oct 2015 at 14:20 Victor Stinner <victor.stinner at gmail.com>
wrote:

> 2015-10-21 23:03 GMT+02:00 Antoine Pitrou <antoine at python.org>:
> > I think adding a version number to dicts would be a very cheap change
> > and would potentially help several CPython-based projects (including
> > perhaps Numba some day, though nowadays it wouldn't benefit).
>
> I tried to leave the dict type unchanged (it's not true in the PoC
> right now, but I'm working on fixing this). Currently, the versionned
> dictionary type (fat.verdict) is only used when Python starts in FAT
> mode and only in a few places: module, class and instance
> dictionaries. I'm not sure that it's worth to add the versionning
> feature to the base dict type. It uses a little bit more memory, and
> IMHO the dict type is already heavy in term of memory usage (compared
> to a compact C array).
>

I'm not sure if Antoine was truly proposing adding version number support
to all dicts, but I know I wasn't. I would propose creating a dict subclass
and use it in exactly the places you outlined, Victor: modules, classes,
and instance dictionaries (and I would honestly be fine with doing it
piecemeal in that order until we have experience as to how truly dynamic
people treat instances compared to modules and classes). I would start with
a global counter for every time __setitem__() is successful and see where
that gets us (both in terms of performance and optimizations).


>
> I'm trying to keep all FAT features optional to be able to easily
> compare performances, but also ensure that the FAT mode can be added
> to CPython without hurting performances (CPU and memory) when it's
> disabled.
>
> > However, I would also like to challenge the idea that an accelerator
> > has to be 100% compatible. If your API is opt-in (which Numba's is),
> > then you can happily relax some constraints (provided it is documented)
> > in exchange for much faster code. Our experience with Numba is that
> > nobody cares if you can't monkeypatch the len() builtin, for example.
>
> Sure, it will be possible to skip some guards if you know that
> builtins are never monkey-patched, that a module variable is constant,
> that a class will never be modified, etc. This will reduce even more
> the cost of guards (especially when all guards are skipped :-)).
>
> But I prefer to start with something that doesn't change the Python
> semantic at all, and then provide opt-in options to give more freedom
> to the optimizer.
>

I agree that not all optimizations/accelerators need to be 100% compatible.
Numba is in a certain kind of position, though, in that it is applied at
the function/method level while what Victor and I are working on applies to
all code. So it's just differing levels of granularity which influences how
much one feels like they can break the rules in the name of performance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151022/ff8751e9/attachment.html>

From antoine at python.org  Thu Oct 22 19:33:26 2015
From: antoine at python.org (Antoine Pitrou)
Date: Thu, 22 Oct 2015 17:33:26 +0000 (UTC)
Subject: [Python-ideas] Add specialized bytecode with guards to functions
References: <CAMpsgwZgWXGtm4FfxKOp8T7ViVLLvyRknPJAA25dYESavETMCw@mail.gmail.com>
 <CAP7+vJL9wXRvW=9nGx0WXCtBNtfzLONjvV91mDdqcUb-zmX0eA@mail.gmail.com>
 <CAMpsgwY0nxgUTdT04NTYJ4=TfrdgF8r0rr4pbcPGuZpzUQDrnA@mail.gmail.com>
 <CAP7+vJ+mVaRc7tvh0ckjHOmAU7v6r2Vr_7ne+cSDpupT-K_NxQ@mail.gmail.com>
 <CAP1=2W7KaNnL2epv5vMMfW0HiSZ1ywEKFMOivamLU3sj-yC5pw@mail.gmail.com>
 <CACac1F8AJMo6FVFpx0zzjcf-xR2FSh00n1O70aUxY2mvhD-nag@mail.gmail.com>
 <CAP1=2W74-nkHvVjyE6sf_cJqpEMhjj3zmKxm-ViP=B-G7D67Eg@mail.gmail.com>
 <loom.20151021T225918-737@post.gmane.org>
 <CAMpsgwZdPFKt2Qzg-eox5e7j1Oj-ta5teRBGV-f7OqZXNCLLkA@mail.gmail.com>
 <CAP1=2W5JqBJ1Uixi_8=L3F18njx+eC2QRQyKo08s6eLApKJc-A@mail.gmail.com>
Message-ID: <loom.20151022T192642-644@post.gmane.org>

Brett Cannon <brett at ...> writes:
> 
> I'm not sure if Antoine was truly proposing adding version number
> support to all dicts, but I know I wasn't.

I am.  An empty dict is currently 288 bytes on 64-bit Linux.
Even an empty dict with key sharing (e.g. an instance dict, which
would be versioned anyway in your proposal) is 96 bytes long.
Adding a size_t-sized field would not make a significant difference
in memory consumption, and is probably much simpler than
ensuring you use the right dict subtype at each place,
add some type-checking to all PyDict concrete APIs, etc.

And incrementing the version number is cheaper if you don't have
a conditional branch based on whether the dict is a versioned or
non-versioned dict.

Regards

Antoine.



From srkunze at mail.de  Fri Oct 23 09:47:23 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 23 Oct 2015 15:47:23 +0200
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562969B4.5080406@gmail.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
Message-ID: <562A3A6B.1060207@mail.de>

I think you shouldn't push it, Yury.

Most people here seem not to like it that much for many reasons.

Best,
Sven

On 23.10.2015 00:56, Yury Selivanov wrote:
> Hi Barry,
>
> On 2015-10-22 5:17 PM, Barry Warsaw wrote:
>>> I think that it might be a good idea to state the following in PEP 8:
>>> >
>>> >- use r'...' strings for raw strings that describe regular 
>>> expressions;
>>> >these strings might be highlighted specially in some editors.
>>> >
>>> >- use R'...' strings for raw strings; editors*should not* highlight
>>> >any escaped characters in them.
>> Aside from other comments in this thread, I think it's generally a 
>> bad idea to
>> codify semantics where there are none defined by the language. So r'' 
>> and R''
>> are equivalent.  It would be like assigning different semantics or 
>> syntax
>> highlighting for 'foo' and "foo".
>
> The thing is that *a lot* of Python users have already made
> the choice -- many tools (including GitHub) do highlight
> r'' strings as regexps, and people love this feature, and
> complain when it's not implemented in their tool of choice.
> This is an extremely useful feature, since any regexp longer
> than 20 characters is unreadable at a glance.
>
> On the other hand, *a lot* of users are annoyed, when their
> raw strings are highlighted as regexps (why \n is highlighted
> as escaped?).  Often that results not in just wrongly
> highlighted raw strings, but in broken highlighting of
> the whole source file [1].
>
> In principle, there is no reason why *both* of these groups
> of users can't use one tool and be happy.  I propose to
> establish a convention in PEP 8, explaining that, while both
> literals are semantically equivalent,
>
> - r'..' strings *should* be used for regexps,
>
> - R'..' strings *should* be used for unstyled raw strings,
>
> and tools *should* treat them as such.
>
> All of this is merely about codifying the current status quo.
>
> Thanks,
> Yury
>
>
> [1] https://gist.github.com/1st1/d3da5d69a9b6c088c26f
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From eric at trueblade.com  Sun Oct 25 17:06:16 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Sun, 25 Oct 2015 17:06:16 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562969B4.5080406@gmail.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
Message-ID: <562D4448.6090208@trueblade.com>

On 10/22/2015 06:56 PM, Yury Selivanov wrote:

> On the other hand, *a lot* of users are annoyed, when their
> raw strings are highlighted as regexps (why \n is highlighted
> as escaped?).  Often that results not in just wrongly
> highlighted raw strings, but in broken highlighting of
> the whole source file [1].
...

> [1] https://gist.github.com/1st1/d3da5d69a9b6c088c26f

I don't see how this matters to the subject at hand. No matter what you
think an r or R string contains, not detecting the end of the string is
a failure of the syntax highlighter. It's failing to find the end of the
triple-quoted r"""[""" token in this case.

Eric.


From srkunze at mail.de  Sat Oct 24 04:39:59 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Sat, 24 Oct 2015 10:39:59 +0200
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
Message-ID: <562B43DF.9030809@mail.de>

On 23.10.2015 18:09, Nick Coghlan wrote:
> It's a big deal due to opportunity cost - any time that is spent on
> making a second spelling of "elif" possible is time not spent working
> on something that directly improves developer productivity (like
> format strings) or application correctness (like gradual typing
> support).

1) I may re-iterate. Such thinking is dangerous. It could mean 
standstill. It prevents thinking and progress. That is python-ideas; not 
python-controlling.

2) Is Python not an open-source project? Could not anybody contribute? 
To me that means, the sky is the limit.

3) Well, that the features you mentioned really do what they promise can 
be only be proved if they are implemented and used. So, we will see how 
they fare. Same, btw., would be for "else if". All of your and mine 
great thoughts are just that: thoughts, assumptions, arguments. Not facts.

> Time spent on frivolous changes also (justifiably) annoys
> users of the language given the large numbers of known problems we've
> yet to figure out how to find the time and resources to resolve.

Cf. above.

Okay, enough of resource-thinking. Back to topic.

>  From a learning perspective, remembering that "elif" is an
> abbreviation of "else if" in Python isn't any more complicated than
> remembering that "don't" is an abbreviation of "do not" in English.
> It's certainly easier to remember than what "def" or "class" means.

Let's remember that.

>  From a readability perspective, inserting an "se " in the middle of
> every elif clause doesn't really improve things:
>
>      if x == 1:
>          ...
>      elif x == 2:
>          ...
>      elif x == 3:
>          ...
>      else:
>          ...
>
> vs:
>
>      if x == 1:
>          ...
>      else if x == 2:
>          ...
>      else if x == 3:
>          ...
>      else:
>          ...
>
> It can actually be argued that it's *harder* to read, since the
> typical English phrasing for this kind of formulation is more along
> the lines of "X if case A, else Y if case B, else Z", which is the way
> conditional expressions are written.

I thought "elif" is an abbreviation of "else if" as is "don't" to "do 
not". So, why are you arguing with differing semantics when it comes to 
readability?

I for one think it improves things (as does several other people on the 
list), that is why it is on the list. ;-)
Why you may ask? Because it just reads like an English sentence: "If 
that condition, then (==colon) do this, else if another condition, then 
(==colon) do that, etc. etc." So, we got "if", ":", and "else", this is 
just all we need to learn.


Something, I would be interested in is: what was the reason for 
introducing the non-English keyword "elif"? Parsing difficulties?

Best,
Sven

From abarnert at yahoo.com  Sun Oct 25 17:49:57 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 25 Oct 2015 14:49:57 -0700
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <562B43DF.9030809@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
Message-ID: <BB391F0E-8DC2-4540-B853-47EDE912A48F@yahoo.com>

On Oct 24, 2015, at 01:39, Sven R. Kunze <srkunze at mail.de> wrote:
> 
> I for one think it improves things (as does several other people on the list), that is why it is on the list. ;-)
> Why you may ask? Because it just reads like an English sentence: "If that condition, then (==colon) do this, else if another condition, then (==colon) do that, etc. etc."

Nobody has ever spoken a sentence in that form. Maybe with "otherwise", even more likely with no connective at all (which you'd spell with a semicolon), but never with "else". That's what's wrong with this whole argument: it's an attempt to make Python "more like English" in a way that's not actually like English.

> Something, I would be interested in is: what was the reason for introducing the non-English keyword "elif"? Parsing difficulties?

Think about the way "else if" works in C and its descendants and influencees: it's just an "else" clause whose body happens to be an "if" statement. By convention, you don't wrap the "if" statement in braces or indent it an extra level, but there's no syntactic reason not to do so. (And formatter tools need a special case to make them not do so.) In Python, indentation is not optional or conventional, it's syntactic. So the if would have to be indented under the else.

Either that, or it would have to work differently from the way it works in all those other languages. Of course that's possible, but that demolishes the other argument for this change: to make Python more like other languages, we'd make it misleadingly appear to be like all those other languages, when it actually works differently. Which would be a great way to confuse people coming to Python from Java, or alternating Python. C, and JS code in their work, etc.

From ncoghlan at gmail.com  Fri Oct 23 12:09:26 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Oct 2015 18:09:26 +0200
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <56290D47.9030800@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de> <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
Message-ID: <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>

On 22 October 2015 at 18:22, Sven R. Kunze <srkunze at mail.de> wrote:
> On 22.10.2015 14:19, Geoffrey Spear wrote:
>>
>> Obviously this is sometimes a cause of confusion to people new to the
>> language, when people think "oh, Python is just like English, so I should
>> use 'is' instead of '=='" or "why doesn't `if foo or bar == 'baz'` do what I
>> mean?", but for the most part having easy-to-read actual English words for
>> keywords and functions with names that actually say what they do instead of
>> making some obscure reference to a badly-named C library function written at
>> a time when limiting identifiers to 6 letters seemed like a good idea to
>> save precious disk space (or room on a punchcard), in my opinion, has
>> greatly contributed to Python's reputation as a readable language.
>
>
> So, why stop there? ;)
>
>> "elif" is a wart, but again, one I think we're stuck with.
>
>
> Dangerous thinking.
>
> I don't regard changing it to be a big deal given 10 years or so of parallel
> usage.

It's a big deal due to opportunity cost - any time that is spent on
making a second spelling of "elif" possible is time not spent working
on something that directly improves developer productivity (like
format strings) or application correctness (like gradual typing
support). Time spent on frivolous changes also (justifiably) annoys
users of the language given the large numbers of known problems we've
yet to figure out how to find the time and resources to resolve.

>From a learning perspective, remembering that "elif" is an
abbreviation of "else if" in Python isn't any more complicated than
remembering that "don't" is an abbreviation of "do not" in English.
It's certainly easier to remember than what "def" or "class" means.

>From a readability perspective, inserting an "se " in the middle of
every elif clause doesn't really improve things:

    if x == 1:
        ...
    elif x == 2:
        ...
    elif x == 3:
        ...
    else:
        ...

vs:

    if x == 1:
        ...
    else if x == 2:
        ...
    else if x == 3:
        ...
    else:
        ...

It can actually be argued that it's *harder* to read, since the
typical English phrasing for this kind of formulation is more along
the lines of "X if case A, else Y if case B, else Z", which is the way
conditional expressions are written.

By contrast, the statement form has an implied "then" after each
condition: "if case A, then X, else if case B, then Y, else Z".
Compared to dropping "then" entirely, abbreviating "else if" to "elif"
is a fairly minor change.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rymg19 at gmail.com  Sun Oct 25 19:03:12 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Sun, 25 Oct 2015 18:03:12 -0500
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <562B43DF.9030809@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
Message-ID: <CAO41-mN+NNLfGA_046zgCGoEzG-RGigkDOxC2rk+1b0GEZRTPQ@mail.gmail.com>

On Sat, Oct 24, 2015 at 3:39 AM, Sven R. Kunze <srkunze at mail.de> wrote:

> On 23.10.2015 18:09, Nick Coghlan wrote:
>
>> It's a big deal due to opportunity cost - any time that is spent on
>> making a second spelling of "elif" possible is time not spent working
>> on something that directly improves developer productivity (like
>> format strings) or application correctness (like gradual typing
>> support).
>>
>
> 1) I may re-iterate. Such thinking is dangerous. It could mean standstill.
> It prevents thinking and progress. That is python-ideas; not
> python-controlling.
>
> 2) Is Python not an open-source project? Could not anybody contribute? To
> me that means, the sky is the limit.
>
> 3) Well, that the features you mentioned really do what they promise can
> be only be proved if they are implemented and used. So, we will see how
> they fare. Same, btw., would be for "else if". All of your and mine great
> thoughts are just that: thoughts, assumptions, arguments. Not facts.
>
> Time spent on frivolous changes also (justifiably) annoys
>> users of the language given the large numbers of known problems we've
>> yet to figure out how to find the time and resources to resolve.
>>
>
> Cf. above.
>
> Okay, enough of resource-thinking. Back to topic.
>
>  From a learning perspective, remembering that "elif" is an
>> abbreviation of "else if" in Python isn't any more complicated than
>> remembering that "don't" is an abbreviation of "do not" in English.
>> It's certainly easier to remember than what "def" or "class" means.
>>
>
> Let's remember that.
>
>  From a readability perspective, inserting an "se " in the middle of
>> every elif clause doesn't really improve things:
>>
>>      if x == 1:
>>          ...
>>      elif x == 2:
>>          ...
>>      elif x == 3:
>>          ...
>>      else:
>>          ...
>>
>> vs:
>>
>>      if x == 1:
>>          ...
>>      else if x == 2:
>>          ...
>>      else if x == 3:
>>          ...
>>      else:
>>          ...
>>
>> It can actually be argued that it's *harder* to read, since the
>> typical English phrasing for this kind of formulation is more along
>> the lines of "X if case A, else Y if case B, else Z", which is the way
>> conditional expressions are written.
>>
>
> I thought "elif" is an abbreviation of "else if" as is "don't" to "do
> not". So, why are you arguing with differing semantics when it comes to
> readability?
>
> I for one think it improves things (as does several other people on the
> list), that is why it is on the list. ;-)
> Why you may ask? Because it just reads like an English sentence: "If that
> condition, then (==colon) do this, else if another condition, then
> (==colon) do that, etc. etc." So, we got "if", ":", and "else", this is
> just all we need to learn.
>
>
Two things:

1. That doesn't read like an English sentence. It sounds like an English
sentence spoken by someone who's been through a 48-hour C hackathon.
2. Well, elif is used almost everywhere, so 99% of the people who learn
Python will have to learn 'elif' the day after 'else if'.


>
> Something, I would be interested in is: what was the reason for
> introducing the non-English keyword "elif"? Parsing difficulties?
>
>
> Best,
> Sven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something?s wrong.
http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151025/caced7b2/attachment-0001.html>

From ben+python at benfinney.id.au  Sun Oct 25 18:16:34 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 26 Oct 2015 09:16:34 +1100
Subject: [Python-ideas] =?utf-8?q?Limits_to_ideas_for_proposal_in_?=
 =?utf-8?b?4oCYcHl0aG9uLWlkZWFz4oCZICh3YXM6ICJlbHNlIGlmIiBhcyBlcXVpdmFs?=
 =?utf-8?b?ZW50IGZvciAiZWxpZiIp?=
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de> <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
Message-ID: <858u6qoapp.fsf_-_@benfinney.id.au>

"Sven R. Kunze" <srkunze at mail.de> writes:

> 2) Is Python not an open-source project? Could not anybody contribute?
> To me that means, the sky is the limit.

Easy for you to say, but that's not the limit. The limit in *this* forum
is the finite attention of the Python core developers.

If you want to discuss sky-is-the-limit ideas, use ?python-list? which
is a forum for all topics Python-related.

-- 
 \       ?Repetition leads to boredom, boredom to horrifying mistakes, |
  `\       horrifying mistakes to God-I-wish-I-was-still-bored, and it |
_o__)              goes downhill from there.? ?Will Larson, 2008-11-04 |
Ben Finney


From barry at python.org  Fri Oct 23 14:18:57 2015
From: barry at python.org (Barry Warsaw)
Date: Fri, 23 Oct 2015 14:18:57 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org>
 <562969B4.5080406@gmail.com>
Message-ID: <20151023141857.0aec87e9@anarchist.wooz.org>

On Oct 22, 2015, at 06:56 PM, Yury Selivanov wrote:

>In principle, there is no reason why *both* of these groups
>of users can't use one tool and be happy.  I propose to
>establish a convention in PEP 8, explaining that, while both
>literals are semantically equivalent,
>
>- r'..' strings *should* be used for regexps,
>
>- R'..' strings *should* be used for unstyled raw strings,
>
>and tools *should* treat them as such.
>
>All of this is merely about codifying the current status quo.

What about all the regexps that are written without raw strings?  Won't PEP 8
codification shame everyone into turn them into raw strings when they really
don't need to?

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151023/75af1dc9/attachment.sig>

From rosuav at gmail.com  Sun Oct 25 21:02:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 26 Oct 2015 12:02:22 +1100
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <562B43DF.9030809@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de> <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
Message-ID: <CAPTjJmrzAn__nTT6RLuHFRtyjTWNaQ6QjgOPVGgP7ajQvhuKNQ@mail.gmail.com>

On Sat, Oct 24, 2015 at 7:39 PM, Sven R. Kunze <srkunze at mail.de> wrote:
> 1) I may re-iterate. Such thinking is dangerous. It could mean standstill.
> It prevents thinking and progress. That is python-ideas; not
> python-controlling.

The point of discussing ideas is to sort out the good ones from the
bad ones. One of the most effective ways to do this is to have someone
advocate an idea, and someone else criticize it, with the resulting
discussion being (hopefully!) a fruitful analysis of the idea's
qualities. So while you're correct that this is called "python-ideas"
for a reason, Nick is also absolutely correct in criticizing the idea.
A good idea will be able to stand up to critics.

> 2) Is Python not an open-source project? Could not anybody contribute? To me
> that means, the sky is the limit.

"Open source" and "free software" and so on do not mean that anyone
can make changes to the core. It does mean, however, that you're
legally, morally, and ethically permitted to make your own version of
CPython that includes the "else if" notation, play around with it, and
demonstrate your idea in action. This is actually a great way to
support an idea; you can find out exactly how much code would have to
change to make this work, and start playing around with it. And then
if the idea does end up being accepted, your patch can be incorporated
into the main body of code.

> 3) Well, that the features you mentioned really do what they promise can be
> only be proved if they are implemented and used. So, we will see how they
> fare. Same, btw., would be for "else if". All of your and mine great
> thoughts are just that: thoughts, assumptions, arguments. Not facts.
>

Yep. And that's why it helps to spin up a patch. Unfortunately this is
a change to the language grammar, which is not the easiest thing to
tweak (if you were proposing a change to a pure Python module in the
standard library, you could play with it much more easily); but if you
write the patch yourself, you are directly answering (part of) Nick's
objection about opportunity cost.

Personally, I'm not in favour of adding another way of spelling elif.
If every other language in the world spelled it "else if" and Python
alone used "elif", then I'd support adding the alias, for the
convenience of people working with multiple languages; but it's not
that consistent (some languages use "elsif", "elseif", and various
other forms). The readability argument is weak, and I don't see any
reason to create two distinct ways of spelling the exact same thing.
But hey! Play with the patch, have some fun with it.

ChrisA

From vgr255 at live.ca  Sun Oct 25 21:28:16 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Sun, 25 Oct 2015 21:28:16 -0400
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CAPTjJmrzAn__nTT6RLuHFRtyjTWNaQ6QjgOPVGgP7ajQvhuKNQ@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>,
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>, <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>,
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>,
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>,
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>,
 <56290D47.9030800@mail.de>,
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>,
 <562B43DF.9030809@mail.de>,
 <CAPTjJmrzAn__nTT6RLuHFRtyjTWNaQ6QjgOPVGgP7ajQvhuKNQ@mail.gmail.com>
Message-ID: <BLU172-W12F2A7E7B6F7797108852E91230@phx.gbl>

Here is yet another argument against this change -- indentation. Indentation is, as we all well know, one of the core aspects of Python's syntax. In every project I have worked on, and in PEP 8 ( https://www.python.org/dev/peps/pep-0008/#indentation ), 4-space indents are used. 'elif' is exactly 4 characters long, which means it lines up pretty well with other similar keywords such as 'else' or 'for' (with the space that follows it). In that regard, 'else if' would actually make code *harder* to read, not easier! In the same sense that 'def' (followed by a space) makes it very easy to spot the functions' names (if for some reason you don't have syntax highlighting ... ), 'elif' makes it easy to spot the beginning of another condition statement within a block.
Cheers,-Emanuel Barry 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151025/cd7e0183/attachment.html>

From guido at python.org  Mon Oct 26 00:07:51 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 25 Oct 2015 21:07:51 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <20151023141857.0aec87e9@anarchist.wooz.org>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org>
 <562969B4.5080406@gmail.com> <20151023141857.0aec87e9@anarchist.wooz.org>
Message-ID: <CAP7+vJ+kZZvVhSKpY6E6h=MEzpf34dLh4JWLZbyVaqOd=P15pA@mail.gmail.com>

It's not going to be codified in PEP 8.

On Fri, Oct 23, 2015 at 11:18 AM, Barry Warsaw <barry at python.org> wrote:

> On Oct 22, 2015, at 06:56 PM, Yury Selivanov wrote:
>
> >In principle, there is no reason why *both* of these groups
> >of users can't use one tool and be happy.  I propose to
> >establish a convention in PEP 8, explaining that, while both
> >literals are semantically equivalent,
> >
> >- r'..' strings *should* be used for regexps,
> >
> >- R'..' strings *should* be used for unstyled raw strings,
> >
> >and tools *should* treat them as such.
> >
> >All of this is merely about codifying the current status quo.
>
> What about all the regexps that are written without raw strings?  Won't
> PEP 8
> codification shame everyone into turn them into raw strings when they
> really
> don't need to?
>
> Cheers,
> -Barry
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151025/56b73231/attachment-0001.html>

From breamoreboy at yahoo.co.uk  Mon Oct 26 00:32:09 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 26 Oct 2015 04:32:09 +0000
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <562B43DF.9030809@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
Message-ID: <n0kacb$sr2$1@ger.gmane.org>

On 24/10/2015 09:39, Sven R. Kunze wrote:
> On 23.10.2015 18:09, Nick Coghlan wrote:
>> It's a big deal due to opportunity cost - any time that is spent on
>> making a second spelling of "elif" possible is time not spent working
>> on something that directly improves developer productivity (like
>> format strings) or application correctness (like gradual typing
>> support).
>
> 1) I may re-iterate. Such thinking is dangerous. It could mean
> standstill. It prevents thinking and progress. That is python-ideas; not
> python-controlling.
>
> 2) Is Python not an open-source project? Could not anybody contribute?
> To me that means, the sky is the limit.
>
> 3) Well, that the features you mentioned really do what they promise can
> be only be proved if they are implemented and used. So, we will see how
> they fare. Same, btw., would be for "else if". All of your and mine
> great thoughts are just that: thoughts, assumptions, arguments. Not facts.
>

I have a very strong feeling that you haven't got the faintest idea how 
the Python development process works, from the time that an idea gets 
put forward here, until the first full release with that shiny new 
feature that everybody has been wanting for since forever.

To me the most important thing that needs sorting is the core workflow. 
  Sure other things need doing, but to me "else if" doesn't even get on 
the bottom of the infinitely long list of Python jobs that need doing.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From ncoghlan at gmail.com  Fri Oct 23 12:14:37 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 23 Oct 2015 18:14:37 +0200
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <CAPJVwB=e0jPFkFOFpFztqoPP6GSJJwE18pjgdfsEMWo9NVAp8A@mail.gmail.com>
References: <562848C2.4090509@gmail.com> <858u6vpqpt.fsf@benfinney.id.au>
 <CAPJVwB=e0jPFkFOFpFztqoPP6GSJJwE18pjgdfsEMWo9NVAp8A@mail.gmail.com>
Message-ID: <CADiSq7dUvYihUSFznveqBm-Zf7bduOgOjVzxo9sgGD58+G0vxg@mail.gmail.com>

On 22 October 2015 at 05:31, Nathaniel Smith <njs at pobox.com> wrote:
> On Wed, Oct 21, 2015 at 7:44 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> Yury Selivanov <yselivanov.ml at gmail.com> writes:
>>> In the process, we had to make a decision on how to highlight raw
>>> string literals -- r''. Many existing highlighters assume that all raw
>>> strings are regexps, and highlight them as such, i.e. '\s' and '\n'
>>> will be highlighted.
>>
>> That is evidently a simple mistake. Merely knowing that a token is a raw
>> string does not justify the assumption that the string is a regular
>> expression, or a filesystem entry name, or a line in a network protocol,
>> or anything except plain text.
>
> This isn't necessarily true, just as a matter of like... epistemology.
> For example, if hypothetically it turned out that 99% of raw strings
> are in fact regular expressions, then knowing something is a raw
> string would give you quite a bit of evidence that it's a regular
> expression -- quite possibly enough to justify treating it as such for
> something like code highlighting.
>
> I haven't actually gathered any data to find out how strong the
> association between raw strings and regexen is, but it'd be pretty
> easy for someone to do. (Parse a large corpus of python code to
> extract all raw strings, randomly subsample 100 of them, review
> manually to decide if each is a regex.)

I'd expect Windows filesystem paths to win handily if we could scan
all the Python code in the world, but they wouldn't show up in a scan
of POSIX specific open source code.

Cheers,
Nick

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ian.g.kelly at gmail.com  Mon Oct 26 01:58:09 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 25 Oct 2015 23:58:09 -0600
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <BLU172-W12F2A7E7B6F7797108852E91230@phx.gbl>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com> <5627D667.8020301@mail.de>
 <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
 <CAPTjJmrzAn__nTT6RLuHFRtyjTWNaQ6QjgOPVGgP7ajQvhuKNQ@mail.gmail.com>
 <BLU172-W12F2A7E7B6F7797108852E91230@phx.gbl>
Message-ID: <CALwzidkreOAsPvmJpcGiQ45ivfxMGMhfuEr+1-yK=vWNWfxZJA@mail.gmail.com>

On Sun, Oct 25, 2015 at 7:28 PM, Emanuel Barry <vgr255 at live.ca> wrote:
> Here is yet another argument against this change -- indentation. Indentation
> is, as we all well know, one of the core aspects of Python's syntax. In
> every project I have worked on, and in PEP 8 (
> https://www.python.org/dev/peps/pep-0008/#indentation ), 4-space indents are
> used. 'elif' is exactly 4 characters long, which means it lines up pretty
> well with other similar keywords such as 'else' or 'for' (with the space
> that follows it). In that regard, 'else if' would actually make code
> *harder* to read, not easier! In the same sense that 'def' (followed by a
> space) makes it very easy to spot the functions' names (if for some reason
> you don't have syntax highlighting ... ), 'elif' makes it easy to spot the
> beginning of another condition statement within a block.

I'm not following. Can you provide an example of how this hurts
indentation? It seems a stretch to suggest that "elif" lines up with
"for" because you're including the space that follows "for", but
you're excluding the space that follows "elif".

I note that "else if" has the same length as "finally", if that matters.

From tritium-list at sdamon.com  Mon Oct 26 02:02:00 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 02:02:00 -0400
Subject: [Python-ideas] Allowing def to assign to anything
Message-ID: <562DC1D8.8060003@sdamon.com>

In my code, I write a lot of dispatch dictionaries (for lack of a switch 
statement, but I will not hold my breath for that).  In trying to make 
writing these dictionaries less annoying, I tend to use many lambdas.  I 
can let you guess at what problems that has resulted in.  Of course, the 
preferred way to write such dictionaries is by using a regular function, 
and adding that function to a dictionary.  This isn't exactly a problem 
- it works, and works well, but it is annoying to write, and leaves 
artifacts of those functions in module scope.  I propose a little bit of 
sugar to make this a little less annoying.

If `def` is allowed to assign to anything (anything that is legal at the 
left hand side of an = in that scope), annoying artifacts go away.  The 
syntax I propose should be backwards compatible.

```
dispatch = {}

def dispatch['foo'](bar):
     return bar * bar
```

Does this make anything possible that is impossible now?  No.  But it 
does make the intent of the module author clear - the function is only 
ever intended to live inside that dict, or list, or other structure.  
This, to me, is less annoying to write, and is more readable.  This 
obviously could be used outside of creating dispatch dictionaries, but 
that is the use case I would benefit from.

From tritium-list at sdamon.com  Mon Oct 26 02:19:43 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 02:19:43 -0400
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>
References: <562DC1D8.8060003@sdamon.com>
 <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>
Message-ID: <562DC5FF.1090508@sdamon.com>

This too leaves artifacts in the module, which is one of the annoyances 
I am trying to eliminate.  Though I do admit that it makes the intent of 
the author clear.  It still feels less pythonic to me than allowing a 
statement that assigns to just assign to anything assignable.

On 10/26/2015 02:15, Bruce Leban wrote:
> Decorators easily support this:
>
> dispatch_table = {}
> def dispatch(arg):
>     def inner(func):
> dispatch_table[arg] = func
>         return func
>     return inner
>
> @dispatch('foo')
> def dispatch_foo(bar):
>     pass

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/8750e73a/attachment-0001.html>

From rosuav at gmail.com  Mon Oct 26 02:20:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 26 Oct 2015 17:20:37 +1100
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC1D8.8060003@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
Message-ID: <CAPTjJmqXgGHW9PKqHcd_rgCLoFtkdju0uzxvdQCv_DAOoRMMJA@mail.gmail.com>

On Mon, Oct 26, 2015 at 5:02 PM, Alexander Walters
<tritium-list at sdamon.com> wrote:
> If `def` is allowed to assign to anything (anything that is legal at the
> left hand side of an = in that scope), annoying artifacts go away.  The
> syntax I propose should be backwards compatible.
>
> ```
> dispatch = {}
>
> def dispatch['foo'](bar):
>     return bar * bar
> ```
>
> Does this make anything possible that is impossible now?  No.  But it does
> make the intent of the module author clear - the function is only ever
> intended to live inside that dict, or list, or other structure.  This, to
> me, is less annoying to write, and is more readable.  This obviously could
> be used outside of creating dispatch dictionaries, but that is the use case
> I would benefit from.

I agree; the idea has been raised a few times, and I think it'd be
helpful. It's probably not necessary to allow the _entire_ scope of
"anything legal on the left of =", as that's pretty broad; even if the
only form allowed were obj[key], it'd be useful.

But for building a dispatch dictionary, you could simply decorate your
functions with a capturer:

dispatch = {}

def cmd(func):
    dispatch[func.__name__] = func
    return func

@cmd
def foo(bar):
    return bar * bar

You can even merge the decorator and the dict itself:

class DispatchDict(dict):
    def __call__(self, func):
        self[func.__name__] = func
        return func

dispatch = DispatchDict()

@dispatch
def foo(bar):
    return bar * bar

This does require that your dict keys be legal identifiers (you can't
do "def dispatch['!'](x):" as "@cmd def !(x)"), but for a lot of
common cases, this does work. I've used this style for building
argparse UIs and such, and it's a lot easier than most other options
I've played with.

ChrisA

From abarnert at yahoo.com  Mon Oct 26 02:19:43 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Sun, 25 Oct 2015 23:19:43 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC1D8.8060003@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
Message-ID: <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>

On Oct 25, 2015, at 23:02, Alexander Walters <tritium-list at sdamon.com> wrote:
> 
> In my code, I write a lot of dispatch dictionaries (for lack of a switch statement, but I will not hold my breath for that).  In trying to make writing these dictionaries less annoying, I tend to use many lambdas.  I can let you guess at what problems that has resulted in.  Of course, the preferred way to write such dictionaries is by using a regular function, and adding that function to a dictionary.  This isn't exactly a problem - it works, and works well, but it is annoying to write, and leaves artifacts of those functions in module scope.  I propose a little bit of sugar to make this a little less annoying.
> 
> If `def` is allowed to assign to anything (anything that is legal at the left hand side of an = in that scope), annoying artifacts go away.  The syntax I propose should be backwards compatible.

Seems interesting.

What's the name of the defined function? For an attribution like "spam.eggs" you'd probably want it to be "eggs", and I guess for "spam['eggs']" as well, but what about "spam['two words']" or "spam[2]"?

I assume the qualname is just the name.

Also, would this go through the descriptor mechanism if you def an attribution?

From bruce at leban.us  Mon Oct 26 02:15:44 2015
From: bruce at leban.us (Bruce Leban)
Date: Sun, 25 Oct 2015 23:15:44 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC1D8.8060003@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
Message-ID: <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>

Decorators easily support this:

dispatch_table = {}
def dispatch(arg):
    def inner(func):
        dispatch_table[arg] = func
        return func
    return inner

@dispatch('foo')
def dispatch_foo(bar):
    pass


--- Bruce
Check out my new puzzle book: http://J.mp/ingToConclusions
Get it free here: http://J.mp/ingToConclusionsFree (available on iOS)



On Sun, Oct 25, 2015 at 11:02 PM, Alexander Walters <tritium-list at sdamon.com
> wrote:

> In my code, I write a lot of dispatch dictionaries (for lack of a switch
> statement, but I will not hold my breath for that).  In trying to make
> writing these dictionaries less annoying, I tend to use many lambdas.  I
> can let you guess at what problems that has resulted in.  Of course, the
> preferred way to write such dictionaries is by using a regular function,
> and adding that function to a dictionary.  This isn't exactly a problem -
> it works, and works well, but it is annoying to write, and leaves artifacts
> of those functions in module scope.  I propose a little bit of sugar to
> make this a little less annoying.
>
> If `def` is allowed to assign to anything (anything that is legal at the
> left hand side of an = in that scope), annoying artifacts go away.  The
> syntax I propose should be backwards compatible.
>
> ```
> dispatch = {}
>
> def dispatch['foo'](bar):
>     return bar * bar
> ```
>
> Does this make anything possible that is impossible now?  No.  But it does
> make the intent of the module author clear - the function is only ever
> intended to live inside that dict, or list, or other structure.  This, to
> me, is less annoying to write, and is more readable.  This obviously could
> be used outside of creating dispatch dictionaries, but that is the use case
> I would benefit from.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151025/6d76845a/attachment.html>

From ned at nedbatchelder.com  Fri Oct 23 14:40:29 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Fri, 23 Oct 2015 14:40:29 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562969B4.5080406@gmail.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
Message-ID: <562A7F1D.90108@nedbatchelder.com>

On 10/22/15 6:56 PM, Yury Selivanov wrote:
> In principle, there is no reason why *both* of these groups
> of users can't use one tool and be happy.  I propose to
> establish a convention in PEP 8, explaining that, while both
> literals are semantically equivalent,
>
> - r'..' strings *should* be used for regexps,
>
> - R'..' strings *should* be used for unstyled raw strings,
>
> and tools *should* treat them as such.
>
> All of this is merely about codifying the current status quo. 
But you are not codifying the status quo.  The distinction you are 
proposing is one that you have invented.  I have never used R"" strings.

I think the best solution to the problem is to improve the highlighters, 
and luckily you have written one!  To me, it is clear which of these 
strings is the regex:

     r"\d+"
     r"\dir"

If the highlighters tried some heuristics, they could do a better job 
"being helpful" by making better guesses about the meaning of programs.  
I don't mind when highlighters make wrong guesses, as long as they don't 
ruin the entire rest of the file.  But better guesses will be better. :)

--Ned.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151023/89a012cf/attachment-0001.html>

From tritium-list at sdamon.com  Mon Oct 26 02:23:54 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 02:23:54 -0400
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
Message-ID: <562DC6FA.3060309@sdamon.com>

I imagine in the case of assigning to a class or instance, the name 
would be the same (and binding to self would act the same) as if it were 
assigned in the traditional way.  I do not propose that assigning to 
classes in this way be considered a good idea.

as for when assigned to a data structure, my admittedly naive idea would 
be to set the name to '__none__' or some such.  I am open to other ideas 
though.

On 10/26/2015 02:19, Andrew Barnert wrote:
> Seems interesting.
>
> What's the name of the defined function? For an attribution like "spam.eggs" you'd probably want it to be "eggs", and I guess for "spam['eggs']" as well, but what about "spam['two words']" or "spam[2]"?
>
> I assume the qualname is just the name.
>
> Also, would this go through the descriptor mechanism if you def an attribution?


From tritium-list at sdamon.com  Mon Oct 26 02:30:28 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 02:30:28 -0400
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <CAPTjJmqXgGHW9PKqHcd_rgCLoFtkdju0uzxvdQCv_DAOoRMMJA@mail.gmail.com>
References: <562DC1D8.8060003@sdamon.com>
 <CAPTjJmqXgGHW9PKqHcd_rgCLoFtkdju0uzxvdQCv_DAOoRMMJA@mail.gmail.com>
Message-ID: <562DC884.2080707@sdamon.com>



On 10/26/2015 02:20, Chris Angelico wrote:
> I agree; the idea has been raised a few times, and I think it'd be
> helpful. It's probably not necessary to allow the _entire_ scope of
> "anything legal on the left of =", as that's pretty broad; even if the
> only form allowed were obj[key], it'd be useful.

I agree, that perhaps the scope could be a little wide when put this 
way, but my instinct is that 'allow anything already legal for =' would 
be the path of least frustration when implementing this.  I could be 
woefully wrong.  I do at least wish to assign to object with __setitem__ 
defined.

>
> But for building a dispatch dictionary, you could simply decorate your
> functions with a capturer:
>
> dispatch = {}
>
> def cmd(func):
>      dispatch[func.__name__] = func
>      return func
>
> @cmd
> def foo(bar):
>      return bar * bar
>
> You can even merge the decorator and the dict itself:
>
> class DispatchDict(dict):
>      def __call__(self, func):
>          self[func.__name__] = func
>          return func
>
> dispatch = DispatchDict()
>
> @dispatch
> def foo(bar):
>      return bar * bar
>
> This does require that your dict keys be legal identifiers (you can't
> do "def dispatch['!'](x):" as "@cmd def !(x)"), but for a lot of
> common cases, this does work. I've used this style for building
> argparse UIs and such, and it's a lot easier than most other options
> I've played with.

This does, indeed, make life a bit easier in the here and now (and is 
similar to kitbashed techniques I already use).  I am hoping to make 
that obsolete.

>
> ChrisA
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
- Alex

From bruce at leban.us  Mon Oct 26 02:49:18 2015
From: bruce at leban.us (Bruce Leban)
Date: Sun, 25 Oct 2015 23:49:18 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC5FF.1090508@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
 <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>
 <562DC5FF.1090508@sdamon.com>
Message-ID: <CAGu0AnvzgWd9OEEnhhORaFNkO+GtkNXy+GNnTgDwPHrXYpAbuw@mail.gmail.com>

On Sun, Oct 25, 2015 at 11:19 PM, Alexander Walters <tritium-list at sdamon.com
> wrote:

> This too leaves artifacts in the module, which is one of the annoyances I
> am trying to eliminate.  Though I do admit that it makes the intent of the
> author clear.  It still feels less pythonic to me than allowing a statement
> that assigns to just assign to anything assignable.
>

Well, to some extent features that are already in the language feel more
pythonic than features that aren't. Given that this is already easy to do,
and the decorator solution is more powerful, the bar for modifying the
language is higher.

That aside, I like the fact that I can find these functions in the module.
For one thing, it makes them testable in an obvious way (without going
through the dispatch dictionary). Generally, module_name.function_name
doesn't change during the course of program execution but there's no such
expectation for dispatch_dict[function_name].

Furthermore, consider that

def dispatch['foo'](bar): pass


might look like a simple dict reference but is actually a call to
__setitem__ which means this can have arbitrary side effects. Yes
@decorators can have arbitrary side effects too. How do I find out the
respective side effects? Perhaps:

help(decorator)  # obvious

help(dispatch)  # nope
help(dispatch.__setitem__)  # not obvious to me


--- Bruce
Check out my new puzzle book: http://J.mp/ingToConclusions
Get it free here: http://J.mp/ingToConclusionsFree (available on iOS)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151025/ae0a595b/attachment.html>

From abarnert at yahoo.com  Mon Oct 26 03:05:02 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 00:05:02 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC6FA.3060309@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
Message-ID: <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>

On Oct 25, 2015, at 23:23, Alexander Walters <tritium-list at sdamon.com> wrote:
> 
> I imagine in the case of assigning to a class or instance, the name would be the same (and binding to self would act the same) as if it were assigned in the traditional way.

In what traditional way? When you def a function, its name is the name given in the def statement. If you later assign it to a member of an object, that doesn't change its name. So, that doesn't answer the question.

As for the binding to self, that doesn't happen at assignment time, so it can't happen the same way as at assignment time. The binding happens later, each time the method is looked up. If an object doesn't have an attribute of the looked up name, but its type does, the type's attribute's __get__ method is called with the instance. So, there's nothing to do here at all; functions already have a __get__ method that returns a bound method (and if you assign a function to an instance rather than a type, it doesn't get bound). This is all explained pretty clearly in the descriptor HOWTO.

But descriptors also work for setting, not just getting (which is how @property works), and that's what I was asking about. If I write "spam.eggs = 0", and type(spam) has a member named "eggs", its __set__ method will get called with the instance (spam) and the value (0). So, if I write "def spam.eggs(): pass", does it call the same descriptor method?

>  I do not propose that assigning to classes in this way be considered a good idea.

OK, but unless you're actually proposing to not allow it, you still need to work out what it would do.

> as for when assigned to a data structure, my admittedly naive idea would be to set the name to '__none__' or some such.  I am open to other ideas though.

Functions defined with lambda get the name "<lambda>", so you'd probably want angle brackets here as well. But otherwise that seems reasonable, I guess. 

> 
>> On 10/26/2015 02:19, Andrew Barnert wrote:
>> Seems interesting.
>> 
>> What's the name of the defined function? For an attribution like "spam.eggs" you'd probably want it to be "eggs", and I guess for "spam['eggs']" as well, but what about "spam['two words']" or "spam[2]"?
>> 
>> I assume the qualname is just the name.
>> 
>> Also, would this go through the descriptor mechanism if you def an attribution?
> 

From tritium-list at sdamon.com  Mon Oct 26 03:09:13 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 03:09:13 -0400
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
Message-ID: <562DD199.1040004@sdamon.com>

to be clear, I propose the following to be equivalent (to clear up what 
I mean)

```
# classes:

class Foo:
     def bar(self):
         pass

# vs

class Foo:
     pass

def Foo.bar(self):
     pass

# Instances:

def foo(args): pass

inst = Class()
inst.foo = foo

# vs

inst =  Class()

def inst.foo(args): pass

```

does this make that clear?

On 10/26/2015 03:05, Andrew Barnert wrote:
> On Oct 25, 2015, at 23:23, Alexander Walters <tritium-list at sdamon.com> wrote:
>> I imagine in the case of assigning to a class or instance, the name would be the same (and binding to self would act the same) as if it were assigned in the traditional way.
> In what traditional way? When you def a function, its name is the name given in the def statement. If you later assign it to a member of an object, that doesn't change its name. So, that doesn't answer the question.
>
> As for the binding to self, that doesn't happen at assignment time, so it can't happen the same way as at assignment time. The binding happens later, each time the method is looked up. If an object doesn't have an attribute of the looked up name, but its type does, the type's attribute's __get__ method is called with the instance. So, there's nothing to do here at all; functions already have a __get__ method that returns a bound method (and if you assign a function to an instance rather than a type, it doesn't get bound). This is all explained pretty clearly in the descriptor HOWTO.
>
> But descriptors also work for setting, not just getting (which is how @property works), and that's what I was asking about. If I write "spam.eggs = 0", and type(spam) has a member named "eggs", its __set__ method will get called with the instance (spam) and the value (0). So, if I write "def spam.eggs(): pass", does it call the same descriptor method?
>
>>   I do not propose that assigning to classes in this way be considered a good idea.
> OK, but unless you're actually proposing to not allow it, you still need to work out what it would do.
>
>> as for when assigned to a data structure, my admittedly naive idea would be to set the name to '__none__' or some such.  I am open to other ideas though.
> Functions defined with lambda get the name "<lambda>", so you'd probably want angle brackets here as well. But otherwise that seems reasonable, I guess.
>
>>> On 10/26/2015 02:19, Andrew Barnert wrote:
>>> Seems interesting.
>>>
>>> What's the name of the defined function? For an attribution like "spam.eggs" you'd probably want it to be "eggs", and I guess for "spam['eggs']" as well, but what about "spam['two words']" or "spam[2]"?
>>>
>>> I assume the qualname is just the name.
>>>
>>> Also, would this go through the descriptor mechanism if you def an attribution?


From ian.g.kelly at gmail.com  Mon Oct 26 03:27:58 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 26 Oct 2015 01:27:58 -0600
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>
References: <562DC1D8.8060003@sdamon.com>
 <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>
Message-ID: <CALwzid=TTJ04OQRPY=MYxyrMsu88rWhTWGUmbyd3C_aUG1DboQ@mail.gmail.com>

> On Sun, Oct 25, 2015 at 11:02 PM, Alexander Walters
> <tritium-list at sdamon.com> wrote:
>>
>> In my code, I write a lot of dispatch dictionaries (for lack of a switch
>> statement, but I will not hold my breath for that).  In trying to make
>> writing these dictionaries less annoying, I tend to use many lambdas.  I can
>> let you guess at what problems that has resulted in.  Of course, the
>> preferred way to write such dictionaries is by using a regular function, and
>> adding that function to a dictionary.  This isn't exactly a problem - it
>> works, and works well, but it is annoying to write, and leaves artifacts of
>> those functions in module scope.  I propose a little bit of sugar to make
>> this a little less annoying.
>>
>> If `def` is allowed to assign to anything (anything that is legal at the
>> left hand side of an = in that scope), annoying artifacts go away.  The
>> syntax I propose should be backwards compatible.
>>
>> ```
>> dispatch = {}
>>
>> def dispatch['foo'](bar):
>>     return bar * bar
>> ```

What about:

def foo(bar)[baz](x):
    return x

This seems like it would complicate parsing as the parser can't be
sure whether (bar) is a parameter list or an argument list until it
reaches the following [baz].

From tritium-list at sdamon.com  Mon Oct 26 03:23:31 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 03:23:31 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562A7F1D.90108@nedbatchelder.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com>
Message-ID: <562DD4F3.5000901@sdamon.com>



On 10/23/2015 14:40, Ned Batchelder wrote:
> On 10/22/15 6:56 PM, Yury Selivanov wrote:
>> In principle, there is no reason why *both* of these groups
>> of users can't use one tool and be happy.  I propose to
>> establish a convention in PEP 8, explaining that, while both
>> literals are semantically equivalent,
>>
>> - r'..' strings *should* be used for regexps,
>>
>> - R'..' strings *should* be used for unstyled raw strings,
>>
>> and tools *should* treat them as such.
>>
>> All of this is merely about codifying the current status quo. 
> But you are not codifying the status quo.  The distinction you are 
> proposing is one that you have invented.  I have never used R"" strings.
>
> I think the best solution to the problem is to improve the 
> highlighters, and luckily you have written one!  To me, it is clear 
> which of these strings is the regex:
>
>     r"\d+"
>     r"\dir"
>
> If the highlighters tried some heuristics, they could do a better job 
> "being helpful" by making better guesses about the meaning of 
> programs.  I don't mind when highlighters make wrong guesses, as long 
> as they don't ruin the entire rest of the file.  But better guesses 
> will be better. :)
>
> --Ned.
>

it should be noted that most regexes are also valid paths on NTFS. is 
r'\dir[a-zA-Z0-9]\\' a path or a regex?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/f404e51d/attachment.html>

From abarnert at yahoo.com  Mon Oct 26 03:55:18 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 00:55:18 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DD199.1040004@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
 <562DD199.1040004@sdamon.com>
Message-ID: <00A91850-DD3B-4BA8-B78A-CA6EF7C16906@yahoo.com>

On Oct 26, 2015, at 00:09, Alexander Walters <tritium-list at sdamon.com> wrote:
> 
> to be clear, I propose the following to be equivalent (to clear up what I mean)

Yes, that clears it up. This is what I thought you probably wanted. But I think it would be better to keep it simpler. Instead of this:

> # classes:
> 
> class Foo:
>    def bar(self):
>        pass

? your first example would be simpler if it were equivalent to this:

class Foo:
    pass
def _dummy(self):
    pass
Foo.bar = _dummy

(except of course not binding anything to "_dummy").

Notice that this is now exactly like your second example, with an instance, so we only need one rule instead of two. And in simple cases it does exactly what you want here, too. And in complex cases, e.g., involving "__class__" or no-argument "super" or names accessible in the class-definition scope, the implementation doesn't need to do any magic, while with your version, it would. (That magic might be convenient in some cases, but since you don't want to encourage this kind of use anyway, why make things more complicated just to make the discouraged use more convenient?)

In fact, I would make this as simple as possible: Just change the "funcname" in the syntax from an identifier to a target. Then, any "def" statement whose works by defining a function, then assigning it to the target (which is legal iff the target is a legal assignment target). It's a perfectly normal assignment, following all the same __setitem__, __setattr__, descriptor, etc. rules as any other assignment.

That does mean the name of the function ends up being "Foo.bar" or "foo.bar" or "foo['bar']", and at least for the first two could end up being confusing, and even more so for the qualname, so maybe you'd want to add one minor tweak: if the target is not an identifier, the name is just "<none>" or "<anon>" or something. But I don't think you want to go any farther. (In particular, making any of the examples come out as "bar" is just extra complexity that only improves uses you don't want to encourage.)


From tritium-list at sdamon.com  Mon Oct 26 04:01:22 2015
From: tritium-list at sdamon.com (Alexander Walters)
Date: Mon, 26 Oct 2015 04:01:22 -0400
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <00A91850-DD3B-4BA8-B78A-CA6EF7C16906@yahoo.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
 <562DD199.1040004@sdamon.com>
 <00A91850-DD3B-4BA8-B78A-CA6EF7C16906@yahoo.com>
Message-ID: <562DDDD2.4090106@sdamon.com>


On 10/26/2015 03:55, Andrew Barnert wrote:
> ? your first example would be simpler if it were equivalent to this:
>
> class Foo:
>      pass
> def _dummy(self):
>      pass
> Foo.bar = _dummy
>
> (except of course not binding anything to "_dummy").
>
> Notice that this is now exactly like your second example, with an instance, so we only need one rule instead of two. And in simple cases it does exactly what you want here, too. And in complex cases, e.g., involving "__class__" or no-argument "super" or names accessible in the class-definition scope, the implementation doesn't need to do any magic, while with your version, it would. (That magic might be convenient in some cases, but since you don't want to encourage this kind of use anyway, why make things more complicated just to make the discouraged use more convenient?)

Agreed, that is a better idea.

> In fact, I would make this as simple as possible: Just change the "funcname" in the syntax from an identifier to a target. Then, any "def" statement whose works by defining a function, then assigning it to the target (which is legal iff the target is a legal assignment target). It's a perfectly normal assignment, following all the same __setitem__, __setattr__, descriptor, etc. rules as any other assignment.
>
> That does mean the name of the function ends up being "Foo.bar" or "foo.bar" or "foo['bar']", and at least for the first two could end up being confusing, and even more so for the qualname, so maybe you'd want to add one minor tweak: if the target is not an identifier, the name is just "<none>" or "<anon>" or something. But I don't think you want to go any farther. (In particular, making any of the examples come out as "bar" is just extra complexity that only improves uses you don't want to encourage.)
>
The more I think about it function.__name__ when assigned like this 
might be better served as `'<anon: %s>' % (whatever_you_assigned_to,)`

From p.f.moore at gmail.com  Mon Oct 26 04:37:50 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 26 Oct 2015 08:37:50 +0000
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <562B43DF.9030809@mail.de>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de> <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
Message-ID: <CACac1F-MGifm+dBnrhn9wOLzP-q7=UywE3EM8FKCXuVXfivgbQ@mail.gmail.com>

On 24 October 2015 at 09:39, Sven R. Kunze <srkunze at mail.de> wrote:
> 2) Is Python not an open-source project? Could not anybody contribute? To me
> that means, the sky is the limit.

The problem is that many people contribute ideas (hence this list) but
only a few contribute code, and even fewer contribute the commitment
to maintain that code long-term.

One of the roles needed on this list is for people to say that the
*current* developers (i.e, those contributing code and maintenance)
are unlikely to spend their time implementing and maintaining an idea
proposed here. That doesn't mean the idea is rejected out of hand, it
just means that to progress the idea the person proposing it is likely
to have to go further, and offer code, and potentially ongoing
maintenance if it's a change that can't be covered by "normal"
maintenance (e.g., a inclusion of a new or specialist library).

So ideas are welcome, but they need to be backed by an implementation,
and "I'm not interested in implementing this idea for you" is an
entirely legitimate (and not uncommon) response that people need to be
prepared to accept.

Paul

From python at 2sn.net  Mon Oct 26 04:57:29 2015
From: python at 2sn.net (Alexander Heger)
Date: Mon, 26 Oct 2015 19:57:29 +1100
Subject: [Python-ideas] Allowing def to assign to anything
Message-ID: <CAN3CYHw-kLTO7e-7Qv5jp1AfKrLrqY8TzbWZOhUY5ZaCo8DxVg@mail.gmail.com>

how about

class Dispatch(object):
    @staticmethod
    def foo(bar):
        return bar * bar
    def __getitem__(self, key):
        return self.__getattribute__(key)
dispatch = Dispatch()

>>> test.dispatch['foo'](2)
 4

this leaves the module namespace less polluted.

-Alexander


On 26 October 2015 at 18:27, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On Sun, Oct 25, 2015 at 11:02 PM, Alexander Walters
>> <tritium-list at sdamon.com> wrote:
>>>
>>> In my code, I write a lot of dispatch dictionaries (for lack of a switch
>>> statement, but I will not hold my breath for that).  In trying to make
>>> writing these dictionaries less annoying, I tend to use many lambdas.  I can
>>> let you guess at what problems that has resulted in.  Of course, the
>>> preferred way to write such dictionaries is by using a regular function, and
>>> adding that function to a dictionary.  This isn't exactly a problem - it
>>> works, and works well, but it is annoying to write, and leaves artifacts of
>>> those functions in module scope.  I propose a little bit of sugar to make
>>> this a little less annoying.
>>>
>>> If `def` is allowed to assign to anything (anything that is legal at the
>>> left hand side of an = in that scope), annoying artifacts go away.  The
>>> syntax I propose should be backwards compatible.
>>>
>>> ```
>>> dispatch = {}
>>>
>>> def dispatch['foo'](bar):
>>>     return bar * bar
>>> ```
>
> What about:
>
> def foo(bar)[baz](x):
>     return x
>
> This seems like it would complicate parsing as the parser can't be
> sure whether (bar) is a parameter list or an argument list until it
> reaches the following [baz].
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From abarnert at yahoo.com  Mon Oct 26 05:08:18 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 02:08:18 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DDDD2.4090106@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
 <562DD199.1040004@sdamon.com>
 <00A91850-DD3B-4BA8-B78A-CA6EF7C16906@yahoo.com>
 <562DDDD2.4090106@sdamon.com>
Message-ID: <CFAD0699-5D52-427C-AADC-53E9975BA3A4@yahoo.com>

On Oct 26, 2015, at 01:01, Alexander Walters <tritium-list at sdamon.com> wrote:
> 
> 
>> On 10/26/2015 03:55, Andrew Barnert wrote:
>> ? your first example would be simpler if it were equivalent to this:
>> 
>> class Foo:
>>     pass
>> def _dummy(self):
>>     pass
>> Foo.bar = _dummy
>> 
>> (except of course not binding anything to "_dummy").
>> 
>> Notice that this is now exactly like your second example, with an instance, so we only need one rule instead of two. And in simple cases it does exactly what you want here, too. And in complex cases, e.g., involving "__class__" or no-argument "super" or names accessible in the class-definition scope, the implementation doesn't need to do any magic, while with your version, it would. (That magic might be convenient in some cases, but since you don't want to encourage this kind of use anyway, why make things more complicated just to make the discouraged use more convenient?)
> 
> Agreed, that is a better idea.
> 
>> In fact, I would make this as simple as possible: Just change the "funcname" in the syntax from an identifier to a target. Then, any "def" statement whose works by defining a function, then assigning it to the target (which is legal iff the target is a legal assignment target). It's a perfectly normal assignment, following all the same __setitem__, __setattr__, descriptor, etc. rules as any other assignment.
>> 
>> That does mean the name of the function ends up being "Foo.bar" or "foo.bar" or "foo['bar']", and at least for the first two could end up being confusing, and even more so for the qualname, so maybe you'd want to add one minor tweak: if the target is not an identifier, the name is just "<none>" or "<anon>" or something. But I don't think you want to go any farther. (In particular, making any of the examples come out as "bar" is just extra complexity that only improves uses you don't want to encourage.)
> The more I think about it function.__name__ when assigned like this might be better served as `'<anon: %s>' % (whatever_you_assigned_to,)`

Actually, now that I think about it, I think that would be up to each implementation?I don't think Python specifies "<lambda>" as the name for lambdas, that's just what CPython does, so the same would be true here, right?

But yeah, now that I see the idea, I definitely like '<anon: foo.bar>' a lot better than '<anon>' or 'foo.bar'.

From ncoghlan at gmail.com  Mon Oct 26 05:52:16 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Oct 2015 10:52:16 +0100
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CACac1F-MGifm+dBnrhn9wOLzP-q7=UywE3EM8FKCXuVXfivgbQ@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de> <5627E6F7.50200@mrabarnett.plus.com>
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>
 <56290D47.9030800@mail.de>
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>
 <562B43DF.9030809@mail.de>
 <CACac1F-MGifm+dBnrhn9wOLzP-q7=UywE3EM8FKCXuVXfivgbQ@mail.gmail.com>
Message-ID: <CADiSq7fuDpU4qZ18jFtkPaeWZ3kRvLVu+Nx_NCTVU3MMXZ2Gtw@mail.gmail.com>

On 26 October 2015 at 09:37, Paul Moore <p.f.moore at gmail.com> wrote:
> So ideas are welcome, but they need to be backed by an implementation,
> and "I'm not interested in implementing this idea for you" is an
> entirely legitimate (and not uncommon) response that people need to be
> prepared to accept.

It's also the case that the number of ways we can make Python worse is
uncountably infinite, while the number of ways we can make it easier
to use without also making it harder to learn is vanishingly small by
comparison. (Hence the controversy over concepts like gradual typing,
which improves usability for larger projects and teams, but introduces
a new concept that people are likely to eventually have to learn as
they gain further experience with the language)

This means that *both* python-ideas and python-dev serve primarily as
filters when it comes to new features: python-ideas finds the ideas
that *might* have potential, and helps refine them into a concrete
pitch to python-dev (or an enhancement request on the issue tracker
for simpler ideas that don't need full python-dev review), while
python-dev is more focused on delivering a relatively prompt yes/no
answer on whether or not an idea is acceptable in principle, and then
refining the practical details if the answer is "yes".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ned at nedbatchelder.com  Mon Oct 26 07:33:38 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Mon, 26 Oct 2015 07:33:38 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562DD4F3.5000901@sdamon.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
Message-ID: <562E0F92.7050400@nedbatchelder.com>


On 10/26/15 3:23 AM, Alexander Walters wrote:
>
> On 10/23/2015 14:40, Ned Batchelder wrote:
>> On 10/22/15 6:56 PM, Yury Selivanov wrote:
>>> In principle, there is no reason why *both* of these groups
>>> of users can't use one tool and be happy.  I propose to
>>> establish a convention in PEP 8, explaining that, while both
>>> literals are semantically equivalent,
>>>
>>> - r'..' strings *should* be used for regexps,
>>>
>>> - R'..' strings *should* be used for unstyled raw strings,
>>>
>>> and tools *should* treat them as such.
>>>
>>> All of this is merely about codifying the current status quo. 
>> But you are not codifying the status quo.  The distinction you are 
>> proposing is one that you have invented.  I have never used R"" strings.
>>
>> I think the best solution to the problem is to improve the 
>> highlighters, and luckily you have written one!  To me, it is clear 
>> which of these strings is the regex:
>>
>>     r"\d+"
>>     r"\dir"
>>
>> If the highlighters tried some heuristics, they could do a better job 
>> "being helpful" by making better guesses about the meaning of 
>> programs.  I don't mind when highlighters make wrong guesses, as long 
>> as they don't ruin the entire rest of the file.  But better guesses 
>> will be better. :)
>>
>> --Ned.
>>
>
> it should be noted that most regexes are also valid paths on NTFS.  is 
> r'\dir[a-zA-Z0-9]\\' a path or a regex?
I understand developers' penchant for getting everything precisely right 
and accounting for the darkest of corners and the farthest reaches of 
obscure edge cases.  But I'm talking about making a reasonable guess.  
If the string contains square brackets, especially paired brackets with 
hyphens inside, it's probably a regex.

--Ned.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/cf75f351/attachment-0001.html>

From abarnert at yahoo.com  Mon Oct 26 07:55:55 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 04:55:55 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <562E0F92.7050400@nedbatchelder.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
 <562E0F92.7050400@nedbatchelder.com>
Message-ID: <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>

On Oct 26, 2015, at 04:33, Ned Batchelder <ned at nedbatchelder.com> wrote:
> 
> 
>> On 10/26/15 3:23 AM, Alexander Walters wrote:
>> 
>>> On 10/23/2015 14:40, Ned Batchelder wrote:
>>>> On 10/22/15 6:56 PM, Yury Selivanov wrote:
>>>> In principle, there is no reason why *both* of these groups 
>>>> of users can't use one tool and be happy.  I propose to 
>>>> establish a convention in PEP 8, explaining that, while both 
>>>> literals are semantically equivalent, 
>>>> 
>>>> - r'..' strings *should* be used for regexps, 
>>>> 
>>>> - R'..' strings *should* be used for unstyled raw strings, 
>>>> 
>>>> and tools *should* treat them as such. 
>>>> 
>>>> All of this is merely about codifying the current status quo.
>>> But you are not codifying the status quo.  The distinction you are proposing is one that you have invented.  I have never used R"" strings.
>>> 
>>> I think the best solution to the problem is to improve the highlighters, and luckily you have written one!  To me, it is clear which of these strings is the regex:
>>> 
>>>     r"\d+"
>>>     r"\dir"
>>> 
>>> If the highlighters tried some heuristics, they could do a better job "being helpful" by making better guesses about the meaning of programs.  I don't mind when highlighters make wrong guesses, as long as they don't ruin the entire rest of the file.  But better guesses will be better. :)
>>> 
>>> --Ned.
>> 
>> it should be noted that most regexes are also valid paths on NTFS.  is r'\dir[a-zA-Z0-9]\\' a path or a regex?
> I understand developers' penchant for getting everything precisely right and accounting for the darkest of corners and the farthest reaches of obscure edge cases.  But I'm talking about making a reasonable guess.  If the string contains square brackets, especially paired brackets with hyphens inside, it's probably a regex.

From working on music tagging software, I can tell you that an awful lot of users have mp3s with square brackets, hyphens, and other such things in their filenames, so if your software makes any assumptions about what filenames look like, their libraries will break your software.

And to verify that this isn't some weird artifact of the way people used to name files on piracy networks back when people traded individual songs, I went to The Pirate Bay and checked the most popular current download in any category, and its first file is named:

    [ www.CpasBien.pw ] Tomorrowland.2015.TRUEFRENCH.BDRip.VxiD-EXTREME.avi

So, I don't think you can assume that paired square brackets or hyphens mean something is not a Windows pathname.

Of course with a wide enough corpus of filenames people have to deal with, you could come up with a better heuristic. (Not many regexes have character classes that are dotted domain names, or match a standard language code followed by "-sub", or most of the other examples I see from a quick scan.) But just guessing based on what you guess filenames are like without looking around is not going to get you that far.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/2ea0decc/attachment.html>

From steve at pearwood.info  Mon Oct 26 09:45:39 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 27 Oct 2015 00:45:39 +1100
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC1D8.8060003@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
Message-ID: <20151026134537.GX3813@ando.pearwood.info>

On Mon, Oct 26, 2015 at 02:02:00AM -0400, Alexander Walters wrote:

> In my code, I write a lot of dispatch dictionaries (for lack of a switch 
> statement, but I will not hold my breath for that).  In trying to make 
> writing these dictionaries less annoying, I tend to use many lambdas.  I 
> can let you guess at what problems that has resulted in.  Of course, the 
> preferred way to write such dictionaries is by using a regular function, 
> and adding that function to a dictionary.  This isn't exactly a problem 
> - it works, and works well, but it is annoying to write, and leaves 
> artifacts of those functions in module scope.  I propose a little bit of 
> sugar to make this a little less annoying.

In this case, leaving "artifacts" in the module scope is a feature. If 
your function is simple enough to express in a simple expression, then a 
lambda may be the right solution. But if it requires a full block, then 
chances are that it's too complex for it to be obviously correct, which 
means you should test it. Giving the function a name and module scope 
supports testing.

But if you really want to get rid of it:

del the_function

after adding it to the dispatch table.

Or, stick them in their own namespace. For a package, that might mean 
moving the dispatch table and its associated functions into its own 
module. Or, put them in a class:

class Switch:
    def cheese(x):
        ...
    def spam(x):
        ...
    def eggs(x):
        ...

dispatch = {}
dispatch.update(Switch.__dict__)

result = dispatch[key](arg)


Or one could write a class decorator (or a metaclass) to post-process
the class and returns whatever you like. If I were doing this a lot, I 
would invest the time in building a nice switch construct, before 
looking for new syntax.


> If `def` is allowed to assign to anything (anything that is legal at the 
> left hand side of an = in that scope), annoying artifacts go away.  The 
> syntax I propose should be backwards compatible.


Assign to *anything*? 

a, b, c = 1, 2, 3

def a, b, c(x, y):
    ...


I don't see that, at least, working.


> ```
> dispatch = {}
> 
> def dispatch['foo'](bar):
>     return bar * bar
> ```
> 
> Does this make anything possible that is impossible now?  No.  But it 
> does make the intent of the module author clear - the function is only 
> ever intended to live inside that dict, or list, or other structure.  

It's not clear to me. To me, it looks like you've badly mistyped a 
function annotation. If you hadn't explained what you wanted, I wouldn't 
have a clue what "def dispatch['foo'](bar)" meant.


-- 
Steve

From vgr255 at live.ca  Mon Oct 26 09:57:01 2015
From: vgr255 at live.ca (Emanuel Barry)
Date: Mon, 26 Oct 2015 09:57:01 -0400
Subject: [Python-ideas] "else if" as equivalent for "elif"
In-Reply-To: <CALwzidkreOAsPvmJpcGiQ45ivfxMGMhfuEr+1-yK=vWNWfxZJA@mail.gmail.com>
References: <CALwzid=uikZyfnbkhb_=Tc_AK7QvE86B4=s8b1Z480xwFYd1TQ@mail.gmail.com>,
 <D6CC3A0D-0AF1-40E6-B64E-206521DDE976@gmail.com>
 <5627D667.8020301@mail.de>, <5627E6F7.50200@mrabarnett.plus.com>,
 <CAGifb9EP27goQyCjD8w72wKXc-9w8MPvbMEkT3S4guMtaN7bUQ@mail.gmail.com>,
 <CAP7h-xa=xZhwncjOYBKhkESxxqE+YznMud5i6Tx0PCcW9ax8uQ@mail.gmail.com>,
 <CAGifb9FUNtAyAaX+rqw_HiZgfBt1mcjDei_niYLPTe-Pcuwdsg@mail.gmail.com>,
 <56290D47.9030800@mail.de>,
 <CADiSq7ezJ3CdBLSUVJS6=LkEDcm-0UW5oBWFn3ix7K9Dk_dS_w@mail.gmail.com>,
 <562B43DF.9030809@mail.de>,
 <CAPTjJmrzAn__nTT6RLuHFRtyjTWNaQ6QjgOPVGgP7ajQvhuKNQ@mail.gmail.com>,
 <BLU172-W12F2A7E7B6F7797108852E91230@phx.gbl>,
 <CALwzidkreOAsPvmJpcGiQ45ivfxMGMhfuEr+1-yK=vWNWfxZJA@mail.gmail.com>
Message-ID: <BLU172-W297932B0D2C7E7CDC0F6ED91230@phx.gbl>

> From: ian.g.kelly at gmail.com
> Date: Sun, 25 Oct 2015 23:58:09 -0600
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] "else if" as equivalent for "elif"
> 
> On Sun, Oct 25, 2015 at 7:28 PM, Emanuel Barry <vgr255 at live.ca> wrote:
> > Here is yet another argument against this change -- indentation. Indentation
> > is, as we all well know, one of the core aspects of Python's syntax. In
> > every project I have worked on, and in PEP 8 (
> > https://www.python.org/dev/peps/pep-0008/#indentation ), 4-space indents are
> > used. 'elif' is exactly 4 characters long, which means it lines up pretty
> > well with other similar keywords such as 'else' or 'for' (with the space
> > that follows it). In that regard, 'else if' would actually make code
> > *harder* to read, not easier! In the same sense that 'def' (followed by a
> > space) makes it very easy to spot the functions' names (if for some reason
> > you don't have syntax highlighting ... ), 'elif' makes it easy to spot the
> > beginning of another condition statement within a block.
> 
> I'm not following. Can you provide an example of how this hurts
> indentation? It seems a stretch to suggest that "elif" lines up with
> "for" because you're including the space that follows "for", but
> you're excluding the space that follows "elif".
> 
> I note that "else if" has the same length as "finally", if that matters.

I agree it might sound a bit inconsistent that I take into account the space following 'for' but not the one following 'elif' - my point is that a human reader can easily look at the first characters before the next indentation level kicks in. This might not be a good example though now that I think of it, as a human reader would see that the line is not just 'else' (which was pretty much my original point). I recognize that you're right though, and I take back my words.
Cheers,-Emanuel Barry 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/333c48d4/attachment-0001.html>

From rymg19 at gmail.com  Mon Oct 26 10:30:12 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Mon, 26 Oct 2015 09:30:12 -0500
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC1D8.8060003@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
Message-ID: <6CF8150C-9696-46C0-B72D-6D9ED0095E78@gmail.com>

You can just do:

dispatch = {}
def dispatcher(f):
    dispatch[f.__name__] = f

@dispatcher
def foo(bar):
    return bar * bar


I really think that might be weird and kind of jarring to read...

On October 26, 2015 1:02:00 AM CDT, Alexander Walters <tritium-list at sdamon.com> wrote:
>In my code, I write a lot of dispatch dictionaries (for lack of a
>switch 
>statement, but I will not hold my breath for that).  In trying to make 
>writing these dictionaries less annoying, I tend to use many lambdas. 
>I 
>can let you guess at what problems that has resulted in.  Of course,
>the 
>preferred way to write such dictionaries is by using a regular
>function, 
>and adding that function to a dictionary.  This isn't exactly a problem
>
>- it works, and works well, but it is annoying to write, and leaves 
>artifacts of those functions in module scope.  I propose a little bit
>of 
>sugar to make this a little less annoying.
>
>If `def` is allowed to assign to anything (anything that is legal at
>the 
>left hand side of an = in that scope), annoying artifacts go away.  The
>
>syntax I propose should be backwards compatible.
>
>```
>dispatch = {}
>
>def dispatch['foo'](bar):
>     return bar * bar
>```
>
>Does this make anything possible that is impossible now?  No.  But it 
>does make the intent of the module author clear - the function is only 
>ever intended to live inside that dict, or list, or other structure.  
>This, to me, is less annoying to write, and is more readable.  This 
>obviously could be used outside of creating dispatch dictionaries, but 
>that is the use case I would benefit from.
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>https://mail.python.org/mailman/listinfo/python-ideas
>Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/191718b3/attachment.html>

From yselivanov.ml at gmail.com  Mon Oct 26 11:17:43 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Mon, 26 Oct 2015 11:17:43 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <20151023014208.GQ3813@ando.pearwood.info>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <BC1F7A8A-D951-4F37-B2DD-D8B91481584A@yahoo.com>
 <20151023014208.GQ3813@ando.pearwood.info>
Message-ID: <562E4417.4090709@gmail.com>

On 2015-10-22 9:42 PM, Steven D'Aprano wrote:
> If you want my advice, it would be to either just pick whichever style
> of formatting you like, and people will either like your code
> highlighter and use it, or they won't, or you should consider a plug-in
> architecture where your users can choose the string-highlighter style
> they want, and even extend it to other DSLs.

Yeah, I finally figured out a way to make the highlighter easily
configurable, so that users can turn on/off highlighting of
regexes, formatting literals, python2 support etc.

This discussion was actually quite useful to me.
Thanks to everybody for feedback!

Yury

From abarnert at yahoo.com  Mon Oct 26 15:30:05 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 12:30:05 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <20151026134537.GX3813@ando.pearwood.info>
References: <562DC1D8.8060003@sdamon.com>
 <20151026134537.GX3813@ando.pearwood.info>
Message-ID: <B8CBF485-2FE5-4234-BFD8-A1837742A75F@yahoo.com>

On Oct 26, 2015, at 06:45, Steven D'Aprano <steve at pearwood.info> wrote:
> 
>> On Mon, Oct 26, 2015 at 02:02:00AM -0400, Alexander Walters wrote:
>> 
>> In my code, I write a lot of dispatch dictionaries (for lack of a switch 
>> statement, but I will not hold my breath for that).  In trying to make 
>> writing these dictionaries less annoying, I tend to use many lambdas.  I 
>> can let you guess at what problems that has resulted in.  Of course, the 
>> preferred way to write such dictionaries is by using a regular function, 
>> and adding that function to a dictionary.  This isn't exactly a problem 
>> - it works, and works well, but it is annoying to write, and leaves 
>> artifacts of those functions in module scope.  I propose a little bit of 
>> sugar to make this a little less annoying.
> 
> In this case, leaving "artifacts" in the module scope is a feature. If 
> your function is simple enough to express in a simple expression, then a 
> lambda may be the right solution. But if it requires a full block, then 
> chances are that it's too complex for it to be obviously correct, which 
> means you should test it.

There is a bit of a boundary here: some functions are a single statement, which can't be written as a lambda, but are just as trivial as functions that can be. And functions that require two statements aren't that much more complex.

And meanwhile, there are plenty of people who twist things into knots to fit things in expressions that don't belong as expressions just so they can use lambda, and this proposal could give their colleagues/teacher/conscience a better way to say "that should be a def" and answer the objections about "but that would mean 3 lines of boilerplate for a 2-line function".

Of course without a realistic example instead of just the empty toy definitions in the original proposal, it's hard to see if there really would be such benefits here, or if it's just a theoretical possibility that would almost never arise, so it's definitely worth you pointing out this issue.

> Giving the function a name and module scope 
> supports testing.

It's not that hard to write unit tests that call dispatcher['spam'] instead of calling spam. Your unit test framework may not do this out of the box, but if you're going to be testing lots of functions like this, that's just something you have to add support for once.

> But if you really want to get rid of it:
> 
> del the_function
> 
> after adding it to the dispatch table.

Or, if you're worried about people calling them accidentally, just have the decorator return None so that's an error. Or, if you're worried about them showing up in tab completion, import *, etc., prefix them with _ and/or leave them out of __all__. The only time "polluting the namespace" is literally a concern is if you might have something else with the same name and you don't want one of them to erase the other; usually it's one of these other things you're really concerned with.

> Or, stick them in their own namespace. For a package, that might mean 
> moving the dispatch table and its associated functions into its own 
> module. Or, put them in a class:

Or just local to a function:

    def make_dispatch_table():
        def spam(x): pass
        def eggs(x): pass
        def cheese(x): pass
        return {'spam': spam, 'eggs': eggs, 'cheese': cheese}

Or, if you really want, you could even replace the last line with:

        return {k: v for (k, v) in locals().items() if callable(v)}

Anyway, it's worth noting that your submodule idea and the local function idea share something in common: because there is a scope the functions are defined in, and they still live in that scope, they can call each other. The OP's suggestion, and your del suggestion, make that impossible. Also, inspect, or manual inspection by hacking away at the REPL, will work a lot more easily.

I still think this proposal (suitably worked out) isn't a bad idea, but I agree that you've given good reasons why it isn't necessary for the stated use case, and why we need better and/or more complete examples to evaluate it.

>> If `def` is allowed to assign to anything (anything that is legal at the 
>> left hand side of an = in that scope), annoying artifacts go away.  The 
>> syntax I propose should be backwards compatible.
> 
> Assign to *anything*? 

This is part of why I was trying to get him to refine it into a complete specification instead of a vague idea.

The start of the answer here is pretty obvious: "def" takes an assignment target?not a target list, or another assignment, even though of course both of those are legal on the left side of an "=". (And further extensions to iterable unpacking, or even full pattern-matching assignment, wouldn't change what "target" means.)

But that still isn't a complete answer. There are things that are allowed as targets by the grammar, but are still syntax errors ("*spam = 2") in an assignment, and I'm pretty sure the list of such things would be slightly different for def statements. (For example, a target can be a parenthesized target list, or a slicing. For a single assignment, that's not a syntax error, although of course it's a type error at runtime if the value isn't iterable; for a function definition, that should probably be a syntax error.) So someone still needs to write out a semantic specification like the one in the docs for assignment statements.

But at least starting with "an assignment target" instead of "anything to the left of '='" is a start. It's as good as writing out the grammar, and a lot easier for most people to understand, and I think it gets the intuitive idea across. 

From abarnert at yahoo.com  Mon Oct 26 15:38:42 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 12:38:42 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
 <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
Message-ID: <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>

On Oct 26, 2015, at 04:55, Andrew Barnert via Python-ideas <python-ideas at python.org> wrote:
> 
>> On Oct 26, 2015, at 04:33, Ned Batchelder <ned at nedbatchelder.com> wrote:
>> 
>> 
>>> On 10/26/15 3:23 AM, Alexander Walters wrote:
>>> 
>>>> On 10/23/2015 14:40, Ned Batchelder wrote:
>>>>> On 10/22/15 6:56 PM, Yury Selivanov wrote:
>>>>> In principle, there is no reason why *both*           of these groups 
>>>>> of users can't use one tool and be happy.  I propose to 
>>>>> establish a convention in PEP 8, explaining that, while both 
>>>>> literals are semantically equivalent, 
>>>>> 
>>>>> - r'..' strings *should* be used for regexps, 
>>>>> 
>>>>> - R'..' strings *should* be used for unstyled raw strings, 
>>>>> 
>>>>> and tools *should* treat them as such. 
>>>>> 
>>>>> All of this is merely about codifying the current status quo.
>>>> But you are not codifying the status quo.  The distinction you are proposing is one that you have invented.  I have never used R"" strings.
>>>> 
>>>> I think the best solution to the problem is to improve the highlighters, and luckily you have written one!  To me, it is clear which of these strings is the regex:
>>>> 
>>>>     r"\d+"
>>>>     r"\dir"
>>>> 
>>>> If the highlighters tried some heuristics, they could do a better job "being helpful" by making better guesses about the meaning of programs.  I don't mind when highlighters make wrong guesses, as long as they don't ruin the entire rest of the file.  But better guesses will be better. :)
>>>> 
>>>> --Ned.
>>> 
>>> it should be noted that most regexes are also valid paths on NTFS.  is r'\dir[a-zA-Z0-9]\\' a path or a regex?
>> I understand developers' penchant for getting everything precisely right and accounting for the darkest of corners and the farthest reaches of obscure edge cases.  But I'm talking about making a reasonable guess.  If the string contains square brackets, especially paired brackets with hyphens inside, it's probably a regex.
> 
> From working on music tagging software, I can tell you that an awful lot of users have mp3s with square brackets, hyphens, and other such things in their filenames, so if your software makes any assumptions about what filenames look like, their libraries will break your software.
> 
> And to verify that this isn't some weird artifact of the way people used to name files on piracy networks back when people traded individual songs, I went to The Pirate Bay and checked the most popular current download in any category, and its first file is named:
> 
>     [ www.CpasBien.pw ] Tomorrowland.2015.TRUEFRENCH.BDRip.VxiD-EXTREME.avi
> 
> So, I don't think you can assume that paired square brackets or hyphens mean something is not a Windows pathname.
> 
> Of course with a wide enough corpus of filenames people have to deal with, you could come up with a better heuristic. (Not many regexes have character classes that are dotted domain names, or match a standard language code followed by "-sub", or most of the other examples I see from a quick scan.) But just guessing based on what you guess filenames are like without looking around is not going to get you that far.

Just for fun: is there a Python regex that matches all valid Python regexes? Obviously there's no actual regular expression that matches all regular expressions (you can't handle matched brackets without recursion or some other extension). I think a Python regex to match all actual regular expressions should be pretty easy. But I'm not sure about a Python regex to match all Python regexes. (Although I'll bet if it's possible for perl, someone has written a CPAN module. Probably without "verbose-mode" whitespace or comments.)

That still wouldn't solve the problem of the many things that are valid regexes and also valid Windows paths (not to mention valid doc strings with embedded code examples, because that includes any possible string?), or detecting things that are obviously intended to be regexes even though they're invalid, etc., so it's probably not very useful for this heuristic anyway. Hence the "just for fun"?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/fdda368d/attachment-0001.html>

From ben+python at benfinney.id.au  Mon Oct 26 17:44:12 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 27 Oct 2015 08:44:12 +1100
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org>
 <562969B4.5080406@gmail.com> <562A7F1D.90108@nedbatchelder.com>
 <562DD4F3.5000901@sdamon.com> <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
Message-ID: <85pp01mhjn.fsf@benfinney.id.au>

Andrew Barnert via Python-ideas
<python-ideas at python.org> writes:

> Just for fun: is there a Python regex that matches all valid Python
> regexes?

Yes: ?.*? matches all valid Python regexes.

> Obviously there's no actual regular expression that matches all
> regular expressions (you can't handle matched brackets without
> recursion or some other extension).

You seem to be seeking something else: a pattern that matches all valid
regex patterns, *and* will never match any string that is not a valid
regex pattern. The latter is rather more difficult.

-- 
 \      ?When I was born I was so surprised I couldn't talk for a year |
  `\                                        and a half.? ?Gracie Allen |
_o__)                                                                  |
Ben Finney


From mertz at gnosis.cx  Mon Oct 26 17:53:48 2015
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 26 Oct 2015 14:53:48 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <85pp01mhjn.fsf@benfinney.id.au>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org>
 <562969B4.5080406@gmail.com> <562A7F1D.90108@nedbatchelder.com>
 <562DD4F3.5000901@sdamon.com> <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
 <85pp01mhjn.fsf@benfinney.id.au>
Message-ID: <CAEbHw4YjZy+SQT4M=G5S0YbdzGRVkKRzbd-3=Od30ffm1TDg7Q@mail.gmail.com>

Obviously there can't be a regex to exclude everything that isn't a regex.
Parentheses can nest to unlimited depths, so you need a formal grammar. But
virtually everything that is a Windows path is also formally grammatical
regex too (as are many things with no plausible likely intention as such)
On Oct 26, 2015 2:44 PM, "Ben Finney" <ben+python at benfinney.id.au> wrote:

> Andrew Barnert via Python-ideas
> <python-ideas at python.org> writes:
>
> > Just for fun: is there a Python regex that matches all valid Python
> > regexes?
>
> Yes: ?.*? matches all valid Python regexes.
>
> > Obviously there's no actual regular expression that matches all
> > regular expressions (you can't handle matched brackets without
> > recursion or some other extension).
>
> You seem to be seeking something else: a pattern that matches all valid
> regex patterns, *and* will never match any string that is not a valid
> regex pattern. The latter is rather more difficult.
>
> --
>  \      ?When I was born I was so surprised I couldn't talk for a year |
>   `\                                        and a half.? ?Gracie Allen |
> _o__)                                                                  |
> Ben Finney
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/3a776358/attachment.html>

From abarnert at yahoo.com  Mon Oct 26 18:23:39 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 15:23:39 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <CAEbHw4YjZy+SQT4M=G5S0YbdzGRVkKRzbd-3=Od30ffm1TDg7Q@mail.gmail.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
 <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
 <85pp01mhjn.fsf@benfinney.id.au>
 <CAEbHw4YjZy+SQT4M=G5S0YbdzGRVkKRzbd-3=Od30ffm1TDg7Q@mail.gmail.com>
Message-ID: <AB307923-E375-4D2D-9AC3-DFE331AEB85E@yahoo.com>

On Oct 26, 2015, at 14:53, David Mertz <mertz at gnosis.cx> wrote:
> Obviously there can't be a regex to exclude everything that isn't a regex. Parentheses can nest to unlimited depths, so you need a formal grammar.
> 
As I said:
>> Obviously there's no actual regular expression that matches all regular expressions (you can't handle matched brackets without recursion or some other extension)

But you can do it trivially with Perl, or with the Regex module for Python, e.g., just by sticking a "(?1)" inside a pair of escaped parens plus a negative lookahead or nongreedy repetition. I'm not sure exactly how powerful Python (re module) regexes are (if I want to match something that isn't a regular language, I tend to reach for or build a dedicated parser rather than try to stretch re), but I know they're somewhere between actual regular expressions and perl regexes.
> But virtually everything that is a Windows path is also formally grammatical regex too (as are many things with no plausible likely intention as such)
> 
That's not true. You can, for example, have unclosed brackets or parens in a Windows path. And if you're wondering why anyone would do that, consider MP3 files auto-named based on their ID3v1/FreeDB metadata, which truncates fields at 29 or 30 bytes.

Anyway, as I said in the same message, it wouldn't be a useful heuristic because there's so much overlap, but you don't need to exaggerate that to make the same point.

>> On Oct 26, 2015 2:44 PM, "Ben Finney" <ben+python at benfinney.id.au> wrote:
>> Andrew Barnert via Python-ideas
>> <python-ideas at python.org> writes:
>> 
>> > Just for fun: is there a Python regex that matches all valid Python
>> > regexes?
>> 
>> Yes: ?.*? matches all valid Python regexes.
>> 
>> > Obviously there's no actual regular expression that matches all
>> > regular expressions (you can't handle matched brackets without
>> > recursion or some other extension).
>> 
>> You seem to be seeking something else: a pattern that matches all valid
>> regex patterns, *and* will never match any string that is not a valid
>> regex pattern. The latter is rather more difficult.
>> 
>> --
>>  \      ?When I was born I was so surprised I couldn't talk for a year |
>>   `\                                        and a half.? ?Gracie Allen |
>> _o__)                                                                  |
>> Ben Finney
>> 
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/8ed3e622/attachment.html>

From mertz at gnosis.cx  Mon Oct 26 18:31:37 2015
From: mertz at gnosis.cx (David Mertz)
Date: Mon, 26 Oct 2015 15:31:37 -0700
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <AB307923-E375-4D2D-9AC3-DFE331AEB85E@yahoo.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org>
 <562969B4.5080406@gmail.com> <562A7F1D.90108@nedbatchelder.com>
 <562DD4F3.5000901@sdamon.com> <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
 <85pp01mhjn.fsf@benfinney.id.au>
 <CAEbHw4YjZy+SQT4M=G5S0YbdzGRVkKRzbd-3=Od30ffm1TDg7Q@mail.gmail.com>
 <AB307923-E375-4D2D-9AC3-DFE331AEB85E@yahoo.com>
Message-ID: <CAEbHw4YKVy8oXUR5SzVa3Mck16GiSCsJznCiGAD8_TAJhKsxdQ@mail.gmail.com>

I think we're down to quibbling over the meaning of "virtually" here. I
recognize it's "not all" and you recognize it's "most" windows paths are
grammatically regexen.

So is it 80%, 95%, 99.9%? And do we mean "paths found in the wild" or
"paths as systematically enumerated" from possibility space?
On Oct 26, 2015 3:23 PM, "Andrew Barnert" <abarnert at yahoo.com> wrote:

> On Oct 26, 2015, at 14:53, David Mertz <mertz at gnosis.cx> wrote:
>
> Obviously there can't be a regex to exclude everything that isn't a regex.
> Parentheses can nest to unlimited depths, so you need a formal grammar.
>
> As I said:
>
> Obviously there's no actual regular expression that matches all regular
>> expressions (you can't handle matched brackets without recursion or some
>> other extension)
>
>
> But you can do it trivially with Perl, or with the Regex module for
> Python, e.g., just by sticking a "(?1)" inside a pair of escaped parens
> plus a negative lookahead or nongreedy repetition. I'm not sure exactly how
> powerful Python (re module) regexes are (if I want to match something that
> isn't a regular language, I tend to reach for or build a dedicated parser
> rather than try to stretch re), but I know they're somewhere between actual
> regular expressions and perl regexes.
>
> But virtually everything that is a Windows path is also formally
> grammatical regex too (as are many things with no plausible likely
> intention as such)
>
> That's not true. You can, for example, have unclosed brackets or parens in
> a Windows path. And if you're wondering why anyone would do that, consider
> MP3 files auto-named based on their ID3v1/FreeDB metadata, which truncates
> fields at 29 or 30 bytes.
>
> Anyway, as I said in the same message, it wouldn't be a useful heuristic
> because there's so much overlap, but you don't need to exaggerate that to
> make the same point.
>
> On Oct 26, 2015 2:44 PM, "Ben Finney" <ben+python at benfinney.id.au> wrote:
>
>> Andrew Barnert via Python-ideas
>> <python-ideas at python.org> writes:
>>
>> > Just for fun: is there a Python regex that matches all valid Python
>> > regexes?
>>
>> Yes: ?.*? matches all valid Python regexes.
>>
>> > Obviously there's no actual regular expression that matches all
>> > regular expressions (you can't handle matched brackets without
>> > recursion or some other extension).
>>
>> You seem to be seeking something else: a pattern that matches all valid
>> regex patterns, *and* will never match any string that is not a valid
>> regex pattern. The latter is rather more difficult.
>>
>> --
>>  \      ?When I was born I was so surprised I couldn't talk for a year |
>>   `\                                        and a half.? ?Gracie Allen |
>> _o__)                                                                  |
>> Ben Finney
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/27adde2c/attachment-0001.html>

From tjreedy at udel.edu  Mon Oct 26 21:43:23 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 26 Oct 2015 21:43:23 -0400
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
 <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
Message-ID: <n0mks0$l1m$1@ger.gmane.org>

On 10/26/2015 3:38 PM, Andrew Barnert via Python-ideas wrote:

> Just for fun: is there a Python regex that matches all valid Python
> regexes?

Could you take this discussion to python-list.  The PEP8 proposal has 
been rejected.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Mon Oct 26 22:04:04 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 26 Oct 2015 22:04:04 -0400
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <20151026134537.GX3813@ando.pearwood.info>
References: <562DC1D8.8060003@sdamon.com>
 <20151026134537.GX3813@ando.pearwood.info>
Message-ID: <n0mm2q$5in$1@ger.gmane.org>

On 10/26/2015 9:45 AM, Steven D'Aprano wrote:
> On Mon, Oct 26, 2015 at 02:02:00AM -0400, Alexander Walters wrote:
>
>> In my code, I write a lot of dispatch dictionaries (for lack of a switch
>> statement, but I will not hold my breath for that).  In trying to make
>> writing these dictionaries less annoying, I tend to use many lambdas.  I
>> can let you guess at what problems that has resulted in.  Of course, the
>> preferred way to write such dictionaries is by using a regular function,
>> and adding that function to a dictionary.  This isn't exactly a problem
>> - it works, and works well, but it is annoying to write, and leaves
>> artifacts of those functions in module scope.  I propose a little bit of
>> sugar to make this a little less annoying.
>
> In this case, leaving "artifacts" in the module scope is a feature. If
> your function is simple enough to express in a simple expression, then a
> lambda may be the right solution. But if it requires a full block, then
> chances are that it's too complex for it to be obviously correct, which
> means you should test it. Giving the function a name and module scope
> supports testing.
>
> But if you really want to get rid of it:
>
> del the_function
>
> after adding it to the dispatch table.
>
> Or, stick them in their own namespace. For a package, that might mean
> moving the dispatch table and its associated functions into its own
> module. Or, put them in a class:
>
> class Switch:
>      def cheese(x):
>          ...
>      def spam(x):
>          ...
>      def eggs(x):
>          ...
>
> dispatch = {}
> dispatch.update(Switch.__dict__)

dispatch = dict(Switch.__dict__)

Except for the extraneous dunder names in __dict__, I like this.
For the fastidious,
dispatch = {k:v for k, v in Switch.__dict__.items() if k[0] != '_'}

This could be wrapped in a function that would take either a module 
(globals()) or class (.__dict__) as the namespace source.

-- 
Terry Jan Reedy


From rosuav at gmail.com  Mon Oct 26 22:20:20 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 27 Oct 2015 13:20:20 +1100
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <562DC884.2080707@sdamon.com>
References: <562DC1D8.8060003@sdamon.com>
 <CAPTjJmqXgGHW9PKqHcd_rgCLoFtkdju0uzxvdQCv_DAOoRMMJA@mail.gmail.com>
 <562DC884.2080707@sdamon.com>
Message-ID: <CAPTjJmpwLPgA=Dxe9LSLOP-HL8gW8w4Fm6241ZZJiQhJ28cBEQ@mail.gmail.com>

On Mon, Oct 26, 2015 at 5:30 PM, Alexander Walters
<tritium-list at sdamon.com> wrote:
>
>
> On 10/26/2015 02:20, Chris Angelico wrote:
>>
>> I agree; the idea has been raised a few times, and I think it'd be
>> helpful. It's probably not necessary to allow the _entire_ scope of
>> "anything legal on the left of =", as that's pretty broad; even if the
>> only form allowed were obj[key], it'd be useful.
>
>
> I agree, that perhaps the scope could be a little wide when put this way,
> but my instinct is that 'allow anything already legal for =' would be the
> path of least frustration when implementing this.  I could be woefully
> wrong.  I do at least wish to assign to object with __setitem__ defined.

If you start by allowing only obj[key] (aka obj.__setitem__), and
maybe with only string literals or simple names for the key, it would
be a strict subset of Python syntax. That means that a future proposal
can easily expand on that (*un*like, for instance, the way
"{obj[key]}".format(obj=obj) operates), while providing only the one
or two most common usages:

def dispatch["foo"](x, y): pass

name = "bar"
def dispatch[name](x, y): pass


>> But for building a dispatch dictionary, you could simply decorate your
>> functions with a capturer:
>>
> This does, indeed, make life a bit easier in the here and now (and is
> similar to kitbashed techniques I already use).  I am hoping to make that
> obsolete.

Sure. You'll need some strong use-cases that justify _not_ using this
technique, though; since there's an existing way to do this, the bar
for new syntax is "show why the current way isn't good enough".

If you're worried about junk in your globals, what you could do is:

dispatch = {}
def cleanup():
    self = cleanup # we're going to delete ourselves, so snapshot to local
    for n,v in globals():
        if v is self: del globals()[n]

def cmd(func):
    dispatch[func.__name__] = func
    return cleanup

@cmd
def foo(): pass
@cmd
def bar(): pass

cleanup() # This function will self-destruct in three seconds. Two... one...


After all, a decorator can return anything it likes!

ChrisA

From rosuav at gmail.com  Mon Oct 26 22:48:36 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 27 Oct 2015 13:48:36 +1100
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
Message-ID: <CAPTjJmpNbyp_tqZg3Yba8gtFj=prZQ3H=G6e3Eg8O3n9G0kzkQ@mail.gmail.com>

On Mon, Oct 26, 2015 at 6:05 PM, Andrew Barnert via Python-ideas
<python-ideas at python.org> wrote:
> On Oct 25, 2015, at 23:23, Alexander Walters <tritium-list at sdamon.com> wrote:
>>
>> I imagine in the case of assigning to a class or instance, the name would be the same (and binding to self would act the same) as if it were assigned in the traditional way.
>
> In what traditional way? When you def a function, its name is the name given in the def statement. If you later assign it to a member of an object, that doesn't change its name. So, that doesn't answer the question.
>

dispatch = {}
def dispatch['foo'](): pass
assert dispatch['foo'].__name__ == "dispatch['foo']"

I have no problem with this, conceptually. Syntactically, it would be
defined as f"dispatch[{key!r}]" if the key is allowed to be an
arbitrary expression.

ChrisA

From rosuav at gmail.com  Mon Oct 26 22:26:23 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 27 Oct 2015 13:26:23 +1100
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <85pp01mhjn.fsf@benfinney.id.au>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org>
 <562969B4.5080406@gmail.com> <562A7F1D.90108@nedbatchelder.com>
 <562DD4F3.5000901@sdamon.com> <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
 <85pp01mhjn.fsf@benfinney.id.au>
Message-ID: <CAPTjJmo+wLhY=LPOC1jGEegu8+=pEgTrqef3=kEj47NbDuhZNg@mail.gmail.com>

On Tue, Oct 27, 2015 at 8:44 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> You seem to be seeking something else: a pattern that matches all valid
> regex patterns, *and* will never match any string that is not a valid
> regex pattern. The latter is rather more difficult.

"Rather more difficult" may be an understatement. A regex can contain
grouping parentheses which can arbitrarily nest, and matching that
with a regex is, AIUI, fundamentally impossible. So I don't think it's
possible to have a regex that validates a regex.

Fortunately, it's easy to write a function that validates a regex.

def is_regex(s):
    try: re.compile(s)
    except re.error: return False
    return True

ChrisA

From abarnert at yahoo.com  Mon Oct 26 23:48:11 2015
From: abarnert at yahoo.com (Andrew Barnert)
Date: Mon, 26 Oct 2015 20:48:11 -0700
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <CAPTjJmpNbyp_tqZg3Yba8gtFj=prZQ3H=G6e3Eg8O3n9G0kzkQ@mail.gmail.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
 <CAPTjJmpNbyp_tqZg3Yba8gtFj=prZQ3H=G6e3Eg8O3n9G0kzkQ@mail.gmail.com>
Message-ID: <C6700365-694E-41CD-A31A-1605F8CF3341@yahoo.com>

On Oct 26, 2015, at 19:48, Chris Angelico <rosuav at gmail.com> wrote:
> 
> On Mon, Oct 26, 2015 at 6:05 PM, Andrew Barnert via Python-ideas
> <python-ideas at python.org> wrote:
>> On Oct 25, 2015, at 23:23, Alexander Walters <tritium-list at sdamon.com> wrote:
>>> 
>>> I imagine in the case of assigning to a class or instance, the name would be the same (and binding to self would act the same) as if it were assigned in the traditional way.
>> 
>> In what traditional way? When you def a function, its name is the name given in the def statement. If you later assign it to a member of an object, that doesn't change its name. So, that doesn't answer the question.
>> 
> 
> dispatch = {}
> def dispatch['foo'](): pass
> assert dispatch['foo'].__name__ == "dispatch['foo']"
> 
> I have no problem with this, conceptually.

The paragraph you're replying to was about the naming for "def foo.bar():", which Alexander had suggested would work differently from "def foo['bar']():" (which he later retracted). So I'm not sure why you're proposing a different alternative for "def foo['bar']():" in response to these paragraphs.

At any rate, later in the thread, Alexander suggested something like "<anon: dispatch['foo']>", which I like better. The angle brackets and "anon" tell you that it's not guaranteed to be a REPL-friendly name?which would be possible in this case, but not in general. Plus, the part after the "anon" is just the text of the target node, so it's dead simple. (And if you're worried that this means the function's repr ends up with nested angle brackets?well, it does, but the same is true for lambda functions, and the hidden functions on comprehensions.)

> Syntactically, it would be
> defined as f"dispatch[{key!r}]" if the key is allowed to be an
> arbitrary expression.

But surely dispatch is also allowed to be an arbitrary expression? And, similarly, foo in foo.bar? And in those cases, the repr of those parts is worse than useless (you'd get the contents of dispatch instead of its name), so there's really no alternative.

Also, your suggestion means that this will assert:

    dispatch = {}
    def dispatch["foo"](): pass
assert dispatch['foo'].__name__ == 'dispatch["foo"]'

? will assert, so it's a pretty weak DWIM anyway.

Alexander's solution doesn't have any of those problems.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151026/3705f756/attachment-0001.html>

From python at mrabarnett.plus.com  Tue Oct 27 00:35:34 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 27 Oct 2015 04:35:34 +0000
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <CALwzid=TTJ04OQRPY=MYxyrMsu88rWhTWGUmbyd3C_aUG1DboQ@mail.gmail.com>
References: <562DC1D8.8060003@sdamon.com>
 <CAGu0AntCxMn5jaF6jrmKbv7ayvHPZXjiTyjTecsOW+RZNQww-w@mail.gmail.com>
 <CALwzid=TTJ04OQRPY=MYxyrMsu88rWhTWGUmbyd3C_aUG1DboQ@mail.gmail.com>
Message-ID: <562EFF16.6000602@mrabarnett.plus.com>

On 2015-10-26 07:27, Ian Kelly wrote:
>> On Sun, Oct 25, 2015 at 11:02 PM, Alexander Walters
>> <tritium-list at sdamon.com> wrote:
>>>
>>> In my code, I write a lot of dispatch dictionaries (for lack of a switch
>>> statement, but I will not hold my breath for that).  In trying to make
>>> writing these dictionaries less annoying, I tend to use many lambdas.  I can
>>> let you guess at what problems that has resulted in.  Of course, the
>>> preferred way to write such dictionaries is by using a regular function, and
>>> adding that function to a dictionary.  This isn't exactly a problem - it
>>> works, and works well, but it is annoying to write, and leaves artifacts of
>>> those functions in module scope.  I propose a little bit of sugar to make
>>> this a little less annoying.
>>>
>>> If `def` is allowed to assign to anything (anything that is legal at the
>>> left hand side of an = in that scope), annoying artifacts go away.  The
>>> syntax I propose should be backwards compatible.
>>>
>>> ```
>>> dispatch = {}
>>>
>>> def dispatch['foo'](bar):
>>>     return bar * bar
>>> ```
>
> What about:
>
> def foo(bar)[baz](x):
>      return x
>
> This seems like it would complicate parsing as the parser can't be
> sure whether (bar) is a parameter list or an argument list until it
> reaches the following [baz].
>
Would they be simpler to parse if they were:

def (bar) as dispatch['foo']:
     return bar * bar

and:

def (x) as foo(bar)[baz]:
     return x

?

In these instances, if there's no name then it should have the "as ..."
part; if there _is_ a name, then it shouldn't have the "as ..." part.

It's possible that:

def foo(bar):
     return x

could just be a shorter way of writing:

def (bar) as foo:
     return x


From rosuav at gmail.com  Tue Oct 27 00:29:20 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 27 Oct 2015 15:29:20 +1100
Subject: [Python-ideas] Allowing def to assign to anything
In-Reply-To: <C6700365-694E-41CD-A31A-1605F8CF3341@yahoo.com>
References: <562DC1D8.8060003@sdamon.com>
 <B317BCA0-1EB9-4200-81E2-54EAF6EA6D98@yahoo.com>
 <562DC6FA.3060309@sdamon.com>
 <9AA8DF99-0A00-478B-91C4-15363CCB480A@yahoo.com>
 <CAPTjJmpNbyp_tqZg3Yba8gtFj=prZQ3H=G6e3Eg8O3n9G0kzkQ@mail.gmail.com>
 <C6700365-694E-41CD-A31A-1605F8CF3341@yahoo.com>
Message-ID: <CAPTjJmqcTW_3YdtkpJ0=m-ThX4mNZ-=mCO3htkj6o4zE-t=x6w@mail.gmail.com>

On Tue, Oct 27, 2015 at 2:48 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> The paragraph you're replying to was about the naming for "def foo.bar():",
> which Alexander had suggested would work differently from "def
> foo['bar']():" (which he later retracted). So I'm not sure why you're
> proposing a different alternative for "def foo['bar']():" in response to
> these paragraphs.

There've been some weirdnesses in how emails get through, and I'm not
sure which previous emails I'd actually received at the time when I
wrote that. Hopefully it's just a temporary outage that's now cured.
If my post makes no sense in the full context of the thread, feel free
to ignore it.

ChrisA

From storchaka at gmail.com  Tue Oct 27 04:33:12 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 27 Oct 2015 10:33:12 +0200
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <CAPTjJmo+wLhY=LPOC1jGEegu8+=pEgTrqef3=kEj47NbDuhZNg@mail.gmail.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
 <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
 <85pp01mhjn.fsf@benfinney.id.au>
 <CAPTjJmo+wLhY=LPOC1jGEegu8+=pEgTrqef3=kEj47NbDuhZNg@mail.gmail.com>
Message-ID: <n0ncs8$jtb$1@ger.gmane.org>

On 27.10.15 04:26, Chris Angelico wrote:
> Fortunately, it's easy to write a function that validates a regex.
>
> def is_regex(s):
>      try: re.compile(s)
>      except re.error: return False
>      return True

re.compile() also can raise OverflowError ('.{,9999999999}') and 
ValueError ('(?ua)').



From storchaka at gmail.com  Tue Oct 27 13:45:37 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 27 Oct 2015 19:45:37 +0200
Subject: [Python-ideas] Specifying constants for functions
Message-ID: <n0od81$pt4$1@ger.gmane.org>

There is known trick to optimize a function:

     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
         ...

It has a side effect: change function's signature. Would be nice to have 
a way to set function's local variables at creation time without 
affecting a signature.

Possible syntax (I'm not sure what is better):

1. Similar to "global" and "nonlocal" declarations with optional 
initializer.

     def foo(x, y=0):
         const len
         const pack=struct.pack, maxsize=1<<BPF
         ...

2. Same as 1, but using "as" instead of "=".

     def foo(x, y=0):
         uses len, struct.pack as pack
         uses 1<<BPF as maxsize
         ...

3. Declaration is moved to function header. The keyword "given" is 
inspired by PEP 3150.

     def foo(x, y=0) given len=len, pack=struct.pack, maxsize=1<<BPF:
         ...

4. Declaration is moved out of the function. The advantage is that bound 
names can be used to evaluate default values for actual parameters (it 
is useful to implement sentinel default value), and all expression are 
evaluated in natural order.

     using len, struct.pack as pack, 1<<BPF as maxsize:
         def foo(x, y=0):
             ...

5. The least wordy syntax. No new keyword needed.

     def foo(x, y=0; len=len, pack=struct.pack, maxsize=1<<BPF):
         ...

All above examples would be roughly equivalent to the following code:

     def create(len=len, pack=struct.pack, maxsize=1<<BPF):
         def foo(x, y=0):
             ...
         return foo
     tmp = create()
     def foo(x, y=0):
         pass
     update_wrapper(tmp, foo)
     foo = tmp
     del create, tmp

This feature is rather ideologically opposite to Victor's approach.


From yselivanov.ml at gmail.com  Tue Oct 27 13:55:30 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Tue, 27 Oct 2015 13:55:30 -0400
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <n0od81$pt4$1@ger.gmane.org>
References: <n0od81$pt4$1@ger.gmane.org>
Message-ID: <562FBA92.1060300@gmail.com>

Serhiy,

On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:
> There is known trick to optimize a function:
>
>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>         ...
>
> It has a side effect: change function's signature. Would be nice to 
> have a way to set function's local variables at creation time without 
> affecting a signature. 

I see this a lot in all kinds of code.  In my experience it doesn't 
actually speed things up in a measurable way.

Is the below code really much slower?

     def foo(x, y=0):
         pack=struct.pack
         maxsize=1<<BPF
         #CODE

If the #CODE is a tight long-running loop - then no, because the loop 
will probably run much longer than an extra attribute lookup + one extra 
bit shift on each "foo()" call.  And if there is no tight loop - then 
you won't probably notice those optimizations anyways.

I think that adding a "const" statement deserves some discussion, but 
not from the standpoint of micro-optimizations.

Thanks,
Yury

From storchaka at gmail.com  Tue Oct 27 14:01:32 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 27 Oct 2015 20:01:32 +0200
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <n0od81$pt4$1@ger.gmane.org>
References: <n0od81$pt4$1@ger.gmane.org>
Message-ID: <n0oe6h$fl3$1@ger.gmane.org>

On 27.10.15 19:45, Serhiy Storchaka wrote:
> There is known trick to optimize a function:
>
>      def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>          ...
>
> It has a side effect: change function's signature. Would be nice to have
> a way to set function's local variables at creation time without
> affecting a signature.
>
> Possible syntax (I'm not sure what is better):
>
> 1. Similar to "global" and "nonlocal" declarations with optional
> initializer.
>
>      def foo(x, y=0):
>          const len
>          const pack=struct.pack, maxsize=1<<BPF
>          ...
>
> 2. Same as 1, but using "as" instead of "=".
>
>      def foo(x, y=0):
>          uses len, struct.pack as pack
>          uses 1<<BPF as maxsize
>          ...
>
> 3. Declaration is moved to function header. The keyword "given" is
> inspired by PEP 3150.
>
>      def foo(x, y=0) given len=len, pack=struct.pack, maxsize=1<<BPF:
>          ...
>
> 4. Declaration is moved out of the function. The advantage is that bound
> names can be used to evaluate default values for actual parameters (it
> is useful to implement sentinel default value), and all expression are
> evaluated in natural order.
>
>      using len, struct.pack as pack, 1<<BPF as maxsize:
>          def foo(x, y=0):
>              ...
>
> 5. The least wordy syntax. No new keyword needed.
>
>      def foo(x, y=0; len=len, pack=struct.pack, maxsize=1<<BPF):
>          ...
>
> All above examples would be roughly equivalent to the following code:
>
>      def create(len=len, pack=struct.pack, maxsize=1<<BPF):
>          def foo(x, y=0):
>              ...
>          return foo
>      tmp = create()
>      def foo(x, y=0):
>          pass
>      update_wrapper(tmp, foo)
>      foo = tmp
>      del create, tmp
>
> This feature is rather ideologically opposite to Victor's approach.

Oh, only after sending my message I had read Scott's message about the 
asconstants decorator, that does just this optimization [1]. With this 
decorator above example can be written as:

     from codetransformer.transformers import asconstants
     @asconstants(len=len, pack=struct.pack, maxsize=1<<BPF)
     def foo(x, y=0):
         ...

But it depends on bytecode implementation details. I wondering if it is 
worth to support this feature by syntax.

[1] http://permalink.gmane.org/gmane.comp.python.ideas/36958



From guido at python.org  Tue Oct 27 14:18:56 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 27 Oct 2015 11:18:56 -0700
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <562FBA92.1060300@gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
Message-ID: <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>

IIRC it's an old micro-optimization; this was a common idiom at Zope. But I
think it's way overused -- people believe it works so they do it all the
time, even for code that's not performance sensitive, just because it's
become a habit. (Like "register" in C in the '80s.)

On Tue, Oct 27, 2015 at 10:55 AM, Yury Selivanov <yselivanov.ml at gmail.com>
wrote:

> Serhiy,
>
> On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:
>
>> There is known trick to optimize a function:
>>
>>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>>         ...
>>
>> It has a side effect: change function's signature. Would be nice to have
>> a way to set function's local variables at creation time without affecting
>> a signature.
>>
>
> I see this a lot in all kinds of code.  In my experience it doesn't
> actually speed things up in a measurable way.
>
> Is the below code really much slower?
>
>     def foo(x, y=0):
>         pack=struct.pack
>         maxsize=1<<BPF
>         #CODE
>
> If the #CODE is a tight long-running loop - then no, because the loop will
> probably run much longer than an extra attribute lookup + one extra bit
> shift on each "foo()" call.  And if there is no tight loop - then you won't
> probably notice those optimizations anyways.
>
> I think that adding a "const" statement deserves some discussion, but not
> from the standpoint of micro-optimizations.
>
> Thanks,
> Yury
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151027/aa9792f4/attachment.html>

From srkunze at mail.de  Tue Oct 27 14:35:24 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 27 Oct 2015 19:35:24 +0100
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
Message-ID: <562FC3EC.3050706@mail.de>

I totally agree here.

On 27.10.2015 19:18, Guido van Rossum wrote:
> IIRC it's an old micro-optimization; this was a common idiom at Zope. 
> But I think it's way overused -- people believe it works so they do it 
> all the time, even for code that's not performance sensitive, just 
> because it's become a habit. (Like "register" in C in the '80s.)
>
> On Tue, Oct 27, 2015 at 10:55 AM, Yury Selivanov 
> <yselivanov.ml at gmail.com <mailto:yselivanov.ml at gmail.com>> wrote:
>
>     Serhiy,
>
>     On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:
>
>         There is known trick to optimize a function:
>
>             def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>                 ...
>
>         It has a side effect: change function's signature. Would be
>         nice to have a way to set function's local variables at
>         creation time without affecting a signature.
>
>
>     I see this a lot in all kinds of code.  In my experience it
>     doesn't actually speed things up in a measurable way.
>
>     Is the below code really much slower?
>
>         def foo(x, y=0):
>             pack=struct.pack
>             maxsize=1<<BPF
>             #CODE
>
>     If the #CODE is a tight long-running loop - then no, because the
>     loop will probably run much longer than an extra attribute lookup
>     + one extra bit shift on each "foo()" call.  And if there is no
>     tight loop - then you won't probably notice those optimizations
>     anyways.
>
>     I think that adding a "const" statement deserves some discussion,
>     but not from the standpoint of micro-optimizations.
>
>     Thanks,
>     Yury
>
>     _______________________________________________
>     Python-ideas mailing list
>     Python-ideas at python.org <mailto:Python-ideas at python.org>
>     https://mail.python.org/mailman/listinfo/python-ideas
>     Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151027/5ce83c1d/attachment-0001.html>

From storchaka at gmail.com  Tue Oct 27 14:36:16 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 27 Oct 2015 20:36:16 +0200
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <562FBA92.1060300@gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
Message-ID: <n0og71$ht3$1@ger.gmane.org>

On 27.10.15 19:55, Yury Selivanov wrote:
> I see this a lot in all kinds of code.  In my experience it doesn't
> actually speed things up in a measurable way.
>
> I think that adding a "const" statement deserves some discussion, but
> not from the standpoint of micro-optimizations.

Agree, we slowly gots rid of such microoptimizations. Seems that 
builtins and globals lookup is much faster in current CPython than was 
when such microoptimizations were added.

Yet one application of this trick is binding globals and module's 
members in __del__ methods and functions that can be called at shutdown 
time after module's content cleared. But this is very limited application.


From breamoreboy at yahoo.co.uk  Tue Oct 27 14:51:42 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 27 Oct 2015 18:51:42 +0000
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <562FBA92.1060300@gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
Message-ID: <n0oh41$1uh$1@ger.gmane.org>

On 27/10/2015 17:55, Yury Selivanov wrote:
> Serhiy,
>
> On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:
>> There is known trick to optimize a function:
>>
>>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>>         ...
>>
>> It has a side effect: change function's signature. Would be nice to
>> have a way to set function's local variables at creation time without
>> affecting a signature.
>
> I see this a lot in all kinds of code.  In my experience it doesn't
> actually speed things up in a measurable way.
>
> Is the below code really much slower?
>
>      def foo(x, y=0):
>          pack=struct.pack
>          maxsize=1<<BPF
>          #CODE
>
> If the #CODE is a tight long-running loop - then no, because the loop
> will probably run much longer than an extra attribute lookup + one extra
> bit shift on each "foo()" call.  And if there is no tight loop - then
> you won't probably notice those optimizations anyways.
>
> I think that adding a "const" statement deserves some discussion, but
> not from the standpoint of micro-optimizations.
>
> Thanks,
> Yury
>

 From my very naive perspective I'd have thought that the only real 
difference between the two implementations is that Yury's has the 
optimization hard coded within the function body, while Serhiy's allows 
you to override the hard coded defaults at run time.  Am I hot, warm, 
tepid, cold or approaching 0 Kelvin?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From bruce at leban.us  Tue Oct 27 15:02:25 2015
From: bruce at leban.us (Bruce Leban)
Date: Tue, 27 Oct 2015 12:02:25 -0700
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
Message-ID: <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>

On Tue, Oct 27, 2015 at 11:18 AM, Guido van Rossum <guido at python.org> wrote:

> IIRC it's an old micro-optimization; this was a common idiom at Zope. But
> I think it's way overused -- people believe it works so they do it all the
> time, even for code that's not performance sensitive, just because it's
> become a habit. (Like "register" in C in the '80s.)
>
> On Tue, Oct 27, 2015 at 10:55 AM, Yury Selivanov <yselivanov.ml at gmail.com>
> wrote:
>
>> Serhiy,
>>
>> On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:
>>
>>> There is known trick to optimize a function:
>>>
>>>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>>>         ...
>>
>>
It's not a micro-optimization in this case:

def foo(x, SENTINEL=object()):
    ...


I don't like mangling function signatures to do this. What I really want is
the equivalent of C's static here:

def foo(x):
    static SENTINEL = object()
    ...


This has two important semantics: (1) the scope of the SENTINEL variable is
limited to the function foo; (2) it is only initialized once.

Hosting the value out of the function into a decorator like @asconstants is
fine.

--- Bruce
Check out my new puzzle book: http://J.mp/ingToConclusions
<http://j.mp/ingToConclusions>
Get it free here: http://J.mp/ingToConclusionsFree
<http://j.mp/ingToConclusionsFree> (available on iOS)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151027/868bbc29/attachment.html>

From ian.g.kelly at gmail.com  Tue Oct 27 14:59:18 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 27 Oct 2015 12:59:18 -0600
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <n0oh41$1uh$1@ger.gmane.org>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <n0oh41$1uh$1@ger.gmane.org>
Message-ID: <CALwzidmGj1=90rJHey4zLKkNADm5Y_K9fTujDv+53L=WMA4wzw@mail.gmail.com>

On Tue, Oct 27, 2015 at 12:51 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 27/10/2015 17:55, Yury Selivanov wrote:
>>
>> Serhiy,
>>
>> On 2015-10-27 1:45 PM, Serhiy Storchaka wrote:
>>>
>>> There is known trick to optimize a function:
>>>
>>>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>>>         ...
>>>
>>> It has a side effect: change function's signature. Would be nice to
>>> have a way to set function's local variables at creation time without
>>> affecting a signature.
>>
>>
>> I see this a lot in all kinds of code.  In my experience it doesn't
>> actually speed things up in a measurable way.
>>
>> Is the below code really much slower?
>>
>>      def foo(x, y=0):
>>          pack=struct.pack
>>          maxsize=1<<BPF
>>          #CODE
>>
>> If the #CODE is a tight long-running loop - then no, because the loop
>> will probably run much longer than an extra attribute lookup + one extra
>> bit shift on each "foo()" call.  And if there is no tight loop - then
>> you won't probably notice those optimizations anyways.
>>
>> I think that adding a "const" statement deserves some discussion, but
>> not from the standpoint of micro-optimizations.
>>
>> Thanks,
>> Yury
>>
>
> From my very naive perspective I'd have thought that the only real
> difference between the two implementations is that Yury's has the
> optimization hard coded within the function body, while Serhiy's allows you
> to override the hard coded defaults at run time.  Am I hot, warm, tepid,
> cold or approaching 0 Kelvin?

Yury's code has to look up the global value of struct, and then get
its pack attribute, every time the function is called. Serhiy's code
only does this once, when the function is created, and when it is
called the local value is loaded directly from the function defaults,
which amounts to a single tuple lookup rather than two sequential dict
lookups.

From Andy.Henshaw at gtri.gatech.edu  Tue Oct 27 15:16:31 2015
From: Andy.Henshaw at gtri.gatech.edu (Henshaw, Andy)
Date: Tue, 27 Oct 2015 19:16:31 +0000
Subject: [Python-ideas] PEP 8: raw strings & regular expressions
In-Reply-To: <CAPTjJmo+wLhY=LPOC1jGEegu8+=pEgTrqef3=kEj47NbDuhZNg@mail.gmail.com>
References: <562848C2.4090509@gmail.com>
 <20151022171703.797395e1@anarchist.wooz.org> <562969B4.5080406@gmail.com>
 <562A7F1D.90108@nedbatchelder.com> <562DD4F3.5000901@sdamon.com>
 <562E0F92.7050400@nedbatchelder.com>
 <72B2B3B1-3F0D-4CB3-9B64-5B3F7114CA2C@yahoo.com>
 <A9C0869B-EB01-4585-8479-8C95F8E457F4@yahoo.com>
 <85pp01mhjn.fsf@benfinney.id.au>
 <CAPTjJmo+wLhY=LPOC1jGEegu8+=pEgTrqef3=kEj47NbDuhZNg@mail.gmail.com>
Message-ID: <2d7acb865d5b4d2ca18ee9ac31b5e4eb@apatlisdmail01.core.gtri.org>

On Monday, October 26, 2015 10:26 PM, Chris Angelico wrote
> 
> On Tue, Oct 27, 2015 at 8:44 AM, Ben Finney
> <ben+python at benfinney.id.au> wrote:
> > You seem to be seeking something else: a pattern that matches all
> > valid regex patterns, *and* will never match any string that is not a
> > valid regex pattern. The latter is rather more difficult.
> 
> "Rather more difficult" may be an understatement...

Indeed, in fact I think that G?del's Incompleteness Theorem applies here.

Andy Henshaw




From yselivanov.ml at gmail.com  Tue Oct 27 15:47:02 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Tue, 27 Oct 2015 15:47:02 -0400
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
Message-ID: <562FD4B6.7060105@gmail.com>



On 2015-10-27 3:02 PM, Bruce Leban wrote:
> It's not a micro-optimization in this case:
>
>     def foo(x, SENTINEL=object()):
>         ...
>
>
> I don't like mangling function signatures to do this. What I really 
> want is the equivalent of C's static here:
>
>     def foo(x):
>     static SENTINEL = object()
>         ...
>
>

Why not

SENTINEL=object()
def foo(x):
     ...

?

That's how I do it usually.

Yury

From rymg19 at gmail.com  Tue Oct 27 16:15:24 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 27 Oct 2015 15:15:24 -0500
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <n0od81$pt4$1@ger.gmane.org>
References: <n0od81$pt4$1@ger.gmane.org>
Message-ID: <CAO41-mMH0xMiF1eKEoRumJxWfhk+p4J-Pmki+ZTiLY24xRB_Rg@mail.gmail.com>

Making a decorator is easy:


def opt(**kw):
    def func(f):
        globals().update(dis.opmap) # Someone's going to kill me for this...
        code = list(f.__code__.co_code)
        offs = 0
        varnames = list(f.__code__.co_varnames)
        nlocals = f.__code__.co_nlocals
        names = list(f.__code__.co_names)
        globls = f.__globals__
        mapping = {}
        anon_count = 0
        for k, v in kw.items():
            if k in names:
                try:
                    i = names.index(k)
                except ValueError:
                    raise ValueError('variable %r must be a global' % k)
from None
                anon_var = '$__varopt__%d$' % anon_count
                anon_count += 1
                names[i] = anon_var
                globls[anon_var] = v
                assert k not in varnames, '%r is both global and local' % k
                varnames.append(k)
                nlocals += 1
                mapping[i] = nlocals-1
                code[:0] = [
                    LOAD_GLOBAL, i, 0,
                    STORE_FAST, nlocals-1, 0,
                ]
                offs += 6
            else:
                raise ValueError('variable %r is not a global' % k)

        i = offs
        while i < len(code):
            if code[i] in dis.hasjabs:
                code[i+1] += offs
            elif code[i] in (LOAD_GLOBAL, STORE_GLOBAL) and code[i+1] in
mapping:
                code[i] = LOAD_FAST if code[i] == LOAD_GLOBAL else
STORE_FAST
                code[i+1] = mapping[code[i+1]]
            i += 3 if code[i] > dis.HAVE_ARGUMENT else 1

        rescode = types.CodeType(f.__code__.co_argcount,
                                 f.__code__.co_kwonlyargcount, nlocals,
                                 f.__code__.co_stacksize,
f.__code__.co_flags,
                                 bytes(code), f.__code__.co_consts,
                                 tuple(names), tuple(varnames),
                                 f.__code__.co_filename, f.__code__.co_name,
                                 f.__code__.co_firstlineno,
                                 f.__code__.co_lnotab,
f.__code__.co_freevars,
                                 f.__code__.co_cellvars)
        return types.FunctionType(rescode, globls,
                                  f.__name__, f.__defaults__, f.__closure__)
    return func


On Tue, Oct 27, 2015 at 12:45 PM, Serhiy Storchaka <storchaka at gmail.com>
wrote:

> There is known trick to optimize a function:
>
>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>         ...
>
> It has a side effect: change function's signature. Would be nice to have a
> way to set function's local variables at creation time without affecting a
> signature.
>
> Possible syntax (I'm not sure what is better):
>
> 1. Similar to "global" and "nonlocal" declarations with optional
> initializer.
>
>     def foo(x, y=0):
>         const len
>         const pack=struct.pack, maxsize=1<<BPF
>         ...
>
> 2. Same as 1, but using "as" instead of "=".
>
>     def foo(x, y=0):
>         uses len, struct.pack as pack
>         uses 1<<BPF as maxsize
>         ...
>
> 3. Declaration is moved to function header. The keyword "given" is
> inspired by PEP 3150.
>
>     def foo(x, y=0) given len=len, pack=struct.pack, maxsize=1<<BPF:
>         ...
>
> 4. Declaration is moved out of the function. The advantage is that bound
> names can be used to evaluate default values for actual parameters (it is
> useful to implement sentinel default value), and all expression are
> evaluated in natural order.
>
>     using len, struct.pack as pack, 1<<BPF as maxsize:
>         def foo(x, y=0):
>             ...
>
> 5. The least wordy syntax. No new keyword needed.
>
>     def foo(x, y=0; len=len, pack=struct.pack, maxsize=1<<BPF):
>         ...
>
> All above examples would be roughly equivalent to the following code:
>
>     def create(len=len, pack=struct.pack, maxsize=1<<BPF):
>         def foo(x, y=0):
>             ...
>         return foo
>     tmp = create()
>     def foo(x, y=0):
>         pass
>     update_wrapper(tmp, foo)
>     foo = tmp
>     del create, tmp
>
> This feature is rather ideologically opposite to Victor's approach.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something?s wrong.
http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151027/4a089f05/attachment.html>

From Nikolaus at rath.org  Tue Oct 27 16:22:22 2015
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Tue, 27 Oct 2015 13:22:22 -0700
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <562FD4B6.7060105@gmail.com> (Yury Selivanov's message of "Tue,
 27 Oct 2015 15:47:02 -0400")
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
 <562FD4B6.7060105@gmail.com>
Message-ID: <87io5sxds1.fsf@thinkpad.rath.org>

On Oct 27 2015, Yury Selivanov <yselivanov.ml-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
> On 2015-10-27 3:02 PM, Bruce Leban wrote:
>> It's not a micro-optimization in this case:
>>
>>     def foo(x, SENTINEL=object()):
>>         ...
>>
>>
>> I don't like mangling function signatures to do this. What I really
>> want is the equivalent of C's static here:
>>
>>     def foo(x):
>>     static SENTINEL = object()
>>         ...
>>
>>
>
> Why not
>
> SENTINEL=object()
> def foo(x):
>     ...
>
> ?

Because it leaks into the enclosing scope.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From yselivanov.ml at gmail.com  Tue Oct 27 16:30:41 2015
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Tue, 27 Oct 2015 16:30:41 -0400
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <87io5sxds1.fsf@thinkpad.rath.org>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
 <562FD4B6.7060105@gmail.com> <87io5sxds1.fsf@thinkpad.rath.org>
Message-ID: <562FDEF1.8050303@gmail.com>



On 2015-10-27 4:22 PM, Nikolaus Rath wrote:
> Because it leaks into the enclosing scope.

Leaks what?
Name?  Name it _SENTINEL then.

Yury

From srkunze at mail.de  Tue Oct 27 17:50:16 2015
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 27 Oct 2015 22:50:16 +0100
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <562FDEF1.8050303@gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
 <562FD4B6.7060105@gmail.com> <87io5sxds1.fsf@thinkpad.rath.org>
 <562FDEF1.8050303@gmail.com>
Message-ID: <562FF198.5080708@mail.de>

On 27.10.2015 21:30, Yury Selivanov wrote:
> On 2015-10-27 4:22 PM, Nikolaus Rath wrote:
>> Because it leaks into the enclosing scope.
>
> Leaks what?
> Name?  Name it _SENTINEL then.

I think because it's missing the relationship with foo. So, better do this

foo._sentinel = object()

It's coupled to foo and moved out of the outer scope.

Best,
Sven

From steve at pearwood.info  Tue Oct 27 19:36:50 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 28 Oct 2015 10:36:50 +1100
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <562FDEF1.8050303@gmail.com>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
 <562FD4B6.7060105@gmail.com> <87io5sxds1.fsf@thinkpad.rath.org>
 <562FDEF1.8050303@gmail.com>
Message-ID: <20151027233650.GB3813@ando.pearwood.info>

On Tue, Oct 27, 2015 at 04:30:41PM -0400, Yury Selivanov wrote:
> 
> 
> On 2015-10-27 4:22 PM, Nikolaus Rath wrote:
> >Because it leaks into the enclosing scope.
> 
> Leaks what?
> Name?  Name it _SENTINEL then.

It is still in the enclosing scope.

Bruce is right that what we really want is something like "static". 
Lacking that feature, one work-around is "make it a global variable, and 
give it a leading underscore", but that's just a work-around.

Consider:

_SENTINEL = object()

# Later:
def spam(): ...
def eggs(): ...
def cheese(): ...
def aardvark(): ...


As the module user, you can see the leading underscore in _SENTINEL 
and immediately forget all about it. But as the module maintainer, you 
cannot ignore _SENTINEL. Leading underscore or not, it is still part of 
the implementation, and maintenance is all about the implementation.

Which of the functions uses _SENTINEL? You can assume that *at least* 
one function uses it, but it may be more. Is it safe to rename it? 
Change it's value? To the maintainer, _SENTINEL is just another global 
variable, with all the disadvantages that has.

Better to put it inside the function, so the maintainer knows that it is 
local to the function:

def spam():
    SENTINEL = object()
    ...


That's not bad, but what if the value is something harder to calculate?

def spam():
    SENTINEL = next_prime_number(2**512)
    ...


At the moment, Python has an obvious way to calculate a value once only 
(make it a module-level global), and there is an obvious way to make a 
value local to a function (put it inside the function body). But there's 
no obvious way to do both together:

"Calculate this thing once only, AND make it local to this scope."

The unobvious and ugly way is to put the calculation in the function 
declaration as a default value:

def spam(SENTINEL=next_prime_number(2**512)):
    ...


which complicates the function signature and risks errors if the caller 
accidentally calls the function with too many arguments.


-- 
Steve

From steve at pearwood.info  Tue Oct 27 19:44:18 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 28 Oct 2015 10:44:18 +1100
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <CAO41-mMH0xMiF1eKEoRumJxWfhk+p4J-Pmki+ZTiLY24xRB_Rg@mail.gmail.com>
References: <n0od81$pt4$1@ger.gmane.org>
 <CAO41-mMH0xMiF1eKEoRumJxWfhk+p4J-Pmki+ZTiLY24xRB_Rg@mail.gmail.com>
Message-ID: <20151027234418.GC3813@ando.pearwood.info>

On Tue, Oct 27, 2015 at 03:15:24PM -0500, Ryan Gonzalez wrote:

> Making a decorator is easy:
[snip 54 lines of hacking the internals of function and code objects]

I like the way you said that with a straight face.

:-)


-- 
Steve

From guido at python.org  Tue Oct 27 19:47:03 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 27 Oct 2015 16:47:03 -0700
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <20151027233650.GB3813@ando.pearwood.info>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
 <562FD4B6.7060105@gmail.com> <87io5sxds1.fsf@thinkpad.rath.org>
 <562FDEF1.8050303@gmail.com> <20151027233650.GB3813@ando.pearwood.info>
Message-ID: <CAP7+vJ+L7XFzPzN-YSg8pVwC7FzfxC2GKfRzd2FxTX8T=-5j1A@mail.gmail.com>

On Tue, Oct 27, 2015 at 4:36 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Tue, Oct 27, 2015 at 04:30:41PM -0400, Yury Selivanov wrote:
> >
> >
> > On 2015-10-27 4:22 PM, Nikolaus Rath wrote:
> > >Because it leaks into the enclosing scope.
> >
> > Leaks what?
> > Name?  Name it _SENTINEL then.
>
> It is still in the enclosing scope.
>
> Bruce is right that what we really want is something like "static".
> Lacking that feature, one work-around is "make it a global variable, and
> give it a leading underscore", but that's just a work-around.
>
> Consider:
>
> _SENTINEL = object()
>
> # Later:
> def spam(): ...
> def eggs(): ...
> def cheese(): ...
> def aardvark(): ...
>
>
> As the module user, you can see the leading underscore in _SENTINEL
> and immediately forget all about it. But as the module maintainer, you
> cannot ignore _SENTINEL. Leading underscore or not, it is still part of
> the implementation, and maintenance is all about the implementation.
>
> Which of the functions uses _SENTINEL? You can assume that *at least*
> one function uses it, but it may be more. Is it safe to rename it?
> Change it's value? To the maintainer, _SENTINEL is just another global
> variable, with all the disadvantages that has.
>
> Better to put it inside the function, so the maintainer knows that it is
> local to the function:
>
> def spam():
>     SENTINEL = object()
>     ...
>
>
> That's not bad, but what if the value is something harder to calculate?
>
> def spam():
>     SENTINEL = next_prime_number(2**512)
>     ...
>
>
> At the moment, Python has an obvious way to calculate a value once only
> (make it a module-level global), and there is an obvious way to make a
> value local to a function (put it inside the function body). But there's
> no obvious way to do both together:
>
> "Calculate this thing once only, AND make it local to this scope."
>
> The unobvious and ugly way is to put the calculation in the function
> declaration as a default value:
>
> def spam(SENTINEL=next_prime_number(2**512)):
>     ...
>
>
> which complicates the function signature and risks errors if the caller
> accidentally calls the function with too many arguments.


Python intentionally doesn't have this feature, and the argument you
present sounds pretty weak (compared to the compelling argument *against*
abusing the default argument syntax -- which also has a maintenance cost,
as you have to explain it over and over).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151027/32ecb08f/attachment.html>

From cory at lukasa.co.uk  Tue Oct 27 20:44:47 2015
From: cory at lukasa.co.uk (Cory Benfield)
Date: Wed, 28 Oct 2015 09:44:47 +0900
Subject: [Python-ideas] ssl module: validated certificate chain
Message-ID: <F165D0E9-61C2-4044-9A86-EEA7AE6CFFA0@lukasa.co.uk>

All,

I?m currently working on adding support for HPKP to the Requests and urllib3 modules. HPKP (HTTP Public Key Pinning), specified in RFC 7469, is an extension to HTTP that allows a web server to specify a whitelist of public keys that are valid for TLS certificates on that domain. This prevents a rogue certificate authority from issuing a certificate that would be trusted by a browser and would allow a man-in-the-middle attack on a domain (as happened to Google in 2013[0]).

Right now, the draft version of the support I have will only work when you use PyOpenSSL for your TLS needs, not the standard library. This is because to get HPKP to work I need access to the validated certificate chain: that is, the certificate chain that OpenSSL has built and validated for the TLS connection. I also need to be able to work with those certificates in order to extract their public keys. The standard library?s ssl module does not expose any of this functionality.

I want to get an idea of whether people believe it should be possible to access this functionality in the standard library. It would require the following things from the standard library:

1. The ability to access the validated certificate chain. This requires saving off the certificate each time the OpenSSL verify callback is called. This is an easy enough change to make.
2. The ability to extract the public key from the saved certificates. This could be done by extending the logic used for getpeercert() to provide a DER-encoded ASN.1 representation of the public key in the dictionary, and then using that representation for each cert in the peer cert chain.

The motivation for making this available in the standard library would be pip. Right now python.org and all its subdomains (including pypi.python.org) are HPKP-enabled. Making this support available in the standard library would ensure that all pip installations are safe from man-in-the-middle attacks on its packaging infrastructure. Without it, a number of third-party packages would be required to add this security.

I?m happy to do the work required to provide this functionality, but I?d only like to start work if people believe there?s a likelihood of getting it merged.

Thanks,

Cory


[0]: https://nakedsecurity.sophos.com/2013/01/08/the-turktrust-ssl-certificate-fiasco-what-happened-and-what-happens-next/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151028/30eb2b61/attachment-0001.sig>

From tjreedy at udel.edu  Tue Oct 27 20:57:31 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 27 Oct 2015 20:57:31 -0400
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <20151027233650.GB3813@ando.pearwood.info>
References: <n0od81$pt4$1@ger.gmane.org> <562FBA92.1060300@gmail.com>
 <CAP7+vJK+dJh7mFpRPdVm6rAZc-iW1Rg8vDsB_72e7qFD=yiSJw@mail.gmail.com>
 <CAGu0Antc6jbTna739wMitfpE75j--0=eS94wd8qUSPoLNUcbqw@mail.gmail.com>
 <562FD4B6.7060105@gmail.com> <87io5sxds1.fsf@thinkpad.rath.org>
 <562FDEF1.8050303@gmail.com> <20151027233650.GB3813@ando.pearwood.info>
Message-ID: <n0p6ht$np9$1@ger.gmane.org>

On 10/27/2015 7:36 PM, Steven D'Aprano wrote:

> The unobvious and ugly way is to put the calculation in the function
> declaration as a default value:
>
> def spam(SENTINEL=next_prime_number(2**512)):
>      ...
>
>
> which complicates the function signature and risks errors if the caller
> accidentally calls the function with too many arguments.

Making constant names keyword only avoids the 'too many arguments' problem.

 >>> def spam(*, _SETINAL=object): pass

 >>> spam(1)
Traceback (most recent call last):
   File "<pyshell#3>", line 1, in <module>
     spam(1)
TypeError: spam() takes 0 positional arguments but 1 was given

Writing 'spam(_SETINAL=3)' would not be an accident ;-)

-- 
Terry Jan Reedy


From victor.stinner at gmail.com  Tue Oct 27 22:48:41 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 28 Oct 2015 11:48:41 +0900
Subject: [Python-ideas] Specifying constants for functions
In-Reply-To: <n0od81$pt4$1@ger.gmane.org>
References: <n0od81$pt4$1@ger.gmane.org>
Message-ID: <CAMpsgwbdJFoJHGpBR5KDd_Lu138Qnm_PFSNyvwWKeNqt-nZe_A@mail.gmail.com>

Hi,

2015-10-28 2:45 GMT+09:00 Serhiy Storchaka <storchaka at gmail.com>:
> There is known trick to optimize a function:
>
>     def foo(x, y=0, len=len, pack=struct.pack, maxsize=1<<BPF):
>         ...

Yeah, it can show a speedup on a micro-benchmark. Probably not a macro
benchmark. As it was said in other answers, this hack is also abused
for bad reasons.

This hack is mainly used in the stdlib to keep symbols alive during
Python shutdown to be able to cleanup properly objects. Just one
example from Lib/subprocess.py of Python 3.6: "def __del__(self,
_maxsize=sys.maxsize):". I guess that sys.maxsize symbol is removed or
set to None during Python shutdown. So depending on the order in which
modules are cleared (subprocess,sys or sys,subprocess), the __del__()
method may fail or may not fail without the "_maxsize=sys.maxsize"
hack.

I would appreciate a syntax to not change the function signature, even
if this hack is mostly used in destructors and destructors must *not*
be called explicitly.

> This feature is rather ideologically opposite to Victor's approach.

I disagree, it's not incompatible with my FAT Python project. In some
cases, we may still see speedup if you combine copying globals to
locals and using FAT Python optimizations. My idea is more to optimize
code without having to modify manually the code to optimize it.

Using FAT Python, you can implement an optimizer producing code like:
---
import builtins

def f(data):
    lengths = []
    for item in data:
        lengths.append(len(item))
    return lengths

def f_copy_globals(data, _len=len):
    lengths = []
    for item in data:
        lengths.append("fast: %s" % _len(item))   # add "fast" to
ensure that we call the "fast" function
    return lengths

i = f.specialize(f_copy_globals)
f.add_dict_guard(i, builtins.__dict__, 'len')
f.add_dict_guard(i, globals(), 'len')

# test specialized function with "fast" _len local symbol
data = ["abc", list(range(5))]
print(f(data))

# test with a mocked len() builtin function
builtins.len = lambda obj: 10
data = ["abc", list(range(5))]
print(f(data))
---

Output:
---
['fast: 3', 'fast: 5']
[10, 10]
---

This optimization doesn't respect strictly Python semantic because the
len() builtin function can be modified during two loop iterations in a
different Python thread. In some cases, it can be worth to optimize
the function and doesn't respect stricly the Python semantic. As it
was discussed in the FAT Python thread, depending on your use case,
you may or may not allow some optimizations.

Note: This example doesn't work with my current implementation of FAT
Python, because f() and f_copy_globals() don't have the same default
values for parameters. You can test with "def f(data, _len=len):". I
have to modify FAT Python to support this example. There is also a bug
if the specialized function uses a free variable, but not the original
function. Again, it should enhance FAT Python to support this case.

Victor

From greg at krypto.org  Wed Oct 28 20:21:52 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 29 Oct 2015 00:21:52 +0000
Subject: [Python-ideas] ssl module: validated certificate chain
In-Reply-To: <F165D0E9-61C2-4044-9A86-EEA7AE6CFFA0@lukasa.co.uk>
References: <F165D0E9-61C2-4044-9A86-EEA7AE6CFFA0@lukasa.co.uk>
Message-ID: <CAGE7PN+yMq6yFpbnP-xB_jTRZ-5kb-5vhd4WSZR2S8sYY8BeFA@mail.gmail.com>

I'm in favor of this. I don't know if we'd ever land this kind of change
(new feature) in existing stable 2.7 or 3.4/3.5 releases, but for strictly
security related things like this we have done so in the past.  If nothing
else, there is no reason not to have this in 3.6.  It is good for the
future.

-gps

On Tue, Oct 27, 2015 at 5:45 PM Cory Benfield <cory at lukasa.co.uk> wrote:

> All,
>
> I?m currently working on adding support for HPKP to the Requests and
> urllib3 modules. HPKP (HTTP Public Key Pinning), specified in RFC 7469, is
> an extension to HTTP that allows a web server to specify a whitelist of
> public keys that are valid for TLS certificates on that domain. This
> prevents a rogue certificate authority from issuing a certificate that
> would be trusted by a browser and would allow a man-in-the-middle attack on
> a domain (as happened to Google in 2013[0]).
>
> Right now, the draft version of the support I have will only work when you
> use PyOpenSSL for your TLS needs, not the standard library. This is because
> to get HPKP to work I need access to the validated certificate chain: that
> is, the certificate chain that OpenSSL has built and validated for the TLS
> connection. I also need to be able to work with those certificates in order
> to extract their public keys. The standard library?s ssl module does not
> expose any of this functionality.
>
> I want to get an idea of whether people believe it should be possible to
> access this functionality in the standard library. It would require the
> following things from the standard library:
>
> 1. The ability to access the validated certificate chain. This requires
> saving off the certificate each time the OpenSSL verify callback is called.
> This is an easy enough change to make.
> 2. The ability to extract the public key from the saved certificates. This
> could be done by extending the logic used for getpeercert() to provide a
> DER-encoded ASN.1 representation of the public key in the dictionary, and
> then using that representation for each cert in the peer cert chain.
>
> The motivation for making this available in the standard library would be
> pip. Right now python.org and all its subdomains (including
> pypi.python.org) are HPKP-enabled. Making this support available in the
> standard library would ensure that all pip installations are safe from
> man-in-the-middle attacks on its packaging infrastructure. Without it, a
> number of third-party packages would be required to add this security.
>
> I?m happy to do the work required to provide this functionality, but I?d
> only like to start work if people believe there?s a likelihood of getting
> it merged.
>
> Thanks,
>
> Cory
>
>
> [0]:
> https://nakedsecurity.sophos.com/2013/01/08/the-turktrust-ssl-certificate-fiasco-what-happened-and-what-happens-next/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151029/5f5e710b/attachment.html>

From tjreedy at udel.edu  Fri Oct 30 21:09:00 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 30 Oct 2015 21:09:00 -0400
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
Message-ID: <n114bh$ih1$1@ger.gmane.org>

This idea results from issue of user files shadowing stdlib files on 
import.  There was a thread on pydev about this yesterday.  There is 
also an opposite issue of builtin modules shadowing user files.

The keyword module provides kwlist and iskeyword function.  One use of 
kwlist is used in some other stdlib modules and can be used by syntax 
highlighters (as in IDLE).  Kwlist is updated by the main function.

A module module would have at least liblist and islibmodule function. 
Liblist would contain all directories with __init__.py and all .py 
files.  (I don't think files within package directories should be 
included, as there is no direct shadowing problem.)  A python oriented 
editor could then warn on save requests "This name matches a stdlib name 
in /Lib. If you run python in this directory, you will not be able to 
import the stdlib module.  Continue?".

The module should also have binlist and isbinmodule for builtin modules. 
  (I do not know how to get such a list.  If necessary, an api could be 
added.)  An editor could than warn "This name matches a builtin stdlib 
name.  You will not be able to import this file.  Continue?".

-- 
Terry Jan Reedy


From guido at python.org  Fri Oct 30 21:19:54 2015
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Oct 2015 18:19:54 -0700
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <n114bh$ih1$1@ger.gmane.org>
References: <n114bh$ih1$1@ger.gmane.org>
Message-ID: <CAP7+vJ+zmEaJfyXNsBK2=fk42aTfVxA4o3tH7297_XEjM9jpkg@mail.gmail.com>

There's sys.builtin_module_names which returns the names of the hardcoded
builtin modules. Dynamically loaded modules can be found by searching
sys.path in the usual way -- importlib shoul know. I wonder if just asking
importlib whether it can locate a given module would be enough?

On Fri, Oct 30, 2015 at 6:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> This idea results from issue of user files shadowing stdlib files on
> import.  There was a thread on pydev about this yesterday.  There is also
> an opposite issue of builtin modules shadowing user files.
>
> The keyword module provides kwlist and iskeyword function.  One use of
> kwlist is used in some other stdlib modules and can be used by syntax
> highlighters (as in IDLE).  Kwlist is updated by the main function.
>
> A module module would have at least liblist and islibmodule function.
> Liblist would contain all directories with __init__.py and all .py files.
> (I don't think files within package directories should be included, as
> there is no direct shadowing problem.)  A python oriented editor could then
> warn on save requests "This name matches a stdlib name in /Lib. If you run
> python in this directory, you will not be able to import the stdlib
> module.  Continue?".
>
> The module should also have binlist and isbinmodule for builtin modules.
> (I do not know how to get such a list.  If necessary, an api could be
> added.)  An editor could than warn "This name matches a builtin stdlib
> name.  You will not be able to import this file.  Continue?".
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151030/124d7091/attachment.html>

From tjreedy at udel.edu  Sat Oct 31 01:56:33 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 31 Oct 2015 01:56:33 -0400
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <CAP7+vJ+zmEaJfyXNsBK2=fk42aTfVxA4o3tH7297_XEjM9jpkg@mail.gmail.com>
References: <n114bh$ih1$1@ger.gmane.org>
 <CAP7+vJ+zmEaJfyXNsBK2=fk42aTfVxA4o3tH7297_XEjM9jpkg@mail.gmail.com>
Message-ID: <n11l6m$q68$1@ger.gmane.org>

On 10/30/2015 9:19 PM, Guido van Rossum wrote:
> There's sys.builtin_module_names which returns the names of the
> hardcoded builtin modules.

Great.  With this solved, I opened an issue for IDLE.
https://bugs.python.org/issue25522

> Dynamically loaded modules can be found by
> searching sys.path in the usual way -- importlib shoul know. I wonder if
> just asking importlib whether it can locate a given module would be enough?

The default search order is stdlib builtins, local user files, /lib 
files, so the shadowing issue the opposite for builtin and /lib modules. 
  Hence a different message is needed.

> On Fri, Oct 30, 2015 at 6:09 PM, Terry Reedy
> <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:
>
>     This idea results from issue of user files shadowing stdlib files on
>     import.  There was a thread on pydev about this yesterday.  There is
>     also an opposite issue of builtin modules shadowing user files.
>
>     The keyword module provides kwlist and iskeyword function.  One use
>     of kwlist is used in some other stdlib modules and can be used by
>     syntax highlighters (as in IDLE).  Kwlist is updated by the main
>     function.
>
>     A module module would have at least liblist and islibmodule
>     function. Liblist would contain all directories with __init__.py and
>     all .py files.  (I don't think files within package directories
>     should be included, as there is no direct shadowing problem.)  A
>     python oriented editor could then warn on save requests "This name
>     matches a stdlib name in /Lib. If you run python in this directory,
>     you will not be able to import the stdlib module.  Continue?".
>
>     The module should also have binlist and isbinmodule for builtin
>     modules.  (I do not know how to get such a list.  If necessary, an
>     api could be added.)  An editor could than warn "This name matches a
>     builtin stdlib name.  You will not be able to import this file.
>     Continue?".
>
>     --
>     Terry Jan Reedy
>
>     _______________________________________________
>     Python-ideas mailing list
>     Python-ideas at python.org
>     <mailto:Python-ideas at python.org>
>     https://mail.python.org/mailman/listinfo/python-ideas
>     Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Terry Jan Reedy


From steve at pearwood.info  Sat Oct 31 04:14:51 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 31 Oct 2015 19:14:51 +1100
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <n114bh$ih1$1@ger.gmane.org>
References: <n114bh$ih1$1@ger.gmane.org>
Message-ID: <20151031081450.GD10946@ando.pearwood.info>

On Fri, Oct 30, 2015 at 09:09:00PM -0400, Terry Reedy wrote:

> A module module would have at least liblist and islibmodule function. 
> Liblist would contain all directories with __init__.py and all .py 
> files.  

`liblist` cannot usefully be a statically created list, since the 
availability of modules on the path is dynamic: the path can change, and 
files can be added or removed.

Nor can it be limited to .py files, since .pyc and other extensions can 
be imported.

I don't know what `islibmodule` is supposed to do. Surely all libraries 
are modules?


> (I don't think files within package directories should be 
> included, as there is no direct shadowing problem.)

Sounds like you want a private "fix_shadowing_of_modules_for_IDLE" 
module, rather than a general-purpose "module" module. For a general 
purpose "module" module, whether or not there is a shadowing problem is 
irrelevant.

By the way, it should be clear from the above that the name "module" is 
atrocious -- it makes it awkward to talk about the module, and it will 
itself be shadowed by anyone (especially beginners) who create a 
"module.py" file. "modtools" might be a better name.


> A python oriented 
> editor could then warn on save requests "This name matches a stdlib name 
> in /Lib. If you run python in this directory, you will not be able to 
> import the stdlib module.  Continue?".

I don't think this is much of a solution to the shadowing problem. For 
starters, it relies on people using a specific editor. It assumes that 
files won't be renamed or moved outside of the editor. It depends on the 
user actually paying attention and reading the error message, which 
beginners notoriously don't do. Many people will either blindly continue 
(and hence shadow), or dismiss the dialog and then get into a flap that 
they can't save their work.

> The module should also have binlist and isbinmodule for builtin modules. 
>  (I do not know how to get such a list.  If necessary, an api could be 
> added.)  An editor could than warn "This name matches a builtin stdlib 
> name.  You will not be able to import this file.  Continue?".

binlist is easy: sys.builtin_module_names.

I don't know what `isbinmodule` means either. Presumably you don't 
actually mean "is binary module", but "is builtin module". I believe 
that the canonical way to do that is hasattr(module, "__file__").


-- 
Steve

From brett at python.org  Sat Oct 31 12:48:45 2015
From: brett at python.org (Brett Cannon)
Date: Sat, 31 Oct 2015 16:48:45 +0000
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <n11l6m$q68$1@ger.gmane.org>
References: <n114bh$ih1$1@ger.gmane.org>
 <CAP7+vJ+zmEaJfyXNsBK2=fk42aTfVxA4o3tH7297_XEjM9jpkg@mail.gmail.com>
 <n11l6m$q68$1@ger.gmane.org>
Message-ID: <CAP1=2W7pQAsGqUj7MrZP=6pAOzLBTDQigyaowipFUOpbhxG5eQ@mail.gmail.com>

On Fri, 30 Oct 2015 at 22:57 Terry Reedy <tjreedy at udel.edu> wrote:

> On 10/30/2015 9:19 PM, Guido van Rossum wrote:
> > There's sys.builtin_module_names which returns the names of the
> > hardcoded builtin modules.
>
> Great.  With this solved, I opened an issue for IDLE.
> https://bugs.python.org/issue25522
>
> > Dynamically loaded modules can be found by
> > searching sys.path in the usual way -- importlib shoul know. I wonder if
> > just asking importlib whether it can locate a given module would be
> enough?
>
> The default search order is stdlib builtins, local user files, /lib
> files, so the shadowing issue the opposite for builtin and /lib modules.
>   Hence a different message is needed.
>

Quick and dirty way to use importlib is to get the location of the stdlib
(os.__file__ should work since it's hard-coded in the interpreter as
representing where the stdlib is) and then use importlib.find_spec() for a
module name to check if the file location in the spec has the same location
prefix as os or not (make sure you use absolute paths since it isn't
guaranteed if you don't execute site.py).

I've now seen this use case, the logging one, and the 2to3 module rename.
I'm starting to wonder if there some general solution that should get added
to the import machinery that  can serve these cases more easily than with
importers or __import__ overrides that can be tricky to get right.

-Brett


>
> > On Fri, Oct 30, 2015 at 6:09 PM, Terry Reedy
> > <tjreedy at udel.edu
> > <mailto:tjreedy at udel.edu>> wrote:
> >
> >     This idea results from issue of user files shadowing stdlib files on
> >     import.  There was a thread on pydev about this yesterday.  There is
> >     also an opposite issue of builtin modules shadowing user files.
> >
> >     The keyword module provides kwlist and iskeyword function.  One use
> >     of kwlist is used in some other stdlib modules and can be used by
> >     syntax highlighters (as in IDLE).  Kwlist is updated by the main
> >     function.
> >
> >     A module module would have at least liblist and islibmodule
> >     function. Liblist would contain all directories with __init__.py and
> >     all .py files.  (I don't think files within package directories
> >     should be included, as there is no direct shadowing problem.)  A
> >     python oriented editor could then warn on save requests "This name
> >     matches a stdlib name in /Lib. If you run python in this directory,
> >     you will not be able to import the stdlib module.  Continue?".
> >
> >     The module should also have binlist and isbinmodule for builtin
> >     modules.  (I do not know how to get such a list.  If necessary, an
> >     api could be added.)  An editor could than warn "This name matches a
> >     builtin stdlib name.  You will not be able to import this file.
> >     Continue?".
> >
> >     --
> >     Terry Jan Reedy
> >
> >     _______________________________________________
> >     Python-ideas mailing list
> >     Python-ideas at python.org
> >     <mailto:Python-ideas at python.org>
> >     https://mail.python.org/mailman/listinfo/python-ideas
> >     Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> >
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151031/d08921ab/attachment.html>

From steve.dower at python.org  Sat Oct 31 14:31:52 2015
From: steve.dower at python.org (Steve Dower)
Date: Sat, 31 Oct 2015 11:31:52 -0700
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <CAP1=2W7pQAsGqUj7MrZP=6pAOzLBTDQigyaowipFUOpbhxG5eQ@mail.gmail.com>
References: <n114bh$ih1$1@ger.gmane.org>
 <CAP7+vJ+zmEaJfyXNsBK2=fk42aTfVxA4o3tH7297_XEjM9jpkg@mail.gmail.com>
 <n11l6m$q68$1@ger.gmane.org>
 <CAP1=2W7pQAsGqUj7MrZP=6pAOzLBTDQigyaowipFUOpbhxG5eQ@mail.gmail.com>
Message-ID: <E1ZsaxV-0006p7-8h@se2-syd.hostedmail.net.au>

Could we add a context manager to importlib (or perhaps site or sys) to temporarily disable imports from non-standard paths? I don't see any safe way to change the default behavior, but no reason we can't make it easy for applications to self-isolate.

Top-posted from my Windows Phone

-----Original Message-----
From: "Brett Cannon" <brett at python.org>
Sent: ?10/?31/?2015 9:50
To: "Terry Reedy" <tjreedy at udel.edu>; "python-ideas at python.org" <python-ideas at python.org>
Subject: Re: [Python-ideas] Add 'module' module, similar to 'keyword' module




On Fri, 30 Oct 2015 at 22:57 Terry Reedy <tjreedy at udel.edu> wrote:

On 10/30/2015 9:19 PM, Guido van Rossum wrote:
> There's sys.builtin_module_names which returns the names of the
> hardcoded builtin modules.

Great.  With this solved, I opened an issue for IDLE.
https://bugs.python.org/issue25522

> Dynamically loaded modules can be found by
> searching sys.path in the usual way -- importlib shoul know. I wonder if
> just asking importlib whether it can locate a given module would be enough?

The default search order is stdlib builtins, local user files, /lib
files, so the shadowing issue the opposite for builtin and /lib modules.
  Hence a different message is needed.



Quick and dirty way to use importlib is to get the location of the stdlib (os.__file__ should work since it's hard-coded in the interpreter as representing where the stdlib is) and then use importlib.find_spec() for a module name to check if the file location in the spec has the same location prefix as os or not (make sure you use absolute paths since it isn't guaranteed if you don't execute site.py).


I've now seen this use case, the logging one, and the 2to3 module rename. I'm starting to wonder if there some general solution that should get added to the import machinery that  can serve these cases more easily than with importers or __import__ overrides that can be tricky to get right.


-Brett
 

> On Fri, Oct 30, 2015 at 6:09 PM, Terry Reedy
> <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:
>
>     This idea results from issue of user files shadowing stdlib files on
>     import.  There was a thread on pydev about this yesterday.  There is
>     also an opposite issue of builtin modules shadowing user files.
>
>     The keyword module provides kwlist and iskeyword function.  One use
>     of kwlist is used in some other stdlib modules and can be used by
>     syntax highlighters (as in IDLE).  Kwlist is updated by the main
>     function.
>
>     A module module would have at least liblist and islibmodule
>     function. Liblist would contain all directories with __init__.py and
>     all .py files.  (I don't think files within package directories
>     should be included, as there is no direct shadowing problem.)  A
>     python oriented editor could then warn on save requests "This name
>     matches a stdlib name in /Lib. If you run python in this directory,
>     you will not be able to import the stdlib module.  Continue?".
>
>     The module should also have binlist and isbinmodule for builtin
>     modules.  (I do not know how to get such a list.  If necessary, an
>     api could be added.)  An editor could than warn "This name matches a
>     builtin stdlib name.  You will not be able to import this file.
>     Continue?".
>
>     --
>     Terry Jan Reedy
>
>     _______________________________________________
>     Python-ideas mailing list
>     Python-ideas at python.org
>     <mailto:Python-ideas at python.org>
>     https://mail.python.org/mailman/listinfo/python-ideas
>     Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


--
Terry Jan Reedy

_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151031/45ec6317/attachment-0001.html>

From tjreedy at udel.edu  Sat Oct 31 18:45:15 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 31 Oct 2015 18:45:15 -0400
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <20151031081450.GD10946@ando.pearwood.info>
References: <n114bh$ih1$1@ger.gmane.org>
 <20151031081450.GD10946@ando.pearwood.info>
Message-ID: <n13ga1$scg$1@ger.gmane.org>

On 10/31/2015 4:14 AM, Steven D'Aprano wrote:

A response to my already half-dead proposal.  Literally 10 minutes after 
I posted it, Guido replied that half of what I wanted already existed as 
sys.builtin_module_names.  I acknowledged that, two hours before 
Steven's post, by saying I would go ahead and use this tuple for IDLE.

General comments: The possibility of name clashes arises because Python 
does not limit import to stdlib.  As Steven noted, Python cannot prevent 
this.  What it could do is search the all possible import sources with 
each import and report clashes before picking one.  I am not proposing 
this.  Whether python picks a user file or stdlib file when both have 
the same name depends on how the stdlib version is implemented.

Specific comment: I limited the proposal to the stdlib because a) the 
stdlib is fixed for a given version of CPython on a particular OS*, and 
b) the reported problems of beginners that I have seen, where they are 
stuck on what to do, involve the stdlib.  (I could have made this 
limitation clearer in my original first paragraph.) *Except as modules 
are omitted in a particular build.

sys.builtin_module_names exists because the information is not otherwise 
exposed and because it is needed and used in several places.

Revised and reduced proposal: If other people would find it useful, add 
all_stdlib_toplevel_module_names - builtin_module_names to sys as 
something equivalent to .other_stdlib_module_names or 
.python_coded_module_names.  Anyone wanting all_toplevel_module names 
could add the two.  Or add the latter, and let others subtract.

In the meanwhile, I will adapt the code in test__all__ that creates such 
a list from the lib directory.

-- 
Terry Jan Reedy


From brenbarn at brenbarn.net  Sat Oct 31 19:07:02 2015
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Sat, 31 Oct 2015 16:07:02 -0700
Subject: [Python-ideas] Add 'module' module, similar to 'keyword' module
In-Reply-To: <n13ga1$scg$1@ger.gmane.org>
References: <n114bh$ih1$1@ger.gmane.org>
 <20151031081450.GD10946@ando.pearwood.info> <n13ga1$scg$1@ger.gmane.org>
Message-ID: <56354996.8030807@brenbarn.net>

On 2015-10-31 15:45, Terry Reedy wrote:
> Revised and reduced proposal: If other people would find it useful, add
> all_stdlib_toplevel_module_names - builtin_module_names to sys as
> something equivalent to .other_stdlib_module_names or
> .python_coded_module_names.  Anyone wanting all_toplevel_module names
> could add the two.  Or add the latter, and let others subtract.

	One thing I've sometimes wondered about for Python 4000 is the idea of 
putting the whole standard library under a single top-level package 
(e.g., from stdlib import sys).  This would be a big change but would 
reduce the surprises that can arise from stdlib modules named things 
like "string", "parser", etc.  I see a brief mention of this in PEP 3108 
but that's it.  Was there more discussion of the idea?

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown