From sjoerdjob at sjoerdjob.com  Wed Jun  1 02:25:39 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Wed, 1 Jun 2016 08:25:39 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574DA1A6.8090906@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
Message-ID: <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>



> On 31 May 2016, at 16:37, Sven R. Kunze <srkunze at mail.de> wrote:
> 
> 
> 
> Btw. there would be huge use-case for Django development. So, given your explanation, it sounds useful.

Could you elaborate on that use-case, because I'm not seeing it. I think maybe you mean the ORM and declarative forms, correct?

Maybe knowing a use-case could better direct the efforts?

From sjoerdjob at sjoerdjob.com  Wed Jun  1 02:25:51 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Wed, 1 Jun 2016 08:25:51 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
Message-ID: <E4E2A40F-E9E5-42F2-BBA5-13E69ED56328@sjoerdjob.com>

I agree that cleaning up is not really needed.

I think the problem is that people feel bad about having to type things twice and immediately think "It's violating DRY". And some are quite adamant about not violating DRY.

Fact of the matter: on nearly all relevant scales, this "DRY violation" does not matter, in fact, I would not even want to call it a violation. DRY is about having 1 representative location for knowledge. The name of a variable (1) hardly counts as knowledge and (2) is also used at other locations (use-sites).

Is repeating the variable name that much of a deal? Granted, I mostly only use namedtuple of all the given examples. And I use that sparingly. Are `TypeVar` and `Symbol` used more often?

At least for namedtuple, we could do some meta-classy logic to allow for

    class Point(namedtuple):
        _fields = ('x', 'y')

And for namedtuple it makes sense. For TypeVar it might too?

    class T(TypeVar): pass

Symbol? Not so much, I think.

I'm not eager to see "improvements" in this area, as I do not see it as a problem, except for those with an overly strict view on DRY. Any new syntax/semantics for this would need to bring a real obvious improvement for the general case, and I doubt that will happen.

The one suggestion I have heard that remotely makes sense (and it was not even in this thread!) is the concept of decorators for variables

    @Something
    x = 5

would become

    x = Something('x', 5)

And even that idea is bad: what if you want multiple arguments? Would you need a tuple? What if you don't need any arguments? (e.g. Symbol).

After seeing all the suggestions so far, I'm not convinced by any of them that they are an improvement. So unless there's a suggestion for syntax/semantics that is reasonable to become the only **obvious** way to do it, I'd say nay.

> On 1 Jun 2016, at 02:52, Michael Selik <michael.selik at gmail.com> wrote:
> 
>> On Mon, May 30, 2016 at 11:08 PM Steven D'Aprano <steve at pearwood.info> wrote:
>> On Mon, May 30, 2016 at 06:16:26PM -0700, Guido van Rossum wrote:
>> 
>> [...]
>> >   T = TypeVar('T')
>> >   x = Symbol('x')
>> >
>> > I'm sure this is not a new idea, but so far I've always thought that
>> > this is pretty esoteric and the approach here is good enough.
>  
>> This comes up a lot and it would be nice to clean it up.
>> 
>> T = type('T', bases, ns)
>> Record = namedtuple('Record', fields)
> 
> I'm not enthusiastic about cleaning it up. It feels appropriately awkward. I don't want my colleagues having too easy a time dynamically creating types or functions.
> 
> You brought up decorators as an example, but I think that was more about how decorators tend to be used essentially as part of the function definition. They might dramatically alter the behavior of the function and yet appear as an afterthought without the decorator syntax. From the PEP 318, "This becomes less readable with longer [functions]". The motivation said nothing about avoiding typing the function name twice.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/ea7daaad/attachment.html>

From mtanous22 at gmail.com  Wed Jun  1 02:38:40 2016
From: mtanous22 at gmail.com (Matthew Tanous)
Date: Wed, 1 Jun 2016 00:38:40 -0600
Subject: [Python-ideas] Proposal to change List Sequence Repetition (*)
 so it is not useless for Mutable Objects
In-Reply-To: <574D4171.3060301@egenix.com>
References: <574D20D9.7060504@gmail.com> <574D4171.3060301@egenix.com>
Message-ID: <574E82F0.1020704@gmail.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Maybe I'm not following, but I don't see there being anything but a
potential overhead penalty from copying immutable objects.  If the
object is immutable, copying a reference to it and copying the object
itself seem transparently "identical" in terms of future use.

I acknowledge that copying the objects is a potential issue, but I think
this would be solved by making the sequence repetition operator
functionally equivalent to the list comprehension, such that [x()] * 5
is the same, semantically, as [x() for i in range(5)].  Alternatively,
these objects could be copied in the same manner as the deepcopy
functionality, although this solution may not be the best way to do it.

Ostensibly, I don't see why this wouldn't apply to all collection
objects that use the sequence repetition operator (lists, tuples, etc.)
to create a sequence.

I agree with your description of the current behavior as "to repeat the
currently existing objects".  But it seems to me that except for some
extremely special cases, this limits it to immutable objects, where
(somewhat ironically) there is no functional difference between
repeating the objects and copying them other than slight differences in
memory usage.

On 5/31/16 1:46 AM, M.-A. Lemburg wrote:
> On 31.05.2016 07:27, Matthew Tanous wrote:
>>
>> Currently, the use of the (*) operator on a list is to duplicate a list
>> by creating multiple references to the same object.  While this works
>> intuitively for immutable objects (like [True] * 5) as these immutable
>> references are replaced when the list is assigned to, it makes the
>> operator nigh unusable for mutable objects.
>>
>> The most obvious case is when the operator is duplicated in a sequence
>> like this:
>>
>> arr = [[True] * 5] * 5
>>
>> This does not create a matrix-like arrangement of the immutable truth
>> variable, but instead creates a list of 5 references to the same list,
>> such that a following assignment like arr[2][3] = False will not change
>> just that one index, but every 4th element of each list in the outer
list.
>>
>> ...
>>
>> It is my opinion that the sequence repetition operator should be
>> modified to make copies of the objects it is repeating, rather than
>> copying references alone.  I believe this would both be more intuitive
>> from a semantic point of view and more useful for the developer.
>>
>> This would change the operator in a way that is mostly unseen in current
>> usage ([5] * 3 would still result in [5, 5, 5]) while treating mutable
>> nesting in a way that is more understandable from the apparent intent of
>> the syntax construction.
>
> Some questions:
>
> * How would you determine whether a list element is mutable
> or not ?
>
> * How would you copy the elements ?
>
> * For which object types would you want to change the behavior ?
>
> I agree that the repeat operator can sometimes create confusing
> and unwanted object structures if not used correctly, but it's main
> purpose it to repeat the already existing objects, not to copy them,
> so the current behavior still is conceptually correct.
>

-----BEGIN PGP SIGNATURE-----

iQEcBAEBCAAGBQJXToLwAAoJEF14rZEhZ/cMxqQIAIpVlRs3PymKNKcgdVnOmf3e
rXWMRmr0T+XZzEsLU5bXy9o2dOx97xoxtmC5k57J5ak7qNiqy3SUPRecOEvuv/Xb
U2c3GNQDyXcHlfcx3C57AD3uyM40u4KXBX4dNsaHMZ6NT986SwS4hV/k2y1gkp8W
lcP5NkTA9PJCnqo+J6/UWSY9jERGScPGaGYygedmdZpUJsQKtW4dUNslMHpdO/cZ
m8b35A5BC8TjRWk/arwLK2vEXHJs4SnZz7JyUgIhigskLQN/vpaDjJuwGOCJVXCx
V5Q1pbk7cu6WlcBs1K+KiwzonpyNzrCjrdu69LDIL63q7mbOH08/xqwX5SdWTWA=
=8zaV
-----END PGP SIGNATURE-----



From mtanous22 at gmail.com  Wed Jun  1 02:49:27 2016
From: mtanous22 at gmail.com (Matthew Tanous)
Date: Wed, 1 Jun 2016 00:49:27 -0600
Subject: [Python-ideas] Proposal to change List Sequence Repetition (*)
 so it is not useless for Mutable Objects
Message-ID: <574E8577.5060308@gmail.com>

I could see the use of the operator '@' to construct a matrix, perhaps.
It is more limited though (essentially a 2D version of sequence
repetition). Semantically, it would seem to me intuitive that [Foo()] *
N would create a list of N Foos, not a list of N references to the same
Foo. I can see that people who are used to how it actually works now
would have issues with such a change in the semantics, however. Didn't
consider the zip(*[it]*3) case being broken, I admit.



From mtanous22 at gmail.com  Wed Jun  1 02:50:18 2016
From: mtanous22 at gmail.com (Matthew Tanous)
Date: Wed, 1 Jun 2016 00:50:18 -0600
Subject: [Python-ideas] Proposal to change List Sequence Repetition (*)
 so it is not useless for Mutable Objects
In-Reply-To: <CAMsGuymV84h=yLEhT1yXg7vhWNNFu1pHD+eWU=X670eQAmujQQ@mail.gmail.com>
References: <CAMsGuymV84h=yLEhT1yXg7vhWNNFu1pHD+eWU=X670eQAmujQQ@mail.gmail.com>
Message-ID: <574E85AA.10805@gmail.com>

I could see the use of the operator '@' to construct a matrix, perhaps.
It is more limited though (essentially a 2D version of sequence
repetition). 

Semantically, it would seem to me intuitive that [Foo()] *
N would create a list of N Foos, not a list of N references to the same
Foo. I can see that people who are used to how it actually works now
would have issues with such a change in the semantics, however. Didn't
consider the zip(*[it]*3) case being broken, I admit.


From mal at egenix.com  Wed Jun  1 04:09:05 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 1 Jun 2016 10:09:05 +0200
Subject: [Python-ideas] Proposal to change List Sequence Repetition (*)
 so it is not useless for Mutable Objects
In-Reply-To: <574E82F0.1020704@gmail.com>
References: <574D20D9.7060504@gmail.com> <574D4171.3060301@egenix.com>
 <574E82F0.1020704@gmail.com>
Message-ID: <574E9821.9030908@egenix.com>

On 01.06.2016 08:38, Matthew Tanous wrote:
> 
> Maybe I'm not following, but I don't see there being anything but a
> potential overhead penalty from copying immutable objects.  If the
> object is immutable, copying a reference to it and copying the object
> itself seem transparently "identical" in terms of future use.
> 
> I acknowledge that copying the objects is a potential issue, but I think
> this would be solved by making the sequence repetition operator
> functionally equivalent to the list comprehension, such that [x()] * 5
> is the same, semantically, as [x() for i in range(5)].  Alternatively,
> these objects could be copied in the same manner as the deepcopy
> functionality, although this solution may not be the best way to do it.
> 
> Ostensibly, I don't see why this wouldn't apply to all collection
> objects that use the sequence repetition operator (lists, tuples, etc.)
> to create a sequence.
> 
> I agree with your description of the current behavior as "to repeat the
> currently existing objects".  But it seems to me that except for some
> extremely special cases, this limits it to immutable objects, where
> (somewhat ironically) there is no functional difference between
> repeating the objects and copying them other than slight differences in
> memory usage.

What you are describing is a duplication mechanism, not a repeat
mechanism, so essentially reassigning the meaning of seq * number.
I don't think that's in line with the way we handle backwards
compatibility in Python.

Please note that there are indeed valid use cases for
repeating even mutable types, namely when you don't intend
to mutate the contents of the objects, but are only interested
in producing a readable data structure with repeated entries,
e.g. for iteration or use as multi-dimensional constant in
calculations.

The duplication mechanism you have in mind can be implemented
using a list comprehension, for example:

import copy

arr = [copy.deepcopy([True] * 5) for i in range(5)]

It's probably better to add a copy.duplicate() API of sorts
than to try to change the * operator on built-in sequences.


> On 5/31/16 1:46 AM, M.-A. Lemburg wrote:
>> On 31.05.2016 07:27, Matthew Tanous wrote:
>>>
>>> Currently, the use of the (*) operator on a list is to duplicate a list
>>> by creating multiple references to the same object.  While this works
>>> intuitively for immutable objects (like [True] * 5) as these immutable
>>> references are replaced when the list is assigned to, it makes the
>>> operator nigh unusable for mutable objects.
>>>
>>> The most obvious case is when the operator is duplicated in a sequence
>>> like this:
>>>
>>> arr = [[True] * 5] * 5
>>>
>>> This does not create a matrix-like arrangement of the immutable truth
>>> variable, but instead creates a list of 5 references to the same list,
>>> such that a following assignment like arr[2][3] = False will not change
>>> just that one index, but every 4th element of each list in the outer
> list.
>>>
>>> ...
>>>
>>> It is my opinion that the sequence repetition operator should be
>>> modified to make copies of the objects it is repeating, rather than
>>> copying references alone.  I believe this would both be more intuitive
>>> from a semantic point of view and more useful for the developer.
>>>
>>> This would change the operator in a way that is mostly unseen in current
>>> usage ([5] * 3 would still result in [5, 5, 5]) while treating mutable
>>> nesting in a way that is more understandable from the apparent intent of
>>> the syntax construction.
> 
>> Some questions:
> 
>> * How would you determine whether a list element is mutable
>> or not ?
> 
>> * How would you copy the elements ?
> 
>> * For which object types would you want to change the behavior ?
> 
>> I agree that the repeat operator can sometimes create confusing
>> and unwanted object structures if not used correctly, but it's main
>> purpose it to repeat the already existing objects, not to copy them,
>> so the current behavior still is conceptually correct.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 01 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From ncoghlan at gmail.com  Wed Jun  1 07:03:47 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 1 Jun 2016 21:03:47 +1000
Subject: [Python-ideas] Declaring immutability (was Re: Proposal to change
 List Sequence Repetition (*) so it is not useless for Mutable Objects)
In-Reply-To: <CADiSq7cmj=b39uywL+F6ZQ3b+hNYNbhL_v2N6OMJfRSr0+6+Yw@mail.gmail.com>
References: <CADiSq7fhj8mWc9vethm1Hqnf41w6E8qSAQgcMYr8Yd-qHqfKmg@mail.gmail.com>
 <CADiSq7eQsv+CLoCtK5NgBX+QpL1rK=zRdn==hVn2tn1ypM5GQg@mail.gmail.com>
 <CADiSq7cERSuyuRJkyw+DbZ7TL7VcA+t1g=sQEzDkf9fjrinboA@mail.gmail.com>
 <CADiSq7cipatBxuh5uj_LvfjjCPVrZ8vz2K9GjWeqd-TV__oeTQ@mail.gmail.com>
 <CADiSq7eOa4XWwYf5tsa-RaEao-rJmy0y5ReB=OUBP3BbU9tUjQ@mail.gmail.com>
 <CADiSq7fhgzpNRTm-sfHmv8F2tfVV4iBd=EzAjChB8EAsgYcd7g@mail.gmail.com>
 <CADiSq7d8vNQpVYus5DTEJKhWRUeiUJW+nq7qZSeH4bcz8-5N3g@mail.gmail.com>
 <CADiSq7fijo13k_P-Rtv3ZWFLVhWVven53SJif5Q8JQ4=+rZcTg@mail.gmail.com>
 <CADiSq7e7YaEsZA=42bwFYv65wPkfwHfdMQKm+xu+qMwhvr=aFA@mail.gmail.com>
 <CADiSq7d0+SA2jyGeLNdkymdE62bJL40FZ5W=FsZmARtG1mKvAA@mail.gmail.com>
 <CADiSq7fp2-J43FweyvRKaptH8eJSndeGWPMgRNg7+ZD9LtNi-g@mail.gmail.com>
 <CADiSq7c4FnERfdvOUHw04DkeULXTr2CKPUqDQTX__MTYxno1-Q@mail.gmail.com>
 <CADiSq7cwtgNN3oUGDGVFtUZfySOeDHArXXwQpihSiewQYhgs7w@mail.gmail.com>
 <CADiSq7dWVeC1A+b+FTrAsgFzM9y=YVmzNdQ=S8OCRjWXzN=XQA@mail.gmail.com>
 <CADiSq7e7UTE-Hfz1w-Xr+TDwc==sZcDCzv=cUs+Oe4vA5Xy3VQ@mail.gmail.com>
 <CADiSq7ddcSZ4aV8GU7Y3231cVJej6oZwLVLcdyNT15MnifVTrg@mail.gmail.com>
 <CADiSq7ek6YuBxkFrJ-S0y1WMaHE5TZbXOCUWJ2JCV9zAXPtOnQ@mail.gmail.com>
 <CADiSq7cCm0Q_5TjWvv_L7jDdSo5bpd-_npuA0fyac4im6q_E1Q@mail.gmail.com>
 <CADiSq7eL5790tDijLZ062NRH-ZofyH18n3TGu3c9uZSD2ymLqA@mail.gmail.com>
 <CADiSq7fhvuq_VDCTFckkp0uxubC5N5dB5LQCZjWRKgpfyRC+yg@mail.gmail.com>
 <CADiSq7dP=DiOPtpqGot6jd6z57xzYku_xYaJGEjsRwmPCL=JsA@mail.gmail.com>
 <CADiSq7e3VCAB41Di9pC+UKzaDWhXKUkHd+fqKfJ5JzgP509eYg@mail.gmail.com>
 <CADiSq7cgmCoua2_QiMYgsbbgODzm93pqc+r0jsj1UN4pT=kLig@mail.gmail.com>
 <CADiSq7fA7uDuk94VTMSBXgf_6HqOL+84Z1O1h453J2w7EmjK3A@mail.gmail.com>
 <CADiSq7eeEvx2kTgEt+PpZPQ_Fk4ZoUOyT6qxhtcTw0LeSaAB6A@mail.gmail.com>
 <CADiSq7cfDWKfhhaqJ3mX_eqEwL7V2fvVSLFyXJivxcWZ_7d6_w@mail.gmail.com>
 <CADiSq7cmj=b39uywL+F6ZQ3b+hNYNbhL_v2N6OMJfRSr0+6+Yw@mail.gmail.com>
Message-ID: <CADiSq7dmwtpyG62kdLVYs1CysupfG6g0257fxV_O4bKAp9ifRQ@mail.gmail.com>

On 31 May 2016 07:17, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> On Tue, May 31, 2016 at 01:36:31PM +0000, Joseph Martinot-Lagarde wrote:
> > > I can only agree here. Even today, despite knowing the fact, it's
> > > causing some headaches in some cases.
> >
> > How about raising an exception if mutable objects are in the list ?
>
> -1
>
> It's a gratuitous breakage that cannot solve the problem, because you
> cannot tell in advance which objects are mutable and which are not.

Steven raised the iterator windowing example I was going to point to as the
canonical use case that means the current behaviour won't change.

However, if folks want a somewhat-related genuinely interesting language
design problem to chew on: try to come up with a way a way to formally
define-and-enforce instance immutability in Python. As a pragmatic matter,
you'll probably still need to allow mutation at least until __new__
finishes running, and some use cases, like weak referencing and some
caching techniques, involve storing mutable state on semantically immutable
instances, so this is likely to be better approached as a type hinting
problem rather than as a runtime feature.

The closest related concept I'm personally familiar with is const pointers
and references in C and C++, so there may be some insights to be gained
there (e.g. being able to declare __new__ as returning "->
typing.Immutable" or "-> typing.Mutable", and being able to explicitly
annotate function parameters as either Immutable parameters, which
definitely won't be modified, and Mutable parameters, which definitely will
be modified. Pure and impure methods could then use those annotations on
their "self" parameter to indicate whether or not they had side effects)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/a6102dfb/attachment.html>

From greg.ewing at canterbury.ac.nz  Wed Jun  1 07:32:21 2016
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 01 Jun 2016 23:32:21 +1200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <E4E2A40F-E9E5-42F2-BBA5-13E69ED56328@sjoerdjob.com>
References: <E4E2A40F-E9E5-42F2-BBA5-13E69ED56328@sjoerdjob.com>
Message-ID: <574EC7C5.8030301@canterbury.ac.nz>

Sjoerd Job Postmus wrote:
> I would not even want to call it a violation. 
> DRY is about having 1 representative location for knowledge. The name of 
> a variable (1) hardly counts as knowledge and (2) is also used at other 
> locations (use-sites).

I don't know whether it meets the strict definition of DRY or
not, but the reason it's annoying is that it's a repitition
that doesn't convey any information.

Using a variable name in multiple places tells you something:
"this thing here refers to that thing defined over there".
But having the variable name in two places in the *definition*
tells you nothing.

-- 
Greg

From srkunze at mail.de  Wed Jun  1 09:19:59 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 15:19:59 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574E225C.6090400@feete.org>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
Message-ID: <574EE0FF.8060308@mail.de>

On 01.06.2016 01:46, Ian Foote wrote:
> This is the first syntax I've seen in this thread that seems reasonably
> intuitive and pythonic. I don't mind particularly the keyword used (def
> or something else), but this structure draws the right parallels in my mind.

Still nobody explained why it needs a special syntax at all.

Normal assignment could do it as well as long as the assignment tells 
the RHS under what name it will be referred to.


Best,
Sven

From srkunze at mail.de  Wed Jun  1 09:22:56 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 15:22:56 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
Message-ID: <574EE1B0.3070106@mail.de>

On 01.06.2016 08:25, Sjoerd Job Postmus wrote:
>> On 31 May 2016, at 16:37, Sven R. Kunze <srkunze at mail.de> wrote:
>>
>> Btw. there would be huge use-case for Django development. So, given your explanation, it sounds useful.
> Could you elaborate on that use-case, because I'm not seeing it. I think maybe you mean the ORM and declarative forms, correct?

Exactly. That is one reason why Django needs metaclasses all over the 
place. Because fields just don't know their name.


This is especially true for forms as django does not magic there. It 
always causes headaches if you need it.


Best,
Sven

From srkunze at mail.de  Wed Jun  1 09:29:36 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 15:29:36 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574E19DB.7050307@canterbury.ac.nz>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz>
Message-ID: <574EE340.7020502@mail.de>

On 01.06.2016 01:10, Greg Ewing wrote:
> Steven D'Aprano wrote:
>> name -> Function(args)
>>
>> will be expanded to:
>>
>> name = Function('name', args)
>
> Another possibility would be to expand it to
>
>    name = Function(args)
>    name.__name__ = "name"
>

Another possibility would be (requiring no syntax change):

     name = function(args)

would always be expanded to

     name = function(args)
     name.__name__ = "name"

or to

     name = function(args)
     # and function.__name__ is already available while function is called


Maybe, somebody knows a reason why a called callable shouldn't know that 
piece of information. I can't think of one.


If there's no assignment, __name__ is None.


Best,
Sven

From srkunze at mail.de  Wed Jun  1 09:37:10 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 15:37:10 +0200
Subject: [Python-ideas] pattern matching proof-of-concept
In-Reply-To: <CACac1F_p78rFgFrNwyse2DO4Y677PwnH8nLYNPqhkTcjT+QG5g@mail.gmail.com>
References: <CAGgTfkP33QPEsrC8kpSds1nLMufGGPxN+W4yWqU5sjD++5HGLQ@mail.gmail.com>
 <CACac1F_p78rFgFrNwyse2DO4Y677PwnH8nLYNPqhkTcjT+QG5g@mail.gmail.com>
Message-ID: <574EE506.9030100@mail.de>

On 28.05.2016 12:26, Paul Moore wrote:
> On 28 May 2016 at 08:22, Michael Selik <michael.selik at gmail.com> wrote:
>> My original dict unpacking proposal was very short and lacked a motivating
>> usage. Toy examples made my proposal look unnecessarily verbose and
>> suggested obvious alternatives with easy current syntax.
>>
>> Nested/recursive unpacking is much more troublesome, especially when
>> combined with name-binding. I wrote an example to compare my proposal with
>> current syntax.
>>
>> Example usage.
>> https://github.com/selik/destructure/blob/master/examples/fips.py
>>
>> Implementation.
>> https://github.com/selik/destructure/blob/master/destructure.py
>>
>> The design of my module I'm least happy with is the name-binding. I extended
>> a SimpleNamespace to create an Erlang-style distinction between bound and
>> unbound names. Though the API is a bit awkward, now that the module is
>> built, I'm less enthusiastic about introducing new syntax. Funny how that
>> works.
>>
>> I haven't yet decided how to add post-binding guards to the cases.
> Interesting! Thanks for taking the time to make a real-world use case.
> I haven't looked at the module yet, just the example, but the code
> does look pretty clean and readable. The example is certainly complex
> enough that I'd probably end up with pretty messy and fragile code if
> I just tried to put something together with pure Python code.
>
> And yes, it's interesting how finding a good API for a module can make
> the need for a dedicated syntax less pressing. But working out that
> good API can be really hard (I don't think I'd ever have thought of
> doing it the way you did).

That's exactly what I had in mind. :) Far more flexible than a 
monolithic switch-case and looks quite Pythonic (= to me: minimum amount 
of special characters ;-) ).

Thanks a lot, Michael.

Best,
Sven

From sjoerdjob at sjoerdjob.com  Wed Jun  1 09:37:47 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Wed, 1 Jun 2016 15:37:47 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EE1B0.3070106@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
 <574EE1B0.3070106@mail.de>
Message-ID: <2FC3778A-59C7-4564-8E94-83CFDFD25C0C@sjoerdjob.com>



> On 1 Jun 2016, at 15:22, Sven R. Kunze <srkunze at mail.de> wrote:
> 
> On 01.06.2016 08:25, Sjoerd Job Postmus wrote:
>>> On 31 May 2016, at 16:37, Sven R. Kunze <srkunze at mail.de> wrote:
>>> 
>>> Btw. there would be huge use-case for Django development. So, given your explanation, it sounds useful.
>> Could you elaborate on that use-case, because I'm not seeing it. I think maybe you mean the ORM and declarative forms, correct?
> 
> Exactly. That is one reason why Django needs metaclasses all over the place. Because fields just don't know their name.
> 
> 
> This is especially true for forms as django does not magic there. It always causes headaches if you need it.
> 
> 
> Best,
> Sven
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

As an alternative suggestion, maybe there could be a `get_assignment_target` function which returns either a string, or `None`. No new syntax required at all, but it comes across to me as completely hacky.

From mal at egenix.com  Wed Jun  1 09:53:04 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 1 Jun 2016 15:53:04 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EE0FF.8060308@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de>
Message-ID: <574EE8C0.1070800@egenix.com>

On 01.06.2016 15:19, Sven R. Kunze wrote:
> On 01.06.2016 01:46, Ian Foote wrote:
>> This is the first syntax I've seen in this thread that seems reasonably
>> intuitive and pythonic. I don't mind particularly the keyword used (def
>> or something else), but this structure draws the right parallels in my
>> mind.
> 
> Still nobody explained why it needs a special syntax at all.
> 
> Normal assignment could do it as well as long as the assignment tells
> the RHS under what name it will be referred to.

Right. Essentially, we'd only need a way to tell the compiler
to invoke e.g. a method on the RHS object which gets called
with the name of the variable it binds (and perhaps the line
number to allow for e.g. recording the order of definitions).

The current byte code for:

    x = obj

reads like this:

  2           0 LOAD_GLOBAL              0 (obj)
              3 STORE_FAST               0 (x)

so if we could get the compiler to generate a method call
right after the assignment, e.g. obj.recordbinding('x', 2)
(taking as arguments the name of the variable and the line
number) we could do lots of interesting stuff.

This could be done via a decorator:

    @recordbinding
    x = obj

to result in the compiler generating the following code:

    x = obj
    obj.recordbinding('x', 2)

Alternativey, you could do all this without any changes
to the interpeter by using a trace function which traces
the execution of a block:

    start_recordbingings() # enable trace function
    x = obj # trace function detects assignment and calls
            # obj.recordbinding('x', 2)
    stop_recordbindings() # disable trace function

However, this is very slow.

A second option would be to place the above into a function
definition and have a decorator apply the byte code
manipulations to implement the implicit call:

@recordbindings
def code():
    x = obj

but this would create problems with the local/global
namespaces.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 01 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From p.f.moore at gmail.com  Wed Jun  1 10:06:20 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 1 Jun 2016 15:06:20 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EE340.7020502@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de>
 <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
Message-ID: <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>

On 1 June 2016 at 14:29, Sven R. Kunze <srkunze at mail.de> wrote:
> Another possibility would be (requiring no syntax change):
>
>     name = function(args)
>
> would always be expanded to
>
>     name = function(args)
>     name.__name__ = "name"
>
> or to
>
>     name = function(args)
>     # and function.__name__ is already available while function is called
>
>
> Maybe, somebody knows a reason why a called callable shouldn't know that
> piece of information. I can't think of one.
>
>
> If there's no assignment, __name__ is None.

I'm not sure I follow this proposal. Could you explain a little please?

Given

    x = Symbol('x')

as the current approach, how would you modify that to work with your
proposal? If you could show how you'd write a wrapper function
AutoSymbol, so that

    x = AutoSymbol()

did the same as the above, that'd help me a lot.

Thanks,
Paul

From srkunze at mail.de  Wed Jun  1 11:10:47 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 17:10:47 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
Message-ID: <574EFAF7.7030502@mail.de>

On 01.06.2016 16:06, Paul Moore wrote:
> I'm not sure I follow this proposal. Could you explain a little please?
>
> Given
>
>      x = Symbol('x')
>
> as the current approach, how would you modify that to work with your
> proposal? If you could show how you'd write a wrapper function
> AutoSymbol, so that
>
>      x = AutoSymbol()
>
> did the same as the above, that'd help me a lot.
>
> Thanks,
> Paul

Okay, let's see.


class Symbol:
     def __init__(self, name):
         self.name = name # that's me :)


That's the current way. If now, the compiler would assign "x" to a 
special dunder attribute/variable, that would allow __init__ to extract 
that name and use it as if it were a parameter:

class AutoSymbol:
     def __init__(self):
         self.name = __assigned_name__ # that's me :)


That's just it. I consider __assigned_name__ the same as __module__. A 
special variable available if you need it.

A two-step approach like the following would also be possible:

class AutoSymbol:

     def __init__(self):
         pass # no name available :(

     def __name__(self, name):
         self.name = name  # that's me

In this case, name is not available in the constructor which could be 
regarded as a drawback. A advantage would be possible optimization. No 
__name__ member, no implicit call of __name__


On the caller site, nothing changes. It's just:


x = AutoSymbol()


Autosymbol either has __assigned_name__ set during constructor execution 
or __name__() will be set later. The same logic could be applied to 
other callables, functions, lambdas, etc.



This post does not really care about the name of the special 
variable/member. It could also be '__lhs__'. I think others will come up 
with better names. In the end, one name needs to be chosen of course. ;-)


Best,
Sven


From srkunze at mail.de  Wed Jun  1 11:17:25 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 17:17:25 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EE8C0.1070800@egenix.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
Message-ID: <574EFC85.9050303@mail.de>

On 01.06.2016 15:53, M.-A. Lemburg wrote:
> On 01.06.2016 15:19, Sven R. Kunze wrote:
>> On 01.06.2016 01:46, Ian Foote wrote:
>>> This is the first syntax I've seen in this thread that seems reasonably
>>> intuitive and pythonic. I don't mind particularly the keyword used (def
>>> or something else), but this structure draws the right parallels in my
>>> mind.
>> Still nobody explained why it needs a special syntax at all.
>>
>> Normal assignment could do it as well as long as the assignment tells
>> the RHS under what name it will be referred to.
> Right. Essentially, we'd only need a way to tell the compiler
> to invoke e.g. a method on the RHS object which gets called
> with the name of the variable it binds (and perhaps the line
> number to allow for e.g. recording the order of definitions).

I think that's the way I would think of it. Although I don't know if I 
would prefer a two-step solution (that's what you've described), or a 
one-step solution (like an implicitly set magic variable); cf. my reply 
to Paul): 
https://mail.python.org/pipermail/python-ideas/2016-June/040677.html


In terms of usability, I think the magic variable is easier but it might 
be harder to optimize.

> The current byte code for:
>
>      x = obj
>
> reads like this:
>
>    2           0 LOAD_GLOBAL              0 (obj)
>                3 STORE_FAST               0 (x)
>
> so if we could get the compiler to generate a method call
> right after the assignment, e.g. obj.recordbinding('x', 2)
> (taking as arguments the name of the variable and the line
> number) we could do lots of interesting stuff.
>
> This could be done via a decorator:
>
>      @recordbinding
>      x = obj
>
> to result in the compiler generating the following code:
>
>      x = obj
>      obj.recordbinding('x', 2)
>
> Alternativey, you could do all this without any changes
> to the interpeter by using a trace function which traces
> the execution of a block:
>
>      start_recordbingings() # enable trace function
>      x = obj # trace function detects assignment and calls
>              # obj.recordbinding('x', 2)
>      stop_recordbindings() # disable trace function
>
> However, this is very slow.
>
> A second option would be to place the above into a function
> definition and have a decorator apply the byte code
> manipulations to implement the implicit call:
>
> @recordbindings
> def code():
>      x = obj
>
> but this would create problems with the local/global
> namespaces.

I hope that's not the only way to optimize it. :(


I am not that deep into CPython development in order to find a better 
solution but I think it should be possible for the compiler to find out 
whether a callee actually USES that information or not without any 
special decoration or something.


Sven

From steve at pearwood.info  Wed Jun  1 11:19:46 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 01:19:46 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574E19DB.7050307@canterbury.ac.nz>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz>
Message-ID: <20160601151946.GB12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 11:10:19AM +1200, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >name -> Function(args)
> >
> >will be expanded to:
> >
> >name = Function('name', args)
> 
> Another possibility would be to expand it to
> 
>    name = Function(args)
>    name.__name__ = "name"

That priviledges one specific use of the name over all others. You'll 
note that I was very careful to describe the use-case as "objects that 
need to know their own name" but didn't specify what they did with the 
name. Maybe they bind it to self.__name__, but maybe they use self.name 
instead. Or they write it to a database. Who knows?

I don't think we should specify what the object does with the name.


-- 
Steve

From marky1991 at gmail.com  Wed Jun  1 11:24:35 2016
From: marky1991 at gmail.com (marky1991 .)
Date: Wed, 1 Jun 2016 11:24:35 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EFAF7.7030502@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
Message-ID: <CAG3cHaabjnmdv=LnUUmG=UWTOXfi7XLMFvs3mdVGXBq1yoze7Q@mail.gmail.com>

What is the value of __assigned_name__ here:

x = y = z = AutoSymbol()

? Is it a list?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/dd58672a/attachment.html>

From steve at pearwood.info  Wed Jun  1 11:29:23 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 01:29:23 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EFAF7.7030502@mail.de>
References: <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
Message-ID: <20160601152923.GC12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 05:10:47PM +0200, Sven R. Kunze wrote:

> class Symbol:
>     def __init__(self, name):
>         self.name = name # that's me :)
> 
> 
> That's the current way. If now, the compiler would assign "x" to a 
> special dunder attribute/variable, that would allow __init__ to extract 
> that name and use it as if it were a parameter:
> 
> class AutoSymbol:
>     def __init__(self):
>         self.name = __assigned_name__ # that's me :)

I don't think that will work. Greg's proposal was to expand the syntax 
x -> Function(args) to

    x = Function(args)
    x.__name__ = 'x'

which means that the name won't be available until after __new__ and 
__init__ have completed.

If you want instead to use a magic local variable somehow injected 
automatically into the __init__ method at runtime, I think that's very 
clever. TOO clever, and far too magical. Automatically providing a 
function argument is quite easy to understand, since the argument is 
right there in the method definition, and it is conceptually just like 
"self". It also means that both of these will work will work exactly the 
same way:

x = Function('x', args)  # explicitly provide the name

x -> Function(args)  # use the new syntax


But with your magic __assigned_name__ local variable, the signature of 
the function must change depending on whether it is being called with 
one argument or two.


> That's just it. I consider __assigned_name__ the same as __module__. A 
> special variable available if you need it.

I'm not sure what you mean by __module__.

py> class A:
...     def __init__(self):
...             print(__module__)
...
py> a = A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __init__
NameError: name '__module__' is not defined




-- 
Steve

From steve at pearwood.info  Wed Jun  1 11:36:40 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 01:36:40 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAG3cHaabjnmdv=LnUUmG=UWTOXfi7XLMFvs3mdVGXBq1yoze7Q@mail.gmail.com>
References: <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
 <CAG3cHaabjnmdv=LnUUmG=UWTOXfi7XLMFvs3mdVGXBq1yoze7Q@mail.gmail.com>
Message-ID: <20160601153640.GD12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 11:24:35AM -0400, marky1991 . wrote:
> What is the value of __assigned_name__ here:
> 
> x = y = z = AutoSymbol()
> 
> ? Is it a list?

With my proposal, that would be a syntax error. However this would be 
allowed:


x, y, z -> Symbol()  # there's nothing "auto" about it


and Symbol would receive a single argument, a tuple ('x', 'y', 'z'). 
Presumably then it could:

* raise an exception, to indicate that it doesn't support creating 
  three symbols at once;

* return three different symbols, one for each of the names;

* return a single symbol duplicated three times

whatever makes sense for the application.

But the important thing is, this would be *exactly* the same as:

x, y, z = Symbol(('x', 'y', 'z'))

The only magic is that you don't have to manually copy the names from 
the left hand side over onto the right and quote them. All Symbol knows 
is that it got passed a three-tuple as argument. It cannot know how it 
got there.


-- 
Steve

From p.f.moore at gmail.com  Wed Jun  1 11:42:39 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 1 Jun 2016 16:42:39 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EFAF7.7030502@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de>
 <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
Message-ID: <CACac1F8fUSSzPEdjBzmqf0DqSn5rdp+OgU2oysJ6PxACHoKzEQ@mail.gmail.com>

On 1 June 2016 at 16:10, Sven R. Kunze <srkunze at mail.de> wrote:
> That's the current way. If now, the compiler would assign "x" to a special
> dunder attribute/variable, that would allow __init__ to extract that name
> and use it as if it were a parameter:
>
> class AutoSymbol:
>     def __init__(self):
>         self.name = __assigned_name__ # that's me :)
>

Why doesn't the assignment here set __assigned_name__ too, and
override the value assigned by the "outer" assignment?

Also, as someone else pointed out, in

a = b = c = AutoSymbol()

what would __assigned_name__ be? The advantage of a special operation
is that it gives you a way of pointing out precisely *which* name
assignment we want to remember...

Paul

From mal at egenix.com  Wed Jun  1 11:53:26 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 1 Jun 2016 17:53:26 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EFC85.9050303@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <574EFC85.9050303@mail.de>
Message-ID: <574F04F6.5050606@egenix.com>

On 01.06.2016 17:17, Sven R. Kunze wrote:
> ...
> I hope that's not the only way to optimize it. :(

Well, if you restrict yourself to functions and objects,
you can implement the assigning from within the code:

Instead of writing:

    x = func('x')

you'd write

    func('x')

and have func register itself with the globals under the given
name:

    def func(bindto):
        value = 'Hello World !'
        globals()[bindto] = value
        return value

    func(bindto='x')

    >>> x
    'Hello World !'

> I am not that deep into CPython development in order to find a better
> solution but I think it should be possible for the compiler to find out
> whether a callee actually USES that information or not without any
> special decoration or something.

I don't see how that could work without being more explicit
about the intention.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 01 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From steve at pearwood.info  Wed Jun  1 11:59:09 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 01:59:09 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAEbHw4YoPNKbL1M9NN6RzGNduxtNS84gW8+JDu_dZGLcrDFGCA@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <20160531154327.GY12028@ando.pearwood.info>
 <CAEbHw4YoPNKbL1M9NN6RzGNduxtNS84gW8+JDu_dZGLcrDFGCA@mail.gmail.com>
Message-ID: <20160601155909.GE12028@ando.pearwood.info>

On Tue, May 31, 2016 at 10:35:01AM -0700, David Mertz wrote:

> For things like namedtuple that need more arguments, you'd need to use
> functools.partial to create the single argument callable to match the
> syntax.

As far as I am concerned, limiting it to single-argument functions 
cripples this proposal to the point that it is of no interest.

So far I've seen exactly four real-world use-cases, plus potentially 
Django. Only supporting a single name makes this useless for two of 
those use-cases, and I predict all of the Django examples[1]. Using 
partial is *more work than the status quo* which means nobody in their 
right mind is going to use it.



# (1) status quo
Record = namedtuple("Record", fields)


# (2a) my suggestion
Record -> namedtuple(fields)

# (2b) or if you insist
def Record = namedtuple(fields)


# (3) if it only supports a single argument
from functools import partial
Record -> partial(namedtuple, field_names=fields)()


That is significantly more annoying than the status quo. But it gets 
worse! Remember that partial() binds positional arguments from the left, 
which is the wrong side for what we need. Fortunately namedtuple 
supports keyword arguments, but what about those that don't?

For the sake of the exercise, let's pretend that namedtuple doesn't 
support keyword arguments, just to get an idea of how to solve it:

Record -> partial(lambda fields, name: namedtuple(name, fields), fields)()

And let's just hope nobody wants to call namedtuple with the other 
arguments, verbose or rename.

It is easy to wave your hands and say "just use partial", but not so 
easy to *actually do so*.





[1] On the basis that very few objects need to know *only* their name. I 
expect that if Django uses this pattern, it will take a name plus at 
least one more argument.


-- 
Steve

From sjoerdjob at sjoerdjob.com  Wed Jun  1 12:10:22 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Wed, 1 Jun 2016 18:10:22 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601155909.GE12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <20160531154327.GY12028@ando.pearwood.info>
 <CAEbHw4YoPNKbL1M9NN6RzGNduxtNS84gW8+JDu_dZGLcrDFGCA@mail.gmail.com>
 <20160601155909.GE12028@ando.pearwood.info>
Message-ID: <2DD61275-9B7B-4C7B-8454-7C0B3D5678EB@sjoerdjob.com>



> On 1 Jun 2016, at 17:59, Steven D'Aprano <steve at pearwood.info> wrote:
> 
>> On Tue, May 31, 2016 at 10:35:01AM -0700, David Mertz wrote:
>> 
>> For things like namedtuple that need more arguments, you'd need to use
>> functools.partial to create the single argument callable to match the
>> syntax.
> 
> As far as I am concerned, limiting it to single-argument functions 
> cripples this proposal to the point that it is of no interest.
> 
> So far I've seen exactly four real-world use-cases, plus potentially 
> Django. Only supporting a single name makes this useless for two of 
> those use-cases, and I predict all of the Django examples[1]. Using 
> partial is *more work than the status quo* which means nobody in their 
> right mind is going to use it.
> 
> 
> 
> # (1) status quo
> Record = namedtuple("Record", fields)
> 
> 
> # (2a) my suggestion
> Record -> namedtuple(fields)
> 
> # (2b) or if you insist
> def Record = namedtuple(fields)
> 
> 
> # (3) if it only supports a single argument
> from functools import partial
> Record -> partial(namedtuple, field_names=fields)()
> 
> 
> That is significantly more annoying than the status quo. But it gets 
> worse! Remember that partial() binds positional arguments from the left, 
> which is the wrong side for what we need. Fortunately namedtuple 
> supports keyword arguments, but what about those that don't?
> 
> For the sake of the exercise, let's pretend that namedtuple doesn't 
> support keyword arguments, just to get an idea of how to solve it:
> 
> Record -> partial(lambda fields, name: namedtuple(name, fields), fields)()
> 
> And let's just hope nobody wants to call namedtuple with the other 
> arguments, verbose or rename.
> 
> It is easy to wave your hands and say "just use partial", but not so 
> easy to *actually do so*.

I'm working under the assumption that "eventually" the functions get reworked so as to return a 1-argument function.

So (using the -> syntax)

    x -> Symbol
    Point -> namedtuple_wrapper(['x', 'y'])

Becomes

    x = Symbol('x')
    Point = namedtuple_wrapper(['x', 'y'])('Point')

Adding the name as a first argument looks even more hacky to me, to be honest.

As for

    x, y, z -> expr

Should raise a syntax-error for all I care. Unless everybody agrees that it should call the naming creator 3 times? I would find it confusing, to be honest.

So it would be

    identifier -> expression

Not

    identifier_list -> expression

From michael.selik at gmail.com  Wed Jun  1 12:12:59 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 01 Jun 2016 16:12:59 +0000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EE1B0.3070106@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
 <574EE1B0.3070106@mail.de>
Message-ID: <CAGgTfkN77yv6OqsZYG2MHZ7W0oO8eEwMM+Z8WA+85W-5RiTdbQ@mail.gmail.com>

On Wed, Jun 1, 2016 at 9:23 AM Sven R. Kunze <srkunze at mail.de> wrote:

> On 01.06.2016 08:25, Sjoerd Job Postmus wrote:
> >> On 31 May 2016, at 16:37, Sven R. Kunze <srkunze at mail.de> wrote:
> >>
> >> Btw. there would be huge use-case for Django development. So, given
> your explanation, it sounds useful.
> > Could you elaborate on that use-case, because I'm not seeing it. I think
> maybe you mean the ORM and declarative forms, correct?
>
> Exactly. That is one reason why Django needs metaclasses all over the
> place. Because fields just don't know their name.
>
> This is especially true for forms as django does not magic there. It
> always causes headaches if you need it.
>

Are you referring to code like this [0]?
    your_name = forms.CharField(label='Your name')

If so, I don't think we actually want the LHS to get passed in as the
label. Not all labels will be valid identifiers. If you're talking about
the CharField knowing what attribute it has been assigned to, I couldn't
find that code in the Django source. Could you point it out?

[0] https://docs.djangoproject.com/en/1.9/topics/forms/#the-form-class
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/2dedd16d/attachment.html>

From steve at pearwood.info  Wed Jun  1 12:14:31 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 02:14:31 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
Message-ID: <20160601161431.GF12028@ando.pearwood.info>

On Tue, May 31, 2016 at 01:09:05PM -0300, Alan Cristhian wrote:
> > The fundamental use-case here is "any object that needs to know its own
> > name". Most objects need more than just a name.
> 
> A more general solution could be:
>
>     def x, y, z as <expression>
> 
> Where <expression> should return a callable that accept only one argument.

You should consider how your suggestion will work with the existing 
use-cases already shown.

We've got two dead-simple use-cases to consider. If this proposal makes 
those use-cases harder than the status quo, then this proposal is dead 
in the water. Nobody is going to use it.


# Status quo.
T = Typevar('T')
x = sympy.Symbol('x')

# My proposal.
T -> Typevar()
x -> sympy.Symbol()


# Your proposal.
def T as (lambda: Typevar)()
def x as (lambda: sympy.Symbol)()


You specify that the expression on the right must return a function that 
takes one variable, so you cannot use Typevar or Symbol directly. You 
have to call a function that returns the function you actually want.

Syntactic sugar is supposed to make things easier, not harder.



-- 
Steve

From michael.selik at gmail.com  Wed Jun  1 12:24:13 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 01 Jun 2016 16:24:13 +0000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkN77yv6OqsZYG2MHZ7W0oO8eEwMM+Z8WA+85W-5RiTdbQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
 <574EE1B0.3070106@mail.de>
 <CAGgTfkN77yv6OqsZYG2MHZ7W0oO8eEwMM+Z8WA+85W-5RiTdbQ@mail.gmail.com>
Message-ID: <CAGgTfkMdz+Z2rZ1simEN7fx5OfjQEKwFC48OhUheH2cKMQf-aQ@mail.gmail.com>

On Wed, Jun 1, 2016 at 12:12 PM Michael Selik <michael.selik at gmail.com>
wrote:

> On Wed, Jun 1, 2016 at 9:23 AM Sven R. Kunze <srkunze at mail.de> wrote:
>
>> On 01.06.2016 08:25, Sjoerd Job Postmus wrote:
>> >> On 31 May 2016, at 16:37, Sven R. Kunze <srkunze at mail.de> wrote:
>> >>
>> >> Btw. there would be huge use-case for Django development. So, given
>> your explanation, it sounds useful.
>> > Could you elaborate on that use-case, because I'm not seeing it. I
>> think maybe you mean the ORM and declarative forms, correct?
>>
>> Exactly. That is one reason why Django needs metaclasses all over the
>> place. Because fields just don't know their name.
>>
>> This is especially true for forms as django does not magic there. It
>> always causes headaches if you need it.
>>
>
> Are you referring to code like this [0]?
>     your_name = forms.CharField(label='Your name')
>
> If so, I don't think we actually want the LHS to get passed in as the
> label. Not all labels will be valid identifiers. If you're talking about
> the CharField knowing what attribute it has been assigned to, I couldn't
> find that code in the Django source. Could you point it out?
>

I tracked it down
https://github.com/django/django/blob/master/django/forms/forms.py

BaseForm looks like a Mixin that needs the subclass to implement the
collection of the base_fields class attribute. Form uses the
DeclarativeFieldsMetaclass to inspect the constructed attributes and
capture the appropriate ones as base_fields.

I don't see how that would be improved by a special syntax for the LHS to
know the identifier it's being assigned to. Could you clarify that?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/b1d99a6d/attachment.html>

From p.f.moore at gmail.com  Wed Jun  1 12:33:19 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 1 Jun 2016 17:33:19 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601161431.GF12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
Message-ID: <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>

On 1 June 2016 at 17:14, Steven D'Aprano <steve at pearwood.info> wrote:
> You should consider how your suggestion will work with the existing
> use-cases already shown.
>
> We've got two dead-simple use-cases to consider. If this proposal makes
> those use-cases harder than the status quo, then this proposal is dead
> in the water. Nobody is going to use it.

There's almost no room for improvement in those two use cases, and yet
people still like the idea of "not repeating the name". I think (but
don't know for certain) that there are some users who would accept a
cost of a few extra characters to avoid the duplication.

> # Status quo.
> T = Typevar('T')
> x = sympy.Symbol('x')
>
> # My proposal.
> T -> Typevar()
> x -> sympy.Symbol()
>
>
> # Your proposal.
> def T as (lambda: Typevar)()
> def x as (lambda: sympy.Symbol)()

That's hardly fair. The two examples you quote are the ones that
really *do* work with the "one-argument function" restriction:

def T as Typevar
def x as sympy.Symbol

If you want to knock down the alternative proposal, you should use
namedtuple or type. But for those, the people arguing for them are
assuming (for better or worse) that additional helpers with a
signature compatible with the new syntax would be added to the stdlib
at the same time as the new syntax.

> You specify that the expression on the right must return a function that
> takes one variable, so you cannot use Typevar or Symbol directly. You
> have to call a function that returns the function you actually want.
>
> Syntactic sugar is supposed to make things easier, not harder.

The biggest problem I have with your proposal is how it would extend
to more complex cases (or probably more realistically, how the
language definition would disallow cases that can't be handled).

Can you give the actual syntax of your proposal, in the form used in
the Python language reference? The best I can come up with is

IDENTIFIER '->' CALL

but CALL (see https://docs.python.org/3/reference/expressions.html#grammar-token-call)
allows the form

    function(x for x in something)

and you can't inject a name at the front of that. So I guess you'd
have to split CALL into two parts, and only allow one of them here?

Paul

From steve at pearwood.info  Wed Jun  1 12:47:32 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 02:47:32 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
Message-ID: <20160601164732.GG12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 05:33:19PM +0100, Paul Moore wrote:

> > # Your proposal.
> > def T as (lambda: Typevar)()
> > def x as (lambda: sympy.Symbol)()
> 
> That's hardly fair. The two examples you quote are the ones that
> really *do* work with the "one-argument function" restriction:

Ah yes, you're right, I misread Alan's proposal. Typevar would work, 
because it is an expression that evaluates to a callable of one 
function. I read Alan as saying that the expression would be called.



-- 
Steve

From christoph at grothesque.org  Wed Jun  1 12:48:58 2016
From: christoph at grothesque.org (Christoph Groth)
Date: Wed, 01 Jun 2016 18:48:58 +0200
Subject: [Python-ideas] async ProcessPoolExecutor
Message-ID: <877fe8lurp.fsf@grothesque.org>

Hi,

While working on a parallel numerical quadrature routine that features
speculative evaluation, I got interested in async programming in Python.
Most async programming seems to be done for network applications, but it
seems to me that controlling complex parallel computations could be also
a promising application.

The asyncio package has the run_in_executor function that seemed to be
just what I need.  When looking at how it works I noticed that
dispatching tasks from an asyncio event loop to a
concurrent.futures.ProcessPoolExecutor involves a lot of wrapping,
locking and hence complexity and inefficiency.

Shouldn't it be possible to write an async version of
ProcessPoolExecutor?

I did just this and the results seem quite promising.  Consider the
following program:

****************************************************************
from concurrent import futures
import time

def work(x):
    return x**2

def main():
    executor = futures.ProcessPoolExecutor(48)

    now = time.time()
    fs = [executor.submit(work, x) for x in range(10000)]
    print(sum(f.result() for f in futures.as_completed(fs)))
    print(time.time() - now)

if __name__ == "__main__":
    main()
****************************************************************

It takes more than 5 seconds on my machine.  It becomes even worse (7
seconds) when asyncio with run_in_executor is used.

Using the async executor, the following variant of the above script runs
in 2 seconds:
****************************************************************
import asyncio
import time
import aexecutor

def work(x):
    return x**2

async def main_coro():
    async with aexecutor.ProcessPoolExecutor(48) as executor:
        now = time.time()
        fs = [executor.submit(work, x) for x in range(10000)]
        acc = 0
        for f in asyncio.as_completed(fs):
            acc += await f
        print(acc)
        print(time.time() - now)

def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main_coro())

if __name__ == "__main__":
    main()
****************************************************************

That difference becomes less when the size of the worker pool gets
reduced, but it's significant even for a few workers.  And I am actually
interested in controlling a huge number of workers (I do computations on
a cluster with 20 nodes of 48 cores each).

What's more, the source code of the executor got reduced from around
1100 lines to 600.

Please see the attachment for the source of the "aexecutor" package as
well as the three test programs.  (All to be run with Python 3.5.)

I expect that the async executor will be useful for me.  Not just for
the gain in efficiency, but also for having a simple executor to play
with.  (I think I that I will need futures with a priority and I might
also want to write a MPI version of it.)  Perhaps this is something that
could be interesting for the wider python community?

Bear in mind that this is just a quick hack and my first project that
uses asyncio.  It's most likely buggy and non-portable (it does work on
Unix).  It can be probably simplified further and there must be a way to
improve the inner loop of the scheduler coroutine which currently looks
like this:

****************************************************************
    reader = result_queue._reader
    event = asyncio.Event(loop=loop)
    loop.add_reader(reader._handle, event.set)

    while True:
        (...)

        while True:
            await event.wait()
            if reader.poll(0):
                break
            else:
                event.clear()
        result_item = reader.recv()
        event.clear()
        (...)
****************************************************************

Any ideas?

Christoph

-------------- next part --------------
A non-text attachment was scrubbed...
Name: aexecutor-test.tar.gz
Type: application/gzip
Size: 6950 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/4452f1a3/attachment.bin>

From steve at pearwood.info  Wed Jun  1 12:49:24 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 02:49:24 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601161431.GF12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
Message-ID: <20160601164924.GH12028@ando.pearwood.info>

On Thu, Jun 02, 2016 at 02:14:31AM +1000, Steven D'Aprano wrote:

> # Your proposal.
> def T as (lambda: Typevar)()
> def x as (lambda: sympy.Symbol)()

This is wrong, I misunderstood Alan. I wrongly interpreted him as saying 
that the expression on the right would be *called* and must return a 
one-argument function, but that's not what he said.

Apologies for the confusion.


-- 
Steve

From michael.selik at gmail.com  Wed Jun  1 12:54:22 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 01 Jun 2016 16:54:22 +0000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
Message-ID: <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>

On Wed, Jun 1, 2016 at 12:33 PM Paul Moore <p.f.moore at gmail.com> wrote:

> On 1 June 2016 at 17:14, Steven D'Aprano <steve at pearwood.info> wrote:
> > We've got two dead-simple use-cases to consider.
>
> There's almost no room for improvement in those two use cases, and yet
> people still like the idea of "not repeating the name".
>

And some people don't!

> T = Typevar('T')
> > x = sympy.Symbol('x')
>

In fact, I often use namedtuple with a dynamic __name__ and fields, without
changing the identifier.

    Record = namedtuple(name, fields)

If I use that in half my code, but then in the other half, something like

    Record => namedtuple(fields)

That's TIMTOWTDI. I'd rather standardize.

Besides, the easy stuff is already easy. Let's focus on what's *hard* in
Python. I haven't seen an explanation of how this proposal would simplify
Django.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/fd86f20d/attachment-0001.html>

From p.f.moore at gmail.com  Wed Jun  1 12:59:50 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 1 Jun 2016 17:59:50 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
Message-ID: <CACac1F_e3A_9NdZ-pr=xajNjzZG10jTLonwaibnrbM6=SDKKtA@mail.gmail.com>

On 1 June 2016 at 17:54, Michael Selik <michael.selik at gmail.com> wrote:
> Besides, the easy stuff is already easy. Let's focus on what's *hard* in
> Python. I haven't seen an explanation of how this proposal would simplify
> Django.

Django's been mentioned, but nobody has explained (or shown) the code
that needs improving and would benefit from a proposal like this.

Paul

From sjoerdjob at sjoerdjob.com  Wed Jun  1 13:16:39 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Wed, 1 Jun 2016 19:16:39 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601161431.GF12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
Message-ID: <D038BB79-08A9-4B5C-8DE4-1ADFC0E9C3C4@sjoerdjob.com>



> On 1 Jun 2016, at 18:14, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> On Tue, May 31, 2016 at 01:09:05PM -0300, Alan Cristhian wrote:
>>> The fundamental use-case here is "any object that needs to know its own
>>> name". Most objects need more than just a name.
>> 
>> A more general solution could be:
>> 
>>    def x, y, z as <expression>
>> 
>> Where <expression> should return a callable that accept only one argument.
> 
> You should consider how your suggestion will work with the existing 
> use-cases already shown.
> 
> We've got two dead-simple use-cases to consider. If this proposal makes 
> those use-cases harder than the status quo, then this proposal is dead 
> in the water. Nobody is going to use it.
> 
> 
> # Status quo.
> T = Typevar('T')
> x = sympy.Symbol('x')
> 
> # My proposal.
> T -> Typevar()
> x -> sympy.Symbol()
> 
> 
> # Your proposal.
> def T as (lambda: Typevar)()
> def x as (lambda: sympy.Symbol)()

Correct me if I'm wrong, but wouldn't it be

    def T as Typevar

Unless you're being (rightfully) pedantic about "must return" implying that the expression will be called as a 0-argument function and that result called with the name?

> 
> You specify that the expression on the right must return a function that 
> takes one variable, so you cannot use Typevar or Symbol directly. You 
> have to call a function that returns the function you actually want.
> 
> Syntactic sugar is supposed to make things easier, not harder.
> 
> 
> 
> -- 
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From steve at pearwood.info  Wed Jun  1 13:26:23 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 03:26:23 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
Message-ID: <20160601172623.GI12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 04:54:22PM +0000, Michael Selik wrote:
> On Wed, Jun 1, 2016 at 12:33 PM Paul Moore <p.f.moore at gmail.com> wrote:

> > There's almost no room for improvement in those two use cases, and yet
> > people still like the idea of "not repeating the name".
> >
> 
> And some people don't!
[...]
> In fact, I often use namedtuple with a dynamic __name__ and fields, without
> changing the identifier.
> 
>     Record = namedtuple(name, fields)
> 
> If I use that in half my code, but then in the other half, something like
> 
>     Record => namedtuple(fields)
> 
> That's TIMTOWTDI. I'd rather standardize.

That's fine. Nobody is going to force you to change.

I'm not proposing any changes to namedtuple at all -- its others who 
want to change it to take just a single argument and have an wrapper 
function to specify the field names. I'm against that rather strongly.


[...]
> I haven't seen an explanation of how this proposal would simplify
> Django.

I fear I may have been mislead by Sven, who made comments about this 
being useful for Django, but hasn't given any examples yet.


-- 
Steve

From brenbarn at brenbarn.net  Wed Jun  1 14:05:42 2016
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Wed, 01 Jun 2016 11:05:42 -0700
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574EE340.7020502@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
Message-ID: <574F23F6.9060105@brenbarn.net>

On 2016-06-01 06:29, Sven R. Kunze wrote:
> Another possibility would be (requiring no syntax change):
>
>       name = function(args)
>
> would always be expanded to
>
>       name = function(args)
>       name.__name__ = "name"

	I'm really against approaches like this that involve explicitly setting 
a special dunder attribute.  What I want from this syntax is for the RHS 
expression to be able to use the string value of the variable name in 
whatever way it wants to; restricting it to setting a hard-coded dunder 
name is not solving the problem.  What if, for instance, `function` is a 
factory function that defines a class at runtime, and that class has its 
own __name__, but the class (or `function`) wants to use the variable 
name for something else?

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown

From steve at pearwood.info  Wed Jun  1 14:22:16 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 04:22:16 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <BLU403-EAS153B19DAF583780CD047BBC91460@phx.gbl>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <87k2ia8843.fsf@thinkpad.rath.org>
 <BLU403-EAS153B19DAF583780CD047BBC91460@phx.gbl>
Message-ID: <20160601182216.GK12028@ando.pearwood.info>

On Tue, May 31, 2016 at 02:08:08PM -0400, ?manuel Barry wrote:

> I think that the idea of implicit call or magically inserting items at 
> the front (or any arbitrary position, for that matter) isn't going to 
> work. For it to work, we need to address cases such as those:

I thought I had.


> x -> foo("bar")("baz") # is this foo("x", "bar")("baz") or foo("bar")("x", "baz") ?

Syntax error.

> x -> foo() # is this foo("x") or foo()("x") ?

foo('x')

> x -> foo # hmm, are we doing foo("x") here, or should this be a syntax error?

Syntax error.


> x -> foo(?, "bar") # this is foo("x", "bar")
> x -> foo(?) # foo("x")
> x -> foo(?, "bar", ?) # is this foo("x", "bar", "x") or a syntax error?
> 
> This looks fine. What about this, though?

Not too keen on a placeholder, mostly because I think YAGNI. Until 
somebody points out some real-world use-cases where the callable expects 
the name to be given in something other than the first position, I'm not 
too worried about supporting arbitrary positions. All the actual 
use-cases I've seen have the name as the first argument.


> a, b, c -> foo(?) # probably foo(("a", "b", "c")), right?
> a, b, c -> foo(?, "bar", ?, "baz", ?) # is this foo("a", "bar",  "b", "baz", "c") or a syntax error?

Another YAGNI.


> In the face of ambiguity, refuse the temptation to guess and throw a 
> syntax error in all ambiguous cases. Oh wait, that's how Python works 
> right now - every case is ambiguous for that.

Under my proposal, there's no ambiguity. Any ambiguous cases will be a 
syntax error.

The rule is simple: the right hand side must be a "simple" expression 
that ends with being called, similar to the rules for @decorator syntax. 
And then the left hand name is substituted for the first parameter.

If you don't like that, okay, but I don't think there's any ambiguity, 
any more than:

@functools.wraps(func)
def function(x, y):
    ...

is ambiguous.


> The main point is that 
> there's no way this is going to fly if there isn't an explicit way to 
> declare the implicitness

I'm afraid I don't know what that means.


> - at that point I'd rather hand-roll my own 
> function (which I can probably customize to fit the needs of my 
> module). Something like:
> 
> def set_global(func, name, *args, **kwargs):
>     globals()[name] = func(name, *args, **kwargs)

What makes you think the variables are always globals?


> In fact, I would go for some decorator magic (I love decorators):
> 
> def call_with_name(func):
>     fn, args, kwargs = func()
>     return fn(func.__name__, *args, **kwargs)
> 
> @call_with_name
> def x():
>     return namedtuple, (fields,), {}
> 
> There you go, and you only typed the name once, in the function definition. Magic enough for you?

Are you suggesting this actually works? When I try it, I get:

NameError: name 'fields' is not defined

so I think there's something you've forgotten. Besides, I *really* don't 
think that:

@call_with_name
def T:
    return Typevar, (), {}


is an improvement over:

T = Typevar('T')

but feel free to use it in your own code.



> I love syntactic sugar and Python's magic, and this would be just fine 
> for me. And, as everyone knows, code is read much more often than it 
> is written. As such,
> 
> x -> foo()
> 
> is easier to write, but much harder to read

Harder to read than the decorator you give above?

I bet that if Python didn't have a multiplication operator, and somebody 
suggested adding * for multiplication, there would be a flood of 
rejections saying

"Oh sure, 10*x is easier to write, but it's much harder to read 
than x+x+x+x+x+x+x+x+x+x. * is too confusing and magical."

Heaven help us if somebody suggested ** for exponentiation...

*wink*


> turning the fun of 
> writing something that works using compiler magic into a maintenance 
> hell.

Maintenance hell. Right. Because that's not even slightly a gross 
exaggeration.


[...]
> I also think that adding new syntax might become an attractive 
> nuisance, where people use it all the time everywhere 

Yeah, I can see it now. 

y -> x + 1  # Oh wait, that's a syntax error
print(x -> 'Hello world!')  # Oh, that's a syntax error too

I cannot imagine how you think people will be able to use this syntax 
"all the time everywhere". If anything, this syntax could be rejected as 
being too limited and not useful enough, rather than because it can be 
applied everywhere.



-- 
Steve

From steve at pearwood.info  Wed Jun  1 14:30:51 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 04:30:51 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574DD44A.8020609@brenbarn.net>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info> <574DD44A.8020609@brenbarn.net>
Message-ID: <20160601183051.GL12028@ando.pearwood.info>

On Tue, May 31, 2016 at 11:13:30AM -0700, Brendan Barnwell wrote:

> ># okay
> >name -> sympy.Symbol()
> 
> 	I don't like this that much, because it looks odd to have the 
> expression on the right without the argument.  The syntax doesn't make 
> it at all clear how the T gets used on the RHS.

*shrug*

You have to learn it, just like you have to learn decorator syntax. It's 
literally a *one sentence explanation*:

"The name on the left hand side is passed to the function on the right 
hand side, as a string, as the first positional argument."


[...]
> > But this is allowed:
> >
> > my_name -> f(g())  # like f('my_name', g())
> 
> 	So. . . what exactly will be allowed?

Guido suggested that we restrict it similar to the restrictions on 
decorator syntax. I'd be okay with that. Without looking up the syntax, 
from memory:

@decorate
@name.decorate
@factory()  # no arguments, returns a decorator
@factory(args, keyword=foo)

and possibly a few others. So by analogy:

x -> Function()
x -> module.Function()
x -> module.Function(args, keyword=foo)

etc.


-- 
Steve

From srkunze at mail.de  Wed Jun  1 14:32:44 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 20:32:44 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAG3cHaabjnmdv=LnUUmG=UWTOXfi7XLMFvs3mdVGXBq1yoze7Q@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
 <CAG3cHaabjnmdv=LnUUmG=UWTOXfi7XLMFvs3mdVGXBq1yoze7Q@mail.gmail.com>
Message-ID: <574F2A4C.4080804@mail.de>

On 01.06.2016 17:24, marky1991 . wrote:
> What is the value of __assigned_name__ here:
>
> x = y = z = AutoSymbol()
>
> ? Is it a list?

I don't think so.

It's basically the first assignment that defines the __assigned_name. If 
you want to initialize 3 symbols at once:


x, y, z = AutoSymbol(), AutoSymbol(), AutoSymbol()


Best,
Sven

From steve at pearwood.info  Wed Jun  1 14:35:23 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 04:35:23 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkP7dgrc4z5LrGY8o3H2+N7T41y2KmCMRuGWCiRNYT8yXA@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CAGgTfkP7dgrc4z5LrGY8o3H2+N7T41y2KmCMRuGWCiRNYT8yXA@mail.gmail.com>
Message-ID: <20160601183523.GM12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 12:52:58AM +0000, Michael Selik wrote:

> You brought up decorators as an example, but I think that was more about
> how decorators tend to be used essentially as part of the function
> definition. They might dramatically alter the behavior of the function and
> yet appear as an afterthought without the decorator syntax. From the PEP
> 318, "This becomes less readable with longer [functions]". The motivation
> said nothing about avoiding typing the function name twice.

Three times, not twice, and the PEP does mention it, right at the 
beginning:

"It also seems less than pythonic to name the function three times for 
what is conceptually a single declaration."

https://www.python.org/dev/peps/pep-0318/#motivation



-- 
Steve

From srkunze at mail.de  Wed Jun  1 14:36:50 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 20:36:50 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574F23F6.9060105@brenbarn.net>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <574F23F6.9060105@brenbarn.net>
Message-ID: <574F2B42.3070207@mail.de>

On 01.06.2016 20:05, Brendan Barnwell wrote:
> On 2016-06-01 06:29, Sven R. Kunze wrote:
>> Another possibility would be (requiring no syntax change):
>>
>>       name = function(args)
>>
>> would always be expanded to
>>
>>       name = function(args)
>>       name.__name__ = "name"
>
>     I'm really against approaches like this that involve explicitly 
> setting a special dunder attribute.  What I want from this syntax is 
> for the RHS expression to be able to use the string value of the 
> variable name in whatever way it wants to; restricting it to setting a 
> hard-coded dunder name is not solving the problem.  What if, for 
> instance, `function` is a factory function that defines a class at 
> runtime, and that class has its own __name__, but the class (or 
> `function`) wants to use the variable name for something else?
>

Each of both approaches do have its own advantages and drawbacks.


It seems, I need to repeat myself: the post you quoted explicitly said 
it does not care about the name of the dunder attribute. So, pick one 
appropriate, like "__to_be_assigned_lhs_name__", which should not clash 
with most usecases.


Furthermore, the same argument could be used against all dunder names.



Best,
Sven

From christoph at grothesque.org  Wed Jun  1 14:47:09 2016
From: christoph at grothesque.org (Christoph Groth)
Date: Wed, 01 Jun 2016 20:47:09 +0200
Subject: [Python-ideas] async ProcessPoolExecutor
References: <877fe8lurp.fsf@grothesque.org>
Message-ID: <87wpm8n3v6.fsf@grothesque.org>

My first post only hints on how "executor" works.  Here is some more
information:

concurrent.futures.ProcessPoolExecutor uses several queues to keep track
of the pending work.  There's one (potentially long) queue.Queue
instance that holds the work items that have been submitted.  And there
are two (short) queues from multiprocessing that are shared among the
master process and the worker processes: one for dispatching work, and
one for receiving results.  These queues are managed by a routine that
runs in a separate thread of the master process.  Because of the
separate manager thread the Futures from concurrent.futures need to be
thread-safe.

The "executor" package began as a copy of concurrent.features.
Subsequently, I replaced bits by equivalents from asyncio as far as
possible.  This necessitated also some changes to how it works, but the
interface remained mostly unchanged.  The specific changes are most
clearly visible by diffing against process.py from concurrent.futures.

The resulting package is free of locks and threads, except as used
internally by the multiprocessing.Queue instances.


From srkunze at mail.de  Wed Jun  1 14:48:34 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 20:48:34 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CACac1F8fUSSzPEdjBzmqf0DqSn5rdp+OgU2oysJ6PxACHoKzEQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
 <CACac1F8fUSSzPEdjBzmqf0DqSn5rdp+OgU2oysJ6PxACHoKzEQ@mail.gmail.com>
Message-ID: <574F2E02.9050206@mail.de>

On 01.06.2016 17:42, Paul Moore wrote:
> On 1 June 2016 at 16:10, Sven R. Kunze <srkunze at mail.de> wrote:
>> That's the current way. If now, the compiler would assign "x" to a special
>> dunder attribute/variable, that would allow __init__ to extract that name
>> and use it as if it were a parameter:
>>
>> class AutoSymbol:
>>      def __init__(self):
>>          self.name = __assigned_name__ # that's me :)
>>
> Why doesn't the assignment here set __assigned_name__ too, and
> override the value assigned by the "outer" assignment?

What's point with regards to the current discussion?

We need the name from the outer assignment. __assigned_name__ would 
solve that as it provides us with the needed piece of information. I 
don't see why an assignment on the same level where __assigned_name__ is 
used, should override this very __assigned_name__ variable. The 
assignment inside of __init__ might do so at a even deeper level but 
__assigned_name__ would stay constant *for* __init__ throughout the 
execution *of* __init__.

To me __assigned_name__ is just a magic variable as __module__ is. It is 
different from module to module. __assigned_name__ is different from 
function execution to function execution.


> Also, as someone else pointed out, in
>
> a = b = c = AutoSymbol()
>
> what would __assigned_name__ be?


That depends on which approach you choose. It's either dynamic and 
changes with each assignment or it is just set on the first assignment 
and never changes then.


I suspect most usecases need the latter. So, that speaks for the magic 
variable approach (independently of how characters are used for 
assignment operator and how we name that variable).


> The advantage of a special operation
> is that it gives you a way of pointing out precisely *which* name
> assignment we want to remember...

I don't know. I can't make a difference between = and ->. Just special 
characters. The one I know has a relatively clear meaning is =. So, I 
stick to it.


Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/e87de0f8/attachment-0001.html>

From brenbarn at brenbarn.net  Wed Jun  1 14:53:31 2016
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Wed, 01 Jun 2016 11:53:31 -0700
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601183051.GL12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info> <574DD44A.8020609@brenbarn.net>
 <20160601183051.GL12028@ando.pearwood.info>
Message-ID: <574F2F2B.70804@brenbarn.net>

On 2016-06-01 11:30, Steven D'Aprano wrote:
> On Tue, May 31, 2016 at 11:13:30AM -0700, Brendan Barnwell wrote:
>
>>> > ># okay
>>> > >name -> sympy.Symbol()
>> >
>> >	I don't like this that much, because it looks odd to have the
>> >expression on the right without the argument.  The syntax doesn't make
>> >it at all clear how the T gets used on the RHS.
> *shrug*
>
> You have to learn it, just like you have to learn decorator syntax. It's
> literally a*one sentence explanation*:
>
> "The name on the left hand side is passed to the function on the right
> hand side, as a string, as the first positional argument."

	It is simple, but if the functionality is that limited, I don't think 
I'm in favor of adding it.  Your proposal would handle essentially 
nothing but the two or three mentioned usecases, and would not be 
generalizable to others, because of the restrictions on the kinds of 
expressions allowed on the RHS.  That's a fair amount of magic to 
accomplish relatively little.

	Decorators are different, because the decorator applies to the *result* 
of the thing it wraps.  That is, the normal function definition results 
in a function object, and the decorator acts on the resulting object. 
Since the decorator is only getting one thing, there isn't much choice 
about "where" that thing should go in the call expression.  But in the 
name-passing case, the name is meant to actually be used within the 
expression on the right, so it has to get inserted before that 
expression is evaluated, and then you have questions about where to 
insert it.

	Your proposal gets around this by restricting the form of the RHS so 
that there are fewer possible insertion points.  But that's also not 
parallel to decorators, because decorators apply to functions (and 
classes), and there's no way to define more than one function (or class) 
"at the same time", so stipulating that a decorator can apply only to a 
single definition doesn't lose anything.  In other words, there's 
nothing you can do in a normal function definition that you can't do in 
a decorated function definition.  But with this new syntax, there is a 
great deal you could do in a normal expression that you can't do in a 
name-passing expression.

	I agree that your proposal is more simple and straightforward than some 
of the others that have been proposed.  But for my money, it's not worth 
adding new syntax just to handle simple cases like `x = Symbol('x')`, 
and as I see it, that's essentially all this new syntax would handle.

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown

From srkunze at mail.de  Wed Jun  1 15:04:29 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 21:04:29 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601152923.GC12028@ando.pearwood.info>
References: <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
Message-ID: <574F31BD.6070604@mail.de>

On 01.06.2016 17:29, Steven D'Aprano wrote:
> If you want instead to use a magic local variable somehow injected
> automatically into the __init__ method at runtime, I think that's very
> clever. TOO clever, and far too magical.

I consider a special syntax for this narrow kind of usecase way too much 
as it doesn't seem very flexible and extensible in the future.

Dunder methods/attributes/variables allow far more.

Especially while pondering over a reply to Michael's post, there's more 
to consider than just the name of the defining scope. One would need the 
scope itself. We could add another dunder attribute/method/variable for 
this as well; or provide a combined dunder access to it.

> Automatically providing a
> function argument is quite easy to understand, since the argument is
> right there in the method definition, and it is conceptually just like
> "self". It also means that both of these will work will work exactly the
> same way:
>
> x = Function('x', args)  # explicitly provide the name
>
> x -> Function(args)  # use the new syntax

I admit it looks looks neat but to me it's not really worth such 
restricting syntax as explained above.

> But with your magic __assigned_name__ local variable, the signature of
> the function must change depending on whether it is being called with
> one argument or two.

I don't think so:

class Symbol:
     def __init__(self, name=None):
         self.name = name or __assigned_name__

>
>> That's just it. I consider __assigned_name__ the same as __module__. A
>> special variable available if you need it.
> I'm not sure what you mean by __module__.

Sorry, if that was not clear. Here you are:

 >>> class X:
...     pass
...
 >>> X.__module__
'__main__'


So, we already have this kind of "let me know what my defining scope 
looked like" dunder attributes.


Best,
Sven

From michael.selik at gmail.com  Wed Jun  1 15:04:33 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 01 Jun 2016 19:04:33 +0000
Subject: [Python-ideas] pattern matching proof-of-concept
In-Reply-To: <574EE506.9030100@mail.de>
References: <CAGgTfkP33QPEsrC8kpSds1nLMufGGPxN+W4yWqU5sjD++5HGLQ@mail.gmail.com>
 <CACac1F_p78rFgFrNwyse2DO4Y677PwnH8nLYNPqhkTcjT+QG5g@mail.gmail.com>
 <574EE506.9030100@mail.de>
Message-ID: <CAGgTfkNsDHCx7aNyUGB+yDzg3hqNfzEMSomUvr5a4=YiBh6mSw@mail.gmail.com>

On Wed, Jun 1, 2016 at 9:37 AM Sven R. Kunze <srkunze at mail.de> wrote:

> more flexible than a monolithic switch-case
>

That wasn't my initial intention, but I guess that's a side-effect of
needing to break it down into pieces viable in current syntax.

Thanks a lot, Michael.
>

You're welcome. I hope the effort informs the discussion of a special
matching syntax.


Some things that really stood out as tough to implement or not possible.

1. Avoid NameErrors when specifying identifiers to bind in a schema.

I see two options to implement this in current syntax. I chose to require a
binding object and all identifiers must be attributes of that binding
object. Another option would be postponing the lookup of the identifiers by
requiring the schema to be defined in a function, then doing some magic.

2. Matching an object and unpacking some of its attributes.

If the schema is a fully-specified object, we can rely on its __eq__ to do
the match. If the schema specifies only the type and does not unpack
attributes, that's even easier. The tough part is a schema that has a
half-specified object where where its __eq__ cannot be relied on. If all
Unbounds are equal to everything, that would help a half-specified schema
compare well, but it breaks all sorts of other code, such as
``list.index``. I chose to compare all public, non-Unbound attributes, but
that may not have been the implementation of its __eq__. Alternatively, one
could trace __eq__ execution and revise Unbound comparisons, but that's
beyond my wizarding abilities.

3. Unpacking the attributes of an object that aggressively type-checks
input.

Given my solution of a Binding object and unpacking into its attributes,
one cannot, for example, unpack the first argument of a ``range`` as it
raises a TypeError: ``bind=Binding(); schema = range(bind.x)``.

I'm considering removing object-unpacking from the module. To match and
unpack an object, the schema could be specified as a mapping or sequence
and the user would transform the object into a mapping (like
``obj.__dict__``) before doing the match.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/b8a7b5e1/attachment.html>

From srkunze at mail.de  Wed Jun  1 15:06:36 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 21:06:36 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601151946.GB12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz>
 <20160601151946.GB12028@ando.pearwood.info>
Message-ID: <574F323C.4000006@mail.de>



On 01.06.2016 17:19, Steven D'Aprano wrote:
> On Wed, Jun 01, 2016 at 11:10:19AM +1200, Greg Ewing wrote:
>> Steven D'Aprano wrote:
>>> name -> Function(args)
>>>
>>> will be expanded to:
>>>
>>> name = Function('name', args)
>> Another possibility would be to expand it to
>>
>>     name = Function(args)
>>     name.__name__ = "name"
> That priviledges one specific use of the name over all others. You'll
> note that I was very careful to describe the use-case as "objects that
> need to know their own name" but didn't specify what they did with the
> name. Maybe they bind it to self.__name__, but maybe they use self.name
> instead. Or they write it to a database. Who knows?
>
> I don't think we should specify what the object does with the name.
>

A dunder method could help here. So, the object can decide what to do 
with it.


Best,
Sven

From michael.selik at gmail.com  Wed Jun  1 15:12:25 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 01 Jun 2016 19:12:25 +0000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601183523.GM12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CAGgTfkP7dgrc4z5LrGY8o3H2+N7T41y2KmCMRuGWCiRNYT8yXA@mail.gmail.com>
 <20160601183523.GM12028@ando.pearwood.info>
Message-ID: <CAGgTfkMEi=M7mLKnnJT9wmeFbDp16qMeDJcYxKZYsyZ5SxDf0g@mail.gmail.com>

On Wed, Jun 1, 2016 at 2:35 PM Steven D'Aprano <steve at pearwood.info> wrote:

> the PEP does mention it, right at the beginning:
>
> "It also seems less than pythonic to name the function three times for
> what is conceptually a single declaration."
>

That's what I get for reading too fast. Still, it does come after the
sentence "This becomes less readable with longer methods," making the
retyping-avoidance feel like an afterthought.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/7f162989/attachment.html>

From srkunze at mail.de  Wed Jun  1 15:14:19 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 1 Jun 2016 21:14:19 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkMdz+Z2rZ1simEN7fx5OfjQEKwFC48OhUheH2cKMQf-aQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
 <574EE1B0.3070106@mail.de>
 <CAGgTfkN77yv6OqsZYG2MHZ7W0oO8eEwMM+Z8WA+85W-5RiTdbQ@mail.gmail.com>
 <CAGgTfkMdz+Z2rZ1simEN7fx5OfjQEKwFC48OhUheH2cKMQf-aQ@mail.gmail.com>
Message-ID: <574F340B.4080008@mail.de>

On 01.06.2016 18:24, Michael Selik wrote:
> I don't see how that would be improved by a special syntax for the LHS 
> to know the identifier it's being assigned to. Could you clarify that?

I wasn't referring to labels here although somebody might find that 
useful too.


I was referring to the issue that fields don't know neither their 
containing form classes nor their form instances. In some circumstances, 
one need to do that in order to correctly work with the field (think of 
the prefix of the form which needs to be reflected in the field but 
there are more missing pieces of information).


As mentioned to Steven, in this case the name does not suffices at all. 
So, I am not sure how to handle this right now. This might belong to a 
greater issue of *referring back* which currently needs to be handled 
manually in all cases.


Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/529fc3e7/attachment.html>

From k7hoven at gmail.com  Wed Jun  1 15:22:39 2016
From: k7hoven at gmail.com (Koos Zevenhoven)
Date: Wed, 1 Jun 2016 22:22:39 +0300
Subject: [Python-ideas] Object for accessing identifiers/names
Message-ID: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>

Inspired by the discussions
regarding
 Guido's
?"M?
atch statement
? brainstorm?
" and
?"?
Quick idea: defining variables from functions..." threads, here's an idea
regarding names/identifiers.

Currently, there is no direct way to work with variable names from within
Python. Yes, you can fiddle with __dict__s and locals() and globals(), but
there is no
?convenient ?
general way. To solve this, there could be a way
?(probably new syntax)?
 for creating an object that
??
can examine and manipulate a name binding conveniently.

The object could be created for instance as follows:

? ?
name_object = identifier some_name

However, 'identifier' is so long that I'll use the keyword 'def' instead,
regardless of whether it is optimal or not:

?  ?
name_obj = def some_name

Now this name_obj
? thing?
could provide functionality like assigning to the name some_name, getting
the assigned object, and determining whether
?something has been assigned to the name or not
. The object would also be aware of the name 'some_name'.

?T
he functionality might
?work
 as follows:

?  ?
bool(
?name_obj
)
?   ?
# True if
?something? is
 assigned to some_name
?  ?
name_obj.name()

?    ?
# == 'some_name'
??
?  name_obj
.set(value) #
?equivalent to `some_name = value`?
??
?  name_obj.get()      # equivalent to just `some_name?`
  name_obj.unset()    # like `del some_name`

??Then you could pass this to a function:

  func(name_obj)

Now func will be able to assign to the variable some_name, but with
some_name referring to the scope where `name_obj = def some_name` was
executed. This is similar to things that can be done with closures.

This would also allow things like:

  if def some_name:
      #do stuff if the name some_name is bound to something

or

  if not def some_name:
      some_name = something()


- Koos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/e7a19faf/attachment.html>

From michael.selik at gmail.com  Wed Jun  1 15:48:56 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 01 Jun 2016 19:48:56 +0000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574F340B.4080008@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
 <574EE1B0.3070106@mail.de>
 <CAGgTfkN77yv6OqsZYG2MHZ7W0oO8eEwMM+Z8WA+85W-5RiTdbQ@mail.gmail.com>
 <CAGgTfkMdz+Z2rZ1simEN7fx5OfjQEKwFC48OhUheH2cKMQf-aQ@mail.gmail.com>
 <574F340B.4080008@mail.de>
Message-ID: <CAGgTfkOXFHM8rsx135vkLcz8jeDw-qizG7BwEYrhw89Logp2Mg@mail.gmail.com>

On Wed, Jun 1, 2016 at 3:14 PM Sven R. Kunze <srkunze at mail.de> wrote:

> On 01.06.2016 18:24, Michael Selik wrote:
>
> I don't see how that would be improved by a special syntax for the [RHS]
> to know the identifier it's being assigned to. Could you clarify that?
>
> I was referring to the issue that fields don't know neither their
> containing form classes nor their form instances. In some circumstances,
> one need to do that in order to correctly work with the field (think of the
> prefix of the form which needs to be reflected in the field but there are
> more missing pieces of information).
>
> As mentioned to Steven, in this case the name does not suffices at all.
> So, I am not sure how to handle this right now. This might belong to a
> greater issue of referring back which currently needs to be handled
> manually in all cases.
>

I always find bi-directional has-a relationships to be troublesome. If it
were necessary frequently, I'd expect the Form metaclass to loop over the
declared_fields and set some .form attribute to the new_class. Django is
way too big for me to grok from light reading, so I won't presume to
understand the choices made there.

https://github.com/django/django/blob/master/django/forms/forms.py#L27
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/e48be266/attachment-0001.html>

From random832 at fastmail.com  Wed Jun  1 16:23:59 2016
From: random832 at fastmail.com (Random832)
Date: Wed, 01 Jun 2016 16:23:59 -0400
Subject: [Python-ideas] Object for accessing identifiers/names
In-Reply-To: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
References: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
Message-ID: <1464812639.3025635.625194193.0B50E2B5@webmail.messagingengine.com>

On Wed, Jun 1, 2016, at 15:22, Koos Zevenhoven wrote:
> name_object = identifier some_name
...
> This is similar to things that can be done with closures.

It sounds like for locals this could be implemented by adding your
methods to the cell type (right now cell objects cannot be modified from
python code), and having this statement return the cell.

Any local variable that is used in this way gets a cell, the same as any
local that is used in a closure does now.

From robertc at robertcollins.net  Wed Jun  1 16:47:47 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Thu, 2 Jun 2016 08:47:47 +1200
Subject: [Python-ideas] Declaring immutability (was Re: Proposal to
 change List Sequence Repetition (*) so it is not useless for Mutable
 Objects)
In-Reply-To: <CADiSq7dmwtpyG62kdLVYs1CysupfG6g0257fxV_O4bKAp9ifRQ@mail.gmail.com>
References: <CADiSq7fhj8mWc9vethm1Hqnf41w6E8qSAQgcMYr8Yd-qHqfKmg@mail.gmail.com>
 <CADiSq7eQsv+CLoCtK5NgBX+QpL1rK=zRdn==hVn2tn1ypM5GQg@mail.gmail.com>
 <CADiSq7cERSuyuRJkyw+DbZ7TL7VcA+t1g=sQEzDkf9fjrinboA@mail.gmail.com>
 <CADiSq7cipatBxuh5uj_LvfjjCPVrZ8vz2K9GjWeqd-TV__oeTQ@mail.gmail.com>
 <CADiSq7eOa4XWwYf5tsa-RaEao-rJmy0y5ReB=OUBP3BbU9tUjQ@mail.gmail.com>
 <CADiSq7fhgzpNRTm-sfHmv8F2tfVV4iBd=EzAjChB8EAsgYcd7g@mail.gmail.com>
 <CADiSq7d8vNQpVYus5DTEJKhWRUeiUJW+nq7qZSeH4bcz8-5N3g@mail.gmail.com>
 <CADiSq7fijo13k_P-Rtv3ZWFLVhWVven53SJif5Q8JQ4=+rZcTg@mail.gmail.com>
 <CADiSq7e7YaEsZA=42bwFYv65wPkfwHfdMQKm+xu+qMwhvr=aFA@mail.gmail.com>
 <CADiSq7d0+SA2jyGeLNdkymdE62bJL40FZ5W=FsZmARtG1mKvAA@mail.gmail.com>
 <CADiSq7fp2-J43FweyvRKaptH8eJSndeGWPMgRNg7+ZD9LtNi-g@mail.gmail.com>
 <CADiSq7c4FnERfdvOUHw04DkeULXTr2CKPUqDQTX__MTYxno1-Q@mail.gmail.com>
 <CADiSq7cwtgNN3oUGDGVFtUZfySOeDHArXXwQpihSiewQYhgs7w@mail.gmail.com>
 <CADiSq7dWVeC1A+b+FTrAsgFzM9y=YVmzNdQ=S8OCRjWXzN=XQA@mail.gmail.com>
 <CADiSq7e7UTE-Hfz1w-Xr+TDwc==sZcDCzv=cUs+Oe4vA5Xy3VQ@mail.gmail.com>
 <CADiSq7ddcSZ4aV8GU7Y3231cVJej6oZwLVLcdyNT15MnifVTrg@mail.gmail.com>
 <CADiSq7ek6YuBxkFrJ-S0y1WMaHE5TZbXOCUWJ2JCV9zAXPtOnQ@mail.gmail.com>
 <CADiSq7cCm0Q_5TjWvv_L7jDdSo5bpd-_npuA0fyac4im6q_E1Q@mail.gmail.com>
 <CADiSq7eL5790tDijLZ062NRH-ZofyH18n3TGu3c9uZSD2ymLqA@mail.gmail.com>
 <CADiSq7fhvuq_VDCTFckkp0uxubC5N5dB5LQCZjWRKgpfyRC+yg@mail.gmail.com>
 <CADiSq7dP=DiOPtpqGot6jd6z57xzYku_xYaJGEjsRwmPCL=JsA@mail.gmail.com>
 <CADiSq7e3VCAB41Di9pC+UKzaDWhXKUkHd+fqKfJ5JzgP509eYg@mail.gmail.com>
 <CADiSq7cgmCoua2_QiMYgsbbgODzm93pqc+r0jsj1UN4pT=kLig@mail.gmail.com>
 <CADiSq7fA7uDuk94VTMSBXgf_6HqOL+84Z1O1h453J2w7EmjK3A@mail.gmail.com>
 <CADiSq7eeEvx2kTgEt+PpZPQ_Fk4ZoUOyT6qxhtcTw0LeSaAB6A@mail.gmail.com>
 <CADiSq7cfDWKfhhaqJ3mX_eqEwL7V2fvVSLFyXJivxcWZ_7d6_w@mail.gmail.com>
 <CADiSq7cmj=b39uywL+F6ZQ3b+hNYNbhL_v2N6OMJfRSr0+6+Yw@mail.gmail.com>
 <CADiSq7dmwtpyG62kdLVYs1CysupfG6g0257fxV_O4bKAp9ifRQ@mail.gmail.com>
Message-ID: <CAJ3HoZ1pNEYkwmahPwZFihdVrMWwiJCAoCW8ssg+m8eJUaaoBQ@mail.gmail.com>

On 1 Jun 2016 11:04 PM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
>
>
>... immutable instances, so this is likely to be better approached as a
type hinting problem rather than as a runtime feature.

+1

> The closest related concept I'm personally familiar with is const
pointers and references in C and C++, so there

I think rust borrows are a likely better fit for inspiration.

-rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/017be82a/attachment.html>

From alan.cristh at gmail.com  Wed Jun  1 16:48:31 2016
From: alan.cristh at gmail.com (Alan Cristhian)
Date: Wed, 1 Jun 2016 17:48:31 -0300
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574F323C.4000006@mail.de>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de>
 <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz>
 <20160601151946.GB12028@ando.pearwood.info>
 <574F323C.4000006@mail.de>
Message-ID: <CAPncOvL5A3_hPwUk7DVy7JsVZuzAm0Pzi6MAQRw+eWk8ixBkCQ@mail.gmail.com>

> > > # Your proposal.
> > > def T as (lambda: Typevar)()
> > > def x as (lambda: sympy.Symbol)()
> >
> > That's hardly fair. The two examples you quote are the ones that
> > really *do* work with the "one-argument function" restriction:
>
> Ah yes, you're right, I misread Alan's proposal. Typevar would work,
> because it is an expression that evaluates to a callable of one
> function. I read Alan as saying that the expression would be called.

I still have trouble with english :)

My proposal is:

    def x as <expression-that-creates-a-callable>  # single variable
    def x, y, z as <expression-that-creates-a-callable> # multiple variables

Where <expression-that-creates-a-callable> should be evaluated and assigned
to a temporary variable. E.g:

    def T as typing.TypeVar

Where "typing.TypeVar" it's what I mean with
<expression-that-creates-a-callable>.

Then the statement

   def T as typing.TypeVar

Could be semantically equivalent to

    _temporary = typing.TypeVar
    T = _temporary("T")

This solves the complex use cases when you need to use more than one
argument. For example, in sympy you can indicate that you want an integer
symbolic variable:

    x = Symbol("x", integer=True)

Whit my syntax you can do:

    SymbolicInteger = lambda name: Symbol(name, integer=True)
    def x as SymbolicInteger

But for me, this is useless. When I use sympy, in all cases I assign more
than one variable because the "Functions of Several Variables" are very
common. If you go to the live interactive sympy console (
http://live.sympy.org) you will find this tree lines:

   >>> x, y, z, t = symbols('x y z t')
   >>> k, m, n = symbols('k m n', integer=True)
   >>> f, g, h = symbols('f g h', cls=Function)

That is the reason by which I propose the alternative whit multiple
variables. Sympy have the "Symbol" class that creates one symbolic variable
and the "symbols" function that creates multiple variables at once.

The full proposal is:

    def x, y, z as <expression-that-creates-a-callable>

That should be semantically equivalent to

    _temporary = <expression-that-creates-a-callable>
    x = _temporary("x")
    y = _temporary("y")
    z = _temporary("z")

I think that my proposal solves more use cases, don't need new keywords and
is more like english grammar.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/c9b78f92/attachment.html>

From greg.ewing at canterbury.ac.nz  Wed Jun  1 16:50:11 2016
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Jun 2016 08:50:11 +1200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601161431.GF12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
Message-ID: <574F4A83.3060207@canterbury.ac.nz>

Steven D'Aprano wrote:
> # Your proposal.
> def T as (lambda: Typevar)()
> def x as (lambda: sympy.Symbol)()

No, you would write those as

    def T as TypeVar
    def x as sympy.Symbol

 > Syntactic sugar is supposed to make things easier, not harder.

Which it does, because you have less parentheses to type!

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Jun  1 17:01:03 2016
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Jun 2016 09:01:03 +1200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601172623.GI12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
Message-ID: <574F4D0F.8000809@canterbury.ac.nz>

Steven D'Aprano wrote:
> I'm not proposing any changes to namedtuple at all -- its others who 
> want to change it to take just a single argument and have an wrapper 
> function to specify the field names. I'm against that rather strongly.

I don't think this would be creating TMWTDI. Given the
proposed feature, and a suitable helper, then

   def Foo as NamedTuple('x', 'y', 'z')

would become the *obvious* way to define a named tuple
type. The old way

   foo = namedtuple(something, 'x', 'y', 'z')

would become something low-level that you only use in
special circumstances -- like calling type() directly
to create a class.

-- 
Greg

From rob.cliffe at btinternet.com  Wed Jun  1 18:26:07 2016
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Wed, 1 Jun 2016 23:26:07 +0100
Subject: [Python-ideas] Wild idea about mutability
Message-ID: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>

I was prompted to post this after seeing comments on "Proposal to change 
List Sequence Repetition (*) so it is not useless for Mutable Objects" 
that you cannot in general tell whether an object is mutable or not.

Well, why shouldn't you be able to tell?  Python is superb at 
introspection, but ISTM there's a gap here.  Here's an idea for Python N 
(N>=4):  Every object has a boolean __mutable__ attribute.  It would 
e.g. be False for ints and strings,  True by default for objects that 
can be mutable, such as lists and dicts. It could be an optional extra 
argument to mutable object constructors (so you have the option of 
making them immutable):

     L = list(somesequence, mutable=False)

There could also be a syntax for list/set literals and dictionary 
displays to indicate that they should be immutable (just as we preface 
literal strings with 'r' to indicate raw strings).  I'm not sure what; I 
thought of f[1,2,3] for a literal immutable ("frozen") list, but that's 
already valid syntax.

You can "freeze" a mutable object by changing its __mutable__ attribute 
from True to False.  You are not allowed to change it from False to 
True, nor to mutate an object that has it False.

I am sure there must be advantages in being able to tell if an object is 
mutable or not.  (Perhaps "hashable" equals "not mutable" ?)

Now:  You don't need the frozenset class.  You just have a set with 
__mutable__ = False.  And you can freeze dicts (classes? modules?) etc., 
to make sure users of your code don't mess with them.

And (fasten your safety belts): You don't need tuples.  You just have a 
list with __mutable__ = False.  And you don't have to explain to 
beginners one of the most FAQ "Why do we need lists and tuples?"

Best wishes (and sorry I can't seem to get my e-mail out of double 
vertical spacing mode),

Rob Cliffe



From random832 at fastmail.com  Wed Jun  1 19:29:28 2016
From: random832 at fastmail.com (Random832)
Date: Wed, 01 Jun 2016 19:29:28 -0400
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
Message-ID: <1464823768.2906208.625337665.41A15F1E@webmail.messagingengine.com>

On Wed, Jun 1, 2016, at 18:26, Rob Cliffe wrote:
> And (fasten your safety belts): You don't need tuples.  You just have a 
> list with __mutable__ = False.  And you don't have to explain to 
> beginners one of the most FAQ "Why do we need lists and tuples?"

Er... so does setting __mutable__ = False cause/require contained
objects to be transitively mutable? Because if it does, you still need
tuples to have an "immutable list of mutable objects". If it doesn't,
then this doesn't solve the other thread's problem.

From jcgoble3 at gmail.com  Wed Jun  1 19:50:33 2016
From: jcgoble3 at gmail.com (Jonathan Goble)
Date: Wed, 1 Jun 2016 19:50:33 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574F4D0F.8000809@canterbury.ac.nz>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
Message-ID: <CAK256p3Gx=gHEwxek24rxu5Td_xVujxwcPzLvVCkvqAts1wbhQ@mail.gmail.com>

I have a completely different idea here. Forget clunky syntax, overloading
of 'def' and 'as', and all that. Introduce the magic parameter '$'. To wit:

    def func(*args, **kwargs, $):  # positioning of the $ is discussed below
        ...  # function code here

$ would be a special-cased parameter that does not receive an explicit
argument, and is thus ignored when calling the function (i.e. the function
above would be called simply as 'func(*args, **kwargs)'). Instead, the $
parameter receives as its value a string representing the name that the
function's return value will be bound to, or None if the function is called
as a statement (where the return value is simply thrown away). Now any
function (e.g. namedtuple) requiring the name can simply toss in a $
parameter, and it automatically receives the appropriate name through
compiler magic. So assuming that namedtuple is patched to take a $
parameter instead of an explicit argument with the name, this common code
today:

    Record = namedtuple('Record', ['name', 'address', 'phone', 'age'])

becomes simply this:

    Record = namedtuple(['name', 'address', 'phone', 'age'])

Some open questions here are where to require the $ to be placed in the
argument list, and whether and how an advanced user can override it if
needed. For the former, I propose to require it to be the last argument,
after **kwargs. I considered making it the first argument, but that would
create a competition for first argument in a class method or instance
method (particularly __init__). Making it last also enables it to be
overridden as a keyword-only argument. If the user really wanted to, they
could change the above namedtuple call to:

    Record = namedtuple('name, address, phone, age', $='BusinessCard')

and $ would receive the explicit argument provided without compiler magic
being applied.

Another concern is backward compatibility for namedtuple and other
functions during the transition to the $ parameter. The easy solution would
seem to be to just keep the explicit parameter and deprecate/ignore it, but
there is then no way to transition to eliminating it completely since new
code will have to continue providing a useless argument (in the common case
that it is the first parameter), or else the function will have to give
default values to every argument so that the other arguments can be passed
as keyword arguments (which is not always desirable or practical), and
sufficient time given for all users to switch to keywords, before the old
first argument can be deleted in a subsequent release.

Instead, I suggest that the recommended method of transition ought to be to
simply provide new functions and deprecate the existing functions (except
where they are already built to handle omitting the explicit parameter in
the call, if such cases exist) in a minor release, and remove the
deprecated functions in a subsequent major release.

Thoughts? (I'm sunburned after spending six hours in a waterpark today on
four hours of sleep last night, so if I sound crazy, that's why.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160601/87e8d567/attachment.html>

From xavier.combelle at gmail.com  Wed Jun  1 21:12:47 2016
From: xavier.combelle at gmail.com (Xavier Combelle)
Date: Thu, 2 Jun 2016 03:12:47 +0200
Subject: [Python-ideas] Match statement brainstorm
In-Reply-To: <20160530183517.GQ12028@ando.pearwood.info>
References: <CAGgTfkPVzLYhugMO8Wguv24CyPxcC_Fkp3fnqsEBBTT7BBtXHQ@mail.gmail.com>
 <CACac1F8hi8Dwwb6eQ_mdj1P6SsFeNnPae_XaVFj_ZicR5G3ZYw@mail.gmail.com>
 <22341.12808.512932.821488@turnbull.sk.tsukuba.ac.jp>
 <CAPTjJmozK-gz7xrBXE-xYhs58_Ds1Qyh4mBY_RSg61gktSiebQ@mail.gmail.com>
 <CADiSq7fjO0EjnuUR-gLGePS-OM-MHcKq+6kAFO8+7griPdC=ug@mail.gmail.com>
 <CAB_e7iy7-Ab43fweiVarahR9i+dv3nKQoQXi3hpPXD8Qc3ZA+A@mail.gmail.com>
 <87shx4d6ip.fsf@thinkpad.rath.org>
 <57478250.6000707@canterbury.ac.nz>
 <22343.62785.209111.270692@turnbull.sk.tsukuba.ac.jp>
 <5748FECD.2030203@canterbury.ac.nz>
 <20160530183517.GQ12028@ando.pearwood.info>
Message-ID: <CAEQcUJT0-QLKXLrYBEj99LCqmOzWgpUuF87p+WSnrTMR+B0JPA@mail.gmail.com>

What about a `__match__()` magic function:
so

given object:
     case Point(x,y,3):print(x,y)
     case Point(1,2,3):something_magic()
     case x,y,Point(a,b,c),1:this_is_my_tuple()


be same than writing

try:
   x,y=Point.__match__(object,("x","y",None),(None,None,3))
except MatchError:
    try:
         Point.__match__(object,(None,None,None),(1,2,3))
    except MatchError:
         try:
              x,y,var1,var2=object

              #literal testing
              if var2 != 1:
                    raise MatchError()

              a,b,c=Point.__match__(var1,("a","b","c"),(None,None,None))
              #var1 and var2 would not be really created, they are
placeholders
         except MatchError:
              #nodefault
              raise
          else:
              this_is_my_tuple
    else:
         something_magic()
else:
   print(x,y)

and let the __match__ functions know what to do to check if arguments are
correct

I purposely did let drop the dictionary and keyword arguments cases,
finding them too
much hard to understand

2016-05-30 20:35 GMT+02:00 Steven D'Aprano <steve at pearwood.info>:

> On Sat, May 28, 2016 at 02:13:33PM +1200, Greg Ewing wrote:
> [...]
> > Finally we could allow "case class" to be abbreviated to
> > just "class":
> >
> >    switch obj:
> >       class Intersection(
> >          class Line(class Point(?x1, ?y1), class Point(?x2, ?y2)),
> >          class Line(class Point(?x3, ?y3), class Point(?x4, ?y4))):
> >
> > Is that unacceptably verbose? I don't know.
>
> Verbose, not so much. Cryptic? Hell yes! Who is going to be able to
> guess what it means? We're no longer even in the same galaxy as
> executable pseudo-code.
>
> Try this as an exercise: given the above, explain in plain English what
> it does and what the result will be. What exactly is being matched?
>
> Those familiar with C-like switches are going to be totally confused.
> They'll probably wonder if you are matching "if obj == the class
> Intersection", and have no idea what's going on with the nested Line and
> Point stuff.
>
> Those familiar with classes and the class keyword are going to wonder
> what class definitions are doing inside the class declaration. Does this
> mean we can now do this?
>
> class Child(class Parent: pass):
>     def method(self): ...
>
> Obviously not, but that's what it looks like.
>
> I wonder whether the real problem here is that pattern matching as a
> concept is simply too concise for mere mortals? Once you get past the
> trivial Haskell-esque:
>
> def factorial(n):
>     switch n:
>         0: return 1
>         n: return n*fact(n-1)
>
> its just doing too much in too little code to be easily comprehensible.
>
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/704f798e/attachment-0001.html>

From xavier.combelle at gmail.com  Wed Jun  1 21:34:41 2016
From: xavier.combelle at gmail.com (Xavier Combelle)
Date: Thu, 2 Jun 2016 03:34:41 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAK256p3Gx=gHEwxek24rxu5Td_xVujxwcPzLvVCkvqAts1wbhQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
 <CAK256p3Gx=gHEwxek24rxu5Td_xVujxwcPzLvVCkvqAts1wbhQ@mail.gmail.com>
Message-ID: <CAEQcUJTV45+8--rQN7Uo75JuRzmnjrm7NUVWXPcUkR5zesztyA@mail.gmail.com>

2016-06-02 1:50 GMT+02:00 Jonathan Goble <jcgoble3 at gmail.com>:

> I have a completely different idea here. Forget clunky syntax, overloading
> of 'def' and 'as', and all that. Introduce the magic parameter '$'. To wit:
>
>     def func(*args, **kwargs, $):  # positioning of the $ is discussed
> below
>         ...  # function code here
>


Why not a method similar to what is done with *args and **kwargs
(I chose pretty arbitrarely /namearg)

so for definition it would be
    def func(*args, **kwargs, /namearg):

and for override it would be
Record = namedtuple('name, address, phone, age', /'BusinessCard')
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/1215990c/attachment.html>

From steve at pearwood.info  Wed Jun  1 21:34:20 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 11:34:20 +1000
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
Message-ID: <20160602013420.GN12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 11:26:07PM +0100, Rob Cliffe wrote:
> I was prompted to post this after seeing comments on "Proposal to change 
> List Sequence Repetition (*) so it is not useless for Mutable Objects" 
> that you cannot in general tell whether an object is mutable or not.
> 
> Well, why shouldn't you be able to tell?  Python is superb at 
> introspection, but ISTM there's a gap here.  Here's an idea for Python N 
> (N>=4):  Every object has a boolean __mutable__ attribute.  It would 
> e.g. be False for ints and strings,  True by default for objects that 
> can be mutable, such as lists and dicts.

I hate to break it to you, but merely setting a flag on an object 
doesn't make it mutable or immutable *wink*

That means that every class (possibly including builtins) needs to be 
re-written so that every mutator method checks this flag and decides 
whether or not to allow the mutation. For example, you suggest:

"You don't need tuples.  You just have a list with __mutable__ = False."

Now the code for list append must be:

    def append(self, item):
        # only written in C, because its a built-in
        if self.__mutable__:
            ...
        else:
            raise SomeError

Now multiply that by *every* mutator class and method. This will be 
especially burdensome for people writing classes intended to be mutable, 
since they have to support immutability whether they want it or not.

(Since the caller might change obj.__mutable__ to False, which is 
allowed.)

This will break type-checking and duck-typing and make feature 
detection a pain:

# currently this works
if hasattr(obj, 'append'):
    handle_things_that_can_append(obj)
else:
    handle_things_that_cant_append(obj)

# with your scheme
if hasattr(obj, 'append'):
    if obj.__mutable__:
        handle_things_that_can_append(obj)
    else:
        handle_things_that_cant_append(obj)
else:
    handle_things_that_cant_append(obj)


> It could be an optional extra 
> argument to mutable object constructors (so you have the option of 
> making them immutable):
> 
>     L = list(somesequence, mutable=False)

This goes against the "no constant bool arguments" design guideline.


> There could also be a syntax for list/set literals and dictionary 
> displays to indicate that they should be immutable (just as we preface 
> literal strings with 'r' to indicate raw strings).  I'm not sure what; I 
> thought of f[1,2,3] for a literal immutable ("frozen") list, but that's 
> already valid syntax.

We already have syntax for a built-in immutable list-like sequence 
object (a "frozen list" if you will):

(1, 2, 3)


> You can "freeze" a mutable object by changing its __mutable__ attribute 
> from True to False.  You are not allowed to change it from False to 
> True, nor to mutate an object that has it False.
> 
> I am sure there must be advantages in being able to tell if an object is 
> mutable or not.  (Perhaps "hashable" equals "not mutable" ?)

No it does not.

(1, 2, [3]) is both immutable and unhashable.


> Now:  You don't need the frozenset class.  You just have a set with 
> __mutable__ = False.  And you can freeze dicts (classes? modules?) etc., 
> to make sure users of your code don't mess with them.

And we get this functionality for free, right? *wink*

Somebody has to write this code. I'm not sure they will appreciate 
having to throw away all the perfectly good frozenset code and tests, 
and re-writing set to handle both cases. Not to mention the breaking of 
backwards compatibility to get rid of frozenset.


> And (fasten your safety belts): You don't need tuples.  You just have a 
> list with __mutable__ = False.  And you don't have to explain to 
> beginners one of the most FAQ "Why do we need lists and tuples?"

(1) I don't remember ever seeing a beginner confused by this difference, 
but I see plenty of experienced programmers who are sure that they must 
be.

(2) Have you actually read the FAQ? Because if you have, you should 
realise that having an immutable list is not the same as having a tuple.

Tuples are intended for "record-like" structures, where the items' 
positions are meaningful, while lists are intended for "array-like" 
structures where all the items have the same type and their position is 
not meaningful. While it's not compulsory to treat tuples and lists 
this way, it is very useful to do so.



-- 
Steve

From brenbarn at brenbarn.net  Wed Jun  1 21:40:18 2016
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Wed, 01 Jun 2016 18:40:18 -0700
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAK256p3Gx=gHEwxek24rxu5Td_xVujxwcPzLvVCkvqAts1wbhQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
 <CAK256p3Gx=gHEwxek24rxu5Td_xVujxwcPzLvVCkvqAts1wbhQ@mail.gmail.com>
Message-ID: <574F8E82.5010308@brenbarn.net>

On 2016-06-01 16:50, Jonathan Goble wrote:
> I have a completely different idea here. Forget clunky syntax,
> overloading of 'def' and 'as', and all that. Introduce the magic
> parameter '$'. To wit:
>
>      def func(*args, **kwargs, $):  # positioning of the $ is discussed
> below
>          ...  # function code here
>
> $ would be a special-cased parameter that does not receive an explicit
> argument, and is thus ignored when calling the function (i.e. the
> function above would be called simply as 'func(*args, **kwargs)').
> Instead, the $ parameter receives as its value a string representing the
> name that the function's return value will be bound to, or None if the
> function is called as a statement (where the return value is simply
> thrown away).

	I think that's way too general and dangerous.  I don't want arbitrary 
objects to be able to do arbitrary things based on what name I'm 
assigning them to.  I think the fact that Python doesn't allow 
overriding simple assignment, although it makes some things difficult, 
makes it simpler to reason about in many cases.  If we do add something 
to handle this case, I'd rather it be something explicit, so you know 
when you're crossing the barrier between identifiers and string values.

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown

From Nikolaus at rath.org  Wed Jun  1 23:23:21 2016
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 01 Jun 2016 20:23:21 -0700
Subject: [Python-ideas] Unpacking a dict
In-Reply-To: <CACac1F9HfQU4cy8j80Pqnq9D-2YYPzAjg-4Zd_4VQ=8vcj+KGA@mail.gmail.com>
 (Paul Moore's message of "Tue, 31 May 2016 21:06:08 +0100")
References: <CAGgTfkMu=FzEzoNh2VGog35Y-x-5_MFZQHdYbmJVO9xcHRB1sQ@mail.gmail.com>
 <CAP7+vJLmm9JXcFoJka3OFTc=6Ydc4q52KN1tzQV9Uxqiek_Gig@mail.gmail.com>
 <CAMiohoiwfaH3NqFGmMDDX9ezOWzEfyxqpFXaDL0+JH94GtOZWg@mail.gmail.com>
 <57472944.7020102@mail.de> <20160526174016.GD12028@ando.pearwood.info>
 <57473890.6050400@stoneleaf.us>
 <CACac1F98i9dGHmy5XRLq8LX--6WaK221HhDAtFCazdAX1MGnpg@mail.gmail.com>
 <87d1o77d1c.fsf@thinkpad.rath.org> <57487AFB.8020800@stoneleaf.us>
 <87vb1z5q80.fsf@thinkpad.rath.org>
 <CAP7+vJKCHa9tZ_3oE8QSUmnE+LQPbQqfMVvUNYMO0DF3HkimUw@mail.gmail.com>
 <CABDEq2mEgMKqsTseGHn_LbYi9zWQtQ9QuBUyRwohgyvOsYiXVQ@mail.gmail.com>
 <CAP7+vJJU5LduRQWFhFNyo-q3uBLR3C5naVWj5zJrcDdGL8Xenw@mail.gmail.com>
 <87twhgr5ns.fsf@kosh.rath.org>
 <CAP7+vJLFrguNF5tR9ATgZzeXzNBqH=Z5gzuZnZLcJEYr_2Dyfg@mail.gmail.com>
 <87bn3na8jf.fsf@kosh.rath.org>
 <CACac1F9CsHHwm0RnEvQ5KPezz20yu-f8VhAU5=-PZs=s48urtQ@mail.gmail.com>
 <87pos28aze.fsf@thinkpad.rath.org>
 <CACac1F-GO_qTZFO5WYbYgLKdArjDQ+CymSMkkRakqvsDMeAsgg@mail.gmail.com>
 <87bn3m80hb.fsf@thinkpad.rath.org>
 <CACac1F9HfQU4cy8j80Pqnq9D-2YYPzAjg-4Zd_4VQ=8vcj+KGA@mail.gmail.com>
Message-ID: <878tyo5l5i.fsf@vostro.rath.org>

On May 31 2016, Paul Moore <p.f.moore-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
> On 31 May 2016 at 20:57, Nikolaus Rath <Nikolaus-BTH8mxji4b0 at public.gmane.org> wrote:
>> On May 31 2016, Paul Moore <p.f.moore-Re5JQEeQqe8AvxtiuMwx3w-XMD5yJDbdMReXY1tMh2IBg at public.gmane.org> wrote:
>>> Technically, the "export key" approach could probably me made readable
>>> in a way I'd be OK with, for example:
>>>
>>> export key from query_result (
>>>     product_id,
>>>     quantity,
>>>     distributor,
>>>     description
>>> )
>>>
>>> but I'm not at all sure that needing 2 new keywords ("export" and
>>> "key") and a reuse of an existing one ("from") is going to fly - it's
>>> too wordy, feels like SQL or COBOL to me. Maybe if someone comes up
>>> with a one-word option for "export key from" then it would be
>>> viable...
>>
>> How about
>>
>>   unravel query_result (this, that, something)
>
> Meh. Don't assume that if you come up with a good word, I'll be in
> favour. At best, I'll go from -1 to -0 or maybe +0. Should we get to
> something that looks reasonably attractive to me, there's still issues
> with the whole thing being a niche problem, limited applicability (by
> leaping at "unravel" you lost the ability to extract attributes from
> an object - did you mean to do that?), etc.
>
> Basically, don't waste too much time trying to convince me. A better
> bet would be to get sufficient support from others that my opinion is
> irrelevant (which it may well be anyway :-))

Nah, I think I don't like my own idea anymore. I think the whole issue
is much better addressed by using an assignment with a placeholder on
the LHS. This also solves the DRY problem when creating things like
namedtuples:

d = query_result
product_id = q[$lhs]
quantity = q[$lhs]
distributor = q[$lhs]
description = q[$lhs]

# or

my_favorite_named_tuple = namedtuple($lhs, "foo bar com")


$lhs would be a special construct that is only allowed in the RHS of an
assignment statement and evaluates to a string representation of the
identifier on the LHS.


But since I don't have time to learn how to extend the Python parser, I
will shut up about this now.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From srkunze at mail.de  Thu Jun  2 06:46:29 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 2 Jun 2016 12:46:29 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkOXFHM8rsx135vkLcz8jeDw-qizG7BwEYrhw89Logp2Mg@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574DA1A6.8090906@mail.de>
 <42A829DD-9910-45C4-B7D2-1B820AE2CB94@sjoerdjob.com>
 <574EE1B0.3070106@mail.de>
 <CAGgTfkN77yv6OqsZYG2MHZ7W0oO8eEwMM+Z8WA+85W-5RiTdbQ@mail.gmail.com>
 <CAGgTfkMdz+Z2rZ1simEN7fx5OfjQEKwFC48OhUheH2cKMQf-aQ@mail.gmail.com>
 <574F340B.4080008@mail.de>
 <CAGgTfkOXFHM8rsx135vkLcz8jeDw-qizG7BwEYrhw89Logp2Mg@mail.gmail.com>
Message-ID: <57500E85.8030100@mail.de>

On 01.06.2016 21:48, Michael Selik wrote:
> On Wed, Jun 1, 2016 at 3:14 PM Sven R. Kunze <srkunze at mail.de 
> <mailto:srkunze at mail.de>> wrote:
>
>     On 01.06.2016 18:24, Michael Selik wrote:
>>     I don't see how that would be improved by a special syntax for
>>     the [RHS] to know the identifier it's being assigned to. Could
>>     you clarify that?
>     I was referring to the issue that fields don't know neither their
>     containing form classes nor their form instances. In some
>     circumstances, one need to do that in order to correctly work with
>     the field (think of the prefix of the form which needs to be
>     reflected in the field but there are more missing pieces of
>     information).
>
>     As mentioned to Steven, in this case the name does not suffices at
>     all. So, I am not sure how to handle this right now. This might
>     belong to a greater issue of referring back which currently needs
>     to be handled manually in all cases.
>
>
> I always find bi-directional has-a relationships to be troublesome. If 
> it were necessary frequently, I'd expect the Form metaclass to loop 
> over the declared_fields and set some .form attribute to the 
> new_class. Django is way too big for me to grok from light reading, so 
> I won't presume to understand the choices made there.
>
> https://github.com/django/django/blob/master/django/forms/forms.py#L27

That would be a hand-made solution and would work quite efficiently in 
terms of productivity for Django projects.

Recently, I started a private side project where I would need that 
back-relationship as well. This time, it's about rigid body simulation 
where I would like to pick a point in body space of a body and always 
get the correct world space coordinates when accessing. Same thing, but 
also hand-made by myself and it requires special attention from time to 
time.

I don't say I have a nice, readable solution for this but it seems to me 
that *providing context* to Python objects would as much help as it 
helps Google to provide extremely accurate search results.


Best,
Sven


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/e01a76a0/attachment.html>

From steve at pearwood.info  Thu Jun  2 08:45:21 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 2 Jun 2016 22:45:21 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574F31BD.6070604@mail.de>
References: <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de>
Message-ID: <20160602124521.GO12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 09:04:29PM +0200, Sven R. Kunze wrote:

> I consider a special syntax for this narrow kind of usecase way too much 
> as it doesn't seem very flexible and extensible in the future.
> 
> Dunder methods/attributes/variables allow far more.

There's nothing to extend. It's a simple problem: how to give the right 
hand side of an assignment access to the name of the assignment target 
without having to type the name again as a string.

If you think that problem is too trivial to bother solving, that's a 
reasonable opinion to hold.

But if you want to extend it to solve "far more" (whatever you mean by 
that), please don't hijack this thread. If you think of some other 
problems you would like to solve (whatever they might be), please feel 
free to propose a solution to them in another thread. But *this* 
discussion is about solving *this problem*:

    Some (but not all) functions need to know the name of their 
    assignment target, so that the object they return can give 
    itself a name that matches that target. Or to put it another 
    way, some objects need to know their own name. The existing
    solution to that is simple, but inelegant and annoying: you
    have to manually type the name as a string and pass it as an
    argument to the function.

If you want to solve some other problem, please do! But that's not part 
of this thread, and if the proposed solution to *this* problem doesn't 
solve *your* problem, that's irrelevant.


-- 
Steve

From duda.piotr at gmail.com  Thu Jun  2 08:55:25 2016
From: duda.piotr at gmail.com (Piotr Duda)
Date: Thu, 2 Jun 2016 14:55:25 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160602124521.GO12028@ando.pearwood.info>
References: <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de>
 <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de>
 <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de>
 <20160602124521.GO12028@ando.pearwood.info>
Message-ID: <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>

2016-06-02 14:45 GMT+02:00 Steven D'Aprano <steve at pearwood.info>:

> But if you want to extend it to solve "far more" (whatever you mean by
> that), please don't hijack this thread. If you think of some other
> problems you would like to solve (whatever they might be), please feel
> free to propose a solution to them in another thread. But *this*
> discussion is about solving *this problem*:
>
>     Some (but not all) functions need to know the name of their
>     assignment target, so that the object they return can give
>     itself a name that matches that target. Or to put it another
>     way, some objects need to know their own name. The existing
>     solution to that is simple, but inelegant and annoying: you
>     have to manually type the name as a string and pass it as an
>     argument to the function.
>
> If you want to solve some other problem, please do! But that's not part
> of this thread, and if the proposed solution to *this* problem doesn't
> solve *your* problem, that's irrelevant.
>

I want to extend it to: some object also need to know module they're
declared and their own qualified name (for example for pickling support).


-- 
????????
??????
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/4f928cec/attachment.html>

From rob.cliffe at btinternet.com  Thu Jun  2 09:05:10 2016
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 2 Jun 2016 14:05:10 +0100
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <20160602013420.GN12028@ando.pearwood.info>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
Message-ID: <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>



On 02/06/2016 02:34, Steven D'Aprano wrote:
> On Wed, Jun 01, 2016 at 11:26:07PM +0100, Rob Cliffe wrote:
>> I was prompted to post this after seeing comments on "Proposal to change
>> List Sequence Repetition (*) so it is not useless for Mutable Objects"
>> that you cannot in general tell whether an object is mutable or not.
>>
>> Well, why shouldn't you be able to tell?  Python is superb at
>> introspection, but ISTM there's a gap here.  Here's an idea for Python N
>> (N>=4):  Every object has a boolean __mutable__ attribute.  It would
>> e.g. be False for ints and strings,  True by default for objects that
>> can be mutable, such as lists and dicts.
> I hate to break it to you, but merely setting a flag on an object
> doesn't make it mutable or immutable *wink*
>
> That means that every class (possibly including builtins) needs to be
> re-written so that every mutator method checks this flag and decides
> whether or not to allow the mutation. For example, you suggest:
>
> "You don't need tuples.  You just have a list with __mutable__ = False."
>
> Now the code for list append must be:
>
>      def append(self, item):
>          # only written in C, because its a built-in
>          if self.__mutable__:
>              ...
>          else:
>              raise SomeError
>
> Now multiply that by *every* mutator class and method. This will be
> especially burdensome for people writing classes intended to be mutable,
> since they have to support immutability whether they want it or not.
>
> (Since the caller might change obj.__mutable__ to False, which is
> allowed.)
>
> This will break type-checking and duck-typing and make feature
> detection a pain:
>
> # currently this works
> if hasattr(obj, 'append'):
>      handle_things_that_can_append(obj)
> else:
>      handle_things_that_cant_append(obj)
>
> # with your scheme
> if hasattr(obj, 'append'):
>      if obj.__mutable__:
>          handle_things_that_can_append(obj)
>      else:
>          handle_things_that_cant_append(obj)
> else:
>      handle_things_that_cant_append(obj)
>
>
>> It could be an optional extra
>> argument to mutable object constructors (so you have the option of
>> making them immutable):
>>
>>      L = list(somesequence, mutable=False)
> This goes against the "no constant bool arguments" design guideline.
>
>
>> There could also be a syntax for list/set literals and dictionary
>> displays to indicate that they should be immutable (just as we preface
>> literal strings with 'r' to indicate raw strings).  I'm not sure what; I
>> thought of f[1,2,3] for a literal immutable ("frozen") list, but that's
>> already valid syntax.
> We already have syntax for a built-in immutable list-like sequence
> object (a "frozen list" if you will):
>
> (1, 2, 3)
>
>
>> You can "freeze" a mutable object by changing its __mutable__ attribute
>> from True to False.  You are not allowed to change it from False to
>> True, nor to mutate an object that has it False.
>>
>> I am sure there must be advantages in being able to tell if an object is
>> mutable or not.  (Perhaps "hashable" equals "not mutable" ?)
> No it does not.
>
> (1, 2, [3]) is both immutable and unhashable.
>
>
>> Now:  You don't need the frozenset class.  You just have a set with
>> __mutable__ = False.  And you can freeze dicts (classes? modules?) etc.,
>> to make sure users of your code don't mess with them.
> And we get this functionality for free, right? *wink*
>
> Somebody has to write this code. I'm not sure they will appreciate
> having to throw away all the perfectly good frozenset code and tests,
> and re-writing set to handle both cases. Not to mention the breaking of
> backwards compatibility to get rid of frozenset.
>
>
> [snip]
Sure, there are many difficulties.
This was intended to be a blue-sky idea.  I would like to pose the 
question "If I had to redesign Python from scratch, would I think this 
is a good idea?"
Rob


From srkunze at mail.de  Thu Jun  2 09:35:59 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 2 Jun 2016 15:35:59 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
References: <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <574D8B3C.5060406@mail.de> <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de> <20160602124521.GO12028@ando.pearwood.info>
 <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
Message-ID: <5750363F.20002@mail.de>

On 02.06.2016 14:55, Piotr Duda wrote:
> I want to extend it to: some object also need to know module they're 
> declared and their own qualified name (for example for pickling support).

Here we go.

I can also weigh in the relation between field+forms in Django.


@Steven
The name is simply not enough. You might think so and that's your right 
to do. However, it's just not that simple and it does not reflect a good 
technical solution. As a consequence this is relevant to the 
"only-the-name-please" thread. Before accusing somebody of "hijack[ing 
a] thread", I'd appreciate if you think why somebody contributed 
something first. Furthermore, your tone seems disrespectful to me.


Best,
Sven

From srkunze at mail.de  Thu Jun  2 09:55:01 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 2 Jun 2016 15:55:01 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
Message-ID: <57503AB5.7090708@mail.de>

On 02.06.2016 15:05, Rob Cliffe wrote:
> Sure, there are many difficulties.
> This was intended to be a blue-sky idea.  I would like to pose the 
> question "If I had to redesign Python from scratch, would I think this 
> is a good idea?"

I don't think that's an overly stupid idea. In such regard, Python 
wouldn't be the first project dealing with mutability in this manner. 
PostgreSQL already does.

One question I would have: would it be enforced? Or can objects designer 
choose between enforced immutability and informative immutability? Or 
would it be enforced inconsistently at some places where at other places 
it does not matter.
To me the 100% enforced variant would be the least troublesome.

One definite advantage of broader use of immutable objects would be 
optimization: using the same memory parts, re-using the same object etc. 
It's not my field in computer science but other people will be able to 
imagine a lot of other possibilities here.

Best,
Sven

From michael.selik at gmail.com  Thu Jun  2 10:11:04 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Thu, 02 Jun 2016 14:11:04 +0000
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <20160602013420.GN12028@ando.pearwood.info>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
Message-ID: <CAGgTfkOjGuFkbYFVu+M2HR0CgXWPVFyPA7wjxErrwCCgkzUL1Q@mail.gmail.com>

On Wed, Jun 1, 2016 at 9:35 PM Steven D'Aprano <steve at pearwood.info> wrote:

> On Wed, Jun 01, 2016 at 11:26:07PM +0100, Rob Cliffe wrote:
> > It could be an optional extra
> > argument to mutable object constructors (so you have the option of
> > making them immutable):
> >
> >     L = list(somesequence, mutable=False)
>
> This goes against the "no constant bool arguments" design guideline.
>

That was a recent discussion and Guido suggested it was unlikely anyone
would need an optional boolean flag to be dynamic in the code. It would be
much more likely that the call site would always pick one or the other and
that flag could be constant. If it's a constant, then it'd be better to
have two different well-named constructors, rather than a boolean parameter.

Thus to evaluate this idea, we should ask:
Would any call site need to set that flag dynamically?

If it's simply a tool for introspection, rather than construction, how
often do you find yourself checking for is-mutable?
    immutables = (str, bytes, int, float, complex, bool, tuple, frozenset)
    if isinstance(obj, immutables):
        ...

I don't think I've ever made that check. I've checked for is-basic-type,
but that's different.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/ff00a290/attachment.html>

From jsbueno at python.org.br  Thu Jun  2 10:18:45 2016
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Thu, 2 Jun 2016 11:18:45 -0300
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <57503AB5.7090708@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
Message-ID: <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>

Just remembering that one is free to implement whatever "switchable"
containers one wants in the language as it is now - no need to force
(full or slightly) incompatible  changes that imply on performance
degradation onto every single Python user for a feature that was not
needed up to this day.

Implementing this in  sequence-abc, and mapping-abc classes is trivial.

A reasonable request for the language could be exactly to allow a
"__mutable__" property on the sequence, container and mapping
protocols, and a "mutable" built-in callable, that in the absence of
"__mutable__" could check for the methods signature (if it does not
have "__mutable__" but has "__setitem__" , then mutable(obj) returns
True, for example).


On 2 June 2016 at 10:55, Sven R. Kunze <srkunze at mail.de> wrote:
> On 02.06.2016 15:05, Rob Cliffe wrote:
>>
>> Sure, there are many difficulties.
>> This was intended to be a blue-sky idea.  I would like to pose the
>> question "If I had to redesign Python from scratch, would I think this is a
>> good idea?"
>
>
> I don't think that's an overly stupid idea. In such regard, Python wouldn't
> be the first project dealing with mutability in this manner. PostgreSQL
> already does.
>
> One question I would have: would it be enforced? Or can objects designer
> choose between enforced immutability and informative immutability? Or would
> it be enforced inconsistently at some places where at other places it does
> not matter.
> To me the 100% enforced variant would be the least troublesome.
>
> One definite advantage of broader use of immutable objects would be
> optimization: using the same memory parts, re-using the same object etc.
> It's not my field in computer science but other people will be able to
> imagine a lot of other possibilities here.
>
> Best,
> Sven
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From michael.selik at gmail.com  Thu Jun  2 11:28:56 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Thu, 02 Jun 2016 15:28:56 +0000
Subject: [Python-ideas] Object for accessing identifiers/names
In-Reply-To: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
References: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
Message-ID: <CAGgTfkP8uby3fEFKZ_Mrr-oy=ACzjG-NfmgJturzaWMU7x=tFA@mail.gmail.com>

On Wed, Jun 1, 2016 at 3:23 PM Koos Zevenhoven <k7hoven at gmail.com> wrote:

> Inspired by the discussions
> regarding
>  Guido's
> ?"M?
> atch statement
> ? brainstorm?
> " and
> ?"?
> Quick idea: defining variables from functions..." threads, here's an idea
> regarding names/identifiers.
>
> Currently, there is no direct way to work with variable names from within
> Python. Yes, you can fiddle with __dict__s and locals() and globals(), but
> there is no
> ?convenient ?
> general way. To solve this, there could be a way
> ?(probably new syntax)?
>  for creating an object that
> ??
> can examine and manipulate a name binding conveniently.
>
> The object could be created for instance as follows:
>
> ? ?
> name_object = identifier some_name
>
> However, 'identifier' is so long that I'll use the keyword 'def' instead,
> regardless of whether it is optimal or not:
>
> ?  ?
> name_obj = def some_name
>
> Now this name_obj
> ? thing?
> could provide functionality like assigning to the name some_name, getting
> the assigned object, and determining whether
> ?something has been assigned to the name or not
> . The object would also be aware of the name 'some_name'.
>
> ?T
> he functionality might
> ?work
>  as follows:
>
> ?  ?
> bool(
> ?name_obj
> )
> ?   ?
> # True if
> ?something? is
>  assigned to some_name
> ?  ?
> name_obj.name()
>
> ?    ?
> # == 'some_name'
> ??
> ?  name_obj
> .set(value) #
> ?equivalent to `some_name = value`?
> ??
> ?  name_obj.get()      # equivalent to just `some_name?`
>   name_obj.unset()    # like `del some_name`
>
> ??Then you could pass this to a function:
>
>   func(name_obj)
>
> Now func will be able to assign to the variable some_name, but with
> some_name referring to the scope where `name_obj = def some_name` was
> executed. This is similar to things that can be done with closures.
>
> This would also allow things like:
>
>   if def some_name:
>       #do stuff if the name some_name is bound to something
>
> or
>
>   if not def some_name:
>       some_name = something()
>




Here's an implementation. Can you show a complete example of where it would
be useful in a program?



class Identifier:
    '''
    Manipulate a variable in the current scope.
    '''

    def __init__(self, name):
        self.name = name
        self.scope = None

    def __enter__(self):
        caller = inspect.stack()[1][0]
        try:
            self.scope = caller.f_locals
            return self
        finally:
            del caller

    def __exit__(self, *args):
        self.scope = None

    def set(self, value):
        if self.scope is None:
            raise RuntimeError('must be used as a context manager')
        self.scope[self.name] = value

    def get(self):
        if self.scope is None:
            raise RuntimeError('must be used as a context manager')
        return self.scope[self.name]

    def del(self):
        if self.scope is None:
            raise RuntimeError('must be used as a context manager')
        del self.scope[self.name]

if __name__ == '__main__':
    with Identifier('x') as name_x:
        name_x.set(5)
        print(name_x.get())
    print(x)



It's a context manager because I got nervous about reference cycles as
warned by the docs of the inspect module. It might not be an issue, because
this is referencing the calling frame rather than the current frame, but
it's better to be safe than sorry.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160602/ca513982/attachment-0001.html>

From stephen at xemacs.org  Thu Jun  2 13:50:25 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 3 Jun 2016 02:50:25 +0900
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160601182216.GK12028@ando.pearwood.info>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <87k2ia8843.fsf@thinkpad.rath.org>
 <BLU403-EAS153B19DAF583780CD047BBC91460@phx.gbl>
 <20160601182216.GK12028@ando.pearwood.info>
Message-ID: <22352.29153.334489.986841@turnbull.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > If anything, this syntax could be rejected as being too limited and
 > not useful enough,

+1 on that sentiment.

I'm willing to bet that all of these cases are optimizations in any
case.  Otherwise you'd be using "class".  People expect optimized code
to be somewhat uglier than the textbook "teach the idiom" version.
Case in point: the partition exchange sort (more commonly called
"quicksort").  It's actually very easy to understand, it's just the
optimization of in-place sort (well, OK, C don't help) that makes it
look hard.  Don't believe me?  Check it out![1]

https://www.youtube.com/watch?v=bSfe5M_zG2s?start=1895

I don't have anything against nice syntax for optimized code, but I
don't consider your syntax an improvement over the status quo.  Not
that it's outright ugly, just not an improvement.

Steve

Footnotes: 
[1]  The start parm is if you're in a hurry.  The whole keynote is
highly recommended!


From rob.cliffe at btinternet.com  Thu Jun  2 14:12:35 2016
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 2 Jun 2016 19:12:35 +0100
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <57503AB5.7090708@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
Message-ID: <cddb8bd9-3eeb-384d-b334-c6131abbaa02@btinternet.com>



On 02/06/2016 14:55, Sven R. Kunze wrote:
> On 02.06.2016 15:05, Rob Cliffe wrote:
>> Sure, there are many difficulties.
>> This was intended to be a blue-sky idea.  I would like to pose the 
>> question "If I had to redesign Python from scratch, would I think 
>> this is a good idea?"
>
> I don't think that's an overly stupid idea. In such regard, Python 
> wouldn't be the first project dealing with mutability in this manner. 
> PostgreSQL already does.
>
> One question I would have: would it be enforced? Or can objects 
> designer choose between enforced immutability and informative 
> immutability? Or would it be enforced inconsistently at some places 
> where at other places it does not matter.
> To me the 100% enforced variant would be the least troublesome.
+1


From steve at pearwood.info  Thu Jun  2 18:10:31 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 3 Jun 2016 08:10:31 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <5750363F.20002@mail.de>
References: <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de> <20160602124521.GO12028@ando.pearwood.info>
 <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
 <5750363F.20002@mail.de>
Message-ID: <20160602221031.GS12028@ando.pearwood.info>

On Thu, Jun 02, 2016 at 03:35:59PM +0200, Sven R. Kunze wrote:

> The name is simply not enough. You might think so and that's your right 
> to do. However, it's just not that simple and it does not reflect a good 
> technical solution.

A good technical solution to *what*? That's the point I am making. The 
name alone is a perfect technical solution to the problem Guido raised 
in the first place. 

> As a consequence this is relevant to the 
> "only-the-name-please" thread. Before accusing somebody of "hijack[ing 
> a] thread", I'd appreciate if you think why somebody contributed 
> something first. Furthermore, your tone seems disrespectful to me.

I am sorry, my post was not intended to be disrespectful. But I stand by 
it: if you wish to solve some other problem, then start a new thread.

In particular, you need to state what problem you are trying to solve. 
It is not enough to just say "this isn't enough".


-- 
Steve

From steve at pearwood.info  Fri Jun  3 01:15:13 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 3 Jun 2016 15:15:13 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
References: <574D8B3C.5060406@mail.de>
 <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de> <20160602124521.GO12028@ando.pearwood.info>
 <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
Message-ID: <20160603051513.GT12028@ando.pearwood.info>

On Thu, Jun 02, 2016 at 02:55:25PM +0200, Piotr Duda wrote:

> I want to extend it to: some object also need to know module they're
> declared and their own qualified name (for example for pickling support).

If you are serious about this, then you should start a thread about it.

You should give some examples of which objects need this, and how people 
currently solve the problem, and why it is unsuitable.

If I have understood you correctly, the usual solution to this is to use 
sys._getframe(1). I don't know what Jython and IronPython use.


-- 
Steve

From steve at pearwood.info  Fri Jun  3 02:05:54 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 3 Jun 2016 16:05:54 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <574F4D0F.8000809@canterbury.ac.nz>
References: <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
Message-ID: <20160603060554.GU12028@ando.pearwood.info>

On Thu, Jun 02, 2016 at 09:01:03AM +1200, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >I'm not proposing any changes to namedtuple at all -- its others who 
> >want to change it to take just a single argument and have an wrapper 
> >function to specify the field names. I'm against that rather strongly.
> 
> I don't think this would be creating TMWTDI.

I'm not objecting to it because it creates more than one way to do it.

I'm objecting to it because it needlessly doubles the number of 
callables needed. Instead of there being one namedtuple function, you 
need two: a single parameter version, and a wrapper that performs 
whatever magic is needed to crowbar the multiple parameter version into 
the constraint of a single parameter version.

For example, we have int(), which takes either one or two arguments:

int("23")
=> returns 23

int("23", 16)
=> returns 35

We don't have int, and int_wrapper:

# Python doesn't do this.
int_wrapper(16)("23")
=> returns 35



> Given the
> proposed feature, and a suitable helper, then
> 
>   def Foo as NamedTuple('x', 'y', 'z')
> 
> would become the *obvious* way to define a named tuple
> type.

I'm not really sure that the difference between namedtuple and 
NamedTuple is obvious, and I'm confident that the subtle difference 
between them will be a bug-magnet. But maybe we could solve that 
somehow.

In any case, I agree that the new syntax would become the One Obvious 
Way. There's no real semantic difference between:

def Record as namedtuple('x y z')

Record -> namedtuple('x y z')

and while I'm not a big fan of the def ... as syntax, I could live with 
it. One thing I dislike is that it reverses the usual sense of "as":

    import module as foo  # module is bound to name "foo"

    with cm() as foo  # context manager is bound to name "foo"

    try:... except E as foo  # exception is bound to name "foo"

All other uses have "as foo" set the name. But you have:

    def foo as thing_that_is_called

which backwards is, as speaks Yoda.


But there is a rather big semantic difference once you insist that the 
thing that is called must only accept a single argument.


-- 
Steve

From steve at pearwood.info  Fri Jun  3 02:22:45 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 3 Jun 2016 16:22:45 +1000
Subject: [Python-ideas] Object for accessing identifiers/names
In-Reply-To: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
References: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
Message-ID: <20160603062245.GV12028@ando.pearwood.info>

On Wed, Jun 01, 2016 at 10:22:39PM +0300, Koos Zevenhoven wrote:

> Currently, there is no direct way to work with variable names from within
> Python. Yes, you can fiddle with __dict__s and locals() and globals(),

The usual way is to write the name as a string and operate on the 
namespace.

That means that variables (names) are not themselves values: a name is 
bound to a value, but there is no value which is itself a name. Instead, 
we use a string as a form of indirection.

> but
> there is no ?convenient? general way. To solve this, there could be a way
> ?(probably new syntax)? for creating an object that
> can examine and manipulate a name binding conveniently.

Before getting too interested in the syntax for creating these, can you 
explain what you want to do with them? Most uses of names don't need to 
be values:

spam = 23
import spam
del spam

all work well on their own. Can you give some use-cases?


> The object could be created for instance as follows:
> ? ?
> name_object = identifier some_name

How would that be different from this?

name_object = 'some_name'


> Now this name_obj? thing?
> could provide functionality like assigning to the name some_name, getting
> the assigned object, and determining whether
> ?something has been assigned to the name or not.

Most of these things already work using strings:

# bind a name to the global scope
globals()[name_object] = 999

# lookup a name in some module's namespace
vars(module)[name_object]

# check for existence
name_object in vars(module)

What functionality do you think is missing?



-- 
Steve

From duda.piotr at gmail.com  Fri Jun  3 03:02:47 2016
From: duda.piotr at gmail.com (Piotr Duda)
Date: Fri, 3 Jun 2016 09:02:47 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160603051513.GT12028@ando.pearwood.info>
References: <574D8B3C.5060406@mail.de>
 <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de> <20160602124521.GO12028@ando.pearwood.info>
 <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
 <20160603051513.GT12028@ando.pearwood.info>
Message-ID: <CAJ1Wxn1b-StSZ9nw=gginShUL-K7ZNiyPwxGU3k60NTS9n37rA@mail.gmail.com>

2016-06-03 7:15 GMT+02:00 Steven D'Aprano <steve at pearwood.info>:
>
> On Thu, Jun 02, 2016 at 02:55:25PM +0200, Piotr Duda wrote:
>
> > I want to extend it to: some object also need to know module they're
> > declared and their own qualified name (for example for pickling support).
>
> If you are serious about this, then you should start a thread about it.
>
> You should give some examples of which objects need this, and how people
> currently solve the problem, and why it is unsuitable.
>
> If I have understood you correctly, the usual solution to this is to use
> sys._getframe(1). I don't know what Jython and IronPython use.


Examples would be any type object created by function that need
pickling support, like stdlib namedtuple or Enum, both of them use
_getframe hack as default, but this doesn't work in Jython or
IronPython, and doesn't work if they're wrapped by another function,
Enum also allow explicity specify module and qualname via keyword
arguments, like:

Animal = Enum('Animal', 'ant bee cat dog', module=__name__,
qualname='SomeData.Animal')

with new syntax:

def Animal = Enum('ant bee cat dog')

From p.f.moore at gmail.com  Fri Jun  3 03:45:37 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 3 Jun 2016 08:45:37 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160603060554.GU12028@ando.pearwood.info>
References: <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
 <20160603060554.GU12028@ando.pearwood.info>
Message-ID: <CACac1F9TJr9Xmb-nMdRGHG=o5D+PPCyfbOjak8mgE0=noq+wyQ@mail.gmail.com>

On 3 June 2016 at 07:05, Steven D'Aprano <steve at pearwood.info> wrote:
> I'm objecting to it because it needlessly doubles the number of
> callables needed. Instead of there being one namedtuple function, you
> need two: a single parameter version, and a wrapper that performs
> whatever magic is needed to crowbar the multiple parameter version into
> the constraint of a single parameter version.

Thanks for clarifying your objection, I hadn't really understood what
you had an issue with until now.

OK, so the question is between needing extra callables, vs a somewhat
"magical" process for injecting an argument.

We still differ on which is the more acceptable answer, but I'm happy
to offer both up for consideration, now that the differences are
clear.

Paul

From duda.piotr at gmail.com  Fri Jun  3 03:55:44 2016
From: duda.piotr at gmail.com (Piotr Duda)
Date: Fri, 3 Jun 2016 09:55:44 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CACac1F9TJr9Xmb-nMdRGHG=o5D+PPCyfbOjak8mgE0=noq+wyQ@mail.gmail.com>
References: <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
 <20160603060554.GU12028@ando.pearwood.info>
 <CACac1F9TJr9Xmb-nMdRGHG=o5D+PPCyfbOjak8mgE0=noq+wyQ@mail.gmail.com>
Message-ID: <CAJ1Wxn21aY_NGMPDov=JJKdMS=mD8EMHOV8bQiZwUB1NQqdD=g@mail.gmail.com>

2016-06-03 9:45 GMT+02:00 Paul Moore <p.f.moore at gmail.com>:
> On 3 June 2016 at 07:05, Steven D'Aprano <steve at pearwood.info> wrote:
>> I'm objecting to it because it needlessly doubles the number of
>> callables needed. Instead of there being one namedtuple function, you
>> need two: a single parameter version, and a wrapper that performs
>> whatever magic is needed to crowbar the multiple parameter version into
>> the constraint of a single parameter version.
>
> Thanks for clarifying your objection, I hadn't really understood what
> you had an issue with until now.
>
> OK, so the question is between needing extra callables, vs a somewhat
> "magical" process for injecting an argument.
>
> We still differ on which is the more acceptable answer, but I'm happy
> to offer both up for consideration, now that the differences are
> clear.

There is also third option, return normal object that have special
dunder method (ex. __def__), which allow set name (and maybe module
and qualname) for object.

From p.f.moore at gmail.com  Fri Jun  3 04:01:14 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 3 Jun 2016 09:01:14 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAJ1Wxn21aY_NGMPDov=JJKdMS=mD8EMHOV8bQiZwUB1NQqdD=g@mail.gmail.com>
References: <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
 <20160603060554.GU12028@ando.pearwood.info>
 <CACac1F9TJr9Xmb-nMdRGHG=o5D+PPCyfbOjak8mgE0=noq+wyQ@mail.gmail.com>
 <CAJ1Wxn21aY_NGMPDov=JJKdMS=mD8EMHOV8bQiZwUB1NQqdD=g@mail.gmail.com>
Message-ID: <CACac1F-qG2PS8BUYGmceCJMefPkYgQroQYmYV42mOA_u6TnBpA@mail.gmail.com>

On 3 June 2016 at 08:55, Piotr Duda <duda.piotr at gmail.com> wrote:
> 2016-06-03 9:45 GMT+02:00 Paul Moore <p.f.moore at gmail.com>:
>> On 3 June 2016 at 07:05, Steven D'Aprano <steve at pearwood.info> wrote:
>>> I'm objecting to it because it needlessly doubles the number of
>>> callables needed. Instead of there being one namedtuple function, you
>>> need two: a single parameter version, and a wrapper that performs
>>> whatever magic is needed to crowbar the multiple parameter version into
>>> the constraint of a single parameter version.
>>
>> Thanks for clarifying your objection, I hadn't really understood what
>> you had an issue with until now.
>>
>> OK, so the question is between needing extra callables, vs a somewhat
>> "magical" process for injecting an argument.
>>
>> We still differ on which is the more acceptable answer, but I'm happy
>> to offer both up for consideration, now that the differences are
>> clear.
>
> There is also third option, return normal object that have special
> dunder method (ex. __def__), which allow set name (and maybe module
> and qualname) for object.

Sorry, yes, there may well be other options as well. I haven't been
following all of the proposals in this thread.
Paul

From sjoerdjob at sjoerdjob.com  Fri Jun  3 04:26:00 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Fri, 3 Jun 2016 10:26:00 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAJ1Wxn21aY_NGMPDov=JJKdMS=mD8EMHOV8bQiZwUB1NQqdD=g@mail.gmail.com>
References: <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <22349.20907.341754.999142@turnbull.sk.tsukuba.ac.jp>
 <CACac1F-kQyivaF6vwiez-hzxA0NaALCeu9eQOxkaDcp492+AUg@mail.gmail.com>
 <CAEbHw4aJ2wT-S9ObEhT9AtsDuzXD-bYxvLhUKUW+aK_-d0KZ7A@mail.gmail.com>
 <CAPncOv++_H2uXhOSAF0eCXYxJePOqiySFj1fvoWM2cjWKgDWiA@mail.gmail.com>
 <20160601161431.GF12028@ando.pearwood.info>
 <CACac1F9QPmYsrPPPSVF6Frm-WP0DiwmjiG+v9pQFD2p7yOyB4w@mail.gmail.com>
 <CAGgTfkNC7SXkgNn7hTA9NOo5Xr2vk4GsBdPJj8JyhOA_5w-5qw@mail.gmail.com>
 <20160601172623.GI12028@ando.pearwood.info>
 <574F4D0F.8000809@canterbury.ac.nz>
 <20160603060554.GU12028@ando.pearwood.info>
 <CACac1F9TJr9Xmb-nMdRGHG=o5D+PPCyfbOjak8mgE0=noq+wyQ@mail.gmail.com>
 <CAJ1Wxn21aY_NGMPDov=JJKdMS=mD8EMHOV8bQiZwUB1NQqdD=g@mail.gmail.com>
Message-ID: <F9D0072A-693C-4A06-AC55-5BD54E17C36A@sjoerdjob.com>



> On 3 Jun 2016, at 09:55, Piotr Duda <duda.piotr at gmail.com> wrote:
> 
> 2016-06-03 9:45 GMT+02:00 Paul Moore <p.f.moore at gmail.com>:
>>> On 3 June 2016 at 07:05, Steven D'Aprano <steve at pearwood.info> wrote:
>>> I'm objecting to it because it needlessly doubles the number of
>>> callables needed. Instead of there being one namedtuple function, you
>>> need two: a single parameter version, and a wrapper that performs
>>> whatever magic is needed to crowbar the multiple parameter version into
>>> the constraint of a single parameter version.
>> 
>> Thanks for clarifying your objection, I hadn't really understood what
>> you had an issue with until now.
>> 
>> OK, so the question is between needing extra callables, vs a somewhat
>> "magical" process for injecting an argument.
>> 
>> We still differ on which is the more acceptable answer, but I'm happy
>> to offer both up for consideration, now that the differences are
>> clear.
> 
> There is also third option, return normal object that have special
> dunder method (ex. __def__), which allow set name (and maybe module
> and qualname) for object.

With the added benefit that in Python 3.something you can already emulate this for construction during class construction: in the __prepare__ of the metaclass, return a dictionary wrapper which does that.

Of course, that's also what Django does now already (in a Python 2 compatible way) from metaclass.__new__.

From srkunze at mail.de  Fri Jun  3 08:15:11 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 3 Jun 2016 14:15:11 +0200
Subject: [Python-ideas] pattern matching proof-of-concept
In-Reply-To: <CAGgTfkNsDHCx7aNyUGB+yDzg3hqNfzEMSomUvr5a4=YiBh6mSw@mail.gmail.com>
References: <CAGgTfkP33QPEsrC8kpSds1nLMufGGPxN+W4yWqU5sjD++5HGLQ@mail.gmail.com>
 <CACac1F_p78rFgFrNwyse2DO4Y677PwnH8nLYNPqhkTcjT+QG5g@mail.gmail.com>
 <574EE506.9030100@mail.de>
 <CAGgTfkNsDHCx7aNyUGB+yDzg3hqNfzEMSomUvr5a4=YiBh6mSw@mail.gmail.com>
Message-ID: <575174CF.3030208@mail.de>

On 01.06.2016 21:04, Michael Selik wrote:
> On Wed, Jun 1, 2016 at 9:37 AM Sven R. Kunze <srkunze at mail.de 
> <mailto:srkunze at mail.de>> wrote:
>
>     more flexible than a monolithic switch-case
>
>
> That wasn't my initial intention, but I guess that's a side-effect of 
> needing to break it down into pieces viable in current syntax.

Which I for one doesn't regard as a drawback.

>
>     Thanks a lot, Michael.
>
>
> You're welcome. I hope the effort informs the discussion of a special 
> matching syntax.
>
>
> Some things that really stood out as tough to implement or not possible.
>
> 1. Avoid NameErrors when specifying identifiers to bind in a schema.
>
> I see two options to implement this in current syntax. I chose to 
> require a binding object and all identifiers must be attributes of 
> that binding object. Another option would be postponing the lookup of 
> the identifiers by requiring the schema to be defined in a function, 
> then doing some magic.
>
> 2. Matching an object and unpacking some of its attributes.
>
> If the schema is a fully-specified object, we can rely on its __eq__ 
> to do the match. If the schema specifies only the type and does not 
> unpack attributes, that's even easier. The tough part is a schema that 
> has a half-specified object where where its __eq__ cannot be relied 
> on. If all Unbounds are equal to everything, that would help a 
> half-specified schema compare well, but it breaks all sorts of other 
> code, such as ``list.index``. I chose to compare all public, 
> non-Unbound attributes, but that may not have been the implementation 
> of its __eq__. Alternatively, one could trace __eq__ execution and 
> revise Unbound comparisons, but that's beyond my wizarding abilities.
>
> 3. Unpacking the attributes of an object that aggressively type-checks 
> input.
>
> Given my solution of a Binding object and unpacking into its 
> attributes, one cannot, for example, unpack the first argument of a 
> ``range`` as it raises a TypeError: ``bind=Binding(); schema = 
> range(bind.x)``.
>
> I'm considering removing object-unpacking from the module. To match 
> and unpack an object, the schema could be specified as a mapping or 
> sequence and the user would transform the object into a mapping (like 
> ``obj.__dict__``) before doing the match.
>

This all matches with my expectations I expressed on the other thread. 
Maybe, it's another indication that one should not split attributes from 
an object at all but rather use the object as a whole. This said, 
lists/tuples and dictionaries are more like a loose collection of 
separate objects (allowing unpacking/matching) whereas an object 
represents more an atomic entity.

Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160603/b56ef4de/attachment.html>

From srkunze at mail.de  Fri Jun  3 08:30:23 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 3 Jun 2016 14:30:23 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160602221031.GS12028@ando.pearwood.info>
References: <20160531133717.GT12028@ando.pearwood.info>
 <574E19DB.7050307@canterbury.ac.nz> <574EE340.7020502@mail.de>
 <CACac1F_ohT0M3-Jbyh+uucbzqrUmBXhx5KRffBfiWL=jQsoqwg@mail.gmail.com>
 <574EFAF7.7030502@mail.de> <20160601152923.GC12028@ando.pearwood.info>
 <574F31BD.6070604@mail.de> <20160602124521.GO12028@ando.pearwood.info>
 <CAJ1Wxn3VzRN7e3kAU-Tny1m_-b7e_PD=ySp2mvEfERZkROzacA@mail.gmail.com>
 <5750363F.20002@mail.de> <20160602221031.GS12028@ando.pearwood.info>
Message-ID: <5751785F.3080503@mail.de>

On 03.06.2016 00:10, Steven D'Aprano wrote:
> I am sorry, my post was not intended to be disrespectful. But I stand by
> it: if you wish to solve some other problem, then start a new thread.

The point was that solving "how to get the name" is only part of the 
bigger problem "how to get more context". So, putting things into 
perspective is important in order to avoid overfitting to a single use-case.

Maybe, my intention wasn't clear but it seems that I am not the only one 
seeing issues with a particular solution.


Sven

NOTE: sys._getframe(1) is not really a to-go solution. It tastes more 
like an internal API which normal devs shouldn't touch except for 
debugging purposes. Maybe, I am wrong here, but the _underscore supports 
that view.

From sebastian at realpath.org  Fri Jun  3 12:06:14 2016
From: sebastian at realpath.org (Sebastian Krause)
Date: Fri, 03 Jun 2016 18:06:14 +0200
Subject: [Python-ideas] solving multi-core Python
In-Reply-To: <CALFfu7Cpv7mTfG7GcV0BuVKVMbuWQg5J6=4a3Nawd3QCkRDfbw@mail.gmail.com>
 (Eric Snow's message of "Sat, 20 Jun 2015 15:42:33 -0600")
References: <CALFfu7Cpv7mTfG7GcV0BuVKVMbuWQg5J6=4a3Nawd3QCkRDfbw@mail.gmail.com>
Message-ID: <m2wpm6ntop.fsf@news.realpath.org>

Eric Snow wrote:
> tl;dr Let's exploit multiple cores by fixing up subinterpreters,
> exposing them in Python, and adding a mechanism to safely share
> objects between them.
>
> This proposal is meant to be a shot over the bow, so to speak.  I plan
> on putting together a more complete PEP some time in the future, with
> content that is more refined along with references to the appropriate
> online resources.

I've heard very little about this since the original announcement
(https://lwn.net/Articles/650521/), so I was wondering if this is
still an active idea being worked on? Or has it turned out to be too
difficult?

Sebastian

From nas-pythonideas at arctrix.com  Fri Jun  3 09:17:30 2016
From: nas-pythonideas at arctrix.com (Neil Schemenauer)
Date: Fri, 3 Jun 2016 06:17:30 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
Message-ID: <20160603131730.GA29255@python.ca>

I'm nearly finished porting a decently size application from Python
2 to Python 3.  It has been a lot of work.  I am dubious as to
whether the port was really the best use of time.  I can imagine
there is still millions, possibly billions of lines of Python 2 code
that has not yet been converted.  Further, my guess is the lines of
Python 2 code are still growing faster than the lines of Python 3
compatible code.  IMHO, we need to do better in making it easier for
people to port code.

Here is a thought that occured to me.  Create a patched version of
Python 3.x, making a stepping stone version for people porting from
Python 2.  I know this would have been useful for me.  Specific
changes that would be helpful, all generating warnings so code can
be eventually fixed:

- comparision with None, smaller than all other types

- comparision of distinct types: use Python 2 behavior (i.e. order
  by type name)

- mixing of unicode strings with byte strings: decode/encode
  using latin-1

- dict objects: make keys() items() values() return special sequence
  that warns if iterated over multiple times or indexed as sequence

Changes that are not necessary as porting code is easy:

- print function syntax

- try/except syntax

- standard module renaming

- __next__/next()

- long type, literal syntax

- true division as default


From ethan at stoneleaf.us  Fri Jun  3 16:33:04 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 03 Jun 2016 13:33:04 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160603131730.GA29255@python.ca>
References: <20160603131730.GA29255@python.ca>
Message-ID: <5751E980.5000105@stoneleaf.us>

On 06/03/2016 06:17 AM, Neil Schemenauer wrote:

> Here is a thought that occured to me.  Create a patched version of
> Python 3.x, making a stepping stone version for people porting from
> Python 2.

This is highly unlikely.

I believe the efforts being made to make porting easier is in the 
optional type annotations, and using them is a double win:

- clarifies intent by specifying expected types going in and
   expected types coming back out

- type checker can then find certain classes of bugs to help with
   porting

--
~Ethan~

From ericsnowcurrently at gmail.com  Fri Jun  3 19:55:10 2016
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 3 Jun 2016 17:55:10 -0600
Subject: [Python-ideas] solving multi-core Python
In-Reply-To: <m2wpm6ntop.fsf@news.realpath.org>
References: <CALFfu7Cpv7mTfG7GcV0BuVKVMbuWQg5J6=4a3Nawd3QCkRDfbw@mail.gmail.com>
 <m2wpm6ntop.fsf@news.realpath.org>
Message-ID: <CALFfu7Ct3KhjBjteHt=Z+Co=+C9-E=pzCQzGzyBPhxTpzjAb1Q@mail.gmail.com>

On Fri, Jun 3, 2016 at 10:06 AM, Sebastian Krause
<sebastian at realpath.org> wrote:
> Eric Snow wrote:
>> tl;dr Let's exploit multiple cores by fixing up subinterpreters,
>> exposing them in Python, and adding a mechanism to safely share
>> objects between them.
>
> I've heard very little about this since the original announcement
> (https://lwn.net/Articles/650521/), so I was wondering if this is
> still an active idea being worked on? Or has it turned out to be too
> difficult?

Sorry for the lack of communication.  I tabled the project a while
back due to lack of time.  I'm still planning on writing something up
in the near future on where things are at, what's left, and what good
ideas may come out of this regardless.

-eric

From g.rodola at gmail.com  Fri Jun  3 20:13:07 2016
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 4 Jun 2016 02:13:07 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160603131730.GA29255@python.ca>
References: <20160603131730.GA29255@python.ca>
Message-ID: <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>

On Fri, Jun 3, 2016 at 3:17 PM, Neil Schemenauer <
nas-pythonideas at arctrix.com> wrote:

> I'm nearly finished porting a decently size application from Python
> 2 to Python 3.  It has been a lot of work.  I am dubious as to
> whether the port was really the best use of time.  I can imagine
> there is still millions, possibly billions of lines of Python 2 code
> that has not yet been converted.  Further, my guess is the lines of
> Python 2 code are still growing faster than the lines of Python 3
> compatible code.  IMHO, we need to do better in making it easier for
> people to port code.
>
> Here is a thought that occured to me.  Create a patched version of
> Python 3.x, making a stepping stone version for people porting from
> Python 2.  I know this would have been useful for me.  Specific
> changes that would be helpful, all generating warnings so code can
> be eventually fixed:
>
> - comparision with None, smaller than all other types
>
> - comparision of distinct types: use Python 2 behavior (i.e. order
>   by type name)
>
> - mixing of unicode strings with byte strings: decode/encode
>   using latin-1
>

To me this sounds more like going back to Python 2 rather than facilitating
the transition, especially #3.
Unfortunately there's no middle ground in there: it's either "do this or
that" and that's why it's so painful.
Incidentally, all these things you mentioned are bad practices which may
hide bugs you may not even be aware of.
The more I think about the porting subject, the more I realize that the
difficulty mainly resides in how good your Python 2 code base is. Python 3
attempted to be a better language (mainly) by becoming stricter. As such, a
badly written Python 2 code base is harder to port because Python 3 stands
in your way acting as a referee for all the mistakes you've made in the
past, forcing you to fix them all at once. That's its biggest advantage and
damnation at the same time.


-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160604/6faf7f7a/attachment.html>

From random832 at fastmail.com  Fri Jun  3 20:41:14 2016
From: random832 at fastmail.com (Random832)
Date: Fri, 03 Jun 2016 20:41:14 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>
Message-ID: <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>

On Fri, Jun 3, 2016, at 20:13, Giampaolo Rodola' wrote:
> On Fri, Jun 3, 2016 at 3:17 PM, Neil Schemenauer <
> nas-pythonideas at arctrix.com> wrote:
> > - mixing of unicode strings with byte strings: decode/encode
> >   using latin-1
> >
> 
> To me this sounds more like going back to Python 2 rather than
> facilitating
> the transition, especially #3.

What about moving forward to unify the types? For example, we could go
with the Emacs way: A single string abstract type*, which is a sequence
whose elements can be Unicode characters, or raw non-ASCII bytes, all of
which are distinct from each other (Emacs' representation is to assign
the high bytes "code points" between "U+3FFF80" and "U+3FFFFF").

*Emacs has two concrete types: "byte strings" which can contain no
non-ASCII characters, and "unicode strings" which use UTF-8 (plus those
extra code points) underlying representation [various indexing
operations are O(N)]. Python would use the FSR as it is now, along with
perhaps a "byte string" type which likewise can contain only ASCII
characters and high bytes. It might be worthwhile to have a 16-bit mode
for "BMP + high bytes" which omits, say, surrogates.

With only one type, mixing them becomes the easiest thing in the world.

From chris.barker at noaa.gov  Fri Jun  3 21:19:14 2016
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 3 Jun 2016 18:19:14 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>
References: <20160603131730.GA29255@python.ca>
 <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>
 <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>
Message-ID: <CALGmxEJq13y488ygVGLC7-wQmBOZYOYBO-AzzgLEVHQJySeVYA@mail.gmail.com>

On Fri, Jun 3, 2016 at 5:41 PM, Random832 <random832 at fastmail.com> wrote:

> *Emacs has two concrete types: "byte strings" which can contain no
> non-ASCII characters, and "unicode strings" which use UTF-8 (plus those
> extra code points)


Uhm, I'm confused, isn't the exactly what py3 has now? byte strings and
unicode strings?

_maybe_ you want a little more interoperability, but those types are there.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160603/96ad8dab/attachment.html>

From steve at pearwood.info  Sat Jun  4 03:31:13 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 4 Jun 2016 17:31:13 +1000
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160603131730.GA29255@python.ca>
References: <20160603131730.GA29255@python.ca>
Message-ID: <20160604073113.GW12028@ando.pearwood.info>

On Fri, Jun 03, 2016 at 06:17:30AM -0700, Neil Schemenauer wrote:

> Here is a thought that occured to me.  Create a patched version of
> Python 3.x, making a stepping stone version for people porting from
> Python 2.  I know this would have been useful for me.  Specific
> changes that would be helpful, all generating warnings so code can
> be eventually fixed:
> 
> - comparision with None, smaller than all other types

Sometimes I think that would be useful as a feature, not just for 2.x 
compatibility...


> - comparision of distinct types: use Python 2 behavior (i.e. order
>   by type name)
> - mixing of unicode strings with byte strings: decode/encode
>   using latin-1

These would be a reversion to harmful behaviour and I guarantee they 
would be abused.

But I don't understand why you want 3.x to raise a warning? It will 
raise an exception, which you then fix. How is this better than a 
warning that you will ignore?

If this is purely a budget issue ("my boss has given me six weeks to 
port, but it will take eight, however if we can get warnings and 
suppress them, we'll squeeze in under budget") then you have my sympathy 
but not much else.


> - dict objects: make keys() items() values() return special sequence
>   that warns if iterated over multiple times or indexed as sequence

This is surely unnecessary -- you just need to mechanically change your 
dict iteration to use dict.viewkeys() etc in Python 2, fix any 
problems, then mechanically remove the "view" in Python 3.



-- 
Steve

From ncoghlan at gmail.com  Sat Jun  4 04:05:37 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Jun 2016 01:05:37 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160603131730.GA29255@python.ca>
References: <20160603131730.GA29255@python.ca>
Message-ID: <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>

On 3 Jun 2016 13:17, "Neil Schemenauer" <nas-pythonideas at arctrix.com> wrote:
>
> I'm nearly finished porting a decently size application from Python
> 2 to Python 3.  It has been a lot of work.  I am dubious as to
> whether the port was really the best use of time.  I can imagine
> there is still millions, possibly billions of lines of Python 2 code
> that has not yet been converted.  Further, my guess is the lines of
> Python 2 code are still growing faster than the lines of Python 3
> compatible code.  IMHO, we need to do better in making it easier for
> people to port code.
>
> Here is a thought that occured to me.  Create a patched version of
> Python 3.x, making a stepping stone version for people porting from
> Python 2.  I know this would have been useful for me.  Specific
> changes that would be helpful, all generating warnings so code can
> be eventually fixed:
>
> - comparision with None, smaller than all other types

Static type checkers should already be able to find these cases today,
allowing them to be fixed incrementally pre-migration.

> - comparision of distinct types: use Python 2 behavior (i.e. order
>   by type name)

As above (and we definitely won't revert it - it's one of the more
appreciated changes reported by educators)

> - mixing of unicode strings with byte strings: decode/encode
>   using latin-1

Converting with latin-1 would be even less correct than Python 2's
behaviour of converting with ASCII (since it would allow arbitrary binary
data to be implicitly interpreted as text)

Type checkers don't generally help here yet, but are expected to in the
future.

> - dict objects: make keys() items() values() return special sequence
>   that warns if iterated over multiple times or indexed as sequence

These can be mechanically converted by futurize in a way that avoids a
redundant copy on Python 2 (modernize will insert a redundant call to list
for the Python 2 case)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160604/30f15076/attachment.html>

From steve at pearwood.info  Sat Jun  4 05:05:22 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 4 Jun 2016 19:05:22 +1000
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>
References: <20160603131730.GA29255@python.ca>
 <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>
 <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>
Message-ID: <20160604090522.GY12028@ando.pearwood.info>

On Fri, Jun 03, 2016 at 08:41:14PM -0400, Random832 wrote:

> What about moving forward to unify the types? For example, we could go
> with the Emacs way: A single string abstract type*, which is a sequence
> whose elements can be Unicode characters, or raw non-ASCII bytes, all of
> which are distinct from each other (Emacs' representation is to assign
> the high bytes "code points" between "U+3FFF80" and "U+3FFFFF").

I am fascinated by this concept. I think it might help solve the problem 
of "mixed text and bytes" files, but at the cost of throwing memory at 
it. Reading a file in binary mode would return a sequence of 32-bit 
"Unicode-plus-bytes" code points, rather than 8-bit bytes. Working in 
bytes would be more expensive, unless you happened to be lucky enough to 
only be dealing with bytes with the high bit cleared.

Basically, instead of having two types:

bytes: valid values are \x00 through \xFF;
text:  valid values are U+0000 through U+10FFFF

we'd have one:

text+bytes

interpreted as:


U+0000 through U+007F: bytes, or Unicode, depending on context;
U+0080 through U+10FFFF: only Unicode;
U+3FFF80 through U+3FFFFF: bytes \x80 through \xFF.

Values outside of those ranges are presumably impossible.

I'm not sure what implications there are for codecs.

The downside is that reading from a binary file would give a sequence of 
code points that require four bytes rather than one (except in the 
unusual case that *no* byte had had its high-bit set). Nor could you 
tell the difference between a bunch of ASCII bytes or ASCII text.

But maybe we could live with that?

> *Emacs has two concrete types: "byte strings" which can contain no
> non-ASCII characters, and "unicode strings" which use UTF-8 (plus those
> extra code points) underlying representation [various indexing
> operations are O(N)]. Python would use the FSR as it is now, along with
> perhaps a "byte string" type which likewise can contain only ASCII
> characters and high bytes.


Presumably for backwards compatibility we would keep bytes as they are 
now, and either add a new Mixed string type, or modify str to be mixed.

I'm still not entirely sure what the implications for encodings would 
be, but this is a promising idea with respect to mixed text/bytes.


-- 
Steve

From tritium-list at sdamon.com  Sat Jun  4 06:33:56 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Sat, 4 Jun 2016 06:33:56 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160603131730.GA29255@python.ca>
References: <20160603131730.GA29255@python.ca>
Message-ID: <087001d1be4c$99423520$cbc69f60$@hotmail.com>



> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Neil Schemenauer
> Sent: Friday, June 3, 2016 9:18 AM
> To: python-ideas at python.org
> Subject: [Python-ideas] Smoothing transition to Python 3
> 
...
 > - comparision with None, smaller than all other types

In a hypothetical python 2.8, the python 3 behavior would be in my backport
list...

> - comparision of distinct types: use Python 2 behavior (i.e. order
>   by type name)

...and this

> - mixing of unicode strings with byte strings: decode/encode
>   using latin-1

...and this
 
> - dict objects: make keys() items() values() return special sequence
>   that warns if iterated over multiple times or indexed as sequence

...and this.  In fact most of the things you list here are the GOOD ideas
that python 3 enforces that reduces bugs when avoided in python 2.

What would actually help the transition, in my world-view at least is 

* A bytes type like the string type in python 2 (*without implicit
conversion!*) There are too many real world use cases that the bytes type
makes painful, including anything dealing with networking.
* an alias to the string type named 'unicode' (this just makes polyglot a
whole heck of a lot less stressful... yes I do this myself, it's annoying,
if it was there by default, like bytes is in 2.7, it would make life a lot
easier.  One just never just never references `str`)
* a "magic" mapping from old to new module names.  In my experience, this is
actually a bigger pain than it looks.

I had discussed, informally on IRC, the concept of building a hypothetical
'final version of python 2' that is, in fact, libpython3 with a python2
parser and shims in front of it.  The general response was the theme of the
entire python 3 transition story:  "I don't see the value added."  This was
in response to more than just my crazy idea, but to the entire process and
existence of python 3.  I cannot disagree; the entire situation is a lot of
effort for very little value added.  Maybe a 'more compatible python 3' is a
solution, maybe not.

I have seen the pypi download numbers[1], and it is not encouraging for
python 3; python 2 outstrips python 3 by a factor of 10 in terms of package
downloads on linux, and a factor of 6 on mac.  The only platform where
python 3 has a real story against 2 is windows.

If python 3 really is to be the future, a lot more needs to be done to bring
the vast majority of users on board.  Telling them "Just port your code" is
not good enough.

 [1] https://s.caremad.io/WPVkP3Ruhg/ (query:
https://bpaste.net/show/56443668b83f)


From nas-pythonideas at arctrix.com  Sat Jun  4 04:12:39 2016
From: nas-pythonideas at arctrix.com (Neil Schemenauer)
Date: Sat, 4 Jun 2016 01:12:39 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
Message-ID: <20160604081239.GA20628@python.ca>

On 2016-06-04, Nick Coghlan wrote:
> On 3 Jun 2016 13:17, "Neil Schemenauer" <nas-pythonideas at arctrix.com> wrote:
> > - comparision with None, smaller than all other types
> 
> Static type checkers should already be able to find these cases today,
> allowing them to be fixed incrementally pre-migration.

Really?  Please point me to them as I have about 100k lines of code
that needs to be checked.  I can't imagine how a static checker
could find this.

> > - comparision of distinct types: use Python 2 behavior (i.e. order
> >   by type name)
> 
> As above (and we definitely won't revert it - it's one of the more
> appreciated changes reported by educators)

I'm not proposing to revert it.  Sometimes I wonder if people
actually read my proposal.  I want a version of Python between 2.7.x
and 3.x that allows it with a warning.

Do you not see how such a version of Python is useful?  Why was the
whole __future__ mechanism designed in the first place?

> > - mixing of unicode strings with byte strings: decode/encode
> >   using latin-1
> 
> Converting with latin-1 would be even less correct than Python 2's
> behaviour of converting with ASCII (since it would allow arbitrary binary
> data to be implicitly interpreted as text)

I'm not sure how to best handle this, maybe UTF-8 would be better.
An implicit conversion only happens in the cases where Python 3
raises a TypeError.  Getting rid of the conversion is easy, make the
object either a str or bytes and call encode/decode as necessary.

> Type checkers don't generally help here yet, but are expected to in the
> future.

Python 3 was released in 2008.  How long until we have these tools
that help people to port their code?

> > - dict objects: make keys() items() values() return special sequence
> >   that warns if iterated over multiple times or indexed as sequence
> 
> These can be mechanically converted by futurize in a way that avoids a
> redundant copy on Python 2 (modernize will insert a redundant call to list
> for the Python 2 case)

I think you are correct.  Blindly putting list() calls around the
method calls works.  People can change their Python 2 code to iter*
if they want to avoid that.

From nas-pythonideas at arctrix.com  Sat Jun  4 04:27:15 2016
From: nas-pythonideas at arctrix.com (Neil Schemenauer)
Date: Sat, 4 Jun 2016 01:27:15 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <087001d1be4c$99423520$cbc69f60$@hotmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
Message-ID: <20160604082715.GB20628@python.ca>

On 2016-06-04, tritium-list at sdamon.com wrote:
> In fact most of the things you list here are the GOOD ideas
> that python 3 enforces that reduces bugs when avoided in python 2.

Sure, and I'm not proposing that standard Python 3.x change the
behavior.

> What would actually help the transition, in my world-view at least is 
> 
> * A bytes type like the string type in python 2 (*without implicit
> conversion!*) There are too many real world use cases that the bytes type
> makes painful, including anything dealing with networking.

The Python 3 bytes type should gain whatever features it needs to
make things not painful.  The %-based formating in 3.5 is a big one.
Is there something else you miss?

> * an alias to the string type named 'unicode' (this just makes polyglot a
> whole heck of a lot less stressful... yes I do this myself, it's annoying,
> if it was there by default, like bytes is in 2.7, it would make life a lot
> easier.  One just never just never references `str`)

Maybe too late now but there should have been 'unicode',
'basestring' as aliases for 'str'.

> * a "magic" mapping from old to new module names.  In my experience, this is
> actually a bigger pain than it looks.

I would like to add this to my "pragmatic" version.

> The general response was the theme of the entire python 3
> transition story:  "I don't see the value added."

Yes, and here we are.  Python 3 is not yet winning and I'm not sure
it will.  I believe Dropbox, Facebook and Google are all still using
Python 2.  If porting code was so easy, why are they not moved over?
I see VMWare released some new IoT SDK:

    https://github.com/vmware/liota

This is new code, written this year.  It is not compatible with
Python 3 as far as I see.  I can't understand why people don't see
we have a problem.

  Neil

From me at jasonfried.info  Sat Jun  4 11:52:11 2016
From: me at jasonfried.info (Jason Fried)
Date: Sat, 04 Jun 2016 15:52:11 +0000
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604082715.GB20628@python.ca>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
Message-ID: <CAEYSsaQopjvyUN7X9JUTuPuZjVNb9Ee1M7T=z7Opwtokx6Tw0g@mail.gmail.com>

As said during pycon,
Facebook is using Python 3 and all new code is Python 3 by default. Google
is working on it.

Sounds like VMware just released a big ole load of tech debt

On Sat, Jun 4, 2016 at 08:20 Neil Schemenauer <nas-pythonideas at arctrix.com>
wrote:

> On 2016-06-04, tritium-list at sdamon.com wrote:
> > In fact most of the things you list here are the GOOD ideas
> > that python 3 enforces that reduces bugs when avoided in python 2.
>
> Sure, and I'm not proposing that standard Python 3.x change the
> behavior.
>
> > What would actually help the transition, in my world-view at least is
> >
> > * A bytes type like the string type in python 2 (*without implicit
> > conversion!*) There are too many real world use cases that the bytes type
> > makes painful, including anything dealing with networking.
>
> The Python 3 bytes type should gain whatever features it needs to
> make things not painful.  The %-based formating in 3.5 is a big one.
> Is there something else you miss?
>
> > * an alias to the string type named 'unicode' (this just makes polyglot a
> > whole heck of a lot less stressful... yes I do this myself, it's
> annoying,
> > if it was there by default, like bytes is in 2.7, it would make life a
> lot
> > easier.  One just never just never references `str`)
>
> Maybe too late now but there should have been 'unicode',
> 'basestring' as aliases for 'str'.
>
> > * a "magic" mapping from old to new module names.  In my experience,
> this is
> > actually a bigger pain than it looks.
>
> I would like to add this to my "pragmatic" version.
>
> > The general response was the theme of the entire python 3
> > transition story:  "I don't see the value added."
>
> Yes, and here we are.  Python 3 is not yet winning and I'm not sure
> it will.  I believe Dropbox, Facebook and Google are all still using
> Python 2.  If porting code was so easy, why are they not moved over?
> I see VMWare released some new IoT SDK:
>
>     https://github.com/vmware/liota
>
> This is new code, written this year.  It is not compatible with
> Python 3 as far as I see.  I can't understand why people don't see
> we have a problem.
>
>   Neil
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160604/91024f85/attachment.html>

From p.f.moore at gmail.com  Sat Jun  4 11:52:36 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 4 Jun 2016 16:52:36 +0100
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604081239.GA20628@python.ca>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
Message-ID: <CACac1F9444A3wB2Mjc17nT2iCDdSi2Xnc7yfGatPdQ_jsWgrGA@mail.gmail.com>

On 4 June 2016 at 09:12, Neil Schemenauer <nas-pythonideas at arctrix.com> wrote:
>> As above (and we definitely won't revert it - it's one of the more
>> appreciated changes reported by educators)
>
> I'm not proposing to revert it.  Sometimes I wonder if people
> actually read my proposal.  I want a version of Python between 2.7.x
> and 3.x that allows it with a warning.

Who do you propose develops that version? Will you do so to help the
next group of people in your situation? Even though there has been a
formal pronouncement that Python 2.8 will not happen, someone could
produce a fork, call it "transition Python" and add all the features
you're asking for. Would people use it? I don't know. But the people
who've asked for such a thing in the past have apparently never been
sufficiently convinced it was a worthwhile effort to put their own
time into it.

People with large Python 2 codebases don't have to port. The lack of
ongoing support for Python 2 from the core devs can be handled by
other means (pay your distribution vendor for extended support, accept
that Python 2 will freeze - it's not as if it's going to stop working
- or maintain your own patches for Python 2.7, lots of options).
Which, of course, may or may not suit your needs.

The big part of the equation that's yet to become a major issue is
libraries. At some point, library authors will start to face the
decision of whether maintaining Python 2 support is worth it. Maybe
only once Python 2 is no longer supported by the core devs, I don't
know. But I don't see any major libraries that are *not* supporting
Python 3 (even Twisted, probably the highest-profile example of a
library that's had major problems porting, is getting Python 3
support). So the Python 2 community faces the prospect of what to do
when libraries like pip, requests, numpy, ... stop supporting Python
2. (Equally, it would be a big issue if one of those major libraries
announced that it would no longer support Python 3, but my instinct
says that's unlikely).

So it seems to me that for Python 2 users, the options are:

1. Stay where you are:
   Advantages - no porting costs, no risk of introducing new bugs in the port
   Disadvantages - platform will become less well supported (may cost
money for support), key dependencies may also desupport Python 2,
adding to the support question.
2. Port to Python 3:
   Advantages - Ongoing free support from the Python community,
possibly better or cheaper vendor support
   Disadvantages - Porting cost, possibility of introducing new bugs
during the port.
3. Your proposal of creating a transitional version:
   Advantages - ??? Porting costs are smaller? But there's the cost of
developing the transitional version (or paying someone to do so).
Unless you were hoping someone would do that for free, or you think
that spreading the cost over a lot of people with the same needs will
reduce the cost sufficiently...
   Disadvantages - At the moment, this option doesn't exist. So delays
before you can even use it. And potential users (who might contribute
help to the effort) may not be willing to wait.

Any others? For me the killer there is "porting cost" which is why I
say you don't have to port. The costs may easily outweigh the
benefits. Maybe in your case, they did but nobody realised that in
time. Maybe there's not enough impartial information on how to
quantify the costs. I don't know.

Paul

From stephen at xemacs.org  Sat Jun  4 12:36:55 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 5 Jun 2016 01:36:55 +0900
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604090522.GY12028@ando.pearwood.info>
References: <20160603131730.GA29255@python.ca>
 <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>
 <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>
 <20160604090522.GY12028@ando.pearwood.info>
Message-ID: <22355.935.162553.989649@turnbull.sk.tsukuba.ac.jp>

Re: mixing text and bytes: We have mixed bytes and text already (PEP
383 surrogate escape codecs), as well as a space-efficient
representation of text (PEP 393).  Python already has improved
versions of Emacs's facilities.

PEPs tl;dr version: Emacs's rawbytes representation is obnoxiously
expensive (minimum 4 bytes per "raw byte"), Python's half as much (PEP
383 surrogates).  PEP 393 gives 1-byte representation not only of pure
ASCII (as characters), but Latin-1 as well.  I'm not sure if Emacs
still implements the horrible as-unibyte APIs (which give you access
to the internal represention, guaranteeing you'll cause data
corruption, and sometimes even crashes), but they've long been
deprecated and I suppose Python 3 can do something similar with memory
views.  (It was never automatic in Emacs, and it was extremely
dangerous.)

As for the OPs other desiderata, they are all similarly bad ideas, and
they *won't help in porting* because Python 3 will never get them,
Guido has many times said so (except perhaps None as "bottom" in all
comparisons, which as far as I know hasn't been categorically rejected).

Steve


From random832 at fastmail.com  Sat Jun  4 13:24:41 2016
From: random832 at fastmail.com (Random832)
Date: Sat, 04 Jun 2016 13:24:41 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CALGmxEJq13y488ygVGLC7-wQmBOZYOYBO-AzzgLEVHQJySeVYA@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CAFYqXL8SQBBUu_4OCALXkdEpS6bsTJV-rc0R9d5GWdrmKZbsxQ@mail.gmail.com>
 <1465000874.1726295.627514497.5D948182@webmail.messagingengine.com>
 <CALGmxEJq13y488ygVGLC7-wQmBOZYOYBO-AzzgLEVHQJySeVYA@mail.gmail.com>
Message-ID: <1465061081.3479014.627930409.2111394C@webmail.messagingengine.com>

On Fri, Jun 3, 2016, at 21:19, Chris Barker wrote:
> On Fri, Jun 3, 2016 at 5:41 PM, Random832 <random832 at fastmail.com> wrote:
> 
> > *Emacs has two concrete types: "byte strings" which can contain no
> > non-ASCII characters, and "unicode strings" which use UTF-8 (plus those
> > extra code points)
> 
> 
> Uhm, I'm confused, isn't the exactly what py3 has now? byte strings and
> unicode strings?
> 
> _maybe_ you want a little more interoperability, but those types are
> there.

Er, no, I'm talking about two concrete implementations of the abstract
string type. Same as how Python has four: ASCII, Latin-1, UCS-2, and
UCS-4.

From nas-pythonideas at arctrix.com  Sat Jun  4 07:34:06 2016
From: nas-pythonideas at arctrix.com (Neil Schemenauer)
Date: Sat, 4 Jun 2016 04:34:06 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CACac1F9444A3wB2Mjc17nT2iCDdSi2Xnc7yfGatPdQ_jsWgrGA@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CACac1F9444A3wB2Mjc17nT2iCDdSi2Xnc7yfGatPdQ_jsWgrGA@mail.gmail.com>
Message-ID: <20160604113406.GA30634@python.ca>

On 2016-06-04, Paul Moore wrote:
> Who do you propose develops that version? Will you do so to help the
> next group of people in your situation?

I'm proposing to help.  Here is a very early start:

https://github.com/nascheme/ppython

> But the people who've asked for such a thing in the past have
> apparently never been sufficiently convinced it was a worthwhile
> effort to put their own time into it.

Given the choice of porting your code to Python 3 or developing this
transitional Python to help you, it is easy to see why people choose
the former option.  However, if you look at the millions of lines of
Python 2 code that is yet to be ported, I think the idea has some merit.

  Neil

From p.f.moore at gmail.com  Sat Jun  4 15:28:07 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 4 Jun 2016 20:28:07 +0100
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604113406.GA30634@python.ca>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CACac1F9444A3wB2Mjc17nT2iCDdSi2Xnc7yfGatPdQ_jsWgrGA@mail.gmail.com>
 <20160604113406.GA30634@python.ca>
Message-ID: <CACac1F-wRc7EdKjtFyD16QV9nnMRfn-2MixLZ9ubiMj62RO4Mw@mail.gmail.com>

On 4 June 2016 at 12:34, Neil Schemenauer <nas-pythonideas at arctrix.com> wrote:
> On 2016-06-04, Paul Moore wrote:
>> Who do you propose develops that version? Will you do so to help the
>> next group of people in your situation?
>
> I'm proposing to help.  Here is a very early start:
>
> https://github.com/nascheme/ppython

Cool!

>> But the people who've asked for such a thing in the past have
>> apparently never been sufficiently convinced it was a worthwhile
>> effort to put their own time into it.
>
> Given the choice of porting your code to Python 3 or developing this
> transitional Python to help you, it is easy to see why people choose
> the former option.  However, if you look at the millions of lines of
> Python 2 code that is yet to be ported, I think the idea has some merit.

Sounds like you know where you want to go with this, I hope you get
the interest to keep the project going. I can't say I'd recommend
people go this route myself, but maybe that's because I'm not looking
at projects on the scale you are - typically the things I've been
involved in haven't been that hard to port (I work mostly on
libraries, and working in the common Python 2/3 subset, with the aid
of libraries like six, has been a perfectly practical solution for
me).

Paul

From tritium-list at sdamon.com  Sat Jun  4 19:29:10 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Sat, 4 Jun 2016 19:29:10 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604082715.GB20628@python.ca>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
Message-ID: <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>



> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Neil Schemenauer
> Sent: Saturday, June 4, 2016 4:27 AM
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Smoothing transition to Python 3
> 
> On 2016-06-04, tritium-list at sdamon.com wrote:
> > In fact most of the things you list here are the GOOD ideas
> > that python 3 enforces that reduces bugs when avoided in python 2.
> 
> Sure, and I'm not proposing that standard Python 3.x change the
> behavior.
> 
> > What would actually help the transition, in my world-view at least is
> >
> > * A bytes type like the string type in python 2 (*without implicit
> > conversion!*) There are too many real world use cases that the bytes
type
> > makes painful, including anything dealing with networking.
> 
> The Python 3 bytes type should gain whatever features it needs to
> make things not painful.  The %-based formating in 3.5 is a big one.
> Is there something else you miss?
> 

b'Foo'[0] returning b'F' instead of the int 70

> > * an alias to the string type named 'unicode' (this just makes polyglot
a
> > whole heck of a lot less stressful... yes I do this myself, it's
annoying,
> > if it was there by default, like bytes is in 2.7, it would make life a
lot
> > easier.  One just never just never references `str`)
> 
> Maybe too late now but there should have been 'unicode',
> 'basestring' as aliases for 'str'.
> 
> > * a "magic" mapping from old to new module names.  In my experience,
> this is
> > actually a bigger pain than it looks.
> 
> I would like to add this to my "pragmatic" version.
> 
> > The general response was the theme of the entire python 3
> > transition story:  "I don't see the value added."
> 
> Yes, and here we are.  Python 3 is not yet winning and I'm not sure
> it will.  I believe Dropbox, Facebook and Google are all still using
> Python 2.  If porting code was so easy, why are they not moved over?
> I see VMWare released some new IoT SDK:
> 
>     https://github.com/vmware/liota
> 
> This is new code, written this year.  It is not compatible with
> Python 3 as far as I see.  I can't understand why people don't see
> we have a problem.
> 
>   Neil
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From guido at python.org  Sat Jun  4 19:37:40 2016
From: guido at python.org (Guido van Rossum)
Date: Sat, 4 Jun 2016 16:37:40 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
Message-ID: <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>

The bytes -> int behavior is widely considered a mistake. We're just
not able to fix it without yet another round of layoffs ^W
deprecations. And I'm not ready for that -- not even Python 4 should
be allowed to change this unilaterally. Though maybe we could do
something with a __future__ import.

On Sat, Jun 4, 2016 at 4:29 PM,  <tritium-list at sdamon.com> wrote:
>
>
>> -----Original Message-----
>> From: Python-ideas [mailto:python-ideas-bounces+tritium-
>> list=sdamon.com at python.org] On Behalf Of Neil Schemenauer
>> Sent: Saturday, June 4, 2016 4:27 AM
>> To: python-ideas at python.org
>> Subject: Re: [Python-ideas] Smoothing transition to Python 3
>>
>> On 2016-06-04, tritium-list at sdamon.com wrote:
>> > In fact most of the things you list here are the GOOD ideas
>> > that python 3 enforces that reduces bugs when avoided in python 2.
>>
>> Sure, and I'm not proposing that standard Python 3.x change the
>> behavior.
>>
>> > What would actually help the transition, in my world-view at least is
>> >
>> > * A bytes type like the string type in python 2 (*without implicit
>> > conversion!*) There are too many real world use cases that the bytes
> type
>> > makes painful, including anything dealing with networking.
>>
>> The Python 3 bytes type should gain whatever features it needs to
>> make things not painful.  The %-based formating in 3.5 is a big one.
>> Is there something else you miss?
>>
>
> b'Foo'[0] returning b'F' instead of the int 70
>
>> > * an alias to the string type named 'unicode' (this just makes polyglot
> a
>> > whole heck of a lot less stressful... yes I do this myself, it's
> annoying,
>> > if it was there by default, like bytes is in 2.7, it would make life a
> lot
>> > easier.  One just never just never references `str`)
>>
>> Maybe too late now but there should have been 'unicode',
>> 'basestring' as aliases for 'str'.
>>
>> > * a "magic" mapping from old to new module names.  In my experience,
>> this is
>> > actually a bigger pain than it looks.
>>
>> I would like to add this to my "pragmatic" version.
>>
>> > The general response was the theme of the entire python 3
>> > transition story:  "I don't see the value added."
>>
>> Yes, and here we are.  Python 3 is not yet winning and I'm not sure
>> it will.  I believe Dropbox, Facebook and Google are all still using
>> Python 2.  If porting code was so easy, why are they not moved over?
>> I see VMWare released some new IoT SDK:
>>
>>     https://github.com/vmware/liota
>>
>> This is new code, written this year.  It is not compatible with
>> Python 3 as far as I see.  I can't understand why people don't see
>> we have a problem.
>>
>>   Neil
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
--Guido van Rossum (python.org/~guido)

From tritium-list at sdamon.com  Sat Jun  4 20:17:46 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Sat, 4 Jun 2016 20:17:46 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
Message-ID: <092901d1bebf$b001e440$1005acc0$@hotmail.com>

Unless you changed your mind about version numbing (and the digit count in each slot of a version number), I don?t think we will be considering breaking changes till python 5 :)

I am totally unopposed to locking these kinds of shims/hacks/shames in a __future__ (__past__?) import.

> -----Original Message-----
> From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf
> Of Guido van Rossum
> Sent: Saturday, June 4, 2016 7:38 PM
> To: Alexander Walters <tritium-list at sdamon.com>
> Cc: Neil Schemenauer <nas-pythonideas at arctrix.com>; Python-Ideas
> <python-ideas at python.org>
> Subject: Re: [Python-ideas] Smoothing transition to Python 3
> 
> The bytes -> int behavior is widely considered a mistake. We're just
> not able to fix it without yet another round of layoffs ^W
> deprecations. And I'm not ready for that -- not even Python 4 should
> be allowed to change this unilaterally. Though maybe we could do
> something with a __future__ import.
> 
> On Sat, Jun 4, 2016 at 4:29 PM,  <tritium-list at sdamon.com> wrote:
> >
> >
> >> -----Original Message-----
> >> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> >> list=sdamon.com at python.org] On Behalf Of Neil Schemenauer
> >> Sent: Saturday, June 4, 2016 4:27 AM
> >> To: python-ideas at python.org
> >> Subject: Re: [Python-ideas] Smoothing transition to Python 3
> >>
> >> On 2016-06-04, tritium-list at sdamon.com wrote:
> >> > In fact most of the things you list here are the GOOD ideas
> >> > that python 3 enforces that reduces bugs when avoided in python 2.
> >>
> >> Sure, and I'm not proposing that standard Python 3.x change the
> >> behavior.
> >>
> >> > What would actually help the transition, in my world-view at least is
> >> >
> >> > * A bytes type like the string type in python 2 (*without implicit
> >> > conversion!*) There are too many real world use cases that the bytes
> > type
> >> > makes painful, including anything dealing with networking.
> >>
> >> The Python 3 bytes type should gain whatever features it needs to
> >> make things not painful.  The %-based formating in 3.5 is a big one.
> >> Is there something else you miss?
> >>
> >
> > b'Foo'[0] returning b'F' instead of the int 70
> >
> >> > * an alias to the string type named 'unicode' (this just makes polyglot
> > a
> >> > whole heck of a lot less stressful... yes I do this myself, it's
> > annoying,
> >> > if it was there by default, like bytes is in 2.7, it would make life a
> > lot
> >> > easier.  One just never just never references `str`)
> >>
> >> Maybe too late now but there should have been 'unicode',
> >> 'basestring' as aliases for 'str'.
> >>
> >> > * a "magic" mapping from old to new module names.  In my experience,
> >> this is
> >> > actually a bigger pain than it looks.
> >>
> >> I would like to add this to my "pragmatic" version.
> >>
> >> > The general response was the theme of the entire python 3
> >> > transition story:  "I don't see the value added."
> >>
> >> Yes, and here we are.  Python 3 is not yet winning and I'm not sure
> >> it will.  I believe Dropbox, Facebook and Google are all still using
> >> Python 2.  If porting code was so easy, why are they not moved over?
> >> I see VMWare released some new IoT SDK:
> >>
> >>     https://github.com/vmware/liota
> >>
> >> This is new code, written this year.  It is not compatible with
> >> Python 3 as far as I see.  I can't understand why people don't see
> >> we have a problem.
> >>
> >>   Neil
> >> _______________________________________________
> >> Python-ideas mailing list
> >> Python-ideas at python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> 
> --
> --Guido van Rossum (python.org/~guido)


From random832 at fastmail.com  Sat Jun  4 20:23:31 2016
From: random832 at fastmail.com (Random832)
Date: Sat, 04 Jun 2016 20:23:31 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
Message-ID: <1465086211.921728.628108937.68DF4326@webmail.messagingengine.com>

On Sat, Jun 4, 2016, at 19:37, Guido van Rossum wrote:
> The bytes -> int behavior is widely considered a mistake. We're just
> not able to fix it without yet another round of layoffs ^W
> deprecations. And I'm not ready for that -- not even Python 4 should
> be allowed to change this unilaterally. Though maybe we could do
> something with a __future__ import.

I did have some thoughts (in the discussion a week or so ago about int
division returning float vs Fraction) on how a __future__ flag (which
have to be file scoped) effecting a change in the behavior of an
operator of a class could be implemented without a
__truediv__/__floordiv__ dichotomy intended to stick around forever.

In this case: Change the actual behavior of bytes.__getitem__ (direct
callers aren't important enough to matter) to return a 1-length bytes
instance. Add a compile flag (enabled by default, disabled by the future
import) which causes foo[bar] to become __oldgetitem__(foo, bar) where:

def __oldgetitem__(obj, i):
    if isinstance(obj, bytes) and not isinstance(i, slice):
        return ord(obj.__getitem__(i))
    else:
        return obj.__getitem__(i)

Or instead of literally calling a function, we could add another byte
code that directly implements this behavior. The question is how big the
performance hit is and how important is the performance of code that
doesn't use the future import, considering dynamic typing means every []
operator (except perhaps those statically knowable to be slices) goes
through this.

From tritium-list at sdamon.com  Sat Jun  4 20:28:04 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Sat, 4 Jun 2016 20:28:04 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <1465086211.921728.628108937.68DF4326@webmail.messagingengine.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <1465086211.921728.628108937.68DF4326@webmail.messagingengine.com>
Message-ID: <092b01d1bec1$2095f880$61c1e980$@hotmail.com>

In my experience (and if this is wrong, ignore the rest of this message),
python 3 is faster than python 2.  Since these are hacks primarily to get
people off of python 2... you only really have to hit python 2 speeds.  A
minor performance hit is acceptable.

> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Random832
> Sent: Saturday, June 4, 2016 8:24 PM
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Smoothing transition to Python 3
> 
> On Sat, Jun 4, 2016, at 19:37, Guido van Rossum wrote:
> > The bytes -> int behavior is widely considered a mistake. We're just
> > not able to fix it without yet another round of layoffs ^W
> > deprecations. And I'm not ready for that -- not even Python 4 should
> > be allowed to change this unilaterally. Though maybe we could do
> > something with a __future__ import.
> 
> I did have some thoughts (in the discussion a week or so ago about int
> division returning float vs Fraction) on how a __future__ flag (which
> have to be file scoped) effecting a change in the behavior of an
> operator of a class could be implemented without a
> __truediv__/__floordiv__ dichotomy intended to stick around forever.
> 
> In this case: Change the actual behavior of bytes.__getitem__ (direct
> callers aren't important enough to matter) to return a 1-length bytes
> instance. Add a compile flag (enabled by default, disabled by the future
> import) which causes foo[bar] to become __oldgetitem__(foo, bar) where:
> 
> def __oldgetitem__(obj, i):
>     if isinstance(obj, bytes) and not isinstance(i, slice):
>         return ord(obj.__getitem__(i))
>     else:
>         return obj.__getitem__(i)
> 
> Or instead of literally calling a function, we could add another byte
> code that directly implements this behavior. The question is how big the
> performance hit is and how important is the performance of code that
> doesn't use the future import, considering dynamic typing means every []
> operator (except perhaps those statically knowable to be slices) goes
> through this.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From ncoghlan at gmail.com  Sun Jun  5 01:14:19 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Jun 2016 22:14:19 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604081239.GA20628@python.ca>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
Message-ID: <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>

On 4 Jun 2016 8:06 am, "Neil Schemenauer" <nas-pythonideas at arctrix.com>
wrote:
>
> On 2016-06-04, Nick Coghlan wrote:
> > On 3 Jun 2016 13:17, "Neil Schemenauer" <nas-pythonideas at arctrix.com>
wrote:
> > > - comparision with None, smaller than all other types
> >
> > Static type checkers should already be able to find these cases today,
> > allowing them to be fixed incrementally pre-migration.
>
> Really?  Please point me to them as I have about 100k lines of code
> that needs to be checked.  I can't imagine how a static checker
> could find this.

It depends on APIs being correctly flagged as returning Optional results
(although I don't know if the current crop of type checkers are clever
enough to know Optional values aren't orderable)

> > > - comparision of distinct types: use Python 2 behavior (i.e. order
> > >   by type name)
> >
> > As above (and we definitely won't revert it - it's one of the more
> > appreciated changes reported by educators)
>
> I'm not proposing to revert it.  Sometimes I wonder if people
> actually read my proposal.  I want a version of Python between 2.7.x
> and 3.x that allows it with a warning.
>
> Do you not see how such a version of Python is useful?  Why was the
> whole __future__ mechanism designed in the first place?

The -3 switch in Python 2.7 should already warn about these cases. If it
doesn't, new -3 warnings are permitted in 2.7 maintenance releases.

> > > - mixing of unicode strings with byte strings: decode/encode
> > >   using latin-1
> >
> > Converting with latin-1 would be even less correct than Python 2's
> > behaviour of converting with ASCII (since it would allow arbitrary
binary
> > data to be implicitly interpreted as text)
>
> I'm not sure how to best handle this, maybe UTF-8 would be better.
> An implicit conversion only happens in the cases where Python 3
> raises a TypeError.  Getting rid of the conversion is easy, make the
> object either a str or bytes and call encode/decode as necessary.
>
> > Type checkers don't generally help here yet, but are expected to in the
> > future.
>
> Python 3 was released in 2008.  How long until we have these tools
> that help people to port their code?

We already have a number of them, many linked from the porting guide in the
main documentation. python-future.org is one of the most comprehensive in
providing a Python 3 like experience atop Python 2 (but also most invasive
for existing Python 2 projects).

Red Hat's Python maintenance team are working on a more explicitly
minimalistic porting guide based on their experiences porting Fedora
components, sometimes in the context of upstream projects that are more
interested in keeping Python 2.4 support than they are in Python 3:
http://portingguide.readthedocs.io/en/latest/

For projects with the code coverage needed to facilitate significant
refactorings, these existing tools should be sufficient in most cases.
Where the static analysers being worked on by Dropbox, Google, etc are
likely to help most is with code bases that *don't* have that kind of test
coverage - good static analysis tools can help compensate for the gaps in
test coverage.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160604/64f99f3e/attachment-0001.html>

From jelle.zijlstra at gmail.com  Sun Jun  5 03:36:42 2016
From: jelle.zijlstra at gmail.com (Jelle Zijlstra)
Date: Sun, 5 Jun 2016 00:36:42 -0700
Subject: [Python-ideas] Exposing regular expression bytecode
In-Reply-To: <CAK256p1GFepP+q_0B-rf-=v1yCqorA8-_=gR1VgtTb645+RfQw@mail.gmail.com>
References: <CAK256p0RwZSxSOtAbS9sD3e+cSJQLXFVkLPZ=02B+9tLWxDSdw@mail.gmail.com>
 <CACfEFw_E4UkKvTTOFgjJH=m_Tm0USSiFq_NEZdRKh7t0qbQfjA@mail.gmail.com>
 <CAK256p1TgVOadvAUt2vscetY2=wDPOqdGFxq65wx9E3Qt31hAw@mail.gmail.com>
 <CAK256p1PVJ_ELb4BrzNxROx6yQs8FRvX3N2sruhAo88N4SVR1A@mail.gmail.com>
 <D1E4933A-1B91-46AA-8F68-2FF80A909454@yahoo.com>
 <CAK256p0B2xLS_j-xgxKTke33osZOyDz1aiXQiKNV3NL4Q4kkDA@mail.gmail.com>
 <475EF1D2-E27E-4AB6-9729-66B00470D60F@yahoo.com>
 <CAK256p0u9As0ATW9ez4E5ZJCRs4TfpUy0jSY=APt7R_mZ6HbBw@mail.gmail.com>
 <56C34D84.40509@gmail.com>
 <CAK256p3zFtMfC0UpczVk1E6TdyDm=_v0anqMO9ufzKkRD_Kjdw@mail.gmail.com>
 <CACac1F_Pm0Bm8CTAKFWpmhZ8vw1QHvT4+1gFWRLoFhb+LLXsSA@mail.gmail.com>
 <CAK256p1GFepP+q_0B-rf-=v1yCqorA8-_=gR1VgtTb645+RfQw@mail.gmail.com>
Message-ID: <CAFp3-p8hmx+duTtrwtdkdfqdNvOGMpvBJPy3JdTyiXdiXUukew@mail.gmail.com>

2016-02-19 11:42 GMT-08:00 Jonathan Goble <jcgoble3 at gmail.com>:

> FWIW, I've decided to shelve this idea for the time being, at least,
> as I've had some things come up unexpectedly that are going to eat
> into my available time for the foreseeable future, so I no longer have
> time to pursue this myself. (Suffice it to say that Real Life always
> has crappy timing. :-P)
>
> Maybe I'll have time to resurrect it in the future; in the meantime,
> the issue on the bug tracker remains open in the event someone gets
> bored and decides to take a crack at it.
>
I found the bugtracker issue during the PyCon sprints and decided to
implement the feature: http://bugs.python.org/issue26336.

> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160605/6d99231b/attachment.html>

From jcgoble3 at gmail.com  Sun Jun  5 03:43:00 2016
From: jcgoble3 at gmail.com (Jonathan Goble)
Date: Sun, 5 Jun 2016 03:43:00 -0400
Subject: [Python-ideas] Exposing regular expression bytecode
In-Reply-To: <CAFp3-p8hmx+duTtrwtdkdfqdNvOGMpvBJPy3JdTyiXdiXUukew@mail.gmail.com>
References: <CAK256p0RwZSxSOtAbS9sD3e+cSJQLXFVkLPZ=02B+9tLWxDSdw@mail.gmail.com>
 <CACfEFw_E4UkKvTTOFgjJH=m_Tm0USSiFq_NEZdRKh7t0qbQfjA@mail.gmail.com>
 <CAK256p1TgVOadvAUt2vscetY2=wDPOqdGFxq65wx9E3Qt31hAw@mail.gmail.com>
 <CAK256p1PVJ_ELb4BrzNxROx6yQs8FRvX3N2sruhAo88N4SVR1A@mail.gmail.com>
 <D1E4933A-1B91-46AA-8F68-2FF80A909454@yahoo.com>
 <CAK256p0B2xLS_j-xgxKTke33osZOyDz1aiXQiKNV3NL4Q4kkDA@mail.gmail.com>
 <475EF1D2-E27E-4AB6-9729-66B00470D60F@yahoo.com>
 <CAK256p0u9As0ATW9ez4E5ZJCRs4TfpUy0jSY=APt7R_mZ6HbBw@mail.gmail.com>
 <56C34D84.40509@gmail.com>
 <CAK256p3zFtMfC0UpczVk1E6TdyDm=_v0anqMO9ufzKkRD_Kjdw@mail.gmail.com>
 <CACac1F_Pm0Bm8CTAKFWpmhZ8vw1QHvT4+1gFWRLoFhb+LLXsSA@mail.gmail.com>
 <CAK256p1GFepP+q_0B-rf-=v1yCqorA8-_=gR1VgtTb645+RfQw@mail.gmail.com>
 <CAFp3-p8hmx+duTtrwtdkdfqdNvOGMpvBJPy3JdTyiXdiXUukew@mail.gmail.com>
Message-ID: <CAK256p1deFjacEKoEwtm-3EGPOgA42VY0eguEyB6MCrn_5kD_A@mail.gmail.com>

On Sun, Jun 5, 2016 at 3:36 AM, Jelle Zijlstra <jelle.zijlstra at gmail.com> wrote:
>
> 2016-02-19 11:42 GMT-08:00 Jonathan Goble <jcgoble3 at gmail.com>:
>>
>> FWIW, I've decided to shelve this idea for the time being, at least,
>> as I've had some things come up unexpectedly that are going to eat
>> into my available time for the foreseeable future, so I no longer have
>> time to pursue this myself. (Suffice it to say that Real Life always
>> has crappy timing. :-P)
>>
>> Maybe I'll have time to resurrect it in the future; in the meantime,
>> the issue on the bug tracker remains open in the event someone gets
>> bored and decides to take a crack at it.
>
> I found the bugtracker issue during the PyCon sprints and decided to implement the feature: http://bugs.python.org/issue26336.

Great, thanks! I had actually forgotten about this, as real life is
still pretty hectic (I'm currently in the middle of searching for a
job). I'm glad someone is interested in working on it.

From g.rodola at gmail.com  Sun Jun  5 12:37:48 2016
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sun, 5 Jun 2016 18:37:48 +0200
Subject: [Python-ideas] 1 + True = 2
Message-ID: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>

>>> 1 + True
2
>>> 1 + False
1
>>>

I bumped into this today by accident and I can't recall ever being aware of
this. Why isn't this a TypeError in Python 3?

-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160605/71c9deaf/attachment.html>

From rosuav at gmail.com  Sun Jun  5 12:48:50 2016
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 6 Jun 2016 02:48:50 +1000
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
Message-ID: <CAPTjJmpGS_1baCMQMSpND=-FhKOaS5Ov=DwVmbps+uikpLWBsw@mail.gmail.com>

On Mon, Jun 6, 2016 at 2:37 AM, Giampaolo Rodola' <g.rodola at gmail.com> wrote:
>>>> 1 + True
> 2
>>>> 1 + False
> 1
>>>>
>
> I bumped into this today by accident and I can't recall ever being aware of
> this. Why isn't this a TypeError in Python 3?
>

Because it's a good thing. bool is a subclass of int, True is exactly
1, False is exactly 0. You can sum a series of conditions and get the
number that are true, for instance. There's no point doing this:

sum(1 if x < 5 else 0 for x in stuff)

when you can just do this:

sum(x < 5 for x in stuff)

to quickly count the values that fit some condition.

ChrisA

From steve at pearwood.info  Sun Jun  5 13:23:13 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 6 Jun 2016 03:23:13 +1000
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
Message-ID: <20160605172313.GA12028@ando.pearwood.info>

On Sun, Jun 05, 2016 at 06:37:48PM +0200, Giampaolo Rodola' wrote:
> >>> 1 + True
> 2
> >>> 1 + False
> 1
> >>>
> 
> I bumped into this today by accident and I can't recall ever being aware of
> this. Why isn't this a TypeError in Python 3?

Because bool is a subclass of int, and has been since it was first 
introduced back in Python 2.2. I'm not quite sure what your question is 
about... are you asking what was the original justification for making 
True and False equal to 1 and 0? If so, see the PEP:

https://www.python.org/dev/peps/pep-0285/

Or do you mean to ask why it wasn't changed in Python 3? Well, changed 
to what? All the reasons given in the PEP still hold, and there's no 
really compelling reason to make bool something else. Since bools aren't 
broken, why change them?


-- 
Steve

From g.rodola at gmail.com  Sun Jun  5 14:37:29 2016
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sun, 5 Jun 2016 20:37:29 +0200
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <20160605172313.GA12028@ando.pearwood.info>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
 <20160605172313.GA12028@ando.pearwood.info>
Message-ID: <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>

On Sun, Jun 5, 2016 at 7:23 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sun, Jun 05, 2016 at 06:37:48PM +0200, Giampaolo Rodola' wrote:
> > >>> 1 + True
> > 2
> > >>> 1 + False
> > 1
> > >>>
> >
> > I bumped into this today by accident and I can't recall ever being aware
> of
> > this. Why isn't this a TypeError in Python 3?
>
> Because bool is a subclass of int, and has been since it was first
> introduced back in Python 2.2. I'm not quite sure what your question is
> about... are you asking what was the original justification for making
> True and False equal to 1 and 0? If so, see the PEP:
>
> https://www.python.org/dev/peps/pep-0285/
>
> Or do you mean to ask why it wasn't changed in Python 3? Well, changed
> to what? All the reasons given in the PEP still hold, and there's no
> really compelling reason to make bool something else. Since bools aren't
> broken, why change them?
>

I've read through the PEP and I understand the rationale about why
True/False is a subclass of int and I'm OK with that. PEP-285 at chapter 6
suggests a strong practical reason for this:

*       In an ideal world, bool might be better implemented as a*
*       separate integer type that knows how to perform mixed-mode*
*       arithmetic.  However, inheriting bool from int eases the*
*       implementation enormously (in part since all C code that calls*
*       PyInt_Check() will continue to work -- this returns true for*
*       subclasses of int)*

What I find odd though is that a bool can be used in arithmetical
operations as if it was the exact same thing as a number. I mean, to me
this is just odd:

>>> 1 + True
2

I would have expected that to be treated the same as:

>>> 1 + "1"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

A bool may be a subclass of int but conceptually (to me at least) it's
different almost the same way "1" is different than 1. I say "almost" only
because bool is a subclass of int, but then again, to my understanding that
was done for practical reasons (implementation details), not because the
main intention was to explicitly allow mixing bools and numbers in
arithmetical operations.

On the other hand (and I'm gonna contradict myself with what I've just said
above) on chapter 4:

*       There's a small but vocal minority that would prefer to see*
*       "textbook" bools that don't support arithmetic operations at*
*       all, but most reviewers agree with me that bools should always*
*       allow arithmetic operations.*

...so maybe supporting arithmetical operations was also a primary
intention, in which case my question is "why?".

-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160605/165a08c5/attachment.html>

From ncoghlan at gmail.com  Sun Jun  5 15:04:34 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 5 Jun 2016 12:04:34 -0700
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
 <20160605172313.GA12028@ando.pearwood.info>
 <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
Message-ID: <CADiSq7d444u_dS0H-zzb_UmfzaJPzC056ePmq9PfoA=ZYz9qyA@mail.gmail.com>

On 5 Jun 2016 11:38, "Giampaolo Rodola'" <g.rodola at gmail.com> wrote:
>
> On the other hand (and I'm gonna contradict myself with what I've just
said above) on chapter 4:
>
>        There's a small but vocal minority that would prefer to see
>        "textbook" bools that don't support arithmetic operations at
>        all, but most reviewers agree with me that bools should always
>        allow arithmetic operations.
>
> ...so maybe supporting arithmetical operations was also a primary
intention, in which case my question is "why?".

The inheritance from int meant the default behaviour was to support
type-promoting integer arithmetic operations in addition to bitwise
arithmetic.

That changes the question from "Why support that?" to "Why do the extra
design, documentation and implementation work needed to prevent that?".

The fact that "1 + True == 2" is surprising hasn't proven to be enough to
motivate anyone to define the precise subset of operations they want to
prevent, and then make the case for those restrictions as Python's native
behaviour.

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160605/38c90bbc/attachment-0001.html>

From tjreedy at udel.edu  Sun Jun  5 15:53:31 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 5 Jun 2016 15:53:31 -0400
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
Message-ID: <nj2001$aff$1@ger.gmane.org>

On 6/5/2016 12:37 PM, Giampaolo Rodola' wrote:
>>>> 1 + True
> 2
>>>> 1 + False
> 1
>>>>
>
> I bumped into this today by accident and I can't recall ever being aware
> of this. Why isn't this a TypeError in Python 3?

Questions like this belong on python-list (where it has been discussed 
before).

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Sun Jun  5 21:20:23 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 5 Jun 2016 21:20:23 -0400
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
 <20160605172313.GA12028@ando.pearwood.info>
 <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
Message-ID: <nj2j4v$mon$1@ger.gmane.org>

On 6/5/2016 2:37 PM, Giampaolo Rodola' wrote:
>
> On Sun, Jun 5, 2016 at 7:23 PM, Steven D'Aprano
> <steve at pearwood.info
> <mailto:steve at pearwood.info>> wrote:
>
>     On Sun, Jun 05, 2016 at 06:37:48PM +0200, Giampaolo Rodola' wrote:
>     > >>> 1 + True
>     > 2
>     > >>> 1 + False
>     > 1
>     > >>>
>     >
>     > I bumped into this today by accident and I can't recall ever being aware of
>     > this. Why isn't this a TypeError in Python 3?
>
>     Because bool is a subclass of int, and has been since it was first
>     introduced back in Python 2.2. I'm not quite sure what your question is
>     about... are you asking what was the original justification for making
>     True and False equal to 1 and 0? If so, see the PEP:
>
>     https://www.python.org/dev/peps/pep-0285/
>
>     Or do you mean to ask why it wasn't changed in Python 3? Well, changed
>     to what? All the reasons given in the PEP still hold, and there's no
>     really compelling reason to make bool something else. Since bools aren't
>     broken, why change them?
>
>
> I've read through the PEP and I understand the rationale about why
> True/False is a subclass of int

'subclass of int' means 'instances are ints'

> What I find odd though is that a bool can be used in arithmetical
> operations as if it was the exact same thing as a number.

That is what subclass means.

 > I mean, to me this is just odd:
>
>>>> 1 + True
> 2
>
> I would have expected that to be treated the same as:
>
>>>> 1 + "1"
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unsupported operand type(s) for +: 'int' and 'str'

This would only be sensible if Bools were strings.

> On the other hand (and I'm gonna contradict myself with what I've just
> said above) on chapter 4:
> /
> /
> /       There's a small but vocal minority that would prefer to see/
> /       "textbook" bools that don't support arithmetic operations at/
> /       all, but most reviewers agree with me that bools should always/
> /       allow arithmetic operations./
>
> ...so maybe supporting arithmetical operations was also a primary
> intention,

For some people, yes.

> in which case my question is "why?".

Just a few days ago, I read someone, while discussing an fairly large 
application, saying how nice python is because it allows simple code like

score = sum(s==c for s, c in zip(student, correct))

(This is not the code from the application, but close enough.)  There is 
real code in the wild exploiting issubclass(bool, int), which could be 
broken if that were changed.

-- 
Terry Jan Reedy


From srkunze at mail.de  Mon Jun  6 03:05:40 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Mon, 6 Jun 2016 09:05:40 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
Message-ID: <575520C4.2010900@mail.de>

On 02.06.2016 16:18, Joao S. O. Bueno wrote:
> Just remembering that one is free to implement whatever "switchable"
> containers one wants in the language as it is now - no need to force
> (full or slightly) incompatible  changes that imply on performance
> degradation onto every single Python user for a feature that was not
> needed up to this day.
>
> Implementing this in  sequence-abc, and mapping-abc classes is trivial.
>
> A reasonable request for the language could be exactly to allow a
> "__mutable__" property on the sequence, container and mapping
> protocols, and a "mutable" built-in callable, that in the absence of
> "__mutable__" could check for the methods signature (if it does not
> have "__mutable__" but has "__setitem__" , then mutable(obj) returns
> True, for example).

Is __setitem__ the only way to change an object?

> On 2 June 2016 at 10:55, Sven R. Kunze <srkunze at mail.de> wrote:
>> On 02.06.2016 15:05, Rob Cliffe wrote:
>>> Sure, there are many difficulties.
>>> This was intended to be a blue-sky idea.  I would like to pose the
>>> question "If I had to redesign Python from scratch, would I think this is a
>>> good idea?"
>>
>> I don't think that's an overly stupid idea. In such regard, Python wouldn't
>> be the first project dealing with mutability in this manner. PostgreSQL
>> already does.
>>
>> One question I would have: would it be enforced? Or can objects designer
>> choose between enforced immutability and informative immutability? Or would
>> it be enforced inconsistently at some places where at other places it does
>> not matter.
>> To me the 100% enforced variant would be the least troublesome.
>>
>> One definite advantage of broader use of immutable objects would be
>> optimization: using the same memory parts, re-using the same object etc.
>> It's not my field in computer science but other people will be able to
>> imagine a lot of other possibilities here.
>>
>> Best,
>> Sven
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/


From robertc at robertcollins.net  Mon Jun  6 03:55:55 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 6 Jun 2016 19:55:55 +1200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <575520C4.2010900@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
Message-ID: <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>

On 6 June 2016 at 19:05, Sven R. Kunze <srkunze at mail.de> wrote:
> On 02.06.2016 16:18, Joao S. O. Bueno wrote:
>>
>> Just remembering that one is free to implement whatever "switchable"
>> containers one wants in the language as it is now - no need to force
>> (full or slightly) incompatible  changes that imply on performance
>> degradation onto every single Python user for a feature that was not
>> needed up to this day.
>>
>> Implementing this in  sequence-abc, and mapping-abc classes is trivial.
>>
>> A reasonable request for the language could be exactly to allow a
>> "__mutable__" property on the sequence, container and mapping
>> protocols, and a "mutable" built-in callable, that in the absence of
>> "__mutable__" could check for the methods signature (if it does not
>> have "__mutable__" but has "__setitem__" , then mutable(obj) returns
>> True, for example).
>
>
> Is __setitem__ the only way to change an object?

No. Trivially a method on an extension object can be storing state in
a C struct. Let trivially:

Class IsThisMutable:
    def __init__(self):
        self._cache = {}

    def random(self):
        self._cache[len(self._cache)] = random.random()
        return self._cache[len(self._cache)-1]

    def size(self):
        return len(self._cache)

this has interior mutability - it uses __setitem__ on self._cache to
track that state. You could remove __setitem__ from IsThisMutable
instances post construction and it would still be getting mutated.

-Rob

From srkunze at mail.de  Mon Jun  6 06:44:10 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Mon, 6 Jun 2016 12:44:10 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
Message-ID: <575553FA.9070807@mail.de>

On 06.06.2016 09:55, Robert Collins wrote:
> On 6 June 2016 at 19:05, Sven R. Kunze <srkunze at mail.de> wrote:
>> On 02.06.2016 16:18, Joao S. O. Bueno wrote:
>>> Just remembering that one is free to implement whatever "switchable"
>>> containers one wants in the language as it is now - no need to force
>>> (full or slightly) incompatible  changes that imply on performance
>>> degradation onto every single Python user for a feature that was not
>>> needed up to this day.
>>>
>>> Implementing this in  sequence-abc, and mapping-abc classes is trivial.
>>>
>>> A reasonable request for the language could be exactly to allow a
>>> "__mutable__" property on the sequence, container and mapping
>>> protocols, and a "mutable" built-in callable, that in the absence of
>>> "__mutable__" could check for the methods signature (if it does not
>>> have "__mutable__" but has "__setitem__" , then mutable(obj) returns
>>> True, for example).
>>
>> Is __setitem__ the only way to change an object?
> No.

That's what I thought as well. So, I see a huge benefit for enforced 
immutability when it comes to reducing errors and not re-inventing the 
wheel.


Thinking this further, __init__ would be the only function to change the 
state of an immutable object. Once created, it will never change. 
Immutable also implies hashability IMHO. Moreover, immutable object 
would not be allowed to query data from global/external variables as 
those can change and would change the observable state of the object 
without the object noticing. So, the allowed way of creating a state for 
an immutable object would be using a new container as you did (by 
defining self._cache) and store immutable objects only there. Would this 
make sense?

Sven

> Trivially a method on an extension object can be storing state in
> a C struct. Let trivially:
>
> Class IsThisMutable:
>      def __init__(self):
>          self._cache = {}
>
>      def random(self):
>          self._cache[len(self._cache)] = random.random()
>          return self._cache[len(self._cache)-1]
>
>      def size(self):
>          return len(self._cache)
>
> this has interior mutability - it uses __setitem__ on self._cache to
> track that state. You could remove __setitem__ from IsThisMutable
> instances post construction and it would still be getting mutated.
>
> -Rob


From random832 at fastmail.com  Mon Jun  6 09:34:24 2016
From: random832 at fastmail.com (Random832)
Date: Mon, 06 Jun 2016 09:34:24 -0400
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <575553FA.9070807@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
Message-ID: <1465220064.1064030.629241337.66238363@webmail.messagingengine.com>

On Mon, Jun 6, 2016, at 06:44, Sven R. Kunze wrote:
> Thinking this further, __init__ would be the only function to change the 
> state of an immutable object. Once created, it will never change. 
> Immutable also implies hashability IMHO.

You're continuing to fail to address the fact that some aspects of this
proposal implicitly claim that immutable objects shall not hold
references to mutable objects (a tuple containing a reference to a list
is not hashable*, does this mean tuples are not immutable?) while other
claims (OP's claim that an "immutable list" could substitute for a
tuple) imply that they can.

> Moreover, immutable object 
> would not be allowed to query data from global/external variables as 
> those can change and would change the observable state of the object 
> without the object noticing.

How are you gonna stop them?

*Java seems to work fine with mutable lists being hashable and trusting
users to responsibly manage the life cycles of dictionaries using them
as keys, but this is an argument that we don't really need to start up
again here and now.

From steve at pearwood.info  Mon Jun  6 10:40:34 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 7 Jun 2016 00:40:34 +1000
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <575553FA.9070807@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
Message-ID: <20160606144034.GB12028@ando.pearwood.info>

On Mon, Jun 06, 2016 at 12:44:10PM +0200, Sven R. Kunze wrote:

> That's what I thought as well. So, I see a huge benefit for enforced 
> immutability when it comes to reducing errors and not re-inventing the 
> wheel.

Python classes have supported "cooperative immutability" for 20+ years. 
For example, the pure Python Decimal and Fraction classes are immutable 
only by convention, if you modify the _private attributes you can change 
them. And yet they work fine.

I'm not saying that "real immutability" wouldn't be nice to have, but I 
think that describing it as huge benefit is possible overstating it.


> Thinking this further, __init__ would be the only function to change the 
> state of an immutable object. Once created, it will never change. 

How would you enforce that?


> Immutable also implies hashability IMHO.

That's incorrect. We all agree that tuples are immutable, I trust.

py> t = (1, 2, {}, 3)
py> hash(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'


Immutable collections are only hashable if all their components are 
hashable.


> Moreover, immutable object 
> would not be allowed to query data from global/external variables as 
> those can change and would change the observable state of the object 
> without the object noticing.

How would you enforce that?


> So, the allowed way of creating a state for 
> an immutable object would be using a new container as you did (by 
> defining self._cache) and store immutable objects only there. Would this 
> make sense?

I don't understand this paragraph, sorry.


-- 
Steve

From vgr255 at live.ca  Mon Jun  6 13:05:02 2016
From: vgr255 at live.ca (=?iso-8859-1?Q?=C9manuel_Barry?=)
Date: Mon, 6 Jun 2016 13:05:02 -0400
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
Message-ID: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>

I'm currently writing a class that can take either a str of bytes object,
and is meant to work with either type. For my purpose, it makes sense to
return the underlying bytes object when calling bytes() on it. However,
defining __bytes__ on the class means that I also have to return bytes even
if the underlying object is a str. The obvious way to fix this is to define
two different classes, one which handles str and one which handles bytes.
However, I was wondering if returning NotImplemented from __bytes__ would be
a viable alternative.

I'm not sure if there'd be any performance hit (there is probably some
optimized C code that simply checks for the existence of __bytes__ in the
namespace without calling it), but hopefully there aren't.

Another alternative would be to define object.__bytes__, which one can call
to fall back to the default behaviour.

While I'm at it, maybe other methods could use that, too (thinking mainly of
__int__, __float__, __abs__, __complex__, __round__, __index__, and maybe
__len__ and __iter__).

Thoughts? Is it reasonable to define a single class that can work with both
str and bytes, or should I fix my code?

-Emanuel

From guido at python.org  Mon Jun  6 14:18:45 2016
From: guido at python.org (Guido van Rossum)
Date: Mon, 6 Jun 2016 11:18:45 -0700
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
In-Reply-To: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
References: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
Message-ID: <CAP7+vJJW3yuG3T0ts5Ahssuqv87pGRsJ7UREbPoK56jHOwWRJg@mail.gmail.com>

Can't you just raise an exception when __bytes__() is called if the
underlying representation uses str?

Returning NotImplemented is not the right thing here -- that's not
supposed to make it into user code, it's only meant to be used when
there's a binary operation, to give the other argument a chance at
providing an implementation. (And if they both return NotImplemented,
Python will always turn that into a TypeError or a default answer.)
But for unary operators like bytes() there's no such thing, and the
NotImplemented would leak into user code, where it can do more damage
than good.

On Mon, Jun 6, 2016 at 10:05 AM, ?manuel Barry <vgr255 at live.ca> wrote:
> I'm currently writing a class that can take either a str of bytes object,
> and is meant to work with either type. For my purpose, it makes sense to
> return the underlying bytes object when calling bytes() on it. However,
> defining __bytes__ on the class means that I also have to return bytes even
> if the underlying object is a str. The obvious way to fix this is to define
> two different classes, one which handles str and one which handles bytes.
> However, I was wondering if returning NotImplemented from __bytes__ would be
> a viable alternative.
>
> I'm not sure if there'd be any performance hit (there is probably some
> optimized C code that simply checks for the existence of __bytes__ in the
> namespace without calling it), but hopefully there aren't.
>
> Another alternative would be to define object.__bytes__, which one can call
> to fall back to the default behaviour.
>
> While I'm at it, maybe other methods could use that, too (thinking mainly of
> __int__, __float__, __abs__, __complex__, __round__, __index__, and maybe
> __len__ and __iter__).
>
> Thoughts? Is it reasonable to define a single class that can work with both
> str and bytes, or should I fix my code?
>
> -Emanuel
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
--Guido van Rossum (python.org/~guido)

From vgr255 at live.ca  Mon Jun  6 14:36:53 2016
From: vgr255 at live.ca (=?utf-8?Q?=C3=89manuel_Barry?=)
Date: Mon, 6 Jun 2016 14:36:53 -0400
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
In-Reply-To: <CAP7+vJJW3yuG3T0ts5Ahssuqv87pGRsJ7UREbPoK56jHOwWRJg@mail.gmail.com>
References: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
 <CAP7+vJJW3yuG3T0ts5Ahssuqv87pGRsJ7UREbPoK56jHOwWRJg@mail.gmail.com>
Message-ID: <BLU403-EAS395BEAAFA8A8AC0C249EDC915C0@phx.gbl>

> From: Guido van Rossum
> Sent: Monday, June 06, 2016 2:19 PM
> To: ?manuel Barry
> Cc: Python-Ideas
> Subject: Re: [Python-ideas] Allow __bytes__ to return NotImplemented
> 
> Can't you just raise an exception when __bytes__() is called if the
> underlying representation uses str?

Sure (that's what I'm doing right now), but my intention was that returning NotImplemented would make bytes() fall back to the default behaviour. But for my use case, it makes no difference as an error is raised either way, just a different one.

> Returning NotImplemented is not the right thing here -- that's not
> supposed to make it into user code, it's only meant to be used when
> there's a binary operation, to give the other argument a chance at
> providing an implementation. (And if they both return NotImplemented,
> Python will always turn that into a TypeError or a default answer.)
> But for unary operators like bytes() there's no such thing, and the
> NotImplemented would leak into user code, where it can do more damage
> than good.

My understanding was that returning NotImplemented would tell the caller (here, bytes()) "I can't do that, figure it out yourself" like binary operations do, which would eventually become an error. It made sense to me at the time, but now I realize that it doesn't make sense if there's only one argument, so I'll raise the error myself.

Thanks for the reply,
-Emanuel

From guido at python.org  Mon Jun  6 14:40:58 2016
From: guido at python.org (Guido van Rossum)
Date: Mon, 6 Jun 2016 11:40:58 -0700
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
In-Reply-To: <BLU403-EAS395BEAAFA8A8AC0C249EDC915C0@phx.gbl>
References: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
 <CAP7+vJJW3yuG3T0ts5Ahssuqv87pGRsJ7UREbPoK56jHOwWRJg@mail.gmail.com>
 <BLU403-EAS395BEAAFA8A8AC0C249EDC915C0@phx.gbl>
Message-ID: <CAP7+vJJBjM8NwOA7-8=8g5g6tSy8-6oYcropb7QUs_zoKhNgbw@mail.gmail.com>

On Mon, Jun 6, 2016 at 11:36 AM, ?manuel Barry <vgr255 at live.ca> wrote:
>> From: Guido van Rossum
>> Can't you just raise an exception when __bytes__() is called if the
>> underlying representation uses str?
>
> Sure (that's what I'm doing right now), but my intention was that returning NotImplemented would make bytes() fall back to the default behaviour. But for my use case, it makes no difference as an error is raised either way, just a different one.

To invoke the default behavior just return super().__bytes().

>> Returning NotImplemented is not the right thing here -- that's not
>> supposed to make it into user code, it's only meant to be used when
>> there's a binary operation, to give the other argument a chance at
>> providing an implementation. (And if they both return NotImplemented,
>> Python will always turn that into a TypeError or a default answer.)
>> But for unary operators like bytes() there's no such thing, and the
>> NotImplemented would leak into user code, where it can do more damage
>> than good.
>
> My understanding was that returning NotImplemented would tell the caller (here, bytes()) "I can't do that, figure it out yourself" like binary operations do, which would eventually become an error. It made sense to me at the time, but now I realize that it doesn't make sense if there's only one argument, so I'll raise the error myself.

Right. NotImplemented is frequently misunderstood (and it doesn't help
that there's also a completely unrelated exception
NotImplementedError). If you want to contribute some docs about this
we'd be delighted! (bugs.python.org)

-- 
--Guido van Rossum (python.org/~guido)

From vgr255 at live.ca  Mon Jun  6 15:00:10 2016
From: vgr255 at live.ca (=?utf-8?Q?=C3=89manuel_Barry?=)
Date: Mon, 6 Jun 2016 15:00:10 -0400
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
In-Reply-To: <CAP7+vJJBjM8NwOA7-8=8g5g6tSy8-6oYcropb7QUs_zoKhNgbw@mail.gmail.com>
References: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
 <CAP7+vJJW3yuG3T0ts5Ahssuqv87pGRsJ7UREbPoK56jHOwWRJg@mail.gmail.com>
 <BLU403-EAS395BEAAFA8A8AC0C249EDC915C0@phx.gbl>
 <CAP7+vJJBjM8NwOA7-8=8g5g6tSy8-6oYcropb7QUs_zoKhNgbw@mail.gmail.com>
Message-ID: <BLU403-EAS1515B0867A750CF3098ECEF915C0@phx.gbl>

> From: Guido van Rossum
> Subject: Re: [Python-ideas] Allow __bytes__ to return NotImplemented
> 
> To invoke the default behavior just return super().__bytes().

Well, object.__bytes__() doesn't exist, so that would just be another unhelpful error.

> Right. NotImplemented is frequently misunderstood (and it doesn't help
> that there's also a completely unrelated exception
> NotImplementedError). If you want to contribute some docs about this
> we'd be delighted! (bugs.python.org)

Done! http://bugs.python.org/issue27242

-Emanuel

From nas-pythonideas at arctrix.com  Mon Jun  6 15:35:30 2016
From: nas-pythonideas at arctrix.com (Neil Schemenauer)
Date: Mon, 6 Jun 2016 12:35:30 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
Message-ID: <20160606193530.GA27695@python.ca>

On 2016-06-04, Guido van Rossum wrote:
> The bytes -> int behavior is widely considered a mistake. We're just
> not able to fix it without yet another round of layoffs ^W
> deprecations. And I'm not ready for that -- not even Python 4 should
> be allowed to change this unilaterally. Though maybe we could do
> something with a __future__ import.

Maybe the following would work:

- add a new method to 'bytes' that returns a view object with the
  current index/iteration behavior

- enable a deprecation warning for code that uses indexing/iteration
  on bytes

- when sufficient time has passed, revert to Python 2 behavior for
  indexing/iteration

Another, probably crazy and unworkable idea:

- have bytes indexing/iteration return a special type that behaves like
  an int or length one byte.

- ord() of this object would return a real int

- code that utilizes this object as an int would generate a warning
  (suggest adding an ord() call to fix code).

- eventually just return length one byte strings

From matt.ruffalo at gmail.com  Mon Jun  6 16:07:26 2016
From: matt.ruffalo at gmail.com (Matt Ruffalo)
Date: Mon, 6 Jun 2016 16:07:26 -0400
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
 <20160605172313.GA12028@ando.pearwood.info>
 <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
Message-ID: <5755D7FE.5020609@gmail.com>

On 2016-06-05 14:37, Giampaolo Rodola' wrote:
> I've read through the PEP and I understand the rationale about why
> True/False is a subclass of int and I'm OK with that. PEP-285 at
> chapter 6 suggests a strong practical reason for this:
>
> /       In an ideal world, bool might be better implemented as a/
> /       separate integer type that knows how to perform mixed-mode/
> /       arithmetic.  However, inheriting bool from int eases the/
> /       implementation enormously (in part since all C code that calls/
> /       PyInt_Check() will continue to work -- this returns true for/
> /       subclasses of int)/
>
> What I find odd though is that a bool can be used in arithmetical
> operations as if it was the exact same thing as a number. I mean, to
> me this is just odd:
>
> >>> 1 + True
> 2
>
> I would have expected that to be treated the same as:
>
> >>> 1 + "1"
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unsupported operand type(s) for +: 'int' and 'str'
>
> A bool may be a subclass of int but conceptually (to me at least) it's
> different almost the same way "1" is different than 1. I say "almost"
> only because bool is a subclass of int, but then again, to my
> understanding that was done for practical reasons (implementation
> details), not because the main intention was to explicitly allow
> mixing bools and numbers in arithmetical operations. 
>
> On the other hand (and I'm gonna contradict myself with what I've just
> said above) on chapter 4:
> /
> /
> /       There's a small but vocal minority that would prefer to see/
> /       "textbook" bools that don't support arithmetic operations at/
> /       all, but most reviewers agree with me that bools should always/
> /       allow arithmetic operations./
>
> ...so maybe supporting arithmetical operations was also a primary
> intention, in which case my question is "why?".
>
> -- 
> Giampaolo - http://grodola.blogspot.com
>
When using something like NumPy or Pandas or related packages, it is
quite common and useful to compute the sum of a boolean vector to check
how many elements pass some criterion:

"""
In [1]: import numpy as np

In [2]: A = np.random.randn(25)

In [3]: A
Out[3]:
array([-1.20821997,  0.8731083 ,  0.68458201,  1.86704545,  0.67679372,
       -0.63936162, -1.2518435 , -0.55477108,  0.00940205,  1.61347396,
        1.51399244,  0.57676897,  0.86984802,  0.96965798, -0.0726013 ,
       -0.35246648,  0.12149487, -0.42062617, -0.22227402, -0.3525525 ,
       -1.04447944, -0.39717087, -0.23223961,  0.81008826,  0.34763992])

In [4]: A > 0
Out[4]:
array([False,  True,  True,  True,  True, False, False, False,  True,
        True,  True,  True,  True,  True, False, False,  True, False,
       False, False, False, False, False,  True,  True], dtype=bool)

In [5]: (A > 0).sum()
Out[5]: 13
"""

It's true that none of this behavior strictly depends on the behavior of
the Python bool object -- the behavior of NumPy arrays already diverges
quite a bit from Python lists and built-in data types, so this behavior
would probably still be part of NumPy even if arithmetic operations on
Python bools raised a TypeError. This is still a good example of why it
can be very useful to have True + True + False == 2, though.

MMR...



From leewangzhong+python at gmail.com  Mon Jun  6 16:08:05 2016
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Mon, 6 Jun 2016 16:08:05 -0400
Subject: [Python-ideas] 1 + True = 2
In-Reply-To: <nj2j4v$mon$1@ger.gmane.org>
References: <CAFYqXL-LnANdC=mqFJ=p2UPGN1QJceTLbeJDqx9ep3uTR3iBrg@mail.gmail.com>
 <20160605172313.GA12028@ando.pearwood.info>
 <CAFYqXL-Ly87A1Q3yyHrzR4=n9+DJFo7ew9ngYpKBrLUhaNk0Wg@mail.gmail.com>
 <nj2j4v$mon$1@ger.gmane.org>
Message-ID: <CAB_e7iw3WEq5uRyxZQAT5p_FoMYyhrc2VdsnhTs4sQVCVENZfg@mail.gmail.com>

On Jun 5, 2016 9:20 PM, "Terry Reedy" <tjreedy at udel.edu> wrote:
>
> On 6/5/2016 2:37 PM, Giampaolo Rodola' wrote:
>>
>>
>> On Sun, Jun 5, 2016 at 7:23 PM, Steven D'Aprano
>> <steve at pearwood.info
>> <mailto:steve at pearwood.info>> wrote:
>>
>>     On Sun, Jun 05, 2016 at 06:37:48PM +0200, Giampaolo Rodola' wrote:
>>     > >>> 1 + True
>>     > 2
>>     > >>> 1 + False
>>     > 1
>>     > >>>
>>     >
>>     > I bumped into this today by accident and I can't recall ever being
aware of
>>     > this. Why isn't this a TypeError in Python 3?
>>
>>     Because bool is a subclass of int, and has been since it was first
>>     introduced back in Python 2.2. I'm not quite sure what your question
is
>>     about... are you asking what was the original justification for
making
>>     True and False equal to 1 and 0? If so, see the PEP:
>>
>>     https://www.python.org/dev/peps/pep-0285/
>>
>>     Or do you mean to ask why it wasn't changed in Python 3? Well,
changed
>>     to what? All the reasons given in the PEP still hold, and there's no
>>     really compelling reason to make bool something else. Since bools
aren't
>>     broken, why change them?
>>
>>
>> I've read through the PEP and I understand the rationale about why
>> True/False is a subclass of int
>
>
> 'subclass of int' means 'instances are ints'
>
>
>> What I find odd though is that a bool can be used in arithmetical
>> operations as if it was the exact same thing as a number.
>
>
> That is what subclass means.
>
>
> > I mean, to me this is just odd:
>>
>>
>>>>> 1 + True
>>
>> 2
>>
>> I would have expected that to be treated the same as:
>>
>>>>> 1 + "1"
>>
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: unsupported operand type(s) for +: 'int' and 'str'
>
>
> This would only be sensible if Bools were strings.
>
>> On the other hand (and I'm gonna contradict myself with what I've just
>> said above) on chapter 4:
>> /
>> /
>> /       There's a small but vocal minority that would prefer to see/
>> /       "textbook" bools that don't support arithmetic operations at/
>> /       all, but most reviewers agree with me that bools should always/
>> /       allow arithmetic operations./
>>
>>
>> ...so maybe supporting arithmetical operations was also a primary
>> intention,
>
>
> For some people, yes.
>
>
>> in which case my question is "why?".
>
>
> Just a few days ago, I read someone, while discussing an fairly large
application, saying how nice python is because it allows simple code like
>
> score = sum(s==c for s, c in zip(student, correct))
>
> (This is not the code from the application, but close enough.)  There is
real code in the wild exploiting issubclass(bool, int), which could be
broken if that were changed.

Some of that reply is too pedantic, like the part about `1+"1"`. Discuss
what people are _trying_ to say, at least in addition to what they actually
say.

Anyway, I'd argue that, from a purity standpoint, summing an iterator of
bools is hacky, and what you really want is this:

        # Count student answers that match the respective correct answers.
        count(s for s, c in zip(student, correct) if s == c)

(Alas, the library for iterator operations, `itertools`, already has a very
different function named `count`.)

But I'm not pushing for non-int bools. Purity is not a primary Python
principle. Python has `if lst:` for emptiness, and I doubt that will ever
go away. (Though it makes Numpy arrays needlessly incompatible with some
libraries.)

Int-like bools can also be used for:
- Hacky `if-else`:
        # Works for numbers, strs, lists, tuples, bytess, and bytearrays!
        (x == y)*yes + (x != y)*no
- Hacky `if-else`:
        yes**(x == y) * no**(x != y)
- Hacky `if-else`:
        [no, yes][x==y]
- Hacky `if-else`:
        (x == y and [yes] or [no])[0]
- Hacky `if/if-not` (control flow):
        (x == y) > 0 > (print("yes") or 1)
        (x == y) < 1 < (print("no") or 0)
- Generalization of XOR: "Exactly N out of M conditions hold."

* Fourth example from:
    https://mail.python.org/pipermail/python-dev/2002-March/020835.html

Earlier discussions:

- 2002 March: PEP 285 review.
    https://mail.python.org/pipermail/python-dev/2002-March/020750.html

- 2002 March: Against int subclassing in PEP 285.
    https://mail.python.org/pipermail/python-list/2002-March/129054.html

- 2015 January: Against int subclassing.
    https://mail.python.org/pipermail/python-ideas/2015-January/031233.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/8e368a3e/attachment.html>

From sjoerdjob at sjoerdjob.com  Mon Jun  6 16:09:01 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Mon, 6 Jun 2016 22:09:01 +0200
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <20160606193530.GA27695@python.ca>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
Message-ID: <3240D18F-18D3-4E1C-B5CE-36FB0A220E33@sjoerdjob.com>

I like this idea!

    >>> k = b'hello world'
    >>> v = k[0]
    >>> type(v)
    <class 'byte'>
    >>> ord(v)
    104
    >>> v + 5
    __main__:1: DeprecationWarning: bytes items will become byte instances. Use ord().
    109

On the other hand, maybe 'hello'[0] should be a ch(a)r instead of str?

The 'wrapping' class could be a `bytearray`?

    >>> k = bytearray(b'hello')
    >>> k[0]
    104
    >>> k[2:4]
    bytearray(b'll')

The only 'downside' is that a bytearray is mutable.

> On 6 Jun 2016, at 21:35, Neil Schemenauer <nas-pythonideas at arctrix.com> wrote:
> 
>> On 2016-06-04, Guido van Rossum wrote:
>> The bytes -> int behavior is widely considered a mistake. We're just
>> not able to fix it without yet another round of layoffs ^W
>> deprecations. And I'm not ready for that -- not even Python 4 should
>> be allowed to change this unilaterally. Though maybe we could do
>> something with a __future__ import.
> 
> Maybe the following would work:
> 
> - add a new method to 'bytes' that returns a view object with the
>  current index/iteration behavior
> 
> - enable a deprecation warning for code that uses indexing/iteration
>  on bytes
> 
> - when sufficient time has passed, revert to Python 2 behavior for
>  indexing/iteration
> 
> Another, probably crazy and unworkable idea:
> 
> - have bytes indexing/iteration return a special type that behaves like
>  an int or length one byte.
> 
> - ord() of this object would return a real int
> 
> - code that utilizes this object as an int would generate a warning
>  (suggest adding an ord() call to fix code).
> 
> - eventually just return length one byte strings
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From srkunze at mail.de  Mon Jun  6 16:23:55 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Mon, 6 Jun 2016 22:23:55 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <20160606144034.GB12028@ando.pearwood.info>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de> <20160606144034.GB12028@ando.pearwood.info>
Message-ID: <5755DBDB.9050104@mail.de>

On 06.06.2016 16:40, Steven D'Aprano wrote:
> Python classes have supported "cooperative immutability" for 20+ years.
> For example, the pure Python Decimal and Fraction classes are immutable
> only by convention, if you modify the _private attributes you can change
> them. And yet they work fine.

Nobody says it does not work fine. But sometimes, people need an 
immutability class blueprint. I saw several implementation attempts for 
such immutable-appearing objects. They tend to be hard to maintain.

>
> I'm not saying that "real immutability" wouldn't be nice to have, but I
> think that describing it as huge benefit is possible overstating it.

Since Python became a Turing-complete language, most PEPs are just that: 
preventing the re-invention of the wheel. And that's always a huge 
benefit; especially viewed economically.

>> Thinking this further, __init__ would be the only function to change the
>> state of an immutable object. Once created, it will never change.
> How would you enforce that?

That's an implementation detail.

>
>
>> Immutable also implies hashability IMHO.
> That's incorrect. We all agree that tuples are immutable, I trust.
>
> py> t = (1, 2, {}, 3)
> py> hash(t)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: unhashable type: 'dict'
>
>
> Immutable collections are only hashable if all their components are
> hashable.

So what? I don't see your point in quoting current behavior. Especially, 
because this is a theoretical discussion about a hypothetical behavior 
which might differ from the current one.

Furthermore, it depends on how you define what a tuple is. Is it the 
mere container to n (possibly mutable) objects? Or does it also comprise 
its components?

In the former case, it's the current behavior IIRC. In the later case, 
its components need to be immutable as well. That's basically what I was 
trying to say below.

>
>> Moreover, immutable object
>> would not be allowed to query data from global/external variables as
>> those can change and would change the observable state of the object
>> without the object noticing.
> How would you enforce that?

Don't give access to them in the first place. That again seems to me 
like another implementation detail.

>> So, the allowed way of creating a state for
>> an immutable object would be using a new container as you did (by
>> defining self._cache) and store immutable objects only there. Would this
>> make sense?
> I don't understand this paragraph, sorry.

(this might be because you split it up into several pieces? ;) )

I might refer here to your tuple example above. The tuple itself might 
look immutable but its components are not.

What immutability means instead for a domain-specific object is that it 
needs to look and to feel immutable. Usually, ones does not inspect such 
objects but use it via its public API. So, the output of its API 
definitely needs to be constant through out the life-time of the object.

Internally, the object could need to store more than a single number to 
represent its state. Maybe, even a container of containers could be 
necessary. But then all those containers and the containers' contents 
need to be immutable as well to assist the designer of the object's class.


That's at least the way I understand it how immutability would need to 
work in order to prevent errors and simplify developers' lives.


Sven

From guido at python.org  Mon Jun  6 16:28:12 2016
From: guido at python.org (Guido van Rossum)
Date: Mon, 6 Jun 2016 13:28:12 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <20160606193530.GA27695@python.ca>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
Message-ID: <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>

I think the approach using a new method for the old behavior is workable.
Maybe we should add another (temporary) new method for the new behavior.

The magic object approach is too magical.

We should do the same for bytearray and memoryview.

On Mon, Jun 6, 2016 at 12:35 PM, Neil Schemenauer <
nas-pythonideas at arctrix.com> wrote:

> On 2016-06-04, Guido van Rossum wrote:
> > The bytes -> int behavior is widely considered a mistake. We're just
> > not able to fix it without yet another round of layoffs ^W
> > deprecations. And I'm not ready for that -- not even Python 4 should
> > be allowed to change this unilaterally. Though maybe we could do
> > something with a __future__ import.
>
> Maybe the following would work:
>
> - add a new method to 'bytes' that returns a view object with the
>   current index/iteration behavior
>
> - enable a deprecation warning for code that uses indexing/iteration
>   on bytes
>
> - when sufficient time has passed, revert to Python 2 behavior for
>   indexing/iteration
>
> Another, probably crazy and unworkable idea:
>
> - have bytes indexing/iteration return a special type that behaves like
>   an int or length one byte.
>
> - ord() of this object would return a real int
>
> - code that utilizes this object as an int would generate a warning
>   (suggest adding an ord() call to fix code).
>
> - eventually just return length one byte strings
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/92cbdd78/attachment.html>

From srkunze at mail.de  Mon Jun  6 16:32:24 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Mon, 6 Jun 2016 22:32:24 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <1465220064.1064030.629241337.66238363@webmail.messagingengine.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
 <1465220064.1064030.629241337.66238363@webmail.messagingengine.com>
Message-ID: <5755DDD8.3040204@mail.de>

On 06.06.2016 15:34, Random832 wrote:
> (a tuple containing a reference to a list
> is not hashable*, does this mean tuples are not immutable?)

Your question in brackets is actually the most important question here.

I would say it depends on who one defines it but I don't see a 
contradiction to have both variants of tuples in the same languages. 
Whatever fits your needs more in a certain situation.

>> Moreover, immutable object
>> would not be allowed to query data from global/external variables as
>> those can change and would change the observable state of the object
>> without the object noticing.
> How are you gonna stop them?

With a hammer? I don't know how you want me to answer that. It's an 
implementation detail to me as other languages can handle it properly. 
There is a way of stopping them by preventing all ways of manipulating 
state variables of an object. What those ways are is more a topic for 
long-served devs of CPython 4 or 5 ;-)


Sven

From michael.selik at gmail.com  Mon Jun  6 18:13:13 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Mon, 06 Jun 2016 22:13:13 +0000
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
In-Reply-To: <BLU403-EAS1515B0867A750CF3098ECEF915C0@phx.gbl>
References: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
 <CAP7+vJJW3yuG3T0ts5Ahssuqv87pGRsJ7UREbPoK56jHOwWRJg@mail.gmail.com>
 <BLU403-EAS395BEAAFA8A8AC0C249EDC915C0@phx.gbl>
 <CAP7+vJJBjM8NwOA7-8=8g5g6tSy8-6oYcropb7QUs_zoKhNgbw@mail.gmail.com>
 <BLU403-EAS1515B0867A750CF3098ECEF915C0@phx.gbl>
Message-ID: <CAGgTfkM2dBqcrm-JhXaqJxUThZ8bBxoM_g-LBh_5YPe+8+dsmA@mail.gmail.com>

On Mon, Jun 6, 2016 at 3:01 PM ?manuel Barry <vgr255 at live.ca> wrote:

> > From: Guido van Rossum
>
> > Right. NotImplemented is frequently misunderstood (and it doesn't help
> > that there's also a completely unrelated exception
> > NotImplementedError). If you want to contribute some docs about this
> > we'd be delighted! (bugs.python.org)
>
> Done! http://bugs.python.org/issue27242



"The appropriate way to signal that an operation is unsupported is to leave
the relevant method undefined."

+1
Thanks for writing that. It's unfortunate that some folks see things like
__hash__ returning None and they assume that's the appropriate pattern for
signaling an operation is not supported.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/70d4089f/attachment-0001.html>

From barry at python.org  Mon Jun  6 18:23:13 2016
From: barry at python.org (Barry Warsaw)
Date: Mon, 6 Jun 2016 18:23:13 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
Message-ID: <20160606182313.601d69db@anarchist.wooz.org>

On May 31, 2016, at 01:08 PM, Steven D'Aprano wrote:

>This comes up a lot and it would be nice to clean it up.
>
>T = type('T', bases, ns)
>Record = namedtuple('Record', fields)
>
>It came up in the discussion on dict unpacking:
>
>a, b, c = **dict  # unpacks keys 'a', 'b', 'c'
>
>which was rejected as too magical. But maybe it will seem less magical if we
>have a special assignment operator that takes the left hand symbol(s) and
>copies them to the right?

It also comes up in @public for constants, which I currently support as:

public(a=SomeA())

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/366b2be8/attachment.sig>

From barry at python.org  Mon Jun  6 18:32:21 2016
From: barry at python.org (Barry Warsaw)
Date: Mon, 6 Jun 2016 18:32:21 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
Message-ID: <20160606183221.1c916e52@anarchist.wooz.org>

On Jun 01, 2016, at 03:53 PM, M.-A. Lemburg wrote:

>This could be done via a decorator:
>
>    @recordbinding
>    x = obj
>
>to result in the compiler generating the following code:
>
>    x = obj
>    obj.recordbinding('x', 2)

I like the idea of using a decorator because it's familiar syntax and we all
already (think we) know what it means.  However, in this case, wouldn't

    @recordbinding
    x = obj

translate to

    x = obj
    recordbinding(x)

?

The way you've written it, obj must be a type that implements the
recordbinding method, but that's not what I'd expect.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/b9dc99c7/attachment.sig>

From mal at egenix.com  Mon Jun  6 18:53:23 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 7 Jun 2016 00:53:23 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160606183221.1c916e52@anarchist.wooz.org>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
Message-ID: <5755FEE3.9020702@egenix.com>

On 07.06.2016 00:32, Barry Warsaw wrote:
> On Jun 01, 2016, at 03:53 PM, M.-A. Lemburg wrote:
> 
>> This could be done via a decorator:
>>
>>    @recordbinding
>>    x = obj
>>
>> to result in the compiler generating the following code:
>>
>>    x = obj
>>    obj.recordbinding('x', 2)
> 
> I like the idea of using a decorator because it's familiar syntax and we all
> already (think we) know what it means.  However, in this case, wouldn't
> 
>     @recordbinding
>     x = obj
> 
> translate to
> 
>     x = obj
>     recordbinding(x)
> 
> ?
> 
> The way you've written it, obj must be a type that implements the
> recordbinding method, but that's not what I'd expect.

This would work as well and indeed reads better, but you'd need
to have the compiler generate:

      x = obj
      recordbinding(obj, 'x', 2)

ie. pass in the object, the bound name and the line number
and recordbinding would then have to decide what to do with the
parameters.

I used the method variant, because a very common use case
is to let the object know about the name under which it is
now known to Python. This can be used to eg. define records,
forms, mappings, etc.

I just wonder how we could tell the compiler to special
case this decorator in a clean way.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 07 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From barry at python.org  Mon Jun  6 19:14:49 2016
From: barry at python.org (Barry Warsaw)
Date: Mon, 6 Jun 2016 19:14:49 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <5755FEE3.9020702@egenix.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
 <5755FEE3.9020702@egenix.com>
Message-ID: <20160606191449.12dd6f67@anarchist.wooz.org>

On Jun 07, 2016, at 12:53 AM, M.-A. Lemburg wrote:

>This would work as well and indeed reads better, but you'd need
>to have the compiler generate:
>
>      x = obj
>      recordbinding(obj, 'x', 2)
>
>ie. pass in the object, the bound name and the line number
>and recordbinding would then have to decide what to do with the
>parameters.

+1 although I'd bikeshed on the order of the arguments.

>I used the method variant, because a very common use case
>is to let the object know about the name under which it is
>now known to Python. This can be used to eg. define records,
>forms, mappings, etc.

Yep.  So given the above, `recordbinding(obj, 'x', 2)` would of course be free
to delegate to `obj.recordbinding('x', 2)`, but it would be up to the
decorator (and its author), not the compiler to do that delegation.

>I just wonder how we could tell the compiler to special
>case this decorator in a clean way.

That's the rub, but I do think you're on to a good general solution to a
common set of problems.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/ea105785/attachment.sig>

From guido at python.org  Mon Jun  6 19:20:07 2016
From: guido at python.org (Guido van Rossum)
Date: Mon, 6 Jun 2016 16:20:07 -0700
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160606191449.12dd6f67@anarchist.wooz.org>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
 <5755FEE3.9020702@egenix.com> <20160606191449.12dd6f67@anarchist.wooz.org>
Message-ID: <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>

On Mon, Jun 6, 2016 at 4:14 PM, Barry Warsaw <barry at python.org> wrote:

> On Jun 07, 2016, at 12:53 AM, M.-A. Lemburg wrote:
>
> >This would work as well and indeed reads better, but you'd need
> >to have the compiler generate:
> >
> >      x = obj
> >      recordbinding(obj, 'x', 2)
> >
> >ie. pass in the object, the bound name and the line number
> >and recordbinding would then have to decide what to do with the
> >parameters.
>
> +1 although I'd bikeshed on the order of the arguments.
>
> >I used the method variant, because a very common use case
> >is to let the object know about the name under which it is
> >now known to Python. This can be used to eg. define records,
> >forms, mappings, etc.
>
> Yep.  So given the above, `recordbinding(obj, 'x', 2)` would of course be
> free
> to delegate to `obj.recordbinding('x', 2)`, but it would be up to the
> decorator (and its author), not the compiler to do that delegation.
>
> >I just wonder how we could tell the compiler to special
> >case this decorator in a clean way.
>
> That's the rub, but I do think you're on to a good general solution to a
> common set of problems.
>

I think we're on to something here. Maybe. Perhaps.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/5d479238/attachment-0001.html>

From steve at pearwood.info  Mon Jun  6 23:57:40 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 7 Jun 2016 13:57:40 +1000
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <20160606193530.GA27695@python.ca>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
Message-ID: <20160607035740.GD12028@ando.pearwood.info>

On Mon, Jun 06, 2016 at 12:35:30PM -0700, Neil Schemenauer wrote:

> Maybe the following would work:
> 
> - add a new method to 'bytes' that returns a view object with the
>   current index/iteration behavior
>
> - enable a deprecation warning for code that uses indexing/iteration
>   on bytes
> 
> - when sufficient time has passed, revert to Python 2 behavior for
>   indexing/iteration

Sounds reasonable. That could even be a __future__ import, so that those 
without a big investment in the current behaviour could get the new 
behaviour straight away.


-- 
Steve

From storchaka at gmail.com  Tue Jun  7 00:41:59 2016
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 7 Jun 2016 07:41:59 +0300
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
 <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
Message-ID: <nj5jan$80l$1@ger.gmane.org>

On 06.06.16 23:28, Guido van Rossum wrote:
> I think the approach using a new method for the old behavior is
> workable. Maybe we should add another (temporary) new method for the new
> behavior.
>
> The magic object approach is too magical.
>
> We should do the same for bytearray and memoryview.

I often use bytearray and memoryview as arrays of ints.



From guido at python.org  Tue Jun  7 00:48:01 2016
From: guido at python.org (Guido van Rossum)
Date: Mon, 6 Jun 2016 21:48:01 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <nj5jan$80l$1@ger.gmane.org>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
 <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
 <nj5jan$80l$1@ger.gmane.org>
Message-ID: <CAP7+vJKxn5RoO069iYO0WrQN6M-OEnZAiM=NxBObsQ12H3jKSg@mail.gmail.com>

On Monday, June 6, 2016, Serhiy Storchaka <storchaka at gmail.com> wrote:

> On 06.06.16 23:28, Guido van Rossum wrote:
>
>> I think the approach using a new method for the old behavior is
>> workable. Maybe we should add another (temporary) new method for the new
>> behavior.
>>
>> The magic object approach is too magical.
>>
>> We should do the same for bytearray and memoryview.
>>
>
> I often use bytearray and memoryview as arrays of ints.


Really short ints. :-). But you should really use the array module instead.

Anyway we should do this for all three or for none.

--Guido


-- 
--Guido (mobile)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160606/f55d6543/attachment.html>

From storchaka at gmail.com  Tue Jun  7 00:51:26 2016
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 7 Jun 2016 07:51:26 +0300
Subject: [Python-ideas] Allow __bytes__ to return NotImplemented
In-Reply-To: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
References: <BLU403-EAS360EFA580035969EAF1A1C4915C0@phx.gbl>
Message-ID: <nj5jse$fnn$1@ger.gmane.org>

On 06.06.16 20:05, ?manuel Barry wrote:
> While I'm at it, maybe other methods could use that, too (thinking mainly of
> __int__, __float__, __abs__, __complex__, __round__, __index__, and maybe
> __len__ and __iter__).

This would break a lot of code that just checks the existence of 
corresponding methods. Even if change all this code to call 
corresponding methods and check the result, this would affect the 
performance.

> Thoughts? Is it reasonable to define a single class that can work with both
> str and bytes, or should I fix my code?

Define tree classes. A base class, its constructor returns an instance 
of appropriate child class, and two child classes for str and bytes.



From storchaka at gmail.com  Tue Jun  7 01:06:10 2016
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 7 Jun 2016 08:06:10 +0300
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <CAP7+vJKxn5RoO069iYO0WrQN6M-OEnZAiM=NxBObsQ12H3jKSg@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
 <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
 <nj5jan$80l$1@ger.gmane.org>
 <CAP7+vJKxn5RoO069iYO0WrQN6M-OEnZAiM=NxBObsQ12H3jKSg@mail.gmail.com>
Message-ID: <nj5ko2$r3a$1@ger.gmane.org>

On 07.06.16 07:48, Guido van Rossum wrote:
> On Monday, June 6, 2016, Serhiy Storchaka
> <storchaka at gmail.com
> <mailto:storchaka at gmail.com>> wrote:
>
>     On 06.06.16 23:28, Guido van Rossum wrote:
>
>         I think the approach using a new method for the old behavior is
>         workable. Maybe we should add another (temporary) new method for
>         the new
>         behavior.
>
>         The magic object approach is too magical.
>
>         We should do the same for bytearray and memoryview.
>
>
>     I often use bytearray and memoryview as arrays of ints.
>
>
> Really short ints. :-). But you should really use the array module instead.

Yes, ints of known width (even 1-bit ints in sre_compile.py). Using the 
array module requires additional copying. Aren't bytearray and 
memoryview here for avoiding unnecessary copying?

> Anyway we should do this for all three or for none.

I think representing bytes as an array of ints was good decision. If you 
need indexing to return a substring, you should use str instead. It is 
as well memory efficient thanks to PEP 393.



From jcgoble3 at gmail.com  Tue Jun  7 01:48:48 2016
From: jcgoble3 at gmail.com (Jonathan Goble)
Date: Tue, 7 Jun 2016 01:48:48 -0400
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <nj5ko2$r3a$1@ger.gmane.org>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
 <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
 <nj5jan$80l$1@ger.gmane.org>
 <CAP7+vJKxn5RoO069iYO0WrQN6M-OEnZAiM=NxBObsQ12H3jKSg@mail.gmail.com>
 <nj5ko2$r3a$1@ger.gmane.org>
Message-ID: <CAK256p3yKrR+dn-W9x686SF1+ZKAfU9nKMqQF_Mas4YZMsY1Ng@mail.gmail.com>

On Tue, Jun 7, 2016 at 1:06 AM, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 07.06.16 07:48, Guido van Rossum wrote:
>>
>> On Monday, June 6, 2016, Serhiy Storchaka
>> <storchaka at gmail.com
>> <mailto:storchaka at gmail.com>> wrote:
>>
>>     On 06.06.16 23:28, Guido van Rossum wrote:
>>
>>         I think the approach using a new method for the old behavior is
>>         workable. Maybe we should add another (temporary) new method for
>>         the new
>>         behavior.
>>
>>         The magic object approach is too magical.
>>
>>         We should do the same for bytearray and memoryview.
>>
>>
>>     I often use bytearray and memoryview as arrays of ints.
>>
>>
>> Really short ints. :-). But you should really use the array module
>> instead.
>
>
> Yes, ints of known width (even 1-bit ints in sre_compile.py). Using the
> array module requires additional copying. Aren't bytearray and memoryview
> here for avoiding unnecessary copying?

Actually, array.array implements the buffer protocol, and memoryview
can operate on any object that implements the buffer protocol. Thus,
you should be able to create a memoryview of an array.array instance
and operate on that (and in fact, you can, as I just tested it in an
interactive session).

From desmoulinmichel at gmail.com  Tue Jun  7 05:24:06 2016
From: desmoulinmichel at gmail.com (Michel Desmoulin)
Date: Tue, 7 Jun 2016 11:24:06 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160606183221.1c916e52@anarchist.wooz.org>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
Message-ID: <575692B6.40506@gmail.com>

So for namedtuple:

foo = namedtuple('foo', ['a', 'b'])

Would become:

@recordbinding
foo = namedtuple(['a', 'b'])

?

This doesn't look like an improvement.


Le 07/06/2016 00:32, Barry Warsaw a ?crit :
> On Jun 01, 2016, at 03:53 PM, M.-A. Lemburg wrote:
>
>> This could be done via a decorator:
>>
>>     @recordbinding
>>     x = obj
>>
>> to result in the compiler generating the following code:
>>
>>     x = obj
>>     obj.recordbinding('x', 2)
>
> I like the idea of using a decorator because it's familiar syntax and we all
> already (think we) know what it means.  However, in this case, wouldn't
>
>      @recordbinding
>      x = obj
>
> translate to
>
>      x = obj
>      recordbinding(x)
>
> ?
>
> The way you've written it, obj must be a type that implements the
> recordbinding method, but that's not what I'd expect.
>
> Cheers,
> -Barry
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>

From guettliml at thomas-guettler.de  Tue Jun  7 06:42:44 2016
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Tue, 7 Jun 2016 12:42:44 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160604082715.GB20628@python.ca>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
Message-ID: <5756A524.1010600@thomas-guettler.de>

Am 04.06.2016 um 10:27 schrieb Neil Schemenauer:
 > I can't understand why people don't see
 > we have a problem.

Imagine you are father of a three years old boy. Some stranger
tells you: "your kid is ugly!"

How does the average man react? Some get sad, some get angry.

Only very few would ask: Why do you think my kid is ugly?

I am not married with Python. Up to now I see no alternative for me,
but I guess sooner or later I will switch to a different
language.

I see only few benefits from porting my code to Python3. I will
use Python2 at least for the next 12 months.

Regards,
   Thomas G?ttler

-- 
Thomas Guettler http://www.thomas-guettler.de/

From antoine at python.org  Tue Jun  7 07:27:47 2016
From: antoine at python.org (Antoine Pitrou)
Date: Tue, 7 Jun 2016 11:27:47 +0000 (UTC)
Subject: [Python-ideas] bytes indexing behavior
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
Message-ID: <loom.20160607T132327-826@post.gmane.org>


Hi,

While I prefer Python 2's indexing behaviour, IMHO that ship has sailed.
Python 3.5 is probably seeing massive adoption now, and by introducing
another behaviour switch we only confuse users and make their lives
miserable.

Let's move on and stop obsessing about errors made in the past.  Every
language has some of those.

Regards

Antoine.



From steve at pearwood.info  Tue Jun  7 07:34:40 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 7 Jun 2016 21:34:40 +1000
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <5756A524.1010600@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
Message-ID: <20160607113440.GF12028@ando.pearwood.info>

On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas G?ttler wrote:

> I am not married with Python. Up to now I see no alternative for me,
> but I guess sooner or later I will switch to a different
> language.
> 
> I see only few benefits from porting my code to Python3. I will
> use Python2 at least for the next 12 months.

That's your right, of course. Some people love to experiment with new 
languages, some don't. But...

What will this other language be, and what does it offer that makes it 
better than Python 3?

When you move to Language X, will you re-write your code to X, or leave 
it in Python 2 forever?

If the answer is "re-write in X", do you think that will be easier than 
porting it to Python 3?

If the answer is "leave it in Python 2 forever", then why can't you do 
that for your existing code and write new code in Python 3?


-- 
Steve

From rob.cliffe at btinternet.com  Tue Jun  7 09:24:22 2016
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 7 Jun 2016 14:24:22 +0100
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <575553FA.9070807@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
Message-ID: <8455818d-ce86-5978-abae-255bcc28e7c2@btinternet.com>



On 06/06/2016 11:44, Sven R. Kunze wrote:
>
>
> Thinking this further, __init__ would be the only function to change 
> the state of an immutable object. Once created, it will never change. 
> Immutable also implies hashability IMHO. Moreover, immutable object 
> would not be allowed to query data from global/external variables as 
> those can change and would change the observable state of the object 
> without the object noticing. So, the allowed way of creating a state 
> for an immutable object would be using a new container as you did (by 
> defining self._cache) and store immutable objects only there. Would 
> this make sense?
>
> Sven
That would be one viable design.
But aren't there use cases for initially creating an object as mutable, 
then at some point "freezing" it by changing the mutable flag from True 
to False?
(Obviously the converse would not be allowed.)
Regards,
Rob Cliffe

From jsbueno at python.org.br  Tue Jun  7 10:39:12 2016
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Tue, 7 Jun 2016 11:39:12 -0300
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <loom.20160607T132327-826@post.gmane.org>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca> <loom.20160607T132327-826@post.gmane.org>
Message-ID: <CAH0mxTSA4yW+Wc6BctDd708W26KOrLkcp2ZrjRVC7sHD-deKAw@mail.gmail.com>

On 7 June 2016 at 08:27, Antoine Pitrou <antoine at python.org> wrote:
>
> Hi,
>
> While I prefer Python 2's indexing behaviour, IMHO that ship has sailed.
> Python 3.5 is probably seeing massive adoption now, and by introducing
> another behaviour switch we only confuse users and make their lives
> miserable.
>
> Let's move on and stop obsessing about errors made in the past.  Every
> language has some of those.
>

+1

Why not just stick in a new attribute over withch one can iterate as 1
byte "bytes sequence" instead?
Backwards compatible, and everyone gets happy:

for code, onebyte in zip(mybytes, mybytes.as_bytes):
     assert code == ord(onebyte)

I just can't view how any other thing could justify a major
breakearage in backward compatibility that would take
another 10-12 years to settle.  Maybe just add an "as_ints" iterator
as well, and deprecate interation on the
object itself - but reverse the behavior should just be a major no no
as in "python 4 scale" major no no!

And on the other hand, 'bytearray's have always behaved this way, even
in Python 2 - and it is
actually quite handy.

> Regards
>
> Antoine.
>

From guido at python.org  Tue Jun  7 11:26:08 2016
From: guido at python.org (Guido van Rossum)
Date: Tue, 7 Jun 2016 08:26:08 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <5756A524.1010600@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
Message-ID: <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>

On Tue, Jun 7, 2016 at 3:42 AM, Thomas G?ttler <guettliml at thomas-guettler.de
> wrote:

> Am 04.06.2016 um 10:27 schrieb Neil Schemenauer:
> > I can't understand why people don't see
> > we have a problem.
>
> Imagine you are father of a three years old boy. Some stranger
> tells you: "your kid is ugly!"
>

Just in case you didn't realize this can be turned around too, that's
exactly how it feels to *me* when people complain about Python 3.


> How does the average man react? Some get sad, some get angry.
>
> Only very few would ask: Why do you think my kid is ugly?
>
> I am not married with Python. Up to now I see no alternative for me,
> but I guess sooner or later I will switch to a different
> language.
>
> I see only few benefits from porting my code to Python3. I will
> use Python2 at least for the next 12 months.
>

That sounds like a threat (or another underhanded insult), but it's really
up to you. You have until 2020, and in fact after that date Python 2 won't
suddenly implode -- but none of the Python core devs who currently still
keep 2.7 alive will help you maintain it.

Of course it's open source so you are free to fork it any time. Just give
your fork a distinctive name so potential users aren't confused about what
they'll be getting and where they'd have to go for support.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160607/99101136/attachment-0001.html>

From ethan at stoneleaf.us  Tue Jun  7 11:59:22 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 07 Jun 2016 08:59:22 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <loom.20160607T132327-826@post.gmane.org>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca> <loom.20160607T132327-826@post.gmane.org>
Message-ID: <5756EF5A.106@stoneleaf.us>

On 06/07/2016 04:27 AM, Antoine Pitrou wrote:

> While I prefer Python 2's indexing behaviour, IMHO that ship has sailed.
> Python 3.5 is probably seeing massive adoption now, and by introducing
> another behaviour switch we only confuse users and make their lives
> miserable.

I agree on both counts.  Wasn't there a proposal/issue about adding some 
convenience methods to bytes to help ease this pain?  My quick search 
did not find it.

> Let's move on and stop obsessing about errors made in the past.  Every
> language has some of those.

Yup.

--
~Ethan~

From michael.selik at gmail.com  Tue Jun  7 13:32:06 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Tue, 07 Jun 2016 17:32:06 +0000
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>
Message-ID: <CAGgTfkModMgMnbWWSw6aqAxpFNvqNK6ySLS2Re2bjh67NaSQmw@mail.gmail.com>

On Tue, Jun 7, 2016 at 11:26 AM Guido van Rossum <guido at python.org> wrote:

> Of course it's open source so you are free to fork it any time. Just give
> your fork a distinctive name so potential users aren't confused about what
> they'll be getting and where they'd have to go for support
>

If only we were talking about Python 4 vs 5, then the obvious name would be
"Python 4evah"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160607/545bca9a/attachment.html>

From phd at phdru.name  Tue Jun  7 13:35:54 2016
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 7 Jun 2016 19:35:54 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CAGgTfkModMgMnbWWSw6aqAxpFNvqNK6ySLS2Re2bjh67NaSQmw@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <5756A524.1010600@thomas-guettler.de>
 <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>
 <CAGgTfkModMgMnbWWSw6aqAxpFNvqNK6ySLS2Re2bjh67NaSQmw@mail.gmail.com>
Message-ID: <20160607173554.GA11195@phdru.name>

On Tue, Jun 07, 2016 at 05:32:06PM +0000, Michael Selik <michael.selik at gmail.com> wrote:
> On Tue, Jun 7, 2016 at 11:26 AM Guido van Rossum <guido at python.org> wrote:
> 
> > Of course it's open source so you are free to fork it any time. Just give
> > your fork a distinctive name so potential users aren't confused about what
> > they'll be getting and where they'd have to go for support
> >
> 
> If only we were talking about Python 4 vs 5, then the obvious name would be
> "Python 4evah"

   Python 2therescue

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From guettliml at thomas-guettler.de  Tue Jun  7 14:27:52 2016
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Tue, 7 Jun 2016 20:27:52 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <20160607113440.GF12028@ando.pearwood.info>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
Message-ID: <57571228.3030806@thomas-guettler.de>

Am 07.06.2016 um 13:34 schrieb Steven D'Aprano:
> On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas G?ttler wrote:
> 
>> I am not married with Python. Up to now I see no alternative for me,
>> but I guess sooner or later I will switch to a different
>> language.
>>
>> I see only few benefits from porting my code to Python3. I will
>> use Python2 at least for the next 12 months.
> 
> That's your right, of course. Some people love to experiment with new
> languages, some don't. But...
> 
> What will this other language be, and what does it offer that makes it
> better than Python 3?

I would love to see "compile to javascript" support.

I would love to see a package management that is easy to use and which
focuses on simple data structures.
 

> When you move to Language X, will you re-write your code to X, or leave
> it in Python 2 forever?

I guess I will use language X for a new project.

> If the answer is "re-write in X", do you think that will be easier than
> porting it to Python 3?
> 
> If the answer is "leave it in Python 2 forever", then why can't you do
> that for your existing code and write new code in Python 3?

I could write new code in Python3, but I see big no benefit. (I
see benefit, but no big)

Does Python3 run in web browsers? Or is it possible to compile it to
javascript?

I guess I will port my python2 code to python3 sooner or later. If
Django drops support for Python2, then I will do it.

Regards,
  Thomas G?ttler

-- 
http://www.thomas-guettler.de/

From mal at egenix.com  Tue Jun  7 14:29:15 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 7 Jun 2016 20:29:15 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
 <20160606191449.12dd6f67@anarchist.wooz.org>
 <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>
Message-ID: <5757127B.4050300@egenix.com>

On 07.06.2016 01:20, Guido van Rossum wrote:
> On Mon, Jun 6, 2016 at 4:14 PM, Barry Warsaw <barry at python.org> wrote:
> 
>> On Jun 07, 2016, at 12:53 AM, M.-A. Lemburg wrote:
>>
>>> This would work as well and indeed reads better, but you'd need
>>> to have the compiler generate:
>>>
>>>      x = obj
>>>      recordbinding(obj, 'x', 2)
>>>
>>> ie. pass in the object, the bound name and the line number
>>> and recordbinding would then have to decide what to do with the
>>> parameters.
>>
>> +1 although I'd bikeshed on the order of the arguments.

:-)

>>> I used the method variant, because a very common use case
>>> is to let the object know about the name under which it is
>>> now known to Python. This can be used to eg. define records,
>>> forms, mappings, etc.
>>
>> Yep.  So given the above, `recordbinding(obj, 'x', 2)` would of course be
>> free
>> to delegate to `obj.recordbinding('x', 2)`, but it would be up to the
>> decorator (and its author), not the compiler to do that delegation.

Right, your approach is more generic.

>>> I just wonder how we could tell the compiler to special
>>> case this decorator in a clean way.
>>
>> That's the rub, but I do think you're on to a good general solution to a
>> common set of problems.
>>
> 
> I think we're on to something here. Maybe. Perhaps.

One approach would be to define what a decorator means in front
of an expression (since that's currently a SyntaxError), without
adding any new syntactic sugar. Here's a sketch...

@decorator
x = obj

compiles to:

x = obj
decorator('x', obj, lineno)

(I'm using obj in the calls to decorator here - those would not
be new lookups, but instead work using DUP_TOP in the byte
code)

Things get a bit tricky when you use tuples on the left hand
side:

@decorator
a, b, c = obj

compiles to:

a, b, c = obj
decorator('a, b, c', obj, lineno)

Not sure whether this would be useful. Perhaps it's better not
to allow for such a use.

Same problem for star assignments:

@decorator
a, *b, c = obj

Multiple assignments would work, though:

@decorator
a = b = c = obj

compiles to:

a = b = c = obj
decorator('a', obj, lineno)
decorator('b', obj, lineno)
decorator('c', obj, lineno)

Augmented assigns are tricky, but would work as well:

@decorator
a += obj

compiles to:

a += obj
decorator('a', value, lineno)

(here value stands for the result of the += operation; in
byte code this would be a DUP_TOP before the STORE_FAST to 'a')

More complex right hand sides should not pose a problem,
since it's clear that only the result matters (which is pushed
to the stack and can be DUP_TOPed as necessary).

Looking at https://docs.python.org/3/reference/grammar.html, I think
those are all cases to consider, unless I missed one.

The grammar would need to be extended with:

"""
decorated: decorators (classdef | funcdef | async_funcdef | assign_expr)

assign_expr: NAME (augassign (yield_expr|testlist) |
                   ('=' (yield_expr|assign_expr))*)
"""

The above disallows testlist or star assignments.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 07 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From phd at phdru.name  Tue Jun  7 14:37:15 2016
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 7 Jun 2016 20:37:15 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <57571228.3030806@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
Message-ID: <20160607183715.GA14939@phdru.name>

On Tue, Jun 07, 2016 at 08:27:52PM +0200, Thomas G??ttler <guettliml at thomas-guettler.de> wrote:
> I would love to see "compile to javascript" support.

   http://www.brython.info/
   http://www.transcrypt.org/
   http://pypyjs.org/
   https://github.com/azazel75/metapensiero.pj
   https://github.com/amirouche/pythonium

> I would love to see a package management that is easy to use and which
> focuses on simple data structures.

   virtualenv + pip

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From ian.g.kelly at gmail.com  Tue Jun  7 14:37:26 2016
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 7 Jun 2016 12:37:26 -0600
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <57571228.3030806@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
Message-ID: <CALwzid=UtLJZn9Phb2QaqBfU2Zo4y3iGUvQeMxxG6JZUJR+2+Q@mail.gmail.com>

On Tue, Jun 7, 2016 at 12:27 PM, Thomas G?ttler
<guettliml at thomas-guettler.de> wrote:
> Does Python3 run in web browsers? Or is it possible to compile it to
> javascript?

http://brython.info/
https://pypi.python.org/pypi/pythonium
http://pypyjs.org/
http://www.skulpt.org/

Brython explicitly claims Python 3 support. I don't know how close the
others are.

From stephen at xemacs.org  Tue Jun  7 15:01:08 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 8 Jun 2016 04:01:08 +0900
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <nj5ko2$r3a$1@ger.gmane.org>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
 <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
 <nj5jan$80l$1@ger.gmane.org>
 <CAP7+vJKxn5RoO069iYO0WrQN6M-OEnZAiM=NxBObsQ12H3jKSg@mail.gmail.com>
 <nj5ko2$r3a$1@ger.gmane.org>
Message-ID: <22359.6644.44125.765027@turnbull.sk.tsukuba.ac.jp>

Serhiy Storchaka writes:

 > I think representing bytes as an array of ints was good decision. If you 
 > need indexing to return a substring, you should use str instead. It is 
 > as well memory efficient thanks to PEP 393.

You can do this by using latin-1 as the codec, but that's pretty
unpleasant, because of the risk of combining with another str and
getting mojibake.

I have long thought that it would be interesting to have a codec and
an extension to PEP 393 that gives "asciibytes" behavior.  That is,
the codec simply slops the bytes into the 8-bit storage of a string,
but when joined with another string the result types are:

asciibytes        other arg        result
 has 8bit           type            type
   yes            pure ascii     asciibytes
   yes            asciibytes     asciibytes
   yes            other str      str with 8bit bytes from asciibytes
                                 encoded as PEP 383 surrogateescape
                                 (note: promotes latin1 to 2-byte-wide)
    no             whatever      whatever

I think Nick actually had a module that worked pretty much like this,
but he never pushed it.  I've never had time to reason out the
possible failure modes, though, or the performance issues.  And it's
not an itch I personally need to scratch.

I believe (but haven't proved) that the failure modes with the above
operation table are the same as for str containing PEP 383
surrogates.  I'm not sure what other issues you might run into.  Also,
I'm not sure it's reasonable to have an asciibytes with no 8bit bytes.

Steve


From ian at feete.org  Tue Jun  7 15:08:20 2016
From: ian at feete.org (Ian Foote)
Date: Tue, 7 Jun 2016 20:08:20 +0100
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <57571228.3030806@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
Message-ID: <57571BA4.5080607@feete.org>

On 07/06/16 19:27, Thomas G?ttler wrote:
> Am 07.06.2016 um 13:34 schrieb Steven D'Aprano:
>> On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas G?ttler wrote:
>>
>>> I am not married with Python. Up to now I see no alternative for me,
>>> but I guess sooner or later I will switch to a different
>>> language.
>>>
>>> I see only few benefits from porting my code to Python3. I will
>>> use Python2 at least for the next 12 months.
>>
>> That's your right, of course. Some people love to experiment with new
>> languages, some don't. But...
>>
>> What will this other language be, and what does it offer that makes it
>> better than Python 3?
> 
> I would love to see "compile to javascript" support.
> 
> I would love to see a package management that is easy to use and which
> focuses on simple data structures.
>  
> 
>> When you move to Language X, will you re-write your code to X, or leave
>> it in Python 2 forever?
> 
> I guess I will use language X for a new project.
> 
>> If the answer is "re-write in X", do you think that will be easier than
>> porting it to Python 3?
>>
>> If the answer is "leave it in Python 2 forever", then why can't you do
>> that for your existing code and write new code in Python 3?
> 
> I could write new code in Python3, but I see big no benefit. (I
> see benefit, but no big)
> 
> Does Python3 run in web browsers? Or is it possible to compile it to
> javascript?
> 
> I guess I will port my python2 code to python3 sooner or later. If
> Django drops support for Python2, then I will do it.
> 
> Regards,
>   Thomas G?ttler
> 

Batavia (https://github.com/pybee/batavia) is worth keeping an eye on
(for browser support), as is the rest of the BeeWare suite
(http://pybee.org/).

Regards,
Ian F

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160607/84d2e972/attachment.sig>

From ncoghlan at gmail.com  Tue Jun  7 15:57:55 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2016 12:57:55 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <5756EF5A.106@stoneleaf.us>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca> <loom.20160607T132327-826@post.gmane.org>
 <5756EF5A.106@stoneleaf.us>
Message-ID: <CADiSq7cP2f+gsZfOqqejx+ONvFYEhjzbku1gHoAg8Bk4kmL-tg@mail.gmail.com>

On 7 June 2016 at 08:59, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 06/07/2016 04:27 AM, Antoine Pitrou wrote:
>
>> While I prefer Python 2's indexing behaviour, IMHO that ship has sailed.
>> Python 3.5 is probably seeing massive adoption now, and by introducing
>> another behaviour switch we only confuse users and make their lives
>> miserable.
>
>
> I agree on both counts.  Wasn't there a proposal/issue about adding some
> convenience methods to bytes to help ease this pain?  My quick search did
> not find it.

It was in PEP 467:
https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-methods-that-produce-bytes-objects

If someone wanted to take that PEP and drive it through to resolution,
I'd be happy to hand it over (my recollection is that the sticking
point in the previous discussion was the proposed constructor changes,
so dropping those may make it easier to get the additional method
accepted).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Tue Jun  7 16:07:14 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2016 13:07:14 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <22359.6644.44125.765027@turnbull.sk.tsukuba.ac.jp>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca>
 <CAP7+vJ+LPc5bc7F30UwDx4LmBDomr3ofDed0oxe9UpimQNg2AQ@mail.gmail.com>
 <nj5jan$80l$1@ger.gmane.org>
 <CAP7+vJKxn5RoO069iYO0WrQN6M-OEnZAiM=NxBObsQ12H3jKSg@mail.gmail.com>
 <nj5ko2$r3a$1@ger.gmane.org>
 <22359.6644.44125.765027@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7dMv+rk4AYNbYWMVux159gAB3N7LSg_bV96i8bHp0CTig@mail.gmail.com>

On 7 June 2016 at 12:01, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Serhiy Storchaka writes:
>
>  > I think representing bytes as an array of ints was good decision. If you
>  > need indexing to return a substring, you should use str instead. It is
>  > as well memory efficient thanks to PEP 393.
>
> You can do this by using latin-1 as the codec, but that's pretty
> unpleasant, because of the risk of combining with another str and
> getting mojibake.
>
> I have long thought that it would be interesting to have a codec and
> an extension to PEP 393 that gives "asciibytes" behavior.  That is,
> the codec simply slops the bytes into the 8-bit storage of a string,
> but when joined with another string the result types are:
>
> asciibytes        other arg        result
>  has 8bit           type            type
>    yes            pure ascii     asciibytes
>    yes            asciibytes     asciibytes
>    yes            other str      str with 8bit bytes from asciibytes
>                                  encoded as PEP 383 surrogateescape
>                                  (note: promotes latin1 to 2-byte-wide)
>     no             whatever      whatever
>
> I think Nick actually had a module that worked pretty much like this,
> but he never pushed it.  I've never had time to reason out the
> possible failure modes, though, or the performance issues.  And it's
> not an itch I personally need to scratch.

Benno Rice, rather than me (although I gave Benno the idea):
https://github.com/jeamland/asciicompat

Managing extra C dependencies is a pain though, and it's a dubious
idea at best, so neither of us seriously pushed for anyone to use it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Tue Jun  7 15:49:01 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2016 12:49:01 -0700
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <5757127B.4050300@egenix.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
 <5755FEE3.9020702@egenix.com> <20160606191449.12dd6f67@anarchist.wooz.org>
 <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>
 <5757127B.4050300@egenix.com>
Message-ID: <CADiSq7fKOnw0CjwoTbDYEj-yEnDVg8uGQJ6WFkTL4etdQ5vRzQ@mail.gmail.com>

On 7 June 2016 at 11:29, M.-A. Lemburg <mal at egenix.com> wrote:
> One approach would be to define what a decorator means in front
> of an expression (since that's currently a SyntaxError), without
> adding any new syntactic sugar. Here's a sketch...
>
> @decorator
> x = obj
>
> compiles to:
>
> x = obj
> decorator('x', obj, lineno)

As a possible guide to designing the signatures for binding
decorators, it's probably worth asking what would be needed to make:

    @bindfunction
    f = lambda : None

equivalent to:

    def f(): pass

Since the interpreter already sets __module__ and __globals__
correctly on lambda functions, the main considerations would be to get
f.__name__ and f.__qualname__ set correctly, which means just the
immediate target would be insufficient - you'd also want the scope
naming information that gets included in __qualname__, but is omitted
from __name__.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Tue Jun  7 16:21:57 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2016 13:21:57 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <57571228.3030806@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
Message-ID: <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>

On 7 June 2016 at 11:27, Thomas G?ttler <guettliml at thomas-guettler.de> wrote:
> Am 07.06.2016 um 13:34 schrieb Steven D'Aprano:
>> On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas G?ttler wrote:
>>
>>> I am not married with Python. Up to now I see no alternative for me,
>>> but I guess sooner or later I will switch to a different
>>> language.
>>>
>>> I see only few benefits from porting my code to Python3. I will
>>> use Python2 at least for the next 12 months.
>>
>> That's your right, of course. Some people love to experiment with new
>> languages, some don't. But...
>>
>> What will this other language be, and what does it offer that makes it
>> better than Python 3?
>
> I would love to see "compile to javascript" support.

Not a problem for python-dev to tackle - there are plenty of
Python-in-the-browser projects, and they don't need to wait for
python-dev's permission (since they're taking the existing language
and making it available in a new context, rather than changing the
language). They'd only need to come back to python-dev if they wanted
language level support for browser concepts like WebWorkers and the
Domain Object Model.

> I would love to see a package management that is easy to use and which
> focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA
and the PSF (but unfortunately is never going to be entirely simple
given Python's broad scope of use).

> Does Python3 run in web browsers? Or is it possible to compile it to
> javascript?

Again, these are software distribution questions, not language design questions.

> I guess I will port my python2 code to python3 sooner or later. If
> Django drops support for Python2, then I will do it.

Exactly, and this is what we expect most folks with problems that are
already well solved by Python 2 to do: it won't be the availability of
alternatives that prompts them to find something else (since there are
already a number of those in the form of both Python 3 and other
language ecosystems), it will be folks ceasing to support the Python 2
ecosystem for free. By the time upstream Python 2.7 support ends in
2020, there will have been ~14 years of parallel development on both
Python 2.x and 3.x, and ~12 years of parallel support.

This isn't a new phenomenon - it's one that's seen regularly at the
operating system level, to the point where people will pay exhorbitant
amounts of money to keep old software supported, even though they
could attain both happier staff and lower operating costs by doing a
bit more forward planning and budgeting for their infrastructure
upgrades.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ethan at stoneleaf.us  Tue Jun  7 16:31:19 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 07 Jun 2016 13:31:19 -0700
Subject: [Python-ideas] bytes indexing behavior
In-Reply-To: <CADiSq7cP2f+gsZfOqqejx+ONvFYEhjzbku1gHoAg8Bk4kmL-tg@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <08e901d1beb8$e60915b0$b21b4110$@hotmail.com>
 <CAP7+vJKCAE97sa_yUgrW_1onjp=0qSoJ_GBay0t-BQVx6H8XMQ@mail.gmail.com>
 <20160606193530.GA27695@python.ca> <loom.20160607T132327-826@post.gmane.org>
 <5756EF5A.106@stoneleaf.us>
 <CADiSq7cP2f+gsZfOqqejx+ONvFYEhjzbku1gHoAg8Bk4kmL-tg@mail.gmail.com>
Message-ID: <57572F17.4010009@stoneleaf.us>

On 06/07/2016 12:57 PM, Nick Coghlan wrote:
> On 7 June 2016 at 08:59, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 06/07/2016 04:27 AM, Antoine Pitrou wrote:
>>
>>> While I prefer Python 2's indexing behaviour, IMHO that ship has sailed.
>>> Python 3.5 is probably seeing massive adoption now, and by introducing
>>> another behaviour switch we only confuse users and make their lives
>>> miserable.
>>
>>
>> I agree on both counts.  Wasn't there a proposal/issue about adding some
>> convenience methods to bytes to help ease this pain?  My quick search did
>> not find it.
>
> It was in PEP 467:
> https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-methods-that-produce-bytes-objects
>
> If someone wanted to take that PEP and drive it through to resolution,
> I'd be happy to hand it over (my recollection is that the sticking
> point in the previous discussion was the proposed constructor changes,
> so dropping those may make it easier to get the additional method
> accepted).

Ah, thanks.

Split new thread to resolve that PEP.

--
~Ethan~


From michael.selik at gmail.com  Tue Jun  7 18:51:58 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Tue, 07 Jun 2016 22:51:58 +0000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CADiSq7fKOnw0CjwoTbDYEj-yEnDVg8uGQJ6WFkTL4etdQ5vRzQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
 <5755FEE3.9020702@egenix.com> <20160606191449.12dd6f67@anarchist.wooz.org>
 <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>
 <5757127B.4050300@egenix.com>
 <CADiSq7fKOnw0CjwoTbDYEj-yEnDVg8uGQJ6WFkTL4etdQ5vRzQ@mail.gmail.com>
Message-ID: <CAGgTfkMjKLADk_wEnaYKc0Txk7HFYcv+gSnWB7LzHPu+VvKxYg@mail.gmail.com>

On Tue, Jun 7, 2016 at 4:13 PM Nick Coghlan <ncoghlan at gmail.com> wrote:

> As a possible guide to designing the signatures for binding
> decorators, it's probably worth asking what would be needed to make:
>
>     @bindfunction
>     f = lambda : None
>
> equivalent to:
>
>     def f(): pass
>
> Since the interpreter already sets __module__ and __globals__
> correctly on lambda functions, the main considerations would be to get
> f.__name__ and f.__qualname__ set correctly, which means just the
> immediate target would be insufficient - you'd also want the scope
> naming information that gets included in __qualname__, but is omitted
> from __name__.
>

The ``bindfunction`` function could look up the __module__ and use that to
assign the __qualname__. Would that satisfy?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160607/8b8ebdb1/attachment.html>

From ncoghlan at gmail.com  Tue Jun  7 19:09:28 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2016 16:09:28 -0700
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CAGgTfkMjKLADk_wEnaYKc0Txk7HFYcv+gSnWB7LzHPu+VvKxYg@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org>
 <5755FEE3.9020702@egenix.com> <20160606191449.12dd6f67@anarchist.wooz.org>
 <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>
 <5757127B.4050300@egenix.com>
 <CADiSq7fKOnw0CjwoTbDYEj-yEnDVg8uGQJ6WFkTL4etdQ5vRzQ@mail.gmail.com>
 <CAGgTfkMjKLADk_wEnaYKc0Txk7HFYcv+gSnWB7LzHPu+VvKxYg@mail.gmail.com>
Message-ID: <CADiSq7ddQ1TLSbS_=J4Srwg8RfA7DB2orh6=ULUNe+34W+TTiA@mail.gmail.com>

On 7 June 2016 at 15:51, Michael Selik <michael.selik at gmail.com> wrote:
> On Tue, Jun 7, 2016 at 4:13 PM Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> As a possible guide to designing the signatures for binding
>> decorators, it's probably worth asking what would be needed to make:
>>
>>     @bindfunction
>>     f = lambda : None
>>
>> equivalent to:
>>
>>     def f(): pass
>>
>> Since the interpreter already sets __module__ and __globals__
>> correctly on lambda functions, the main considerations would be to get
>> f.__name__ and f.__qualname__ set correctly, which means just the
>> immediate target would be insufficient - you'd also want the scope
>> naming information that gets included in __qualname__, but is omitted
>> from __name__.
>
>
> The ``bindfunction`` function could look up the __module__ and use that to
> assign the __qualname__. Would that satisfy?

Not really, due to class and function nesting:

    >>> class A:
    ...     class B:
    ...        def f(self):
    ...             def g(): pass
    ...             return g
    ...
    >>> A.B().f.__qualname__
    'A.B.f'
    >>> A.B().f().__qualname__
    'A.B.f.<locals>.g'

That's why I brought it up as a design problem worth considering -
it's *very* easy to inadvertently design namebinding feature that only
do the right thing at module scope, and misbehave at class and
function scope.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Tue Jun  7 22:40:56 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 8 Jun 2016 12:40:56 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <5755FEE3.9020702@egenix.com>
References: <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
Message-ID: <20160608024056.GH12028@ando.pearwood.info>

On Tue, Jun 07, 2016 at 12:53:23AM +0200, M.-A. Lemburg wrote:
> On 07.06.2016 00:32, Barry Warsaw wrote:
> > On Jun 01, 2016, at 03:53 PM, M.-A. Lemburg wrote:
> > 
> >> This could be done via a decorator:
> >>
> >>    @recordbinding
> >>    x = obj
> >>
> >> to result in the compiler generating the following code:
> >>
> >>    x = obj
> >>    obj.recordbinding('x', 2)

I don't understand the purpose of the second argument, given as 2. 
Earlier you say that it's the line number. Why is that needed?

I don't see how this actually solves the problem. Apply it to 
namedtuple:

Record = namedtuple('Record', fields)

If you naively try the decorator:

@recordbinding
Record = namedtuple(fields)

that gets converted to:

Record = namedtuple(fields)  # TypeError
Record.recordbinding('Record')  # AttributeError

There's no point in calling a special method on the RHS object, as that 
requires the object to have been constructed before the method can be 
called. And that requires the name. So your recordbinding decorator gets 
called too late.

Even if it worked, I strongly dislike that this turns a one-line 
expression into a two line statement.

# Status quo:
x = sympy.Symbol('x')

# Becomes:
@recordbinding
x = sympy.Symbol()

which doesn't seem like an improvement to me.

[...]
> This would work as well and indeed reads better, but you'd need
> to have the compiler generate:
> 
>       x = obj
>       recordbinding(obj, 'x', 2)
> 
> ie. pass in the object, the bound name and the line number
> and recordbinding would then have to decide what to do with the
> parameters.

Again, this is called too late. Your namedtuple needs its name when it 
is constructed, it can't be constructed first and then the name injected 
in, even if recordbinding() knew where to inject the name.


-- 
Steve

From eric at trueblade.com  Tue Jun  7 23:17:26 2016
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 7 Jun 2016 23:17:26 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160608024056.GH12028@ando.pearwood.info>
References: <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
 <20160608024056.GH12028@ando.pearwood.info>
Message-ID: <b3c28bea-78ae-5a4c-7385-e6dce638f721@trueblade.com>

On 6/7/2016 10:40 PM, Steven D'Aprano wrote:
> On Tue, Jun 07, 2016 at 12:53:23AM +0200, M.-A. Lemburg wrote:
>> On 07.06.2016 00:32, Barry Warsaw wrote:
>>> On Jun 01, 2016, at 03:53 PM, M.-A. Lemburg wrote:
>>>
>>>> This could be done via a decorator:
>>>>
>>>>    @recordbinding
>>>>    x = obj
>>>>
>>>> to result in the compiler generating the following code:
>>>>
>>>>    x = obj
>>>>    obj.recordbinding('x', 2)
>
> I don't understand the purpose of the second argument, given as 2.
> Earlier you say that it's the line number. Why is that needed?
>
> I don't see how this actually solves the problem. Apply it to
> namedtuple:
>
> Record = namedtuple('Record', fields)
>
> If you naively try the decorator:
>
> @recordbinding
> Record = namedtuple(fields)
>
> that gets converted to:
>
> Record = namedtuple(fields)  # TypeError
> Record.recordbinding('Record')  # AttributeError

I think we might need some helpers, and a slight change to the 
specifics. I'd have:

@binding_method
x = obj

result in:
x = binding_method('x', obj)

Then you could just say:

@namedtuple
Point = 'x y z'

Which would become:

Point = namedtuple('Point', 'x y z')

(or equivalently in this case:
@namedtuple
Point = ['x', 'y', 'z']
)

As has been pointed out, it's possible a different signature would be 
needed (not sure about line numbers, but qualname seems desirable). In 
that case, "namedtuple" above might not be "collections.namedtuple", but 
some helper.

The question for me is: do we want to have something that tells the 
compiler that "binding_method" or "namedtuple" above are special, or is 
this just what the compiler does for all uses of what looks like a 
decorated assignment statement?

Eric.

> There's no point in calling a special method on the RHS object, as that
> requires the object to have been constructed before the method can be
> called. And that requires the name. So your recordbinding decorator gets
> called too late.
>
> Even if it worked, I strongly dislike that this turns a one-line
> expression into a two line statement.
>
> # Status quo:
> x = sympy.Symbol('x')
>
> # Becomes:
> @recordbinding
> x = sympy.Symbol()
>
> which doesn't seem like an improvement to me.
>
> [...]
>> This would work as well and indeed reads better, but you'd need
>> to have the compiler generate:
>>
>>       x = obj
>>       recordbinding(obj, 'x', 2)
>>
>> ie. pass in the object, the bound name and the line number
>> and recordbinding would then have to decide what to do with the
>> parameters.
>
> Again, this is called too late. Your namedtuple needs its name when it
> is constructed, it can't be constructed first and then the name injected
> in, even if recordbinding() knew where to inject the name.
>
>


From pavol.lisy at gmail.com  Wed Jun  8 03:34:07 2016
From: pavol.lisy at gmail.com (Pavol Lisy)
Date: Wed, 8 Jun 2016 09:34:07 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
Message-ID: <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>

2016-06-05 7:14 GMT+02:00, Nick Coghlan <ncoghlan at gmail.com>:

> Red Hat's Python maintenance team are working on a more explicitly
> minimalistic porting guide based on their experiences porting Fedora
> components, sometimes in the context of upstream projects that are more
> interested in keeping Python 2.4 support than they are in Python 3:
> http://portingguide.readthedocs.io/en/latest/

do you have some drafts how to write (new) code supporting python from
2.4 - 2.7 with intention to port it to python3 in future?

From mal at egenix.com  Wed Jun  8 03:59:52 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 8 Jun 2016 09:59:52 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CADiSq7fKOnw0CjwoTbDYEj-yEnDVg8uGQJ6WFkTL4etdQ5vRzQ@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <20160531030810.GR12028@ando.pearwood.info>
 <CACac1F_qaxHnKJ7ytEOkf5sjZD3FDzRy2S05m6Rpn6Cn1HVP7w@mail.gmail.com>
 <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
 <20160606191449.12dd6f67@anarchist.wooz.org>
 <CAP7+vJ+XUoM9pMMNYA_uW9svC5hBfpy0p6=K96sKFf7mAHYdtw@mail.gmail.com>
 <5757127B.4050300@egenix.com>
 <CADiSq7fKOnw0CjwoTbDYEj-yEnDVg8uGQJ6WFkTL4etdQ5vRzQ@mail.gmail.com>
Message-ID: <5757D078.6040209@egenix.com>

On 07.06.2016 21:49, Nick Coghlan wrote:
> On 7 June 2016 at 11:29, M.-A. Lemburg <mal at egenix.com> wrote:
>> One approach would be to define what a decorator means in front
>> of an expression (since that's currently a SyntaxError), without
>> adding any new syntactic sugar. Here's a sketch...
>>
>> @decorator
>> x = obj
>>
>> compiles to:
>>
>> x = obj
>> decorator('x', obj, lineno)
> 
> As a possible guide to designing the signatures for binding
> decorators, it's probably worth asking what would be needed to make:
> 
>     @bindfunction
>     f = lambda : None
> 
> equivalent to:
> 
>     def f(): pass
> 
> Since the interpreter already sets __module__ and __globals__
> correctly on lambda functions, the main considerations would be to get
> f.__name__ and f.__qualname__ set correctly, which means just the
> immediate target would be insufficient - you'd also want the scope
> naming information that gets included in __qualname__, but is omitted
> from __name__.

Well, in your example it would still be enough:

 >>> f = lambda : None
 >>> f.__qualname__
 '<lambda>'
 >>> f.__name__
 '<lambda>'
 >>> def f(): pass
 ...
 >>> f.__qualname__
 'f'
 >>> f.__name__
 'f'

and even at deeper levels, you could base the new .__qualname__
on the one that is set on the lambda function:

 >>> class C:
 ...  f = lambda : None
 ...
 >>> C.f
 <function C.<lambda> at 0x7f5f838736a8>
 >>> C.f.__qualname__
 'C.<lambda>'
 >>> C.f.__name__
 '<lambda>'

The .__qualname__ is set at the time the lambda is built,
so it is known even correct after making the assignment
in the class definition:

 >>> class C:
 ...  f = lambda : None
 ...  print(f.__qualname__)
 ...
 C.<lambda>

so I guess this all works - much to my own surprise, because
I wouldn't have expected e.g. the last experiment to
actually succeed. Someone did a good job there :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 08 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From ben+python at benfinney.id.au  Wed Jun  8 04:04:39 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 08 Jun 2016 18:04:39 +1000
Subject: [Python-ideas] Smoothing transition to Python 3
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
Message-ID: <85y46g14yw.fsf@benfinney.id.au>

Pavol Lisy <pavol.lisy at gmail.com> writes:

> do you have some drafts how to write (new) code supporting python from
> 2.4 - 2.7 with intention to port it to python3 in future?

I would advise:

* Write the code targeting Python 3.5 primarily.

* Back-port the code from the Python 3 source to Python 2.

This is a much easier transition because, instead of hunting
hard-to-find mistakes (e.g. the broken ?text and bytes are the same?
assumption in Python 2 code), instead it entails adding some temporary
work-arounds. If the code works on Python 2 with your unit tests, your
job is done.

-- 
 \                ?I got fired from my job the other day. They said my |
  `\          personality was weird. ? That's okay, I have four more.? |
_o__)                                       ?Bug-Eyed Earl, _Red Meat_ |
Ben Finney


From guettliml at thomas-guettler.de  Wed Jun  8 06:56:44 2016
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Wed, 8 Jun 2016 12:56:44 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>
Message-ID: <5757F9EC.1090708@thomas-guettler.de>



Am 07.06.2016 um 17:26 schrieb Guido van Rossum:
> On Tue, Jun 7, 2016 at 3:42 AM, Thomas G?ttler <guettliml at thomas-guettler.de <mailto:guettliml at thomas-guettler.de>> wrote:
>
>     Am 04.06.2016 um 10:27 schrieb Neil Schemenauer:
>     > I can't understand why people don't see
>     > we have a problem.
>
>     Imagine you are father of a three years old boy. Some stranger
>     tells you: "your kid is ugly!"
>
>
> Just in case you didn't realize this can be turned around too, that's exactly how it feels to *me* when people complain
> about Python 3.

Yes, this hurts. But nevertheless you listen and try to understand
the reasons. That's why Python is successful. Thank you Guido!

>
>     How does the average man react? Some get sad, some get angry.
>
>     Only very few would ask: Why do you think my kid is ugly?
>
>     I am not married with Python. Up to now I see no alternative for me,
>     but I guess sooner or later I will switch to a different
>     language.
>
>     I see only few benefits from porting my code to Python3. I will
>     use Python2 at least for the next 12 months.
>
>
> That sounds like a threat (or another underhanded insult), but it's really up to you.

 From Wikipedia: ... a devil's advocate is someone who, given a certain argument, takes a position they do not 
necessarily agree with (or simply an alternative position from the accepted norm), for the sake of debate or to explore 
the thought further.

Back to Python3: I compare the 3.2 and current docs about "Porting Python 2 Code to Python 3"

https://docs.python.org/3.2/howto/pyporting.html
https://docs.python.org/3.5/howto/pyporting.html

The 3.2 version started with "Choosing a Strategy" the current docs start with "1. ..., 2. ..., 3. ..."

I read the porting docs years ago, and I was not able to choose a strategy. I was unsure. (sheeps don't like fog)
I think, at least sometimes, docs are more important than code.

The current docs give me a nice roadmap. I will follow them, but not in 2016.

Regards,
   Thomas G?ttler




Regards,
   Thomas G?ttler

-- 
Thomas Guettler http://www.thomas-guettler.de/

From guettliml at thomas-guettler.de  Wed Jun  8 07:02:03 2016
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Wed, 8 Jun 2016 13:02:03 +0200
Subject: [Python-ideas] package management is not python-dev's problem :-(
In-Reply-To: <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
 <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
Message-ID: <5757FB2B.90604@thomas-guettler.de>

>> I would love to see a package management that is easy to use and which
>> focuses on simple data structures.
>
> Also not python-dev's problem - that one falls on distutils-sig, PyPA
> and the PSF (but unfortunately is never going to be entirely simple
> given Python's broad scope of use).

This statement make me sad. Why is this not python-dev's problem?

Regards,
   Thomas G?ttler


-- 
Thomas Guettler http://www.thomas-guettler.de/

From p.f.moore at gmail.com  Wed Jun  8 07:10:01 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Jun 2016 12:10:01 +0100
Subject: [Python-ideas] package management is not python-dev's problem
 :-(
In-Reply-To: <5757FB2B.90604@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
 <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
 <5757FB2B.90604@thomas-guettler.de>
Message-ID: <CACac1F8vugrHTYL-sK82+sovk8Dytxoi5dCPezFQpPBPNDDBpA@mail.gmail.com>

On 8 June 2016 at 12:02, Thomas G?ttler <guettliml at thomas-guettler.de> wrote:
>>> I would love to see a package management that is easy to use and which
>>> focuses on simple data structures.
>>
>>
>> Also not python-dev's problem - that one falls on distutils-sig, PyPA
>> and the PSF (but unfortunately is never going to be entirely simple
>> given Python's broad scope of use).
>
>
> This statement make me sad. Why is this not python-dev's problem?

Because distutils-sig is the central point for packaging related issues.
Paul

From donald at stufft.io  Wed Jun  8 07:14:22 2016
From: donald at stufft.io (Donald Stufft)
Date: Wed, 8 Jun 2016 07:14:22 -0400
Subject: [Python-ideas] package management is not python-dev's problem
 :-(
In-Reply-To: <5757FB2B.90604@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
 <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
 <5757FB2B.90604@thomas-guettler.de>
Message-ID: <B1B534BB-1EC4-431F-B06A-24FC820AD433@stufft.io>


> On Jun 8, 2016, at 7:02 AM, Thomas G?ttler <guettliml at thomas-guettler.de> wrote:
> 
>>> I would love to see a package management that is easy to use and which
>>> focuses on simple data structures.
>> 
>> Also not python-dev's problem - that one falls on distutils-sig, PyPA
>> and the PSF (but unfortunately is never going to be entirely simple
>> given Python's broad scope of use).
> 
> This statement make me sad. Why is this not python-dev's problem?
> 

Historically python-dev hasn?t been involved much in it and almost all of the
tooling for packaging is developed externally to Python itself. This is generally
a strength, since packaging tools tend to work best when they?re not lied to the
lifetime of the language itself.

In the abstract, it *could* be python-dev?s problem, but Python as a community
didn?t choose that solution, it chose to delegate that task outside of it and
I (though I am biased) don?t think that trying to move it into the python-dev
domain brings much in the way of benefits.

?
Donald Stufft




From srkunze at mail.de  Wed Jun  8 09:53:50 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 8 Jun 2016 15:53:50 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <8455818d-ce86-5978-abae-255bcc28e7c2@btinternet.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
 <8455818d-ce86-5978-abae-255bcc28e7c2@btinternet.com>
Message-ID: <5758236E.7050408@mail.de>

On 07.06.2016 15:24, Rob Cliffe wrote:
>
>
> On 06/06/2016 11:44, Sven R. Kunze wrote:
>>
>>
>> Thinking this further, __init__ would be the only function to change 
>> the state of an immutable object. Once created, it will never change. 
>> Immutable also implies hashability IMHO. Moreover, immutable object 
>> would not be allowed to query data from global/external variables as 
>> those can change and would change the observable state of the object 
>> without the object noticing. So, the allowed way of creating a state 
>> for an immutable object would be using a new container as you did (by 
>> defining self._cache) and store immutable objects only there. Would 
>> this make sense?
>>
>> Sven
> That would be one viable design.
> But aren't there use cases for initially creating an object as 
> mutable, then at some point "freezing" it by changing the mutable flag 
> from True to False?
> (Obviously the converse would not be allowed.)

Object freezing sounds like a crazy dynamic idea.

Unfreezing would be even crazier.


I think that would be even better than a static mutable flag. I like 
that. :)


Sven

From eric at trueblade.com  Wed Jun  8 10:33:29 2016
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 8 Jun 2016 10:33:29 -0400
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <5758236E.7050408@mail.de>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
 <8455818d-ce86-5978-abae-255bcc28e7c2@btinternet.com>
 <5758236E.7050408@mail.de>
Message-ID: <F97A782A-8B80-418B-B212-EF881904FEF1@trueblade.com>

> On Jun 8, 2016, at 9:53 AM, Sven R. Kunze <srkunze at mail.de> wrote:
> 
>> On 07.06.2016 15:24, Rob Cliffe wrote:
>> 
>> 
>>> On 06/06/2016 11:44, Sven R. Kunze wrote:
>>> 
>>> 
>>> Thinking this further, __init__ would be the only function to change the state of an immutable object. Once created, it will never change. Immutable also implies hashability IMHO. Moreover, immutable object would not be allowed to query data from global/external variables as those can change and would change the observable state of the object without the object noticing. So, the allowed way of creating a state for an immutable object would be using a new container as you did (by defining self._cache) and store immutable objects only there. Would this make sense?
>>> 
>>> Sven
>> That would be one viable design.
>> But aren't there use cases for initially creating an object as mutable, then at some point "freezing" it by changing the mutable flag from True to False?
>> (Obviously the converse would not be allowed.)
> 
> Object freezing sounds like a crazy dynamic idea.
> 
> Unfreezing would be even crazier.
> 
> 
> I think that would be even better than a static mutable flag. I like that. :)
> 

See the rejected PEP-351 for one version of the freeze protocol. 

--
Eric.



From steve at pearwood.info  Wed Jun  8 11:19:37 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 9 Jun 2016 01:19:37 +1000
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <b3c28bea-78ae-5a4c-7385-e6dce638f721@trueblade.com>
References: <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
 <20160608024056.GH12028@ando.pearwood.info>
 <b3c28bea-78ae-5a4c-7385-e6dce638f721@trueblade.com>
Message-ID: <20160608151937.GL12028@ando.pearwood.info>

On Tue, Jun 07, 2016 at 11:17:26PM -0400, Eric V. Smith wrote:

> I think we might need some helpers, and a slight change to the 
> specifics. I'd have:
> 
> @binding_method
> x = obj
> 
> result in:
> x = binding_method('x', obj)

That's a bit more promising. 

Disadvantage:
- what was one line is now two;
- confusing to pass more than a single argument (plus the implicit name) 
  to the function;
- fails the principle "things that look similar should be similar".

Status quo:

    Record = namedtuple('Record', fields)

would become:

    @namedtuple
    Record = fields

which doesn't look awful. I'm sad that it needs two lines.

But what if you want to pass more than one argument?

    @namedtuple
    Record = fields, True

That will be equivalent to

    namedtuple('Record', (fields, True))

which is not what is wanted. And it gets worse if you use a keyword 
argument:

    Record = fields, verbose=True

I don't really like the way the @ syntax is being used for two 
completely different things.

    @function
    def spam(): ...

does one thing, but

   @function
   spam = ...

does a completely different and unrelated thing. I'm not saying that @ 
cannot be used for anything but decorators, but I think it is confusing 
to use something which looks so close to decorator syntax for something 
that is nothing like a decorator.


> The question for me is: do we want to have something that tells the 
> compiler that "binding_method" or "namedtuple" above are special, or is 
> this just what the compiler does for all uses of what looks like a 
> decorated assignment statement?

I'm surprised you ask that question :-) What does the Zen say about 
special cases?

I don't think it is desirable to have developers have to go cap in hand 
to the core devs and say "Please sir, I have a function that needs to 
know its name, can you please add it to the privileged list of special 
functions that work with @ please?"

*wink*

It should either work for any name after the @ or not at all. Hard 
coding support for just namedtuple would be bad.


-- 
Steve

From brett at python.org  Wed Jun  8 11:48:14 2016
From: brett at python.org (Brett Cannon)
Date: Wed, 08 Jun 2016 15:48:14 +0000
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <5757F9EC.1090708@thomas-guettler.de>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <CAP7+vJK7fGVFhwR_LcTSkqyhGwm+V8_PJp1oSKcRMWyA1i0faw@mail.gmail.com>
 <5757F9EC.1090708@thomas-guettler.de>
Message-ID: <CAP1=2W5dd4JvtnYGU-UA7VJzdbnfxMuV4mJhGJEMtQybeY10OA@mail.gmail.com>

On Wed, Jun 8, 2016, 03:57 Thomas G?ttler <guettliml at thomas-guettler.de>
wrote:

>
>
> Am 07.06.2016 um 17:26 schrieb Guido van Rossum:
> > On Tue, Jun 7, 2016 at 3:42 AM, Thomas G?ttler <
> guettliml at thomas-guettler.de <mailto:guettliml at thomas-guettler.de>> wrote:
> >
> >     Am 04.06.2016 um 10:27 schrieb Neil Schemenauer:
> >     > I can't understand why people don't see
> >     > we have a problem.
> >
> >     Imagine you are father of a three years old boy. Some stranger
> >     tells you: "your kid is ugly!"
> >
> >
> > Just in case you didn't realize this can be turned around too, that's
> exactly how it feels to *me* when people complain
> > about Python 3.
>
> Yes, this hurts. But nevertheless you listen and try to understand
> the reasons. That's why Python is successful. Thank you Guido!
>
> >
> >     How does the average man react? Some get sad, some get angry.
> >
> >     Only very few would ask: Why do you think my kid is ugly?
> >
> >     I am not married with Python. Up to now I see no alternative for me,
> >     but I guess sooner or later I will switch to a different
> >     language.
> >
> >     I see only few benefits from porting my code to Python3. I will
> >     use Python2 at least for the next 12 months.
> >
> >
> > That sounds like a threat (or another underhanded insult), but it's
> really up to you.
>
>  From Wikipedia: ... a devil's advocate is someone who, given a certain
> argument, takes a position they do not
> necessarily agree with (or simply an alternative position from the
> accepted norm), for the sake of debate or to explore
> the thought further.
>

The point is that you don't state you're playing devil's advocate and you
phrase your statements in such a way that they can be (mis)construed as an
insult. There's nothing wrong with stating the other view of a discussion,
but you still have to take care to state it politely.


> Back to Python3: I compare the 3.2 and current docs about "Porting Python
> 2 Code to Python 3"
>
> https://docs.python.org/3.2/howto/pyporting.html
> https://docs.python.org/3.5/howto/pyporting.html
>
> The 3.2 version started with "Choosing a Strategy" the current docs start
> with "1. ..., 2. ..., 3. ..."
>
> I read the porting docs years ago, and I was not able to choose a
> strategy. I was unsure. (sheeps don't like fog)
> I think, at least sometimes, docs are more important than code.
>
> The current docs give me a nice roadmap. I will follow them, but not in
> 2016.
>

I changed the docs when it became obvious the community had chosen the 2/3
support structure for porting code.

-brett



> Regards,
>    Thomas G?ttler
>
>
>
>
> Regards,
>    Thomas G?ttler
>
> --
> Thomas Guettler http://www.thomas-guettler.de/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160608/cfd87431/attachment.html>

From eric at trueblade.com  Wed Jun  8 12:01:53 2016
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 8 Jun 2016 12:01:53 -0400
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160608151937.GL12028@ando.pearwood.info>
References: <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
 <20160608024056.GH12028@ando.pearwood.info>
 <b3c28bea-78ae-5a4c-7385-e6dce638f721@trueblade.com>
 <20160608151937.GL12028@ando.pearwood.info>
Message-ID: <57584171.2040700@trueblade.com>

On 06/08/2016 11:19 AM, Steven D'Aprano wrote:
> On Tue, Jun 07, 2016 at 11:17:26PM -0400, Eric V. Smith wrote:
> 
>> I think we might need some helpers, and a slight change to the 
>> specifics. I'd have:
>>
>> @binding_method
>> x = obj
>>
>> result in:
>> x = binding_method('x', obj)
> 
> That's a bit more promising. 
> 
> Disadvantage:
> - what was one line is now two;
> - confusing to pass more than a single argument (plus the implicit name) 
>   to the function;
> - fails the principle "things that look similar should be similar".
> 
> Status quo:
> 
>     Record = namedtuple('Record', fields)
> 
> would become:
> 
>     @namedtuple
>     Record = fields
> 
> which doesn't look awful. I'm sad that it needs two lines.

All true!

> But what if you want to pass more than one argument?
> 
>     @namedtuple
>     Record = fields, True
> 
> That will be equivalent to
> 
>     namedtuple('Record', (fields, True))
> 
> which is not what is wanted. And it gets worse if you use a keyword 
> argument:
> 
>     Record = fields, verbose=True

I'd say it would be:

   namedtuple('Record', fields, True)

But I'm just thinking out loud.

> I don't really like the way the @ syntax is being used for two 
> completely different things.
> 
>     @function
>     def spam(): ...
> 
> does one thing, but
> 
>    @function
>    spam = ...
> 
> does a completely different and unrelated thing. I'm not saying that @ 
> cannot be used for anything but decorators, but I think it is confusing 
> to use something which looks so close to decorator syntax for something 
> that is nothing like a decorator.

I agree. I'm just riffing on someone else's proposal.

>> The question for me is: do we want to have something that tells the 
>> compiler that "binding_method" or "namedtuple" above are special, or is 
>> this just what the compiler does for all uses of what looks like a 
>> decorated assignment statement?
> 
> I'm surprised you ask that question :-) What does the Zen say about 
> special cases?
> 
> I don't think it is desirable to have developers have to go cap in hand 
> to the core devs and say "Please sir, I have a function that needs to 
> know its name, can you please add it to the privileged list of special 
> functions that work with @ please?"
> 
> *wink*
> 
> It should either work for any name after the @ or not at all. Hard 
> coding support for just namedtuple would be bad.

I completely agree about special cases! But what I was really thinking
about here was that maybe you'd have to tell the compiler in advance
that you're defining a special type of "decorator", something along the
lines of:

############

# somehow tell the compiler that namedtuple is special
__special_assignment_psuedo_decorators__.append(namedtuple)

@namedtuple
Record = fields

############

Eric.



From steve at pearwood.info  Wed Jun  8 12:09:04 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 9 Jun 2016 02:09:04 +1000
Subject: [Python-ideas] Proposal for special name and qualname symbols
Message-ID: <20160608160904.GM12028@ando.pearwood.info>

This is an new thread inspired by the "Quick idea: defining variables 
from functions that take the variable name" thread.

For the benefit of those who haven't read that thread, in a nutshell, 
there are some functions or classes that find it useful or necessary to 
know their own name or qualified name. At the moment, apart from a few 
special cases where the compiler does the work behind the scenes 
(function and class definitions), the only way to do this is to 
hard-code the name/qualname as a string.

The classic example from the standard library is namedtuple:

Record = namedtuple('Record', fields)

Proposal: we introduce two special symbols to represent the name and 
qualname at any point. They can be used anywhere an identifier can be 
used.

We could use special dunder names, let's say __NAME__ and __QNAME__, but 
for now I'm going to use two special symbols:

@ for the unqualified bare name;
@@ for the qualified name.

The namedtuple example would become:

Record = namedtuple(@, fields)

If your function takes its name parameter somewhere other than the first 
argument, that's not a problem:

MyClass = make_a_class(spam, eggs, name=@, fnord=True)

'@' takes on the name, or dotted name, on the left-hand side of an 
assignment:

x = function(@)       # @ == 'x'
spam.x = function(@)  # @ == 'spam.x'

If there are multiple targets, and they are all simple or dotted names, 
@ is a tuple of names:

a, b, c.d = function(@)    # @ == ('a', 'b', 'c.d')
a = b = c.d = function(@)  # @ == ('a', 'b', 'c.d')

Those two cases are indistinguishable by @. I'm not sure if that's a bug 
or a feature :-)

For now, any other assignment target will be a syntax error. That 
restriction may be relaxed in the future, if necessary.

spam['key'] = function(@)  # SyntaxError


@ is not restricted to appear inside function calls:

element = mapping[@]  # like mapping['element']


The qualified name @@ gets the module name:

x = function(@@)  # @@ = 'module.x'

or, inside a class or function, the class/function name:

class K:
   attr = @@   # 'module.K.attr'
   def method(self):
       x = @@  # 'module.K.method.x'


Whether dunder names or symbols, assignment to them will be illegal:

@ = "myname"  # SyntaxError


Questions and answers:

Q: Why @ rather than $? 

A: $ is too similar to S for my liking, and I believe that Guido has 
said that $ will never be used for syntax in Python.

Q: What's the use-case for qualified names?

A: Classes need to know their qualified name for pickling.

Q: Isn't this rather magical?

A: Of course it is, but the `def` and `class` statements already make 
use of that compile magic, this just extends it to other things.

Q: Are you sure about the module being part of the qualified name?

A: I'm not wedded to that idea. If it's not necessary, take it out.

Q: Wait a minute, isn't @ used for matrix multiplication now? And @@ may 
be used for matrix exponentiation in the future?

A: Oh yeah, I forgot that. Damn. Guess we'll have to use the dunder 
names then.

Q: But the dunder names are too long to type.

A: Then think of your own symbols and propose them :-)



-- 
Steve

From ethan at stoneleaf.us  Wed Jun  8 12:45:06 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 08 Jun 2016 09:45:06 -0700
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608160904.GM12028@ando.pearwood.info>
References: <20160608160904.GM12028@ando.pearwood.info>
Message-ID: <57584B92.90905@stoneleaf.us>

On 06/08/2016 09:09 AM, Steven D'Aprano wrote:

> @ for the unqualified bare name;
> @@ for the qualified name.
>
> The namedtuple example would become:
>
> Record = namedtuple(@, fields)

-1

I suppose I could be convinced that this is a good idea, but I don't see 
it happening.

--
~Ethan~

From rymg19 at gmail.com  Wed Jun  8 13:06:14 2016
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 8 Jun 2016 12:06:14 -0500
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608160904.GM12028@ando.pearwood.info>
References: <20160608160904.GM12028@ando.pearwood.info>
Message-ID: <CAO41-mOVNmbbH1mBJztj-LB7a-aS4nNTiwso+mt8wHsmUd5VFA@mail.gmail.com>

On Jun 8, 2016 11:09 AM, "Steven D'Aprano" <steve at pearwood.info> wrote:
>
> This is an new thread inspired by the "Quick idea: defining variables
> from functions that take the variable name" thread.
>
> For the benefit of those who haven't read that thread, in a nutshell,
> there are some functions or classes that find it useful or necessary to
> know their own name or qualified name. At the moment, apart from a few
> special cases where the compiler does the work behind the scenes
> (function and class definitions), the only way to do this is to
> hard-code the name/qualname as a string.
>
> The classic example from the standard library is namedtuple:
>
> Record = namedtuple('Record', fields)
>
> Proposal: we introduce two special symbols to represent the name and
> qualname at any point. They can be used anywhere an identifier can be
> used.
>
> We could use special dunder names, let's say __NAME__ and __QNAME__, but
> for now I'm going to use two special symbols:
>
> @ for the unqualified bare name;
> @@ for the qualified name.
>
> The namedtuple example would become:
>
> Record = namedtuple(@, fields)
>
> If your function takes its name parameter somewhere other than the first
> argument, that's not a problem:
>
> MyClass = make_a_class(spam, eggs, name=@, fnord=True)
>
> '@' takes on the name, or dotted name, on the left-hand side of an
> assignment:
>
> x = function(@)       # @ == 'x'
> spam.x = function(@)  # @ == 'spam.x'
>
> If there are multiple targets, and they are all simple or dotted names,
> @ is a tuple of names:
>
> a, b, c.d = function(@)    # @ == ('a', 'b', 'c.d')
> a = b = c.d = function(@)  # @ == ('a', 'b', 'c.d')
>
> Those two cases are indistinguishable by @. I'm not sure if that's a bug
> or a feature :-)
>
> For now, any other assignment target will be a syntax error. That
> restriction may be relaxed in the future, if necessary.
>
> spam['key'] = function(@)  # SyntaxError
>
>
> @ is not restricted to appear inside function calls:
>
> element = mapping[@]  # like mapping['element']
>
>
> The qualified name @@ gets the module name:
>
> x = function(@@)  # @@ = 'module.x'
>
> or, inside a class or function, the class/function name:
>
> class K:
>    attr = @@   # 'module.K.attr'
>    def method(self):
>        x = @@  # 'module.K.method.x'
>
>
> Whether dunder names or symbols, assignment to them will be illegal:
>
> @ = "myname"  # SyntaxError
>
>
> Questions and answers:
>
> Q: Why @ rather than $?
>
> A: $ is too similar to S for my liking, and I believe that Guido has
> said that $ will never be used for syntax in Python.
>
> Q: What's the use-case for qualified names?
>
> A: Classes need to know their qualified name for pickling.
>
> Q: Isn't this rather magical?
>
> A: Of course it is, but the `def` and `class` statements already make
> use of that compile magic, this just extends it to other things.
>
> Q: Are you sure about the module being part of the qualified name?
>
> A: I'm not wedded to that idea. If it's not necessary, take it out.
>
> Q: Wait a minute, isn't @ used for matrix multiplication now? And @@ may
> be used for matrix exponentiation in the future?
>
> A: Oh yeah, I forgot that. Damn. Guess we'll have to use the dunder
> names then.
>

Wouldn't it still be unambiguous, since they're both binary operators?

> Q: But the dunder names are too long to type.
>
> A: Then think of your own symbols and propose them :-)
>
>

Well, since I'll probably get killed if I mention `?`...

What about & or ! (the exclamation point)? IIRC the latter isn't used at
all right now. It also reminds me (for better or worse) of hashbangs.

Only other symbols I can think of would be \ and ` (the backtick), but I'm
not sure what the heck the idea behind them is (although \ could probably
be referenced to lambdas or something?), and the backtick is kind of
unreadable...

>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
--
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something?s wrong.
http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160608/cf3280f0/attachment-0001.html>

From python at mrabarnett.plus.com  Wed Jun  8 13:06:54 2016
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 8 Jun 2016 18:06:54 +0100
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608160904.GM12028@ando.pearwood.info>
References: <20160608160904.GM12028@ando.pearwood.info>
Message-ID: <6a57d344-432e-3734-f55d-f986aeda85a1@mrabarnett.plus.com>

On 2016-06-08 17:09, Steven D'Aprano wrote:
[snip]
> Q: Wait a minute, isn't @ used for matrix multiplication now? And @@ may
> be used for matrix exponentiation in the future?
>
> A: Oh yeah, I forgot that. Damn. Guess we'll have to use the dunder
> names then.
>
> Q: But the dunder names are too long to type.
>
> A: Then think of your own symbols and propose them :-)
>
I don't think that using @ as you did will be a problem.

After all, * is used both as an infix operator and a prefix to pack/unpack.


From mafagafogigante at gmail.com  Wed Jun  8 13:12:59 2016
From: mafagafogigante at gmail.com (Bernardo Sulzbach)
Date: Wed, 8 Jun 2016 14:12:59 -0300
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <CAO41-mOVNmbbH1mBJztj-LB7a-aS4nNTiwso+mt8wHsmUd5VFA@mail.gmail.com>
References: <20160608160904.GM12028@ando.pearwood.info>
 <CAO41-mOVNmbbH1mBJztj-LB7a-aS4nNTiwso+mt8wHsmUd5VFA@mail.gmail.com>
Message-ID: <6f1ee621-7524-47b2-6b06-3e9cb7ecfb03@gmail.com>

On 06/08/2016 02:06 PM, Ryan Gonzalez wrote:
> What about & or ! (the exclamation point)? IIRC the latter isn't used at
> all right now. It also reminds me (for better or worse) of hashbangs.
>
> Only other symbols I can think of would be \ and ` (the backtick), but I'm
> not sure what the heck the idea behind them is (although \ could probably
> be referenced to lambdas or something?), and the backtick is kind of
> unreadable...
>

& is bitwise "and", right? I don't think it should be overloaded.

"Kind of unreadable" is very generous for the backtick. Also, 
backslashes are just not right for this.

From sjoerdjob at sjoerdjob.com  Wed Jun  8 13:14:51 2016
From: sjoerdjob at sjoerdjob.com (Sjoerd Job Postmus)
Date: Wed, 8 Jun 2016 19:14:51 +0200
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <20160608151937.GL12028@ando.pearwood.info>
References: <574D8B81.9070108@canterbury.ac.nz>
 <20160531140416.GU12028@ando.pearwood.info>
 <574E1EE8.1080601@canterbury.ac.nz> <574E225C.6090400@feete.org>
 <574EE0FF.8060308@mail.de> <574EE8C0.1070800@egenix.com>
 <20160606183221.1c916e52@anarchist.wooz.org> <5755FEE3.9020702@egenix.com>
 <20160608024056.GH12028@ando.pearwood.info>
 <b3c28bea-78ae-5a4c-7385-e6dce638f721@trueblade.com>
 <20160608151937.GL12028@ando.pearwood.info>
Message-ID: <D6FF3698-B036-45DC-B2AE-26D0F4876E31@sjoerdjob.com>



> On 8 Jun 2016, at 17:19, Steven D'Aprano <steve at pearwood.info> wrote:
> 
>> On Tue, Jun 07, 2016 at 11:17:26PM -0400, Eric V. Smith wrote:
>> 
>> I think we might need some helpers, and a slight change to the 
>> specifics. I'd have:
>> 
>> @binding_method
>> x = obj
>> 
>> result in:
>> x = binding_method('x', obj)
> 
> That's a bit more promising. 
> 
> Disadvantage:
> - what was one line is now two;
> - confusing to pass more than a single argument (plus the implicit name) 
>  to the function;
> - fails the principle "things that look similar should be similar".
> 
> Status quo:
> 
>    Record = namedtuple('Record', fields)
> 
> would become:
> 
>    @namedtuple
>    Record = fields
> 
> which doesn't look awful. I'm sad that it needs two lines.
> 
> But what if you want to pass more than one argument?
> 
>    @namedtuple
>    Record = fields, True
> 
> That will be equivalent to
> 
>    namedtuple('Record', (fields, True))
> 
> which is not what is wanted. And it gets worse if you use a keyword 
> argument:
> 
>    Record = fields, verbose=True

TypeError, can't iterate over boolean

@namedtuple(verbose=True)
Record = fields

Although for namedtuple in particular, I'd rather have namedtuple be a class-generator:

class Record(namedtuple(fields)):
    pass

Or

class Record(metaclass=namedtuple):
    fields = fields

And namedtuple could abuse all the g(l)ory details of metaclasses and/of eval to do its job. And that is the right solution for namedtuple. 

The cases that interest me more are typevar/symbol/Django modelfields/sqlalchemy declarative fields and all the cases where you're not constructing a class(like) thingy.
> 
> I don't really like the way the @ syntax is being used for two 
> completely different things.
> 
>    @function
>    def spam(): ...
> 
> does one thing, but
> 
>   @function
>   spam = ...
> 
> does a completely different and unrelated thing. I'm not saying that @ 
> cannot be used for anything but decorators, but I think it is confusing 
> to use something which looks so close to decorator syntax for something 
> that is nothing like a decorator.
> 
> 
>> The question for me is: do we want to have something that tells the 
>> compiler that "binding_method" or "namedtuple" above are special, or is 
>> this just what the compiler does for all uses of what looks like a 
>> decorated assignment statement?
> 
> I'm surprised you ask that question :-) What does the Zen say about 
> special cases?
> 
> I don't think it is desirable to have developers have to go cap in hand 
> to the core devs and say "Please sir, I have a function that needs to 
> know its name, can you please add it to the privileged list of special 
> functions that work with @ please?"
> 
> *wink*
> 
> It should either work for any name after the @ or not at all. Hard 
> coding support for just namedtuple would be bad.
> 
> 
> -- 
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From brenbarn at brenbarn.net  Wed Jun  8 14:28:00 2016
From: brenbarn at brenbarn.net (Brendan Barnwell)
Date: Wed, 08 Jun 2016 11:28:00 -0700
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608160904.GM12028@ando.pearwood.info>
References: <20160608160904.GM12028@ando.pearwood.info>
Message-ID: <575863B0.1060604@brenbarn.net>

On 2016-06-08 09:09, Steven D'Aprano wrote:
> Proposal: we introduce two special symbols to represent the name and
> qualname at any point. They can be used anywhere an identifier can be
> used.

	Do you mean "anywhere an identifier can be used on the right-hand side 
of an assignment statement"?  Presumably these would not be allowed 
outside of assignment statements.  Would they be allowed on augmented 
assignment statements?

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown

From Nikolaus at rath.org  Wed Jun  8 15:43:41 2016
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 08 Jun 2016 12:43:41 -0700
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608160904.GM12028@ando.pearwood.info> (Steven D'Aprano's
 message of "Thu, 9 Jun 2016 02:09:04 +1000")
References: <20160608160904.GM12028@ando.pearwood.info>
Message-ID: <87oa7bqxea.fsf@thinkpad.rath.org>

On Jun 09 2016, Steven D'Aprano <steve-iDnA/YwAAsAk+I/owrrOrA at public.gmane.org> wrote:
> This is an new thread inspired by the "Quick idea: defining variables 
> from functions that take the variable name" thread.
>
> For the benefit of those who haven't read that thread, in a nutshell, 
> there are some functions or classes that find it useful or necessary to 
> know their own name or qualified name. At the moment, apart from a few 
> special cases where the compiler does the work behind the scenes 
> (function and class definitions), the only way to do this is to 
> hard-code the name/qualname as a string.
>
> The classic example from the standard library is namedtuple:
>
> Record = namedtuple('Record', fields)
>
> Proposal: we introduce two special symbols to represent the name and 
> qualname at any point. They can be used anywhere an identifier can be 
> used.
>
> We could use special dunder names, let's say __NAME__ and __QNAME__, but 
> for now I'm going to use two special symbols:
>
> @ for the unqualified bare name;
> @@ for the qualified name.
>
> The namedtuple example would become:
>
> Record = namedtuple(@, fields)
[...]

This is so similar (in both semantics and syntax) to the

Record = namedtuple($lhs, fields)

proposal that I've made several times in that thread that I am a little
offended. Either you've not read that thread all that thorougly, or you
have a rather selective memory. 


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From ben+python at benfinney.id.au  Wed Jun  8 16:12:05 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 09 Jun 2016 06:12:05 +1000
Subject: [Python-ideas] package management is not python-dev's problem
 :-(
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
 <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
 <5757FB2B.90604@thomas-guettler.de>
Message-ID: <85inxj1luy.fsf@benfinney.id.au>

Thomas G?ttler
<guettliml at thomas-guettler.de> writes:

> >> I would love to see a package management that is easy to use and which
> >> focuses on simple data structures.
> >
> > Also not python-dev's problem - that one falls on distutils-sig, PyPA
> > and the PSF (but unfortunately is never going to be entirely simple
> > given Python's broad scope of use).
>
> This statement make me sad. Why is this not python-dev's problem?

Because the management of packaging and distribution tools is not very
much related to development of the core language.

The other teams mentioned above do a demonstrably better job, so
?python-dev? gets out of their way.

-- 
 \     ?Don't be misled by the enormous flow of money into bad defacto |
  `\    standards for unsophisticated buyers using poor adaptations of |
_o__)                                     incomplete ideas.? ?Alan Kay |
Ben Finney


From Nikolaus at rath.org  Wed Jun  8 16:17:32 2016
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 08 Jun 2016 13:17:32 -0700
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <87oa7bqxea.fsf@thinkpad.rath.org> (Nikolaus Rath's message of
 "Wed, 08 Jun 2016 12:43:41 -0700")
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
Message-ID: <87lh2fqvtv.fsf@thinkpad.rath.org>

On Jun 08 2016, Nikolaus Rath <Nikolaus-BTH8mxji4b0 at public.gmane.org> wrote:
> On Jun 09 2016, Steven D'Aprano <steve-iDnA/YwAAsAk+I/owrrOrA at public.gmane.org> wrote:
>> This is an new thread inspired by the "Quick idea: defining variables 
>> from functions that take the variable name" thread.
>>
>> For the benefit of those who haven't read that thread, in a nutshell, 
>> there are some functions or classes that find it useful or necessary to 
>> know their own name or qualified name. At the moment, apart from a few 
>> special cases where the compiler does the work behind the scenes 
>> (function and class definitions), the only way to do this is to 
>> hard-code the name/qualname as a string.
>>
>> The classic example from the standard library is namedtuple:
>>
>> Record = namedtuple('Record', fields)
>>
>> Proposal: we introduce two special symbols to represent the name and 
>> qualname at any point. They can be used anywhere an identifier can be 
>> used.
>>
>> We could use special dunder names, let's say __NAME__ and __QNAME__, but 
>> for now I'm going to use two special symbols:
>>
>> @ for the unqualified bare name;
>> @@ for the qualified name.
>>
>> The namedtuple example would become:
>>
>> Record = namedtuple(@, fields)
> [...]
>
> This is so similar (in both semantics and syntax) to the
>
> Record = namedtuple($lhs, fields)
>
> proposal that I've made several times in that thread that I am a little
> offended. Either you've not read that thread all that thorougly, or you
> have a rather selective memory.

...or there might have been another reason. Sorry, that wasn't meant to
come out so aggressively.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From barry at python.org  Wed Jun  8 17:10:16 2016
From: barry at python.org (Barry Warsaw)
Date: Wed, 8 Jun 2016 17:10:16 -0400
Subject: [Python-ideas] Proposal for special name and qualname symbols
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
Message-ID: <20160608171016.074950ca@anarchist.wooz.org>

On Jun 08, 2016, at 12:43 PM, Nikolaus Rath wrote:

>This is so similar (in both semantics and syntax) to the
>
>Record = namedtuple($lhs, fields)
>
>proposal that I've made several times in that thread that I am a little
>offended. Either you've not read that thread all that thorougly, or you
>have a rather selective memory. 

It's of course quite easy to miss a specific suggestion in the various
millithreads on this list, or not quite connect the dots to see the
similarities on the initial read.  Just be happy that great minds think
alike. :)

On to the topic at hand, when I saw your suggestion and Steve's I immediately
thought I'd like to see a special symbol like __LHS__, e.g.

Record = namedtuple(__LHS__, fields)

which would at least have the benefit of not introducing a special character,
and one that could be confusing in other contexts (e.g. PEP 292 strings).

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160608/2bf258fd/attachment.sig>

From michael.selik at gmail.com  Wed Jun  8 18:57:41 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 08 Jun 2016 22:57:41 +0000
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608171016.074950ca@anarchist.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
Message-ID: <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>

On Wed, Jun 8, 2016 at 5:10 PM Barry Warsaw <barry at python.org> wrote:

> On Jun 08, 2016, at 12:43 PM, Nikolaus Rath wrote:
> On to the topic at hand, when I saw your suggestion and Steve's I
> immediately thought I'd like to see a special symbol like __LHS__, e.g.
>
> Record = namedtuple(__LHS__, fields)
>
> which would at least have the benefit of not introducing a special
> character, and one that could be confusing in other contexts (e.g. PEP 292
> strings).
>

For what it's worth, __LHS__ seems best to me, of the suggestions so far.
The fact that it's all caps suggests even more magic than most dunders.

To recap the problems this might solve:
1. pass identifier as string arg (seems trivial, status quo is fine)
2. get name and qualname for pickling (instead of sys._getframe)
3. @public for constants (as per Barry's email)

I don't know what that last one was referring to. Do you mind clarifying,
Barry (or anyone else)?

Any other problems this could help with?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160608/ee37db85/attachment.html>

From barry at python.org  Wed Jun  8 19:28:29 2016
From: barry at python.org (Barry Warsaw)
Date: Wed, 8 Jun 2016 19:28:29 -0400
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
Message-ID: <20160608192829.22a444b3@subdivisions.wooz.org>

On Jun 08, 2016, at 10:57 PM, Michael Selik wrote:

>3. @public for constants (as per Barry's email)
>
>I don't know what that last one was referring to. Do you mind clarifying,
>Barry (or anyone else)?

Sure!

I propose that it's difficult to keep __all__ up to date.  Witness the
plethora of bugs open on similar issues in the stdlib.  Furthermore, the
definition of __all__ is usually far removed from the object it names.  And
it's not obvious when looking at the object whether it is intended to be
exported or not.

@public solves these problems and I contend that its meaning is pretty
obvious, given that (unknown to me at the time) there have been at least two
other independent inventions of nearly exactly the same decorator.

My tracker issue proposing to add `public` to builtins is here:

http://bugs.python.org/issue26632

I gave a lightning talk at the Pycon 2016 language summit about it but it got
a rather frosty reception. ;)  OTOH, other people like it, I've begun to use it
in my own code, and I like it too.  Now you don't have to wait for Python to
catch up <wink>:

http://public.readthedocs.io/en/latest/
https://pypi.io/project/atpublic/
https://gitlab.com/warsaw/public

The issue in this context is that @public as a decorator only works for things
with an __name__, e.g. functions and classes.  There have been different
proposals to make it work for constants, and my approach supports calling
public() with keyword arguments, e.g.

public(SEVEN=7)
public(a_bar=Bar())

Admittedly this is rather un-Pythonic since it not only modifies the module's
__all__ but it pokes a variable binding into the module globals.  It's yucky
and I'd rather be able to do something like:

@public
SEVEN = 7

@public
a_bar = Bar()

Thus the relevance to this thread.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160608/8a415977/attachment.sig>

From robertc at robertcollins.net  Wed Jun  8 19:31:21 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Thu, 9 Jun 2016 11:31:21 +1200
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608192829.22a444b3@subdivisions.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
Message-ID: <CAJ3HoZ20JPO3kDCoSJYS-LKPK+bLprHqEnPcXrGtwGbzr5vjJw@mail.gmail.com>

On 9 June 2016 at 11:28, Barry Warsaw <barry at python.org> wrote:

> Admittedly this is rather un-Pythonic since it not only modifies the module's
> __all__ but it pokes a variable binding into the module globals.  It's yucky
> and I'd rather be able to do something like:
>
> @public
> SEVEN = 7
>
> @public
> a_bar = Bar()
>
> Thus the relevance to this thread.

I'd like to be able to do that too.

-Rob

From ethan at stoneleaf.us  Wed Jun  8 19:58:47 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 08 Jun 2016 16:58:47 -0700
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608192829.22a444b3@subdivisions.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
Message-ID: <5758B137.6050608@stoneleaf.us>

On 06/08/2016 04:28 PM, Barry Warsaw wrote:
> On Jun 08, 2016, at 10:57 PM, Michael Selik wrote:

> The issue in this context is that @public as a decorator only works for things
> with an __name__, e.g. functions and classes.  There have been different
> proposals to make it work for constants, and my approach supports calling
> public() with keyword arguments, e.g.
>
> public(SEVEN=7)
> public(a_bar=Bar())
>
> Admittedly this is rather un-Pythonic since it not only modifies the module's
> __all__ but it pokes a variable binding into the module globals.  It's yucky
> and I'd rather be able to do something like:
>
> @public
> SEVEN = 7
>
> @public
> a_bar = Bar()
>
> Thus the relevance to this thread.

How would those examples look if we had the __LHS__ working?

--
~Ethan~


From eric at trueblade.com  Wed Jun  8 21:30:24 2016
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 8 Jun 2016 21:30:24 -0400
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <5758B137.6050608@stoneleaf.us>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
 <5758B137.6050608@stoneleaf.us>
Message-ID: <58455581-8ac1-2638-5570-657ade55e5fc@trueblade.com>

On 6/8/2016 7:58 PM, Ethan Furman wrote:
> On 06/08/2016 04:28 PM, Barry Warsaw wrote:
>> On Jun 08, 2016, at 10:57 PM, Michael Selik wrote:
>
>> The issue in this context is that @public as a decorator only works
>> for things
>> with an __name__, e.g. functions and classes.  There have been different
>> proposals to make it work for constants, and my approach supports calling
>> public() with keyword arguments, e.g.
>>
>> public(SEVEN=7)
>> public(a_bar=Bar())
>>
>> Admittedly this is rather un-Pythonic since it not only modifies the
>> module's
>> __all__ but it pokes a variable binding into the module globals.  It's
>> yucky
>> and I'd rather be able to do something like:
>>
>> @public
>> SEVEN = 7
>>
>> @public
>> a_bar = Bar()
>>
>> Thus the relevance to this thread.
>
> How would those examples look if we had the __LHS__ working?

Presumably:

SEVEN = public(__LHS__, 7)
a_bar = public(__LHS__, Bar())

Although this might be a different version of public (or not!).

Eric.



From python at mrabarnett.plus.com  Wed Jun  8 22:29:07 2016
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 9 Jun 2016 03:29:07 +0100
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608192829.22a444b3@subdivisions.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
Message-ID: <4fdaf368-b9c4-16b1-42d9-b87f274bc0ca@mrabarnett.plus.com>

On 2016-06-09 00:28, Barry Warsaw wrote:
> On Jun 08, 2016, at 10:57 PM, Michael Selik wrote:
>
>>3. @public for constants (as per Barry's email)
>>
>>I don't know what that last one was referring to. Do you mind clarifying,
>>Barry (or anyone else)?
>
> Sure!
>
> I propose that it's difficult to keep __all__ up to date.  Witness the
> plethora of bugs open on similar issues in the stdlib.  Furthermore, the
> definition of __all__ is usually far removed from the object it names.  And
> it's not obvious when looking at the object whether it is intended to be
> exported or not.
>
> @public solves these problems and I contend that its meaning is pretty
> obvious, given that (unknown to me at the time) there have been at least two
> other independent inventions of nearly exactly the same decorator.
>
[snip]

A workaround is to use a comment:

#@public
SEVEN = 7

and a script that parses the file and updates the __all__.


From steve at pearwood.info  Wed Jun  8 23:43:23 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 9 Jun 2016 13:43:23 +1000
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <4fdaf368-b9c4-16b1-42d9-b87f274bc0ca@mrabarnett.plus.com>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
 <4fdaf368-b9c4-16b1-42d9-b87f274bc0ca@mrabarnett.plus.com>
Message-ID: <20160609034323.GQ12028@ando.pearwood.info>

On Thu, Jun 09, 2016 at 03:29:07AM +0100, MRAB wrote:

> A workaround is to use a comment:
> 
> #@public
> SEVEN = 7
> 
> and a script that parses the file and updates the __all__.

Ooooh! I like that! Except that instead of a scripting updating __all__, 
it's a unit test that fails.

I don't know if this should go into unittest or test.support.

https://docs.python.org/3/library/test.html#module-test.support

Off the top of my head, something like:

class TestAll(unittest.CheckAll):
    module = name_of_module_to_check
    skip = [names, to, skip]

which scans the source of module, extracts all the public names, and 
fails if any of them (other than those listed in skip) are missing from 
__all__. If the source is unavailable, the test is skipped rather than 
passing.



-- 
Steve

From barry at barrys-emacs.org  Thu Jun  9 05:01:15 2016
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu, 09 Jun 2016 10:01:15 +0100
Subject: [Python-ideas] Quick idea: defining variables from functions
 that take the variable name
In-Reply-To: <CADiSq7ddQ1TLSbS_=J4Srwg8RfA7DB2orh6=ULUNe+34W+TTiA@mail.gmail.com>
References: <CAP7+vJKCR_eaUVjZG+FNhBz=VhGjXXxCon3d=5af4bjwaeNCjg@mail.gmail.com>
 <CAGgTfkMjKLADk_wEnaYKc0Txk7HFYcv+gSnWB7LzHPu+VvKxYg@mail.gmail.com>
 <CADiSq7ddQ1TLSbS_=J4Srwg8RfA7DB2orh6=ULUNe+34W+TTiA@mail.gmail.com>
Message-ID: <2373496.dCS0Eprb4j@varric.chelsea.private>

Problem is to remove the duplication of 'name' in expressions like this:

	name = cls( 'name', arg, kw=val )

Would using a builtin that gives the name of the variable being assigned to be 
enough? A builtin like lhs_variable_name() - not a great name.

	name = cls( lhs_variable_name(), arg, kw=val )

This escapes from the problem of knowing the name space that
name is in and needing to understand the details of the expression.

It does not need a decorator with the complexity of needing a
break down of the RHS expression.

Now I do not need the 'name' to be the first argument:

	name = function( arg, kw=lhs_variable_name() )

I'm not sure what the behavior  should be for multiple LHS names.

	name1, name2 = cls( lhs_variable_name(), arg )

Is this an error and an exception is raised?
Always return the first name, 'name1'?
Does lhs_variable_name() return a tuple of ('name1','name2')?
Does lhs_variable_name() take an argument to choose?

Barry


From barry at barrys-emacs.org  Thu Jun  9 05:12:17 2016
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu, 09 Jun 2016 10:12:17 +0100
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608171016.074950ca@anarchist.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
Message-ID: <2564196.hmf4OFIST4@varric.chelsea.private>

On Wednesday 08 June 2016 17:10:16 Barry Warsaw wrote:
> On Jun 08, 2016, at 12:43 PM, Nikolaus Rath wrote:
> >This is so similar (in both semantics and syntax) to the
> >
> >Record = namedtuple($lhs, fields)
> >
> >proposal that I've made several times in that thread that I am a little
> >offended. Either you've not read that thread all that thorougly, or you
> >have a rather selective memory.
> 
> It's of course quite easy to miss a specific suggestion in the various
> millithreads on this list, or not quite connect the dots to see the
> similarities on the initial read.  Just be happy that great minds think
> alike. :)

Indeed I just posted along the same line, and missing both the @ and $lhs
before posting. I had thought to use a builtin to get the variable names.
I think the __LHS__ is the more general and powerful.

> On to the topic at hand, when I saw your suggestion and Steve's I
> immediately thought I'd like to see a special symbol like __LHS__, e.g.
> 
> Record = namedtuple(__LHS__, fields)
> 
> which would at least have the benefit of not introducing a special
> character, and one that could be confusing in other contexts (e.g. PEP 292
> strings).



> 
> Cheers,
> -Barry


From rob.cliffe at btinternet.com  Thu Jun  9 06:30:46 2016
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 9 Jun 2016 11:30:46 +0100
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <4fdaf368-b9c4-16b1-42d9-b87f274bc0ca@mrabarnett.plus.com>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
 <4fdaf368-b9c4-16b1-42d9-b87f274bc0ca@mrabarnett.plus.com>
Message-ID: <fd3deee6-9b25-be13-e6c3-41836066b63b@btinternet.com>



On 09/06/2016 03:29, MRAB wrote:
> On 2016-06-09 00:28, Barry Warsaw wrote:
>> On Jun 08, 2016, at 10:57 PM, Michael Selik wrote:
>>
>>> 3. @public for constants (as per Barry's email)
>>>
>>> I don't know what that last one was referring to. Do you mind 
>>> clarifying,
>>> Barry (or anyone else)?
>>
>> Sure!
>>
>> I propose that it's difficult to keep __all__ up to date. Witness the
>> plethora of bugs open on similar issues in the stdlib. Furthermore, the
>> definition of __all__ is usually far removed from the object it 
>> names.  And
>> it's not obvious when looking at the object whether it is intended to be
>> exported or not.
>>
>> @public solves these problems and I contend that its meaning is pretty
>> obvious, given that (unknown to me at the time) there have been at 
>> least two
>> other independent inventions of nearly exactly the same decorator.
>>
> [snip]
>
> A workaround is to use a comment:
>
> #@public
> SEVEN = 7
>
> and a script that parses the file and updates the __all__.
>
Another workaround which I use is to initialise __all__ to [] (or 
whatever), and then write
__all__.extend([ ..... ])
to add some thing(s) I've just defined.
Rob Cliffe

From phd at phdru.name  Thu Jun  9 07:04:52 2016
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 9 Jun 2016 13:04:52 +0200
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <fd3deee6-9b25-be13-e6c3-41836066b63b@btinternet.com>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
 <4fdaf368-b9c4-16b1-42d9-b87f274bc0ca@mrabarnett.plus.com>
 <fd3deee6-9b25-be13-e6c3-41836066b63b@btinternet.com>
Message-ID: <20160609110452.GA2328@phdru.name>

On Thu, Jun 09, 2016 at 11:30:46AM +0100, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
> On 09/06/2016 03:29, MRAB wrote:
> >On 2016-06-09 00:28, Barry Warsaw wrote:
> >>On Jun 08, 2016, at 10:57 PM, Michael Selik wrote:
> >>
> >>>3. @public for constants (as per Barry's email)
> >>>
> >>>I don't know what that last one was referring to. Do you mind
> >>>clarifying,
> >>>Barry (or anyone else)?
> >>
> >>Sure!
> >>
> >>I propose that it's difficult to keep __all__ up to date. Witness the
> >>plethora of bugs open on similar issues in the stdlib. Furthermore, the
> >>definition of __all__ is usually far removed from the object it names.
> >>And
> >>it's not obvious when looking at the object whether it is intended to be
> >>exported or not.
> >>
> >>@public solves these problems and I contend that its meaning is pretty
> >>obvious, given that (unknown to me at the time) there have been at least
> >>two
> >>other independent inventions of nearly exactly the same decorator.
> >>
> >[snip]
> >
> >A workaround is to use a comment:
> >
> >#@public
> >SEVEN = 7
> >
> >and a script that parses the file and updates the __all__.
> >
> Another workaround which I use is to initialise __all__ to [] (or whatever),
> and then write
> __all__.extend([ ..... ])
> to add some thing(s) I've just defined.

   @public is just a special syntax for that.

> Rob Cliffe

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From christoph at grothesque.org  Thu Jun  9 09:37:27 2016
From: christoph at grothesque.org (Christoph Groth)
Date: Thu, 09 Jun 2016 15:37:27 +0200
Subject: [Python-ideas] Statically checking purity of functions
Message-ID: <87eg86tre0.fsf@grothesque.org>

Hello,

As far as I can tell the following has not been discussed yet on 
the searchable internet.  It may seem that my idea concerns only 
mypy, but since "typing" is part of the stdlib, I believe it is 
appropriate for this mailing list.

Even though Python is a very dynamic language, in practice many 
routines (when called with arguments of the intended type) are 
pure in the functional programming sense, i.e. they have no 
observable side effects and the result reproducibly depends on the 
arguments.

I think there could be many interesting applications of knowing 
whether a function is pure or not.  Example: Long-running 
computations often need to be parallelized.  There are plenty of 
ways to parallelize Python code (think of concurrent.futures or 
ipyparallel) and most of them make implicit assumptions that some 
functions are pure.

Another example: if the purity of functions would be known, 
Jupyter notebooks could automatically deduce the side effects of 
code cells and thus calculate their dependencies on each 
other. Changes to a cell could thus invalidate other (dependent) 
cells.

With plain Python (without type annotations) there is no way to 
verify the purity of a function as has been well explained here: 
http://stackoverflow.com/a/31664102

But couldn't this be done with a static type checker like mypy? If 
mypy would track functions as pure or not pure, it should be able 
to verify the claimed purity of any new function.  For example, 
the following function would pass the test,

@pure
def add(a: int, b: int) -> int:
   return a + b

while the next one would produce an error

@pure
def add_random(a: int) -> int:
   return a + random.randrange(10)

because the static checker would know that random.randrange() is 
not pure (it modifies the state of the random module).

Am I overseeing any problems with the above?

Christoph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/2d2e1283/attachment.sig>

From p.f.moore at gmail.com  Thu Jun  9 10:14:49 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Jun 2016 15:14:49 +0100
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <87eg86tre0.fsf@grothesque.org>
References: <87eg86tre0.fsf@grothesque.org>
Message-ID: <CACac1F92wSCyv5BWGcHPssAHJojruGtnVL2o61F384N2cEqy7A@mail.gmail.com>

On 9 June 2016 at 14:37, Christoph Groth <christoph at grothesque.org> wrote:
> With plain Python (without type annotations) there is no way to verify the
> purity of a function as has been well explained here:
> http://stackoverflow.com/a/31664102
>
> But couldn't this be done with a static type checker like mypy? If mypy
> would track functions as pure or not pure, it should be able to verify the
> claimed purity of any new function.  For example, the following function
> would pass the test,
>
> @pure
> def add(a: int, b: int) -> int:
>   return a + b
>
> while the next one would produce an error
>
> @pure
> def add_random(a: int) -> int:
>   return a + random.randrange(10)
>
> because the static checker would know that random.randrange() is not pure
> (it modifies the state of the random module).
>
> Am I overseeing any problems with the above?

That sounds both reasonable and potentially useful, albeit of
relatively niche interest. It's quite similar to Perl's "taint checks"
that track whether data is affected by external input (although the
latter marks *data*, not functions).

I've not looked into mypy yet, so all I know about it is from skimming
some of the typing discussions here. But if it's possible to write a
plugin for mypy to do something like this, that would be pretty
awesome.

I'm not sure how much a capability like that would affect Python
itself, though. (In fact, I'd say it ought not to - being able to do
something like this without any changes to the language would be
ideal).

Paul

From cory at lukasa.co.uk  Thu Jun  9 10:22:29 2016
From: cory at lukasa.co.uk (Cory Benfield)
Date: Thu, 9 Jun 2016 15:22:29 +0100
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <87eg86tre0.fsf@grothesque.org>
References: <87eg86tre0.fsf@grothesque.org>
Message-ID: <D97655A3-47A1-4991-B1E2-DE7220E729C3@lukasa.co.uk>


> On 9 Jun 2016, at 14:37, Christoph Groth <christoph at grothesque.org> wrote:
> 
> But couldn't this be done with a static type checker like mypy? If mypy would track functions as pure or not pure, it should be able to verify the claimed purity of any new function.  For example, the following function would pass the test,
> 
> @pure
> def add(a: int, b: int) -> int:
>  return a + b

Here?s where this falls down (NB, the print is just a simple example of non-purity, but you can imagine that you can just do arbitrary code).

class SideEffectyInt(int):
    def __add__(self, other):
        if other == 3:
            print(?YAY")
        return super().__add__(other)

This SideEffectyInt() object is suitable for passing into your pure ?add? function, but has an impure add logic. I?m not sure how mypy could resolve this problem: I presume it?d need purity declarations for every possible function on every type *and* the ability to see all function invocations, which is pretty difficult given that Python?s dynamism makes it possible to invoke arbitrary functions.

So I don?t really see how this pure decorator could *enforce* purity.

Cory
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/65cbb55d/attachment.sig>

From robertc at robertcollins.net  Thu Jun  9 15:48:27 2016
From: robertc at robertcollins.net (Robert Collins)
Date: Fri, 10 Jun 2016 07:48:27 +1200
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <87eg86tre0.fsf@grothesque.org>
References: <87eg86tre0.fsf@grothesque.org>
Message-ID: <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>

On 10 June 2016 at 01:37, Christoph Groth <christoph at grothesque.org> wrote:
> Hello,
>
> As far as I can tell the following has not been discussed yet on the
> searchable internet.  It may seem that my idea concerns only mypy, but since
> "typing" is part of the stdlib, I believe it is appropriate for this mailing
> list.
>...
> But couldn't this be done with a static type checker like mypy? If mypy
> would track functions as pure or not pure, it should be able to verify the
> claimed purity of any new function.  For example, the following function
> would pass the test,
>
> @pure
> def add(a: int, b: int) -> int:
>   return a + b
>
> while the next one would produce an error
>
> @pure
> def add_random(a: int) -> int:
>   return a + random.randrange(10)
>
> because the static checker would know that random.randrange() is not pure
> (it modifies the state of the random module).
>
> Am I overseeing any problems with the above?

The problem, as Cory demonstrates, is that purity is transitive: you
have to know that all the runtime interactions are also pure, or your
function stops being pure.

external names like random in add_random can never be pure (because
any piece of impure code can change the module globals to rebind
random to something else). This means that any pure function would
have to accept as parameters everything it operates on, and you'd need
to error on any call to a pure function with impure arguments. It
would be an interesting exercise to see how it plays out in practice
though.

-Rob

From random832 at fastmail.com  Thu Jun  9 16:58:07 2016
From: random832 at fastmail.com (Random832)
Date: Thu, 09 Jun 2016 16:58:07 -0400
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
Message-ID: <1465505887.2056792.633188081.6CA4A04F@webmail.messagingengine.com>

On Thu, Jun 9, 2016, at 15:48, Robert Collins wrote:
> and you'd need
> to error on any call to a pure function with impure arguments.

What do you mean by impure arguments?

From joejev at gmail.com  Thu Jun  9 16:59:50 2016
From: joejev at gmail.com (Joseph Jevnik)
Date: Thu, 9 Jun 2016 16:59:50 -0400
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <1465505887.2056792.633188081.6CA4A04F@webmail.messagingengine.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 <1465505887.2056792.633188081.6CA4A04F@webmail.messagingengine.com>
Message-ID: <CAHGq92Vv5iGLc5k2AapKqSZ0fm9MaaxS9fUikLL_GVKW_PtcVg@mail.gmail.com>

I assume it means arguments with impure behavior, for example, an object
that has impure methods.

On Thu, Jun 9, 2016 at 4:58 PM, Random832 <random832 at fastmail.com> wrote:

> On Thu, Jun 9, 2016, at 15:48, Robert Collins wrote:
> > and you'd need
> > to error on any call to a pure function with impure arguments.
>
> What do you mean by impure arguments?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/03e08896/attachment.html>

From ncoghlan at gmail.com  Thu Jun  9 17:32:12 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2016 14:32:12 -0700
Subject: [Python-ideas] package management is not python-dev's problem
 :-(
In-Reply-To: <B1B534BB-1EC4-431F-B06A-24FC820AD433@stufft.io>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca> <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
 <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
 <5757FB2B.90604@thomas-guettler.de>
 <B1B534BB-1EC4-431F-B06A-24FC820AD433@stufft.io>
Message-ID: <CADiSq7ccAB4-JKwWavBTMXQbJvRcBFZC0vMa4+V-skVwJ-Cthg@mail.gmail.com>

On 8 June 2016 at 04:14, Donald Stufft <donald at stufft.io> wrote:
>
>> On Jun 8, 2016, at 7:02 AM, Thomas G?ttler <guettliml at thomas-guettler.de> wrote:
>>
>>>> I would love to see a package management that is easy to use and which
>>>> focuses on simple data structures.
>>>
>>> Also not python-dev's problem - that one falls on distutils-sig, PyPA
>>> and the PSF (but unfortunately is never going to be entirely simple
>>> given Python's broad scope of use).
>>
>> This statement make me sad. Why is this not python-dev's problem?
>>
>
> Historically python-dev hasn?t been involved much in it and almost all of the
> tooling for packaging is developed externally to Python itself. This is generally
> a strength, since packaging tools tend to work best when they?re not lied to the
> lifetime of the language itself.

To help put some specifics on that:

* for the core runtime and standard library, having features appear
only in new Python versions is normal and expected, and the related
update cycles are measured in months (maintenance releases) or years
(feature releases)
* for build and distribution tools, it's highly desirable to offer a
consistent feature set across all widely deployed Python versions
(since there is only one PyPI shared amongst all versions), and the
related update cycles tend to be measured in weeks (maintenance
releases) or months (feature releases)

As such, distutils-sig/PyPA maintain Python 2/3 compatible tooling on
behalf of not only CPython, but also PyPy, Jython, IronPython, and
other sufficiently compatible alternate implementations. There are
some *people* that participate in both communities (including folks
that are both PyPA contributors and CPython core developers), and
there are some issues that remain specifically python-dev's
responsibility (such as providing recent versions of pip by default as
part of CPython installations), but the day-to-day design discussions
are separate.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From christoph at grothesque.org  Thu Jun  9 17:59:32 2016
From: christoph at grothesque.org (Christoph Groth)
Date: Thu, 09 Jun 2016 23:59:32 +0200
Subject: [Python-ideas] Statically checking purity of functions
References: <87eg86tre0.fsf@grothesque.org>
 <D97655A3-47A1-4991-B1E2-DE7220E729C3@lukasa.co.uk>
Message-ID: <87twh2dnwb.fsf@grothesque.org>

Cory Benfield wrote:

> Here?s where this falls down (NB, the print is just a simple 
> example of non-purity, but you can imagine that you can just do 
> arbitrary code).
>
> class SideEffectyInt(int):
>     def __add__(self, other):
>         if other == 3:
>             print(?YAY")
>         return super().__add__(other)

Of course, an instance of SideEffectInt is also an instance of int 
and as such mypy will not complain when it is passed to the "add" 
function.

However, using the above technique any type checking by mypy can 
be circumvented.  Mypy will happily nod through the following 
program:

class FloatingInt(int):
    def __add__(self, other):
        return float(super().__add__(other))

def add(a: int, b: int) -> int:
    return a + b

print(add(FloatingInt(1), 2))

However, if we change the above class definition to

class FloatingInt(int):
    def __add__(self, other) -> float:
        return float(super().__add__(other))

mypy will start to complain with the following error:

test.py:2: error: Return type of "__add__" incompatible with 
supertype "int"

Now, mypy could be probably more strict and flag already the first 
definition of FloatingInt.__add__() as an error, because it knows 
(as the above error message shows) what its return type should be 
From inheritance.  It also knows that float()  produces a float, 
since it does not accept

def foo() -> int:
    return float(5)

I guess it's a choice of mypy not to complain about bits of code 
that don't use type declarations.

In an exactly analogous way, mypy could detect that calling 
print() in SideEffectlyInt.__add__ is incompatible with the 
(supposedly declared) purity of int.__add__() if SideEffectlyInt 
was declared as:

class SideEffectyInt(int):
    @pure
    def __add__(self, other):
        if other == 3:
            print(?YAY")
        return super().__add__(other)

And just like before, if mypy was more strict, it could detect the 
purity-breaking of SideEffectyInt.__add__() from inheritance.

> This SideEffectyInt() object is suitable for passing into your 
> pure ?add? function, but has an impure add logic. I?m not sure 
> how mypy could resolve this problem: I presume it?d need purity 
> declarations for every possible function on every type *and* the 
> ability to see all function invocations, which is pretty 
> difficult given that Python?s dynamism makes it possible to 
> invoke arbitrary functions.

This is correct, but I believe your statement remains correct when 
one replaces "purity" by "typing": In order to verify with 
certainty that the typing is correct for a function, mypy would 
have to know the typing declaration of every function that is 
called inside it.

> So I don?t really see how this pure decorator could *enforce* 
> purity.

It seems to me that purity could be enforced to the same degree 
that correct typing can be enforced.  In other words, the goal of 
mypy seems not to be to prove that typing of a program is 
consistent (Python is too dynamic for that), but rather to help 
finding possible errors.  It seems to me that the same service 
could be provided for purity.  That should be good enough for many 
applications.

In that spirit, perhaps it would make more sense to assume purity 
by default if unspecified (just like mypy assumes type correctness 
when no typing information is present), and have both "pure" and 
"impure" (dirty? ;-) decorators.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/fdc01fcd/attachment-0001.sig>

From egregius313 at gmail.com  Thu Jun  9 18:10:24 2016
From: egregius313 at gmail.com (Ed Minnix)
Date: Thu, 9 Jun 2016 18:10:24 -0400
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAHGq92Vv5iGLc5k2AapKqSZ0fm9MaaxS9fUikLL_GVKW_PtcVg@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 <1465505887.2056792.633188081.6CA4A04F@webmail.messagingengine.com>
 <CAHGq92Vv5iGLc5k2AapKqSZ0fm9MaaxS9fUikLL_GVKW_PtcVg@mail.gmail.com>
Message-ID: <960E2B5B-120D-42A0-933A-FF15A48D7AA9@gmail.com>

Perhaps this would work better with something like property based testing of some sort. You would take a function and say for the allowed types, run a memoization series and check if the return value is the same every time for the arguments.

E.g., if you know the function can only take subclasses of numbers.Number whose arithmetic operators are pure, the type checker / test suite can verify.

Code such as:

>>> def add(a: int, b: int) -> int:
...    return a + b 
...

would allow for a test:

@given(integers(), integers())
def add_testing(a, b):
    result_1 = add(a,b)

    assert isinstance(result_1, int)

    for _ in range(100):
        assert result_1 == add(a,b)


- Ed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/4882b60a/attachment.html>

From egregius313 at gmail.com  Thu Jun  9 18:10:49 2016
From: egregius313 at gmail.com (Ed Minnix)
Date: Thu, 9 Jun 2016 18:10:49 -0400
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAHGq92Vv5iGLc5k2AapKqSZ0fm9MaaxS9fUikLL_GVKW_PtcVg@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 <1465505887.2056792.633188081.6CA4A04F@webmail.messagingengine.com>
 <CAHGq92Vv5iGLc5k2AapKqSZ0fm9MaaxS9fUikLL_GVKW_PtcVg@mail.gmail.com>
Message-ID: <7D944FD8-C11A-4287-A07E-5BB38397CDDD@gmail.com>

Perhaps this would work better with something like property based testing of some sort. You would take a function and say for the allowed types, run a memoization series and check if the return value is the same every time for the arguments.

E.g., if you know the function can only take subclasses of numbers.Number whose arithmetic operators are pure, the type checker / test suite can verify.

Code such as:

>>> def add(a: int, b: int) -> int:
...    return a + b 
...

would allow for a test:

@given(integers(), integers())
def add_testing(a, b):
    result_1 = add(a,b)

    assert isinstance(result_1, int)

    for _ in range(100):
        assert result_1 == add(a,b)


- Ed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/162b0de0/attachment.html>

From ncoghlan at gmail.com  Thu Jun  9 18:20:41 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2016 15:20:41 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
Message-ID: <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>

On 8 June 2016 at 00:34, Pavol Lisy <pavol.lisy at gmail.com> wrote:
> 2016-06-05 7:14 GMT+02:00, Nick Coghlan <ncoghlan at gmail.com>:
>
>> Red Hat's Python maintenance team are working on a more explicitly
>> minimalistic porting guide based on their experiences porting Fedora
>> components, sometimes in the context of upstream projects that are more
>> interested in keeping Python 2.4 support than they are in Python 3:
>> http://portingguide.readthedocs.io/en/latest/
>
> do you have some drafts how to write (new) code supporting python from
> 2.4 - 2.7 with intention to port it to python3 in future?

Unfortunately, the only practical solution we've found for that case
is to fork the code base until the maintainers of the original project
are willing to raise the minimum version requirement to Python 2.6:
http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-and-lower

While it's theoretically possible to support 2.4+ and 3.x in the same
code base, it's painful in practice due to the lack of the forward
compatibility features added in Python 2.6 (such as the Python 3 style
syntax for name binding in except clauses)

For applications (rather than libraries), an alternate recommendation
is to bundle a Python 2.7 runtime on platforms where the system Python
is ancient, but whether or not that's a suitable solution will depend
a great deal on the specifics of the application.

My own view is that it doesn't make sense for *new* software (or new
versions of existing software) being written in 2016 to support a
version of Python that has been end-of-life for more than 7 years
(last release December 2008, first release November 2004), just to
allow folks to continue avoiding upgrading a base operating system
that is itself now two major releases behind (assuming your interest
in Python 2.4 support is driven by RHEL/CentOS 5 users).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From carl at oddbird.net  Thu Jun  9 18:35:56 2016
From: carl at oddbird.net (Carl Meyer)
Date: Thu, 9 Jun 2016 16:35:56 -0600
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
 <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
Message-ID: <5759EF4C.9000209@oddbird.net>

On 06/09/2016 04:20 PM, Nick Coghlan wrote:
> On 8 June 2016 at 00:34, Pavol Lisy <pavol.lisy at gmail.com> wrote:
>> 2016-06-05 7:14 GMT+02:00, Nick Coghlan <ncoghlan at gmail.com>:
>>
>>> Red Hat's Python maintenance team are working on a more explicitly
>>> minimalistic porting guide based on their experiences porting Fedora
>>> components, sometimes in the context of upstream projects that are more
>>> interested in keeping Python 2.4 support than they are in Python 3:
>>> http://portingguide.readthedocs.io/en/latest/
>>
>> do you have some drafts how to write (new) code supporting python from
>> 2.4 - 2.7 with intention to port it to python3 in future?
> 
> Unfortunately, the only practical solution we've found for that case
> is to fork the code base until the maintainers of the original project
> are willing to raise the minimum version requirement to Python 2.6:
> http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-and-lower
> 
> While it's theoretically possible to support 2.4+ and 3.x in the same
> code base, it's painful in practice due to the lack of the forward
> compatibility features added in Python 2.6 (such as the Python 3 style
> syntax for name binding in except clauses)

While I agree that it's by far the best option to raise the minimum
supported version to 2.6 (or even 2.7) before trying to straddle to 3,
we shouldn't make it sound impossible to straddle all the way back to
2.4. It's painful, for sure, but every issue is surmountable if you're
dedicated enough. E.g. if you need to capture the exception in an
`except` clause, you use `sys.exc_info()` instead. Pip did that for
years :-)

Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/5c5561ec/attachment-0001.sig>

From ncoghlan at gmail.com  Thu Jun  9 20:24:14 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2016 17:24:14 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <5759EF4C.9000209@oddbird.net>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
 <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
 <5759EF4C.9000209@oddbird.net>
Message-ID: <CADiSq7ce9yt=Y5qpAzEJB3a7u+2KPmnp6c+7q+=9AcXbN4g70A@mail.gmail.com>

On 9 June 2016 at 15:35, Carl Meyer <carl at oddbird.net> wrote:
> On 06/09/2016 04:20 PM, Nick Coghlan wrote:
>> While it's theoretically possible to support 2.4+ and 3.x in the same
>> code base, it's painful in practice due to the lack of the forward
>> compatibility features added in Python 2.6 (such as the Python 3 style
>> syntax for name binding in except clauses)
>
> While I agree that it's by far the best option to raise the minimum
> supported version to 2.6 (or even 2.7) before trying to straddle to 3,
> we shouldn't make it sound impossible to straddle all the way back to
> 2.4. It's painful, for sure, but every issue is surmountable if you're
> dedicated enough. E.g. if you need to capture the exception in an
> `except` clause, you use `sys.exc_info()` instead. Pip did that for
> years :-)

Right, but there's also a time related aspect to this advice: most
folks enthusiastic enough about Python 3 to adopt those kinds of
invasive measures already support it, and Python 2.4 is now several
years older than it was when folks first started adding Python 3
support to their projects (even RHEL 5 is reaching the point where Red
Hat starts requiring the Extended Life Support addon for ongoing
support beyond March 2017).

That means that for anyone that's still holding off on porting due to
a desire to retain Python 2.4 compatibility, "wait 12 months or so,
and then reassess our need to support Python 2.4" is likely to make
more sense than putting a lot of work into supporting Python 2.4 and
3.x in the same code base. Similarly, maintaining a Python 3
compatibility changeset with a reasonable hope of being able to merge
it back into the parent project sometime in the next year is a
significantly more attractive option than creating and maintaining a
full fork of the project indefinitely.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From Nikolaus at rath.org  Thu Jun  9 23:01:11 2016
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Thu, 09 Jun 2016 20:01:11 -0700
Subject: [Python-ideas] Smoothing transition to Python 3
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
 <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
 <5759EF4C.9000209@oddbird.net>
Message-ID: <87eg85ycg8.fsf@vostro.rath.org>

On Jun 09 2016, Carl Meyer <carl-faUO1KlGllLR7s880joybQ at public.gmane.org> wrote:
> On 06/09/2016 04:20 PM, Nick Coghlan wrote:
>> On 8 June 2016 at 00:34, Pavol Lisy <pavol.lisy-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
>>> 2016-06-05 7:14 GMT+02:00, Nick Coghlan <ncoghlan-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>:
>>>
>>>> Red Hat's Python maintenance team are working on a more explicitly
>>>> minimalistic porting guide based on their experiences porting Fedora
>>>> components, sometimes in the context of upstream projects that are more
>>>> interested in keeping Python 2.4 support than they are in Python 3:
>>>> http://portingguide.readthedocs.io/en/latest/
>>>
>>> do you have some drafts how to write (new) code supporting python from
>>> 2.4 - 2.7 with intention to port it to python3 in future?
>> 
>> Unfortunately, the only practical solution we've found for that case
>> is to fork the code base until the maintainers of the original project
>> are willing to raise the minimum version requirement to Python 2.6:
>> http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-and-lower
>> 
>> While it's theoretically possible to support 2.4+ and 3.x in the same
>> code base, it's painful in practice due to the lack of the forward
>> compatibility features added in Python 2.6 (such as the Python 3 style
>> syntax for name binding in except clauses)
>
> While I agree that it's by far the best option to raise the minimum
> supported version to 2.6 (or even 2.7) before trying to straddle to 3,
> we shouldn't make it sound impossible to straddle all the way back to
> 2.4. It's painful, for sure, but every issue is surmountable if you're
> dedicated enough.

Sure, e.g. you could implement a Python 3 runtime in Python 2.4. It's
Turing-complete after all.

SCNR.

Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             ?Time flies like an arrow, fruit flies like a Banana.?

From jelle.zijlstra at gmail.com  Thu Jun  9 23:47:52 2016
From: jelle.zijlstra at gmail.com (Jelle Zijlstra)
Date: Thu, 9 Jun 2016 20:47:52 -0700
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608171016.074950ca@anarchist.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
Message-ID: <CAFp3-p8zfe-LfiJucd5jSesECtG5JxFL7bynfwCBko9gNaNZwQ@mail.gmail.com>

2016-06-08 14:10 GMT-07:00 Barry Warsaw <barry at python.org>:

> On Jun 08, 2016, at 12:43 PM, Nikolaus Rath wrote:
>
> >This is so similar (in both semantics and syntax) to the
> >
> >Record = namedtuple($lhs, fields)
> >
> >proposal that I've made several times in that thread that I am a little
> >offended. Either you've not read that thread all that thorougly, or you
> >have a rather selective memory.
>
> It's of course quite easy to miss a specific suggestion in the various
> millithreads on this list, or not quite connect the dots to see the
> similarities on the initial read.  Just be happy that great minds think
> alike. :)
>
> On to the topic at hand, when I saw your suggestion and Steve's I
> immediately
> thought I'd like to see a special symbol like __LHS__, e.g.
>
> Record = namedtuple(__LHS__, fields)
>
> which would at least have the benefit of not introducing a special
> character,
> and one that could be confusing in other contexts (e.g. PEP 292 strings).
>

I implemented Barry's variation of this idea at
https://github.com/JelleZijlstra/cpython/tree/lhs in case somebody wants to
play with it.

>>> Record = namedtuple(__LHS__, 'a b')

>>> Record
<class '__main__.Record' at 0x7fb12af03028>


>
> Cheers,
> -Barry
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160609/9e0266b4/attachment.html>

From pavol.lisy at gmail.com  Fri Jun 10 02:29:05 2016
From: pavol.lisy at gmail.com (Pavol Lisy)
Date: Fri, 10 Jun 2016 08:29:05 +0200
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CADiSq7ce9yt=Y5qpAzEJB3a7u+2KPmnp6c+7q+=9AcXbN4g70A@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
 <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
 <5759EF4C.9000209@oddbird.net>
 <CADiSq7ce9yt=Y5qpAzEJB3a7u+2KPmnp6c+7q+=9AcXbN4g70A@mail.gmail.com>
Message-ID: <CABDEq2=QgLASVLkFA4KfSjUu-rr=rEZJYojDZHsy+rrtD6xiWA@mail.gmail.com>

2016-06-10 2:24 GMT+02:00, Nick Coghlan <ncoghlan at gmail.com>:

> Right, but there's also a time related aspect to this advice: most
> folks enthusiastic enough about Python 3 to adopt those kinds of
> invasive measures already support it, and Python 2.4 is now several
> years older than it was when folks first started adding Python 3
> support to their projects (even RHEL 5 is reaching the point where Red
> Hat starts requiring the Extended Life Support addon for ongoing
> support beyond March 2017).

If I am not wrong then RHEL5 is stil about 2.4.

2016-06-10 0:20 GMT+02:00, Nick Coghlan <ncoghlan at gmail.com>:

> Unfortunately, the only practical solution we've found for that case
> is to fork the code base until the maintainers of the original project
> are willing to raise the minimum version requirement to Python 2.6:
> http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-and-lower

If you need support 2.4 it is highly probably because very
conservative customer or environement. Conservative environment could
not approve to raise higher version. :/

> While it's theoretically possible to support 2.4+ and 3.x in the same
> code base, it's painful in practice due to the lack of the forward
> compatibility features added in Python 2.6 (such as the Python 3 style
> syntax for name binding in except clauses)

It is why I like to see some guidlines how to do it. One practical
possibility is (sorry for blasphemy) to (temporary!) drop support for
python3.

We need to understand that it is not only "fight" for python3 but also
"fight" for python at all. So people who "want" to write code in 2.4
are not enemies.

---

Btw. six library (now in version 1.10) dropped support for python 2.4
in version 1.5. and I am surprised. (They/we had probably to rename it
to seven_and_half too :) Interesting for me is reason why they/we did
it -> "Removed support for Python 2.4. This is because py.test no
longer supports 2.4."

From p.f.moore at gmail.com  Fri Jun 10 04:47:24 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Jun 2016 09:47:24 +0100
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CABDEq2=QgLASVLkFA4KfSjUu-rr=rEZJYojDZHsy+rrtD6xiWA@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
 <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
 <5759EF4C.9000209@oddbird.net>
 <CADiSq7ce9yt=Y5qpAzEJB3a7u+2KPmnp6c+7q+=9AcXbN4g70A@mail.gmail.com>
 <CABDEq2=QgLASVLkFA4KfSjUu-rr=rEZJYojDZHsy+rrtD6xiWA@mail.gmail.com>
Message-ID: <CACac1F-pvF3cWFVRgEzPyUh13gZuRBhJqB7p_Zoqad9d_+i4DQ@mail.gmail.com>

On 10 June 2016 at 07:29, Pavol Lisy <pavol.lisy at gmail.com> wrote:
> It is why I like to see some guidlines how to do it. One practical
> possibility is (sorry for blasphemy) to (temporary!) drop support for
> python3.

That's certainly an option. If you're writing an in-house application,
then it's entirely reasonable to decide not to support a version of
Python you don't plan to go to. If you're writing in-house libraries,
saying "our company is not yet ready to support Python 3 for our
internal code" is perfectly reasonable.

If you're publishing code for external use, you have to decide whether
alienating customers who want Python 3 support is worth it - that's
again your decision. Only you know how many such customers you have.
There is no requirement on anyone to support Python 3. However,
there's a growing trend [1] to think of public software that doesn't
support Python 3 as stagnant, and for people to look for "more up to
date" replacements. That's something you have to consider when
deciding not to support Python 3.

Paul

[1] We can debate endlessly on how fast the growth is. But even
Twisted, the most highly visible project that I know of to struggle
with supporting Python 3, has felt enough pressure to invest massive
amounts of time in the porting effort.

From christoph at grothesque.org  Fri Jun 10 06:10:37 2016
From: christoph at grothesque.org (Christoph Groth)
Date: Fri, 10 Jun 2016 12:10:37 +0200
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CACac1F92wSCyv5BWGcHPssAHJojruGtnVL2o61F384N2cEqy7A@mail.gmail.com>
 (Paul Moore's message of "Thu, 9 Jun 2016 15:14:49 +0100")
References: <87eg86tre0.fsf@grothesque.org>
 <CACac1F92wSCyv5BWGcHPssAHJojruGtnVL2o61F384N2cEqy7A@mail.gmail.com>
Message-ID: <87k2hxe4ma.fsf@grothesque.org>

Paul Moore wrote:

> It's quite similar to Perl's "taint checks" that track whether 
> data is affected by external input (although the latter marks 
> *data*, not functions).

Indeed!  Thanks for the pointer, I haven't heard of taint checks 
before.

> I've not looked into mypy yet, so all I know about it is from 
> skimming some of the typing discussions here. But if it's 
> possible to write a plugin for mypy to do something like this, 
> that would be pretty awesome.
>
> I'm not sure how much a capability like that would affect Python 
> itself, though. (In fact, I'd say it ought not to - being able 
> to do something like this without any changes to the language 
> would be ideal).

Testing for purity, like testing for type correctness, could be 
kept separate from the language.  (Although I have the gut feeling 
that one day there will be a switch to CPython that enables 
mypy-like testing at runtime.)  There is one exception: there must 
be some syntax to declare (non-)purity.

Function annotations (used by mypy for type checking) set the 
__annotations__ attribute.  In the same way there could be, for 
example, a __pure__ attribute that, if present, can be True or 
False.  It could be, for example, set by the decorators 
"typing.pure" and "typing.impure".

The problem with using decorators, as compared to having a 
dedicated syntax, is that the name of a decorator is just a name, 
and can be assigned some different value. But I do not see a clear 
majority for adding syntax to Python for declaring purity right 
now... ;-)

Perhaps this is going too far, but I note that there are other 
traits of functions that could be tracked as well.  For example, 
one could track whether a function is potentially blocking.  If 
such alternative traits are judged important enough (I do not 
consider "blocking" important enough, it's just an example of the 
concept), one could also store several such traits in a __traits__ 
attribute.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160610/4443549a/attachment.sig>

From srkunze at mail.de  Fri Jun 10 06:16:15 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Fri, 10 Jun 2016 12:16:15 +0200
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
Message-ID: <575A936F.8020601@mail.de>

On 09.06.2016 21:48, Robert Collins wrote:
> The problem, as Cory demonstrates, is that purity is transitive: you
> have to know that all the runtime interactions are also pure, or your
> function stops being pure.

That reminds me of the discussion in the other thread about (im-)mutability.

>
> external names like random in add_random can never be pure (because
> any piece of impure code can change the module globals to rebind
> random to something else). This means that any pure function would
> have to accept as parameters everything it operates on, and you'd need
> to error on any call to a pure function with impure arguments. It
> would be an interesting exercise to see how it plays out in practice
> though.

From christoph at grothesque.org  Fri Jun 10 06:57:34 2016
From: christoph at grothesque.org (Christoph Groth)
Date: Fri, 10 Jun 2016 12:57:34 +0200
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 (Robert Collins's message of "Fri, 10 Jun 2016 07:48:27 +1200")
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
Message-ID: <87d1npe2g1.fsf@grothesque.org>

Robert Collins wrote:

> external names like random in add_random can never be pure 
> (because any piece of impure code can change the module globals 
> to rebind random to something else).

True, but when checking for type correctness one faces exactly the 
same problem.  Consider the following script.  Uncommenting any of 
the commented-out blocks will make it fail in the same way, but 
mypy will only detect the first failure.

That's already not bad at all, since most of such renaming of 
global variables will typically happen at global scope. 
Sufficiently insidious monkey-patching can never be detected by a 
linter that only looks at the Python source. (Hey, it could happen 
inside a C extension module!)

import math

def bad_sin(x: float) -> str:
    return str(x)

### Mypy will detect this as a problem:
# math.sin = bad_sin

### Mypy will not mind the following:
# def do_evil():
#     math.sin = bad_sin
# do_evil()

def sin_squared(x: float) -> float:
    return math.sin(x)**2

print(sin_squared(math.pi / 6))

> This means that any pure function would have to accept as 
> parameters everything it operates on, and you'd need to error on 
> any call to a pure function with impure arguments.

You mean because one can never be sure about the purity of global 
variables?  That's true, but exactly to the same extent that one 
cannot be sure about the typing annotations of global variables.

I believe that such "weak" checking for purity would still be 
useful, just as the equally weak verification of typing by mypy is 
useful.

Christoph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160610/4c6fe6c9/attachment.sig>

From barry at python.org  Fri Jun 10 12:42:13 2016
From: barry at python.org (Barry Warsaw)
Date: Fri, 10 Jun 2016 12:42:13 -0400
Subject: [Python-ideas] Proposal for special name and qualname symbols
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
 <5758B137.6050608@stoneleaf.us>
 <58455581-8ac1-2638-5570-657ade55e5fc@trueblade.com>
Message-ID: <20160610124213.4dc2d506@anarchist.wooz.org>

On Jun 08, 2016, at 09:30 PM, Eric V. Smith wrote:

>SEVEN = public(__LHS__, 7)
>a_bar = public(__LHS__, Bar())

Yes, if we had __LHS__ that's how I'd write it.

>Although this might be a different version of public (or not!).

It would be, because public() wouldn't then need to poke values into the
module globals.  It would just have to return the second argument and put the
first argument in __all__.  Tools like pyflakes would stop complaining too.
Win!

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160610/94180ec5/attachment.sig>

From tjreedy at udel.edu  Fri Jun 10 13:43:26 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 10 Jun 2016 13:43:26 -0400
Subject: [Python-ideas] Smoothing transition to Python 3
In-Reply-To: <CACac1F-pvF3cWFVRgEzPyUh13gZuRBhJqB7p_Zoqad9d_+i4DQ@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <CADiSq7fkg8EgfJ5gF034077-DjrSxrqkyro2Pt0eoBbKyr1hDA@mail.gmail.com>
 <20160604081239.GA20628@python.ca>
 <CADiSq7cj-shaHEnXkxrZFB5J-Kh_rZc82gA5Fn2Mqth1oQod4g@mail.gmail.com>
 <CABDEq2=dTuZOOrHv_x9oqvTgVSyxDg4-Yh0LRr2-=zmXTiEpNw@mail.gmail.com>
 <CADiSq7fZhUNo8heDG2j+VsQKd1wXF4=DJrEnd9j30sQOohVE_A@mail.gmail.com>
 <5759EF4C.9000209@oddbird.net>
 <CADiSq7ce9yt=Y5qpAzEJB3a7u+2KPmnp6c+7q+=9AcXbN4g70A@mail.gmail.com>
 <CABDEq2=QgLASVLkFA4KfSjUu-rr=rEZJYojDZHsy+rrtD6xiWA@mail.gmail.com>
 <CACac1F-pvF3cWFVRgEzPyUh13gZuRBhJqB7p_Zoqad9d_+i4DQ@mail.gmail.com>
Message-ID: <njeu80$5ug$1@ger.gmane.org>

On 6/10/2016 4:47 AM, Paul Moore wrote:
> On 10 June 2016 at 07:29, Pavol Lisy <pavol.lisy at gmail.com> wrote:
>> It is why I like to see some guidlines how to do it. One practical
>> possibility is (sorry for blasphemy) to (temporary!) drop support for
>> python3.
>
> That's certainly an option. If you're writing an in-house application,
> then it's entirely reasonable to decide not to support a version of
> Python you don't plan to go to. If you're writing in-house libraries,
> saying "our company is not yet ready to support Python 3 for our
> internal code" is perfectly reasonable.
>
> If you're publishing code for external use, you have to decide whether
> alienating customers who want Python 3 support is worth it - that's
> again your decision. Only you know how many such customers you have.
> There is no requirement on anyone to support Python 3.

Or to support Python 2 ;-).  For a standalone application, it almost 
does not matter which Python it runs on, as long as the needed version 
is available for the machine, and 2.7 and some 3.x are available for 
most anything now.

  However,
> there's a growing trend [1] to think of public software that doesn't
> support Python 3 as stagnant, and for people to look for "more up to
> date" replacements. That's something you have to consider when
> deciding not to support Python 3.
>
> Paul
>
> [1] We can debate endlessly on how fast the growth is. But even
> Twisted, the most highly visible project that I know of to struggle
> with supporting Python 3, has felt enough pressure to invest massive
> amounts of time in the porting effort.

-- 
Terry Jan Reedy


From leewangzhong+python at gmail.com  Fri Jun 10 15:31:20 2016
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Fri, 10 Jun 2016 15:31:20 -0400
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160610124213.4dc2d506@anarchist.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <CAGgTfkMUGJzJw2JzpWh8bGS80UhpD54jd=w-92CpP1BLSDWmOA@mail.gmail.com>
 <20160608192829.22a444b3@subdivisions.wooz.org>
 <5758B137.6050608@stoneleaf.us>
 <58455581-8ac1-2638-5570-657ade55e5fc@trueblade.com>
 <20160610124213.4dc2d506@anarchist.wooz.org>
Message-ID: <CAB_e7iz0CKj8ii11F9C4CJ7R91Qk4ZAOZw9MJq=G+XHbMuW6VA@mail.gmail.com>

(API bikeshedding follows.)

In `@public` => `public(__LHS__, value)`, I'm a little bothered by
__LHS__ being the first argument. The first argument in a regular
decorator is the value.

Instead, how about this?

    SEVEN = public(7, <some_keyword>=__LHS__)

... where the keyword is, for example, `name` or `target`. I thought
about `__name__`, too, but that sort of implies the name will be
attached to the object, which is true for `namedtuple` but not for
`public`.

Now the problem with that is, the assignment is now reversed.

(I'm not so bothered, somehow, that __LHS__ is magically substituted in.)

----

By the way, should these LHSs also have API?
- `indexable[index] = value`
- `obj.attrname = value`
- `first, *middle, last = value`
- ... and more, if we get unpacking generalizations like dict
unpacking and pattern-match.
(`obj` and `indexable` can be expressions, but that's not an issue.)

How should these be passed? Or should they just be disallowed?

Here's an ugly straw man proposal, using `decorator(name, value)`:
- `decorator((indexable.__setitem__, index), value)`
- `decorator((obj.__setattr__, attrname), value)`
- `decorator(['first', ..., 'middle', 'last'], value)`

Another (ugly) possibility, using kw params and `functools.partial`:
- `decorator(value, target=partial(indexable.__setitem__, index))`
- `decorator(value, target=partial(obj.__setattr__, attrname))`
- `decorator(value, name=['first', 'middle', 'last'], splat=1)`

Possible usecases:
???

On Fri, Jun 10, 2016 at 12:42 PM, Barry Warsaw <barry at python.org> wrote:
> On Jun 08, 2016, at 09:30 PM, Eric V. Smith wrote:
>
>>SEVEN = public(__LHS__, 7)
>>a_bar = public(__LHS__, Bar())
>
> Yes, if we had __LHS__ that's how I'd write it.
>
>>Although this might be a different version of public (or not!).
>
> It would be, because public() wouldn't then need to poke values into the
> module globals.  It would just have to return the second argument and put the
> first argument in __all__.  Tools like pyflakes would stop complaining too.
> Win!
>
> -Barry
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From barry at barrys-emacs.org  Sat Jun 11 12:00:42 2016
From: barry at barrys-emacs.org (Barry Scott)
Date: Sat, 11 Jun 2016 17:00:42 +0100
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160608171016.074950ca@anarchist.wooz.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
Message-ID: <D7585789-FF8E-4CBB-A9B7-4D31E28CB334@barrys-emacs.org>

What is the behavour in this case:

   mydict[ index ] = Record( __LHS__ )

I expect ther are others to consider like +=.

I suspose if there is reasonable semantics raise SyntaxError?

Barry


Barry


> On 8 Jun 2016, at 22:10, Barry Warsaw <barry at python.org> wrote:
> 
>> On Jun 08, 2016, at 12:43 PM, Nikolaus Rath wrote:
>> 
>> This is so similar (in both semantics and syntax) to the
>> 
>> Record = namedtuple($lhs, fields)
>> 
>> proposal that I've made several times in that thread that I am a little
>> offended. Either you've not read that thread all that thorougly, or you
>> have a rather selective memory.
> 
> It's of course quite easy to miss a specific suggestion in the various
> millithreads on this list, or not quite connect the dots to see the
> similarities on the initial read.  Just be happy that great minds think
> alike. :)
> 
> On to the topic at hand, when I saw your suggestion and Steve's I immediately
> thought I'd like to see a special symbol like __LHS__, e.g.
> 
> Record = namedtuple(__LHS__, fields)
> 
> which would at least have the benefit of not introducing a special character,
> and one that could be confusing in other contexts (e.g. PEP 292 strings).
> 
> Cheers,
> -Barry
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From jab at math.brown.edu  Sun Jun 12 19:52:28 2016
From: jab at math.brown.edu (jab at math.brown.edu)
Date: Sun, 12 Jun 2016 19:52:28 -0400
Subject: [Python-ideas] bool.from_config_str()
Message-ID: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>

Dear Python-Ideas,

(New here... hope this idea contributes something!)

Several times I've had to implement a silly function that converts the
odd environment variable (or some other value from the outside world
that, perforce, comes in as a string) to the boolean it actually
represents.

I say "silly" because, for other commonly-needed types of primitive
values, we have concise, idiomatic ways to convert them from strings,
leaving no need for wheel reinvention:

int(os.getenv('NUM_FJORDS'))
float(os.getenv('PRICE_LIMBURGER'))
etc.

So I thought it might be nice if we could do something like this for
booleans, too:

Python 3.6.0a1+ (default:0b18f7d262cc+, Jun 12 2016, 18:21:54)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> bool.from_config_str('false')
False
>>> bool.from_config_str('False')
False
>>> bool.from_config_str('True')
True
>>> bool.from_config_str('true')
True
>>> bool.from_config_str('0')
False
>>> bool.from_config_str('')
False
>>> bool.from_config_str('1')
True


I seized the opportunity to work up my first (tiny) patch for CPython
which implements this, along with tests (please see attached patch,
which applies cleanly against current tip).

Is there any interest in this? If so, I'd be happy to make it work for
bytes too, or to make any other changes that would help get it landed.

Thanks for your consideration, and looking forward to your feedback.

Josh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bool_from_config_str.patch
Type: application/octet-stream
Size: 3402 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160612/c68d50a8/attachment.obj>

From michael.selik at gmail.com  Sun Jun 12 21:00:57 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Mon, 13 Jun 2016 01:00:57 +0000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <CAGgTfkNmGXtqhKHr6Uyq865npMR2SqHbsT+vRkfTfh4mHO0aNg@mail.gmail.com>

On Sun, Jun 12, 2016 at 7:53 PM <jab at math.brown.edu> wrote:

> >>> bool.from_config_str('false')
> False
>

Why put the word "config" in the method name rather than just
``bool.from_str``?

Why not just check for the truth values?

    config = 'False'
    try:
        value = bool(float(config))
    except (TypeError, ValueError):
        value = config.casefold() == 'true'

I agree it's frustrating to have the 4 lines. Perhaps this is yet another
case that would be improved by exception-catching expressions (PEP 463)?

Unless those 4 lines occur very frequently, this feels like a bit too
special-case for a bool method. It feels more like it's part of a library
offering environment or config utilities.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/ffa6e630/attachment.html>

From tjreedy at udel.edu  Sun Jun 12 23:10:05 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 12 Jun 2016 23:10:05 -0400
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <njl86i$ql$1@ger.gmane.org>

On 6/12/2016 7:52 PM, jab at math.brown.edu wrote:
> Dear Python-Ideas,
>
> (New here... hope this idea contributes something!)
>
> Several times I've had to implement a silly function that converts the
> odd environment variable (or some other value from the outside world
> that, perforce, comes in as a string) to the boolean it actually
> represents.
>
> I say "silly" because, for other commonly-needed types of primitive
> values, we have concise, idiomatic ways to convert them from strings,
> leaving no need for wheel reinvention:
>
> int(os.getenv('NUM_FJORDS'))
> float(os.getenv('PRICE_LIMBURGER'))
> etc.
>
> So I thought it might be nice if we could do something like this for
> booleans, too:

We already can, with much more flexibility.  See below.

> Python 3.6.0a1+ (default:0b18f7d262cc+, Jun 12 2016, 18:21:54)
> [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> bool.from_config_str('false')
> False
>>>> bool.from_config_str('False')
> False
>>>> bool.from_config_str('True')
> True
>>>> bool.from_config_str('true')
> True

This is all specific to English.

>>>> bool.from_config_str('0')
> False

Duplicates bool(int('0')).  Ditto for '1'.

>>>> bool.from_config_str('')
> False

Duplicates bool('').

>>>> bool.from_config_str('1')
> True

These collectively duplicate "s in {'True', 'true', '1'}"
One can use any set of strings to be mapped to True.  The set can be 
reduced by lowercasing s. For an entry form, one might convert with
"man = sex.lower() in {'m', 'man', 'male'}".  One might even decide that 
"man = sex.lower.beginswith('m') is good enough.

For standard .cfg configuration files, configparser.RawConfigParser has 
this class attribute and method.

# Possible boolean values in the configuration.
BOOLEAN_STATES = {'1': True, 'yes': True, 'true': True, 'on': True,
                   '0': False, 'no': False, 'false': False, 'off': False}

def _convert_to_boolean(self, value):
     """Return a boolean value translating from other types if necessary.
     """
     if value.lower() not in self.BOOLEAN_STATES:
         raise ValueError('Not a boolean: %s' % value)
     return self.BOOLEAN_STATES[value.lower()]


One can customize this for any langauge or application.  A bool function 
would need duplicate attribute.


-- 
Terry Jan Reedy


From steve at pearwood.info  Sun Jun 12 23:12:39 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 13 Jun 2016 13:12:39 +1000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <20160613031237.GR27919@ando.pearwood.info>

Hi Josh, and welcome!

On Sun, Jun 12, 2016 at 07:52:28PM -0400, jab at math.brown.edu wrote:

[...]

> >>> bool.from_config_str('false')
> False
> >>> bool.from_config_str('False')
> False
> >>> bool.from_config_str('True')
> True
> >>> bool.from_config_str('true')
> True
> >>> bool.from_config_str('0')
> False
> >>> bool.from_config_str('')
> False
> >>> bool.from_config_str('1')
> True

You have certainly found a need, but tempting as it is, I don't believe 
it should be in the standard library or a builtin. While it is true that 
nearly everyone will, at some point, want to convert human-readable 
strings to bools, the details of how and what they do are too 
individual.

Some people will want to treat:

    true yes 1 on

    false no 0 off

as true and false values, while others will have other ideas:

    v?rit? oui si wahr ?????? ja ??
    aus ab unwahr 

etc. The point is, we cannot assume that all Python users will expect 
"yes" to map to True, and "no" to map to False. Since config files are 
generally written in the end-user's natural language, we can't even 
expect them to want "true" to map to True.

But I do think that this is a great candidate for a utility function in 
your own personal toolbox. There's no need for it to be in the 
*standard* library when it can be in your own *personal* library.


-- 
Steve

From ben+python at benfinney.id.au  Mon Jun 13 00:52:07 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 13 Jun 2016 14:52:07 +1000
Subject: [Python-ideas] bool.from_config_str()
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <857fdtzo5k.fsf@benfinney.id.au>

jab at math.brown.edu writes:

> (New here... hope this idea contributes something!)

Thank you, I think you have indeed contributed here.

> Several times I've had to implement a silly function that converts the
> odd environment variable (or some other value from the outside world
> that, perforce, comes in as a string) to the boolean it actually
> represents.

The obvious (to me) place to look for this is the ?configparser? module
<URL:https://docs.python.org/3/library/configparser.html> in the
standard library. That has functionality to read text, integer, or
boolean values.

(I am also a bit baffled to see some respondents in this thread
expressing the notion there is no consensus on what input text should
convert to a Boolean. The ?configparser? module is an obvious touchstone
here for other Python behaviour, IMO.)

> Is there any interest in this? If so, I'd be happy to make it work for
> bytes too, or to make any other changes that would help get it landed.

Perhaps the most obvious way to expose this functionality is as public
functions on the ?configparser? module. Re-factor the existing functions
to public functions, and expose those as part of the ?configparser? API.

That way, any code could use them directly from the module regardless of
whether a specific-format file were involved.

-- 
 \                ?Science doesn't work by vote and it doesn't work by |
  `\        authority.? ?Richard Dawkins, _Big Mistake_ (The Guardian, |
_o__)                                                      2006-12-27) |
Ben Finney


From lukasz at langa.pl  Mon Jun 13 03:19:17 2016
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Mon, 13 Jun 2016 00:19:17 -0700
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <D0D0F373-7CC4-4041-B51A-8C76B9F1F361@langa.pl>

On Jun 12, 2016, at 4:52 PM, jab at math.brown.edu wrote:

> 
> Python 3.6.0a1+ (default:0b18f7d262cc+, Jun 12 2016, 18:21:54)
> [GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> bool.from_config_str('false')
> False
>>>> bool.from_config_str('False')
> False
>>>> bool.from_config_str('True')
> True
>>>> bool.from_config_str('true')
> True
>>>> bool.from_config_str('0')
> False
>>>> bool.from_config_str('')
> False
>>>> bool.from_config_str('1')
> True

Previous discussion about the said patch: http://bugs.python.org/issue25243 <http://bugs.python.org/issue25243>

The most robust solution here, as Raymond suggests on the issue, is to define an application-level dictionary and use that. In this case you always know what values you support and you control what they are. This is literally a few lines of code. The configparser implementation is 6 without the docstring.

Let me explain why introducing your patch is not as easy as it might seem.

The problem with adding any variant of `from_config_str` to the `bool` type is that there is no clear pre-defined standard to follow here. That means this would likely open a new stream of bike shedding about what should and should not be part of the new API. Some people would like to be able to plug into this to add their own values. Others would like to read the existing dictionaries to reuse them in some library. Somebody would suggest `from_config_bytes()`. And so on, and so on. All in all, this adds a maintenance burden on the current and future CPython contributors. All of which are ultimately volunteers.

To put this into perspective, look at the `itertools` documentation. It's split between describing what the library implements and an ?Itertools Recipes? section that includes over two dozens of ~3-liners that the users might want to use in their programs. This form of documentation sends a message that the user is free to compose existing functionality however they want. It also sends a subtler message that not all three-liners have a place in the standard library, let alone the language. In the days of robust PyPI and ubiquitous `pip`, it?s easy to fill gaps yourself, in which case your solution is also usable with older versions of Python.

Summing up, thank you for your patch but I don?t think it is going to get accepted. I hope you understand. Don?t let that discourage you from tinkering further with CPython.

--
Lukasz Langa | Facebook
Production Engineer | The Ministry of Silly Walks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/f539d2a7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/f539d2a7/attachment.sig>

From marcin at urzenia.net  Mon Jun 13 03:27:21 2016
From: marcin at urzenia.net (Marcin Sztolcman)
Date: Mon, 13 Jun 2016 09:27:21 +0200
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <CAHbdAinbLxJ9xMOpHXRGNfnwe+_Gvdy7siXec5Fpbje-ad+dhw@mail.gmail.com>

On Mon, Jun 13, 2016 at 1:52 AM,  <jab at math.brown.edu> wrote:
> (New here... hope this idea contributes something!)
>
> Several times I've had to implement a silly function that converts the
> odd environment variable (or some other value from the outside world
> that, perforce, comes in as a string) to the boolean it actually
> represents.
>
> I say "silly" because, for other commonly-needed types of primitive
> values, we have concise, idiomatic ways to convert them from strings,
> leaving no need for wheel reinvention:
>
> int(os.getenv('NUM_FJORDS'))
> float(os.getenv('PRICE_LIMBURGER'))
> etc.

My proposal I use everyday:

foo = os.environ.get('FOO', DEFAULT_FOO).lower() in ('1', 'true', 'yes')

It's easy, concise and powerful, allow to specify what do you think is
true in that case.

-- 
Marcin Sztolcman :: http://urzenia.net/ :: http://sztolcman.eu/

From barry at python.org  Mon Jun 13 05:25:54 2016
From: barry at python.org (Barry Warsaw)
Date: Mon, 13 Jun 2016 12:25:54 +0300
Subject: [Python-ideas] Proposal for special name and qualname symbols
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <D7585789-FF8E-4CBB-A9B7-4D31E28CB334@barrys-emacs.org>
Message-ID: <20160613122554.5382385e@python.org>

On Jun 11, 2016, at 05:00 PM, Barry Scott wrote:

>What is the behavour in this case:
>
>   mydict[ index ] = Record( __LHS__ )

That's a good question.  How insane/useless would it be to return in this case
the string "mydict[ index ]"?

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/95df8d81/attachment.sig>

From barry at python.org  Mon Jun 13 05:32:37 2016
From: barry at python.org (Barry Warsaw)
Date: Mon, 13 Jun 2016 12:32:37 +0300
Subject: [Python-ideas] bool.from_config_str()
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
Message-ID: <20160613123237.636172c6@python.org>

On Jun 12, 2016, at 07:52 PM, jab at math.brown.edu wrote:

>So I thought it might be nice if we could do something like this for
>booleans, too:
>
>Python 3.6.0a1+ (default:0b18f7d262cc+, Jun 12 2016, 18:21:54)
>[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
>Type "help", "copyright", "credits" or "license" for more information.
>>>> bool.from_config_str('false')  
>False
>>>> bool.from_config_str('False')  
>False
>>>> bool.from_config_str('True')  
>True
>>>> bool.from_config_str('true')  
>True
>>>> bool.from_config_str('0')  
>False
>>>> bool.from_config_str('')  
>False
>>>> bool.from_config_str('1')  
>True

FWIW, in some projects I use lazr.config, which has (among other type
conversions) an as_boolean() method which does:

    value = value.lower()
    if value in ('true', 'yes', '1', 'on', 'enabled', 'enable'):
        return True
    if value in ('false', 'no', '0', 'off', 'disabled', 'disable'):
        return False
    raise ValueError('Invalid boolean value: %s' % value)

I understand the problem with adding this in a generic method on the bool
type, so I'm just mentioning this in case lazr.config.as_boolean() is useful
to you.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/7d8d9e74/attachment-0001.sig>

From steve at pearwood.info  Mon Jun 13 02:21:49 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 13 Jun 2016 16:21:49 +1000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <857fdtzo5k.fsf@benfinney.id.au>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <857fdtzo5k.fsf@benfinney.id.au>
Message-ID: <20160613062149.GT27919@ando.pearwood.info>

On Mon, Jun 13, 2016 at 02:52:07PM +1000, Ben Finney wrote:

> (I am also a bit baffled to see some respondents in this thread
> expressing the notion there is no consensus on what input text should
> convert to a Boolean. The ?configparser? module is an obvious touchstone
> here for other Python behaviour, IMO.)

To be honest, I had forgotten about configparser. But I stand by my 
earlier comments: in this 21st century globally-connected world, to 
expect all users to write "yes" in a config file when they mean "oui" or 
"ja" is unacceptable.

configparser works because the RawConfigParser class doesn't hard-code a 
list of strings, it sets an attribute BOOLEAN_STATES which can be 
overridden by instances.


-- 
Steve

From eric at trueblade.com  Mon Jun 13 06:40:40 2016
From: eric at trueblade.com (Eric V. Smith)
Date: Mon, 13 Jun 2016 06:40:40 -0400
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160613122554.5382385e@python.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <D7585789-FF8E-4CBB-A9B7-4D31E28CB334@barrys-emacs.org>
 <20160613122554.5382385e@python.org>
Message-ID: <6b036bc6-b382-2008-ed29-4d1de4e3143c@trueblade.com>

On 6/13/2016 5:25 AM, Barry Warsaw wrote:
> On Jun 11, 2016, at 05:00 PM, Barry Scott wrote:
>
>> What is the behavour in this case:
>>
>>   mydict[ index ] = Record( __LHS__ )
>
> That's a good question.  How insane/useless would it be to return in this case
> the string "mydict[ index ]"?

Very insane, and very useless!

Preserving the whitespace would be hard, and of dubious value.

I suggest that if we do this, we limit it to a single identifier on the 
lhs. In the unlikely event that we need it later, we can relax the 
restriction.

Eric.



From pavol.lisy at gmail.com  Mon Jun 13 12:12:30 2016
From: pavol.lisy at gmail.com (Pavol Lisy)
Date: Mon, 13 Jun 2016 18:12:30 +0200
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <20160613062149.GT27919@ando.pearwood.info>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <857fdtzo5k.fsf@benfinney.id.au> <20160613062149.GT27919@ando.pearwood.info>
Message-ID: <CABDEq2msL1Wtk5wwcCFX5qCFKFAXMd4Fwp4L56sS36PJGmjNjw@mail.gmail.com>

2016-06-13 8:21 GMT+02:00, Steven D'Aprano <steve at pearwood.info>:
> On Mon, Jun 13, 2016 at 02:52:07PM +1000, Ben Finney wrote:
>
>> (I am also a bit baffled to see some respondents in this thread
>> expressing the notion there is no consensus on what input text should
>> convert to a Boolean. The ?configparser? module is an obvious touchstone
>> here for other Python behaviour, IMO.)
>
> To be honest, I had forgotten about configparser. But I stand by my
> earlier comments: in this 21st century globally-connected world, to
> expect all users to write "yes" in a config file when they mean "oui" or
> "ja" is unacceptable.

Philosophical question which you are bringing is interesting! :)

Are keywords: "for, while, if, try, catch" good enough for
globally-connected world in 21st century?

Would be better or worse for python ecosystem to have possibility to
write for example:

#!language:fr

pour i en gamme(3):
    publier(i)

Maybe in second fourth of 21 century, when we will have intelligent
editors able automatically translate this to national (or
international) python it could be strong idea?

Or it's better to keep this simple in international/english mode? Not
to build babel tower?

From mafagafogigante at gmail.com  Mon Jun 13 12:19:24 2016
From: mafagafogigante at gmail.com (Bernardo Sulzbach)
Date: Mon, 13 Jun 2016 13:19:24 -0300
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <20160613062149.GT27919@ando.pearwood.info>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <857fdtzo5k.fsf@benfinney.id.au> <20160613062149.GT27919@ando.pearwood.info>
Message-ID: <f11ae820-c3c7-fd94-43a1-4bd465fb5a44@gmail.com>

On 06/13/2016 03:21 AM, Steven D'Aprano wrote:
>
> To be honest, I had forgotten about configparser. But I stand by my
> earlier comments: in this 21st century globally-connected world, to
> expect all users to write "yes" in a config file when they mean "oui" or
> "ja" is unacceptable.
>

Users (the chunk of meat type) should not have access to configuration 
files.

Developers should know the human language the project is being developed 
in (and it should be English).

As far as a configuration file goes, supporting only 1 and 0 for true 
and false is good enough.

This "globally-connected paranoia" would justify having

      rotating = ee

In public codebases. What is ee? True? False? A number?

From pavol.lisy at gmail.com  Mon Jun 13 12:24:47 2016
From: pavol.lisy at gmail.com (Pavol Lisy)
Date: Mon, 13 Jun 2016 18:24:47 +0200
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <6b036bc6-b382-2008-ed29-4d1de4e3143c@trueblade.com>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <D7585789-FF8E-4CBB-A9B7-4D31E28CB334@barrys-emacs.org>
 <20160613122554.5382385e@python.org>
 <6b036bc6-b382-2008-ed29-4d1de4e3143c@trueblade.com>
Message-ID: <CABDEq2nd9nNSiX2nM1B6wSspxKOVE6xoWNj9vjxT11Ek76NOzg@mail.gmail.com>

2016-06-13 12:40 GMT+02:00, Eric V. Smith <eric at trueblade.com>:
> On 6/13/2016 5:25 AM, Barry Warsaw wrote:
>> On Jun 11, 2016, at 05:00 PM, Barry Scott wrote:
>>
>>> What is the behavour in this case:
>>>
>>>   mydict[ index ] = Record( __LHS__ )
>>
>> That's a good question.  How insane/useless would it be to return in this
>> case
>> the string "mydict[ index ]"?

Or f"mydict[ {index} ]" ?

>
> Very insane, and very useless!
>
> Preserving the whitespace would be hard, and of dubious value.
>
> I suggest that if we do this, we limit it to a single identifier on the
> lhs. In the unlikely event that we need it later, we can relax the
> restriction.
>
> Eric.

I am telling nothing about usability, but relaxing later could be more
difficult due to backward compatibility!

What about:

  object.attribute = Record( __LHS__ )

?

From steve at pearwood.info  Mon Jun 13 13:21:27 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 14 Jun 2016 03:21:27 +1000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CABDEq2msL1Wtk5wwcCFX5qCFKFAXMd4Fwp4L56sS36PJGmjNjw@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <857fdtzo5k.fsf@benfinney.id.au> <20160613062149.GT27919@ando.pearwood.info>
 <CABDEq2msL1Wtk5wwcCFX5qCFKFAXMd4Fwp4L56sS36PJGmjNjw@mail.gmail.com>
Message-ID: <20160613172126.GV27919@ando.pearwood.info>

On Mon, Jun 13, 2016 at 06:12:30PM +0200, Pavol Lisy wrote:

> Would be better or worse for python ecosystem to have possibility to
> write for example:
> 
> #!language:fr
> pour i en gamme(3):
>     publier(i)

Localising Python to a non-English variant has already been done at 
least once seriously:

http://www.chinesepython.org/english/english.html
http://www.reganmian.net/blog/2008/11/21/chinese-python-translating-a-programming-language/

I'm pretty sure this is just intended as proof-of-concept:

http://www.fiber-space.de/EasyExtend/doc/teuton/teuton.htm

And if you really want to be silly, there's always LikePython:

http://www.staringispolite.com/likepython/


The idea of localisation does come with solid precedents:

- The ALGOL 68 standard allowed for localised versions, and at least one 
such version became the GOST/???? standard in the USSR.

- Microsoft Office macro language is localised.

- In the 1990s, Apple's Hyperscript and Applescript languages could be 
localised.

- 4th Dimension localised its keywords to French or German.

- There were various versions of BASIC localised to Chinese and other 
languages.

- Perl's parser can be modified at runtime to support not just keyword 
localisation, but changing the grammar as well. Of course Perl hackers 
used this to support Latin and Klingon.




-- 
Steve

From jab at math.brown.edu  Mon Jun 13 15:32:40 2016
From: jab at math.brown.edu (jab at math.brown.edu)
Date: Mon, 13 Jun 2016 15:32:40 -0400
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <D0D0F373-7CC4-4041-B51A-8C76B9F1F361@langa.pl>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <D0D0F373-7CC4-4041-B51A-8C76B9F1F361@langa.pl>
Message-ID: <CANwREeWndNjC5xHkUpHwKSoHoLOpWvrO2JH_7gCKwZCniPdFnA@mail.gmail.com>

On Mon, Jun 13, 2016 at 3:19 AM, ?ukasz Langa <lukasz at langa.pl> wrote:
> Previous discussion about the said patch: http://bugs.python.org/issue25243

(((
I would have linked to that, but:

- That's actually a different patch, and an inaccurate discussion of
it, to boot. For one, in an effort to avoid exactly your bike shedding
concerns, I specifically didn't include "on"/"off" or "yes"/"no" this
time (which I wouldn't have included originally, if it weren't for the
built-in configparser module's using them in getboolean().)

- The mention of the configparser module in that discussion ends up
being distracting, confusing the issue, and taking away from the
productive discussion of it I hoped to have here.

- Case in point, Raymond misunderstood the original patch, and then
never got a chance to respond to my clarifications.

So I was just trying to save people here from wasting time reading an
out-of-date patch and a confusing and inaccurate discussion of it.
)))

--

I know this would only save us from each having to rewrite the same
few-line function. And I know there can be a tradeoff in introducing a
new API like this. If it's just not possible to narrow the scope of
this enough to make it worth it (e.g. would just accepting "True" or
"False", so we offered a round-trip from str(some_bool), settle
this?), I'll leave it at that. But if anyone has any other ideas or
feels it's worth discussing further, I'd be happy to.

Thanks for your consideration!

Josh

From bzvi7919 at gmail.com  Mon Jun 13 17:32:10 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Mon, 13 Jun 2016 21:32:10 +0000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CANwREeWndNjC5xHkUpHwKSoHoLOpWvrO2JH_7gCKwZCniPdFnA@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <D0D0F373-7CC4-4041-B51A-8C76B9F1F361@langa.pl>
 <CANwREeWndNjC5xHkUpHwKSoHoLOpWvrO2JH_7gCKwZCniPdFnA@mail.gmail.com>
Message-ID: <CAMsGuy=ByTjJ6at9c37tcN+ZGOC+ijcBoDy+RsZhDxBtF08ZFQ@mail.gmail.com>

I do believe something like that, but with a better name should exist.
I have seen countless implementations for the same thing done over and over
again.
The whole point of a good standard library is to prevent repetition.

The funniest thing is that I personally use distutils.util.strtobool.

The fact that the same concept exists twice even in the standard library
and I wouldn't be surprised to see third or fourth implementation somewhere
in there, should be somewhat alarming.

Perhaps a utility function module, much like distutils.util should be
introduced.

distutils.dir_util.mkpath is also something I often use.
Maybe even renaming that forgotten module and raising awareness is the
solution.Another nice example is the itertools recipes. Things that people
just re-implement over and over again.
tl;dr: My suggestion is a utility module containing many implemented recipes
shortening many commonly used patterns. The only question is how to prevent
over-extending it with lots of niche and less-used ones.

On Mon, Jun 13, 2016 at 10:33 PM <jab at math.brown.edu> wrote:

> On Mon, Jun 13, 2016 at 3:19 AM, ?ukasz Langa <lukasz at langa.pl> wrote:
> > Previous discussion about the said patch:
> http://bugs.python.org/issue25243
>
> (((
> I would have linked to that, but:
>
> - That's actually a different patch, and an inaccurate discussion of
> it, to boot. For one, in an effort to avoid exactly your bike shedding
> concerns, I specifically didn't include "on"/"off" or "yes"/"no" this
> time (which I wouldn't have included originally, if it weren't for the
> built-in configparser module's using them in getboolean().)
>
> - The mention of the configparser module in that discussion ends up
> being distracting, confusing the issue, and taking away from the
> productive discussion of it I hoped to have here.
>
> - Case in point, Raymond misunderstood the original patch, and then
> never got a chance to respond to my clarifications.
>
> So I was just trying to save people here from wasting time reading an
> out-of-date patch and a confusing and inaccurate discussion of it.
> )))
>
> --
>
> I know this would only save us from each having to rewrite the same
> few-line function. And I know there can be a tradeoff in introducing a
> new API like this. If it's just not possible to narrow the scope of
> this enough to make it worth it (e.g. would just accepting "True" or
> "False", so we offered a round-trip from str(some_bool), settle
> this?), I'll leave it at that. But if anyone has any other ideas or
> feels it's worth discussing further, I'd be happy to.
>
> Thanks for your consideration!
>
> Josh
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/976af43e/attachment-0001.html>

From michael.selik at gmail.com  Mon Jun 13 18:08:54 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Mon, 13 Jun 2016 22:08:54 +0000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CAMsGuy=ByTjJ6at9c37tcN+ZGOC+ijcBoDy+RsZhDxBtF08ZFQ@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <D0D0F373-7CC4-4041-B51A-8C76B9F1F361@langa.pl>
 <CANwREeWndNjC5xHkUpHwKSoHoLOpWvrO2JH_7gCKwZCniPdFnA@mail.gmail.com>
 <CAMsGuy=ByTjJ6at9c37tcN+ZGOC+ijcBoDy+RsZhDxBtF08ZFQ@mail.gmail.com>
Message-ID: <CAGgTfkPAUVyevJ=DniAKDVDq5cAo6bfq659butu_Cdoem1S0sg@mail.gmail.com>

On Mon, Jun 13, 2016 at 5:32 PM Bar Harel <bzvi7919 at gmail.com> wrote:

> The whole point of a good standard library is to prevent repetition.
>

Certain kinds of repetition, yes. Some code looks almost exactly the same,
but the slight differences make it hard to generalize as a function. In
those cases, it's better for the language to provide some simple tools that
can be composed to get the job done. In this case, we have ``value = config
in strings_that_mean_true`` which solves the category of problem quite
nicely.

The fact that the same concept exists twice even in the standard library
>

The repetition is that each case might have a different standard for what
strings are truthy and whether to raise an error if the string is an
unexpected value.

> tl;dr: My suggestion is a utility module containing many implemented recipes
> shortening many commonly used patterns. The only question is how to prevent
> over-extending it with lots of niche and less-used ones.
>
Some possible criteria for evaluating the benefit of a function/class:
- the code pattern it replaces is very common in stdlib and major projects
- the code pattern it replaces is a frequent source of bugs, even if rare
- the code pattern it replaces is hard to write, even if usually correct
- the code pattern it replaces is hard to read, even if rare and easy to
write

Sometimes copy-paste-tweak is OK.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/b79d1720/attachment.html>

From jab at math.brown.edu  Mon Jun 13 18:13:39 2016
From: jab at math.brown.edu (jab at math.brown.edu)
Date: Mon, 13 Jun 2016 18:13:39 -0400
Subject: [Python-ideas] eval_literal
Message-ID: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>

(Riffing off some discussion in another thread, I had another idea I
wanted to throw out there.)

Given that Eval Really Is Dangerous[1], has something like this ever
been considered?:

>>> int(str(42))
42
>>> float(str(42.0))
42.0
>>> bool(str(False))  # :(
True
>>> eval_literal('42')
42
>>> eval_literal('42.0')
42.0
>>> eval_literal('False')  # :)
False
>>> eval_literal('', default=False)  # shout out to PEP 463 / Michael Selik
False

i.e. An extremely limited version of eval, possibly just for literals
or even literal atoms, that would make it safe?


[1] http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html

From bzvi7919 at gmail.com  Mon Jun 13 18:16:15 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Mon, 13 Jun 2016 22:16:15 +0000
Subject: [Python-ideas] eval_literal
In-Reply-To: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
References: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
Message-ID: <CAMsGuy=dHZS8qf0at-eLL-B9VE6+4JWD83Jc58Y1LMtsSk3CYw@mail.gmail.com>

That exact same thing exists mate :-)

See ast.literal_eval

On Tue, Jun 14, 2016 at 1:14 AM <jab at math.brown.edu> wrote:

> (Riffing off some discussion in another thread, I had another idea I
> wanted to throw out there.)
>
> Given that Eval Really Is Dangerous[1], has something like this ever
> been considered?:
>
> >>> int(str(42))
> 42
> >>> float(str(42.0))
> 42.0
> >>> bool(str(False))  # :(
> True
> >>> eval_literal('42')
> 42
> >>> eval_literal('42.0')
> 42.0
> >>> eval_literal('False')  # :)
> False
> >>> eval_literal('', default=False)  # shout out to PEP 463 / Michael Selik
> False
>
> i.e. An extremely limited version of eval, possibly just for literals
> or even literal atoms, that would make it safe?
>
>
> [1] http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/724cc0e6/attachment.html>

From jelle.zijlstra at gmail.com  Mon Jun 13 18:16:55 2016
From: jelle.zijlstra at gmail.com (Jelle Zijlstra)
Date: Mon, 13 Jun 2016 15:16:55 -0700
Subject: [Python-ideas] eval_literal
In-Reply-To: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
References: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
Message-ID: <CAFp3-p-cjmGiCPFH-y-BGi4rS8TuwR=pZUrPs-K+1DNYG_2DXA@mail.gmail.com>

Yes, it's called ast.literal_eval.
https://docs.python.org/3/library/ast.html#ast.literal_eval

2016-06-13 15:13 GMT-07:00 <jab at math.brown.edu>:

> (Riffing off some discussion in another thread, I had another idea I
> wanted to throw out there.)
>
> Given that Eval Really Is Dangerous[1], has something like this ever
> been considered?:
>
> >>> int(str(42))
> 42
> >>> float(str(42.0))
> 42.0
> >>> bool(str(False))  # :(
> True
> >>> eval_literal('42')
> 42
> >>> eval_literal('42.0')
> 42.0
> >>> eval_literal('False')  # :)
> False
> >>> eval_literal('', default=False)  # shout out to PEP 463 / Michael Selik
> False
>
> i.e. An extremely limited version of eval, possibly just for literals
> or even literal atoms, that would make it safe?
>
>
> [1] http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/6b0a1320/attachment.html>

From rosuav at gmail.com  Mon Jun 13 18:17:54 2016
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 14 Jun 2016 08:17:54 +1000
Subject: [Python-ideas] eval_literal
In-Reply-To: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
References: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
Message-ID: <CAPTjJmoBuscDpVQreoRce4Tt+BoBCEnj7ntKdJFvzecoeT4XyA@mail.gmail.com>

On Tue, Jun 14, 2016 at 8:13 AM,  <jab at math.brown.edu> wrote:
> (Riffing off some discussion in another thread, I had another idea I
> wanted to throw out there.)
>
> Given that Eval Really Is Dangerous[1], has something like this ever
> been considered?:
>
>>>> int(str(42))
> 42
>>>> float(str(42.0))
> 42.0
>>>> bool(str(False))  # :(
> True
>>>> eval_literal('42')
> 42
>>>> eval_literal('42.0')
> 42.0
>>>> eval_literal('False')  # :)
> False
>>>> eval_literal('', default=False)  # shout out to PEP 463 / Michael Selik
> False
>
> i.e. An extremely limited version of eval, possibly just for literals
> or even literal atoms, that would make it safe?
>

Check out ast.literal_eval:

https://docs.python.org/3/library/ast.html#ast.literal_eval

It's capable of evaluating all forms of literal, plus a variety of
things that people kinda expect to be literals but aren't, plus some
larger constructs that definitely aren't literals but are still very
useful and safe (eg list display).

ChrisA

From mahmoud at hatnote.com  Mon Jun 13 18:18:19 2016
From: mahmoud at hatnote.com (Mahmoud Hashemi)
Date: Mon, 13 Jun 2016 15:18:19 -0700
Subject: [Python-ideas] eval_literal
In-Reply-To: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
References: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
Message-ID: <CALXjU9EMwe-76W9=5pp3uQK0U7T_YEkWoebNvCWgAA1m+URK+w@mail.gmail.com>

You mean ast.literal_eval?
https://docs.python.org/2/library/ast.html#ast.literal_eval

Mahmoud
https://github.com/mahmoud
http://sedimental.org

On Mon, Jun 13, 2016 at 3:13 PM, <jab at math.brown.edu> wrote:

> (Riffing off some discussion in another thread, I had another idea I
> wanted to throw out there.)
>
> Given that Eval Really Is Dangerous[1], has something like this ever
> been considered?:
>
> >>> int(str(42))
> 42
> >>> float(str(42.0))
> 42.0
> >>> bool(str(False))  # :(
> True
> >>> eval_literal('42')
> 42
> >>> eval_literal('42.0')
> 42.0
> >>> eval_literal('False')  # :)
> False
> >>> eval_literal('', default=False)  # shout out to PEP 463 / Michael Selik
> False
>
> i.e. An extremely limited version of eval, possibly just for literals
> or even literal atoms, that would make it safe?
>
>
> [1] http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/9c7bbbb9/attachment.html>

From jab at math.brown.edu  Mon Jun 13 18:24:03 2016
From: jab at math.brown.edu (jab at math.brown.edu)
Date: Mon, 13 Jun 2016 18:24:03 -0400
Subject: [Python-ideas] eval_literal
In-Reply-To: <CAMsGuy=dHZS8qf0at-eLL-B9VE6+4JWD83Jc58Y1LMtsSk3CYw@mail.gmail.com>
References: <CANwREeVj9EtrjvG8b9rndS+pJukuUW3dEeJg4DYMGWP-73pJyA@mail.gmail.com>
 <CAMsGuy=dHZS8qf0at-eLL-B9VE6+4JWD83Jc58Y1LMtsSk3CYw@mail.gmail.com>
Message-ID: <CANwREeX4-ENe-j_VTnC-fd69C2PRh4jBwB6q38=s+V=qtsG5yQ@mail.gmail.com>

On Mon, Jun 13, 2016 at 6:16 PM, Bar Harel <bzvi7919 at gmail.com> wrote:
> That exact same thing exists mate :-)
>
> See ast.literal_eval

Cool! Perhaps I can consider it validating that I just reinvented it? :)

And to deal with the empty string case (in the absence of PEP 463), to
make it a possible substitute for some bool.from_str() method, I guess
you could do:

>>> ast.literal_eval(os.getenv('FOO', 'False'))

From bzvi7919 at gmail.com  Mon Jun 13 18:34:06 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Mon, 13 Jun 2016 22:34:06 +0000
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <CAGgTfkPAUVyevJ=DniAKDVDq5cAo6bfq659butu_Cdoem1S0sg@mail.gmail.com>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <D0D0F373-7CC4-4041-B51A-8C76B9F1F361@langa.pl>
 <CANwREeWndNjC5xHkUpHwKSoHoLOpWvrO2JH_7gCKwZCniPdFnA@mail.gmail.com>
 <CAMsGuy=ByTjJ6at9c37tcN+ZGOC+ijcBoDy+RsZhDxBtF08ZFQ@mail.gmail.com>
 <CAGgTfkPAUVyevJ=DniAKDVDq5cAo6bfq659butu_Cdoem1S0sg@mail.gmail.com>
Message-ID: <CAMsGuykJuM_7N1YRxgCPHtS_JwNqytMso91Y6aGxXCd0GWtAsg@mail.gmail.com>

Thanks for the feedback. I like your criteria suggestions btw.

The repetition is that each case might have a different standard for what
> strings are truthy and whether to raise an error if the string is an
> unexpected value.
>

Well, both configparser and strtobool evaluate the same strings as True and
False, and both raise an exception but I see what you're saying.

I personally feel that a new utility module has space much like itertool's
common recipes and their function "equivalent" explanations. Heck, even
itertool itself is basically a utility module, but another option is
creating a wiki page similar to PerformanceTips
<https://wiki.python.org/moin/PythonSpeed/PerformanceTips> in
wiki.python.org. It won't require even a single line of code and being
marked as "suggested guidelines" from the python community may be highly
beneficial.

What do you think of that? :-)

On Tue, Jun 14, 2016 at 1:09 AM Michael Selik <michael.selik at gmail.com>
wrote:

> On Mon, Jun 13, 2016 at 5:32 PM Bar Harel <bzvi7919 at gmail.com> wrote:
>
>> The whole point of a good standard library is to prevent repetition.
>>
>
> Certain kinds of repetition, yes. Some code looks almost exactly the same,
> but the slight differences make it hard to generalize as a function. In
> those cases, it's better for the language to provide some simple tools that
> can be composed to get the job done. In this case, we have ``value =
> config in strings_that_mean_true`` which solves the category of problem
> quite nicely.
>
> The fact that the same concept exists twice even in the standard library
>>
>
> The repetition is that each case might have a different standard for what
> strings are truthy and whether to raise an error if the string is an
> unexpected value.
>
>> tl;dr: My suggestion is a utility module containing many implemented recipes
>> shortening many commonly used patterns. The only question is how to prevent
>> over-extending it with lots of niche and less-used ones.
>>
> Some possible criteria for evaluating the benefit of a function/class:
> - the code pattern it replaces is very common in stdlib and major projects
> - the code pattern it replaces is a frequent source of bugs, even if rare
> - the code pattern it replaces is hard to write, even if usually correct
> - the code pattern it replaces is hard to read, even if rare and easy to
> write
>
> Sometimes copy-paste-tweak is OK.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160613/4f072832/attachment.html>

From stephen at xemacs.org  Tue Jun 14 02:44:01 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 14 Jun 2016 15:44:01 +0900
Subject: [Python-ideas] bool.from_config_str()
In-Reply-To: <20160613172126.GV27919@ando.pearwood.info>
References: <CANwREeXs5=hO+LQDr6Zc0SZxzLaZE1NrN5dDBeQQAxu9Wwnuow@mail.gmail.com>
 <857fdtzo5k.fsf@benfinney.id.au>
 <20160613062149.GT27919@ando.pearwood.info>
 <CABDEq2msL1Wtk5wwcCFX5qCFKFAXMd4Fwp4L56sS36PJGmjNjw@mail.gmail.com>
 <20160613172126.GV27919@ando.pearwood.info>
Message-ID: <22367.42929.282689.286032@turnbull.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > The idea of localisation does come with solid precedents:

But the argument against localization of machine-parsed code (which
would include configs) has solid precedents from the days of PEP 263
implementation.  There the issue was the language of comments (which
aren't even machine-parsed!)  Several prominent developers with
impeccable "global" credentials related anecdotes involving comments
that were unreadable to the rest of the team when the author left, as
well as one ridiculous case where the entire dev team of one
nationality left to form a startup or something, and were replaced by
a new team of a different nationality.

As a not-so-prominent developer, I can testify to translating several
hundred lines of Japanese comments in XEmacs sources because the
authors had disappeared and I was the only Japanese reader in the
core.

Localization for educational purposes at the compulsory education
level may make sense (although for children under the age of 12 it
probably has little value, because they absorb new words and whole new
languages very quickly -- OTOH, their teachers already have withered
neurons at ages as low as 23).  Other than that kind of use case, if
localization is so important, make a configuration app and localize
that, I think.

Steve


From stephen at xemacs.org  Tue Jun 14 02:39:56 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 14 Jun 2016 15:39:56 +0900
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <20160613122554.5382385e@python.org>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <D7585789-FF8E-4CBB-A9B7-4D31E28CB334@barrys-emacs.org>
 <20160613122554.5382385e@python.org>
Message-ID: <22367.42684.398488.443412@turnbull.sk.tsukuba.ac.jp>

 > On Jun 11, 2016, at 05:00 PM, Barry Scott wrote:
 > 
 > >What is the behavour in this case:
 > >
 > >   mydict[ index ] = Record( __LHS__ )

I can't imagine why that would ever come up.  More likely would be

    mydict = { name : Record[name] for name in standard_symbol_names }

and the whole issue is just gone, this is TOOWTDI for such iterative
cases.  It's only when you want to inject a name (such as "mydict"
itself) into a namespace normally maintained by the compiler that you
need this.

Barry Warsaw writes:

 > That's a good question.  How insane/useless would it be to return
 > in this case the string "mydict[ index ]"?

Given the assumption that this kind of anaphorism is Pythonic at all,
it would either be that or a canonical flattening of the AST.  It
looks quite useless to me.

But IMO this is unacceptably ugly given that its use cases are so
small.  A dunder keyword that is used by design even in main programs?
C'mon, Barry, you can't mean you're going to litter Mailman code with
this!

I don't like (-0.9) Steven d'Aprano's "->" operator either, but it's
the best of a bad lot so far.  Note that it can't be used iteratively
either unless its signature is changed from <id> -> <constructor> to
<str> -> <constructor>, so that instead of writing

    x -> Symbol()

you'd write

    'x' -> Symbol()

(thus instead of reading the name of an identifier from the namespace
and passing the string to Symbol, it would find or inject an
identifier with that name in the namespace, and assign the value of
the RHS to it).  Then you could write things like

    for name in standard_symbol_names:
        name -> Symbol()

which I don't like but I suspect advocates will.

Steve

From michael.selik at gmail.com  Tue Jun 14 07:58:17 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Tue, 14 Jun 2016 11:58:17 +0000
Subject: [Python-ideas] Proposal for special name and qualname symbols
In-Reply-To: <22367.42684.398488.443412@turnbull.sk.tsukuba.ac.jp>
References: <20160608160904.GM12028@ando.pearwood.info>
 <87oa7bqxea.fsf@thinkpad.rath.org>
 <20160608171016.074950ca@anarchist.wooz.org>
 <D7585789-FF8E-4CBB-A9B7-4D31E28CB334@barrys-emacs.org>
 <20160613122554.5382385e@python.org>
 <22367.42684.398488.443412@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CAGgTfkMX6b7orFBg09_gsbn_NoZLuQGLbF5_h7kUdvHXMKy1VA@mail.gmail.com>

On Tue, Jun 14, 2016 at 3:00 AM Stephen J. Turnbull <stephen at xemacs.org>
wrote:

>  > On Jun 11, 2016, at 05:00 PM, Barry Scott wrote:
>  >
>  > >What is the behavour in this case:
>  > >   mydict[ index ] = Record( __LHS__ )
>
> But IMO this is unacceptably ugly given that its use cases are so
> small.  A dunder keyword that is used by design even in main programs?
>

One use case is replacing usage of ``sys.getframe()``. Perhaps that
suggests the best replacement is a new sys module function ``sys.getlhs()``
or ``sys.getname()``.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160614/742c66fb/attachment.html>

From leewangzhong+python at gmail.com  Tue Jun 14 18:58:06 2016
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Tue, 14 Jun 2016 18:58:06 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without an
 encoding
Message-ID: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>

Current behavior (3.5.1):
    >>> bytes('')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: string argument without an encoding

Suggestion:
If the character size is 1, the `bytes`/`bytearray` constructor
doesn't need a specified encoding.

High-level idea:
If the string only has code points in range(128), encoding is optional
(and useless anyway). The new error message could be
    TypeError: non-ASCII string argument without an encoding

How:
CPython strings store characters in an array, such that each character
takes a single entry. With an entry per character, indexing is just a
regular C array index operation. Since PEP 393, the size( )of the
elements of the array is just the size needed for the largest
character. Thus, CPython strings "know" whether or not they're ASCII.

Other implementations without PEP 393 can do a scan of the code points
to check the 0-127 condition during building. That means O(n) more
checks, but in those implementations, the per-character checks are
already necessary with an explicit encoding, since you'd need to see
if that character needs encoding.

(The `bytes` and ASCII-`str` could in fact share memory, given a few
tweaks. But that's an implementation detail.)

From guido at python.org  Tue Jun 14 19:26:19 2016
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Jun 2016 16:26:19 -0700
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
Message-ID: <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>

-1. Such a check for the contents of the string sounds exactly like the
Python 2 behavior we are trying to get away with.

On Tue, Jun 14, 2016 at 3:58 PM, Franklin? Lee <
leewangzhong+python at gmail.com> wrote:

> Current behavior (3.5.1):
>     >>> bytes('')
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: string argument without an encoding
>
> Suggestion:
> If the character size is 1, the `bytes`/`bytearray` constructor
> doesn't need a specified encoding.
>
> High-level idea:
> If the string only has code points in range(128), encoding is optional
> (and useless anyway). The new error message could be
>     TypeError: non-ASCII string argument without an encoding
>
> How:
> CPython strings store characters in an array, such that each character
> takes a single entry. With an entry per character, indexing is just a
> regular C array index operation. Since PEP 393, the size( )of the
> elements of the array is just the size needed for the largest
> character. Thus, CPython strings "know" whether or not they're ASCII.
>
> Other implementations without PEP 393 can do a scan of the code points
> to check the 0-127 condition during building. That means O(n) more
> checks, but in those implementations, the per-character checks are
> already necessary with an explicit encoding, since you'd need to see
> if that character needs encoding.
>
> (The `bytes` and ASCII-`str` could in fact share memory, given a few
> tweaks. But that's an implementation detail.)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160614/13e11495/attachment.html>

From leewangzhong+python at gmail.com  Tue Jun 14 19:46:34 2016
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Tue, 14 Jun 2016 19:46:34 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>
Message-ID: <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>

On Tue, Jun 14, 2016 at 7:26 PM, Guido van Rossum <guido at python.org> wrote:
> -1. Such a check for the contents of the string sounds exactly like the
> Python 2 behavior we are trying to get away with.

But isn't it really just converting back and forth between two
representations of the same thing? A str with char width 1 is
conceptually an ASCII string; you're just changing how it's exposed to
the program.

As it stands, when you have an ASCII string stored as a str, you can
call str.encode() on it (whereby it will default to encoding='utf-8'),
or you can call `bytes(s, 'utf-8')`, and pass in an argument which is
conceptually ignored. (Unless it is in fact not an ASCII string!) On
the other hand, `bytes(s)` means, "Encoding shall not be necessary."
That could be semantically useful, and a non-ASCII string will trigger
an exception, while the other methods will just encode.

From python at mrabarnett.plus.com  Tue Jun 14 20:17:02 2016
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 15 Jun 2016 01:17:02 +0100
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
Message-ID: <ef4ec113-5fad-7cad-7ad2-76c1e1227b15@mrabarnett.plus.com>

On 2016-06-14 23:58, Franklin? Lee wrote:
> Current behavior (3.5.1):
>     >>> bytes('')
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: string argument without an encoding
>
> Suggestion:
> If the character size is 1, the `bytes`/`bytearray` constructor
> doesn't need a specified encoding.
>
> High-level idea:
> If the string only has code points in range(128), encoding is optional
> (and useless anyway). The new error message could be
>     TypeError: non-ASCII string argument without an encoding
>
Why is the encoding is optional (and useless)? Are you assuming that the 
user will want 'ascii'?

-1


From matt.ruffalo at gmail.com  Tue Jun 14 20:30:37 2016
From: matt.ruffalo at gmail.com (Matt Ruffalo)
Date: Tue, 14 Jun 2016 20:30:37 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>
 <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
Message-ID: <5760A1AD.5090602@gmail.com>

On 2016-06-14 19:46, Franklin? Lee wrote:
> But isn't it really just converting back and forth between two
> representations of the same thing? A str with char width 1 is
> conceptually an ASCII string; you're just changing how it's exposed to
> the program.
Your concept of a "str with char width 1" is not well-defined at all,
and is not true under most ways I can think of to interpret what you've
said.

"""
Python 3.5.1+ (default, Mar 30 2016, 22:46:26)
[GCC 5.3.1 20160330] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> s = '\U0001f4a9'
>>> len(s)
1
>>> len(s.encode('utf-8'))
4
"""

I think you might mean "code point that will be encoded to a single byte
under UTF-8" (i.e. those in the ASCII range), because there are many
code points that can be encoded to single bytes under various legacy
encodings like ISO-8859-[1-15], the KOI variants, and so on.

> As it stands, when you have an ASCII string stored as a str, you can
> call str.encode() on it (whereby it will default to encoding='utf-8'),
> or you can call `bytes(s, 'utf-8')`, and pass in an argument which is
> conceptually ignored. (Unless it is in fact not an ASCII string!) On
> the other hand, `bytes(s)` means, "Encoding shall not be necessary."
> That could be semantically useful, and a non-ASCII string will trigger
> an exception, while the other methods will just encode.

What makes you say that the "utf-8" argument to `bytes` is conceptually
ignored? That is the encoding used to convert the Unicode code points to
bytes! That's by no means the only option either; Python supports
multiple ASCII-incompatible encodings like UTF-16, UTF-32, and (as I
recall) Shift-JIS, etc. 'utf-8' isn't hardcoded here, either; the
encoding used is the one reported by `sys.getdefaultencoding()`.

Saying "Unless it is in fact not an ASCII string!" doesn't make any
sense either; UTF-8 is still used to convert e.g. U+0041 LATIN CAPITAL
LETTER A to the byte 0x41. Even if your text string only contains code
points in the range [U+0000, U+007F], this is a very important semantic
distinction, so a big -1.

MMR...

From tjreedy at udel.edu  Tue Jun 14 20:34:47 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 14 Jun 2016 20:34:47 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
Message-ID: <njq7rd$edj$1@ger.gmane.org>

On 6/14/2016 6:58 PM, Franklin? Lee wrote:
> Current behavior (3.5.1):
>     >>> bytes('')
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: string argument without an encoding

A null case is the only case where an encoding is not needed, but having 
a special rule is not worth the trouble.

> Suggestion:
> If the character size is 1, the `bytes`/`bytearray` constructor
> doesn't need a specified encoding.

By character size, I presume you mean the PEP 393 internal 1,2,4 bytes 
per char.  Non-ascii latin-1 chars can also be size 1.  Not all 
encodings are ascii compatible.  For example, IBM's EBCDIC.

-- 
Terry Jan Reedy


From stephen at xemacs.org  Tue Jun 14 20:50:56 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 15 Jun 2016 09:50:56 +0900
Subject: [Python-ideas] Strings can sometimes convert to bytes without an
 encoding
In-Reply-To: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
Message-ID: <22368.42608.128692.756772@turnbull.sk.tsukuba.ac.jp>

Franklin? Lee writes:
 > Current behavior (3.5.1):
 >     >>> bytes('')
 >     Traceback (most recent call last):
 >       File "<stdin>", line 1, in <module>
 >     TypeError: string argument without an encoding
 > 
 > Suggestion:
 > If the character size is 1, the `bytes`/`bytearray` constructor
 > doesn't need a specified encoding.

-1

The current rule is simple, the error obvious.  The rule you propose
is not only more complex, but also not TOOWTDI.  Many use cases will
want the default encoding, not ASCII.

Also, YAGNI.  People who really truly do want ASCII because of wire
protocol elements that look like English words are likely to be
working at the bytes level and not using str at all, especially after
PEP 461.


From steve at pearwood.info  Tue Jun 14 21:06:35 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 15 Jun 2016 11:06:35 +1000
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>
 <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
Message-ID: <20160615010634.GZ27919@ando.pearwood.info>

On Tue, Jun 14, 2016 at 07:46:34PM -0400, Franklin? Lee wrote:
> On Tue, Jun 14, 2016 at 7:26 PM, Guido van Rossum <guido at python.org> wrote:
> > -1. Such a check for the contents of the string sounds exactly like the
> > Python 2 behavior we are trying to get away with.
> 
> But isn't it really just converting back and forth between two
> representations of the same thing? A str with char width 1 is
> conceptually an ASCII string; 

How do you reason that? There are many one-character strings that aren't 
ASCII, and encode to two or more bytes.

py> s = '\N{CJK UNIFIED IDEOGRAPH-4E4D}'
py> len(s)
1
py> s.encode('big5')
b'\xa5E'

Other multibyte encodings include UTF-16 and UTF-32. Even single-byte 
encodings are not necessarily based on ASCII, e.g. the various EBCDIC 
codecs.

The only string which is likely to return the same bytes under ALL 
encodings is the empty string. And even that is not guaranteed: imagine 
that somebody uses the codec machinary to create a string to bytes 
transformation which is not a pure character translation, e.g. a 
string-to-bytes version of:

py> codecs.encode(b'', 'zip')
b'x\x9c\x03\x00\x00\x00\x00\x01'

Even ignoring such exotic transformations, it's not worth special casing 
the single case of '' --> b''. Who is going to write ''.encode() when 
they could just write b''?


> you're just changing how it's exposed to
> the program.
> 
> As it stands, when you have an ASCII string stored as a str, you can
> call str.encode() on it (whereby it will default to encoding='utf-8'),
> or you can call `bytes(s, 'utf-8')`, and pass in an argument which is
> conceptually ignored. (Unless it is in fact not an ASCII string!) 

Your comment in parentheses is the crux of the matter: your string may 
not be an ASCII string, so you don't know if the encoding is 
conceptually ignored.

Besides, it may not be ignored: the FSR is a CPython implementation 
detail, not a language guarantee. 'A' is not necessarily stored 
internally as the single byte 0x41. Obvious alternatives are UTF-16 (two 
bytes) or UTF-32 (four bytes). I expect that Jython and IronPython will 
use whatever Java and .Net use for strings, which is unlikely to be the 
same as what CPython does.


> On
> the other hand, `bytes(s)` means, "Encoding shall not be necessary."
> That could be semantically useful, and a non-ASCII string will trigger
> an exception, while the other methods will just encode.

Encode how? Just by copying bytes from whatever internal representation 
the Python compiler happens to use? Anything else requires a codec.

You're suggesting raising an exception on non-ASCII strings, so either 
strings need an internal flag to say whether they're ASCII or not 
(possibly a good idea regardless), or bytes(s) has to scan the string, 
not just copy it.

To my mind, it sounds like giving bytes a default encoding of 'ascii' 
might satify you. Or you can write a wrapper:

def bytes(s):
    return builtins.bytes(s, 'ascii')



-- 
Steve

From leewangzhong+python at gmail.com  Tue Jun 14 21:27:10 2016
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Tue, 14 Jun 2016 21:27:10 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <ef4ec113-5fad-7cad-7ad2-76c1e1227b15@mrabarnett.plus.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <ef4ec113-5fad-7cad-7ad2-76c1e1227b15@mrabarnett.plus.com>
Message-ID: <CAB_e7iyGsYq_hGt-5mwPxELbrFfDp6YBgKNsfb1mtgm-nGEf1w@mail.gmail.com>

Withdrawing the proposal because I didn't know `bytes(s, 'ascii')` exists.

On Tue, Jun 14, 2016 at 8:17 PM, MRAB <python at mrabarnett.plus.com> wrote:
> Why is the encoding is optional (and useless)? Are you assuming that the
> user will want 'ascii'?

^ That prompted me to check.

From storchaka at gmail.com  Tue Jun 14 22:09:11 2016
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 15 Jun 2016 05:09:11 +0300
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <njq7rd$edj$1@ger.gmane.org>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <njq7rd$edj$1@ger.gmane.org>
Message-ID: <njqdc8$oes$1@ger.gmane.org>

On 15.06.16 03:34, Terry Reedy wrote:
> A null case is the only case where an encoding is not needed, but having
> a special rule is not worth the trouble.

It is needed even in null case. bytes('', 'utf-8') != bytes('', 'utf-16').



From tjreedy at udel.edu  Wed Jun 15 00:06:35 2016
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 15 Jun 2016 00:06:35 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <njqdc8$oes$1@ger.gmane.org>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <njq7rd$edj$1@ger.gmane.org> <njqdc8$oes$1@ger.gmane.org>
Message-ID: <njqk8g$k1r$1@ger.gmane.org>

On 6/14/2016 10:09 PM, Serhiy Storchaka wrote:
> On 15.06.16 03:34, Terry Reedy wrote:
>> A null case is the only case where an encoding is not needed, but having
>> a special rule is not worth the trouble.
>
> It is needed even in null case. bytes('', 'utf-8') != bytes('', 'utf-16').

& bytes('', 'utf-32') is different yet ;-).

-- 
Terry Jan Reedy


From greg.ewing at canterbury.ac.nz  Wed Jun 15 01:51:46 2016
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Jun 2016 17:51:46 +1200
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
Message-ID: <5760ECF2.3050704@canterbury.ac.nz>

Franklin? Lee wrote:
> If the string only has code points in range(128), encoding is optional
> (and useless anyway).

No, it's not useless. It's possible to have an encoding
that takes code points in the range 0-127 to something
other than their ASCII equivalents. UTF-16, for example.

You're effectively suggesting that ASCII or Latin-1
should be assumed as a default encoding, which seems like
a bad idea.

-- 
Greg

From leewangzhong+python at gmail.com  Wed Jun 15 04:55:21 2016
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Wed, 15 Jun 2016 04:55:21 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <5760ECF2.3050704@canterbury.ac.nz>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <5760ECF2.3050704@canterbury.ac.nz>
Message-ID: <CAB_e7ixyMgVVSe-Ne=0qg-AnOWoeTDdBKUJQC0LEyE73J_QTVQ@mail.gmail.com>

On Jun 15, 2016 1:52 AM, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:
>
> Franklin? Lee wrote:
>>
>> If the string only has code points in range(128), encoding is optional
>> (and useless anyway).
>
>
> No, it's not useless. It's possible to have an encoding
> that takes code points in the range 0-127 to something
> other than their ASCII equivalents. UTF-16, for example.
>
> You're effectively suggesting that ASCII or Latin-1
> should be assumed as a default encoding, which seems like
> a bad idea.

UTF-8 is a default encoding for str.encode and bytes.decode. Latin-1 is the
internal encoding in CPython whenever possible, and PyASCIIObject is an
internal struct in Python 3. It is not exactly alien to Python to choose
ASCII as a default. If it is a bad idea, it is not original to me.

ASCII has a privileged position among single-byte encodings, even in Python
3. There's no 'builtins.latin1', let alone 'builtins.shiftjis' (though,
someone might point out, it's not single-byte). We don't have 're.CA_1'. I
could list more things that Python provides for ASCII but not any
ASCII-incompatible encodings: https://docs.python.org/3/search.html?q=ascii
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160615/9fe0858f/attachment.html>

From ethan at stoneleaf.us  Wed Jun 15 11:27:04 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 15 Jun 2016 08:27:04 -0700
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>
 <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
Message-ID: <576173C8.7070105@stoneleaf.us>

On 06/14/2016 04:46 PM, Franklin? Lee wrote:
> On Tue, Jun 14, 2016 at 7:26 PM, Guido van Rossum wrote:

>> -1. Such a check for the contents of the string sounds exactly like the
>> Python 2 behavior we are trying to get away [from].
>
> But isn't it really just converting back and forth between two
> representations of the same thing? A str with char width 1 is
> conceptually an ASCII string; you're just changing how it's exposed to
> the program.

The main reason Python 3 is not Python 2 is because text is text and 
bytes are bytes and there will be no more automagic encoding/decoding 
betwixt the two.


On 06/15/2016 01:55 AM, Franklin? Lee wrote:
 > UTF-8 is a default encoding for str.encode and bytes.decode. Latin-1
 > is the internal encoding in CPython whenever possible, and
 > PyASCIIObject is an internal struct in Python 3. It is not exactly
 > alien to Python to choose ASCII as a default. If it is a bad idea, it
 > is not original to me.

- cPython is not the only Python

- Latin-1 is an implementation detail, not a language guarantee

- PyASCIIObject is (probably) a name left over from Python 2 (massive
   renames of various structures is usually needless code churn)

- it may not have been a bad idea when Python was created, but it is a
   bad idea now



Please put your energy elsewhere because this particular is not going to 
change.

--
~Ethan~

From random832 at fastmail.com  Wed Jun 15 13:14:45 2016
From: random832 at fastmail.com (Random832)
Date: Wed, 15 Jun 2016 13:14:45 -0400
Subject: [Python-ideas] Strings can sometimes convert to bytes without
 an encoding
In-Reply-To: <20160615010634.GZ27919@ando.pearwood.info>
References: <CAB_e7ixmAuoqxr9wrOp8DjNuPrKT8vcPgkNnB+1+jm_Eh-t=VA@mail.gmail.com>
 <CAP7+vJ+W1gx5ZeaOUnPmnPxi2vDL2dvTxKC3Es_TjM0rpFfvzw@mail.gmail.com>
 <CAB_e7izvAchRLJZhuvMq7RNgZBFrjL1=0y2OEASPR5zyGuXHGQ@mail.gmail.com>
 <20160615010634.GZ27919@ando.pearwood.info>
Message-ID: <1466010885.374279.638709281.6ED38140@webmail.messagingengine.com>

On Tue, Jun 14, 2016, at 21:06, Steven D'Aprano wrote:
> On Tue, Jun 14, 2016 at 07:46:34PM -0400, Franklin? Lee wrote:
> > On Tue, Jun 14, 2016 at 7:26 PM, Guido van Rossum <guido at python.org> wrote:
> > > -1. Such a check for the contents of the string sounds exactly like the
> > > Python 2 behavior we are trying to get away with.
> > 
> > But isn't it really just converting back and forth between two
> > representations of the same thing? A str with char width 1 is
> > conceptually an ASCII string; 
> 
> How do you reason that? There are many one-character strings that aren't 
> ASCII, and encode to two or more bytes.

I think he's using "char width" to mean "width of the character type in
bytes", not "length of the string", and referring to implementation
details of the FSR. But at that point, why say it's conceptually ASCII
rather than conceptually Latin-1?

From srkunze at mail.de  Wed Jun 15 14:07:27 2016
From: srkunze at mail.de (Sven R. Kunze)
Date: Wed, 15 Jun 2016 20:07:27 +0200
Subject: [Python-ideas] Wild idea about mutability
In-Reply-To: <F97A782A-8B80-418B-B212-EF881904FEF1@trueblade.com>
References: <93119c3d-012a-387d-5540-db57d33eea24@btinternet.com>
 <20160602013420.GN12028@ando.pearwood.info>
 <cea9a367-0d17-9a3f-c2f8-1c852bad6cf5@btinternet.com>
 <57503AB5.7090708@mail.de>
 <CAH0mxTQGvAVH2OB46zrjzNSSNPwPPA_43UxebvMe47a8fT-SqQ@mail.gmail.com>
 <575520C4.2010900@mail.de>
 <CAJ3HoZ3_2L6AJw4mUDQYG6k4vRMQF_sR2TDXTc3m-c9-c95CvA@mail.gmail.com>
 <575553FA.9070807@mail.de>
 <8455818d-ce86-5978-abae-255bcc28e7c2@btinternet.com>
 <5758236E.7050408@mail.de>
 <F97A782A-8B80-418B-B212-EF881904FEF1@trueblade.com>
Message-ID: <5761995F.5010904@mail.de>

On 08.06.2016 16:33, Eric V. Smith wrote:
> See the rejected PEP-351 for one version of the freeze protocol.

Wow, still 11 years later equal thoughts arise again.


Reading through the rejection notes, I can see the concerns are as 
abstract as the use-cases (" good design", "net harmful effect", "Liskov 
violation", ...). I think the reservations are similar to those to other 
dynamic-increasing features like duck-typing, lambdas, closures: 
changing/defining an object right there where you need it to be 
changed/defined with the intend to convert it to a traditional (static) 
style later. I have to admit I don't like this style much but I catch 
myself doing it recently quite often. So, just for the sake of 
completeness, the concrete use-cases from my point of view:

1) debugging (like assert which theoretically is not necessary in 
production)

2) providing a correct/drop-in blue-print for immutable objects (the 
"intentional, essential part" from Raymond's post)

3) benefits of immutable objects (like faster development, 
error-reduction, etc.) <- not sure if that qualifies as a use-case


This, said. I for one would not be averse to a immutability 
mixin/freezing protocol although I don't see a point in producing 
freezed copies recursively as PEP-351 did.


Best,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160615/df41c69a/attachment.html>

From k7hoven at gmail.com  Wed Jun 15 14:39:04 2016
From: k7hoven at gmail.com (Koos Zevenhoven)
Date: Wed, 15 Jun 2016 21:39:04 +0300
Subject: [Python-ideas] Object for accessing identifiers/names
In-Reply-To: <20160603062245.GV12028@ando.pearwood.info>
References: <CAMiohohuQaL4UhJRS-ZNtLhG=cRky4LW=SWWEja02jNeM2JWBw@mail.gmail.com>
 <20160603062245.GV12028@ando.pearwood.info>
Message-ID: <CAMiohogE9XVKukTdb-A3fYWex+RKW-GBW7dYU+CG5K8nzrOSqg@mail.gmail.com>

[This almost two-week-old draft email was apparently waiting for me to
hit 'send']

On Fri, Jun 3, 2016 at 9:22 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Jun 01, 2016 at 10:22:39PM +0300, Koos Zevenhoven wrote:
>
>> Currently, there is no direct way to work with variable names from within
>> Python. Yes, you can fiddle with __dict__s and locals() and globals(),
>
> The usual way is to write the name as a string and operate on the
> namespace.
>
> That means that variables (names) are not themselves values: a name is
> bound to a value, but there is no value which is itself a name. Instead,
> we use a string as a form of indirection.
>

Indeed the point is to make it possible to, on demand, create a value
(object) representing a name.

>> but
>> there is no convenient general way. To solve this, there could be a way
>> (probably new syntax) for creating an object that
>> can examine and manipulate a name binding conveniently.
>
> Before getting too interested in the syntax for creating these, can you
> explain what you want to do with them? Most uses of names don't need to
> be values:
>
> spam = 23
> import spam
> del spam
>
> all work well on their own. Can you give some use-cases?
>

This is a variation of the 'given <name>' syntax that I proposed in
the matching syntax thread. So the truth value of the name object
would tell you whether the name is bound to a value or not, as I
explain in the OP of this thread too. But the __str__ of the name
object would give the name, so that would allow things like:

TypeVar(name_object)
sympy.Symbol(name_object)

and those constructors/callables would have access to the name in the
calling scope and would be able to bind the value to it without one
having to repeat oneself as in `alpha = sympy.Symbol('alpha')` etc.

This could also be used as instead of a callback that just needs to
assign to some variable in the calling scope (or the associated
closure).

You could potentially even `await` these things (until some other
thread or task sets the variable), although there is some overlap in
functionality with other awaitables of course.

I believe this might also help a little in the example that Michael
just posted in the other thread for matching and extracting values
within nested JSON-like structures.

I think there is more functionality that this could provide too.

[...]
>> Now this name_obj thing
>> could provide functionality like assigning to the name some_name, getting
>> the assigned object, and determining whether
>> something has been assigned to the name or not.
>
> Most of these things already work using strings:
>
> # bind a name to the global scope
> globals()[name_object] = 999
>
> # lookup a name in some module's namespace
> vars(module)[name_object]

But changing it does not work similarly. From the docs: """
Note
The contents of this dictionary should not be modified; changes may
not affect the values of local and free variables used by the
interpreter.
"""

> # check for existence
> name_object in vars(module)

Again from the docs: """
...however, other objects may have write restrictions on their
__dict__ attributes (for example, classes use a dictproxy to prevent
direct dictionary updates).
"""

>
> What functionality do you think is missing?
>

Well you need to understand locals(), globals(), vars() and __dict__
and their differences to be able to do these things. That's why these
existing functions are not very suitable for the uses I'm describing.
And they certainly don't do anything to help with TypeVar and
sympy.Symbol :-).

-- Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +

From ram at rachum.com  Fri Jun 17 02:00:27 2016
From: ram at rachum.com (Ram Rachum)
Date: Fri, 17 Jun 2016 09:00:27 +0300
Subject: [Python-ideas] collections.abc.Stream
Message-ID: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>

Is there a way to check in Python whether an object is file-like? (Like
`open`, `StringIO`, etc.) I would think that an abc in collections.abc will
be the standard solution, like we have for so many other data types, but I
can't find one.


Thanks,
Ram.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160617/daf4a23d/attachment.html>

From michael.selik at gmail.com  Fri Jun 17 10:38:14 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Fri, 17 Jun 2016 14:38:14 +0000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
Message-ID: <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>

On Fri, Jun 17, 2016, 2:01 AM Ram Rachum <ram at rachum.com> wrote:

> Is there a way to check in Python whether an object is file-like? (Like
> `open`, `StringIO`, etc.) I would think that an abc in collections.abc will
> be the standard solution, like we have for so many other data types, but I
> can't find one.
>

File-like means different things in different places. Some functions want
the read method, others want readline.

How about just trying to use the method and catching AttributeError?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160617/47ec96b1/attachment.html>

From bzvi7919 at gmail.com  Fri Jun 17 18:21:28 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Fri, 17 Jun 2016 22:21:28 +0000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
Message-ID: <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>

+1 although isn't io.IOBase already an ABC for file-like objects?

On Fri, Jun 17, 2016 at 5:38 PM Michael Selik <michael.selik at gmail.com>
wrote:

>
> On Fri, Jun 17, 2016, 2:01 AM Ram Rachum <ram at rachum.com> wrote:
>
>> Is there a way to check in Python whether an object is file-like? (Like
>> `open`, `StringIO`, etc.) I would think that an abc in collections.abc will
>> be the standard solution, like we have for so many other data types, but I
>> can't find one.
>>
>
> File-like means different things in different places. Some functions want
> the read method, others want readline.
>
> How about just trying to use the method and catching AttributeError?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160617/16276d65/attachment.html>

From ram at rachum.com  Fri Jun 17 20:36:36 2016
From: ram at rachum.com (Ram Rachum)
Date: Sat, 18 Jun 2016 03:36:36 +0300
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
Message-ID: <CANXboVZ+FfcvdRHj5Tk9jTViw6AFYn3RtWwnzY16Xd6JOBMyvw@mail.gmail.com>

Thanks for the io.IOBase reference!

On Sat, Jun 18, 2016 at 1:21 AM, Bar Harel <bzvi7919 at gmail.com> wrote:

> +1 although isn't io.IOBase already an ABC for file-like objects?
>
> On Fri, Jun 17, 2016 at 5:38 PM Michael Selik <michael.selik at gmail.com>
> wrote:
>
>>
>> On Fri, Jun 17, 2016, 2:01 AM Ram Rachum <ram at rachum.com> wrote:
>>
>>> Is there a way to check in Python whether an object is file-like? (Like
>>> `open`, `StringIO`, etc.) I would think that an abc in collections.abc will
>>> be the standard solution, like we have for so many other data types, but I
>>> can't find one.
>>>
>>
>> File-like means different things in different places. Some functions want
>> the read method, others want readline.
>>
>> How about just trying to use the method and catching AttributeError?
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/014cb568/attachment.html>

From bzvi7919 at gmail.com  Sat Jun 18 06:20:33 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Sat, 18 Jun 2016 10:20:33 +0000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CANXboVZ+FfcvdRHj5Tk9jTViw6AFYn3RtWwnzY16Xd6JOBMyvw@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CANXboVZ+FfcvdRHj5Tk9jTViw6AFYn3RtWwnzY16Xd6JOBMyvw@mail.gmail.com>
Message-ID: <CAMsGuymttEjeqa4V65gQbugCW5+8WQCZvbC+RAD5dsO+azM2XQ@mail.gmail.com>

You're welcome.

Perhaps there should be some kind of reference to it in collections.abc?
Maybe an alias even?

On Sat, Jun 18, 2016, 3:36 AM Ram Rachum <ram at rachum.com> wrote:

> Thanks for the io.IOBase reference!
>
> On Sat, Jun 18, 2016 at 1:21 AM, Bar Harel <bzvi7919 at gmail.com> wrote:
>
>> +1 although isn't io.IOBase already an ABC for file-like objects?
>>
>> On Fri, Jun 17, 2016 at 5:38 PM Michael Selik <michael.selik at gmail.com>
>> wrote:
>>
>>>
>>> On Fri, Jun 17, 2016, 2:01 AM Ram Rachum <ram at rachum.com> wrote:
>>>
>>>> Is there a way to check in Python whether an object is file-like? (Like
>>>> `open`, `StringIO`, etc.) I would think that an abc in collections.abc will
>>>> be the standard solution, like we have for so many other data types, but I
>>>> can't find one.
>>>>
>>>
>>> File-like means different things in different places. Some functions
>>> want the read method, others want readline.
>>>
>>> How about just trying to use the method and catching AttributeError?
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/cc642348/attachment.html>

From p.f.moore at gmail.com  Sat Jun 18 07:08:57 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 18 Jun 2016 12:08:57 +0100
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
Message-ID: <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>

On 17 June 2016 at 23:21, Bar Harel <bzvi7919 at gmail.com> wrote:
> +1 although isn't io.IOBase already an ABC for file-like objects?

To an extent, but it would likely be an error to typecheck for it.
There's a huge history of people implementing objects with (for
example) just a read() method and expecting it to work in functions
that need files but only actually use that one method. The idea of a
"file-like object" predates ABCs by such a long time that code that
defines "file-like" via any ABC will be seen as "broken" by someone
(for public libraries at least - for private code you can of course
follow whatever conventions you want).

> Perhaps there should be some kind of reference to it in collections.abc? Maybe an alias even?

I would think that would be a mistake. Far too often it wouldn't do
what people would expect, so it'd end up being more of a hindrance
than a help.

Paul

From bzvi7919 at gmail.com  Sat Jun 18 07:16:51 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Sat, 18 Jun 2016 11:16:51 +0000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
Message-ID: <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>

Hmm... If the term "file-like object" is so ambiguous, why does the STL use
it so much?
Perhaps a new term should be introduced clearly stating the required
methods.

On Sat, Jun 18, 2016, 2:08 PM Paul Moore <p.f.moore at gmail.com> wrote:

> On 17 June 2016 at 23:21, Bar Harel <bzvi7919 at gmail.com> wrote:
> > +1 although isn't io.IOBase already an ABC for file-like objects?
>
> To an extent, but it would likely be an error to typecheck for it.
> There's a huge history of people implementing objects with (for
> example) just a read() method and expecting it to work in functions
> that need files but only actually use that one method. The idea of a
> "file-like object" predates ABCs by such a long time that code that
> defines "file-like" via any ABC will be seen as "broken" by someone
> (for public libraries at least - for private code you can of course
> follow whatever conventions you want).
>
> > Perhaps there should be some kind of reference to it in collections.abc?
> Maybe an alias even?
>
> I would think that would be a mistake. Far too often it wouldn't do
> what people would expect, so it'd end up being more of a hindrance
> than a help.
>
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/0f1ad67c/attachment.html>

From p.f.moore at gmail.com  Sat Jun 18 07:27:05 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 18 Jun 2016 12:27:05 +0100
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
Message-ID: <CACac1F_Wn7bYVTDKxdmBBHs-G9-42UQE17gjyxFB+PMB3p1uJA@mail.gmail.com>

On 18 June 2016 at 12:16, Bar Harel <bzvi7919 at gmail.com> wrote:
> Hmm... If the term "file-like object" is so ambiguous, why does the STL use
> it so much?

Not sure what you mean by the STL? Do you mean the standard library?
Because it was an easily-understandable term to use, even though it
does require context (and sometimes a little bit of trial and error,
or reading the source) to understand the precise requirements.
Historically, the stdlib docs were somewhat more informal in their
language than people seem to expect these days. (Personally, I think a
bit of informality is fine, but opinions differ).

> Perhaps a new term should be introduced clearly stating the required
> methods.

The point is there would be many separate terms, each stating
different sets of methods. Or people would be required to implement
unnecessary methods just to satisfy the constraints of an
overly-strict type check.

This is the fundamental idea behind duck typing - you only need to
implement the methods needed by the code you're calling.
Paul

From bzvi7919 at gmail.com  Sat Jun 18 07:47:12 2016
From: bzvi7919 at gmail.com (Bar Harel)
Date: Sat, 18 Jun 2016 11:47:12 +0000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CACac1F_Wn7bYVTDKxdmBBHs-G9-42UQE17gjyxFB+PMB3p1uJA@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
 <CACac1F_Wn7bYVTDKxdmBBHs-G9-42UQE17gjyxFB+PMB3p1uJA@mail.gmail.com>
Message-ID: <CAMsGuy=4ZOSnoqV6p-U8zs5CAxAYst2aUHAtmBUK=hPSKeJu8A@mail.gmail.com>

>
> This is the fundamental idea behind duck typing - you only need to
> implement the methods needed by the code you're calling.
>

But then again, the problem is that you don't know what methods are needed
by the code. Even if you don't type-check, an ABC gives you a general idea
of the methods needed for the code to work whether you inherit from it, or
just look at it.
I believe trial & error, and looking through the source code is a poor
solution compared to the precise documentation of the required methods.

On Sat, Jun 18, 2016 at 2:27 PM Paul Moore <p.f.moore at gmail.com> wrote:

> On 18 June 2016 at 12:16, Bar Harel <bzvi7919 at gmail.com> wrote:
> > Hmm... If the term "file-like object" is so ambiguous, why does the STL
> use
> > it so much?
>
> Not sure what you mean by the STL? Do you mean the standard library?
> Because it was an easily-understandable term to use, even though it
> does require context (and sometimes a little bit of trial and error,
> or reading the source) to understand the precise requirements.
> Historically, the stdlib docs were somewhat more informal in their
> language than people seem to expect these days. (Personally, I think a
> bit of informality is fine, but opinions differ).
>
> > Perhaps a new term should be introduced clearly stating the required
> > methods.
>
> The point is there would be many separate terms, each stating
> different sets of methods. Or people would be required to implement
> unnecessary methods just to satisfy the constraints of an
> overly-strict type check.
>
> This is the fundamental idea behind duck typing - you only need to
> implement the methods needed by the code you're calling.
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/2ceb7085/attachment.html>

From steve at pearwood.info  Sat Jun 18 08:18:43 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jun 2016 22:18:43 +1000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAMsGuy=4ZOSnoqV6p-U8zs5CAxAYst2aUHAtmBUK=hPSKeJu8A@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
 <CACac1F_Wn7bYVTDKxdmBBHs-G9-42UQE17gjyxFB+PMB3p1uJA@mail.gmail.com>
 <CAMsGuy=4ZOSnoqV6p-U8zs5CAxAYst2aUHAtmBUK=hPSKeJu8A@mail.gmail.com>
Message-ID: <20160618121842.GL27919@ando.pearwood.info>

On Sat, Jun 18, 2016 at 11:47:12AM +0000, Bar Harel wrote:
> >
> > This is the fundamental idea behind duck typing - you only need to
> > implement the methods needed by the code you're calling.
> >
> 
> But then again, the problem is that you don't know what methods are needed
> by the code.

It is the responsiblity of the code to document what it needs. It can 
document that it needs an int or a str, or an object with a read() 
method.


-- 
Steve

From steve at pearwood.info  Sat Jun 18 08:16:04 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 18 Jun 2016 22:16:04 +1000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
Message-ID: <20160618121604.GK27919@ando.pearwood.info>

On Sat, Jun 18, 2016 at 11:16:51AM +0000, Bar Harel wrote:
> Hmm... If the term "file-like object" is so ambiguous, why does the STL use
> it so much?

I don't know. What is the STL and how does it relate to Python?

That's a rhetorical question. I know that the STL is C++'s Standard 
Template Library, which has very little to do with Python. Their use of 
the term "file-like object" is not necessarily identical to Python's use 
of it.


> Perhaps a new term should be introduced clearly stating the required
> methods.

But that's the whole point -- there is no one standard set of required 
methods. Some uses of a "file-like object" just require a read() or 
write() method. Some might require a close() method, or a flush(), 
or both. Some might need seek() and/or tell(). There's very little point 
in insisting on the full interface provided by open(...) if all you need 
is to call the write() method.



-- 
Steve

From michael.selik at gmail.com  Sat Jun 18 10:29:37 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Sat, 18 Jun 2016 14:29:37 +0000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAMsGuy=4ZOSnoqV6p-U8zs5CAxAYst2aUHAtmBUK=hPSKeJu8A@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
 <CACac1F_Wn7bYVTDKxdmBBHs-G9-42UQE17gjyxFB+PMB3p1uJA@mail.gmail.com>
 <CAMsGuy=4ZOSnoqV6p-U8zs5CAxAYst2aUHAtmBUK=hPSKeJu8A@mail.gmail.com>
Message-ID: <CAGgTfkMJEyzqCd0MkN0=Zmwd+fMR=jrFMDLHBa2wz9OGWDVqdg@mail.gmail.com>

On Sat, Jun 18, 2016 at 7:47 AM Bar Harel <bzvi7919 at gmail.com> wrote:

> This is the fundamental idea behind duck typing - you only need to
>> implement the methods needed by the code you're calling.
>>
>
> But then again, the problem is that you don't know what methods are needed
> by the code.
>

Pass a stub object into the function and it will raise AttributeError or
TypeError. That's the best way to learn about what interface is necessary.


> I believe trial & error, and looking through the source code is a poor
> solution compared to the precise documentation of the required methods.
>

Documentation can go stale very easily. An AttributeError never lies; it
tells you exactly which method you need to implement.

Take a look at IOBase. There's a whole lot going on there that's
unnecessary for a function that just expects a ``read`` method.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/4c7321c6/attachment.html>

From egregius313 at gmail.com  Sat Jun 18 12:50:54 2016
From: egregius313 at gmail.com (Ed Minnix)
Date: Sat, 18 Jun 2016 12:50:54 -0400
Subject: [Python-ideas] Why aren't slices hashable?
Message-ID: <EB5F73EC-734A-44E1-B4B9-282CA8433F8B@gmail.com>

Hello,

I was experimenting in the repl and I noticed something odd: slices are not hashable. I was wondering why this would be since, I would think slices are immutable, and most immutable objects are hashable.

Thanks,
Ed

From michael.selik at gmail.com  Sat Jun 18 13:11:24 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Sat, 18 Jun 2016 17:11:24 +0000
Subject: [Python-ideas] Why aren't slices hashable?
In-Reply-To: <EB5F73EC-734A-44E1-B4B9-282CA8433F8B@gmail.com>
References: <EB5F73EC-734A-44E1-B4B9-282CA8433F8B@gmail.com>
Message-ID: <CAGgTfkMnB7YpTFdV104R-2hE70VPT+vDv6wHB_Rv2LpZbi38rw@mail.gmail.com>

On Sat, Jun 18, 2016 at 12:51 PM Ed Minnix <egregius313 at gmail.com> wrote:

> I was experimenting in the repl and I noticed something odd: slices are
> not hashable. I was wondering why this would be since, I would think slices
> are immutable, and most immutable objects are hashable.
>

Apparently that was to avoid mistakes. Otherwise there would be a bit of
confusion between assigning to a slice as the key or changing many keys.

https://bugs.python.org/issue408326
http://stackoverflow.com/questions/29980786/why-are-slice-objects-not-hashable-in-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/9d65bbf5/attachment.html>

From rosuav at gmail.com  Sat Jun 18 14:53:05 2016
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 19 Jun 2016 04:53:05 +1000
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <20160618121604.GK27919@ando.pearwood.info>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
 <20160618121604.GK27919@ando.pearwood.info>
Message-ID: <CAPTjJmrT3e4AtckU9oO-taUHKy+UH=fVp07rK8UAsJXNJuVCSQ@mail.gmail.com>

On Sat, Jun 18, 2016 at 10:16 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> But that's the whole point -- there is no one standard set of required
> methods. Some uses of a "file-like object" just require a read() or
> write() method. Some might require a close() method, or a flush(),
> or both. Some might need seek() and/or tell(). There's very little point
> in insisting on the full interface provided by open(...) if all you need
> is to call the write() method.

So then the question is: What can I provide to something which wants a
"file-like object"? Is an io.StringIO valid for this function, or for
that function? It's the most file-like non-file that I can think of,
so I'd generally expect it to work in most situations; it appears to
support a lot of operations. What about io.BytesIO? Right there, we
have to distinguish between "binary file-like objects" and "text
file-like objects". And with other classes, how do you know what you
need? The term "file-like object" is unfortunately made somewhat
useless by its variant uses.

ChrisA

From guido at python.org  Sat Jun 18 20:24:39 2016
From: guido at python.org (Guido van Rossum)
Date: Sat, 18 Jun 2016 17:24:39 -0700
Subject: [Python-ideas] collections.abc.Stream
In-Reply-To: <CAPTjJmrT3e4AtckU9oO-taUHKy+UH=fVp07rK8UAsJXNJuVCSQ@mail.gmail.com>
References: <CANXboVaObO9M47EgBMST06sQ1oATU=xomATtTw2xZpG73emz_w@mail.gmail.com>
 <CAGgTfkOUM=O_w8Qs86no5xePN=EPpQfBNY8U-VGaOgm5G5x2Ew@mail.gmail.com>
 <CAMsGuykELkrKa_fc4hE3Oo5+rH6j_JXqoHV+BpDge=nihBZqXA@mail.gmail.com>
 <CACac1F-MsKvU5pAiSZfVner_4grt_bSZKnK61CUJuKZh=NnipQ@mail.gmail.com>
 <CAMsGuymbEv7CdXcMCbLVRY1t+HG3ojeKBQQc-CF+2ty8tAS+sg@mail.gmail.com>
 <20160618121604.GK27919@ando.pearwood.info>
 <CAPTjJmrT3e4AtckU9oO-taUHKy+UH=fVp07rK8UAsJXNJuVCSQ@mail.gmail.com>
Message-ID: <CAP7+vJ+ur0f5w1sYkBidPQ+q2mLrdCRTNRTz4EOOk0O0vHoTbA@mail.gmail.com>

I think it may be time to gently phase out "file-like" from the stdlib
documentation, in favor of more precise terms like (text | binary) (input |
output) stream.

There are probably a handful of cases where it's worth restricting the
(duck) type more, e.g. to "an object with a readline() method returning
str". But in most cases for the stdlib I favor specifying a relatively wide
interface so that future evolution of the documented module is not
constrained by old documentation.

For example, an API that takes a writable binary stream might currently
only call .readline() on that stream, but in the future might find it more
convenient to use .readlines() or .read().

As to whether we should just stick with the ABCs from 'io', there are a few
problems with that, so I don't want to go there.

First, these ABCs don't distinguish between input and output. This is
intentional because otherwise there would be a bewildering set of
combinations, since for each form would have to support input, output, or
both -- and we already have more than enough ABCs here (IOBase, RawIOBase,
BufferedIOBase, TextIOBase -- I really don't want to have to remember 12 of
these).

We also need to be sensitive to the use of duck typing (which is
particularly common for I/O streams) -- while it's fine to formally define
an API as taking e.g. a text input stream, it's usually not fine to start
asserting that it must be an instance of TextIOBase -- ABC.register()
notwithstanding, that would break a lot of code. (Type annotations are a
different story -- but except for brand new APIs, the stdlib should not yet
start using those.)

PS. I assume the reference to 'STL' was meant to be 'stdlib', an innocent
mistake by someone more familiar with C++, not something to be picked apart.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160618/f7a88f68/attachment.html>

From steve at pearwood.info  Sun Jun 19 11:02:52 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 20 Jun 2016 01:02:52 +1000
Subject: [Python-ideas] Make Python front end to the C math module
Message-ID: <20160619150251.GS27919@ando.pearwood.info>

Currently the math module is written in C, not Python. This greatly 
increases the barrier to contributions to math. I suggest that we treat 
the existing math module as an accelerator module:

- rename math.so to _math.so;
  (or whatever platform-specific name it uses)

- add math.py with something like the following content:

from _math import *


and we're done! That will make it easier for people to contribute pure 
Python functions in the future. Currently all math functions, regardless 
of how trivial or difficult, must be written in C. With this change, 
they can be written in Python, and only written in C if necessary.

For the avoidance of doubt, there is NO requirement for pure-Python 
implementations of the existing math functions, or even future ones. 
Future functions can go straight into the C implementation is desired. 
This is to allow future functions the opportunity to be written in 
Python.



-- 
Steve

From tritium-list at sdamon.com  Sun Jun 19 12:19:47 2016
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Sun, 19 Jun 2016 12:19:47 -0400
Subject: [Python-ideas] Make Python front end to the C math module
In-Reply-To: <20160619150251.GS27919@ando.pearwood.info>
References: <20160619150251.GS27919@ando.pearwood.info>
Message-ID: <000d01d1ca46$65f71c70$31e55550$@hotmail.com>

Is the math library really the place to put functions that are not directly
related to interacting with libm?

> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Steven D'Aprano
> Sent: Sunday, June 19, 2016 11:03 AM
> To: python-ideas at python.org
> Subject: [Python-ideas] Make Python front end to the C math module
> 
> Currently the math module is written in C, not Python. This greatly
> increases the barrier to contributions to math. I suggest that we treat
> the existing math module as an accelerator module:
> 
> - rename math.so to _math.so;
>   (or whatever platform-specific name it uses)
> 
> - add math.py with something like the following content:
> 
> from _math import *
> 
> 
> and we're done! That will make it easier for people to contribute pure
> Python functions in the future. Currently all math functions, regardless
> of how trivial or difficult, must be written in C. With this change,
> they can be written in Python, and only written in C if necessary.
> 
> For the avoidance of doubt, there is NO requirement for pure-Python
> implementations of the existing math functions, or even future ones.
> Future functions can go straight into the C implementation is desired.
> This is to allow future functions the opportunity to be written in
> Python.
> 
> 
> 
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From p.f.moore at gmail.com  Sun Jun 19 13:04:47 2016
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 19 Jun 2016 18:04:47 +0100
Subject: [Python-ideas] Make Python front end to the C math module
In-Reply-To: <20160619150251.GS27919@ando.pearwood.info>
References: <20160619150251.GS27919@ando.pearwood.info>
Message-ID: <CACac1F_iB6szbcS=Y7+DykZgcA=5cnD5BoOqVb4XVBzxhmY6Uw@mail.gmail.com>

On 19 June 2016 at 16:02, Steven D'Aprano <steve at pearwood.info> wrote:
>
> For the avoidance of doubt, there is NO requirement for pure-Python
> implementations of the existing math functions, or even future ones.
> Future functions can go straight into the C implementation is desired.
> This is to allow future functions the opportunity to be written in
> Python.

Is there any reason to do this in advance of an actual requirement for
it? It seems to me that doing this when someone proposes a
Python-coded math function would be fine. Or do you think that the
fact that math is coded in C is stopping people even suggesting
additions, because they believe they'd need to be able to write them
in C?

Paul

From steve at pearwood.info  Sun Jun 19 21:11:54 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 20 Jun 2016 11:11:54 +1000
Subject: [Python-ideas] Make Python front end to the C math module
In-Reply-To: <CACac1F_iB6szbcS=Y7+DykZgcA=5cnD5BoOqVb4XVBzxhmY6Uw@mail.gmail.com>
References: <20160619150251.GS27919@ando.pearwood.info>
 <CACac1F_iB6szbcS=Y7+DykZgcA=5cnD5BoOqVb4XVBzxhmY6Uw@mail.gmail.com>
Message-ID: <20160620011153.GT27919@ando.pearwood.info>

On Sun, Jun 19, 2016 at 06:04:47PM +0100, Paul Moore wrote:
> On 19 June 2016 at 16:02, Steven D'Aprano <steve at pearwood.info> wrote:
> >
> > For the avoidance of doubt, there is NO requirement for pure-Python
> > implementations of the existing math functions, or even future ones.
> > Future functions can go straight into the C implementation is desired.
> > This is to allow future functions the opportunity to be written in
> > Python.
> 
> Is there any reason to do this in advance of an actual requirement for
> it? It seems to me that doing this when someone proposes a
> Python-coded math function would be fine.

Now that you mention it, I do have an actual requirement:

http://bugs.python.org/issue27353


> Or do you think that the
> fact that math is coded in C is stopping people even suggesting
> additions, because they believe they'd need to be able to write them
> in C?

Its hard to say whether its stopping people from *proposing* new 
functionality, since talk is cheap. But its certainly a barrier to 
having them propose actual patches, since they must be written in C.

To anticipate a likely objection, that math was originally conceived as 
a platform-specific thin wrapper around the C math library, I think 
that's well and truly past:

- math is not limited to the C89 math library;
- but nor does it contain all of the C99 math library (there's
  quite a bit missing);
- and it includes functions not found in either C89 or C99.

Python tries to smooth out platform specific differences regarding 
float, e.g. float('nan') works on all platforms, not just POSIX (Windows 
used to require '1.#IND').

I think it is fair to say that math will be closely aligned to your 
platform's C math library, but it is not restricted to that. I think the 
current documentation is clear:

CPython implementation detail: The math module consists MOSTLY of thin 
wrappers around the platform C math library functions. [emphasis added]

https://docs.python.org/3/library/math.html#constants


-- 
Steve

From guido at python.org  Sun Jun 19 23:07:44 2016
From: guido at python.org (Guido van Rossum)
Date: Sun, 19 Jun 2016 20:07:44 -0700
Subject: [Python-ideas] Make Python front end to the C math module
In-Reply-To: <20160620011153.GT27919@ando.pearwood.info>
References: <20160619150251.GS27919@ando.pearwood.info>
 <CACac1F_iB6szbcS=Y7+DykZgcA=5cnD5BoOqVb4XVBzxhmY6Uw@mail.gmail.com>
 <20160620011153.GT27919@ando.pearwood.info>
Message-ID: <CAP7+vJJonf00UWw3h+2aqmSSYt=6JrGBNJnrSjcqt7Qu3wdGzA@mail.gmail.com>

I see your point, but I'm skeptical that it'll make much of a difference,
and I do see a cost: "import math" would become a little slower and require
a little more memory.

If your n'th root function is a classic floating point algorithm, surely
there's someone who can help you translate it into C (just ask), but just
as surely there's probably already some public-domain C code you can copy.
(Is it this? https://en.wikipedia.org/wiki/Nth_root#nth_root_algorithm)

I think the math module ought to stay close to a libm wrapper in spirit (if
not in implementation) because I don't really think that it ought to open
itself up to all of mathematics -- there's simply too much of that. I do
think a more accurate n'th root algorithm makes a fine addition, but I
don't think such additions are common enough to try to encourage
contributions.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160619/9da60d95/attachment.html>

From chris.barker at noaa.gov  Tue Jun 21 01:35:38 2016
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Mon, 20 Jun 2016 22:35:38 -0700
Subject: [Python-ideas] Make Python front end to the C math module
In-Reply-To: <CACac1F_iB6szbcS=Y7+DykZgcA=5cnD5BoOqVb4XVBzxhmY6Uw@mail.gmail.com>
References: <20160619150251.GS27919@ando.pearwood.info>
 <CACac1F_iB6szbcS=Y7+DykZgcA=5cnD5BoOqVb4XVBzxhmY6Uw@mail.gmail.com>
Message-ID: <8774589768421002132@unknownmsgid>

Sent from my iPhone
>
> Is there any reason to do this in advance of an actual requirement for
> it?

This was brought up last year when we added the isclose() function. In
fact Victor even whipped up an implementation.

Guido rejected the idea, and isclose() was implemented in C.

-CHB




> It seems to me that doing this when someone proposes a
> Python-coded math function would be fine. Or do you think that the
> fact that math is coded in C is stopping people even suggesting
> additions, because they believe they'd need to be able to write them
> in C?
>
> Paul
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From victor.stinner at gmail.com  Tue Jun 21 17:07:47 2016
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 21 Jun 2016 23:07:47 +0200
Subject: [Python-ideas] package management is not python-dev's problem
 :-(
In-Reply-To: <CAMpsgwY+PJ=9-x-QUw8Hp_KAsCy72t_he6TUg5nrngxvkhCw4A@mail.gmail.com>
References: <20160603131730.GA29255@python.ca>
 <087001d1be4c$99423520$cbc69f60$@hotmail.com>
 <20160604082715.GB20628@python.ca>
 <5756A524.1010600@thomas-guettler.de>
 <20160607113440.GF12028@ando.pearwood.info>
 <57571228.3030806@thomas-guettler.de>
 <CADiSq7eaNPn2Axg9Ugmn=hpjasBt0m6CM1i01ywiXDpYJREJWg@mail.gmail.com>
 <5757FB2B.90604@thomas-guettler.de>
 <CAMpsgwZLu9LEa91ji8yksckfJq+uixTqodJVDt_20SVMTSTLqg@mail.gmail.com>
 <CAMpsgwY+PJ=9-x-QUw8Hp_KAsCy72t_he6TUg5nrngxvkhCw4A@mail.gmail.com>
Message-ID: <CAMpsgwYAKxzfBq=4wurJp8htSsRkc65nwZoXTBRFYDYsRmBOSQ@mail.gmail.com>

Le 8 juin 2016 1:02 PM, "Thomas G?ttler" <guettliml at thomas-guettler.de> a
?crit :
> This statement make me sad. Why is this not python-dev's problem?

It was the case in the past, but most discussions ended with bikeshedding,
trolls, etc. instead of trying to discuss solutions to fix issues.

Since distutils-sig was created, many PEP were aproved and pip evolved
faster than before.

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160621/c0dbb8e0/attachment.html>

From victor.stinner at gmail.com  Tue Jun 21 17:14:08 2016
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 21 Jun 2016 23:14:08 +0200
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
Message-ID: <CAMpsgwa3Ghb2YmKpB-LkLySEJ+xtsbNRDGQVhWLY++hg5hgOTA@mail.gmail.com>

Le 9 juin 2016 9:48 PM, "Robert Collins" <robertc at robertcollins.net> a
?crit :
> The problem, as Cory demonstrates, is that purity is transitive: you
> have to know that all the runtime interactions are also pure, or your
> function stops being pure.

FAT Python implements optimizations on pure functions like inlining but
disable optimizations as soon as an assumption becomes false. There is no
magic, it runs checks at runtime using guards.

JIT compilers do the same.

Some projects like numba or pythran don't implement guards, but it's a
deliberate choice to maximize performance. It's acceptable because they use
opt-in optimizations.

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160621/dc06d07b/attachment.html>

From jlehtosalo at gmail.com  Wed Jun 22 06:23:46 2016
From: jlehtosalo at gmail.com (Jukka Lehtosalo)
Date: Wed, 22 Jun 2016 11:23:46 +0100
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <87d1npe2g1.fsf@grothesque.org>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 <87d1npe2g1.fsf@grothesque.org>
Message-ID: <CAA_f+Lz9pn7BNU5X7EQ0Hp8iSFB31p+K96kt=4Z6bB2WWVB5Uw@mail.gmail.com>

On Fri, Jun 10, 2016 at 11:57 AM, Christoph Groth <christoph at grothesque.org>
wrote:

> This means that any pure function would have to accept as parameters
>> everything it operates on, and you'd need to error on any call to a pure
>> function with impure arguments.
>>
>
> You mean because one can never be sure about the purity of global
> variables?  That's true, but exactly to the same extent that one cannot be
> sure about the typing annotations of global variables.
>
> I believe that such "weak" checking for purity would still be useful, just
> as the equally weak verification of typing by mypy is useful.
>

I'm late to the discussion, but I agree with Christoph. I doubt purity
checking can reliably enforce anything without making it impractically
restrictive, but even best-effort, partial purity checking could be useful,
similar to how mypy is useful. A checker could catch many (but not all)
purity violations and purity annotations would make programs easier to
reason about for humans and tools. I've thought about this a bit in the
past -- mostly in the context of mypy. I'm not sure whether purity checking
would be more than marginally useful, but at least in some use cases it
might be a valuable a tool. For example, purity checking might help with
reasoning about concurrency and parallelism. However, writing a purity
checking tool for Python that is flexible enough that many programmers
would actually want to use it sounds like a difficult (research?) project.

Here are some practical considerations:

1) We'd need to annotate library functions with respect to purity, as
Python derives much of its value from the available libraries, and we'd
want pure functions to be able to use (pure) libraries. Perhaps stubs in
typeshed would have purity annotations for individual functions.
Alternatively, there could be a typeshed-like repository for purity
information only, and a tool could use information from both repositories.
The safest thing would be to default to impure, unless a function is
explicitly annotated as pure.

2) Type annotations would likely be needed for practical checking of
purity, as otherwise things like method calls would be hard to check for
purity. Thus purity checking would be a stricter form of static type
checking.

3) For consistency with PEP 484 style gradual typing, we may want to allow
functions without annotations to be called from pure functions. Tools might
make this user-configurable, as this is clearly a big hole in a purity
checking system.

4) It may be necessary to allow impure operations to be used within a pure
function as long the impurity won't be visible outside the function. For
example, perhaps this function should be fine, as an external observer
can't tell whether the function uses pure operations only:

def squares(n: int) -> List[int]:
    a = []
    for i in range(n):
        a.append(i * i)
    return a

I`m not sure if I know how to implement this in a useful way, though. I
believe that there is some prior work in this area (not specific to Python).

Jukka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160622/430342f9/attachment.html>

From liik.joonas at gmail.com  Wed Jun 22 08:39:04 2016
From: liik.joonas at gmail.com (Joonas Liik)
Date: Wed, 22 Jun 2016 15:39:04 +0300
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAA_f+Lz9pn7BNU5X7EQ0Hp8iSFB31p+K96kt=4Z6bB2WWVB5Uw@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 <87d1npe2g1.fsf@grothesque.org>
 <CAA_f+Lz9pn7BNU5X7EQ0Hp8iSFB31p+K96kt=4Z6bB2WWVB5Uw@mail.gmail.com>
Message-ID: <CAB1GNpT-Or5icWhA4=4diUE0AFh3jP9291CHMvjxEskd00o4Fg@mail.gmail.com>

On 22 June 2016 at 13:23, Jukka Lehtosalo <jlehtosalo at gmail.com> wrote:
> On Fri, Jun 10, 2016 at 11:57 AM, Christoph Groth <christoph at grothesque.org>
> wrote:
>>>
>>> This means that any pure function would have to accept as parameters
>>> everything it operates on, and you'd need to error on any call to a pure
>>> function with impure arguments.
>>
>>
>> You mean because one can never be sure about the purity of global
>> variables?  That's true, but exactly to the same extent that one cannot be
>> sure about the typing annotations of global variables.
>>
>> I believe that such "weak" checking for purity would still be useful, just
>> as the equally weak verification of typing by mypy is useful.
>
>
> I'm late to the discussion, but I agree with Christoph. I doubt purity
> checking can reliably enforce anything without making it impractically
> restrictive, but even best-effort, partial purity checking could be useful,
> similar to how mypy is useful. A checker could catch many (but not all)
> purity violations and purity annotations would make programs easier to
> reason about for humans and tools. I've thought about this a bit in the past
> -- mostly in the context of mypy. I'm not sure whether purity checking would
> be more than marginally useful, but at least in some use cases it might be a
> valuable a tool. For example, purity checking might help with reasoning
> about concurrency and parallelism. However, writing a purity checking tool
> for Python that is flexible enough that many programmers would actually want
> to use it sounds like a difficult (research?) project.
>
> Here are some practical considerations:
>
> 1) We'd need to annotate library functions with respect to purity, as Python
> derives much of its value from the available libraries, and we'd want pure
> functions to be able to use (pure) libraries. Perhaps stubs in typeshed
> would have purity annotations for individual functions. Alternatively, there
> could be a typeshed-like repository for purity information only, and a tool
> could use information from both repositories. The safest thing would be to
> default to impure, unless a function is explicitly annotated as pure.
>
> 2) Type annotations would likely be needed for practical checking of purity,
> as otherwise things like method calls would be hard to check for purity.
> Thus purity checking would be a stricter form of static type checking.
>
> 3) For consistency with PEP 484 style gradual typing, we may want to allow
> functions without annotations to be called from pure functions. Tools might
> make this user-configurable, as this is clearly a big hole in a purity
> checking system.
>
> 4) It may be necessary to allow impure operations to be used within a pure
> function as long the impurity won't be visible outside the function. For
> example, perhaps this function should be fine, as an external observer can't
> tell whether the function uses pure operations only:
>
> def squares(n: int) -> List[int]:
>     a = []
>     for i in range(n):
>         a.append(i * i)
>     return a
>
> I`m not sure if I know how to implement this in a useful way, though. I
> believe that there is some prior work in this area (not specific to Python).
>
> Jukka
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

we could have eg:

def parallel_map(f, data):
    if is_pure(f):
        # <impl that does special optimisations to take advantage of
the fact f is truly pure>
        # special case that uses light weight threads or sth (maybe
can run bits of it GIL free or sth, less overhead or sth)
        # its ok to share immutable data
    else:
        #<impl that has no issue with impure f>
        # eg: multiprocessing process pools..
        #its ok to mutate everything becauuse we no longer truly share anything.

in other words, even if only a small subset of code can be proven pure
it can be useful to help optimise code (in this example to avoid
locking by knowing that no 2 parrallel instances of the callback can
mock with mutually shared state)

for some scenarios it - might be - ok even if we leave the proving to
the programmer. (eg a C extension could implement the optimised
parallel map in this example and a user defined decorator is enough to
shove a flag on to the function f to trigger the optimised code path)
if this it is not truly pure tho despite the assertion it might cause
issues so in that case it probably is not something that should ship
with python by default?

From cfkaran2 at gmail.com  Wed Jun 22 10:05:10 2016
From: cfkaran2 at gmail.com (CFK)
Date: Wed, 22 Jun 2016 10:05:10 -0400
Subject: [Python-ideas] Statically checking purity of functions
In-Reply-To: <CAENTz_X1s1Haw7xiFMhZNqWES_kneLpyWLr7P_mdwYTNqUKchQ@mail.gmail.com>
References: <87eg86tre0.fsf@grothesque.org>
 <CAJ3HoZ1JE1J5a6vzfEww2u=gerRGpF3JVHtvpCWbab_7xtPFmg@mail.gmail.com>
 <87d1npe2g1.fsf@grothesque.org>
 <CAA_f+Lz9pn7BNU5X7EQ0Hp8iSFB31p+K96kt=4Z6bB2WWVB5Uw@mail.gmail.com>
 <CAB1GNpT-Or5icWhA4=4diUE0AFh3jP9291CHMvjxEskd00o4Fg@mail.gmail.com>
 <CAENTz_X1s1Haw7xiFMhZNqWES_kneLpyWLr7P_mdwYTNqUKchQ@mail.gmail.com>
Message-ID: <CAENTz_X-Pp3otA0Gcihz0vqbe3tSOuEfKO8y=QWXTCvsQYzZgA@mail.gmail.com>

Why does the checking have to be static? How about a 'trust but verify'
approach? If a function is marked pure, python trusts that it is, but turns
on code to check for side effects at runtime. If impure code is executed in
a pure function, then an exception is raised. That will deal with the weird
corner cases brought up earlier, at the cost of not being able to
statically type check code.  It will also deal with old code that wasn't
marked, but happens to be pure.

Not sure how to write the checks though; maybe with Perl's data tainting
approach?

Thanks,
Cem Karan
On Jun 22, 2016 8:39 AM, "Joonas Liik" <liik.joonas at gmail.com> wrote:

On 22 June 2016 at 13:23, Jukka Lehtosalo <jlehtosalo at gmail.com> wrote:
> On Fri, Jun 10, 2016 at 11:57 AM, Christoph Groth <
christoph at grothesque.org>
> wrote:
>>>
>>> This means that any pure function would have to accept as parameters
>>> everything it operates on, and you'd need to error on any call to a pure
>>> function with impure arguments.
>>
>>
>> You mean because one can never be sure about the purity of global
>> variables?  That's true, but exactly to the same extent that one cannot
be
>> sure about the typing annotations of global variables.
>>
>> I believe that such "weak" checking for purity would still be useful,
just
>> as the equally weak verification of typing by mypy is useful.
>
>
> I'm late to the discussion, but I agree with Christoph. I doubt purity
> checking can reliably enforce anything without making it impractically
> restrictive, but even best-effort, partial purity checking could be
useful,
> similar to how mypy is useful. A checker could catch many (but not all)
> purity violations and purity annotations would make programs easier to
> reason about for humans and tools. I've thought about this a bit in the
past
> -- mostly in the context of mypy. I'm not sure whether purity checking
would
> be more than marginally useful, but at least in some use cases it might
be a
> valuable a tool. For example, purity checking might help with reasoning
> about concurrency and parallelism. However, writing a purity checking tool
> for Python that is flexible enough that many programmers would actually
want
> to use it sounds like a difficult (research?) project.
>
> Here are some practical considerations:
>
> 1) We'd need to annotate library functions with respect to purity, as
Python
> derives much of its value from the available libraries, and we'd want pure
> functions to be able to use (pure) libraries. Perhaps stubs in typeshed
> would have purity annotations for individual functions. Alternatively,
there
> could be a typeshed-like repository for purity information only, and a
tool
> could use information from both repositories. The safest thing would be to
> default to impure, unless a function is explicitly annotated as pure.
>
> 2) Type annotations would likely be needed for practical checking of
purity,
> as otherwise things like method calls would be hard to check for purity.
> Thus purity checking would be a stricter form of static type checking.
>
> 3) For consistency with PEP 484 style gradual typing, we may want to allow
> functions without annotations to be called from pure functions. Tools
might
> make this user-configurable, as this is clearly a big hole in a purity
> checking system.
>
> 4) It may be necessary to allow impure operations to be used within a pure
> function as long the impurity won't be visible outside the function. For
> example, perhaps this function should be fine, as an external observer
can't
> tell whether the function uses pure operations only:
>
> def squares(n: int) -> List[int]:
>     a = []
>     for i in range(n):
>         a.append(i * i)
>     return a
>
> I`m not sure if I know how to implement this in a useful way, though. I
> believe that there is some prior work in this area (not specific to
Python).
>
> Jukka
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

we could have eg:

def parallel_map(f, data):
    if is_pure(f):
        # <impl that does special optimisations to take advantage of
the fact f is truly pure>
        # special case that uses light weight threads or sth (maybe
can run bits of it GIL free or sth, less overhead or sth)
        # its ok to share immutable data
    else:
        #<impl that has no issue with impure f>
        # eg: multiprocessing process pools..
        #its ok to mutate everything becauuse we no longer truly share
anything.

in other words, even if only a small subset of code can be proven pure
it can be useful to help optimise code (in this example to avoid
locking by knowing that no 2 parrallel instances of the callback can
mock with mutually shared state)

for some scenarios it - might be - ok even if we leave the proving to
the programmer. (eg a C extension could implement the optimised
parallel map in this example and a user defined decorator is enough to
shove a flag on to the function f to trigger the optimised code path)
if this it is not truly pure tho despite the assertion it might cause
issues so in that case it probably is not something that should ship
with python by default?
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160622/85a4e7c2/attachment.html>

From vxgmichel at gmail.com  Mon Jun 27 17:15:41 2016
From: vxgmichel at gmail.com (Vincent Michel)
Date: Mon, 27 Jun 2016 23:15:41 +0200
Subject: [Python-ideas] Submitting a job to an asyncio event loop
In-Reply-To: <75b3439f-de52-41a0-8b0e-ec7d5438be98@googlegroups.com>
References: <CAFvThkBK5RjLLXgxZ8ePe0xfQ5O2xKKpT_0oULRAz0PB0X--zQ@mail.gmail.com>
 <75b3439f-de52-41a0-8b0e-ec7d5438be98@googlegroups.com>
Message-ID: <CAFvThkDrAPyCn2dKKT47fWjDii-OirvY5cQbReKFKWjNE2cVhg@mail.gmail.com>

I did remove it, because the feature has been implemented after the discussion:
https://docs.python.org/3/library/asyncio-task.html?highlight=run_coroutine#asyncio.run_coroutine_threadsafe
https://docs.python.org/3/library/asyncio-dev.html#asyncio-multithreading
https://github.com/python/asyncio/pull/273

2016-06-27 22:53 GMT+02:00 David Ford <firefighterblu3 at gmail.com>:
> Vincent, your github repo referenced here is 404. did you rename/move it?
>
>
>
> On Saturday, September 26, 2015 at 2:29:43 PM UTC, Vincent Michel wrote:
>>
>> Hi,
>>
>> I noticed there is currently no standard solution to submit a job from a
>> thread to an asyncio event loop.
>>
>> Here's what the asyncio documentation says about concurrency and
>> multithreading:
>>
>> > To schedule a callback from a different thread, the
>> > BaseEventLoop.call_soon_threadsafe() method should be used.
>> > Example to schedule a coroutine from a different thread:
>> >     loop.call_soon_threadsafe(asyncio.async, coro_func())
>>
>> The issue with this method is the loss of the coroutine result.
>>
>> One way to deal with this issue is to connect the asyncio.Future returned
>> by async (or ensure_future) to a concurrent.futures.Future. It is then
>> possible to use a subclass of concurrent.futures.Executor to submit a
>> callback to an asyncio event loop. Such an executor can also be used to set
>> up communication between two event loops using run_in_executor.
>>
>> I posted an implementation called LoopExecutor on GitHub:
>> https://github.com/vxgmichel/asyncio-loopexecutor
>> The repo contains the loopexecutor module along with tests for several use
>> cases. The README describes the whole thing (context, examples, issues,
>> implementation).
>
>
>>
>> [...]

From matt at getpattern.com  Wed Jun 29 13:47:13 2016
From: matt at getpattern.com (Matt Gilson)
Date: Wed, 29 Jun 2016 10:47:13 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
Message-ID: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>

The following code raises an `AttributeError`:

>>> class D(dict):
...     def __missing__(self, k):
...          super(D, self).__missing__(k)
...
>>> d = D()
>>> d['key']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __missing__
AttributeError: 'super' object has no attribute '__missing__'

I find this behavior to be a little bit odd as I would have expected the
default implementation to have a `__missing__` method and I would expect
the `__missing__` method to raise an appropriate `KeyError`.  I think that
this would facilitate a class hierarchy where each class can decide "I
don't know hot to handle this missing key, maybe something further up the
MRO does".  Currently, to do that (and adhere to the ["Super considered
Super"](https://rhettinger.wordpress.com/2011/05/26/super-considered-super/)
methodology) you'd have to add an extra base-class in the hierarchy which
seems unnecessary.

Obviously this is probably a pretty niche case and may not be worth
considering, but I thought I'd throw it out there just in case.

Also, FWIW, I believe that the current behavior is correct based on [the
documentation](https://docs.python.org/2/library/stdtypes.html#dict) (under
`d[key]`).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160629/5f18fdab/attachment.html>

From ethan at stoneleaf.us  Wed Jun 29 13:50:56 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 29 Jun 2016 10:50:56 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
Message-ID: <57740A80.3010507@stoneleaf.us>

On 06/29/2016 10:47 AM, Matt Gilson wrote:

> The following code raises an `AttributeError`:
>
>  >>> class D(dict):
> ...     def __missing__(self, k):
> ...          super(D, self).__missing__(k)
> ...
>  >>> d = D()
>  >>> d['key']
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "<stdin>", line 3, in __missing__
> AttributeError: 'super' object has no attribute '__missing__'
>
> I find this behavior to be a little bit odd as I would have expected the
> default implementation to have a `__missing__` method and I would expect
> the `__missing__` method to raise an appropriate `KeyError`.  I think
> that this would facilitate a class hierarchy where each class can decide
> "I don't know hot to handle this missing key, maybe something further up
> the MRO does".

+1

--
~Ethan~


From michael.selik at gmail.com  Wed Jun 29 14:29:48 2016
From: michael.selik at gmail.com (Michael Selik)
Date: Wed, 29 Jun 2016 18:29:48 +0000
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
Message-ID: <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>

On Wed, Jun 29, 2016 at 1:47 PM Matt Gilson <matt at getpattern.com> wrote:

> The following code raises an `AttributeError`:
> >>> class D(dict):
> ...     def __missing__(self, k):
> ...          super(D, self).__missing__(k)
> ...
>
> I find this behavior to be a little bit odd as I would have expected the
> default implementation to have a `__missing__` method
>

The behavior of a dictionary key-lookup checking for __missing__ parallels
the way a standard class does attribute-lookup, checking for __getattr__.

I'm guessing the non-existence of __missing__ on the built-in dict type
(and the non-existence of __getattr__ on the base object type) allows for
some speed optimizations in the interpreter. If so, then the tradeoff of
needing to provide an extra base class shim for your multiple inheritance
hierarchy seems reasonable. Does anyone know the original reason for not
having __missing__ defined on the builtin type?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160629/b41ba093/attachment.html>

From guido at python.org  Wed Jun 29 14:56:02 2016
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Jun 2016 11:56:02 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
Message-ID: <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>

The argument against adding `dict.__missing__()` is that it would just
a shortcut to raise KeyError with certain parameters.

The purpose of `__missing__` is to give you a hook to override how the
base class handles missing keys in `__getitem__`. In order to define
this hook you must necessarily subclass dict.

IMO the idea that there's always a superclass you can call is silly --
you should distinguish between cases where you *override* a method vs.
cases where you *define* it. In this case you are required to *define*
`__missing__`.

But I may be missing something...

On Wed, Jun 29, 2016 at 11:29 AM, Michael Selik <michael.selik at gmail.com> wrote:
> On Wed, Jun 29, 2016 at 1:47 PM Matt Gilson <matt at getpattern.com> wrote:
>>
>> The following code raises an `AttributeError`:
>> >>> class D(dict):
>> ...     def __missing__(self, k):
>> ...          super(D, self).__missing__(k)
>> ...
>>
>> I find this behavior to be a little bit odd as I would have expected the
>> default implementation to have a `__missing__` method
>
>
> The behavior of a dictionary key-lookup checking for __missing__ parallels
> the way a standard class does attribute-lookup, checking for __getattr__.
>
> I'm guessing the non-existence of __missing__ on the built-in dict type (and
> the non-existence of __getattr__ on the base object type) allows for some
> speed optimizations in the interpreter. If so, then the tradeoff of needing
> to provide an extra base class shim for your multiple inheritance hierarchy
> seems reasonable. Does anyone know the original reason for not having
> __missing__ defined on the builtin type?
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
--Guido van Rossum (python.org/~guido)

From vgr255 at live.ca  Wed Jun 29 15:04:32 2016
From: vgr255 at live.ca (=?iso-8859-1?Q?=C9manuel_Barry?=)
Date: Wed, 29 Jun 2016 15:04:32 -0400
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
 <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
Message-ID: <BLU403-EAS135589069859A2C08B97B1C91230@phx.gbl>

> From: Guido van Rossum
> To: Michael Selik
> Cc: Python-Ideas
> Subject: Re: [Python-ideas] should `dict` supply a default `__missing__`
> implementation?
> 
> The argument against adding `dict.__missing__()` is that it would just
> a shortcut to raise KeyError with certain parameters.

Can't the interpreter optimize that out? Something like 'if
ns["__missing__"] is dict.__missing__: raise KeyError' ? I figure it
wouldn't be hard, doesn't Python already do stuff like that in a bunch of
other places?

> IMO the idea that there's always a superclass you can call is silly --
> you should distinguish between cases where you *override* a method vs.
> cases where you *define* it. In this case you are required to *define*
> `__missing__`.

The difference is that `dict` explicitly supports __missing__ for
subclasses, so that users don't have to re-define __getitem__ for those
purposes. I don't know of many classes explicitly supporting a given feature
but not implementing at least a method for it that does nothing. But maybe
that's just me.

-Emanuel

From ethan at stoneleaf.us  Wed Jun 29 15:07:52 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 29 Jun 2016 12:07:52 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
 <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
Message-ID: <57741C88.7030906@stoneleaf.us>

On 06/29/2016 11:56 AM, Guido van Rossum wrote:

> The argument against adding `dict.__missing__()` is that it would just
> a shortcut to raise KeyError with certain parameters.
>
> The purpose of `__missing__` is to give you a hook to override how the
> base class handles missing keys in `__getitem__`. In order to define
> this hook you must necessarily subclass dict.
>
> IMO the idea that there's always a superclass you can call is silly --
> you should distinguish between cases where you *override* a method vs.
> cases where you *define* it. In this case you are required to *define*
> `__missing__`.

My understanding is a comprehensive base class is necessary when 
multiple inheritance is a concern, since the methods can't know if they 
are being called before or after other methods and so, to be good 
citizens, should call super() -- which of course will fail when the base 
class is searched and the method doesn't exist.

> But I may be missing something...

I can see two possible use-cases for multiple __missing__ methods:

- the default provided may be different based on the key name
- different kinds of tracking/registering of key names is happening

--
~Ethan~

From guido at python.org  Wed Jun 29 15:09:39 2016
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Jun 2016 12:09:39 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <57741C88.7030906@stoneleaf.us>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
 <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
 <57741C88.7030906@stoneleaf.us>
Message-ID: <CAP7+vJLTHo2Uv3V+3yzNs3tjyYwOqZcX25Oe42SNL0XQNJNa-g@mail.gmail.com>

So providing the comprehensive base class is up to the user, not up to
the stdlib. Is that such a big deal?

On Wed, Jun 29, 2016 at 12:07 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 06/29/2016 11:56 AM, Guido van Rossum wrote:
>
>> The argument against adding `dict.__missing__()` is that it would just
>> a shortcut to raise KeyError with certain parameters.
>>
>> The purpose of `__missing__` is to give you a hook to override how the
>> base class handles missing keys in `__getitem__`. In order to define
>> this hook you must necessarily subclass dict.
>>
>> IMO the idea that there's always a superclass you can call is silly --
>> you should distinguish between cases where you *override* a method vs.
>> cases where you *define* it. In this case you are required to *define*
>> `__missing__`.
>
>
> My understanding is a comprehensive base class is necessary when multiple
> inheritance is a concern, since the methods can't know if they are being
> called before or after other methods and so, to be good citizens, should
> call super() -- which of course will fail when the base class is searched
> and the method doesn't exist.
>
>> But I may be missing something...
>
>
> I can see two possible use-cases for multiple __missing__ methods:
>
> - the default provided may be different based on the key name
> - different kinds of tracking/registering of key names is happening
>
> --
> ~Ethan~
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
--Guido van Rossum (python.org/~guido)

From matt at getpattern.com  Wed Jun 29 15:29:13 2016
From: matt at getpattern.com (Matt Gilson)
Date: Wed, 29 Jun 2016 12:29:13 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAP7+vJLTHo2Uv3V+3yzNs3tjyYwOqZcX25Oe42SNL0XQNJNa-g@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
 <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
 <57741C88.7030906@stoneleaf.us>
 <CAP7+vJLTHo2Uv3V+3yzNs3tjyYwOqZcX25Oe42SNL0XQNJNa-g@mail.gmail.com>
Message-ID: <CAJCbRZZw0GnYEqb69CVcHAf1VDO14CxEiVHchJuWNiXMU3HP6w@mail.gmail.com>

On Wed, Jun 29, 2016 at 12:09 PM, Guido van Rossum <guido at python.org> wrote:

> So providing the comprehensive base class is up to the user, not up to
> the stdlib. Is that such a big deal?
>

Nope.  I honestly don't think that it is a big deal.  Mostly I just found
the behavior surprising for the reasons mentioned earlier (`__missing__` is
explicitly supported for dict subclasses but also explicitly not supported
on the superclass).

FWIW, "The argument against adding `dict.__missing__()` is that it would
just a shortcut to raise KeyError with certain parameters." also doesn't
seem like that big of a deal to me :-).  I'm not sure if there would
be noticeable performance issues or if the implementation would be
particularly difficult (both reasons which would be certainly compelling
enough to reject this idea).  But if the rational really is that it would
be a hook method whose implementation is almost trivial -- That seems like
the best kind of hook method to me :-).

Of course, maybe I'm shooting my proposal in the foot here -- If it's not
really a big deal on either side of the argument, maybe keeping the
status-quo is the safest recourse...



>
> On Wed, Jun 29, 2016 at 12:07 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> > On 06/29/2016 11:56 AM, Guido van Rossum wrote:
> >
> >> The argument against adding `dict.__missing__()` is that it would just
> >> a shortcut to raise KeyError with certain parameters.
> >>
> >> The purpose of `__missing__` is to give you a hook to override how the
> >> base class handles missing keys in `__getitem__`. In order to define
> >> this hook you must necessarily subclass dict.
> >>
> >> IMO the idea that there's always a superclass you can call is silly --
> >> you should distinguish between cases where you *override* a method vs.
> >> cases where you *define* it. In this case you are required to *define*
> >> `__missing__`.
> >
> >
> > My understanding is a comprehensive base class is necessary when multiple
> > inheritance is a concern, since the methods can't know if they are being
> > called before or after other methods and so, to be good citizens, should
> > call super() -- which of course will fail when the base class is searched
> > and the method doesn't exist.
> >
> >> But I may be missing something...
> >
> >
> > I can see two possible use-cases for multiple __missing__ methods:
> >
> > - the default provided may be different based on the key name
> > - different kinds of tracking/registering of key names is happening
> >
> > --
> > ~Ethan~
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 

[image: pattern-sig.png]

Matt Gilson // SOFTWARE ENGINEER

E: matt at getpattern.com // P: 603.892.7736

We?re looking for beta testers.  Go here
<https://www.getpattern.com/meetpattern> to sign up!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160629/d57a1695/attachment.html>

From ethan at stoneleaf.us  Wed Jun 29 15:30:58 2016
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 29 Jun 2016 12:30:58 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <CAP7+vJLTHo2Uv3V+3yzNs3tjyYwOqZcX25Oe42SNL0XQNJNa-g@mail.gmail.com>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
 <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
 <57741C88.7030906@stoneleaf.us>
 <CAP7+vJLTHo2Uv3V+3yzNs3tjyYwOqZcX25Oe42SNL0XQNJNa-g@mail.gmail.com>
Message-ID: <577421F2.7060003@stoneleaf.us>

On 06/29/2016 12:09 PM, Guido van Rossum wrote:

> So providing the comprehensive base class is up to the user, not up to
> the stdlib. Is that such a big deal?

No, it's not.  But it makes for a better user experience if the base 
class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because 
subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do 
it.  If it's not, I think we should.  Maybe add it to UserDict if 
performance is a concern?

--
~Ethan~

From guido at python.org  Wed Jun 29 16:57:58 2016
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Jun 2016 13:57:58 -0700
Subject: [Python-ideas] should `dict` supply a default `__missing__`
 implementation?
In-Reply-To: <577421F2.7060003@stoneleaf.us>
References: <CAJCbRZaT7r5M0+w2sM-RMB_RSobw1axrvjrn6Pkx+UH9Qkdf6A@mail.gmail.com>
 <CAGgTfkON9W_QHSxZGKJm4Yxzhy7KHoQDswdPPfhrMnPCRO+QbQ@mail.gmail.com>
 <CAP7+vJKwHSuonGPFFd9J72iXUVuaRvd+S0iiRhj3nP5KFrk+LQ@mail.gmail.com>
 <57741C88.7030906@stoneleaf.us>
 <CAP7+vJLTHo2Uv3V+3yzNs3tjyYwOqZcX25Oe42SNL0XQNJNa-g@mail.gmail.com>
 <577421F2.7060003@stoneleaf.us>
Message-ID: <CAP7+vJJXpKAEioWw1vziKS4kgYCP+vQmWba+543w4VHndX9quQ@mail.gmail.com>

UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't
think we should particularly encourage subclassing it. So -1 on adding
dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 06/29/2016 12:09 PM, Guido van Rossum wrote:
>
>> So providing the comprehensive base class is up to the user, not up to
>> the stdlib. Is that such a big deal?
>
>
> No, it's not.  But it makes for a better user experience if the base class
> has the __missing__ method that raises a KeyError already.
>
> Didn't we add a UserDict that could be subclassed primarily because
> subclassing dict directly was such a poor user experience?
>
> If adding __missing__ to dict is huge (performance hit?), we don't do it.
> If it's not, I think we should.  Maybe add it to UserDict if performance is
> a concern?
>
>
> --
> ~Ethan~
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
--Guido van Rossum (python.org/~guido)

From mehaase at gmail.com  Thu Jun 30 03:23:42 2016
From: mehaase at gmail.com (Mark E. Haase)
Date: Thu, 30 Jun 2016 03:23:42 -0400
Subject: [Python-ideas] Chaining asyncio futures?
Message-ID: <CALb0Rk4ZHkebWnj8d+8Mka30idjTcjRL5LyQkYK4Z_x903_CGQ@mail.gmail.com>

Other async frameworks have constructs similar to asyncio.Future, e.g.
JavaScript has Promise[1] and Dart has a Completer/Future pair[2][3]. (For
the purpose of this e-mail, I'll call the generic concept a "Future".)

JavaScript and Dart allow futures to be chained, such that one Future's
callback is invoked only when the other Future completes. For example,
consider an API that fetches a string from a URL and returns it, but also
has an MRU cache. This API returns a Future because the caller does not
know in advance if the string to be retrieved is cached or not. If the
string is in cache, then the Future can complete immediately, but if the
string it not in cache, then the string needs to be fetched over the
network, perhaps by calling some API that returns a Future string. In this
case, it would be nice to return a Future string that is chained to the
HTTP request's future string.[4]

I tried doing this with asyncio.Future:

    import asyncio

    f1 = asyncio.Future()
    f2 = asyncio.Future()
    f2.add_done_callback(lambda f: print('done: {}'.format(f.result())))
    f2.set_result(f1)
    f1.set_result('hello world!')

    loop = asyncio.get_event_loop()
    loop.run_until_complete(f2)

Output:

    done: <Future finished result='hello world!'>

This does not make f2 dependent on f1; it immediately sets f2's result to
repr(f1). It's easy to modify the example so that f1's callback explicitly
sets f2's result. Abstracting a bit, we can write a chain_futures()
function.

    def chain_futures(a, b):
        a.add_done_callback(lambda _: b.set_result(a.result()))

    f3 = asyncio.Future()
    f4 = asyncio.Future()
    f4.add_done_callback(lambda f: print('done: {}'.format(f.result())))
    chain_futures(f3, f4)
    f3.set_result('hello world!')

    loop.run_until_complete(f4)

Output:

    done: hello world!

Wouldn't this be a nice feature in asyncio.Future? E.g. instead of
`chain_futures(f3, f4)`, either `f3.set_result(f4)` or something more
explicit like `f3.set_result_from(f4)`.


[1]
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
[2] https://api.dartlang.org/stable/1.17.1/dart-async/Completer-class.html
[3] https://api.dartlang.org/stable/1.17.1/dart-async/Future-class.html
[4] Why not return the HTTP request Future directly? That feels hacky --
it's a leaky abstraction.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160630/f4339c80/attachment.html>

From cory at lukasa.co.uk  Thu Jun 30 06:48:00 2016
From: cory at lukasa.co.uk (Cory Benfield)
Date: Thu, 30 Jun 2016 11:48:00 +0100
Subject: [Python-ideas] Chaining asyncio futures?
In-Reply-To: <CALb0Rk4ZHkebWnj8d+8Mka30idjTcjRL5LyQkYK4Z_x903_CGQ@mail.gmail.com>
References: <CALb0Rk4ZHkebWnj8d+8Mka30idjTcjRL5LyQkYK4Z_x903_CGQ@mail.gmail.com>
Message-ID: <52B699FC-DEB6-4657-B774-136B8E570475@lukasa.co.uk>


> On 30 Jun 2016, at 08:23, Mark E. Haase <mehaase at gmail.com> wrote:
> 
> This does not make f2 dependent on f1; it immediately sets f2's result to repr(f1). It's easy to modify the example so that f1's callback explicitly sets f2's result. Abstracting a bit, we can write a chain_futures() function.
> 
>     def chain_futures(a, b):
>         a.add_done_callback(lambda _: b.set_result(a.result()))
> 
>     f3 = asyncio.Future()
>     f4 = asyncio.Future()
>     f4.add_done_callback(lambda f: print('done: {}'.format(f.result())))
>     chain_futures(f3, f4)
>     f3.set_result('hello world!')
> 
>     loop.run_until_complete(f4)
> 
> Output:
> 
>     done: hello world!
> 
> Wouldn't this be a nice feature in asyncio.Future? E.g. instead of `chain_futures(f3, f4)`, either `f3.set_result(f4)` or something more explicit like `f3.set_result_from(f4)`.

For a reference point: Twisted?s Deferreds have this already, written almost exactly in the form you?re considering it: https://twistedmatrix.com/documents/current/core/howto/defer.html#chaining-deferreds <https://twistedmatrix.com/documents/current/core/howto/defer.html#chaining-deferreds>. The biggest difference for Twisted is that it also hooks up the errbacks, which is a semantic that doesn?t make sense for asyncio Futures.

While discussing this, the natural way to write this kind of code in Twisted is actually to return Deferreds from a callback (the asyncio equivalent would be to return a Future from a done callback). In Twisted, if a callback returns a Deferred the next callback in the chain is not called until the Deferred resolves, at which point it is called with the result of that Deferred. Essentially, then, the program shown above returns only a single Deferred with a long series of callbacks that transform the data, some of which operate asynchronously.

As best as I can work out, the only reason that asyncio Future?s don?t behave this way is that asyncio strongly emphasises the use of coroutines to program in this manner. Put another way, the asyncio-native way to write that code is not to chain Futures, but instead to write a coroutine that manages your flow control for you:

    async def do_work():
        result = await check_cache(url)
        if not result:
            result = await do_web_request(url)
        print(?done: {}?.format(f.result()))

    loop.run_until_complete(do_work())

This is not a reason in and of itself not to have chain_futures as a thing that exists. However, I think it may be the case that the asyncio core developers aren?t hugely interested in it. I?m now reading a bit into the mindset of Guido so I?m sure he could step in and correct me, but I seem to recall that asyncio?s Futures were deliberately designed to have less of the complexity of, say, Twisted?s Deferreds, in part because coroutines were intended to supersede some of the more complex functionality of Deferreds.

Cory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160630/511d6593/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160630/511d6593/attachment-0001.sig>

From vxgmichel at gmail.com  Thu Jun 30 08:12:54 2016
From: vxgmichel at gmail.com (Vincent Michel)
Date: Thu, 30 Jun 2016 14:12:54 +0200
Subject: [Python-ideas] Chaining asyncio futures?
In-Reply-To: <mailman.1294.1467283684.2357.python-ideas@python.org>
References: <mailman.1294.1467283684.2357.python-ideas@python.org>
Message-ID: <57750CC6.2000204@gmail.com>

 >> Wouldn't this be a nice feature in asyncio.Future?

It does exist actually, as `asyncio.futures._chain_future`:
https://github.com/python/asyncio/blob/master/asyncio/futures.py#L411

It is used in `loop.run_in_executor` and `run_coroutine_threadsafe` to 
connect asyncio futures and concurrent futures together.

 > As best as I can work out, the only reason that asyncio Future?s 
don?t behave this way is that asyncio strongly emphasises the use of 
coroutines to program in this manner.

I agree, I think that's why this function is not exposed.

/Vincent