From leewangzhong+python at gmail.com  Sat Apr  1 00:33:21 2017
From: leewangzhong+python at gmail.com (Franklin? Lee)
Date: Sat, 1 Apr 2017 00:33:21 -0400
Subject: [Python-ideas] Construct a matrix from a list: matrix
 multiplication
In-Reply-To: <CAPTjJmrrd1HnjP7+BiUv8svEtV1bhgNtpG7kr4Mb_VQPBrC6aA@mail.gmail.com>
References: <CAPTjJmrrd1HnjP7+BiUv8svEtV1bhgNtpG7kr4Mb_VQPBrC6aA@mail.gmail.com>
Message-ID: <CAB_e7iwfEQm9eEKh6B7Tv9XjPAA04-F0mm_AhAUnA93ceU8c0Q@mail.gmail.com>

On Fri, Mar 31, 2017 at 3:58 AM, Chris Angelico <rosuav at gmail.com> wrote:
> This keeps on coming up in one form or another - either someone
> multiplies a list of lists and ends up surprised that they're all the
> same, or is frustrated with the verbosity of the alternatives.
>
> Can we use the matmul operator for this?

In math, a number can be considered a 1-dimensional vector. You can
multiply a 1-dimensional vector by an n-dimensional vector _as
matrices_ to get the same result as scalar multiplication. Using it as
a deep-copy multiplication operator might make things confusing when
someone moves to or from Numpy.

Instead of using multiplication to create homogenous lists, how about
using the list constructor?
    list(0, 4) #list of 4 elements, all 0
    list(0, (4,4)) #4x4

Concerns with my idea:
1. Someone might expect `list(0, 4)` to return `[0, 4]`.
2. Someone might want to write `list(list(0, 4), 4)`, which has the
same issue as `[[0] * 4] * 4`.
3. Contrast with the `bytes` constructor. `bytes(n) == b'\0' * 10`

The `dict` constructor has a `.fromkeys` class method. An analogous
method for `list` can be `list.withshape(4,4, val=0)`. That's likely
to be too big a change to solve a small problem.

From mistersheik at gmail.com  Sat Apr  1 01:17:13 2017
From: mistersheik at gmail.com (Neil Girdhar)
Date: Fri, 31 Mar 2017 22:17:13 -0700 (PDT)
Subject: [Python-ideas] unify usage of mutable and immutable objects
In-Reply-To: <262c49f5.3a0.15a871f3b36.Coremail.mlet_it_bew@126.com>
References: <38e2b548.256.15a86f126b6.Coremail.mlet_it_bew@126.com>
 <CAEbHw4Z_e=ZtqqYMW8OBQfXQYLUx4qCP07xoW5sVffKMwc5rWA@mail.gmail.com>
 <262c49f5.3a0.15a871f3b36.Coremail.mlet_it_bew@126.com>
Message-ID: <7f0ef1ca-e955-4695-8ac6-69e785442626@googlegroups.com>


On Tuesday, February 28, 2017 at 6:48:42 PM UTC-5, ????? wrote:
>
> > A hypothetical frozenset.pop() is also necessarily O(N).  It needs to 
> copy N-1 elements into the new (smaller) frozenset object.  So this isn't 
> an argument.
> Pop tuple/frozenset(standard one) gain no benefit. # O(n)
> It is a different story for balanced tree. # O(log n)
>
> > Sounds like `collections.deque` might be what you want (for concurrency, 
> not for immutability).  But a local copy will require, by definition, a 
> *copy* operation either way.
> My intent is to unify "SET" interface, not for to using deque.
>      I want something that is SET can use anywhere regardless mutable or 
> not.
>     And the idiom SHOULD work for other type.
>
> WHY set.add / list.sort return None?
>      if return self,  someone may think it don't modify the orignal object.
>      so, mutable object will have different methods.
> Such differences are good UNLESS we want to ignore it explictly.
> We need a uniform way to make a interface suitable for both cases. 
>
>
collections.abc.Set is the uniform interface suitable for mutable and 
immutable sets:

https://docs.python.org/3/library/collections.abc.html

It contains:

__contains__, __iter__, __len__ __le__, __lt__, __eq__, __ne__, __gt__, 
__ge__, __and__, __or__, __sub__, __xor__, and isdisjoint

Is there something missing?  I don't think add should be there. I think if 
you want to start mutating a (possibly immutable)  set, you should convert 
it to set (pay the linear cost up-front) and then start mutating it.


>
> At 2017-03-01 07:14:33, "David Mertz" <me... at gnosis.cx <javascript:>> 
> wrote:
>
> On Tue, Feb 28, 2017 at 2:57 PM, ????? <mlet_... at 126.com <javascript:>> 
> wrote:
>
>> 1) coverting to set or list is O(n) in time
>>
>
> A hypothetical frozenset.pop() is also necessarily O(N).  It needs to copy 
> N-1 elements into the new (smaller) frozenset object.  So this isn't an 
> argument.
>  
>
>> 2) if I have to keep the old copy,
>>     standard set solution will be O(n) both in time and space!
>>
>
> Again, nothing gained here. Same applies to frozenset.pop() (assuming it 
> returns both the item popped and a reduced set).  If you just want 
> "something from frozenset" without creating a new almost-copy frozenset, 
> that is spelled `next(iter(fs))`.
>  
>
>> working examples:
>>     1) priority queue:
>>         insert and pop occur
>>     2) share immutable data to difference subroutines:
>>         each one can modify local copy safely and concurrency.
>>
>
> Sounds like `collections.deque` might be what you want (for concurrency, 
> not for immutability).  But a local copy will require, by definition, a 
> *copy* operation either way. 
>
>
> -- 
> Keeping medicines from the bloodstreams of the sick; food 
> from the bellies of the hungry; books from the hands of the 
> uneducated; technology from the underdeveloped; and putting 
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
>
>
>  
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170331/620028c5/attachment-0001.html>

From pavol.lisy at gmail.com  Sat Apr  1 01:22:12 2017
From: pavol.lisy at gmail.com (Pavol Lisy)
Date: Sat, 1 Apr 2017 07:22:12 +0200
Subject: [Python-ideas] Construct a matrix from a list: matrix
 multiplication
In-Reply-To: <CAB_e7iwfEQm9eEKh6B7Tv9XjPAA04-F0mm_AhAUnA93ceU8c0Q@mail.gmail.com>
References: <CAPTjJmrrd1HnjP7+BiUv8svEtV1bhgNtpG7kr4Mb_VQPBrC6aA@mail.gmail.com>
 <CAB_e7iwfEQm9eEKh6B7Tv9XjPAA04-F0mm_AhAUnA93ceU8c0Q@mail.gmail.com>
Message-ID: <CABDEq2neee14XGz90m955a+ROrXFiTVDXos0emAE=DCDccRbJg@mail.gmail.com>

On 4/1/17, Franklin? Lee <leewangzhong+python at gmail.com> wrote:
> On Fri, Mar 31, 2017 at 3:58 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> This keeps on coming up in one form or another - either someone
>> multiplies a list of lists and ends up surprised that they're all the
>> same, or is frustrated with the verbosity of the alternatives.

>> Can we use the matmul operator for this?

> In math, a number can be considered a 1-dimensional vector. You can
> multiply a 1-dimensional vector by an n-dimensional vector _as
> matrices_ to get the same result as scalar multiplication. Using it as
> a deep-copy multiplication operator might make things confusing when
> someone moves to or from Numpy.

Too late. ;)

    [1, 1, 1] * 3
    numpy.ones(3) * 3

PL.

From ram at rachum.com  Sat Apr  1 15:27:34 2017
From: ram at rachum.com (Ram Rachum)
Date: Sat, 1 Apr 2017 21:27:34 +0200
Subject: [Python-ideas] Introduce BaseTimeoutError
Message-ID: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>

Today I got burned because I had code that did this:

    except TimeoutError:

When it should have done this:

    except socket.timeout:

There's also another timeout error class in asyncio.

Initially I thought, why not make them all use the same exception class?
But Guido objects to that:

I considered this, and decided against unifying the two TimeoutErrors.

First the builtin TimeoutError is specifically a subclass of OSError
representing the case where errno is ETIMEDOUT.  But
asyncio.TimeoutError means nothing of the sort.

Second, the precedent is concurrent.futures.TimeoutError. The asyncio
one is used under the same conditions as that one.

I think we should just update the links in the docs to be correct.

So here's my idea: Define a new base class `BaseTimeoutError` and have it
be a base class of all other timeout errors. This way people could do:

    except BaseTimeoutError:

And it'll catch all of them.

I thought of doing it myself as an abc, but if I remember correctly the
except logic ignored __subclasshook__ or __instancecheck__ or whatever is
used by abc, so I think this can only work by modifying CPython.


What do you think?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170401/96c0dce9/attachment.html>

From greg.ewing at canterbury.ac.nz  Sat Apr  1 19:48:20 2017
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 02 Apr 2017 11:48:20 +1200
Subject: [Python-ideas] Construct a matrix from a list: matrix
 multiplication
In-Reply-To: <20170401014415.GD9464@ando.pearwood.info>
References: <CAPTjJmrrd1HnjP7+BiUv8svEtV1bhgNtpG7kr4Mb_VQPBrC6aA@mail.gmail.com>
 <20170401014415.GD9464@ando.pearwood.info>
Message-ID: <58E03C44.4050303@canterbury.ac.nz>

Steven D'Aprano wrote:
> I like the idea of using * for repetition without copying, and @ for 
> repetition with shallow copying.

Shallow copying wouldn't be enough in many cases, e.g.
building a matrix with 3 or more dimensions.

-- 
Greg

From steve at pearwood.info  Sat Apr  1 23:11:38 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 2 Apr 2017 13:11:38 +1000
Subject: [Python-ideas] Introduce BaseTimeoutError
In-Reply-To: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>
References: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>
Message-ID: <20170402031137.GG9464@ando.pearwood.info>

On Sat, Apr 01, 2017 at 09:27:34PM +0200, Ram Rachum wrote:
> Today I got burned because I had code that did this:
> 
>     except TimeoutError:
> 
> When it should have done this:
> 
>     except socket.timeout:

On the face of it, this isn't a serious problem. It seems to me rather 
like mistakenly writing:

    except TypeError

when you actually intended to write:

    except ZeroDivisionError

You failed to catch the exception that you needed, an got an exception. 
That's a bug, and the fix for it is to catch the correct exception. It 
isn't to introduce a new, more complex exception TypeOrZeroDivisionError 
that potentially catches too much.


> There's also another timeout error class in asyncio.
> 
> Initially I thought, why not make them all use the same exception class?

That's the wrong way to think about adding classes:

Wrong: "Why not add this new class?"

Right: "Why add this new class?"

We don't start from a position of "Default allow", that is, we add any 
arbitrary change suggested unless people find a reason to reject it. We 
start from a conservative position of "Default deny". All changes are 
rejected, unless they can show enough positive reasons to accept the 
change.

If you start off thinking "Why not make this change?", you will spend a 
lot of time suggesting things which are rejected. You should spend more 
time thinking critically about what actual positive reasons: "Why make 
this change?"

What is the benefit of this class? Does it unify two or more *related* 
exceptions? Guido doesn't think so, and I don't think so either. A 
socket timeout and an asyncio timeout are, to my mind, *unrelated*.

Just because they both relate to the passage of time doesn't make them 
the same thing.


> But Guido objects to that:
> 
> I considered this, and decided against unifying the two TimeoutErrors.
> 
> First the builtin TimeoutError is specifically a subclass of OSError
> representing the case where errno is ETIMEDOUT.  But
> asyncio.TimeoutError means nothing of the sort.
> 
> Second, the precedent is concurrent.futures.TimeoutError. The asyncio
> one is used under the same conditions as that one.

I agree with Guido's analysis. I don't think that the connection between 
the two timeout exceptions are related. 



> So here's my idea: Define a new base class `BaseTimeoutError` and have it
> be a base class of all other timeout errors. This way people could do:
> 
>     except BaseTimeoutError:
> 
> And it'll catch all of them.

Why would you do that?

This is exactly what I mean when I say that we need to have positive 
reasons for a change. It isn't enough to say that we can catch "all of 
them". We can already do that:

    # Did I miss any?
    except (TimeoutError, 
            asyncio.TimeoutError, 
            concurrent.futures.TimeoutError, 
            multiprocessing.TimeoutError,
            socket.timeout,
            subprocess.TimeoutExpired):

but *why* would I write that code? Under what circumstances will I have 
something that could raise *any* of those exceptions, and I will want to 
treat them *all* the same way?

It seems far more likely that I would *not* want to treat them the same 
way. If I have some networking code that raises socket.timeout, I 
probably want to catch that exception and try again (maybe with 
backoff). But if that same piece of networking code were to raise 
concurrent.futures.TimeoutError or subprocess.TimeoutExpired, I'd want 
to know about it! I'd want to see the exception, which means *not* 
catching it.

Or the other way around... if I am working with Futures, and have to 
deal with futures.TimeoutError, that doesn't mean that a networking 
timeout or a system call timeout should be treated the same way.

I'm having difficulty in seeing any positive benefit to this proposed 
base class.

If you really want this, you can name a tuple of all the unrelated 
exceptions you want to catch:

TimeoutErrors = (TimeoutError, asyncio.TimeoutError, 
        concurrent.futures.TimeoutError, socket.timeout,
        multiprocessing.TimeoutError, subprocess.TimeoutExpired)

try:
    ...
except TimeoutErrors:
    ...

but I cannot possibly recommend that. To me, that looks like it will 
catch too much under nearly all reasonable circumstances.




-- 
Steve

From rosuav at gmail.com  Sun Apr  2 00:38:41 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 2 Apr 2017 14:38:41 +1000
Subject: [Python-ideas] Introduce BaseTimeoutError
In-Reply-To: <20170402031137.GG9464@ando.pearwood.info>
References: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>
 <20170402031137.GG9464@ando.pearwood.info>
Message-ID: <CAPTjJmobeWxc9e+fzz3iqVKzUWApsek+MP9MfwDPTtBLj587Yg@mail.gmail.com>

On Sun, Apr 2, 2017 at 1:11 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On the face of it, this isn't a serious problem. It seems to me rather
> like mistakenly writing:
>
>     except TypeError
>
> when you actually intended to write:
>
>     except ZeroDivisionError
>
> You failed to catch the exception that you needed, an got an exception.
> That's a bug, and the fix for it is to catch the correct exception. It
> isn't to introduce a new, more complex exception TypeOrZeroDivisionError
> that potentially catches too much.

What I'd like to see is a linter/static analyzer that can look at your
code and determine heuristically that it is highly unlikely for this
exception to be raised. With some exceptions, the assumption is "can
be raised anywhere", but with custom or library-specific exceptions,
only a raise statement can cause them. If you had a linter like that,
you could spot the wrong timeout being caught, as it would notify you
that there's no (normal) way for that to catch anything.

Note that I am adamantly NOT suggesting this as a language feature. A
linter is allowed to be wrong occasionally if it's helpful the rest of
the time, and it's allowed to ignore the possibilities of weird
aliasing or monkey-patching.

Does this already exist somewhere? Is there something people would recommend?

ChrisA

From turnbull.stephen.fw at u.tsukuba.ac.jp  Sun Apr  2 15:28:42 2017
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 3 Apr 2017 04:28:42 +0900
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
Message-ID: <22753.20714.236815.996942@turnbull.sk.tsukuba.ac.jp>

Stephan Houben writes:

 > FWIW, I also strongly prefer the Verbal Expression style and consider
 > "normal" regular expressions to become quickly unreadable and
 > unmaintainable.
 > 
 > Verbal Expressions are also much more composable.

So are grammars.

But REs aren't so bad or incomposable if you build them up slowly in a
grammar-like fashion and with a specific convention for groups:

    atom = r"[-%A-Za-z0-9]+"   # incorrect, for example only
                               # each component has different lexical
                               # restrictions
    scheme = user = password = rf"({atom})"
    domain = rf"((?:{atom}\.)+{atom})"
    port = r"([0-9]+)"
    authority = rf"(?:{user}(?::{password})?@)?{domain}(?::{port})?"
    path = rf"((?:/{atom})+/?)"

    # Incorrect, but handles many common URIs.
    url = rf"{scheme}://(?:{authority})?({path})"

Of course this is parsing with regular expressions, which is generally
frowned upon, and it would be even uglier without f-strings.  The
non-capturing groups admittedly are a significant distraction when
reading.  It's about the limit of what I would do if I didn't have a
parsing library but did have REs (more complex than this and I'd write
my own parser).

I will concede that it took me 15 minutes to write that, of which 4
were spent testing and fixing one bug (which was a real bug; there
were no syntax errors in the REs).  Some of the time was spent
deciding how closely to follow the RFC 3986 generic syntax, though.  I
will also concede that I've been writing REs since 1981, although not
as frequently in the last 15 years as in the first 20.

Would you like to write that using VEs and show us the result?  Don't
forget to document the indicies for extracting the scheme, user,
password, domain, port, and path (in my RE, they are 1-6).

Steve


From tjreedy at udel.edu  Sun Apr  2 19:08:44 2017
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 2 Apr 2017 19:08:44 -0400
Subject: [Python-ideas] Introduce BaseTimeoutError
In-Reply-To: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>
References: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>
Message-ID: <obs09m$4up$1@blaine.gmane.org>

On 4/1/2017 3:27 PM, Ram Rachum wrote:
> Today I got burned because I had code that did this:
>
>     except TimeoutError:
>
> When it should have done this:
>
>     except socket.timeout:

Both are subclasses of OSError but mean different things.  TimeoutError 
means that something in 'your system' did not respond.  Socket.timeout 
means that a foreign system did not respond.  (I am leaving out a local 
socket connection.)  The latter can only happen with a socket call with 
timeouts enabled.  If it is possible for TimeoutError to also occur with 
a timeout-enabled socket call, then one might very well want to only 
catch one or catch them separately and respond differently.

> There's also another timeout error class in asyncio.

Which can only happen when using asyncio.

-1 on merging, +1 on staying with the current design.  What I write 
above is similar to Guido's explanation for asyncio.TimeoutError.

-- 
Terry Jan Reedy


From mistersheik at gmail.com  Sun Apr  2 21:22:08 2017
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 2 Apr 2017 18:22:08 -0700 (PDT)
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
Message-ID: <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>

Same.  One day, Python will have a decent parsing library.

On Friday, March 31, 2017 at 4:21:51 AM UTC-4, Stephan Houben wrote:
>
> Hi all, 
>
> FWIW, I also strongly prefer the Verbal Expression style and consider 
> "normal" regular expressions to become quickly unreadable and 
> unmaintainable. 
>
> Verbal Expressions are also much more composable. 
>
> Stephan 
>
> 2017-03-31 9:23 GMT+02:00 Stephen J. Turnbull 
> <turnbull.... at u.tsukuba.ac.jp <javascript:>>: 
> > Abe Dillon writes: 
> > 
> >  > Note that the entire documentation is 250 words while just the syntax 
> >  > portion of Python docs for the re module is over 3000 words. 
> > 
> > Since Verbal Expressions (below, VEs, indicating notation) "compile" 
> > to regular expressions (spelling out indicates the internal matching 
> > implementation), the documentation of VEs presumably ignores 
> > everything except the limited language it's useful for.  To actually 
> > understand VEs, you need to refer to the RE docs.  Not a win IMO. 
> > 
> >  > > You think that example is more readable than the proposed 
> transalation 
> >  > >     ^(http)(s)?(\:\/\/)(www\.)?([^\ ]*)$ 
> >  > > which is better written 
> >  > >     ^https?://(www\.)?[^ ]*$ 
> >  > > or even 
> >  > >     ^https?://[^ ]*$ 
> >  > 
> >  > 
> >  > Yes. I find it *far* more readable. It's not a soup of symbols like 
> Perl 
> >  > code. I can only surmise that you're fluent in regex because it seems 
> >  > difficult for you to see how the above could be less readable than 
> English 
> >  > words. 
> > 
> > Yes, I'm fairly fluent in regular expression notation (below, REs). 
> > I've maintained a compiler for one dialect. 
> > 
> > I'm not interested in the difference between words and punctuation 
> > though.  The reason I find the middle RE most readable is that it 
> > "looks like" what it's supposed to match, in a contiguous string as 
> > the object it will match will be contiguous.  If I need to parse it to 
> > figure out *exactly* what it matches, yes, that takes more effort. 
> > But to understand a VE's semantics correctly, I'd have to look it up 
> > as often as you have to look up REs because many words chosen to notate 
> > VEs have English meanings that are (a) ambiguous, as in all natural 
> > language, and (b) only approximate matches to RE semantics. 
> > 
> >  > I could tell it only matches URLs that are the only thing inside 
> >  > the string because it clearly says: start_of_line() and 
> >  > end_of_line(). 
> > 
> > That's not the problem.  The problem is the semantics of the method 
> > "find".  "then" would indeed read better, although it doesn't exactly 
> > match the semantics of concatenation in REs. 
> > 
> >  > I would have had to refer to a reference to know that "^" doesn't 
> >  > always mean "not", it sometimes means "start of string" and 
> >  > probably other things. I would also have to check a reference to 
> >  > know that "$" can mean "end of string" (and probably other things). 
> > 
> > And you'll still have to do that when reading other people's REs. 
> > 
> >  > > Are those groups capturing in Verbal Expressions?  The use of 
> >  > > "find" (~ "search") rather than "match" is disconcerting to the 
> >  > > experienced user. 
> >  > 
> >  > You can alternately use the word "then". The source code is just 
> >  > one python file. It's very easy to read. I actually like "then" 
> >  > over "find" for the example: 
> > 
> > You're missing the point.  The reader does not get to choose the 
> > notation, the author does.  I do understand what several varieties of 
> > RE mean, but the variations are of two kinds: basic versus extended 
> > (ie, what tokens need to be escaped to be taken literally, which ones 
> > have special meaning if escaped), and extensions (which can be 
> > ignored).  Modern RE facilities are essentially all of the extended 
> > variety.  Once you've learned that, you're in good shape for almost 
> > any RE that should be written outside of an obfuscated code contest. 
> > 
> > This is a fundamental principle of Python design: don't make readers 
> > of code learn new things.  That includes using notation developed 
> > elsewhere in many cases. 
> > 
> >  > What does alternation look like? 
> >  > 
> >  > .OR(option1).OR(option2).OR(option3)... 
> >  > 
> >  > How about alternation of 
> >  > > non-trivial regular expressions? 
> >  > 
> >  > .OR(other_verbal_expression) 
> > 
> > Real examples, rather than pseudo code, would be nice.  I think you, 
> > too, will find that examples of even fairly simple nested alternations 
> > containing other constructs become quite hard to read, as they fall 
> > off the bottom of the screen. 
> > 
> > For example, the VE equivalent of 
> > 
> >     scheme = "(https?|ftp|file):" 
> > 
> > would be (AFAICT): 
> > 
> >     scheme = VerEx().then(VerEx().then("http") 
> >                                  .maybe("s") 
> >                                  .OR("ftp") 
> >                                  .OR("file")) 
> >                     .then(":") 
> > 
> > which is pretty hideous, I think.  And the colon is captured by a 
> > group.  If perversely I wanted to extract that group from a match, 
> > what would its index be? 
> > 
> > I guess you could keep the linear arrangement with 
> > 
> >     scheme = (VerEx().add("(") 
> >                      .then("http") 
> >                      .maybe("s") 
> >                      .OR("ftp") 
> >                      .OR("file") 
> >                      .add(")") 
> >                      .then(":")) 
> > 
> > but is that really an improvement over 
> > 
> >     scheme = VerEx().add("(https?|ftp|file):") 
> > 
> > ;-) 
> > 
> >  > > As far as I can see, Verbal Expressions are basically a way of 
> >  > > making it so painful to write regular expressions that people 
> >  > > will restrict themselves to regular expressions 
> >  > 
> >  > What's so painful to write about them? 
> > 
> > One thing that's painful is that VEs "look like" context-free 
> > grammars, but clumsy and without the powerful semantics.  You can get 
> > the readability you want with greater power using grammars, which is 
> > why I would prefer we work on getting a parser module into the stdlib. 
> > 
> > But if one doesn't know about grammars, it's still not great.  The 
> > main pains about writing VEs for me are (1) reading what I just wrote, 
> > (2) accessing capturing groups, and (3) verbosity.  Even a VE to 
> > accurately match what is normally a fairly short string, such as the 
> > scheme, credentials, authority, and port portions of a "standard" URL, 
> > is going to be hundreds of characters long and likely dozens of lines 
> > if folded as in the examples. 
> > 
> > Another issue is that we already have a perfectly good poor man's 
> > matching library: glob.  The URL example becomes 
> > 
> >     http{,s}://{,www.}* 
> > 
> > Granted you lose the anchors, but how often does that matter?  You 
> > apparently don't use them often enough to remember them. 
> > 
> >  > Does your IDE not have autocompletion? 
> > 
> > I don't want an IDE.  I have Emacs. 
> > 
> >  > I find REs so painful to write that I usually just use string 
> >  > methods if at all feasible. 
> > 
> > Guess what?  That's the right thing to do anyway.  They're a lot more 
> > readable and efficient when partitioning a string into two or three 
> > parts, or recognizing a short list of affixes.  But chaining many 
> > methods, as VEs do, is not a very Pythonic way to write a program. 
> > 
> >  > > I don't think that this failure to respect the developer's taste 
> >  > > is restricted to this particular implementation, either. 
> >  > 
> >  > I generally find it distasteful to write a pseudolanguage in 
> >  > strings inside of other languages (this applies to SQL as well). 
> > 
> > You mean like arithmetic operators?  (Lisp does this right, right? 
> > Only one kind of expression, the function call!)  It's a matter of 
> > what you're used to.  I understand that people new to text-processing, 
> > or who don't do so much of it, don't find REs easy to read.  So how is 
> > this a huge loss?  They don't use regular expressions very often!  In 
> > fact, they're far more likely to encounter, and possibly need to 
> > understand, REs written by others! 
> > 
> >  > Especially when the design principals of that pseudolanguage are 
> >  > *diametrically opposed* to the design principals of the host 
> >  > language. A key principal of Python's design is: "you read code a 
> >  > lot more often than you write code, so emphasize 
> >  > readability". Regex seems to be based on: "Do the most with the 
> >  > fewest key-strokes. 
> > 
> > So is all of mathematics.  There's nothing wrong with concise 
> > expression for use in special cases. 
> > 
> >  > Readability be dammed!". It makes a lot more sense to wrap the 
> >  > psudolanguage in constructs that bring it in-line with the host 
> >  > language than to take on the mental burden of trying to comprehend 
> >  > two different languages at the same time. 
> >  > 
> >  > If you disagree, nothing's stopping you from continuing to write 
> >  > res the old-fashion way. 
> > 
> > I don't think that RE and SQL are "pseudo" languages, no.  And I, and 
> > most developers, will continue to write regular expressions using the 
> > much more compact and expressive RE notation.  (In fact with the 
> > exception of the "word" method, in VEs you still need to use RE notion 
> > to express most of the Python extensions.)  So what you're saying is 
> > that you don't read much code, except maybe your own.  Isn't that your 
> > problem?  Those of us who cooperate widely on applications using 
> > regular expressions will continue to communicate using REs.  If that 
> > leaves you out, that's not good.  But adding VEs to the stdlib (and 
> > thus encouraging their use) will split the community into RE users and 
> > VE users, if VEs are at all useful.  That's a bad.  I don't see that 
> > the potential usefulness of VEs to infrequent users of regular 
> > expressions outweighing the downsides of "many ways to do it" in the 
> > stdlib. 
> > 
> >  > Can we at least agree that baking special re syntax directly into 
> >  > the language is a bad idea? 
> > 
> > I agree that there's no particular need for RE literals.  If one wants 
> > to mark an RE as some special kind of object, re.compile() does that 
> > very well both by converting to a different type internally and as a 
> > marker syntactically. 
> > 
> >  > On Wed, Mar 29, 2017 at 11:49 PM, Nick Coghlan <ncog... at gmail.com 
> <javascript:>> wrote: 
> >  > 
> >  > > We don't really want to ease the use of regexps in Python - while 
> >  > > they're an incredibly useful tool in a programmer's toolkit, 
> >  > > they're so cryptic that they're almost inevitably a 
> >  > > maintainability nightmare. 
> > 
> > I agree with Nick.  Regular expressions, whatever the notation, are a 
> > useful tool (no suspension of disbelief necessary for me, though!). 
> > But they are cryptic, and it's not just the notation.  People (even 
> > experienced RE users) are often surprised by what fairly simple 
> > regular expression match in a given text, because people want to read 
> > a regexp as instructions to a one-pass greedy parser, and it isn't. 
> > 
> > For example, above I wrote 
> > 
> >     scheme = "(https?|ftp|file):" 
> > 
> > rather than 
> > 
> >     scheme = "(\w+):" 
> > 
> > because it's not unlikely that I would want to treat those differently 
> > from other schemes such as mailto, news, and doi.  In many 
> > applications of regular expressions (such as tokenization for a 
> > parser) you need many expressions.  Compactness really is a virtue in 
> > REs. 
> > 
> > Steve 
> > 
> > _______________________________________________ 
> > Python-ideas mailing list 
> > Python... at python.org <javascript:> 
> > https://mail.python.org/mailman/listinfo/python-ideas 
> > Code of Conduct: http://python.org/psf/codeofconduct/ 
> _______________________________________________ 
> Python-ideas mailing list 
> Python... at python.org <javascript:> 
> https://mail.python.org/mailman/listinfo/python-ideas 
> Code of Conduct: http://python.org/psf/codeofconduct/ 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170402/a3364064/attachment-0001.html>

From njs at pobox.com  Sun Apr  2 21:29:31 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 2 Apr 2017 18:29:31 -0700
Subject: [Python-ideas] Introduce BaseTimeoutError
In-Reply-To: <obs09m$4up$1@blaine.gmane.org>
References: <CANXboVYFY+DdCPQWGFKgFjV-y6bnhJBwWxDnbk3iGB5qwm-NnA@mail.gmail.com>
 <obs09m$4up$1@blaine.gmane.org>
Message-ID: <CAPJVwBkJ_-hqNd_RDU_LoAUckeC8e2P+pZC_aGNDyCaVCHRsdA@mail.gmail.com>

On Sun, Apr 2, 2017 at 4:08 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Both are subclasses of OSError but mean different things.  TimeoutError
> means that something in 'your system' did not respond.  Socket.timeout means
> that a foreign system did not respond.  (I am leaving out a local socket
> connection.)  The latter can only happen with a socket call with timeouts
> enabled.  If it is possible for TimeoutError to also occur with a
> timeout-enabled socket call, then one might very well want to only catch one
> or catch them separately and respond differently.

I don't necessarily disagree with the conclusion, but I'm pretty sure
this is rationale is wrong: connect / recv / etc. can return ETIMEDOUT
if the peer is non-responsive and some internal
operating-system-specific timeout expires (see e.g. [1], or the
TCP_USER_TIMEOUT socket option). So both TimeoutError and
socket.timeout can indicate a non-responsive remote system, and the
actual difference is in who implements the timeout: socket.timeout
means that the timeout implemented by the Python interpreter and
controlled by socket.settimeout() expired; TimeoutError means that the
timeout implemented by the kernel expired.

You can also get TimeoutError in other odd places like the pthread
synchronization functions that take a timeout, or when working with
POSIX message queues for IPC. These do describe stuff that's on the
same system timing out, but since these APIs aren't exposed by Python
I'm not actually sure if it's possible in practice to get a
TimeoutError except with a socket talking to a non-responsive peer.

-n

[1] https://stackoverflow.com/questions/8471577/linux-tcp-connect-failure-with-etimedout

-- 
Nathaniel J. Smith -- https://vorpus.org

From breamoreboy at yahoo.co.uk  Mon Apr  3 02:29:40 2017
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 3 Apr 2017 07:29:40 +0100
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
 <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>
Message-ID: <obsq4q$992$1@blaine.gmane.org>

On 03/04/2017 02:22, Neil Girdhar wrote:
> Same.  One day, Python will have a decent parsing library.
>

Nothing here https://wiki.python.org/moin/LanguageParsing suits your needs?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From mistersheik at gmail.com  Mon Apr  3 04:25:55 2017
From: mistersheik at gmail.com (Neil Girdhar)
Date: Mon, 03 Apr 2017 08:25:55 +0000
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <obsq4q$992$1@blaine.gmane.org>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
 <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>
 <obsq4q$992$1@blaine.gmane.org>
Message-ID: <CAA68w_m1gH_ALJ_Me2MfASmWFzb4VHp58rb4tc2g4wRKzihq+A@mail.gmail.com>

On Mon, Apr 3, 2017 at 2:31 AM Mark Lawrence via Python-ideas <
python-ideas at python.org> wrote:

> On 03/04/2017 02:22, Neil Girdhar wrote:
> > Same.  One day, Python will have a decent parsing library.
> >
>
> Nothing here https://wiki.python.org/moin/LanguageParsing suits your
> needs?
>

No, unfortunately.

I tried to make a simple grammar that parses latex code, and it was
basically impossible with these tools.

>From what I remember, you need the match objects to be able to accept or
reject their matched sub-nodes.

It's same thing if you want to parse Python in one pass (not the usual two
passes that CPython does whereby it creates an AST and then validates it).
It would be cooler to validate as you go since the errors can be much
richer since you have the whole parsing context?

It's been a while, so I might be forgetting something, but I remember
thinking that I'll check back in five years and see if anything new has
come out.

>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
>
> Mark Lawrence
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/FSd6xLHowg8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170403/5f32b413/attachment.html>

From desmoulinmichel at gmail.com  Mon Apr  3 04:33:56 2017
From: desmoulinmichel at gmail.com (Michel Desmoulin)
Date: Mon, 3 Apr 2017 10:33:56 +0200
Subject: [Python-ideas] pathlib.Path.walk
Message-ID: <d49e5a55-bd67-f5c3-5d6f-a2ab12556e12@gmail.com>

Like os.walk, but from a Path instance.

We have Path.iterdir, but it's not recursive. Which you use either
os.scandir, or os.walk. In any case, you end up doing:

import os
import pathlib

directory = pathlib.Path(get_dir())

# do things with directory

for root, dirs, files os.walk(directory):
    root = Path(root)
    for f in files:
        f = root / f
            # do something with file

Which is not that bad, but you waste a lot of time discovering how to do
that since you look first for something like Path.walk.


From songofacandy at gmail.com  Mon Apr  3 04:38:37 2017
From: songofacandy at gmail.com (INADA Naoki)
Date: Mon, 3 Apr 2017 17:38:37 +0900
Subject: [Python-ideas] pathlib.Path.walk
In-Reply-To: <d49e5a55-bd67-f5c3-5d6f-a2ab12556e12@gmail.com>
References: <d49e5a55-bd67-f5c3-5d6f-a2ab12556e12@gmail.com>
Message-ID: <CAEfz+TzaH_93JsbJqT6v2P4af1tLi5L1F=tGw6V0TixK8Tt1pQ@mail.gmail.com>

Why not Path.glob?
https://docs.python.org/3.6/library/pathlib.html#pathlib.Path.glob

On Mon, Apr 3, 2017 at 5:33 PM, Michel Desmoulin
<desmoulinmichel at gmail.com> wrote:
> Like os.walk, but from a Path instance.
>
> We have Path.iterdir, but it's not recursive. Which you use either
> os.scandir, or os.walk. In any case, you end up doing:
>
> import os
> import pathlib
>
> directory = pathlib.Path(get_dir())
>
> # do things with directory
>
> for root, dirs, files os.walk(directory):
>     root = Path(root)
>     for f in files:
>         f = root / f
>             # do something with file
>
> Which is not that bad, but you waste a lot of time discovering how to do
> that since you look first for something like Path.walk.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

From songofacandy at gmail.com  Mon Apr  3 07:53:20 2017
From: songofacandy at gmail.com (INADA Naoki)
Date: Mon, 3 Apr 2017 20:53:20 +0900
Subject: [Python-ideas] pathlib.Path.walk
In-Reply-To: <d49e5a55-bd67-f5c3-5d6f-a2ab12556e12@gmail.com>
References: <d49e5a55-bd67-f5c3-5d6f-a2ab12556e12@gmail.com>
Message-ID: <CAEfz+TzAwUFs14hCpNCM0dO0N0tLOT7+W431_7B1JSxByW3RbQ@mail.gmail.com>

On Mon, Apr 3, 2017 at 5:33 PM, Michel Desmoulin
<desmoulinmichel at gmail.com> wrote:
> Like os.walk, but from a Path instance.
>
> We have Path.iterdir, but it's not recursive. Which you use either
> os.scandir, or os.walk. In any case, you end up doing:
>
> import os
> import pathlib
>
> directory = pathlib.Path(get_dir())
>
> # do things with directory
>
> for root, dirs, files os.walk(directory):
>     root = Path(root)
>     for f in files:
>         f = root / f
>             # do something with file
>
> Which is not that bad, but you waste a lot of time discovering how to do
> that since you look first for something like Path.walk.
>

OK, but what Path.walk should be return?
If all of dirs and files are Path object, it will have significant
performance overhead.

for root, dirs, files in directory.walk():
    # all dirs are path object, but not used.
    for f in files:
        f = root / f  # And Path overhead can be avoided for files too.

From rymg19 at gmail.com  Mon Apr  3 08:54:29 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Mon, 3 Apr 2017 07:54:29 -0500
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <CAA68w_m1gH_ALJ_Me2MfASmWFzb4VHp58rb4tc2g4wRKzihq+A@mail.gmail.com>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
 <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>
 <obsq4q$992$1@blaine.gmane.org>
 <CAA68w_m1gH_ALJ_Me2MfASmWFzb4VHp58rb4tc2g4wRKzihq+A@mail.gmail.com>
Message-ID: <CAO41-mNy77ND0uox4Tp3T-nFcNgmrNcKsWjH3cHdBUzTd0iA5A@mail.gmail.com>

Have you tried PyParsing and/or Grako? They're some of my favorites (well,
I like PLY too, but I'm thinking you wouldn't like it too much).

--
Ryan (????)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://refi64.com

On Apr 3, 2017 3:26 AM, "Neil Girdhar" <mistersheik at gmail.com> wrote:

>
>
> On Mon, Apr 3, 2017 at 2:31 AM Mark Lawrence via Python-ideas <
> python-ideas at python.org> wrote:
>
>> On 03/04/2017 02:22, Neil Girdhar wrote:
>> > Same.  One day, Python will have a decent parsing library.
>> >
>>
>> Nothing here https://wiki.python.org/moin/LanguageParsing suits your
>> needs?
>>
>
> No, unfortunately.
>
> I tried to make a simple grammar that parses latex code, and it was
> basically impossible with these tools.
>
> From what I remember, you need the match objects to be able to accept or
> reject their matched sub-nodes.
>
> It's same thing if you want to parse Python in one pass (not the usual two
> passes that CPython does whereby it creates an AST and then validates it).
> It would be cooler to validate as you go since the errors can be much
> richer since you have the whole parsing context?
>
> It's been a while, so I might be forgetting something, but I remember
> thinking that I'll check back in five years and see if anything new has
> come out.
>
>>
>> --
>> My fellow Pythonistas, ask not what our language can do for you, ask
>> what you can do for our language.
>>
>> Mark Lawrence
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "python-ideas" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/
>> topic/python-ideas/FSd6xLHowg8/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> python-ideas+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170403/92c9d70f/attachment.html>

From mistersheik at gmail.com  Mon Apr  3 08:57:30 2017
From: mistersheik at gmail.com (Neil Girdhar)
Date: Mon, 03 Apr 2017 12:57:30 +0000
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <CAO41-mNy77ND0uox4Tp3T-nFcNgmrNcKsWjH3cHdBUzTd0iA5A@mail.gmail.com>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
 <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>
 <obsq4q$992$1@blaine.gmane.org>
 <CAA68w_m1gH_ALJ_Me2MfASmWFzb4VHp58rb4tc2g4wRKzihq+A@mail.gmail.com>
 <CAO41-mNy77ND0uox4Tp3T-nFcNgmrNcKsWjH3cHdBUzTd0iA5A@mail.gmail.com>
Message-ID: <CAA68w_nQ_dToPU6OP-iSiJ4gmKSVJw8z3Yb0rwuV+P2kCLJF2g@mail.gmail.com>

I've tried PyParsing.  I haven't tried Grako.

On Mon, Apr 3, 2017 at 8:54 AM Ryan Gonzalez <rymg19 at gmail.com> wrote:

> Have you tried PyParsing and/or Grako? They're some of my favorites (well,
> I like PLY too, but I'm thinking you wouldn't like it too much).
>
> --
> Ryan (????)
> Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
> http://refi64.com
>
> On Apr 3, 2017 3:26 AM, "Neil Girdhar" <mistersheik at gmail.com> wrote:
>
>
>
> On Mon, Apr 3, 2017 at 2:31 AM Mark Lawrence via Python-ideas <
> python-ideas at python.org> wrote:
>
> On 03/04/2017 02:22, Neil Girdhar wrote:
> > Same.  One day, Python will have a decent parsing library.
> >
>
> Nothing here https://wiki.python.org/moin/LanguageParsing suits your
> needs?
>
>
> No, unfortunately.
>
> I tried to make a simple grammar that parses latex code, and it was
> basically impossible with these tools.
>
> From what I remember, you need the match objects to be able to accept or
> reject their matched sub-nodes.
>
> It's same thing if you want to parse Python in one pass (not the usual two
> passes that CPython does whereby it creates an AST and then validates it).
> It would be cooler to validate as you go since the errors can be much
> richer since you have the whole parsing context?
>
> It's been a while, so I might be forgetting something, but I remember
> thinking that I'll check back in five years and see if anything new has
> come out.
>
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
>
> Mark Lawrence
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/FSd6xLHowg8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170403/467a4c85/attachment-0001.html>

From apalala at gmail.com  Mon Apr  3 13:06:58 2017
From: apalala at gmail.com (=?UTF-8?Q?Juancarlo_A=C3=B1ez?=)
Date: Mon, 3 Apr 2017 13:06:58 -0400
Subject: [Python-ideas] What about regexp string litterals : re".*" ?
In-Reply-To: <CAA68w_nQ_dToPU6OP-iSiJ4gmKSVJw8z3Yb0rwuV+P2kCLJF2g@mail.gmail.com>
References: <20170327151740.GJ6883@tabr.siet.ch>
 <CADiSq7f23kG_c7uhRQYADxT=t90WAnmDP_m17QvNHwPJGnpZeQ@mail.gmail.com>
 <CAKr=oZvaJejTL_xsgb=1G1hvDvb3C2R5=gEK1zfTSfWD1Kqe2Q@mail.gmail.com>
 <22750.1027.251684.274189@turnbull.sk.tsukuba.ac.jp>
 <CAOOa=pNyALtUESHcsYsydB-ORZsn-Avym+3v9s2kyzDf=cA7LQ@mail.gmail.com>
 <cc244c2f-8017-4099-939f-e9dd2dd2c24c@googlegroups.com>
 <obsq4q$992$1@blaine.gmane.org>
 <CAA68w_m1gH_ALJ_Me2MfASmWFzb4VHp58rb4tc2g4wRKzihq+A@mail.gmail.com>
 <CAO41-mNy77ND0uox4Tp3T-nFcNgmrNcKsWjH3cHdBUzTd0iA5A@mail.gmail.com>
 <CAA68w_nQ_dToPU6OP-iSiJ4gmKSVJw8z3Yb0rwuV+P2kCLJF2g@mail.gmail.com>
Message-ID: <CAN1YFWsvKDXDgFoZW0JAeGFCu3E3KxKB-CV=BKTP-AaZSR7JvQ@mail.gmail.com>

On Mon, Apr 3, 2017 at 8:57 AM, Neil Girdhar <mistersheik at gmail.com> wrote:

> I've tried PyParsing.  I haven't tried Grako.
>

Caveat: I'm the author of Grako.

It's very easy to do complex parsing with Grako. The grammar can be
embedded in a Python string, and the compiled grammar can be used for
parsing without generating any Python code. Most of the unit tests under
the distribution's grako/grako/test use those features.

https://pypi.org/project/grako/

One of the ways in which a top-down grammar (as those accepted by Grako)
can be used is to organize a series of regular expressions into a tree to
handle complex cases with clarity.


-- 
Juancarlo *A?ez*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170403/2da19cb0/attachment.html>

From anthony at xtfx.me  Thu Apr  6 11:53:16 2017
From: anthony at xtfx.me (C Anthony Risinger)
Date: Thu, 6 Apr 2017 10:53:16 -0500
Subject: [Python-ideas] Relative import of self or parent package?
Message-ID: <CAGAVQTGWsOid4=j=DoXppSySpS3=XbiwP--YLxfCfeertb45uA@mail.gmail.com>

I occasionally want to do something like this:

import .. as parent

or:

import . as self

The pattern is a little less useful for namespace packages (which I've been
trying to use almost exclusively, along with relative imports) than it is
for __init__.py based packages, but there is still some utility.

Is there a reason one can't import dots directly and bind them to a name? I
did not find any text around this and the closest I've come up with is:

from .. import __name__ as parent_module_name
parent = __import__(parent_module_name)

Could we support relative imports without `from` so long as there is an
`as`?

Thanks,

-- 

C Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170406/ecafa191/attachment.html>

From victor.varvariuc at gmail.com  Sat Apr  8 01:54:50 2017
From: victor.varvariuc at gmail.com (Victor Varvariuc)
Date: Sat, 8 Apr 2017 08:54:50 +0300
Subject: [Python-ideas] Make `import a.b.c as m` is equivalent to `m =
 sys.modules['a.b.c']`
Message-ID: <5FE16C04-A235-413A-A3AA-895F6E107115@gmail.com>

Hi there.

I asked a question <http://stackoverflow.com/questions/41845671/import-as-in-python-3> on Stackoverflow:

(Pdb) import brain.utils.mail
(Pdb) import brain.utils.mail as mail_utils
*** AttributeError: module 'brain.utils' has no attribute 'mail'
I always thought that import a.b.c as m is roughly equivalent to m = sys.modules['a.b.c']. Why AttributeError? Python 3.6

I was pointed out <http://stackoverflow.com/a/24968941/248296> that this is a somewhat weird behavior of Python:

The statement is not quite true, as evidenced by the corner case you met, namely if the required modules already exist in sys.modules but are yet uninitialized. The import ... as requires that the module foo.bar is injected in foo namespace as the attribute bar, in addition to being in sys.modules, whereas the from ... import ... as looks for foo.bar in sys.modules.

Why would `import a.b.c` work when `a.b.c` is not yet fully imported, but `import a.b.c as my_c` would not? I though it would be vice versa.

Using `import a.b.c as my_c` allows avoiding a number of circular import issues. Right now I have to use `from a.b import c as my_c` as a workaround, but this doesn't look right.

Does this make sense? Is there an issue in the Python bug tracker for this? Can/should I create one?


Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170408/007c75a4/attachment.html>

From ncoghlan at gmail.com  Sat Apr  8 02:17:48 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 8 Apr 2017 16:17:48 +1000
Subject: [Python-ideas] Relative import of self or parent package?
In-Reply-To: <CAGAVQTGWsOid4=j=DoXppSySpS3=XbiwP--YLxfCfeertb45uA@mail.gmail.com>
References: <CAGAVQTGWsOid4=j=DoXppSySpS3=XbiwP--YLxfCfeertb45uA@mail.gmail.com>
Message-ID: <CADiSq7cAKa=BTgtVJDqc6Pz=PdWefXpi-wpV4mNzg771F9f+5A@mail.gmail.com>

On 7 April 2017 at 01:53, C Anthony Risinger <anthony at xtfx.me> wrote:
> I occasionally want to do something like this:
>
> import .. as parent
>
> or:
>
> import . as self
>
> The pattern is a little less useful for namespace packages (which I've been
> trying to use almost exclusively, along with relative imports) than it is
> for __init__.py based packages, but there is still some utility.
>
> Is there a reason one can't import dots directly and bind them to a name?

Mainly the fact that globals() already covers most use cases for
module self references, and even when it doesn't the above are (for a
typical module) still only a few lines long:

    import sys
    self = sys.modules[__name__]

and:

    import sys
    parent_name, __, __ = __name__.rpartition(".")
    parent = sys.modules[parent_name]

Getting a reference to the current module or a parent module is also
considered a sufficiently niche requirement that understanding the
existence and behaviour of the sys.modules cache is considered an
acceptable barrier to entry, over adding yet another variant of the
import statement syntax.

It's definitely a case of "Hasn't been considered worth the extra
language complexity so far" rather than "couldn't be done", though -
the requirement to be met would be to establish that the use cases for
self and parent references are sufficiently common that they're
deserving of dedicated syntactic support on par with that already
offered for peer module references.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Apr  8 02:47:34 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 8 Apr 2017 16:47:34 +1000
Subject: [Python-ideas] Make `import a.b.c as m` is equivalent to `m =
 sys.modules['a.b.c']`
In-Reply-To: <5FE16C04-A235-413A-A3AA-895F6E107115@gmail.com>
References: <5FE16C04-A235-413A-A3AA-895F6E107115@gmail.com>
Message-ID: <CADiSq7cCvJgHOBQHMD_+r3bN658LFyGOwxfor69SrWjL41XWpw@mail.gmail.com>

On 8 April 2017 at 15:54, Victor Varvariuc <victor.varvariuc at gmail.com> wrote:
> Hi there.
>
> I asked a question on Stackoverflow:
>
> (Pdb) import brain.utils.mail
>
> (Pdb) import brain.utils.mail as mail_utils
>
> *** AttributeError: module 'brain.utils' has no attribute 'mail'
>
> I always thought that import a.b.c as m is roughly equivalent to m =
> sys.modules['a.b.c']. Why AttributeError? Python 3.6
>
> I was pointed out that this is a somewhat weird behavior of Python:
>
> The statement is not quite true, as evidenced by the corner case you met,
> namely if the required modules already exist in sys.modules but are yet
> uninitialized. The import ... as requires that the module foo.bar is
> injected in foo namespace as the attribute bar, in addition to being in
> sys.modules, whereas the from ... import ... as looks for foo.bar in
> sys.modules.
>
>
> Why would `import a.b.c` work when `a.b.c` is not yet fully imported, but
> `import a.b.c as my_c` would not? I though it would be vice versa.

It as to do with when and how the attribute lookup for the submodule
occurs. Disassembling the 3 examples (plus another variant):

    >>> import dis
    >>> dis.dis("import a.b.c")
     1          0 LOAD_CONST               0 (0)
                 2 LOAD_CONST               1 (None)
                 4 IMPORT_NAME              0 (a.b.c)
                 6 STORE_NAME               1 (a)
                 8 LOAD_CONST               1 (None)
                10 RETURN_VALUE

In the first case, we see that the name bound in the local namespace
is *a*: we won't actually try to access the full "a.b.c" attribute
reference until somewhere later in the module.

That lazy lookup means the import system is satisfied based on the
sys.modules entry, and the module code will be happy as long as "a.b"
has a "c" attribute by the time the module needs it.

    >>> dis.dis("import a.b.c as m")
      1         0 LOAD_CONST               0 (0)
                 2 LOAD_CONST               1 (None)
                 4 IMPORT_NAME              0 (a.b.c)
                 6 LOAD_ATTR                1 (b)
                 8 LOAD_ATTR                2 (c)
               10 STORE_NAME               3 (m)
               12 LOAD_CONST               1 (None)
               14 RETURN_VALUE

In the second case, we see that "import a.b.c as m" produces almost
exactly the same bytecode as "import a.b.c; m = a.b.c", skipping only
the store-and-reload of the "a" reference:

    >>> dis.dis("import a.b.c; m = a.b.c")
     1          0 LOAD_CONST               0 (0)
                 2 LOAD_CONST               1 (None)
                 4 IMPORT_NAME              0 (a.b.c)
                 6 STORE_NAME               1 (a)
                 8 LOAD_NAME                1 (a)
               10 LOAD_ATTR                2 (b)
               12 LOAD_ATTR                3 (c)
               14 STORE_NAME               4 (m)
               16 LOAD_CONST               1 (None)
               18 RETURN_VALUE

That eager lookup means that "a.b" *must* have a "c" attribute at the
time of the import, or the LOAD_ATTR operation will fail.

    >>> dis.dis("from a.b import c as m")
      1         0 LOAD_CONST               0 (0)
                 2 LOAD_CONST               1 (('c',))
                 4 IMPORT_NAME              0 (a.b)
                 6 IMPORT_FROM              1 (c)
                 8 STORE_NAME               2 (m)
               10 POP_TOP
               12 LOAD_CONST               2 (None)
               14 RETURN_VALUE

Finally, when we come to the "from" import case, we can see that the
bytecode changes significantly: the LOAD_ATTRs all disappear, replaced
by a single IMPORT_FROM, and the from_list for the IMPORT_NAME opcode
is a populated tuple, rather than None.

To address the next question that follows on from looking at the
current compile time behaviour, changing those semantics isn't as
simple as just having the compiler reinterpret "import x.y.z as m" as
"from x.y import z as m". The problem is that the two formulations
make different assertions about the nature of "z" (in the first form,
it's expected to be a submodule, in the second, it can be any
attribute of x.y), so adding such a reinterpretation would either move
the discrepancy to having "import x.y.z as m" work in cases where
"import x.y.z" fails, or else require adding a way for the compiler to
indicate that the resolved attribute must *also* exist in sys.modules,
in addition to being an attribute on the parent module.

That's not impossible to resolve, but it would be a lot of work just
to add an alternate spelling of "from a.b import c as m".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From victor.varvariuc at gmail.com  Sun Apr  9 07:03:59 2017
From: victor.varvariuc at gmail.com (Victor Varvariuc)
Date: Sun, 9 Apr 2017 14:03:59 +0300
Subject: [Python-ideas] Make `import a.b.c as m` is equivalent to `m =
 sys.modules['a.b.c']`
In-Reply-To: <CADiSq7fM6yUEHGR8GmEMg+Q9Pd4P1kuXnaxeY1oDCLB71Ov0hw@mail.gmail.com>
References: <5FE16C04-A235-413A-A3AA-895F6E107115@gmail.com>
 <CADiSq7cCvJgHOBQHMD_+r3bN658LFyGOwxfor69SrWjL41XWpw@mail.gmail.com>
 <89BFABB4-6E30-4993-B63D-1AA0F88E8897@gmail.com>
 <CADiSq7ddczmiNbxRKo15O1pWVb_b+8XQZ08e=V1=WE7g01jvkQ@mail.gmail.com>
 <0E02798E-21D5-45E7-847F-3E3ACA46B71D@gmail.com>
 <CADiSq7fM6yUEHGR8GmEMg+Q9Pd4P1kuXnaxeY1oDCLB71Ov0hw@mail.gmail.com>
Message-ID: <0ACC1C61-B388-47BF-B172-EDC915C47C5B@gmail.com>

Thanks for the reply. CC'ing the mailing list.


> On 9 Apr 2017, at 10:05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 9 April 2017 at 16:13, Victor Varvariuc <victor.varvariuc at gmail.com> wrote:
>> I think there are can be several ways to achieve the result.
>> 
>> The question is if what I've presented is a valid concern. I think this would make the language more consistent and help reducing number of errors related to import without breaking old code.
>> 
>> May I create an issue in the bug tracker?
> 
> Aye, go ahead - I would previously have advised against it, but given
> at least one relatively low impact path to resolution, it's worth
> filing it as an enhancement request.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia




> Begin forwarded message:
> 
> From: Nick Coghlan <ncoghlan at gmail.com>
> Subject: Re: [Python-ideas] Make `import a.b.c as m` is equivalent to `m = sys.modules['a.b.c']`
> Date: 9 April 2017 at 08:33:43 GMT+3
> To: Victor Varvariuc <victor.varvariuc at gmail.com>
> 
> On 8 April 2017 at 22:57, Victor Varvariuc <victor.varvariuc at gmail.com> wrote:
>> Thanks, Nick, for the thorough explanation.
>> 
>> changing those semantics isn't as
>> simple as just having the compiler reinterpret "import x.y.z as m" as
>> "from x.y import z as m"
>> 
>> 
>> I don't think this is a solution.
>> 
>> The solution, IMO, is to treat  `import x.y.z as m` as `m =
>> __import__('x.y.z')` (or similar, IIRC).
> 
> That's not quite how the `__import__` hook works: when the fromlist is
> empty, it returns the *base* module, not the most resolved one.
> 
>>>> __import__("unittest.mock")
>    <module 'unittest' from '/usr/lib64/python3.5/unittest/__init__.py'>
> 
> However, the idea does indicate a potential resolution, which would be
> to add a new IMPORT_ATTR opcode with roughly the following semantics:
> 
>    try:
>        _op_result = getattr(obj, ATTR)
>    except AttributeError:
>        _package_name = obj.__package__
>        if not _package_name:
>            raise
>        _op_result = sys.modules[_package_name + "." + ATTR]
> 
> And then have the import statement unconditionally emit IMPORT_ATTR
> instead of LOAD_ATTR for the "import x.* as y" case.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170409/5baa2f9b/attachment.html>

From victor.varvariuc at gmail.com  Sun Apr  9 07:13:20 2017
From: victor.varvariuc at gmail.com (Victor Varvariuc)
Date: Sun, 9 Apr 2017 14:13:20 +0300
Subject: [Python-ideas] Make `import a.b.c as m` is equivalent to `m =
 sys.modules['a.b.c']`
In-Reply-To: <CADiSq7fM6yUEHGR8GmEMg+Q9Pd4P1kuXnaxeY1oDCLB71Ov0hw@mail.gmail.com>
References: <5FE16C04-A235-413A-A3AA-895F6E107115@gmail.com>
 <CADiSq7cCvJgHOBQHMD_+r3bN658LFyGOwxfor69SrWjL41XWpw@mail.gmail.com>
 <89BFABB4-6E30-4993-B63D-1AA0F88E8897@gmail.com>
 <CADiSq7ddczmiNbxRKo15O1pWVb_b+8XQZ08e=V1=WE7g01jvkQ@mail.gmail.com>
 <0E02798E-21D5-45E7-847F-3E3ACA46B71D@gmail.com>
 <CADiSq7fM6yUEHGR8GmEMg+Q9Pd4P1kuXnaxeY1oDCLB71Ov0hw@mail.gmail.com>
Message-ID: <B8593FFC-3A05-4FE9-95BF-1D604319597F@gmail.com>

The issue: http://bugs.python.org/issue30024 <http://bugs.python.org/issue30024>

> On 9 Apr 2017, at 10:05, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 9 April 2017 at 16:13, Victor Varvariuc <victor.varvariuc at gmail.com> wrote:
>> I think there are can be several ways to achieve the result.
>> 
>> The question is if what I've presented is a valid concern. I think this would make the language more consistent and help reducing number of errors related to import without breaking old code.
>> 
>> May I create an issue in the bug tracker?
> 
> Aye, go ahead - I would previously have advised against it, but given
> at least one relatively low impact path to resolution, it's worth
> filing it as an enhancement request.
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170409/0ab385e2/attachment.html>

From eric at trueblade.com  Sun Apr  9 19:49:22 2017
From: eric at trueblade.com (Eric V. Smith)
Date: Sun, 9 Apr 2017 19:49:22 -0400
Subject: [Python-ideas] Proposal: making __str__ count in time's class
In-Reply-To: <CAMgCxhKDcXjPaB8yoeSvyhr+u_bT=cmHCzqoWPiu-gd3_bM+dQ@mail.gmail.com>
References: <CAMgCxhKDcXjPaB8yoeSvyhr+u_bT=cmHCzqoWPiu-gd3_bM+dQ@mail.gmail.com>
Message-ID: <726593d1-20b9-e0be-bfc8-7d18d2d94272@trueblade.com>

On 3/8/2017 11:01 AM, Francesco Franchina wrote:
> Hello everyone,
> 
> I'm shortly writing to you about a reflection I lately made upon the 
> current functioning of __str__ for the time's class.
> 
> Before expressing my thought and proposal, I want to make sure we all 
> agree on a simple and clear fact:
> the __str__ magic method is used to give a literal and human-readable 
> representation to the object (unlike __repr__).
> 
> Generally this is true across the python panorama. It's not true for the 
> time class, for example.
> /
>  >>> import time
>  >>> a = time.localtime()
>  >>> a.__str__()
> 'time.struct_time(tm_year=2017, tm_mon=3, tm_mday=8, tm_hour=16, 
> tm_min=6, tm_sec=16, tm_wday=2, tm_yday=67, tm_isdst=0)'/
> 
> Well, don't get me wrong: the main aim of the __str__ method has been 
> accomplished but, imho, not in the most pythonic way.
> 
> I just wanted to ask you: what do you think about re-writing the __str__ 
> of the time class so it would return something like
> ISO 8601 [https://en.wikipedia.org/wiki/ISO_8601 
> <https://en.wikipedia.org/wiki/ISO_8601>] format? Wouldn't it be more 
> meaningful? Especially in the JS-everywhere-era
> it could be more more productive.
> 
> 
> *TL;DR*
> __str__ for dates should return a human-readable date format (eg: 
> https://en.wikipedia.org/wiki/ISO_8601 
> <https://en.wikipedia.org/wiki/ISO_8601>)
> 
> 
> I'm waiting for your opinions.
> Thank you for your time and ideas!

I don't think we can change __str__ at this point, but I'd support 
adding __format__ to make this easier to control. Presumably it would 
just call strftime.

Eric.


From kamal.mustafa at gmail.com  Tue Apr 11 21:56:40 2017
From: kamal.mustafa at gmail.com (Kamal Mustafa)
Date: Wed, 12 Apr 2017 09:56:40 +0800
Subject: [Python-ideas] "import me" to display some summary of the current
 python installation
Message-ID: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>

On the same theme with "import this" and "import antigravity", "import
me" I think can be both some fun way to introduce Python and also
offer a helpful functionality when troubleshooting python installation
with someone new to the language.

Currently, we have to use some myriad approach like "show me the
output of `which python`" to know the basic info of the python
installation someone having trouble or asking question.

From wes.turner at gmail.com  Tue Apr 11 22:28:22 2017
From: wes.turner at gmail.com (Wes Turner)
Date: Tue, 11 Apr 2017 21:28:22 -0500
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
Message-ID: <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>

python -m site
https://github.com/python/cpython/blob/master/Lib/site.py
- _script()


On Tuesday, April 11, 2017, Kamal Mustafa <kamal.mustafa at gmail.com> wrote:

> On the same theme with "import this" and "import antigravity", "import
> me" I think can be both some fun way to introduce Python and also
> offer a helpful functionality when troubleshooting python installation
> with someone new to the language.
>
> Currently, we have to use some myriad approach like "show me the
> output of `which python`" to know the basic info of the python
> installation someone having trouble or asking question.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170411/664220d8/attachment.html>

From rymg19 at gmail.com  Tue Apr 11 22:41:32 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 11 Apr 2017 21:41:32 -0500
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
Message-ID: <CAO41-mMNpsoy7c1Qs5D0OtwZJMvUmtK+vYY9Ft9gcpZcbsW=Nw@mail.gmail.com>

...except it would break everybody who has a module named `me` (which
admittedly isn't too common...).

Something like `import this.what` or something like that would be cool,
though.

--
Ryan (????)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://refi64.com

On Apr 11, 2017 9:04 PM, "Kamal Mustafa" <kamal.mustafa at gmail.com> wrote:

> On the same theme with "import this" and "import antigravity", "import
> me" I think can be both some fun way to introduce Python and also
> offer a helpful functionality when troubleshooting python installation
> with someone new to the language.
>
> Currently, we have to use some myriad approach like "show me the
> output of `which python`" to know the basic info of the python
> installation someone having trouble or asking question.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170411/c4d7953e/attachment.html>

From kamal.mustafa at gmail.com  Tue Apr 11 23:02:28 2017
From: kamal.mustafa at gmail.com (Kamal Mustafa)
Date: Wed, 12 Apr 2017 11:02:28 +0800
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>
Message-ID: <CAJey_ytp+HAhCSd6H97CmstvEc2njotssfnS72gxqz+8npyEJw@mail.gmail.com>

Cool ! I don't know this.

On 12 April 2017 at 10:28, Wes Turner <wes.turner at gmail.com> wrote:
> python -m site
> https://github.com/python/cpython/blob/master/Lib/site.py
> - _script()
>
>
> On Tuesday, April 11, 2017, Kamal Mustafa <kamal.mustafa at gmail.com> wrote:
>>
>> On the same theme with "import this" and "import antigravity", "import
>> me" I think can be both some fun way to introduce Python and also
>> offer a helpful functionality when troubleshooting python installation
>> with someone new to the language.
>>
>> Currently, we have to use some myriad approach like "show me the
>> output of `which python`" to know the basic info of the python
>> installation someone having trouble or asking question.
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/

From steve at pearwood.info  Tue Apr 11 23:04:50 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 12 Apr 2017 13:04:50 +1000
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>
Message-ID: <20170412030448.GC9464@ando.pearwood.info>

On Tue, Apr 11, 2017 at 09:28:22PM -0500, Wes Turner wrote:

> python -m site
> https://github.com/python/cpython/blob/master/Lib/site.py
> - _script()


Wes, I have no idea what that code snippet is supposed to do, or even 
whether it is supposed to be working code.

Remember that we can't read your mind. If you're experimenting with code 
and trying things out, please don't waste the time of everyone else by 
doing it on the mailing list.



-- 
Steve

From steve at pearwood.info  Tue Apr 11 23:18:42 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 12 Apr 2017 13:18:42 +1000
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
Message-ID: <20170412031841.GD9464@ando.pearwood.info>

On Wed, Apr 12, 2017 at 09:56:40AM +0800, Kamal Mustafa wrote:

> On the same theme with "import this" and "import antigravity", "import
> me" I think can be both some fun way to introduce Python and also
> offer a helpful functionality when troubleshooting python installation
> with someone new to the language.

If your Python installation isn't working, how are you supposed to run 
"import me"?

You're a bit vague about what this "import me" is actually going to do. 
Can you show a mock-up of the output you expect?

Both import this and antigravity are Easter eggs. I'm not entirely sure 
I like the idea of having a non-joke module run as a script on import. 
At the very least, it makes testing harder.

But having said that, I like the idea of a standard, simple way of 
getting a good set of diagnostic information.


> Currently, we have to use some myriad approach like "show me the
> output of `which python`" to know the basic info of the python
> installation someone having trouble or asking question.

What's wrong with that? Apart from not working on Windows.

If you are trouble-shooting calling Python from the shell, you need to 
see what the shell thinks is your Python executable, which may be an 
alias. For example, I might have:

[steve at ando ~]$ which python3.5
alias python3.5='env -u PYTHONSTARTUP python3.5'
        /usr/bin/env

Now, if I forget and wonder why Python 3.5 doesn't run my startup file, 
I can't troubleshoot that from inside Python 3.5 since the problem is in 
the shell, not Python.


-- 
Steve

From carl.input at gmail.com  Wed Apr 12 05:31:05 2017
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 12 Apr 2017 10:31:05 +0100
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <20170412031841.GD9464@ando.pearwood.info>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <20170412031841.GD9464@ando.pearwood.info>
Message-ID: <CAP-uhDe0ywOONAKBxydH8NUUNM8C4x_ZiTCrKQrfw7xmDvc2Jg@mail.gmail.com>

I actually did have `me` defined, but changed it to `my` recently. I use it
in my shell to store my personal details, like my mobile number, API keys
etc. It's handy being able to do `my.mobile` when I need that info quickly.

Either way, the default banner should, and generally does, point new users
to the help they need.

-- Carl Smith
carl.input at gmail.com

On 12 April 2017 at 04:18, Steven D'Aprano <steve at pearwood.info> wrote:

> On Wed, Apr 12, 2017 at 09:56:40AM +0800, Kamal Mustafa wrote:
>
> > On the same theme with "import this" and "import antigravity", "import
> > me" I think can be both some fun way to introduce Python and also
> > offer a helpful functionality when troubleshooting python installation
> > with someone new to the language.
>
> If your Python installation isn't working, how are you supposed to run
> "import me"?
>
> You're a bit vague about what this "import me" is actually going to do.
> Can you show a mock-up of the output you expect?
>
> Both import this and antigravity are Easter eggs. I'm not entirely sure
> I like the idea of having a non-joke module run as a script on import.
> At the very least, it makes testing harder.
>
> But having said that, I like the idea of a standard, simple way of
> getting a good set of diagnostic information.
>
>
> > Currently, we have to use some myriad approach like "show me the
> > output of `which python`" to know the basic info of the python
> > installation someone having trouble or asking question.
>
> What's wrong with that? Apart from not working on Windows.
>
> If you are trouble-shooting calling Python from the shell, you need to
> see what the shell thinks is your Python executable, which may be an
> alias. For example, I might have:
>
> [steve at ando ~]$ which python3.5
> alias python3.5='env -u PYTHONSTARTUP python3.5'
>         /usr/bin/env
>
> Now, if I forget and wonder why Python 3.5 doesn't run my startup file,
> I can't troubleshoot that from inside Python 3.5 since the problem is in
> the shell, not Python.
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170412/00909ee4/attachment.html>

From carl.input at gmail.com  Wed Apr 12 05:32:31 2017
From: carl.input at gmail.com (Carl Smith)
Date: Wed, 12 Apr 2017 10:32:31 +0100
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CAP-uhDe0ywOONAKBxydH8NUUNM8C4x_ZiTCrKQrfw7xmDvc2Jg@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <20170412031841.GD9464@ando.pearwood.info>
 <CAP-uhDe0ywOONAKBxydH8NUUNM8C4x_ZiTCrKQrfw7xmDvc2Jg@mail.gmail.com>
Message-ID: <CAP-uhDdscMx1TDzYR-3S207fN9Rb21JUFk-62qh9ZurNjB66nw@mail.gmail.com>

Sorry. Ignore my last message. I thought this conversation was on the
IPython Dev list.

-- Carl Smith
carl.input at gmail.com

On 12 April 2017 at 10:31, Carl Smith <carl.input at gmail.com> wrote:

> I actually did have `me` defined, but changed it to `my` recently. I use
> it in my shell to store my personal details, like my mobile number, API
> keys etc. It's handy being able to do `my.mobile` when I need that info
> quickly.
>
> Either way, the default banner should, and generally does, point new users
> to the help they need.
>
> -- Carl Smith
> carl.input at gmail.com
>
> On 12 April 2017 at 04:18, Steven D'Aprano <steve at pearwood.info> wrote:
>
>> On Wed, Apr 12, 2017 at 09:56:40AM +0800, Kamal Mustafa wrote:
>>
>> > On the same theme with "import this" and "import antigravity", "import
>> > me" I think can be both some fun way to introduce Python and also
>> > offer a helpful functionality when troubleshooting python installation
>> > with someone new to the language.
>>
>> If your Python installation isn't working, how are you supposed to run
>> "import me"?
>>
>> You're a bit vague about what this "import me" is actually going to do.
>> Can you show a mock-up of the output you expect?
>>
>> Both import this and antigravity are Easter eggs. I'm not entirely sure
>> I like the idea of having a non-joke module run as a script on import.
>> At the very least, it makes testing harder.
>>
>> But having said that, I like the idea of a standard, simple way of
>> getting a good set of diagnostic information.
>>
>>
>> > Currently, we have to use some myriad approach like "show me the
>> > output of `which python`" to know the basic info of the python
>> > installation someone having trouble or asking question.
>>
>> What's wrong with that? Apart from not working on Windows.
>>
>> If you are trouble-shooting calling Python from the shell, you need to
>> see what the shell thinks is your Python executable, which may be an
>> alias. For example, I might have:
>>
>> [steve at ando ~]$ which python3.5
>> alias python3.5='env -u PYTHONSTARTUP python3.5'
>>         /usr/bin/env
>>
>> Now, if I forget and wonder why Python 3.5 doesn't run my startup file,
>> I can't troubleshoot that from inside Python 3.5 since the problem is in
>> the shell, not Python.
>>
>>
>> --
>> Steve
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170412/a0f984b3/attachment.html>

From breamoreboy at yahoo.co.uk  Wed Apr 12 06:22:20 2017
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 12 Apr 2017 11:22:20 +0100
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <20170412030448.GC9464@ando.pearwood.info>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>
 <20170412030448.GC9464@ando.pearwood.info>
Message-ID: <ockv5b$ng0$1@blaine.gmane.org>

On 12/04/2017 04:04, Steven D'Aprano wrote:
> On Tue, Apr 11, 2017 at 09:28:22PM -0500, Wes Turner wrote:
> 
>> python -m site
>> https://github.com/python/cpython/blob/master/Lib/site.py
>> - _script()
>
> Wes, I have no idea what that code snippet is supposed to do, or even
> whether it is supposed to be working code.
> 
> Remember that we can't read your mind. If you're experimenting with code
> and trying things out, please don't waste the time of everyone else by
> doing it on the mailing list.
> 

https://docs.python.org/3/library/site.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From kamal.mustafa at gmail.com  Wed Apr 12 09:35:56 2017
From: kamal.mustafa at gmail.com (Kamal Mustafa)
Date: Wed, 12 Apr 2017 21:35:56 +0800
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
Message-ID: <CAJey_yuhx7KRuzk2XRdn5VCPYduRvxRhzznUr92yRFdDWyq0xw@mail.gmail.com>

Never mind. site._script() as pointed out by Wes Turner is what I need:-

Python 3.4.2 (default, Oct  8 2014, 10:45:20)
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> site._script()
sys.path = [
    '',
    '/usr/lib/python3.4',
    '/usr/lib/python3.4/plat-x86_64-linux-gnu',
    '/usr/lib/python3.4/lib-dynload',
    '/usr/local/lib/python3.4/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/kamal/.local' (exists)
USER_SITE: '/home/kamal/.local/lib/python3.4/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

On 12 April 2017 at 09:56, Kamal Mustafa <kamal.mustafa at gmail.com> wrote:
> On the same theme with "import this" and "import antigravity", "import
> me" I think can be both some fun way to introduce Python and also
> offer a helpful functionality when troubleshooting python installation
> with someone new to the language.
>
> Currently, we have to use some myriad approach like "show me the
> output of `which python`" to know the basic info of the python
> installation someone having trouble or asking question.

From p.f.moore at gmail.com  Wed Apr 12 10:01:41 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 12 Apr 2017 15:01:41 +0100
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <CAJey_yuhx7KRuzk2XRdn5VCPYduRvxRhzznUr92yRFdDWyq0xw@mail.gmail.com>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <CAJey_yuhx7KRuzk2XRdn5VCPYduRvxRhzznUr92yRFdDWyq0xw@mail.gmail.com>
Message-ID: <CACac1F82Oa50NfgyW1t-NmQzwCVWroP-EEWKGkTSRz-UkEqcPQ@mail.gmail.com>

On 12 April 2017 at 14:35, Kamal Mustafa <kamal.mustafa at gmail.com> wrote:
> Never mind. site._script() as pointed out by Wes Turner is what I need:-
>
> Python 3.4.2 (default, Oct  8 2014, 10:45:20)
> [GCC 4.9.1] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import site
>>>> site._script()
> sys.path = [
>     '',
>     '/usr/lib/python3.4',
>     '/usr/lib/python3.4/plat-x86_64-linux-gnu',
>     '/usr/lib/python3.4/lib-dynload',
>     '/usr/local/lib/python3.4/dist-packages',
>     '/usr/lib/python3/dist-packages',
> ]
> USER_BASE: '/home/kamal/.local' (exists)
> USER_SITE: '/home/kamal/.local/lib/python3.4/site-packages' (doesn't exist)
> ENABLE_USER_SITE: True

Unless you have to get this information from within the Python
interactive prompt, you can just run "python -m site" from the shell
prompt to get the same information.

Paul

From wes.turner at gmail.com  Wed Apr 12 10:02:10 2017
From: wes.turner at gmail.com (Wes Turner)
Date: Wed, 12 Apr 2017 09:02:10 -0500
Subject: [Python-ideas] "import me" to display some summary of the
 current python installation
In-Reply-To: <20170412030448.GC9464@ando.pearwood.info>
References: <CAJey_yv2JnUc1TY+RVRMX-vGgFm8e0va4feOETFxpRY2uLz3gQ@mail.gmail.com>
 <CACfEFw9Q3W0WWM116ggq4yeT5gdwGRZz7U4O9WH=DdqtmzdxRw@mail.gmail.com>
 <20170412030448.GC9464@ando.pearwood.info>
Message-ID: <CACfEFw-fY-GC9GUmFXjWYu6dWFP-xaVQ55+ru4ybCxAXfFMGLw@mail.gmail.com>

Steven,

I had intended to answer the question. If I did not answer this question, I
must have misunderstood the question.

> Never mind. site._script() as pointed out by Wes Turner is what I need:

These commands could also be helpful for the inferred use case:


  $(which pip) --version
  $(which python) -m pip freeze
  python -m pip freeze --user
  python -m pip freeze --l
  python -m pip freeze -r ./requirements.txt

  conda --version
  conda config --show
  conda env export # .yaml

  dotfiles_status()   # westurner/dotfiles/develop/etc/bash/
05-bashrc.dotfiles.sh
   echo $VIRTUAL_ENV
   echo $CONDA_ENVS_PATH $WORKON_HOME

In terms of printing the actual versions of {Python, pkgs, }
for reproducibility,
for Jupyter Notebook, there's:

a %version_information magic command extension:
https://github.com/jrjohansson/version_information/blob/master/version_information/version_information.py

and also %watermark:
https://github.com/rasbt/watermark/blob/master/watermark/watermark.py

A way to print all this environment information from the CLI (as e.g. JSON
and/or YAML) would indeed be useful (e.g. for CI build logs / as a build
artifact).



On Tue, Apr 11, 2017 at 10:04 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Tue, Apr 11, 2017 at 09:28:22PM -0500, Wes Turner wrote:
>
> > python -m site
> > https://github.com/python/cpython/blob/master/Lib/site.py
> > - _script()
>
>
> Wes, I have no idea what that code snippet is supposed to do, or even
> whether it is supposed to be working code.
>
> Remember that we can't read your mind. If you're experimenting with code
> and trying things out, please don't waste the time of everyone else by
> doing it on the mailing list.
>
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170412/a83f9b0d/attachment.html>

From steve at pearwood.info  Thu Apr 13 14:20:07 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Apr 2017 04:20:07 +1000
Subject: [Python-ideas] Discourage operator.__dunder__ functions
Message-ID: <20170413182007.GG9464@ando.pearwood.info>

Notice that I said *discourage* rather than *deprecate*.

Quoting the documentation:

    The operator module exports a set of efficient functions
    corresponding to the intrinsic operators of Python. For 
    example, operator.add(x, y) is equivalent to the expression 
    x+y. The function names are those used for special class 
    methods; variants without leading and trailing __ are also 
    provided for convenience.

https://docs.python.org/3/library/operator.html

I propose four simple documentation changes, and no code change:


(1) The quoted paragraph is factually wrong to say:

    "The function names are those used for special class methods"

    We can fix that by changing it to:

    "Some function names are those used for special class methods".


(2) Replace the word "convenience" in the quoted paragraph by 
    "backward compatibility";


(3) Add a note close to the top of the page that the non-dunder 
    names are preferred for new code.


(4) And a note stating that existing dunder functions will 
    remain, but no new ones will be added.


The existing dunder names will remain aliases to the non-dunder names; 
they will not be deprecated (maybe in Python 5000 *wink*). Those who 
really want to use them can continue to do so.

Regarding point (1) above:

- Not all operator functions have a dunder alias.

- The dunder functions don't always correspond to a dunder method. For 
example, there is operator.__concat__ for sequence concatenation, but no 
str.__concat__ or list.__concat__ methods.

- Even when the dunder function corresponds by name to the dunder 
method, they don't mean the same thing. For example, operator.__add__ is 
*not* the same as just calling the first argument's __add__ method.

- And finally, I fail to see how having to type an extra four characters 
is a "convenience".


Long ago, when the operator module was first introduced, there was a 
much stronger correspondence between the operator.__dunder__ functions 
and dunder methods. But I think that correspondence is now so weak that 
we should simply treat it as a historical artifact.



-- 
Steve

From gvanrossum at gmail.com  Thu Apr 13 14:35:01 2017
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu, 13 Apr 2017 11:35:01 -0700
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <20170413182007.GG9464@ando.pearwood.info>
References: <20170413182007.GG9464@ando.pearwood.info>
Message-ID: <CAP7+vJL2Hcz3Gmtf9=xy2CX98cqzjOf+CF8wNNww6d2hnEjaeQ@mail.gmail.com>

+1

On Apr 13, 2017 11:26 AM, "Steven D'Aprano" <steve at pearwood.info> wrote:

> Notice that I said *discourage* rather than *deprecate*.
>
> Quoting the documentation:
>
>     The operator module exports a set of efficient functions
>     corresponding to the intrinsic operators of Python. For
>     example, operator.add(x, y) is equivalent to the expression
>     x+y. The function names are those used for special class
>     methods; variants without leading and trailing __ are also
>     provided for convenience.
>
> https://docs.python.org/3/library/operator.html
>
> I propose four simple documentation changes, and no code change:
>
>
> (1) The quoted paragraph is factually wrong to say:
>
>     "The function names are those used for special class methods"
>
>     We can fix that by changing it to:
>
>     "Some function names are those used for special class methods".
>
>
> (2) Replace the word "convenience" in the quoted paragraph by
>     "backward compatibility";
>
>
> (3) Add a note close to the top of the page that the non-dunder
>     names are preferred for new code.
>
>
> (4) And a note stating that existing dunder functions will
>     remain, but no new ones will be added.
>
>
> The existing dunder names will remain aliases to the non-dunder names;
> they will not be deprecated (maybe in Python 5000 *wink*). Those who
> really want to use them can continue to do so.
>
> Regarding point (1) above:
>
> - Not all operator functions have a dunder alias.
>
> - The dunder functions don't always correspond to a dunder method. For
> example, there is operator.__concat__ for sequence concatenation, but no
> str.__concat__ or list.__concat__ methods.
>
> - Even when the dunder function corresponds by name to the dunder
> method, they don't mean the same thing. For example, operator.__add__ is
> *not* the same as just calling the first argument's __add__ method.
>
> - And finally, I fail to see how having to type an extra four characters
> is a "convenience".
>
>
> Long ago, when the operator module was first introduced, there was a
> much stronger correspondence between the operator.__dunder__ functions
> and dunder methods. But I think that correspondence is now so weak that
> we should simply treat it as a historical artifact.
>
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170413/3541b5f9/attachment.html>

From tjreedy at udel.edu  Thu Apr 13 14:51:59 2017
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 13 Apr 2017 14:51:59 -0400
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <20170413182007.GG9464@ando.pearwood.info>
References: <20170413182007.GG9464@ando.pearwood.info>
Message-ID: <ocohc9$ja4$1@blaine.gmane.org>

On 4/13/2017 2:20 PM, Steven D'Aprano wrote:
> Notice that I said *discourage* rather than *deprecate*.
>
> Quoting the documentation:
>
>     The operator module exports a set of efficient functions
>     corresponding to the intrinsic operators of Python. For
>     example, operator.add(x, y) is equivalent to the expression
>     x+y. The function names are those used for special class
>     methods; variants without leading and trailing __ are also
>     provided for convenience.
>
> https://docs.python.org/3/library/operator.html
>
> I propose four simple documentation changes, and no code change:

I agree with revisiting this part of the doc, *and* with promoting
the dunderless versions as the primary and currently preferred version.

> (1) The quoted paragraph is factually wrong to say:
>
>     "The function names are those used for special class methods"
>
>     We can fix that by changing it to:
>
>     "Some function names are those used for special class methods".

They are instance methods defined on classes rather than 'class 
methods'.  How about

"Many function names are those used for special methods, minus the 
double underscores."

>
> (2) Replace the word "convenience" in the quoted paragraph by
>     "backward compatibility";

Then we would have "variants without leading and trailing __ are also
provided for backward compatibility", which is wrong.  I presume that 
you mean "For backward compatibility, many of these have variants with 
the double underscores kept.

>
> (3) Add a note close to the top of the page that the non-dunder
>     names are preferred for new code.

Literal 'notes' in the doc stand out too much for this.

My combined suggestion: replace the current

"The function names are those used for special class methods; variants 
without leading and trailing __ are also provided for convenience."

with

""Many function names are those used for special methods, minus the 
double underscores.  For backward compatibility, many of these have a 
variant with the double underscores kept.  We recommend using the 
dunderless form.  Note that operator.__add__(x, y), for instance, being 
the same as x + y, is not the same as x.__add__(y)."

Possibly add ", since the first two may result in calling y.__radd__(x)".

> (4) And a note stating that existing dunder functions will
>     remain, but no new ones will be added.

I think that this is implied in 'For backward compatibility'.  In any 
case, I don't think this belongs in the doc itself, but rather might be 
a comment in the module directed at future maintainers.

> The existing dunder names will remain aliases to the non-dunder names;
> they will not be deprecated (maybe in Python 5000 *wink*). Those who
> really want to use them can continue to do so.

Again, this is a future policy comment.

> Regarding point (1) above:
>
> - Not all operator functions have a dunder alias.
>
> - The dunder functions don't always correspond to a dunder method. For
> example, there is operator.__concat__ for sequence concatenation, but no
> str.__concat__ or list.__concat__ methods.
>
> - Even when the dunder function corresponds by name to the dunder
> method, they don't mean the same thing. For example, operator.__add__ is
> *not* the same as just calling the first argument's __add__ method.
>
> - And finally, I fail to see how having to type an extra four characters
> is a "convenience".

It is a burden and, as you noted, a bit misleading.

> Long ago, when the operator module was first introduced, there was a
> much stronger correspondence between the operator.__dunder__ functions
> and dunder methods. But I think that correspondence is now so weak that
> we should simply treat it as a historical artifact.

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Fri Apr 14 10:09:39 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Apr 2017 00:09:39 +1000
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <20170413182007.GG9464@ando.pearwood.info>
References: <20170413182007.GG9464@ando.pearwood.info>
Message-ID: <CADiSq7fWjXsM8qhCuik26cDnrEHZq40VJnrn+rzj50a5gqpb-Q@mail.gmail.com>

On 14 April 2017 at 04:20, Steven D'Aprano <steve at pearwood.info> wrote:
> Long ago, when the operator module was first introduced, there was a
> much stronger correspondence between the operator.__dunder__ functions
> and dunder methods. But I think that correspondence is now so weak that
> we should simply treat it as a historical artifact.

+1 from me, with this rationale. The specifics sounds pretty good to
me, too - happy to review a PR if you put one together :)

Cheers,
Nick.

P.S. To be completely honest, I'd forgotten the dunder names in
operator were there :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From storchaka at gmail.com  Fri Apr 14 12:47:24 2017
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 14 Apr 2017 19:47:24 +0300
Subject: [Python-ideas] Thread-safe generators
Message-ID: <ocquen$iqq$1@blaine.gmane.org>

When use a generator from different threads you can get a ValueError 
"generator already executing". Getting this exception with the single 
thread is a programming error, it in case of different threads it could 
be possible to wait until other thread finish executing the generator. 
The generator can be made thread-safe after wrapping it in a class that 
acquire a lock before calling the generator's __next__ method (for 
example see [1]). But this is not very efficient of course.

I wondering if it is worth to add support of thread-safe generators in 
the stdlib. Either by providing standard decorator (written in C for 
efficiency), or adding threading support just in the generator object. 
The latter may need increasing the size of the generator object for a 
lock and thread identifier (but may be GIL is enough), but should not 
affect performance since locking is used only when you faced with a 
generator running in other thread.

This topic already was raised on Python-Dev [2] but didn't moved too 
much. There are a number of StackOverflow questions about threads and 
generators. We have already encountered this issue in the stdlib. Once 
in regrtest with the -j option ([3], [4]), other time after 
reimplementing tempfile._RandomNameSequence as a generator [5].

[1] http://anandology.com/blog/using-iterators-and-generators/
[2] https://mail.python.org/pipermail/python-dev/2004-February/042390.html
[3] https://bugs.python.org/issue7996
[4] https://bugs.python.org/issue15320
[5] https://bugs.python.org/issue30030


From shoyer at gmail.com  Fri Apr 14 21:06:29 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 14 Apr 2017 18:06:29 -0700
Subject: [Python-ideas] Callable Enum values
Message-ID: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>

Enums are great. They allow you cleanly define a set of statically defined
options.

One way that I've found myself using enums recently is for dispatching (as
keys in a dictionary) between different interchangeable functions or
classes. My code looks something like this:

from enum import Enum

def foo(...):
    ...

def bar(...):
    ...

class Implementation(Enum):
    FOO = 1
    BAR = 2

_IMPLEMENTATIONS = {
    Implementation.FOO: foo,
    Implementation.BAR: bar,
}

def call_implementation(implementation, *args, **kwargs):
    return _IMPLEMENTATIONS[implementation](*args, **kwargs)

The first part of this blog post does a nice job of summarizing the general
use case:
http://lukasz.langa.pl/8/single-dispatch-generic-functions/

Obviously, enums are better than strings, because they're static declared
and already grouped together. But it would be nice if we could do one
better, by eliminating the dictionary, moving the dictionary values to the
enum and making the enum instances.

You might try this by writing a small subclass of Enum:

class CallableEnum(Enum):
    def __call__(self, *args, **kwargs):
        return self.value(*args, **kwargs)

class Implementation(CallableEnum):
    FOO = foo
    BAR = bar

def call_implementation(implementation, *args, **kwargs):
    return implementation(*args, **kwargs)

This looks great, with the Enum serving essentially as a typed namespace
for a group of related functions, but unfortunately it this doesn't work.
The problem is that when you assign a function to an Enum, it treats it as
a method instead of an enum value:
http://stackoverflow.com/questions/40338652/how-to-define-enum-values-that-are-functions

Instead, you need to wrap function values in a callable class, e.g.,

from functools import partial

class Implementation(CallableEnum):
    FOO = partial(foo)
    BAR = partial(bar)

This is OK, but definitely uglier and more error prone than necessary. It's
easy to forget to add a partial, which results in an accidental method
declaration.

It would be nice to have a CallableEnum class that works like Enum, but
adds the __call_ method and doesn't allow defining any new methods: all
functions assigned in the class body become Enum instances instead of
methods.

I would write this myself and throw it up on pypi, but there doesn't seem
to be any way to do this short of delving into the guts of enum.EnumMeta
<https://github.com/python/cpython/blob/0dc5c3169dcd4853612d11ed8c92b12fa210c07f/Lib/enum.py#L93>.
Given that I think that others will also find this pattern useful, rather
than forking the enum module I would like to add this into the standard
library instead.

Note: this proposal would also allow defining enum values using "def"
inside the Enum body. But I think this is actually pretty clean, aside from
general enum confusion over the fact that Implementation.FOO is Enum
instance, not the method.

class Implementation(CallableEnum):
    def FOO(...):
        ...
    def BAR(...):
        ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170414/0e5824b2/attachment.html>

From greg.ewing at canterbury.ac.nz  Fri Apr 14 18:42:35 2017
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 15 Apr 2017 10:42:35 +1200
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <ocquen$iqq$1@blaine.gmane.org>
References: <ocquen$iqq$1@blaine.gmane.org>
Message-ID: <58F1505B.8090009@canterbury.ac.nz>

Serhiy Storchaka wrote:
> but should not 
> affect performance since locking is used only when you faced with a 
> generator running in other thread.

I don't think that's true, because the first thread to use a
generator has to lock it as well. And even if there is only
one thread in existence when __next__ is called, another one
could be created before it finishes and try to use the
generator.

-- 
Greg

From storchaka at gmail.com  Sat Apr 15 00:55:02 2017
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 15 Apr 2017 07:55:02 +0300
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <58F1505B.8090009@canterbury.ac.nz>
References: <ocquen$iqq$1@blaine.gmane.org> <58F1505B.8090009@canterbury.ac.nz>
Message-ID: <ocs931$2fv$1@blaine.gmane.org>

On 15.04.17 01:42, Greg Ewing wrote:
> Serhiy Storchaka wrote:
>> but should not affect performance since locking is used only when you
>> faced with a generator running in other thread.
>
> I don't think that's true, because the first thread to use a
> generator has to lock it as well. And even if there is only
> one thread in existence when __next__ is called, another one
> could be created before it finishes and try to use the
> generator.

The first thread just sets the running flag (as in current code). Due to 
GIL this doesn't need additional synchronization. Other threads check 
this flag and sleep rather than raising an exception. After finishing 
the generator the first thread checks if there are other threads 
awaiting and awake a one of them.



From turnbull.stephen.fw at u.tsukuba.ac.jp  Sat Apr 15 04:55:11 2017
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Sat, 15 Apr 2017 17:55:11 +0900
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <ocs931$2fv$1@blaine.gmane.org>
References: <ocquen$iqq$1@blaine.gmane.org> <58F1505B.8090009@canterbury.ac.nz>
 <ocs931$2fv$1@blaine.gmane.org>
Message-ID: <22769.57327.46563.514010@turnbull.sk.tsukuba.ac.jp>

Serhiy Storchaka writes:

 > The first thread just sets the running flag (as in current code). Due to 
 > GIL this doesn't need additional synchronization.

Can we assume this lack of additional synchronization for other
implementations?  If not, do we care?


From ncoghlan at gmail.com  Sat Apr 15 05:45:16 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Apr 2017 19:45:16 +1000
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <ocquen$iqq$1@blaine.gmane.org>
References: <ocquen$iqq$1@blaine.gmane.org>
Message-ID: <CADiSq7c8tX+mfJ2qDeyhcF9T+nYLt6QmcmK-W-=EhPwpptTg-A@mail.gmail.com>

On 15 April 2017 at 02:47, Serhiy Storchaka <storchaka at gmail.com> wrote:
> When use a generator from different threads you can get a ValueError
> "generator already executing". Getting this exception with the single thread
> is a programming error, it in case of different threads it could be possible
> to wait until other thread finish executing the generator. The generator can
> be made thread-safe after wrapping it in a class that acquire a lock before
> calling the generator's __next__ method (for example see [1]). But this is
> not very efficient of course.
>
> I wondering if it is worth to add support of thread-safe generators in the
> stdlib. Either by providing standard decorator (written in C for
> efficiency), or adding threading support just in the generator object. The
> latter may need increasing the size of the generator object for a lock and
> thread identifier (but may be GIL is enough), but should not affect
> performance since locking is used only when you faced with a generator
> running in other thread.

Allowing multiple worker threads to pull from the same work queue is a
general concurrency problem, and that's why we have queue.Queue in the
standard library: https://docs.python.org/3/library/queue.html

So I'd be opposed to trying to make generator objects natively thread
aware - as Stephen notes, the GIL is an implementation detail of
CPython, so it isn't OK to rely on it when defining changes to
language level semantics (in this case, whether or not it's OK to have
multiple threads all calling the same generator without some form of
external locking).

However, it may make sense to explore possible options for offering a
queue.AutoQueue type, where the queue always has a defined maximum
size (defaulting to 1), disallows explicit calls to put(), and
automatically populates itself based on an iterator supplied to the
constructors. Once the input iterator raises StopIteration, then the
queue will start reporting itself as being empty.

The benefit of going down that path is that it can be used with
arbitrary iterators (not just generators), and can be more easily
generalised to other synchronisation models (such as multiprocessing).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From storchaka at gmail.com  Sat Apr 15 06:26:04 2017
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 15 Apr 2017 13:26:04 +0300
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <22769.57327.46563.514010@turnbull.sk.tsukuba.ac.jp>
References: <ocquen$iqq$1@blaine.gmane.org>
 <58F1505B.8090009@canterbury.ac.nz> <ocs931$2fv$1@blaine.gmane.org>
 <22769.57327.46563.514010@turnbull.sk.tsukuba.ac.jp>
Message-ID: <ocssfn$su1$1@blaine.gmane.org>

On 15.04.17 11:55, Stephen J. Turnbull wrote:
> Serhiy Storchaka writes:
>
>  > The first thread just sets the running flag (as in current code). Due to
>  > GIL this doesn't need additional synchronization.
>
> Can we assume this lack of additional synchronization for other
> implementations?  If not, do we care?

Other implementations should have atomic test-and-set operations for the 
running flag. Or other ways to prevent a race condition. So yes, we can 
assume this.



From gvanrossum at gmail.com  Sat Apr 15 11:05:28 2017
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sat, 15 Apr 2017 08:05:28 -0700
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <ocssfn$su1$1@blaine.gmane.org>
References: <ocquen$iqq$1@blaine.gmane.org> <58F1505B.8090009@canterbury.ac.nz>
 <ocs931$2fv$1@blaine.gmane.org>
 <22769.57327.46563.514010@turnbull.sk.tsukuba.ac.jp>
 <ocssfn$su1$1@blaine.gmane.org>
Message-ID: <CAP7+vJ+Bm_GB7FQxz=YxNipF6cE+WYyGnk9f6vRnOYxNEr5BDw@mail.gmail.com>

My 2 cent's worth, don't even think about it.

On Apr 15, 2017 3:27 AM, "Serhiy Storchaka" <storchaka at gmail.com> wrote:

> On 15.04.17 11:55, Stephen J. Turnbull wrote:
>
>> Serhiy Storchaka writes:
>>
>>  > The first thread just sets the running flag (as in current code). Due
>> to
>>  > GIL this doesn't need additional synchronization.
>>
>> Can we assume this lack of additional synchronization for other
>> implementations?  If not, do we care?
>>
>
> Other implementations should have atomic test-and-set operations for the
> running flag. Or other ways to prevent a race condition. So yes, we can
> assume this.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170415/a3ee7981/attachment.html>

From steve at pearwood.info  Sun Apr 16 04:24:33 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 16 Apr 2017 18:24:33 +1000
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
Message-ID: <20170416082431.GL9464@ando.pearwood.info>

On Fri, Apr 14, 2017 at 06:06:29PM -0700, Stephan Hoyer wrote:

> One way that I've found myself using enums recently is for dispatching (as
> keys in a dictionary) between different interchangeable functions or
> classes. My code looks something like this:
> 
> from enum import Enum
> 
> def foo(...):
>     ...
> 
> def bar(...):
>     ...
> 
> class Implementation(Enum):
>     FOO = 1
>     BAR = 2
> 
> _IMPLEMENTATIONS = {
>     Implementation.FOO: foo,
>     Implementation.BAR: bar,
> }
> 
> def call_implementation(implementation, *args, **kwargs):
>     return _IMPLEMENTATIONS[implementation](*args, **kwargs)


To me, this looks like a case where you want to give each instance a 
custom per-instance method. (Surely this must be a named Design 
Pattern?)

There's not really good syntax for that, although if the method can be 
written with lambda you can at least avoid a global function:


from types import MethodType

class Colour(Enum):
    RED = 1
    BLUE = 2

Colour.RED.paint = MethodType(
        lambda self: print("Painting the town", self.name),
        Colour.RED)
Colour.BLUE.paint = MethodType(
        lambda self: print("I'm feeling", self.name),
        Colour.BLUE)



but of course one can write a helper function or decorator to make it a 
little less ugly. See below.

This doesn't work for dunder methods, not directly. You can't say:

Colour.RED.__call__ = ...

to make the RED instance callable. But you can do this:

class Colour(Enum):
    RED = 1
    BLUE = 2
    def __call__(self):
        return self._call()


Colour.RED._call = MethodType( ... )


and now Colours.RED() will call your per-instance method.

We can use a decorator as a helper:

# untested
def add_per_instance_method(instance):
    def decorator(function):
        instance._call = MethodType(function, instance)
        return function
    return decorator


and now this should work:


@add_per_instance_method(Colour.RED)
def _call(self):
    # implementation goes here
    ...

@add_per_instance_method(Colour.BLUE)
def _call(self):
    ...


The only thing left to do is clean up at the end and remove the left 
over namespace pollution:

del _call


if you can be bothered. And now you have callable Enums with a 
per-instance method each, as well as a pretty enumeration value for 
debugging. Much nicer than <function foo at 0xb7b02cd4>.

Does this solve your problem? If not, what's missing?



[...]
> Obviously, enums are better than strings, because they're static declared
> and already grouped together. But it would be nice if we could do one
> better, by eliminating the dictionary, moving the dictionary values to the
> enum and making the enum instances.

I think you missed a word. Making the enum instances... what? Callable?


[...]
> The problem is that when you assign a function to an Enum, it treats it as
> a method instead of an enum value:
> http://stackoverflow.com/questions/40338652/how-to-define-enum-values-that-are-functions

That's a minor, and not very important, limitation. I consider that 
equivalent to the restriction that functions defined inside a class body 
are automatically converted to instance methods. If you want to avoid 
that, you need to decorate them as a class method or static method or 
similar.


> Instead, you need to wrap function values in a callable class, e.g.,
> 
> from functools import partial
> 
> class Implementation(CallableEnum):
>     FOO = partial(foo)
>     BAR = partial(bar)
> 
> This is OK, but definitely uglier and more error prone than necessary. It's
> easy to forget to add a partial, which results in an accidental method
> declaration.

Python doesn't have to protect the programmer from every possible source 
of human error. I don't think it is Python's responsibility to protect 
people from accidentally doing something like this:

class Colours(Enum):
    RED = partial(red)
    BLUE = partial(blue)
    GREEN = partial(green)
    YELLOW = yellow  # oops

Sometimes the answer is Then Don't Do That.


> It would be nice to have a CallableEnum class that works like Enum, but
> adds the __call_ method and doesn't allow defining any new methods: all
> functions assigned in the class body become Enum instances instead of
> methods.

I wouldn't be happy with the restriction "all methods are Enum 
instances". Seems over-eager. It might be suitable for *your* specific 
use-case, but I expect that other users of Enum will want to have both 
methods and functions as Enum values:

class Thing(Enum):
    WIDGET = 'x'
    DOODAD = 5
    GIZMO = function  # how to protect this?
    THINGAMAJIG = 'something'

    def method(self, arg):
        ...


Given that wanting to use a function as the enumeration value is quite 
unusual in the first place, I don't think this belongs in the standard 
library.


-- 
Steve

From victor.stinner at gmail.com  Sun Apr 16 10:57:46 2017
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sun, 16 Apr 2017 16:57:46 +0200
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <ocquen$iqq$1@blaine.gmane.org>
References: <ocquen$iqq$1@blaine.gmane.org>
Message-ID: <CAMpsgwbOWFNETGPWk+qbF=Di0opzPsz2R=3e05EBGCx6q13Kag@mail.gmail.com>

Thread safety is very complex and has an impact on performance. I dislike
the idea of providing such property to generators which can have a complex
next method.

IMHO it's better to put a generator in wrapper which adds thread safety.

What do you think?

Victor

Le 14 avr. 2017 18:48, "Serhiy Storchaka" <storchaka at gmail.com> a ?crit :

> When use a generator from different threads you can get a ValueError
> "generator already executing". Getting this exception with the single
> thread is a programming error, it in case of different threads it could be
> possible to wait until other thread finish executing the generator. The
> generator can be made thread-safe after wrapping it in a class that acquire
> a lock before calling the generator's __next__ method (for example see
> [1]). But this is not very efficient of course.
>
> I wondering if it is worth to add support of thread-safe generators in the
> stdlib. Either by providing standard decorator (written in C for
> efficiency), or adding threading support just in the generator object. The
> latter may need increasing the size of the generator object for a lock and
> thread identifier (but may be GIL is enough), but should not affect
> performance since locking is used only when you faced with a generator
> running in other thread.
>
> This topic already was raised on Python-Dev [2] but didn't moved too much.
> There are a number of StackOverflow questions about threads and generators.
> We have already encountered this issue in the stdlib. Once in regrtest with
> the -j option ([3], [4]), other time after reimplementing
> tempfile._RandomNameSequence as a generator [5].
>
> [1] http://anandology.com/blog/using-iterators-and-generators/
> [2] https://mail.python.org/pipermail/python-dev/2004-February/042390.html
> [3] https://bugs.python.org/issue7996
> [4] https://bugs.python.org/issue15320
> [5] https://bugs.python.org/issue30030
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170416/c5298ebc/attachment.html>

From ethan at stoneleaf.us  Sun Apr 16 14:14:34 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 16 Apr 2017 11:14:34 -0700
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <20170416082431.GL9464@ando.pearwood.info>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
 <20170416082431.GL9464@ando.pearwood.info>
Message-ID: <58F3B48A.2090305@stoneleaf.us>

On 04/16/2017 01:24 AM, Steven D'Aprano wrote:
> On Fri, Apr 14, 2017 at 06:06:29PM -0700, Stephan Hoyer wrote:

>> One way that I've found myself using enums recently is for dispatching (as
>> keys in a dictionary) between different interchangeable functions or
>> classes.

[...]

> Given that wanting to use a function as the enumeration value is quite
> unusual in the first place, I don't think this belongs in the standard
> library.

I agree with D'Aprano: such unusual usage does not belong in the stdlib.

Fortunately, there is the Advanced Enumeration (aenum) library*:

--- 8< -----------------------------------------------------------------
from aenum import Enum, enum

class CallableEnum(Enum):

     def __new__(cls, *args, **kwds):
         member = object.__new__(cls)
         member._impl = args[0]
         if member._impl.__doc__ is not None:
             member._value_ = member._impl.__doc__
         else:
             member._value_ = repr(member._impl)
         return member

     def __call__(self, *args, **kwds):
         return self._impl(*args, **kwds)
--- 8< -----------------------------------------------------------------

and in use

--- 8< -----------------------------------------------------------------
class TestEnum(CallableEnum):

     @enum
     def hello(text):
         "a pleasant greeting"
         print('hello,', text)

     @enum
     def goodbye(text):
         print('goodbye,', text)

list(TestEnum)
# [
#  <TestEnum.hello: 'a pleasant greeting'>,
#  <TestEnum.goodbye: '<function goodbye at 0xb7264844>'>,
# ]

print(TestEnum.hello)
# TestEnum.hello

TestEnum.hello('how are you?')
# 'hello, how are you?'

TestEnum.goodbye('see you soon!')
# 'goodbye, see you soon!'
--- 8< -----------------------------------------------------------------

Note that it is possible to do the same thing using the stdlib Enum if 
you create your own decorator (it must return a class instance) and a 
slight rewrite of __new__ -- but I'll leave that as an exercise for the 
reader.

--
~Ethan~


* Disclosure:  I am the primary author of the stdlib Enum; I am also the 
author if the enum34 backport and the aenum library.

enum34: https://pypi.python.org/pypi/enum34
  aenum: https://pypi.python.org/pypi/aenum

From gvanrossum at gmail.com  Sun Apr 16 15:55:18 2017
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sun, 16 Apr 2017 12:55:18 -0700
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <CAMpsgwbOWFNETGPWk+qbF=Di0opzPsz2R=3e05EBGCx6q13Kag@mail.gmail.com>
References: <ocquen$iqq$1@blaine.gmane.org>
 <CAMpsgwbOWFNETGPWk+qbF=Di0opzPsz2R=3e05EBGCx6q13Kag@mail.gmail.com>
Message-ID: <CAP7+vJJTgon3d=f7ERNBmEW=DJD3XSEncd-HcQkBJJNoD8DCZQ@mail.gmail.com>

I think the two shouldn't be mixed.

On Apr 16, 2017 7:58 AM, "Victor Stinner" <victor.stinner at gmail.com> wrote:

> Thread safety is very complex and has an impact on performance. I dislike
> the idea of providing such property to generators which can have a complex
> next method.
>
> IMHO it's better to put a generator in wrapper which adds thread safety.
>
> What do you think?
>
> Victor
>
> Le 14 avr. 2017 18:48, "Serhiy Storchaka" <storchaka at gmail.com> a ?crit :
>
>> When use a generator from different threads you can get a ValueError
>> "generator already executing". Getting this exception with the single
>> thread is a programming error, it in case of different threads it could be
>> possible to wait until other thread finish executing the generator. The
>> generator can be made thread-safe after wrapping it in a class that acquire
>> a lock before calling the generator's __next__ method (for example see
>> [1]). But this is not very efficient of course.
>>
>> I wondering if it is worth to add support of thread-safe generators in
>> the stdlib. Either by providing standard decorator (written in C for
>> efficiency), or adding threading support just in the generator object. The
>> latter may need increasing the size of the generator object for a lock and
>> thread identifier (but may be GIL is enough), but should not affect
>> performance since locking is used only when you faced with a generator
>> running in other thread.
>>
>> This topic already was raised on Python-Dev [2] but didn't moved too
>> much. There are a number of StackOverflow questions about threads and
>> generators. We have already encountered this issue in the stdlib. Once in
>> regrtest with the -j option ([3], [4]), other time after reimplementing
>> tempfile._RandomNameSequence as a generator [5].
>>
>> [1] http://anandology.com/blog/using-iterators-and-generators/
>> [2] https://mail.python.org/pipermail/python-dev/2004-February/0
>> 42390.html
>> [3] https://bugs.python.org/issue7996
>> [4] https://bugs.python.org/issue15320
>> [5] https://bugs.python.org/issue30030
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170416/d58063a4/attachment.html>

From p.f.moore at gmail.com  Sun Apr 16 18:00:07 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 16 Apr 2017 23:00:07 +0100
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <CADiSq7c8tX+mfJ2qDeyhcF9T+nYLt6QmcmK-W-=EhPwpptTg-A@mail.gmail.com>
References: <ocquen$iqq$1@blaine.gmane.org>
 <CADiSq7c8tX+mfJ2qDeyhcF9T+nYLt6QmcmK-W-=EhPwpptTg-A@mail.gmail.com>
Message-ID: <CACac1F8M3yfsbOFRKzRx2+HpoCnw--zTE7TBD-g895wmSktUKA@mail.gmail.com>

On 15 April 2017 at 10:45, Nick Coghlan <ncoghlan at gmail.com> wrote:
> So I'd be opposed to trying to make generator objects natively thread
> aware - as Stephen notes, the GIL is an implementation detail of
> CPython, so it isn't OK to rely on it when defining changes to
> language level semantics (in this case, whether or not it's OK to have
> multiple threads all calling the same generator without some form of
> external locking).
>
> However, it may make sense to explore possible options for offering a
> queue.AutoQueue type, where the queue always has a defined maximum
> size (defaulting to 1), disallows explicit calls to put(), and
> automatically populates itself based on an iterator supplied to the
> constructors. Once the input iterator raises StopIteration, then the
> queue will start reporting itself as being empty.

+1 A generator that can have values pulled from it on different
threads sounds like a queue to me, so the AutoQueue class that wraps a
generator seems like a natural abstraction to work with. It also means
that the cost for thread safety is only paid by those applications
that need it.

Paul

From steve at pearwood.info  Sun Apr 16 22:34:31 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 17 Apr 2017 12:34:31 +1000
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <CADiSq7fWjXsM8qhCuik26cDnrEHZq40VJnrn+rzj50a5gqpb-Q@mail.gmail.com>
References: <20170413182007.GG9464@ando.pearwood.info>
 <CADiSq7fWjXsM8qhCuik26cDnrEHZq40VJnrn+rzj50a5gqpb-Q@mail.gmail.com>
Message-ID: <20170417023430.GM9464@ando.pearwood.info>

On Sat, Apr 15, 2017 at 12:09:39AM +1000, Nick Coghlan wrote:
> On 14 April 2017 at 04:20, Steven D'Aprano <steve at pearwood.info> wrote:
> > Long ago, when the operator module was first introduced, there was a
> > much stronger correspondence between the operator.__dunder__ functions
> > and dunder methods. But I think that correspondence is now so weak that
> > we should simply treat it as a historical artifact.
> 
> +1 from me, with this rationale. The specifics sounds pretty good to
> me, too - happy to review a PR if you put one together :)

http://bugs.python.org/issue30085

For some reason, Github won't allow me to log in at the moment. It keeps 
insisting I need to enable cookies, even though I have. Perhaps it 
doesn't like my browser? I'll have to try on another computer with a 
different browser.

Terry, you had some nice suggestions for wording. If you want to submit 
a PR, feel free to take over the issue. I probably won't get a chance to 
do any more on this for a week.



-- 
Steve

From ncoghlan at gmail.com  Mon Apr 17 01:08:22 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 17 Apr 2017 15:08:22 +1000
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <CACac1F8M3yfsbOFRKzRx2+HpoCnw--zTE7TBD-g895wmSktUKA@mail.gmail.com>
References: <ocquen$iqq$1@blaine.gmane.org>
 <CADiSq7c8tX+mfJ2qDeyhcF9T+nYLt6QmcmK-W-=EhPwpptTg-A@mail.gmail.com>
 <CACac1F8M3yfsbOFRKzRx2+HpoCnw--zTE7TBD-g895wmSktUKA@mail.gmail.com>
Message-ID: <CADiSq7c6HDeT-ZFgqa82_OsYaZ-psWm6tOs2Kt8kPVRDJ67Gkg@mail.gmail.com>

On 17 April 2017 at 08:00, Paul Moore <p.f.moore at gmail.com> wrote:
> On 15 April 2017 at 10:45, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> So I'd be opposed to trying to make generator objects natively thread
>> aware - as Stephen notes, the GIL is an implementation detail of
>> CPython, so it isn't OK to rely on it when defining changes to
>> language level semantics (in this case, whether or not it's OK to have
>> multiple threads all calling the same generator without some form of
>> external locking).
>>
>> However, it may make sense to explore possible options for offering a
>> queue.AutoQueue type, where the queue always has a defined maximum
>> size (defaulting to 1), disallows explicit calls to put(), and
>> automatically populates itself based on an iterator supplied to the
>> constructors. Once the input iterator raises StopIteration, then the
>> queue will start reporting itself as being empty.
>
> +1 A generator that can have values pulled from it on different
> threads sounds like a queue to me, so the AutoQueue class that wraps a
> generator seems like a natural abstraction to work with. It also means
> that the cost for thread safety is only paid by those applications
> that need it.

If someone did build something like this, it would be interesting to
benchmark it against a more traditional producer thread model, where
one thread is responsible for adding work items to the queue, while
others are responsible for draining them.

The trick is that an auto-queue would borrow execution time from the
consumer threads when new values are needed, so you'd theoretically
get fewer context switches between threads, but at the cost of
changing the nature of the workload in a given thread, and hence
messing with the working set of objects it has active.

It may also pair well with the concurrent.futures.Executor model,
which is already good for "go handle this predefined list of tasks",
but currently less useful as a replacement for a message queue with a
pool of workers.

Setting the latter up yourself is currently still a bit tedious, since:

1. we don't have a standard threading Pool abstraction in the standard
library, just the one tucked away as part of multiprocessing
2. while queue.Queue has native support for worker pools, we don't
provide a pre-assembled version that makes it easy to say "here is the
producer, here are the consumers, wire them together for me"

There are good reasons for that (mainly that it's hard to come up with
an abstraction that's useful in its own right without becoming so
complex that you're on the verge of reinventing a task manager like
celery or a distributed computation manager like dask), but at the
same time, the notion of "input queue, worker pool, output queue" is
one that comes up a *lot* across different concurrency models, so
there's potential value in providing a low-barrier-to-entry
introduction to that idiom as part of the standard library.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jim.baker at python.org  Mon Apr 17 13:20:46 2017
From: jim.baker at python.org (Jim Baker)
Date: Mon, 17 Apr 2017 11:20:46 -0600
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <CADiSq7c6HDeT-ZFgqa82_OsYaZ-psWm6tOs2Kt8kPVRDJ67Gkg@mail.gmail.com>
References: <ocquen$iqq$1@blaine.gmane.org>
 <CADiSq7c8tX+mfJ2qDeyhcF9T+nYLt6QmcmK-W-=EhPwpptTg-A@mail.gmail.com>
 <CACac1F8M3yfsbOFRKzRx2+HpoCnw--zTE7TBD-g895wmSktUKA@mail.gmail.com>
 <CADiSq7c6HDeT-ZFgqa82_OsYaZ-psWm6tOs2Kt8kPVRDJ67Gkg@mail.gmail.com>
Message-ID: <CAOhO=aMQGxsjuwvEG1SfW9mdH+QzJryiQgHpe_B_Qi9N_DJy-g@mail.gmail.com>

This is a bad idea in the generator itself, as commented earlier by others
here.

>From a cross implementation perspective, in Jython, different threads can
call next on a non running generator, *so long as they coordinate with each
other external to any use of this generator*, and this works fine.

But any reliance on gi_running, as seen here, can only be considered to be
possible help in detecting such races; it would not even come close to
preventing a race:
https://github.com/jythontools/jython/blob/master/src/org/python/core/PyGenerator.java#L146

(We don't even bother making gi_running a volatile to get actual
test-and-set style semantics, because really it makes no sense to pretend
otherwise; and why pay the performance penalty?)

The idea of putting generators behind a queue sounds reasonably workable -
the semantics then are the right ones, although implementing this
efficiently is the trick here.


On Sun, Apr 16, 2017 at 11:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 17 April 2017 at 08:00, Paul Moore <p.f.moore at gmail.com> wrote:
> > On 15 April 2017 at 10:45, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> So I'd be opposed to trying to make generator objects natively thread
> >> aware - as Stephen notes, the GIL is an implementation detail of
> >> CPython, so it isn't OK to rely on it when defining changes to
> >> language level semantics (in this case, whether or not it's OK to have
> >> multiple threads all calling the same generator without some form of
> >> external locking).
> >>
> >> However, it may make sense to explore possible options for offering a
> >> queue.AutoQueue type, where the queue always has a defined maximum
> >> size (defaulting to 1), disallows explicit calls to put(), and
> >> automatically populates itself based on an iterator supplied to the
> >> constructors. Once the input iterator raises StopIteration, then the
> >> queue will start reporting itself as being empty.
> >
> > +1 A generator that can have values pulled from it on different
> > threads sounds like a queue to me, so the AutoQueue class that wraps a
> > generator seems like a natural abstraction to work with. It also means
> > that the cost for thread safety is only paid by those applications
> > that need it.
>
> If someone did build something like this, it would be interesting to
> benchmark it against a more traditional producer thread model, where
> one thread is responsible for adding work items to the queue, while
> others are responsible for draining them.
>
> The trick is that an auto-queue would borrow execution time from the
> consumer threads when new values are needed, so you'd theoretically
> get fewer context switches between threads, but at the cost of
> changing the nature of the workload in a given thread, and hence
> messing with the working set of objects it has active.
>
> It may also pair well with the concurrent.futures.Executor model,
> which is already good for "go handle this predefined list of tasks",
> but currently less useful as a replacement for a message queue with a
> pool of workers.
>
> Setting the latter up yourself is currently still a bit tedious, since:
>
> 1. we don't have a standard threading Pool abstraction in the standard
> library, just the one tucked away as part of multiprocessing
> 2. while queue.Queue has native support for worker pools, we don't
> provide a pre-assembled version that makes it easy to say "here is the
> producer, here are the consumers, wire them together for me"
>
> There are good reasons for that (mainly that it's hard to come up with
> an abstraction that's useful in its own right without becoming so
> complex that you're on the verge of reinventing a task manager like
> celery or a distributed computation manager like dask), but at the
> same time, the notion of "input queue, worker pool, output queue" is
> one that comes up a *lot* across different concurrency models, so
> there's potential value in providing a low-barrier-to-entry
> introduction to that idiom as part of the standard library.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170417/8c7f107f/attachment.html>

From greg.ewing at canterbury.ac.nz  Mon Apr 17 18:41:57 2017
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 18 Apr 2017 10:41:57 +1200
Subject: [Python-ideas] Thread-safe generators
In-Reply-To: <CADiSq7c6HDeT-ZFgqa82_OsYaZ-psWm6tOs2Kt8kPVRDJ67Gkg@mail.gmail.com>
References: <ocquen$iqq$1@blaine.gmane.org>
 <CADiSq7c8tX+mfJ2qDeyhcF9T+nYLt6QmcmK-W-=EhPwpptTg-A@mail.gmail.com>
 <CACac1F8M3yfsbOFRKzRx2+HpoCnw--zTE7TBD-g895wmSktUKA@mail.gmail.com>
 <CADiSq7c6HDeT-ZFgqa82_OsYaZ-psWm6tOs2Kt8kPVRDJ67Gkg@mail.gmail.com>
Message-ID: <58F544B5.80404@canterbury.ac.nz>

Nick Coghlan wrote:
> but at the cost of
> changing the nature of the workload in a given thread, and hence
> messing with the working set of objects it has active.

Does the working set of an individual thread matter to anything?
Or even an individual process? As far as the virtual memory
system is concerned, what matters is the working set of all
active code sharing the RAM.

-- 
Greg



From sanketdasgupta at gmail.com  Tue Apr 18 15:51:51 2017
From: sanketdasgupta at gmail.com (Sanket Dasgupta)
Date: Tue, 18 Apr 2017 19:51:51 +0000
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <20170417023430.GM9464@ando.pearwood.info>
References: <20170413182007.GG9464@ando.pearwood.info>
 <CADiSq7fWjXsM8qhCuik26cDnrEHZq40VJnrn+rzj50a5gqpb-Q@mail.gmail.com>
 <20170417023430.GM9464@ando.pearwood.info>
Message-ID: <CAFOmoUkWtTTXKfXi0oDOqAr2Xhk9xmQpAO_LE2WmPwGm4S8WPQ@mail.gmail.com>

Hi,

I have opened a pull request at https://github.com/python/cpython/pull/1171

I am not sure of the wording used, and I'd love some feedback.

Thanks!

On Mon, Apr 17, 2017 at 8:10 AM Steven D'Aprano <steve at pearwood.info> wrote:

> On Sat, Apr 15, 2017 at 12:09:39AM +1000, Nick Coghlan wrote:
> > On 14 April 2017 at 04:20, Steven D'Aprano <steve at pearwood.info> wrote:
> > > Long ago, when the operator module was first introduced, there was a
> > > much stronger correspondence between the operator.__dunder__ functions
> > > and dunder methods. But I think that correspondence is now so weak that
> > > we should simply treat it as a historical artifact.
> >
> > +1 from me, with this rationale. The specifics sounds pretty good to
> > me, too - happy to review a PR if you put one together :)
>
> http://bugs.python.org/issue30085
>
> For some reason, Github won't allow me to log in at the moment. It keeps
> insisting I need to enable cookies, even though I have. Perhaps it
> doesn't like my browser? I'll have to try on another computer with a
> different browser.
>
> Terry, you had some nice suggestions for wording. If you want to submit
> a PR, feel free to take over the issue. I probably won't get a chance to
> do any more on this for a week.
>
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170418/8d8f93d8/attachment.html>

From toddrjen at gmail.com  Wed Apr 19 08:54:07 2017
From: toddrjen at gmail.com (Todd)
Date: Wed, 19 Apr 2017 08:54:07 -0400
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <20170413182007.GG9464@ando.pearwood.info>
References: <20170413182007.GG9464@ando.pearwood.info>
Message-ID: <CAFpSVpKCy8rxevdY8NikbfQgjgRnWsi7MLogjQeVxkmZpsQ72w@mail.gmail.com>

On Apr 13, 2017 14:25, "Steven D'Aprano" <steve at pearwood.info> wrote:

Notice that I said *discourage* rather than *deprecate*.

Quoting the documentation:

    The operator module exports a set of efficient functions
    corresponding to the intrinsic operators of Python. For
    example, operator.add(x, y) is equivalent to the expression
    x+y. The function names are those used for special class
    methods; variants without leading and trailing __ are also
    provided for convenience.

https://docs.python.org/3/library/operator.html

I propose four simple documentation changes, and no code change:


(1) The quoted paragraph is factually wrong to say:

    "The function names are those used for special class methods"

    We can fix that by changing it to:

    "Some function names are those used for special class methods".


(2) Replace the word "convenience" in the quoted paragraph by
    "backward compatibility";


(3) Add a note close to the top of the page that the non-dunder
    names are preferred for new code.


(4) And a note stating that existing dunder functions will
    remain, but no new ones will be added.


The existing dunder names will remain aliases to the non-dunder names;
they will not be deprecated (maybe in Python 5000 *wink*). Those who
really want to use them can continue to do so.

Regarding point (1) above:

- Not all operator functions have a dunder alias.

- The dunder functions don't always correspond to a dunder method. For
example, there is operator.__concat__ for sequence concatenation, but no
str.__concat__ or list.__concat__ methods.

- Even when the dunder function corresponds by name to the dunder
method, they don't mean the same thing. For example, operator.__add__ is
*not* the same as just calling the first argument's __add__ method.

- And finally, I fail to see how having to type an extra four characters
is a "convenience".


Long ago, when the operator module was first introduced, there was a
much stronger correspondence between the operator.__dunder__ functions
and dunder methods. But I think that correspondence is now so weak that
we should simply treat it as a historical artifact.


What about going a step further and moving the __dunder__ functions to
another section?

Perhaps it could have a statement saying that some of the functions also
come in dunder versions, but that people are encouraged to use the regular
version. Then you can just have a table with the normal version on the left
and the equivalent dunder version (if any) on the right.

This would also have the benefit of making the existing function list
simpler and clearer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170419/b4a548ba/attachment.html>

From ethan at stoneleaf.us  Thu Apr 20 13:58:50 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 20 Apr 2017 10:58:50 -0700
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <CAEQ_Tvfe1NhMTJMOs9=Yt+7APWG=pqV7nb9kbbFd--q+OWpT5g@mail.gmail.com>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
 <20170416082431.GL9464@ando.pearwood.info> <58F3B48A.2090305@stoneleaf.us>
 <CAEQ_Tvfe1NhMTJMOs9=Yt+7APWG=pqV7nb9kbbFd--q+OWpT5g@mail.gmail.com>
Message-ID: <58F8F6DA.3060109@stoneleaf.us>

[redirecting back to the list]

On 04/19/2017 11:15 PM, Stephan Hoyer wrote:
> Ethan and Steven,
>
> Thanks for your feedback on this one. I agree that it probably doesn't make sense for the standard library.
>
> I'm still not really happy with any of the standard approaches for choosing a function based on an enum value -- they
> all seem pretty verbose/ugly -- but clearly I'm bothered by this more than most, and the standard library is not a good
> place for novel solutions.

I'm curious, what did you find ugly with:

     class TestEnum(CallableEnum):

          @enum
          def hello(text):
              "a pleasant greeting"
              print('hello,', text)

          @enum
          def goodbye(text):
              print('goodbye,', text)

--
~Ethan~

From shoyer at gmail.com  Fri Apr 21 12:04:45 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 21 Apr 2017 09:04:45 -0700
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <58F8F6DA.3060109@stoneleaf.us>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
 <20170416082431.GL9464@ando.pearwood.info> <58F3B48A.2090305@stoneleaf.us>
 <CAEQ_Tvfe1NhMTJMOs9=Yt+7APWG=pqV7nb9kbbFd--q+OWpT5g@mail.gmail.com>
 <58F8F6DA.3060109@stoneleaf.us>
Message-ID: <CAEQ_TvfwLPuokDvBm3GB00D2CJVNiSqFORXEvn9SOW3-xt6AZg@mail.gmail.com>

On Thu, Apr 20, 2017 at 10:58 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> I'm curious, what did you find ugly with:
>
>     class TestEnum(CallableEnum):
>
>          @enum
>          def hello(text):
>              "a pleasant greeting"
>              print('hello,', text)
>
>          @enum
>          def goodbye(text):
>              print('goodbye,', text)
>

Yeah, this is actually pretty reasonable.

For my use case, both the functions and their names are pretty long, so I
wouldn't want to write them inside the enum class. With a
decorator/function wrapper, it just gets kind of long -- especially if I
only import modules as required
<https://google.github.io/styleguide/pyguide.html?showone=Imports#Imports>
by the Google style guide. So my real usage looks more like:

class Greeting(callable_enum.Enum):
    HELLO = callable_enum.Value(_greet_hello)
    GOODBYE = callable_enum.Value(_greet_goodbye)
    TO_PYTHON_IDEAS = callable_enum.Value(_welcome_to_python_ideas)

The large callable_enum.Value() wrappers make it harder to track what's
going on.

But perhaps this is really just a good use for an inline declaration, where
I don't need the wrappers at all:

Greeting = callable_enum.Enum('Greeting', {
    'HELLO': _greet_hello,
    'GOODBYE': _greet_goodbye,
    'TO_PYTHON_IDEAS': _welcome_to_python_ideas,
})
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170421/fdca3674/attachment.html>

From ethan at stoneleaf.us  Fri Apr 21 12:43:39 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 21 Apr 2017 09:43:39 -0700
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <CAEQ_TvfwLPuokDvBm3GB00D2CJVNiSqFORXEvn9SOW3-xt6AZg@mail.gmail.com>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
 <20170416082431.GL9464@ando.pearwood.info> <58F3B48A.2090305@stoneleaf.us>
 <CAEQ_Tvfe1NhMTJMOs9=Yt+7APWG=pqV7nb9kbbFd--q+OWpT5g@mail.gmail.com>
 <58F8F6DA.3060109@stoneleaf.us>
 <CAEQ_TvfwLPuokDvBm3GB00D2CJVNiSqFORXEvn9SOW3-xt6AZg@mail.gmail.com>
Message-ID: <58FA36BB.20501@stoneleaf.us>

On 04/21/2017 09:04 AM, Stephan Hoyer wrote:
> On Thu, Apr 20, 2017 at 10:58 AM, Ethan Furman wrote:

> I'm curious, what did you find ugly with:
>
>      class TestEnum(CallableEnum):
>
>           @enum
>           def hello(text):
>               "a pleasant greeting"
>               print('hello,', text)
>
>           @enum
>           def goodbye(text):
>               print('goodbye,', text)
>
> Yeah, this is actually pretty reasonable.
>
> For my use case, both the functions and their names are pretty long, so I wouldn't want to write them inside the enum
> class. With a decorator/function wrapper, it just gets kind of long -- especially if I only import modules as required
> <https://google.github.io/styleguide/pyguide.html?showone=Imports#Imports> by the Google style guide. So my real usage
> looks more like:
>
> class Greeting(callable_enum.Enum):
>      HELLO = callable_enum.Value(_greet_hello)
>      GOODBYE = callable_enum.Value(_greet_goodbye)
>      TO_PYTHON_IDEAS =callable_enum.Value(_welcome_to_python_ideas)
>
> The large callable_enum.Value() wrappers make it harder to track what's going on.

You have to use the complete module name?  Instead of `from callable_enum import Enum, Value`?  Ouch.

--
~Ethan~

From shoyer at gmail.com  Fri Apr 21 13:01:34 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 21 Apr 2017 10:01:34 -0700
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <58FA36BB.20501@stoneleaf.us>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
 <20170416082431.GL9464@ando.pearwood.info> <58F3B48A.2090305@stoneleaf.us>
 <CAEQ_Tvfe1NhMTJMOs9=Yt+7APWG=pqV7nb9kbbFd--q+OWpT5g@mail.gmail.com>
 <58F8F6DA.3060109@stoneleaf.us>
 <CAEQ_TvfwLPuokDvBm3GB00D2CJVNiSqFORXEvn9SOW3-xt6AZg@mail.gmail.com>
 <58FA36BB.20501@stoneleaf.us>
Message-ID: <CAEQ_TvezVS3q6+sQf=+WB+=3+jEqJyxk_E_Fo+O1tZ51QzBgWg@mail.gmail.com>

On Fri, Apr 21, 2017 at 9:43 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> You have to use the complete module name?  Instead of `from callable_enum
> import Enum, Value`?  Ouch


Yeah... it took me a while to come around to this one. But the rule does
start to pay off in large code bases.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170421/ea939f11/attachment.html>

From chris.barker at noaa.gov  Fri Apr 21 15:46:15 2017
From: chris.barker at noaa.gov (Chris Barker)
Date: Fri, 21 Apr 2017 12:46:15 -0700
Subject: [Python-ideas] Callable Enum values
In-Reply-To: <58FA36BB.20501@stoneleaf.us>
References: <CAEQ_TvdKQk4PW9zvZ8_QmeKyKF0w6i-W4r80ssdnVYeKXs53Sw@mail.gmail.com>
 <20170416082431.GL9464@ando.pearwood.info> <58F3B48A.2090305@stoneleaf.us>
 <CAEQ_Tvfe1NhMTJMOs9=Yt+7APWG=pqV7nb9kbbFd--q+OWpT5g@mail.gmail.com>
 <58F8F6DA.3060109@stoneleaf.us>
 <CAEQ_TvfwLPuokDvBm3GB00D2CJVNiSqFORXEvn9SOW3-xt6AZg@mail.gmail.com>
 <58FA36BB.20501@stoneleaf.us>
Message-ID: <CALGmxE+B6E_g=_kEtzVC_8QKyf9ehauwaDLPp2A-avZYCwry0g@mail.gmail.com>

OT, but...

You have to use the complete module name?  Instead of `from callable_enum
> import Enum, Value`?  Ouch.


yes, but:

"""
Use from x import y as z if two modules named y are to be imported or if y is
an inconveniently long name.
"""

(https://google.github.io/styleguide/pyguide.html?showone=Imports#Imports)

So you can rename it if it's long:

import callable_enum as ce

class Greeting(callable_enum.Enum):
     HELLO = ce.Value(_greet_hello)
     GOODBYE = ce.Value(_greet_goodbye)
     TO_PYTHON_IDEAS =ce.Value(_welcome_to_python_ideas)

not too bad, and a pattern that has been pretty universally adopted by,
e.g., numpy:

import numpy as np.

I'm still on the fence for modules that I'm only using a couple names from
though...

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170421/246261c3/attachment.html>

From shoyer at gmail.com  Sun Apr 23 21:23:12 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sun, 23 Apr 2017 18:23:12 -0700
Subject: [Python-ideas] Binary arithmetic does not always call subclasses
 first
Message-ID: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>

I recently filed this as a bug, and was asked to repost to python-dev or
python-ideas for greater visibility:
http://bugs.python.org/issue30140

Without further ado, here is my original report:

---------------------------
We are writing a system for overloading NumPy operations (see PR [1] and
design doc [2]) that is designed to copy and extend Python's system for
overloading binary operators.

The reference documentation on binary arithmetic [3] states:

> Note: If the right operand's type is a subclass of the left operand?s
type and that subclass provides the reflected method for the operation,
this method will be called before the left operand?s non-reflected method.
This behavior allows subclasses to override their ancestors? operations.

However, this isn't actually done if the right operand merely inherits from
the left operand's type. In practice, CPython requires that the right
operand defines a different method before it defers to it. Note that the
behavior is different for comparisons, which defer to subclasses regardless
of whether they implement a new method [4].

I think this behavior is a mistake and should be corrected. It is just as
useful to write generic binary arithmetic methods that are well defined on
subclasses as generic comparison operations. In fact, this is exactly the
design pattern we propose for objects implementing special operators like
NumPy arrays (see NDArrayOperatorsMixin in [1] and [2]).

Here is a simple example, of a well-behaved type that implements addition
by wrapping its value and that returns NotImplemented when the other
operand has the wrong type:

class A:
   def __init__(self, value):
       self.value = value
   def __add__(self, other):
       if not isinstance(other, A):
           return NotImplemented
       return type(self)(self.value + other.value)
   __radd__ = __add__
   def __repr__(self):
       return f'{type(self).__name__}({self.value!r})'

class B(A):
    pass

class C(A):
   def __add__(self, other):
       if not isinstance(other, A):
           return NotImplemented
       return type(self)(self.value + other.value)
   __radd__ = __add__

A does not defer to subclass B:

>>> A(1) + B(1)
A(2)

But it does defer to subclass C, which defines new methods (literally
copied/pasted) for __add__/__radd__:

>>> A(1) + C(1)
C(2)

With the current behavior, special operator implementations need to
explicitly account for the possibility that they are being called from a
subclass by returning NotImplemented. My guess is that this is rarely done,
which means that most of these methods are broken when used with
subclasses, or subclasses needlessly reimplement these methods.

Can we fix this logic for Python 3.7?

[1] https://github.com/numpy/numpy/pull/8247
[2]
https://github.com/charris/numpy/blob/406bbc652424fff332f49b0d2f2e5aedd8191d33/doc/neps/ufunc-overrides.rst
[3] https://docs.python.org/3/reference/datamodel.html#object.__ror__
[4] http://bugs.python.org/issue22052

-----------------------------
Mark Dickinson's response:

This is probably worth bringing up on the python-dev or python-ideas
mailing lists for greater visibility. I can't think of any plausible
non-historical reason why it makes sense for comparisons to behave one way
and arithmetic operators another. Altering this might be a PEP-level
change, though.

The "Coercion rules" section[1] of the Python 2.7 docs is a bit more
explicit about the intent:

"""
Exception to the previous item: if the left operand is an instance of a
built-in type or a new-style class, and the right operand is an instance of
a proper subclass of that type or class and overrides the base?s __rop__()
method, the right operand?s __rop__() method is tried before the left
operand?s __op__() method.
"""

so the check for an override was clearly intentional, rather than an
implementation convenience or accident. (It's also clearly intentional in
the source and comments.) The 3.x docs don't have the "and overrides"
language; I haven't figured out why and when that language changed.

[1]
https://docs.python.org/release/2.7.6/reference/datamodel.html#coercion-rules
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170423/5311e43b/attachment.html>

From steve at pearwood.info  Sun Apr 23 22:54:14 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 24 Apr 2017 12:54:14 +1000
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
Message-ID: <20170424025414.GZ9464@ando.pearwood.info>

On Sun, Apr 23, 2017 at 06:23:12PM -0700, Stephan Hoyer wrote:
> I recently filed this as a bug, and was asked to repost to python-dev or
> python-ideas for greater visibility:
> http://bugs.python.org/issue30140
> 
> Without further ado, here is my original report:
[...]

> The reference documentation on binary arithmetic [3] states:
> 
> > Note: If the right operand's type is a subclass of the left operand?s
> type and that subclass provides the reflected method for the operation,
> this method will be called before the left operand?s non-reflected method.
> This behavior allows subclasses to override their ancestors? operations.
> 
> However, this isn't actually done if the right operand merely inherits from
> the left operand's type. In practice, CPython requires that the right
> operand defines a different method before it defers to it. Note that the
> behavior is different for comparisons, which defer to subclasses regardless
> of whether they implement a new method [4].

I think that's a bug. At the very least, comparisons and operators 
should behave the same.

I think that the comparison behaviour is correct: the subclass 
reflected method should be called, even if the method isn't explicitly 
over-ridden.


> I think this behavior is a mistake and should be corrected. It is just as
> useful to write generic binary arithmetic methods that are well defined on
> subclasses as generic comparison operations.

Agreed.


[...]
> Here is a simple example, of a well-behaved type that implements addition
> by wrapping its value and that returns NotImplemented when the other
> operand has the wrong type:

I often write classes like your example, and it is humbling to realise 
that they are buggy! I must admit I misread the docs and didn't notice 
the "and overrides" language, and clearly I didn't test my classes 
sufficiently or else I would have discovered this myself :-(


> A does not defer to subclass B:
> 
> >>> A(1) + B(1)
> A(2)
> 
> But it does defer to subclass C, which defines new methods (literally
> copied/pasted) for __add__/__radd__:
> 
> >>> A(1) + C(1)
> C(2)

To me, that's the deciding point. We have two ways to go, and one of 
them encourages the copy-and-paste anti-pattern, the other doesn't.

I think that's a Bad Thing and we should treat the comparison behaviour 
as correct, and the other operators are buggy. An explicit over-ride 
shouldn't be necessary.


> Mark Dickinson's response:
>
> The "Coercion rules" section[1] of the Python 2.7 docs is a bit more
> explicit about the intent:
> 
> """
> Exception to the previous item: if the left operand is an instance of a
> built-in type or a new-style class, and the right operand is an instance of
> a proper subclass of that type or class and overrides the base?s __rop__()
> method, the right operand?s __rop__() method is tried before the left
> operand?s __op__() method.
> """
> 
> so the check for an override was clearly intentional, rather than an
> implementation convenience or accident. (It's also clearly intentional in
> the source and comments.)

The next paragraph tells us:

    This is done so that a subclass can completely override binary 
    operators. Otherwise, the left operand?s __op__() method would 
    always accept the right operand: when an instance of a given class 
    is expected, an instance of a subclass of that class is always 
    acceptable.


but as far as I can tell, the comparison behaviour equally accomplishes 
that, without an explicit over-ride.


> The 3.x docs don't have the "and overrides"
> language; I haven't figured out why and when that language changed.
> 
> [1]
> https://docs.python.org/release/2.7.6/reference/datamodel.html#coercion-rules



From greg.ewing at canterbury.ac.nz  Mon Apr 24 01:57:17 2017
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 24 Apr 2017 17:57:17 +1200
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
Message-ID: <58FD93BD.8050908@canterbury.ac.nz>

Stephan Hoyer wrote:
> In practice, CPython requires that the 
> right operand defines a different method before it defers to it.

I'm not sure exactly what the rationale for this behaviour is,
but it's probably something along the lines that the left
method should already know how to deal with that combination
of types, and right methods are only supposed to be called
as a fallback if the left method can't handle the operands,
so calling it in that situation would be wrong.

Following that logic, the wrapper's __add__ method in your
example needs to allow for the subclassing case, e.g.

    def __add__(self, other):
        t1 = type(self)
        t2 = type(other)
        t = t2 if issubclass(t2, t1) else t1
        return t(self.value + other.value)

> the behavior is different for comparisons, which defer to 
> subclasses regardless of whether they implement a new method

Comparisons are a bit different, because they don't have
separate left and right methods, although it's hard to see
exactly how that affects the logic.

> I think this behavior is a mistake and should be corrected.

Possibly, but it's not clear how much breakage might result
from changing it now. Although...

> The 3.x docs don't have the "and overrides" language;

so arguably we would be changing the implementation to match
the docs.

-- 
Greg


From shoyer at gmail.com  Mon Apr 24 12:14:17 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Mon, 24 Apr 2017 09:14:17 -0700
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <58FD93BD.8050908@canterbury.ac.nz>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
Message-ID: <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>

On Sun, Apr 23, 2017 at 10:57 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
wrote:

> Stephan Hoyer wrote:
>
>> In practice, CPython requires that the right operand defines a different
>> method before it defers to it.
>>
>
> I'm not sure exactly what the rationale for this behaviour is,
> but it's probably something along the lines that the left
> method should already know how to deal with that combination
> of types, and right methods are only supposed to be called
> as a fallback if the left method can't handle the operands,
> so calling it in that situation would be wrong.
>
>>
Yes, this could makes sense. But in that case, why check for explicitly
overridden methods on subclasses at all? I can rationalize either not
treating subclasses differently or always trying subclasses first, but not
the current behavior.

Of these two options, I prefer always trying subclasses first because I
agree with the rationale in the docs: "This behavior allows subclasses to
override their ancestors? operations."

In general, code should be written such that subclasses are aware of
super-classes, not the other way around.


> The 3.x docs don't have the "and overrides" language;
>>
>
> so arguably we would be changing the implementation to match
> the docs.
>

Based on the change in the documentation between 2.x and 3.x, I wonder if
this is something that someone intended to clean up as part of Python 3000
but never got around to. I would love to hear from anyone familiar with the
historical context here.

Nonetheless, at this point the behavior has been around for quite some
time. Almost assuredly, there is *someone* relying on this feature/bug,
though probably unintentionally. So I would also love to hear from anyone
who knows of code that this change would actually break (as opposed to fix
and/or allow for removing redundant methods).

More broadly: is this change significant enough that it needs a PEP?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170424/5378708a/attachment.html>

From steve at pearwood.info  Mon Apr 24 14:25:06 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 25 Apr 2017 04:25:06 +1000
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <58FD93BD.8050908@canterbury.ac.nz>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
Message-ID: <20170424182506.GB20708@ando.pearwood.info>

On Mon, Apr 24, 2017 at 05:57:17PM +1200, Greg Ewing wrote:
> Stephan Hoyer wrote:
> >In practice, CPython requires that the 
> >right operand defines a different method before it defers to it.
> 
> I'm not sure exactly what the rationale for this behaviour is,
> but it's probably something along the lines that the left
> method should already know how to deal with that combination
> of types, and right methods are only supposed to be called
> as a fallback if the left method can't handle the operands,
> so calling it in that situation would be wrong.

I've never seen that rationale before, and I don't think I would agree 
with it. And it goes against the rationale in the docs:

    [...] the right operand?s __rop__() method is tried before the 
    left operand?s __op__() method.

    This is done so that a subclass can completely override binary 
    operators. Otherwise, the left operand?s __op__() method would 
    always accept the right operand: when an instance of a given 
    class is expected, an instance of a subclass of that class is 
    always acceptable.


I think your rationale goes against the intention as documented. There's 
no expectation that __rop__ methods are only to be called when the 
__op__ method can't handle the operands. 

In general, which operand "wins" should depend on the classes, not on 
whether they happen to be on the left or right of the operator. (Except 
in the case where we cannot decide between the operands, in which case 
we break ties by preferring the __op__.) The reason is that subclasses 
are usually intended to be more specialised than their parent, and so 
they ought to be given priority in mixed operations.

Given classes X, Y(X), with instances x and y, we should expect that the 
more specialised class (namely Y) gets called first whether we write:

    x ? y

or 

    y ? x

for any operator ?.

As documented in the 3 docs, we get that for free: the interpreter 
correctly calls the __op__ or __rop__ method, as needed, and the class 
author doesn't have to think about it. That's how it's documented, but 
not how it's implemented.

The alternative is that every class has to include boilerplate testing 
for subclasses, as you say:


> Following that logic, the wrapper's __add__ method in your
> example needs to allow for the subclassing case, e.g.
> 
>    def __add__(self, other):
>        t1 = type(self)
>        t2 = type(other)
>        t = t2 if issubclass(t2, t1) else t1
>        return t(self.value + other.value)

but that's bad. That makes each and every class (that might ever be 
subclassed) responsible for checking for subclasses, instead of putting 
the check in one place (whichever part of the interpreter handles 
calling __op__/__rop__ methods).

Remember that a specialised subclass might not overload the __op__ and 
__rop__ methods themselves. It might overload a data attribute, or 
another method that __op__ / __rop__ call.

class A:
    def __add__(self, other):
        self.log()
        ...
    __radd__ = __add__

class B(A):
    def log(self):
        ...

A() + B()

As the more specialised instance (a subclass of A), the right hand 
operand should get the priority.


> >the behavior is different for comparisons, which defer to 
> >subclasses regardless of whether they implement a new method
> 
> Comparisons are a bit different, because they don't have
> separate left and right methods, although it's hard to see
> exactly how that affects the logic.

It doesn't affect the logic, and comparisons implement exactly the 
documented (in 3) behaviour. The only difference is that the 
reversed methods aren't spelled __rop__:

    __eq__ and __ne__ are their own reflection;
    __lt__ and __gt__
    __le__ and __ge__


For example:

py> class A(object):
...     def __lt__(self, other):
...         print("lt", self)
...         return True
...     def __gt__(self, other):
...         print("gt", self)
...         return False
...
py> class B(A):
...     pass
...
py> A() < B()
gt <__main__.B object at 0xb7a9e8ec>
False


The more specialised class (B) has its method called, even though it 
isn't over-ridden.



-- 
Steve

From tjreedy at udel.edu  Mon Apr 24 16:15:10 2017
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 24 Apr 2017 16:15:10 -0400
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
 <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
Message-ID: <odlmca$dql$1@blaine.gmane.org>

On 4/24/2017 12:14 PM, Stephan Hoyer wrote:

> Based on the change in the documentation between 2.x and 3.x, I wonder 
> if this is something that someone intended to clean up as part of Python 
> 3000 but never got around to. I would love to hear from anyone familiar 
> with the historical context here.

We should ask the intention of the person who made the change, which is 
in the repository.  If you email me the doc (the last part of the url) 
and location within, I will look it up.

-- 
Terry Jan Reedy


From prometheus235 at gmail.com  Mon Apr 24 19:29:52 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Mon, 24 Apr 2017 18:29:52 -0500
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <odlmca$dql$1@blaine.gmane.org>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
 <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
 <odlmca$dql$1@blaine.gmane.org>
Message-ID: <CAHkxivcO049ov_XgApic-unqHYmG2MNKw15LNXnzCKNrGiaK3A@mail.gmail.com>

GitHub shows that that note hasn't changed in 10 years:
https://github.com/python/cpython/blame/master/Doc/reference/datamodel.rst#L2210

On Mon, Apr 24, 2017 at 3:15 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 4/24/2017 12:14 PM, Stephan Hoyer wrote:
>
> Based on the change in the documentation between 2.x and 3.x, I wonder if
>> this is something that someone intended to clean up as part of Python 3000
>> but never got around to. I would love to hear from anyone familiar with the
>> historical context here.
>>
>
> We should ask the intention of the person who made the change, which is in
> the repository.  If you email me the doc (the last part of the url) and
> location within, I will look it up.
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170424/cf9c9f5b/attachment-0001.html>

From shoyer at gmail.com  Mon Apr 24 19:46:44 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Mon, 24 Apr 2017 16:46:44 -0700
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAHkxivcO049ov_XgApic-unqHYmG2MNKw15LNXnzCKNrGiaK3A@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
 <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
 <odlmca$dql$1@blaine.gmane.org>
 <CAHkxivcO049ov_XgApic-unqHYmG2MNKw15LNXnzCKNrGiaK3A@mail.gmail.com>
Message-ID: <CAEQ_Tvdt69UoeHBRiD=VEMecPEecccOxjdqCWDEteOfk8zFESA@mail.gmail.com>

+Georg Brandl, in case he remembers where "Move the 3k reST doc tree in
place." moved things from:
https://github.com/python/cpython/commit/116aa62bf54a39697e25f21d6cf6799f7faa1349

On Mon, Apr 24, 2017 at 4:29 PM, Nick Timkovich <prometheus235 at gmail.com>
wrote:

> GitHub shows that that note hasn't changed in 10 years:
> https://github.com/python/cpython/blame/master/
> Doc/reference/datamodel.rst#L2210
>
> On Mon, Apr 24, 2017 at 3:15 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>> On 4/24/2017 12:14 PM, Stephan Hoyer wrote:
>>
>> Based on the change in the documentation between 2.x and 3.x, I wonder if
>>> this is something that someone intended to clean up as part of Python 3000
>>> but never got around to. I would love to hear from anyone familiar with the
>>> historical context here.
>>>
>>
>> We should ask the intention of the person who made the change, which is
>> in the repository.  If you email me the doc (the last part of the url) and
>> location within, I will look it up.
>>
>> --
>> Terry Jan Reedy
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170424/6792473a/attachment.html>

From gvanrossum at gmail.com  Mon Apr 24 19:54:23 2017
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Mon, 24 Apr 2017 16:54:23 -0700
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAEQ_Tvdt69UoeHBRiD=VEMecPEecccOxjdqCWDEteOfk8zFESA@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
 <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
 <odlmca$dql$1@blaine.gmane.org>
 <CAHkxivcO049ov_XgApic-unqHYmG2MNKw15LNXnzCKNrGiaK3A@mail.gmail.com>
 <CAEQ_Tvdt69UoeHBRiD=VEMecPEecccOxjdqCWDEteOfk8zFESA@mail.gmail.com>
Message-ID: <CAP7+vJ+cGKiWG91Ypv-=zCMZJ8TSMMpVXRsvHUQFaiBQ+jX4=Q@mail.gmail.com>

If this is portant I should probably ponder it.

On Apr 24, 2017 4:47 PM, "Stephan Hoyer" <shoyer at gmail.com> wrote:

> +Georg Brandl, in case he remembers where "Move the 3k reST doc tree in
> place." moved things from:
> https://github.com/python/cpython/commit/116aa62bf54a39697e25f21d6cf679
> 9f7faa1349
>
> On Mon, Apr 24, 2017 at 4:29 PM, Nick Timkovich <prometheus235 at gmail.com>
> wrote:
>
>> GitHub shows that that note hasn't changed in 10 years:
>> https://github.com/python/cpython/blame/master/Doc/
>> reference/datamodel.rst#L2210
>>
>> On Mon, Apr 24, 2017 at 3:15 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>>> On 4/24/2017 12:14 PM, Stephan Hoyer wrote:
>>>
>>> Based on the change in the documentation between 2.x and 3.x, I wonder
>>>> if this is something that someone intended to clean up as part of Python
>>>> 3000 but never got around to. I would love to hear from anyone familiar
>>>> with the historical context here.
>>>>
>>>
>>> We should ask the intention of the person who made the change, which is
>>> in the repository.  If you email me the doc (the last part of the url) and
>>> location within, I will look it up.
>>>
>>> --
>>> Terry Jan Reedy
>>>
>>>
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170424/99f16652/attachment.html>

From python at lucidity.plus.com  Mon Apr 24 21:08:05 2017
From: python at lucidity.plus.com (Erik)
Date: Tue, 25 Apr 2017 02:08:05 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
Message-ID: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>

Hi. I suspect that this may have been discussed to death at some point 
in the past, but I've done some searching and I didn't come up with 
much. Apologies if I'm rehashing an old argument ;)

I often find myself writing __init__ methods of the form:

def __init__(self, foo, bar, baz, spam, ham):
   self.foo = foo
   self.bar = bar
   self.baz = baz
   self.spam = spam
   self.ham = ham

This seems a little wordy and uses a lot of vertical space on the 
screen. Occasionally, I have considered something like:

def __init__(self, foo, bar, baz, spam, ham):
   self.foo, self.bar, self.baz, self.spam, self.ham = \
      foo, bar, baz, spam, ham

... just to make it a bit more compact - though in practice, I'd 
probably not do that with a list quite that long ... two or three items 
at most:

def __init__(self, foo, bar, baz):
    self.foo, self.bar, self.baz = foo, bar, baz

When I do that I'm torn because I know it has a runtime impact to create 
and unpack the implicit tuples and I'm also introducing a style 
asymmetry in my code just because of the number of parameters a method 
happens to have.

So why not have an augmented assignment operator for object attributes? 
It addresses one of the same broad issues that the other augmented 
assignment operators were introduced for (that of repeatedly spelling 
names).

The suggestion therefore is:

def __init__(self, foo, bar, baz, spam, ham):
   self .= foo, bar, baz, spam, ham

This is purely syntactic sugar for the original example:

def __init__(self, foo, bar, baz, spam, ham):
   self.foo = foo
   self.bar = bar
   self.baz = baz
   self.spam = spam
   self.ham = ham

... so if any of the attributes have setters, then they are called as 
usual. It's purely a syntactic shorthand. Any token which is not 
suitable on the RHS of the dot in a standard "obj.attr =" assignment is 
a syntax error (no "self .= 1").

The comma-separators in the example are not creating a tuple object, 
they would work at the same level in the parser as the import 
statement's comma-separated lists - in the same way that "from pkg 
import a, b, c" is the same as saying:

import pkg
a = pkg.a
b = pkg.b
c = pkg.c

... "self .= a, b, c" is the same as writing:

self.a = a
self.b = b
self.c = c

E.

From rosuav at gmail.com  Mon Apr 24 21:15:12 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 25 Apr 2017 11:15:12 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <CAPTjJmrorg68=J33jkRtOhKiVmwbXhFb_9uL-n2m01T4O_k8bw@mail.gmail.com>

On Tue, Apr 25, 2017 at 11:08 AM, Erik <python at lucidity.plus.com> wrote:
> The suggestion therefore is:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self .= foo, bar, baz, spam, ham
>
> This is purely syntactic sugar for the original example:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
>
> ... so if any of the attributes have setters, then they are called as usual.
> It's purely a syntactic shorthand. Any token which is not suitable on the
> RHS of the dot in a standard "obj.attr =" assignment is a syntax error (no
> "self .= 1").

Bikeshedding: Your example looks a lot more like tuple assignment than
multiple assignment. I'd rather it link more with the way that
multiple assignment works:

# simple multiple assignment
a = b = c = d = e = 0
# object member assignment
self .= foo .= bar .= baz .= spam .= ham

The trouble is that this syntax is really only going to be used inside
__init__. It's hard to justify new syntax for one purpose like this.
So I'm swaying between -0 and +0.5 on this.

ChrisA

From steve at pearwood.info  Mon Apr 24 22:53:06 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 25 Apr 2017 12:53:06 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <20170425025305.GC20708@ando.pearwood.info>

On Tue, Apr 25, 2017 at 02:08:05AM +0100, Erik wrote:

> I often find myself writing __init__ methods of the form:
> 
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
> 
> This seems a little wordy and uses a lot of vertical space on the 
> screen. 

It does, and while both are annoying, in the grand scheme of things 
they're a very minor annoyance. After all, this is typically only an 
issue once per class, and not even every class, and vertical space is 
quite cheap. In general, the barrier for accepting new syntax is quite 
high, and "minor annoyance" generally doesn't reach it.

I'm completely with you about the annoyance factor here. It is 
especially annoying during the early stages of development when the 
__init__ method is evolving rapidly (parameters are being renamed, added 
or removed). One work-around is this pattern:

    def __init__(self, spam, ham, eggs, cheese, aardvark):
        vars(self).update(locals())
        del self.self
        ...


which is cute but I've never quite been brave enough to use it in 
production. But I don't think the annoyance factor is high enough, or 
common enough, to justify syntactic sugar to "fix" it.



> Occasionally, I have considered something like:
> 
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo, self.bar, self.baz, self.spam, self.ham = \
>      foo, bar, baz, spam, ham
> 
> ... just to make it a bit more compact
[...]
> When I do that I'm torn because I know it has a runtime impact to create 
> and unpack the implicit tuples and I'm also introducing a style 
> asymmetry in my code just because of the number of parameters a method 
> happens to have.

I'm not too worried about the performance impact unless I've 
profiled my code and it is actually significant.

As far as the style asymmetry, if we use your proposed syntax, that's 
introducing style asymmetry too.



> So why not have an augmented assignment operator for object attributes? 

Wrong question. Don't ask "why not add this feature?". The question to 
ask is "why add this feature?" Every new feature has cost, whether it is 
run-time cost, development cost, feature bloat, learning curve cost, 
etc, so the feature must be justified as adding sufficiently more 
value than it costs.

Here are a few reasons why this feature fails to meet the barrier for 
new syntax.

- This feature doesn't add any new capability to the language. There's 
nothing that this feature allows you to do which you couldn't do before.

- It has quite narrow applicability. It's really only useful inside a 
small proportion of __init__ methods.

- In my experience, it is very rare to have a whole set of unadorned 
assignments in the way you show it. MUCH more common is to have some 
sort of pre-processing of the argument before assignment, or for the 
assignment target to be different from the parameter name:


    def __init__(self, spam, eggs, cheese, aardvark, ...):
        # type checks
        if not isinstance(spam, Foo):
            raise TypeError
        self._spam = spam  # not all parameters are public attributes
        # value checks
        if eggs < 0:
            raise ValueError
        self.eggs = eggs
        # replacement of None
        if cheese is None:
            cheese = make_cheese()
        self.cheese = cheese
        # other pre-processing
        self.mammals = [aardvark, get_weasel()]


which reduces the applicability of this syntax even further.

- Having big parameter lists is something of a mild code smell. As the 
parameter list gets bigger, the smell gets worse. Having so many 
parameters that this feature becomes attractive should be painful, 
because its a warning that maybe your class has too many parameters.



> It addresses one of the same broad issues that the other augmented 
> assignment operators were introduced for (that of repeatedly spelling 
> names).
> 
> The suggestion therefore is:
> 
> def __init__(self, foo, bar, baz, spam, ham):
>   self .= foo, bar, baz, spam, ham

But there's a major difference between this and the other augmented 
assignment operators, so major that the analogy between them doesn't 
hold up.

In augmented assignment, the target can be any valid target, and the RHS 
can be any expression

    mylist[0].attr['key'] += (some_value() or another) + 99

Your suggestion is very different: there are *many* targets, and the RHS 
must be a comma-separated list of identifiers. They're quite different 
kinds of assignment. And unlike augmented assignment, which calls one of 
the __iop__ methods, this would have to be handled purely as syntax.

> This is purely syntactic sugar for the original example:
> 
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
> 
> ... so if any of the attributes have setters, then they are called as 
> usual. It's purely a syntactic shorthand. Any token which is not 
> suitable on the RHS of the dot in a standard "obj.attr =" assignment is 
> a syntax error (no "self .= 1").

Right. And that *seriously* limits the applicability of this.

Yes, I've written classes with a dozen or twenty parameters. But 
especially with long parameter lists, invariably at least *some* of 
those arguments are pre-processed before assignment to an attribute:

    self.spam = spam + 1

or are assigned to a different name:

    self._spam = spam

or both:

    self.foods = [spam, eggs, cheese]

So my expectation is that even if this syntax was accepted, I wouldn't 
use it, because by the time I've processed those assignments, the ones 
that are left are too few to bother with the special syntax.

The bottom line is that I think you've identified a very real, but 
minor, annoyance, but the applicability of the solution is too low to 
justify new syntax.



-- 
Steve

From njs at pobox.com  Tue Apr 25 00:04:40 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 24 Apr 2017 21:04:40 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <CAPJVwBkgohydiFvEGaEKovivZCsdCuLgQib2QVy5b2vMBQo1kw@mail.gmail.com>

On Mon, Apr 24, 2017 at 6:08 PM, Erik <python at lucidity.plus.com> wrote:
> Hi. I suspect that this may have been discussed to death at some point in
> the past, but I've done some searching and I didn't come up with much.
> Apologies if I'm rehashing an old argument ;)
>
> I often find myself writing __init__ methods of the form:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham

This isn't a direct response, but you might be interested in the attrs library:
    https://attrs.readthedocs.io/en/stable/examples.html

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From p.f.moore at gmail.com  Tue Apr 25 03:56:51 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 25 Apr 2017 08:56:51 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170425025305.GC20708@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
Message-ID: <CACac1F-5fjzS60GNDzjidbE3pL6E5SJFW7bUrgJJsOFRA0T4JA@mail.gmail.com>

On 25 April 2017 at 03:53, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, Apr 25, 2017 at 02:08:05AM +0100, Erik wrote:
>
>> I often find myself writing __init__ methods of the form:
>>
>> def __init__(self, foo, bar, baz, spam, ham):
>>   self.foo = foo
>>   self.bar = bar
>>   self.baz = baz
>>   self.spam = spam
>>   self.ham = ham
>>
>> This seems a little wordy and uses a lot of vertical space on the
>> screen.
>
> It does, and while both are annoying, in the grand scheme of things
> they're a very minor annoyance. After all, this is typically only an
> issue once per class, and not even every class, and vertical space is
> quite cheap. In general, the barrier for accepting new syntax is quite
> high, and "minor annoyance" generally doesn't reach it.

I suspect that with a suitably creative use of inspect.signature() you
could write a decorator for this:

@auto_attrs
def __init__(self, a, b, c):
    # Remaining init code, called with self.a, self.b and self.c set

I don't have time to experiment right now, but will try to find time
later. If nothing else, such a decorator would be a good prototype for
the proposed functionality, and may well be sufficient for the likely
use cases without needing a syntax change.

Paul

From dmoisset at machinalis.com  Tue Apr 25 07:54:22 2017
From: dmoisset at machinalis.com (Daniel Moisset)
Date: Tue, 25 Apr 2017 12:54:22 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F-5fjzS60GNDzjidbE3pL6E5SJFW7bUrgJJsOFRA0T4JA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <CACac1F-5fjzS60GNDzjidbE3pL6E5SJFW7bUrgJJsOFRA0T4JA@mail.gmail.com>
Message-ID: <CALuYSZWrG4m10LsoV+iyiReUZjwKncrvTooLhFbLamzjf7C_Bw@mail.gmail.com>

I actually saw a decorator like that last week,
https://twitter.com/judy2k/status/854330478068977664

On 25 April 2017 at 08:56, Paul Moore <p.f.moore at gmail.com> wrote:

> On 25 April 2017 at 03:53, Steven D'Aprano <steve at pearwood.info> wrote:
> > On Tue, Apr 25, 2017 at 02:08:05AM +0100, Erik wrote:
> >
> >> I often find myself writing __init__ methods of the form:
> >>
> >> def __init__(self, foo, bar, baz, spam, ham):
> >>   self.foo = foo
> >>   self.bar = bar
> >>   self.baz = baz
> >>   self.spam = spam
> >>   self.ham = ham
> >>
> >> This seems a little wordy and uses a lot of vertical space on the
> >> screen.
> >
> > It does, and while both are annoying, in the grand scheme of things
> > they're a very minor annoyance. After all, this is typically only an
> > issue once per class, and not even every class, and vertical space is
> > quite cheap. In general, the barrier for accepting new syntax is quite
> > high, and "minor annoyance" generally doesn't reach it.
>
> I suspect that with a suitably creative use of inspect.signature() you
> could write a decorator for this:
>
> @auto_attrs
> def __init__(self, a, b, c):
>     # Remaining init code, called with self.a, self.b and self.c set
>
> I don't have time to experiment right now, but will try to find time
> later. If nothing else, such a decorator would be a good prototype for
> the proposed functionality, and may well be sufficient for the likely
> use cases without needing a syntax change.
>
> Paul
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Daniel F. Moisset - UK Country Manager
www.machinalis.com
Skype: @dmoisset
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/7a46dbab/attachment-0001.html>

From george at fischhof.hu  Tue Apr 25 08:33:49 2017
From: george at fischhof.hu (George Fischhof)
Date: Tue, 25 Apr 2017 14:33:49 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F-5fjzS60GNDzjidbE3pL6E5SJFW7bUrgJJsOFRA0T4JA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <CACac1F-5fjzS60GNDzjidbE3pL6E5SJFW7bUrgJJsOFRA0T4JA@mail.gmail.com>
Message-ID: <CAFwcP0gjg1BXeQ2pmv637bXG6+vcXiVu1QNEkdetuHjmoT_yCw@mail.gmail.com>

2017. ?pr. 25. de. 10:04 ezt ?rta ("Paul Moore" <p.f.moore at gmail.com>):

On 25 April 2017 at 03:53, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, Apr 25, 2017 at 02:08:05AM +0100, Erik wrote:
>
>> I often find myself writing __init__ methods of the form:
>>
>> def __init__(self, foo, bar, baz, spam, ham):
>>   self.foo = foo
>>   self.bar = bar
>>   self.baz = baz
>>   self.spam = spam
>>   self.ham = ham
>>
>> This seems a little wordy and uses a lot of vertical space on the
>> screen.
>
> It does, and while both are annoying, in the grand scheme of things
> they're a very minor annoyance. After all, this is typically only an
> issue once per class, and not even every class, and vertical space is
> quite cheap. In general, the barrier for accepting new syntax is quite
> high, and "minor annoyance" generally doesn't reach it.

I suspect that with a suitably creative use of inspect.signature() you
could write a decorator for this:

@auto_attrs
def __init__(self, a, b, c):
    # Remaining init code, called with self.a, self.b and self.c set

I don't have time to experiment right now, but will try to find time
later. If nothing else, such a decorator would be a good prototype for
the proposed functionality, and may well be sufficient for the likely
use cases without needing a syntax change.

Paul
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/



Hi, such a decorator would be very grateful ;-) in the standard as for
example I use class parameters this way too in more than 90% of the cases.

BR
George
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/2d9d20f3/attachment.html>

From tinchester at gmail.com  Tue Apr 25 12:16:36 2017
From: tinchester at gmail.com (=?UTF-8?Q?Tin_Tvrtkovi=C4=87?=)
Date: Tue, 25 Apr 2017 16:16:36 +0000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <mailman.41.1493136006.9763.python-ideas@python.org>
References: <mailman.41.1493136006.9763.python-ideas@python.org>
Message-ID: <CAEJ-mUoZ9UC13W2ybbWcasUhvCBaL9RgPbvMtUExpM8Lq2en4w@mail.gmail.com>

You might want to check out attrs (http://attrs.readthedocs.io/en/stable/).
It can generate the __init__ for you, and much much more.


Date: Tue, 25 Apr 2017 14:33:49 +0200
> From: George Fischhof <george at fischhof.hu>
> To: Paul Moore <p.f.moore at gmail.com>
> Cc: Python-Ideas <python-ideas at python.org>, "Steven D'Aprano"
>         <steve at pearwood.info>
> Subject: Re: [Python-ideas] Augmented assignment syntax for objects.
> Message-ID:
>         <
> CAFwcP0gjg1BXeQ2pmv637bXG6+vcXiVu1QNEkdetuHjmoT_yCw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> 2017. ?pr. 25. de. 10:04 ezt ?rta ("Paul Moore" <p.f.moore at gmail.com>):
>
> On 25 April 2017 at 03:53, Steven D'Aprano <steve at pearwood.info> wrote:
> > On Tue, Apr 25, 2017 at 02:08:05AM +0100, Erik wrote:
> >
> >> I often find myself writing __init__ methods of the form:
> >>
> >> def __init__(self, foo, bar, baz, spam, ham):
> >>   self.foo = foo
> >>   self.bar = bar
> >>   self.baz = baz
> >>   self.spam = spam
> >>   self.ham = ham
> >>
> >> This seems a little wordy and uses a lot of vertical space on the
> >> screen.
> >
> > It does, and while both are annoying, in the grand scheme of things
> > they're a very minor annoyance. After all, this is typically only an
> > issue once per class, and not even every class, and vertical space is
> > quite cheap. In general, the barrier for accepting new syntax is quite
> > high, and "minor annoyance" generally doesn't reach it.
>
> I suspect that with a suitably creative use of inspect.signature() you
> could write a decorator for this:
>
> @auto_attrs
> def __init__(self, a, b, c):
>     # Remaining init code, called with self.a, self.b and self.c set
>
> I don't have time to experiment right now, but will try to find time
> later. If nothing else, such a decorator would be a good prototype for
> the proposed functionality, and may well be sufficient for the likely
> use cases without needing a syntax change.
>
> Paul
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/c13d2065/attachment.html>

From gvanrossum at gmail.com  Tue Apr 25 12:56:19 2017
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Tue, 25 Apr 2017 09:56:19 -0700
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAP7+vJ+cGKiWG91Ypv-=zCMZJ8TSMMpVXRsvHUQFaiBQ+jX4=Q@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
 <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
 <odlmca$dql$1@blaine.gmane.org>
 <CAHkxivcO049ov_XgApic-unqHYmG2MNKw15LNXnzCKNrGiaK3A@mail.gmail.com>
 <CAEQ_Tvdt69UoeHBRiD=VEMecPEecccOxjdqCWDEteOfk8zFESA@mail.gmail.com>
 <CAP7+vJ+cGKiWG91Ypv-=zCMZJ8TSMMpVXRsvHUQFaiBQ+jX4=Q@mail.gmail.com>
Message-ID: <CAP7+vJKcqfSQF6rrKu_3fNhOO4K=VA0t3Jd1ckksA=UGafxzXg@mail.gmail.com>

Now that I am with a real keyboard and screen and have tried to understand
the OP, I can actually write up my thoughts on the matter.

There are two aspects to the behavior. Giving preference to the class of
the right operand if it is a subclass of the left operand's class is
reasonable and explained in the docs.

Only doing this if the right operand actually overrides __rop__ was perhaps
meant as an optimization, and I expect that when I introduced that rule I
hadn't thought of the kind of classes that use type(self) or self.__class__
to return an instance of the runtime class.

However there's also another thing to consider. Consider a variant of the
OP example, where B doesn't override __add__ or __radd__, but now add a
different __init__ signature, one that requires completely different
arguments. This is entirely legal and done often enough (though perhaps not
in numpy circles?) -- constructors are not subject to the Liskov rule. So
the call to type(self)(<value>) may crash or have an undesirable result.

But given that A does call type(self)(<value>), all its subclasses have to
either have a compatible __init__ or override both __add__ and __radd__.

In the end I agree with the OP that we should fix this. I don't see a
reason to require a PEP or require updating whatever PEP described this
behavior originally -- PEPs generally describe what should be done to a
specific version of Python, they don't prevent future alterations, and they
essentially represent the historical record, not current documentation.

I'm a little worried about breaking existing code, but only a little bit,
and this is clearly a gray area, so I think it's okay to change in 3.7
without deprecations. (But I've been overruled on such matters before, so
if you disagree, speak up now and show us your code!)

--Guido

On Mon, Apr 24, 2017 at 4:54 PM, Guido van Rossum <gvanrossum at gmail.com>
wrote:

> If this is portant I should probably ponder it.
>
> On Apr 24, 2017 4:47 PM, "Stephan Hoyer" <shoyer at gmail.com> wrote:
>
>> +Georg Brandl, in case he remembers where "Move the 3k reST doc tree in
>> place." moved things from:
>> https://github.com/python/cpython/commit/116aa62bf54a39697e2
>> 5f21d6cf6799f7faa1349
>>
>> On Mon, Apr 24, 2017 at 4:29 PM, Nick Timkovich <prometheus235 at gmail.com>
>> wrote:
>>
>>> GitHub shows that that note hasn't changed in 10 years:
>>> https://github.com/python/cpython/blame/master/Doc/re
>>> ference/datamodel.rst#L2210
>>>
>>> On Mon, Apr 24, 2017 at 3:15 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>>
>>>> On 4/24/2017 12:14 PM, Stephan Hoyer wrote:
>>>>
>>>> Based on the change in the documentation between 2.x and 3.x, I wonder
>>>>> if this is something that someone intended to clean up as part of Python
>>>>> 3000 but never got around to. I would love to hear from anyone familiar
>>>>> with the historical context here.
>>>>>
>>>>
>>>> We should ask the intention of the person who made the change, which is
>>>> in the repository.  If you email me the doc (the last part of the url) and
>>>> location within, I will look it up.
>>>>
>>>> --
>>>> Terry Jan Reedy
>>>>
>>>>
>>>> _______________________________________________
>>>> Python-ideas mailing list
>>>> Python-ideas at python.org
>>>> https://mail.python.org/mailman/listinfo/python-ideas
>>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>>
>>>
>>>
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/4d6af93b/attachment.html>

From python-ideas at mgmiller.net  Tue Apr 25 14:16:22 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Tue, 25 Apr 2017 11:16:22 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170425025305.GC20708@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
Message-ID: <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>

Agreed with Steven, although I do find myself a little more annoyed and bothered 
by a typical init than him I guess.

Even so I didn't think the current proposals went far enough.  To tilt the 
balance farther, to make it easier, let's go all the way!  Instead of continuing 
duplication:

> >>> def __init__(self, foo, bar, baz, spam, ham):
>   self .= foo, bar, baz, spam, ham

or

> # object member assignment
> self .= foo .= bar .= baz .= spam .= ham

How about?

     def __init__(self, foo, bar, baz, spam, ham):
       self .= *

The asterisk here used to be reminiscent of argument unpacking (minus self). 
That would imply a double asterisk for keyword assignment which could be used as 
well.

Also, I did find the decorator proposal intriguing, though have to say I 
probably wouldn't bother to use it unless it were a builtin or I had a dozen 
parameters to deal with.

-Mike

From srkunze at mail.de  Tue Apr 25 15:45:55 2017
From: srkunze at mail.de (Sven R. Kunze)
Date: Tue, 25 Apr 2017 21:45:55 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <8f8b937f-c470-00a5-1586-527c9293861a@mail.de>

On 25.04.2017 20:16, Mike Miller wrote:
> Also, I did find the decorator proposal intriguing, though have to say 
> I probably wouldn't bother to use it unless it were a builtin or I had 
> a dozen parameters to deal with.

Same here. And practical experience tells me that the usage of this 
decorator wouldn't last long.
In Steven's "early stages of development", the day will come where there 
is a need for a parameter which must not be an attribute on self.

So I prefer the "annoyance" since it's bloody easy to read AND to change.

Sven


From contact at brice.xyz  Tue Apr 25 17:15:19 2017
From: contact at brice.xyz (Brice PARENT)
Date: Tue, 25 Apr 2017 23:15:19 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
Message-ID: <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>



Le 25/04/17 ? 20:16, Mike Miller a ?crit :
> Agreed with Steven, although I do find myself a little more annoyed 
> and bothered by a typical init than him I guess.
>
> Even so I didn't think the current proposals went far enough.  To tilt 
> the balance farther, to make it easier, let's go all the way!  Instead 
> of continuing duplication:
>
>> >>> def __init__(self, foo, bar, baz, spam, ham):
>>   self .= foo, bar, baz, spam, ham
>
> or
>
>> # object member assignment
>> self .= foo .= bar .= baz .= spam .= ham
>
> How about?
>
>     def __init__(self, foo, bar, baz, spam, ham):
>       self .= *
>
> The asterisk here used to be reminiscent of argument unpacking (minus 
> self). That would imply a double asterisk for keyword assignment which 
> could be used as well.
>
> Also, I did find the decorator proposal intriguing, though have to say 
> I probably wouldn't bother to use it unless it were a builtin or I had 
> a dozen parameters to deal with.
>
If you *need* a shorter solution, even though I'm not entirely sure 
there's a real need for it, it may be easier to get something like this 
(I think, as there is no new operator involved) :

def __init__(self, *args, **kwargs):
       self.* = *args
       self.** = **kwargs

And I'm sure this could easily find new use cases outside of a constructor.

But, any of these proposals, mine and yours, if you really need this to 
shorten the code writting time or vertical space only, is not a better 
idea than to propose a macro to your IDE editor, or a pull request if 
it's open source. Such a macro would generate automatically those 
long-to-write lines, and maybe implement some code folding if vertical 
space is an issue.

- Brice


From python-ideas at mgmiller.net  Tue Apr 25 17:27:51 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Tue, 25 Apr 2017 14:27:51 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
Message-ID: <5170eee5-7ba0-0413-15cd-d539713c4d60@mgmiller.net>



On 2017-04-25 14:15, Brice PARENT wrote:
> But, any of these proposals, mine and yours, if you really need this to shorten
> the code writing time or vertical space only, is not a better idea than to
> propose a macro to your IDE editor, or a pull request if it's open source. Such
> a macro would generate automatically those long-to-write lines, and maybe
> implement some code folding if vertical space is an issue.

I'm personally not bothered by vertical space, but rather typing the same thing 
three times with self's and ='s.  That's more boilerplate than Java.  ;)

From p.f.moore at gmail.com  Tue Apr 25 18:05:19 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 25 Apr 2017 23:05:19 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
Message-ID: <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>

On 25 April 2017 at 22:15, Brice PARENT <contact at brice.xyz> wrote:
>> Also, I did find the decorator proposal intriguing, though have to say I
>> probably wouldn't bother to use it unless it were a builtin or I had a dozen
>> parameters to deal with.
>>
> If you *need* a shorter solution, even though I'm not entirely sure there's
> a real need for it, it may be easier to get something like this (I think, as
> there is no new operator involved) :

It seems to me that the number of people for whom both of the following hold:

1. Writing out the assignments "longhand" is an unacceptable burden.
2. Using a decorator (which can be written directly in your project,
doesn't even need to be an external dependency) is unacceptable.

is likely to be small enough that it's not a compelling argument for
adding new syntax. Add to that the fact that these people would be
arguing "I want the ability to avoid writing out the assignments, but
I don't want that capability enough to use a decorator" - which hardly
says that a syntax change is vital - and it's difficult to see this
proposal getting accepted.

Paul

From p.f.moore at gmail.com  Tue Apr 25 18:09:10 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 25 Apr 2017 23:09:10 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <5170eee5-7ba0-0413-15cd-d539713c4d60@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <5170eee5-7ba0-0413-15cd-d539713c4d60@mgmiller.net>
Message-ID: <CACac1F_8H3qwPDarY2vqrvFmZ6N4bSW4iOZQgwPqBXXF0ur0Pg@mail.gmail.com>

On 25 April 2017 at 22:27, Mike Miller <python-ideas at mgmiller.net> wrote:
> On 2017-04-25 14:15, Brice PARENT wrote:
>>
>> But, any of these proposals, mine and yours, if you really need this to
>> shorten
>> the code writing time or vertical space only, is not a better idea than to
>> propose a macro to your IDE editor, or a pull request if it's open source.
>> Such
>> a macro would generate automatically those long-to-write lines, and maybe
>> implement some code folding if vertical space is an issue.
>
>
> I'm personally not bothered by vertical space, but rather typing the same
> thing three times with self's and ='s.  That's more boilerplate than Java.
> ;)

Well,

class MyClass:
    @auto_args
    def __init__(self, a, b, c=None):
        pass

seems ideal for you then (using the auto_args decorator from
https://twitter.com/judy2k/status/854330478068977664 - a 13-line
function including the import of inspect and some comments).

(Arguing for auto_args to be in the stdlib may be a better option than
arguing for new syntax, BTW...)

Paul

From python at lucidity.plus.com  Tue Apr 25 18:14:08 2017
From: python at lucidity.plus.com (Erik)
Date: Tue, 25 Apr 2017 23:14:08 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPTjJmrorg68=J33jkRtOhKiVmwbXhFb_9uL-n2m01T4O_k8bw@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAPTjJmrorg68=J33jkRtOhKiVmwbXhFb_9uL-n2m01T4O_k8bw@mail.gmail.com>
Message-ID: <93e6c872-e478-3c81-41a3-32d0ae3f0b33@lucidity.plus.com>

On 25/04/17 02:15, Chris Angelico wrote:
> Bikeshedding: Your example looks a lot more like tuple assignment than
> multiple assignment.

Well, originally, I thought it was just the spelling-the-same-name-twice 
thing that irritated me and I was just going to suggest a single 
assignment version like:

   self .= foo
   self .= bar

Then I thought that this is similar to importing (referencing an object 
from one namespace in another under the same name). In that scenario, 
instead of:

   from other import foo
   from other import bar

we have:

   from other import foo, bar

That's where the comma-separated idea came from, and I understand it 
looks like a tuple (which is why I explicitly mentioned that) but it 
does in the import syntax too ;)

The single argument version (though it doesn't help with vertical space) 
still reads better to me for the same reason that augmented assignment 
is clearer - there is no need to mentally parse that the same name is 
being used on both sides of the assignment because it's only spelled once.

> self .= foo .= bar .= baz .= spam .= ham

Thanks for being the only person so far to understand that I don't 
necessarily want to bind ALL of the __init__ parameters to the object, 
just the ones I explicitly reference, but I'm not convinced by this 
suggestion. In chained assignment the thing on the RHS is bound to each 
name to the left of it and that is really not happening here.

> The trouble is that this syntax is really only going to be used inside
> __init__.

Even if that was true, who ever writes one of those? :D

E.

From python at lucidity.plus.com  Tue Apr 25 18:30:01 2017
From: python at lucidity.plus.com (Erik)
Date: Tue, 25 Apr 2017 23:30:01 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
Message-ID: <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>

On 25/04/17 23:05, Paul Moore wrote:
> 1. Writing out the assignments "longhand" is an unacceptable burden.

There are reasons why augmented assignment was implemented. One of them 
was to make the code easier to read:

   foil = foil + 1
   foil = foi1 + 1
   foil += 1

Should one be silly enough to have a "foil" and "foi1" variable in 
scope, only one of those is clearly incrementing a variable without 
requiring a slightly harder look ;)

It's not about the time taken to type the line. It's about the clarity 
of what the line is expressing.

> 2. Using a decorator (which can be written directly in your project,
> doesn't even need to be an external dependency) is unacceptable.

All of the decorators (or other language tricks that modify the object's 
dict) suggested so far assume that ALL of the method's arguments are to 
be assigned. I do not want that. I want to be able to say:

def __init__(self, foo, bar, baz, spam):
   self .= foo, bar, spam
   self.baz = baz * 100

It's all still explicit inside the body of the method.

> Add to that the fact that these people would be
> arguing "I want the ability to avoid writing out the assignments, but
> I don't want that capability enough to use a decorator"

As I said above, it's not about the effort writing it out. It's about 
the effort (and accuracy) of reading the code after it has been written.

And as I also said above, decorators don't cut it anyway (at least not 
those proposed) because they blindly assign ALL of the arguments. I'm 
more than happy to hear of something that solves both of those problems 
without needing syntax changes though, as that means I can have it today ;)

E.


From python at lucidity.plus.com  Tue Apr 25 19:31:37 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 00:31:37 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
Message-ID: <7717f19b-a5a9-30a4-ba09-4153e864dea8@lucidity.plus.com>

On 25/04/17 22:15, Brice PARENT wrote:
> it may be easier to get something like this
> (I think, as there is no new operator involved) :

No new operator, but still a syntax change, so that doesn't help from 
that POV.

>
> def __init__(self, *args, **kwargs):
>       self.* = *args
>       self.** = **kwargs

What is "self.* = *args" supposed to do? For each positional argument, 
what name in the object is it bound to?

E.

From matt at getpattern.com  Tue Apr 25 19:51:51 2017
From: matt at getpattern.com (Matt Gilson)
Date: Tue, 25 Apr 2017 16:51:51 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <7717f19b-a5a9-30a4-ba09-4153e864dea8@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <7717f19b-a5a9-30a4-ba09-4153e864dea8@lucidity.plus.com>
Message-ID: <CAJCbRZb7mJd5UR8QFUzSzo27uW-g5fgf-9Jh9BZYXKVdkWT4=Q@mail.gmail.com>

On Tue, Apr 25, 2017 at 4:31 PM, Erik <python at lucidity.plus.com> wrote:

> On 25/04/17 22:15, Brice PARENT wrote:
>
>> it may be easier to get something like this
>> (I think, as there is no new operator involved) :
>>
>
> No new operator, but still a syntax change, so that doesn't help from that
> POV.
>
>
>> def __init__(self, *args, **kwargs):
>>       self.* = *args
>>       self.** = **kwargs
>>
>
> What is "self.* = *args" supposed to do? For each positional argument,
> what name in the object is it bound to?
>
> E.


For what it's worth, that's what I don't really like about the initially
proposed syntax too ...

    self .= foo, bar, baz

works OK, but:

    tup = foo, bar, baz
    self .= tup

doesn't work.  Admittedly, that could be part of the definition of this
feature, but it feels really unexpected to all of a sudden give my tuple a
temporary name and have the code behave in a dramatically different fashion.


>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 

Matt Gilson | Pattern

Software Engineer
getpattern.com
<https://www.getpattern.com?utm_source=email&utm_medium=email&utm_campaign=signature-matt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/47ce63ab/attachment.html>

From rymg19 at gmail.com  Tue Apr 25 20:05:40 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 25 Apr 2017 19:05:40 -0500
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>

FWIW I always liked
Dart's/Ruby's/Crystal's/(Coffee|Moon)Script's/WhateverElse's style:


class Cls {
  Cls(this.a);  // IT'S MAGIC
}


but the Python equivalent is admittedly weirder:


def ___init__(self, self.attr):


partly because, it'd have to work on pretty much any other variable name,
yet there's no other real use case outside of methods.


--
Ryan (????)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://refi64.com

On Apr 24, 2017 8:08 PM, "Erik" <python at lucidity.plus.com> wrote:

> Hi. I suspect that this may have been discussed to death at some point in
> the past, but I've done some searching and I didn't come up with much.
> Apologies if I'm rehashing an old argument ;)
>
> I often find myself writing __init__ methods of the form:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
>
> This seems a little wordy and uses a lot of vertical space on the screen.
> Occasionally, I have considered something like:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo, self.bar, self.baz, self.spam, self.ham = \
>      foo, bar, baz, spam, ham
>
> ... just to make it a bit more compact - though in practice, I'd probably
> not do that with a list quite that long ... two or three items at most:
>
> def __init__(self, foo, bar, baz):
>    self.foo, self.bar, self.baz = foo, bar, baz
>
> When I do that I'm torn because I know it has a runtime impact to create
> and unpack the implicit tuples and I'm also introducing a style asymmetry
> in my code just because of the number of parameters a method happens to
> have.
>
> So why not have an augmented assignment operator for object attributes? It
> addresses one of the same broad issues that the other augmented assignment
> operators were introduced for (that of repeatedly spelling names).
>
> The suggestion therefore is:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self .= foo, bar, baz, spam, ham
>
> This is purely syntactic sugar for the original example:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
>
> ... so if any of the attributes have setters, then they are called as
> usual. It's purely a syntactic shorthand. Any token which is not suitable
> on the RHS of the dot in a standard "obj.attr =" assignment is a syntax
> error (no "self .= 1").
>
> The comma-separators in the example are not creating a tuple object, they
> would work at the same level in the parser as the import statement's
> comma-separated lists - in the same way that "from pkg import a, b, c" is
> the same as saying:
>
> import pkg
> a = pkg.a
> b = pkg.b
> c = pkg.c
>
> ... "self .= a, b, c" is the same as writing:
>
> self.a = a
> self.b = b
> self.c = c
>
> E.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/8201fc95/attachment-0001.html>

From rosuav at gmail.com  Tue Apr 25 20:12:55 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 26 Apr 2017 10:12:55 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
Message-ID: <CAPTjJmq1hfi=HaS4LUKTvaMNQ8gkeDHZRnX93JMzn40FJWtPNg@mail.gmail.com>

On Wed, Apr 26, 2017 at 10:05 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> FWIW I always liked
> Dart's/Ruby's/Crystal's/(Coffee|Moon)Script's/WhateverElse's style:
>
>
> class Cls {
>   Cls(this.a);  // IT'S MAGIC
> }
>
>
> but the Python equivalent is admittedly weirder:
>
>
> def ___init__(self, self.attr):

In a sense, what you have is this (modulo keyword arguments):

def __init__(*args):
    self, self.attr = args

which is perfectly legal, albeit weird. So it needn't actually be
magic per se, just a change in the definition of an argument name
(from NAME to whatever is legal as an assignment target).

I don't think it's particularly useful, though.

ChrisA

From njs at pobox.com  Tue Apr 25 20:39:16 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 25 Apr 2017 17:39:16 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
Message-ID: <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>

On Tue, Apr 25, 2017 at 3:30 PM, Erik <python at lucidity.plus.com> wrote:
> On 25/04/17 23:05, Paul Moore wrote:
>>
>> 1. Writing out the assignments "longhand" is an unacceptable burden.
>
>
> There are reasons why augmented assignment was implemented. One of them was
> to make the code easier to read:
>
>   foil = foil + 1
>   foil = foi1 + 1
>   foil += 1
>
> Should one be silly enough to have a "foil" and "foi1" variable in scope,
> only one of those is clearly incrementing a variable without requiring a
> slightly harder look ;)

But if this were the only argument for +=, then I'm not sure it would
have ever been added :-).

The really compelling cases for += are things like:

foo.some_attr[get_the_key(kwarg=something)] =
foo.some_attr[get_the_key(kwarg=something)] + 1

vs

foo.some_attr[get_the_key(kwarg=something)] += 1

where the latter is *much* more readable, and it only calls
get_the_key once, which (depending on what it does) may be both a
substantial efficiency win and also guarantees this is actually an
increment ? we don't have to read get_the_key's source code first to
figure out if the two calls actually return the same value.

Another example:

arr = arr + 1
arr += 1

If 'arr' is a numpy array then these actually do very different
things: the first one allocates a new array and then rebinds the name
'arr' to the copy; the second modifies the array object in place. The
former is usually what you want, but without augmented assignment you
have to remember to do awkward things like 'arr[...] = arr + 1'. And
it gets worse: since 'arr[...] = arr + 1' has to make a copy, it's
about twice as slow as 'arr += 1'. The real equivalent of 'arr += 1'
is 'np.add(arr, 1, out=arr)', which is *really* bad.

Are there any similar arguments for .=?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From apalala at gmail.com  Tue Apr 25 21:57:06 2017
From: apalala at gmail.com (=?UTF-8?Q?Juancarlo_A=C3=B1ez?=)
Date: Tue, 25 Apr 2017 21:57:06 -0400
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F_8H3qwPDarY2vqrvFmZ6N4bSW4iOZQgwPqBXXF0ur0Pg@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <5170eee5-7ba0-0413-15cd-d539713c4d60@mgmiller.net>
 <CACac1F_8H3qwPDarY2vqrvFmZ6N4bSW4iOZQgwPqBXXF0ur0Pg@mail.gmail.com>
Message-ID: <CAN1YFWtLV8L_mAPUQPrsP9DCOMYQVj=Dhb01-=wn0uoYTgPv2Q@mail.gmail.com>

On Tue, Apr 25, 2017 at 6:09 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> (Arguing for auto_args to be in the stdlib may be a better option than
> arguing for new syntax, BTW...)
>

Having such a decorator in the stdlib would allow IDEs and syntax
highlighters to know what's going on.


-- 
Juancarlo *A?ez*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170425/81fd8dd1/attachment.html>

From p.f.moore at gmail.com  Wed Apr 26 03:59:59 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Apr 2017 08:59:59 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
Message-ID: <CACac1F8rRatCtAxbi+nopSgZOq0yXxwP++2Zi2H7CzgO0eFnLQ@mail.gmail.com>

On 25 April 2017 at 23:30, Erik <python at lucidity.plus.com> wrote:
> As I said above, it's not about the effort writing it out. It's about the
> effort (and accuracy) of reading the code after it has been written.

Well, personally I find all of the syntax proposals relatively
unreadable. So that's definitely a matter of opinion. And the
"explicit is better than implicit" principle argues for the longhand
form.

As has been pointed out, the case for += is more about incremented
complex computed cases than simply avoiding repeating a variable name
(although some people find that simpler case helpful, too - I'm
personally ambivalent).

> And as I also said above, decorators don't cut it anyway (at least not those
> proposed) because they blindly assign ALL of the arguments. I'm more than
> happy to hear of something that solves both of those problems without
> needing syntax changes though, as that means I can have it today ;)

That's something that wasn't clear from your original post, but you're
correct. It should be possible to modify the decorator to take a list
of the variable names you want to assign, but I suspect you won't like
that - it does reduce the number of times you have to name the
variables from 3 to 2, the same as your proposal, though.

class MyClass:
    @auto_args('a', 'b')
    def __init__(self, a, b, c=None):
        pass

Paul

From contact at brice.xyz  Wed Apr 26 07:12:14 2017
From: contact at brice.xyz (Brice PARENT)
Date: Wed, 26 Apr 2017 13:12:14 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>

Why not simply do this :

class MyClass:
     def _set_multiple(self, **kwargs):
         for key, value in kwargs.items():
             setattr(self, key, value)

     def __init__(self, a, b, c):
         self._set_multiple(a=a, b=b, c=c)

-> No new syntax, no inspection, simple to read, parse and understand, 
and explicit.

And you could directly do some simple cleanings or not define a 
variable, like :

class MyClass:
     def _set_multiple(self, **kwargs):
         for key, value in kwargs.items():
             setattr(self, key, value)

     def __init__(self, a, b, c):
         self._set_multiple(a=a * 2, b=clean_field(b))

- Brice

Le 25/04/17 ? 03:08, Erik a ?crit :
> Hi. I suspect that this may have been discussed to death at some point 
> in the past, but I've done some searching and I didn't come up with 
> much. Apologies if I'm rehashing an old argument ;)
>
> I often find myself writing __init__ methods of the form:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
>
> This seems a little wordy and uses a lot of vertical space on the 
> screen. Occasionally, I have considered something like:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo, self.bar, self.baz, self.spam, self.ham = \
>      foo, bar, baz, spam, ham
>
> ... just to make it a bit more compact - though in practice, I'd 
> probably not do that with a list quite that long ... two or three 
> items at most:
>
> def __init__(self, foo, bar, baz):
>    self.foo, self.bar, self.baz = foo, bar, baz
>
> When I do that I'm torn because I know it has a runtime impact to 
> create and unpack the implicit tuples and I'm also introducing a style 
> asymmetry in my code just because of the number of parameters a method 
> happens to have.
>
> So why not have an augmented assignment operator for object 
> attributes? It addresses one of the same broad issues that the other 
> augmented assignment operators were introduced for (that of repeatedly 
> spelling names).
>
> The suggestion therefore is:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self .= foo, bar, baz, spam, ham
>
> This is purely syntactic sugar for the original example:
>
> def __init__(self, foo, bar, baz, spam, ham):
>   self.foo = foo
>   self.bar = bar
>   self.baz = baz
>   self.spam = spam
>   self.ham = ham
>
> ... so if any of the attributes have setters, then they are called as 
> usual. It's purely a syntactic shorthand. Any token which is not 
> suitable on the RHS of the dot in a standard "obj.attr =" assignment 
> is a syntax error (no "self .= 1").
>
> The comma-separators in the example are not creating a tuple object, 
> they would work at the same level in the parser as the import 
> statement's comma-separated lists - in the same way that "from pkg 
> import a, b, c" is the same as saying:
>
> import pkg
> a = pkg.a
> b = pkg.b
> c = pkg.c
>
> ... "self .= a, b, c" is the same as writing:
>
> self.a = a
> self.b = b
> self.c = c
>
> E.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From phd at phdru.name  Wed Apr 26 07:32:31 2017
From: phd at phdru.name (Oleg Broytman)
Date: Wed, 26 Apr 2017 13:32:31 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
Message-ID: <20170426113231.GA31536@phdru.name>

On Wed, Apr 26, 2017 at 01:12:14PM +0200, Brice PARENT <contact at brice.xyz> wrote:
>     def _set_multiple(self, **kwargs):
>         for key, value in kwargs.items():
>             setattr(self, key, value)

    self.__dict__.update(kwargs)

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From tritium-list at sdamon.com  Wed Apr 26 08:02:39 2017
From: tritium-list at sdamon.com (tritium-list at sdamon.com)
Date: Wed, 26 Apr 2017 08:02:39 -0400
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170426113231.GA31536@phdru.name>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
 <20170426113231.GA31536@phdru.name>
Message-ID: <02ad01d2be85$00c935c0$025ba140$@hotmail.com>



> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Oleg Broytman
> Sent: Wednesday, April 26, 2017 7:33 AM
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Augmented assignment syntax for objects.
> 
> On Wed, Apr 26, 2017 at 01:12:14PM +0200, Brice PARENT
> <contact at brice.xyz> wrote:
> >     def _set_multiple(self, **kwargs):
> >         for key, value in kwargs.items():
> >             setattr(self, key, value)
> 
>     self.__dict__.update(kwargs)

Touching __dict__ feels dirty to me.

> Oleg.
> --
>      Oleg Broytman            http://phdru.name/            phd at phdru.name
>            Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


From jsbueno at python.org.br  Wed Apr 26 08:19:34 2017
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Wed, 26 Apr 2017 09:19:34 -0300
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
Message-ID: <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>

On 25 April 2017 at 19:30, Erik <python at lucidity.plus.com> wrote:
> And as I also said above, decorators don't cut it anyway (at least not those
> proposed) because they blindly assign ALL of the arguments. I'm more than
> happy to hear of something that solves both of those problems without
> needing syntax changes though, as that means I can have it today ;)

Sorry -  a decorator won't "blindly assign all argments" - it will do
that just if it is written to do so.

It is perfectly feasible to have a decorator to which you can simply
pass a list of not wanted auto-parameters, or that will just
auto-assign parameters that have been declared on the class body, or
even inspect the function signature and check for special annotations
- there are tens of ways we could spec a list of exceptions to such a
decorator, and still avouid typing three times each name.

That said, I am all in favor of the addition of such a decorator to the stdlib.

   js
  -><-

From miron at python.it  Wed Apr 26 08:16:09 2017
From: miron at python.it (Carlo Miron)
Date: Wed, 26 Apr 2017 14:16:09 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <02ad01d2be85$00c935c0$025ba140$@hotmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
 <20170426113231.GA31536@phdru.name>
 <02ad01d2be85$00c935c0$025ba140$@hotmail.com>
Message-ID: <CAFp98etTnvya5f-R8T1W4rVXQmExtV4CAL+GW_saxpPyzv4XvA@mail.gmail.com>

On Wed, Apr 26, 2017 at 2:02 PM,  <tritium-list at sdamon.com> wrote:

>> On Wed, Apr 26, 2017 at 01:12:14PM +0200, Brice PARENT
>> <contact at brice.xyz> wrote:
>> >     def _set_multiple(self, **kwargs):
>> >         for key, value in kwargs.items():
>> >             setattr(self, key, value)
>>
>>     self.__dict__.update(kwargs)
>
> Touching __dict__ feels dirty to me.

vars(self).update(kwargs)?

?

-- 
|:**THE ?-WARE LICENSE** *(Revision ?)*:
| <miron@?.it> wrote this mail. As long as you retain this
| notice you can do whatever you want with this stuff.
| If we meet some day, and you think this stuff is worth it,
| you can buy me a ? in return. ??

From python at lucidity.plus.com  Wed Apr 26 10:59:15 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 15:59:15 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
Message-ID: <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>

On 26/04/17 13:19, Joao S. O. Bueno wrote:
> On 25 April 2017 at 19:30, Erik <python at lucidity.plus.com> wrote:
>> decorators don't cut it anyway (at least not those
>> proposed) because they blindly assign ALL of the arguments. I'm more than
>> happy to hear of something that solves both of those problems without
>> needing syntax changes though, as that means I can have it today ;)
>
> Sorry -  a decorator won't "blindly assign all argments" - it will do
> that just if it is written to do so.

Right, and the three or four variants suggested (and the 
vars(self).update() suggestion) all do exactly that. I was talking about 
the specific responses (though I can see my language is vague).

[FWIW I've been using Python the whole time that decorators have existed 
and I've yet to need to write one - I've _used_ some non-parameterized 
ones though - so I guess I'd forgotten that they can take parameters]

E.

From prometheus235 at gmail.com  Wed Apr 26 11:10:45 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Wed, 26 Apr 2017 10:10:45 -0500
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
 <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>
Message-ID: <CAHkxivdR11wuftmC=sN+__G0KkPuDCjJmE8Jw4BKP7B78yPkFw@mail.gmail.com>

I was wondering that if there are so many arguments to a function that it
*looks* ugly, that it might just *be* ugly.

For one, too many required arguments to a function (constructor, whatever)
is already strange. Binding them as attributes of the object, unmodified in
a constructor also seems to be rare. Matplotlib, for example, might take 8
(optional keyword) arguments for a Figure (
https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/figure.py#L268),
but then at a bare minimum, the vertical space required is *doubled* by the
need for pesky if-blocks and logic. Self-assignment is a pittance.

Are there any better examples of where adding new syntax could help
readability for programmers, both new and old?

On Wed, Apr 26, 2017 at 9:59 AM, Erik <python at lucidity.plus.com> wrote:

> On 26/04/17 13:19, Joao S. O. Bueno wrote:
>
>> On 25 April 2017 at 19:30, Erik <python at lucidity.plus.com> wrote:
>>
>>> decorators don't cut it anyway (at least not those
>>> proposed) because they blindly assign ALL of the arguments. I'm more than
>>> happy to hear of something that solves both of those problems without
>>> needing syntax changes though, as that means I can have it today ;)
>>>
>>
>> Sorry -  a decorator won't "blindly assign all argments" - it will do
>> that just if it is written to do so.
>>
>
> Right, and the three or four variants suggested (and the
> vars(self).update() suggestion) all do exactly that. I was talking about
> the specific responses (though I can see my language is vague).
>
> [FWIW I've been using Python the whole time that decorators have existed
> and I've yet to need to write one - I've _used_ some non-parameterized ones
> though - so I guess I'd forgotten that they can take parameters]
>
> E.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170426/ed56f919/attachment-0001.html>

From python at lucidity.plus.com  Wed Apr 26 11:17:41 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 16:17:41 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F8rRatCtAxbi+nopSgZOq0yXxwP++2Zi2H7CzgO0eFnLQ@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CACac1F8rRatCtAxbi+nopSgZOq0yXxwP++2Zi2H7CzgO0eFnLQ@mail.gmail.com>
Message-ID: <05b6e2cb-fb2a-d775-a8c5-83bd5312db2d@lucidity.plus.com>

On 26/04/17 08:59, Paul Moore wrote:
> It should be possible to modify the decorator to take a list
> of the variable names you want to assign, but I suspect you won't like
> that

Now you're second-guessing me.

 > class MyClass:
 >     @auto_args('a', 'b')
 >     def __init__(self, a, b, c=None):
 >         pass

I had forgotten that decorators could take parameters. Something like 
that pretty much ticks the boxes for me.

I'd _prefer_ something that sits inside the method body rather than just 
outside it, and I'd probably _prefer_ something that wasn't quite so 
heavyweight at runtime (which may be an irrational concern on my part 
;)), but those aren't deal breakers, depending on the project - and the 
vast majority of what I do in Python is short-lived one-off projects and 
rapid prototyping for later implementation in another language, so I do 
seem to be fleshing out a set of classes from scratch and writing a 
bunch of __init__ methods far more of the time than people with 
long-lived projects would do. Perhaps that's why it irritates me more 
than it does some others ;)

E.

From p.f.moore at gmail.com  Wed Apr 26 11:22:01 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Apr 2017 16:22:01 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <05b6e2cb-fb2a-d775-a8c5-83bd5312db2d@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CACac1F8rRatCtAxbi+nopSgZOq0yXxwP++2Zi2H7CzgO0eFnLQ@mail.gmail.com>
 <05b6e2cb-fb2a-d775-a8c5-83bd5312db2d@lucidity.plus.com>
Message-ID: <CACac1F9HbqiidT5Rb8AdV_krXtVN0k7mp0EH7wKuGaA2ihUtwg@mail.gmail.com>

On 26 April 2017 at 16:17, Erik <python at lucidity.plus.com> wrote:
> On 26/04/17 08:59, Paul Moore wrote:
>>
>> It should be possible to modify the decorator to take a list
>> of the variable names you want to assign, but I suspect you won't like
>> that
>
>
> Now you're second-guessing me.

Sorry :-)

>> class MyClass:
>>     @auto_args('a', 'b')
>>     def __init__(self, a, b, c=None):
>>         pass
>
> I had forgotten that decorators could take parameters. Something like that
> pretty much ticks the boxes for me.

Cool. Glad you liked the idea.

Paul

From python-ideas at mgmiller.net  Wed Apr 26 13:42:03 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Wed, 26 Apr 2017 10:42:03 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
Message-ID: <ac3c632c-b4f1-8fd8-f3f5-2ae9511f8b3b@mgmiller.net>


On 2017-04-25 15:30, Erik wrote:

> All of the decorators (or other language tricks that modify the object's dict)
> suggested so far assume that ALL of the method's arguments are to be assigned. I
> do not want that. I want to be able to say:
>
> def __init__(self, foo, bar, baz, spam):
>   self .= foo, bar, spam
>   self.baz = baz * 100


I don't see ALL being set a big problem, and less work than typing several of 
them out again.

Just because all args get set at once, doesn't mean the init stops there and 
attributes can't be further modified or deleted.  In your example above, you 
modify baz afterward as an argument that might need to be changed, as they often do.

If this doesn't give enough control, the manual way is always available.
Setting them all makes simple things *very* simple, and the medium and 
complicated still possible.

From python-ideas at mgmiller.net  Wed Apr 26 13:46:25 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Wed, 26 Apr 2017 10:46:25 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
Message-ID: <626ce4d3-0407-70af-44ba-d2ace007eb72@mgmiller.net>


On 2017-04-26 04:12, Brice PARENT wrote:
> Why not simply do this :
>
> class MyClass:
>     def _set_multiple(self, **kwargs):
>         for key, value in kwargs.items():
>             setattr(self, key, value)
>
>     def __init__(self, a, b, c):
>         self._set_multiple(a=a, b=b, c=c)

If the goal is to not have to type out argument names three times (in DRY 
fashion), this doesn't quite fit the bill.

-Mike


From apalala at gmail.com  Wed Apr 26 13:59:44 2017
From: apalala at gmail.com (=?UTF-8?Q?Juancarlo_A=C3=B1ez?=)
Date: Wed, 26 Apr 2017 13:59:44 -0400
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <05b6e2cb-fb2a-d775-a8c5-83bd5312db2d@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CACac1F8rRatCtAxbi+nopSgZOq0yXxwP++2Zi2H7CzgO0eFnLQ@mail.gmail.com>
 <05b6e2cb-fb2a-d775-a8c5-83bd5312db2d@lucidity.plus.com>
Message-ID: <CAN1YFWuRF-tt2FhUTuKzzkW7ZGZJoO=tGDM6m86=sYGb08UgWw@mail.gmail.com>

On Wed, Apr 26, 2017 at 11:17 AM, Erik <python at lucidity.plus.com> wrote:

> I had forgotten that decorators could take parameters. Something like that
> pretty much ticks the boxes for me.
>

There are decorators with "include"  and "included" in this SO Q&A:

    http://stackoverflow.com/q/3652851/545637


> I'd _prefer_ something that sits inside the method body rather than just
> outside it, and I'd probably _prefer_ something that wasn't quite so
> heavyweight at runtime (which may be an irrational concern on my part ;)),
>

The same strategies applied by the decorators may be applied by a by a
function called from within __init__.


> but those aren't deal breakers, depending on the project - and the vast
> majority of what I do in Python is short-lived one-off projects and rapid
> prototyping for later implementation in another language, so I do seem to
> be fleshing out a set of classes from scratch and writing a bunch of
> __init__ methods far more of the time than people with long-lived projects
> would do. Perhaps that's why it irritates me more than it does some others
> ;)
>

For the cases I've found in which classes define several attributes that
are initialized in the constructor I think that a library like
https://github.com/python-attrs/attrs does what's needed. The downside is
that code-writing tools (like IDEs) don't understand what's going on under
the hood.

-- 
Juancarlo *A?ez*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170426/d2507a84/attachment.html>

From python at mrabarnett.plus.com  Wed Apr 26 14:11:31 2017
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 26 Apr 2017 19:11:31 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <626ce4d3-0407-70af-44ba-d2ace007eb72@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
 <626ce4d3-0407-70af-44ba-d2ace007eb72@mgmiller.net>
Message-ID: <c13ca72d-0ad2-a508-69cb-4d7f25674ac4@mrabarnett.plus.com>

On 2017-04-26 18:46, Mike Miller wrote:
> 
> On 2017-04-26 04:12, Brice PARENT wrote:
>> Why not simply do this :
>>
>> class MyClass:
>>     def _set_multiple(self, **kwargs):
>>         for key, value in kwargs.items():
>>             setattr(self, key, value)
>>
>>     def __init__(self, a, b, c):
>>         self._set_multiple(a=a, b=b, c=c)
> 
> If the goal is to not have to type out argument names three times (in DRY
> fashion), this doesn't quite fit the bill.
> 
An alternative is to use a special comment, for example:

class MyClass:
     def __init__(self, a, b, c):
         #> a, b, c

And then use, say, a Python script to expand it to:

class MyClass:
     def __init__(self, a, b, c):
         #> a, b, c
         self.a = a
         self.b = b
         self.c = c
         #<

The Python script could be called from the editor.

If you want to add or remove an argument, edit the comment and then run 
the script again. (Hopefully it'll be smart enough to make the necessary 
changes).

From python-ideas at mgmiller.net  Wed Apr 26 14:15:18 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Wed, 26 Apr 2017 11:15:18 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
Message-ID: <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>


On 2017-04-25 15:05, Paul Moore wrote:
> It seems to me that the number of people for whom both of the following hold:
>
> 1. Writing out the assignments "longhand" is an unacceptable burden.
> 2. Using a decorator (which can be written directly in your project,
> doesn't even need to be an external dependency) is unacceptable.
>
> is likely to be small enough that it's not a compelling argument for
> adding new syntax. Add to that the fact that these people would be
> arguing "I want the ability to avoid writing out the assignments, but
> I don't want that capability enough to use a decorator" - which hardly
> says that a syntax change is vital - and it's difficult to see this
> proposal getting accepted.

While the word unacceptable is probably a bit too strong, this is a good 
insight.  We have a "middling" difficulty here.  Not the end of the world, but 
an annoyance, as the existence of the attrs library demonstrates.  Kivy's 
properties solve a similar problem (though a bit different scope).  I've 
overheard Java snobs sneer at Python's init so it is a well known issue.

If there is going to be a solution, I submit it needs to be much easier to use 
and to read.  Making it a little easier is not good enough, not enough reason to 
change.  (Believe you mentioned a 3 to 2 reduction elsewhere in the thread.)  Meh.

If I have install a module in pip and go back to the top of the file and write 
an import line and then come back and write out a decorator it's not going to be 
worth it.  Currently, I use editor snippets to ease authoring of init, but 
readability and DRY are still concerns afterward.

As the new syntax ideas piggyback on existing syntax, it doesn't feel like that 
its a complete impossibility to have this solved.  Could be another "fixed 
papercut" to drive Py3 adoption.  Taken individually not a big deal but they add up.

From contact at brice.xyz  Wed Apr 26 14:16:31 2017
From: contact at brice.xyz (Brice PARENT)
Date: Wed, 26 Apr 2017 20:16:31 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <626ce4d3-0407-70af-44ba-d2ace007eb72@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
 <626ce4d3-0407-70af-44ba-d2ace007eb72@mgmiller.net>
Message-ID: <5949a515-2bc9-d7c7-fb29-9c454b38d5ee@brice.xyz>

Le 26/04/17 ? 19:46, Mike Miller a ?crit :
>
> On 2017-04-26 04:12, Brice PARENT wrote:
>> Why not simply do this :
>>
>> class MyClass:
>>     def _set_multiple(self, **kwargs):
>>         for key, value in kwargs.items():
>>             setattr(self, key, value)
>>
>>     def __init__(self, a, b, c):
>>         self._set_multiple(a=a, b=b, c=c)
>
> If the goal is to not have to type out argument names three times (in 
> DRY fashion), this doesn't quite fit the bill.
>
> -Mike
>
You still save some typing (no need for /self./ for every assignment), 
and a lot of vertical space.

Also, it allows you to use any dict (like **kwargs) as well as doing 
very common things like :

class MyClass:
     def _set_multiple(self, **kwargs):
         for key, value in kwargs.items():
             setattr(self, key, value)

     def __init__(self, a, b, c):
         self._set_multiple(a=a, _b=b)

which is equivalent to the following really common pattern :

class MyClass:
     def __init__(self, a, b, c):
         self.a = a
         self._b = b

(well, I didn't try it, but it should work)
This means your public API (the signature of the constructor here) has 
no incidence on the way the class works on the inside. Meaning an 
attribute can become a property, which is something that occurs from 
time to time, and you just need to update your constructor 
(self._set_multiple(a=a, b=b) becomes self._set_multiple(a=a, _b=b)) and 
you're done.
And if at some point your community doesn't think "a" is a good name for 
your variable, you can update it to something every user will understand 
without any significant change to your class 
(self._set_multiple(a=way_better, _b=b)).

-Brice

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170426/9fd3552e/attachment-0001.html>

From python at lucidity.plus.com  Wed Apr 26 15:41:14 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 20:41:14 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <ac3c632c-b4f1-8fd8-f3f5-2ae9511f8b3b@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <ac3c632c-b4f1-8fd8-f3f5-2ae9511f8b3b@mgmiller.net>
Message-ID: <251586da-3b70-c05c-12c7-210a7bea7494@lucidity.plus.com>

On 26/04/17 18:42, Mike Miller wrote:
> I want to be able to say:
>>
>> def __init__(self, foo, bar, baz, spam):
>>   self .= foo, bar, spam
>>   self.baz = baz * 100
>
>
> I don't see ALL being set a big problem, and less work than typing
> several of them out again.

Because, some of the parameters might be things that are just passed to 
another constructor to create an object that is then referenced by the 
object being created.

If one doesn't want the object's namespace to be polluted by that stuff 
(which may be large and also now can't be garbage collected while the 
object is alive) then a set of "del self.xxx" statements is required 
instead, so you've just replaced one problem with another ;)

I'd rather just explicitly say what I want to happen rather than have 
*everything* happen and then have to tidy that up instead ...

E.

From malaclypse2 at gmail.com  Wed Apr 26 15:54:22 2017
From: malaclypse2 at gmail.com (Jerry Hill)
Date: Wed, 26 Apr 2017 15:54:22 -0400
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
Message-ID: <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>

On Tue, Apr 25, 2017 at 8:05 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> def ___init__(self, self.attr):

I'm not a python developer, I'm just a developer that uses python.
That said, I really like this form.  It eliminates most of the
redundancy, while still being explicit.  It's true that you have to
repeat the 'self' name, but it feels like the right balance in my
mind.  It seems like anyone who's familiar with python function
definitions would immediately grasp what's going on.

-- 
Jerry

From python at lucidity.plus.com  Wed Apr 26 16:24:46 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 21:24:46 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAHkxivdR11wuftmC=sN+__G0KkPuDCjJmE8Jw4BKP7B78yPkFw@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
 <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>
 <CAHkxivdR11wuftmC=sN+__G0KkPuDCjJmE8Jw4BKP7B78yPkFw@mail.gmail.com>
Message-ID: <beee40f8-42de-24a3-00c6-54ee1725841c@lucidity.plus.com>

On 26/04/17 16:10, Nick Timkovich wrote:
> I was wondering that if there are so many arguments to a function that
> it *looks* ugly, that it might just *be* ugly.
>
> For one, too many required arguments to a function (constructor,
> whatever) is already strange. Binding them as attributes of the object,
> unmodified in a constructor also seems to be rare.

Yes, and perhaps it's more of a problem for me because of my 
possibly-atypical use of Python.

The background is that what I find myself doing a lot of for private 
projects is importing data from databases into a structured collection 
of objects and then grouping and analyzing the data in different ways 
before graphing the results.

So yes, I tend to have classes that accept their entire object state as 
parameters to the __init__ method (from the database values) and then 
any other methods in the class are generally to do with the subsequent 
analysis (including dunder methods for iteration, rendering and 
comparison etc).

E.

From rosuav at gmail.com  Wed Apr 26 16:50:58 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 27 Apr 2017 06:50:58 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <beee40f8-42de-24a3-00c6-54ee1725841c@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
 <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>
 <CAHkxivdR11wuftmC=sN+__G0KkPuDCjJmE8Jw4BKP7B78yPkFw@mail.gmail.com>
 <beee40f8-42de-24a3-00c6-54ee1725841c@lucidity.plus.com>
Message-ID: <CAPTjJmqU_4ogrNwQKtcLimCEyWTur+MK5mVPGf-jceAgFJm+fA@mail.gmail.com>

On Thu, Apr 27, 2017 at 6:24 AM, Erik <python at lucidity.plus.com> wrote:
> The background is that what I find myself doing a lot of for private
> projects is importing data from databases into a structured collection of
> objects and then grouping and analyzing the data in different ways before
> graphing the results.
>
> So yes, I tend to have classes that accept their entire object state as
> parameters to the __init__ method (from the database values) and then any
> other methods in the class are generally to do with the subsequent analysis
> (including dunder methods for iteration, rendering and comparison etc).

You may want to try designing your objects as namedtuples. That gives
you a lot of what you're looking for.

ChrisA

From python at lucidity.plus.com  Wed Apr 26 16:51:35 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 21:51:35 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>
Message-ID: <8470f746-2295-b0aa-b396-9b604de3b7b8@lucidity.plus.com>

On 26/04/17 01:39, Nathaniel Smith wrote:
[snip discussion of why current augmented assignment operators are 
better for other reasons]

> Are there any similar arguments for .=?

It doesn't make anything more efficient, however all of the suggestions 
of how to do it with current syntax (mostly decorators) _do_ make things 
less efficient.

So rather than a win/win as with current augmented assignment 
(compact/clearer code *and* potentially a performance improvement), it's 
now a tradeoff (wordy code *or* a performance reduction).

E.

From p.f.moore at gmail.com  Wed Apr 26 17:28:20 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Apr 2017 22:28:20 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <8470f746-2295-b0aa-b396-9b604de3b7b8@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>
 <8470f746-2295-b0aa-b396-9b604de3b7b8@lucidity.plus.com>
Message-ID: <CACac1F-BJL_5V4rTS7gvN_yfhgim+GaDfjvOD_=y8mZee_LsbQ@mail.gmail.com>

On 26 April 2017 at 21:51, Erik <python at lucidity.plus.com> wrote:
> It doesn't make anything more efficient, however all of the suggestions of
> how to do it with current syntax (mostly decorators) _do_ make things less
> efficient.

Is instance creation the performance bottleneck in your application?
That seems unusual. I guess it's possible if you're reading a large
database file and creating objects for each row. But in that case, as
Chris A says, you may be better with something like a named tuple. In
any case, optimising for performance is not what generic solutions are
good at, so it's not surprising that a generic decorator involves a
performance hit.

Paul

From python at lucidity.plus.com  Wed Apr 26 17:42:35 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 22:42:35 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F-BJL_5V4rTS7gvN_yfhgim+GaDfjvOD_=y8mZee_LsbQ@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>
 <8470f746-2295-b0aa-b396-9b604de3b7b8@lucidity.plus.com>
 <CACac1F-BJL_5V4rTS7gvN_yfhgim+GaDfjvOD_=y8mZee_LsbQ@mail.gmail.com>
Message-ID: <22107a28-ae44-dd37-b79f-dc4fc01e6c78@lucidity.plus.com>

On 26/04/17 22:28, Paul Moore wrote:
> On 26 April 2017 at 21:51, Erik <python at lucidity.plus.com> wrote:
>> It doesn't make anything more efficient, however all of the suggestions of
>> how to do it with current syntax (mostly decorators) _do_ make things less
>> efficient.
>
> Is instance creation the performance bottleneck in your application?

No, not at all. This discussion has split into two:

1) How can I personally achieve what I want for my own personal 
use-cases. This should really be on -list, and some variation of the 
decorator thing will probably suffice for me.

2) The original proposal, which does belong on -ideas and has to take 
into account the general case, not just my specific use-case.

The post you are responding to is part of (2), and hence reduced 
performance is a consideration.

Regards, E.

From breamoreboy at yahoo.co.uk  Wed Apr 26 17:50:01 2017
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 26 Apr 2017 22:50:01 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPTjJmqU_4ogrNwQKtcLimCEyWTur+MK5mVPGf-jceAgFJm+fA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
 <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>
 <CAHkxivdR11wuftmC=sN+__G0KkPuDCjJmE8Jw4BKP7B78yPkFw@mail.gmail.com>
 <beee40f8-42de-24a3-00c6-54ee1725841c@lucidity.plus.com>
 <CAPTjJmqU_4ogrNwQKtcLimCEyWTur+MK5mVPGf-jceAgFJm+fA@mail.gmail.com>
Message-ID: <odr4n7$10l$1@blaine.gmane.org>

On 26/04/2017 21:50, Chris Angelico wrote:
> On Thu, Apr 27, 2017 at 6:24 AM, Erik <python at lucidity.plus.com> wrote:
>> The background is that what I find myself doing a lot of for private
>> projects is importing data from databases into a structured collection of
>> objects and then grouping and analyzing the data in different ways before
>> graphing the results.
>>
>> So yes, I tend to have classes that accept their entire object state as
>> parameters to the __init__ method (from the database values) and then any
>> other methods in the class are generally to do with the subsequent analysis
>> (including dunder methods for iteration, rendering and comparison etc).
> 
> You may want to try designing your objects as namedtuples. That gives
> you a lot of what you're looking for.
> 
> ChrisA
> 

Something like https://docs.python.org/3/library/sqlite3.html#row-objects ?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From p.f.moore at gmail.com  Wed Apr 26 18:28:22 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 26 Apr 2017 23:28:22 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <22107a28-ae44-dd37-b79f-dc4fc01e6c78@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>
 <8470f746-2295-b0aa-b396-9b604de3b7b8@lucidity.plus.com>
 <CACac1F-BJL_5V4rTS7gvN_yfhgim+GaDfjvOD_=y8mZee_LsbQ@mail.gmail.com>
 <22107a28-ae44-dd37-b79f-dc4fc01e6c78@lucidity.plus.com>
Message-ID: <CACac1F-wo8n3FzBtH7Q14Gz5caBfq=dMAZYSN3kAm7HV=8h_dw@mail.gmail.com>

On 26 April 2017 at 22:42, Erik <python at lucidity.plus.com> wrote:
> 2) The original proposal, which does belong on -ideas and has to take into
> account the general case, not just my specific use-case.
>
> The post you are responding to is part of (2), and hence reduced performance
> is a consideration.

Ah, OK. I'm discounting the original proposal, as there don't seem to
be sufficient (i.e., any :-)) actual use cases that aren't covered by
the decorator proposal(s). Or to put it another way, if the only
reason for the syntax proposal is performance then show me a case
where performance is so critical that it warrants a language change.

Sorry for not being clear myself.
Paul

From python at lucidity.plus.com  Wed Apr 26 18:29:19 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 23:29:19 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
Message-ID: <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>

On 26/04/17 19:15, Mike Miller wrote:
> As the new syntax ideas piggyback on existing syntax, it doesn't feel
> like that its a complete impossibility to have this solved.  Could be
> another "fixed papercut" to drive Py3 adoption.  Taken individually not
> a big deal but they add up.

*sigh* OK, this has occurred to me over the last couple of days but I 
didn't want to suggest it as I didn't want the discussion to fragment 
even more.

But, if we're going to bikeshed and there is some weight behind the idea 
that this "papercut" should be addressed, then given my previous 
comparisons with importing, what about having 'import' as an operator:

def __init__(self, a, b, c):
    self import a, b
    self.foo = c * 100

Also allows renaming:

def __init__(self, a, b, c):
    self import a, b, c as _c

Because people are conditioned to think the comma-separated values after 
"import" are not tuples, perhaps the use of import as an operator rides 
on that wave ...

(I do realise that blurring the lines between statements and operators 
like this is probably not going to work for technical reasons (and it 
just doesn't quite read correctly anyway), but now we're bikeshedding 
and who knows what someone else might come up with in response ...).

E.

From python at lucidity.plus.com  Wed Apr 26 18:39:53 2017
From: python at lucidity.plus.com (Erik)
Date: Wed, 26 Apr 2017 23:39:53 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F-wo8n3FzBtH7Q14Gz5caBfq=dMAZYSN3kAm7HV=8h_dw@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAPJVwBkUbSDCWxmtsrEux34HSYap=03UcV6v5exmmCFqGUe2sg@mail.gmail.com>
 <8470f746-2295-b0aa-b396-9b604de3b7b8@lucidity.plus.com>
 <CACac1F-BJL_5V4rTS7gvN_yfhgim+GaDfjvOD_=y8mZee_LsbQ@mail.gmail.com>
 <22107a28-ae44-dd37-b79f-dc4fc01e6c78@lucidity.plus.com>
 <CACac1F-wo8n3FzBtH7Q14Gz5caBfq=dMAZYSN3kAm7HV=8h_dw@mail.gmail.com>
Message-ID: <f7d8c3f0-2b81-f030-f28d-4cb0ab128594@lucidity.plus.com>

On 26/04/17 23:28, Paul Moore wrote:
> Or to put it another way, if the only
> reason for the syntax proposal is performance then show me a case
> where performance is so critical that it warrants a language change.

It's the other way around.

The proposal (arguably) makes the code clearer but does not impact 
performance (and is a syntax error today, so does not break existing code).

The suggestions (decorators etc) make the code (arguably) clearer today 
without a syntax change, but impact performance.

So, those who think the decorators make for clearer code have to choose 
between source code clarity and performance.

E.

From python-ideas at mgmiller.net  Wed Apr 26 18:57:44 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Wed, 26 Apr 2017 15:57:44 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
Message-ID: <701f3390-5f12-bf34-0874-8451d7f81add@mgmiller.net>

Yes, I like it too.  Removes the triple repetition, and has precedent in the 
other languages.


On 2017-04-26 12:54, Jerry Hill wrote:
> On Tue, Apr 25, 2017 at 8:05 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> def ___init__(self, self.attr):
>
> I'm not a python developer, I'm just a developer that uses python.
> That said, I really like this form.  It eliminates most of the
> redundancy, while still being explicit.  It's true that you have to
> repeat the 'self' name, but it feels like the right balance in my
> mind.

From apalala at gmail.com  Wed Apr 26 20:52:46 2017
From: apalala at gmail.com (=?UTF-8?Q?Juancarlo_A=C3=B1ez?=)
Date: Wed, 26 Apr 2017 20:52:46 -0400
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <701f3390-5f12-bf34-0874-8451d7f81add@mgmiller.net>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <701f3390-5f12-bf34-0874-8451d7f81add@mgmiller.net>
Message-ID: <CAN1YFWsFYGkWjN=Zn_xrBX97Jn8AYZaevcQzCz9nOaQ8dpSXhA@mail.gmail.com>

On Wed, Apr 26, 2017 at 6:57 PM, Mike Miller <python-ideas at mgmiller.net>
wrote:

> Yes, I like it too.  Removes the triple repetition, and has precedent in
> the other languages.


This discussion has been around how to deal with repetition in object
constructors, and the proposals have been a new assignment statement (patch
assignments?) or decorators to __init__ (patch init).

In my experience, what Python is lacking is a way to declare attributes
outside of the constructor. Take a look at how it's done in C#, Swisft, or
Go.

Object attributes outside of the constructor would solve things more
relevant than the vertical space used when assigning constructor parameters
to attributes. For example, multiple inheritance is well designed in
Python, except that it often requires constructors with no parameters,
which leads to objects with no default attributes.

The namespace after "class"  is taken by class attributes. There's no
namespace for default object attributes outside __init__. Solving that
would likely lead to simple and sensible solutions to the OPs query.

I don't think that a solution to the OPs query should be treated as a
No-No, because there's namedtuple, which is quirky, but solved lots of use
cases (the same goes for Enum).

IMHO, something like the attrs library solves lots of common use cases...
except that tool support is poor because attrs is not part of stdlib.

It is not about having to write a few more lines in an __init__
constructor. There are common and frequent use cases of "objects are mostly
data" that are partially solved in Python (mostly through namedtuple?).

Cheers!

-- 
Juancarlo *A?ez*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170426/3c063082/attachment-0001.html>

From python at mrabarnett.plus.com  Wed Apr 26 21:45:41 2017
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 27 Apr 2017 02:45:41 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
Message-ID: <f3f51c81-e0fa-e8c9-7ea0-3729369f714d@mrabarnett.plus.com>

On 2017-04-26 23:29, Erik wrote:
> On 26/04/17 19:15, Mike Miller wrote:
>> As the new syntax ideas piggyback on existing syntax, it doesn't feel
>> like that its a complete impossibility to have this solved.  Could be
>> another "fixed papercut" to drive Py3 adoption.  Taken individually not
>> a big deal but they add up.
> 
> *sigh* OK, this has occurred to me over the last couple of days but I
> didn't want to suggest it as I didn't want the discussion to fragment
> even more.
> 
> But, if we're going to bikeshed and there is some weight behind the idea
> that this "papercut" should be addressed, then given my previous
> comparisons with importing, what about having 'import' as an operator:
> 
> def __init__(self, a, b, c):
>      self import a, b
>      self.foo = c * 100
> 
> Also allows renaming:
> 
> def __init__(self, a, b, c):
>      self import a, b, c as _c
> 
> Because people are conditioned to think the comma-separated values after
> "import" are not tuples, perhaps the use of import as an operator rides
> on that wave ...
> 
> (I do realise that blurring the lines between statements and operators
> like this is probably not going to work for technical reasons (and it
> just doesn't quite read correctly anyway), but now we're bikeshedding
> and who knows what someone else might come up with in response ...).
> 
If we're going to bikeshed, then:

def __init__(self, a, b, c):
     for self import a, b
     self.foo = c * 100

and:

def __init__(self, a, b, c):
     for self import a, b, c as _c

From ncoghlan at gmail.com  Thu Apr 27 10:07:44 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Apr 2017 00:07:44 +1000
Subject: [Python-ideas] Binary arithmetic does not always call
 subclasses first
In-Reply-To: <CAP7+vJKcqfSQF6rrKu_3fNhOO4K=VA0t3Jd1ckksA=UGafxzXg@mail.gmail.com>
References: <CAEQ_TveT0fPRNrpP1UHjJPjMCrk8cSOe6N_FAvmWa-pWr9J8JQ@mail.gmail.com>
 <58FD93BD.8050908@canterbury.ac.nz>
 <CAEQ_TvdDxChc1+9+FFFHr=PawV5LG9m2y1fdjx-LJ=S0N77osw@mail.gmail.com>
 <odlmca$dql$1@blaine.gmane.org>
 <CAHkxivcO049ov_XgApic-unqHYmG2MNKw15LNXnzCKNrGiaK3A@mail.gmail.com>
 <CAEQ_Tvdt69UoeHBRiD=VEMecPEecccOxjdqCWDEteOfk8zFESA@mail.gmail.com>
 <CAP7+vJ+cGKiWG91Ypv-=zCMZJ8TSMMpVXRsvHUQFaiBQ+jX4=Q@mail.gmail.com>
 <CAP7+vJKcqfSQF6rrKu_3fNhOO4K=VA0t3Jd1ckksA=UGafxzXg@mail.gmail.com>
Message-ID: <CADiSq7e2JHgFcmJELyyxZe8yOekBxS0YbmQWJpE=ufh3-5N0QA@mail.gmail.com>

On 26 April 2017 at 02:56, Guido van Rossum <gvanrossum at gmail.com> wrote:
> In the end I agree with the OP that we should fix this. I don't see a reason
> to require a PEP or require updating whatever PEP described this behavior
> originally -- PEPs generally describe what should be done to a specific
> version of Python, they don't prevent future alterations, and they
> essentially represent the historical record, not current documentation.
>
> I'm a little worried about breaking existing code, but only a little bit,
> and this is clearly a gray area, so I think it's okay to change in 3.7
> without deprecations. (But I've been overruled on such matters before, so if
> you disagree, speak up now and show us your code!)

This is really obscure behaviour to be relying on, so a porting note +
the 3.7 pre-release testing cycles seems like sufficient notice to me.

It's potentially also worth checking how PyPy handles these cases -
for the only other similar case I'm aware of (the quirks with the
relative priority of the nb_* and sq_* slots at the C layer), enough
projects relied on the CPython behaviour for them to decide to
replicate it, so if they *haven't* replicated the quirk described in
the OP, it's a solid data point suggesting there aren't a lot of major
projects relying on it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Thu Apr 27 11:24:58 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Apr 2017 01:24:58 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
Message-ID: <CADiSq7d6dP-1XmNW5=w7dkQLL0zRuosRaQ-0yj-RPm0ryvje9g@mail.gmail.com>

On 25 April 2017 at 11:08, Erik <python at lucidity.plus.com> wrote:
> Hi. I suspect that this may have been discussed to death at some point in
> the past, but I've done some searching and I didn't come up with much.

Hi Erik,

Offering more concise ways of writing more powerful class definitions
is certainly a common topic of exploration in Python development.

A couple of folks have already mentioned Hynek Schlawack's `attrs`
packages, which pairs up a class decorator with particular kinds of
values set as class level attributes:

    from attr import attrs, attrib

    @attrs
    class MyClass:
        foo = attrib()
        bar = attrib()
        baz = attrib()
        ...

Here, the `attrs` decorator not only writes __init__ for you, but also
__repr__, comparison operators, pickle support, and a range of other
things. Glyph Lefkowitz put together a good post last year describing
just how much functionality attrs implicitly adds to a typical class
definition, all while making it shorter and less repetitive:
https://glyph.twistedmatrix.com/2016/08/attrs.html

So I think if Python were to make any changes to the default toolset
in this area, it would be more along the lines of how `attrs` works
(i.e. offering a more declarative approach to the high level task of
defining structured data containers), than it would be along the lines
of a low level syntactic solution that only benefited `__init__` and
`__new__` without addressing any of the other repetitive aspects of
writing comprehensive class definitions.

For example, if we decided we wanted to go down that path, we could
add a new `autoclass` module and write the equivalent of the above as:

    from autoclass import make, field

    @make
    class MyClass:
        foo = field()
        bar = field()
        baz = field()
        ...

That would provide us with a clear terminological distinction between
autoclass fields and class attributes in general, while still letting
people eliminate a lot of the boilerplate currently involved in
writing classes that produce nice and easy to use instances.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rymg19 at gmail.com  Thu Apr 27 11:27:48 2017
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Thu, 27 Apr 2017 10:27:48 -0500
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CADiSq7d6dP-1XmNW5=w7dkQLL0zRuosRaQ-0yj-RPm0ryvje9g@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CADiSq7d6dP-1XmNW5=w7dkQLL0zRuosRaQ-0yj-RPm0ryvje9g@mail.gmail.com>
Message-ID: <CAO41-mOO-Yq8=NubkXwu271O0FE=Akm6=ydgyRyAEJR7Xdr7pg@mail.gmail.com>

*cough* I'll just drop this here a sec *cough*:

https://code.activestate.com/recipes/580790-auto-assign-self-attributes-in-__init__-using-pep-/

On Thu, Apr 27, 2017 at 10:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 25 April 2017 at 11:08, Erik <python at lucidity.plus.com> wrote:
>> Hi. I suspect that this may have been discussed to death at some point in
>> the past, but I've done some searching and I didn't come up with much.
>
> Hi Erik,
>
> Offering more concise ways of writing more powerful class definitions
> is certainly a common topic of exploration in Python development.
>
> A couple of folks have already mentioned Hynek Schlawack's `attrs`
> packages, which pairs up a class decorator with particular kinds of
> values set as class level attributes:
>
>     from attr import attrs, attrib
>
>     @attrs
>     class MyClass:
>         foo = attrib()
>         bar = attrib()
>         baz = attrib()
>         ...
>
> Here, the `attrs` decorator not only writes __init__ for you, but also
> __repr__, comparison operators, pickle support, and a range of other
> things. Glyph Lefkowitz put together a good post last year describing
> just how much functionality attrs implicitly adds to a typical class
> definition, all while making it shorter and less repetitive:
> https://glyph.twistedmatrix.com/2016/08/attrs.html
>
> So I think if Python were to make any changes to the default toolset
> in this area, it would be more along the lines of how `attrs` works
> (i.e. offering a more declarative approach to the high level task of
> defining structured data containers), than it would be along the lines
> of a low level syntactic solution that only benefited `__init__` and
> `__new__` without addressing any of the other repetitive aspects of
> writing comprehensive class definitions.
>
> For example, if we decided we wanted to go down that path, we could
> add a new `autoclass` module and write the equivalent of the above as:
>
>     from autoclass import make, field
>
>     @make
>     class MyClass:
>         foo = field()
>         bar = field()
>         baz = field()
>         ...
>
> That would provide us with a clear terminological distinction between
> autoclass fields and class attributes in general, while still letting
> people eliminate a lot of the boilerplate currently involved in
> writing classes that produce nice and easy to use instances.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Ryan (????)
Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else
http://refi64.com/

From srkunze at mail.de  Thu Apr 27 13:29:22 2017
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 27 Apr 2017 19:29:22 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <odr4n7$10l$1@blaine.gmane.org>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <bac88c93-e388-5616-f418-f2ecd267b46f@lucidity.plus.com>
 <CAH0mxTRYEm2ny9aGFqxU0qK0qPFN7ERxbqE1e=PC6=5jZ_JTnQ@mail.gmail.com>
 <1f9146c9-2a8c-040e-22a7-f7f7b7df2489@lucidity.plus.com>
 <CAHkxivdR11wuftmC=sN+__G0KkPuDCjJmE8Jw4BKP7B78yPkFw@mail.gmail.com>
 <beee40f8-42de-24a3-00c6-54ee1725841c@lucidity.plus.com>
 <CAPTjJmqU_4ogrNwQKtcLimCEyWTur+MK5mVPGf-jceAgFJm+fA@mail.gmail.com>
 <odr4n7$10l$1@blaine.gmane.org>
Message-ID: <4eec2dd5-414c-53b2-7ec9-6cf66a1d805a@mail.de>

On 26.04.2017 23:50, Mark Lawrence via Python-ideas wrote:
> On 26/04/2017 21:50, Chris Angelico wrote:
>> On Thu, Apr 27, 2017 at 6:24 AM, Erik <python at lucidity.plus.com> wrote:
>>> The background is that what I find myself doing a lot of for private
>>> projects is importing data from databases into a structured 
>>> collection of
>>> objects and then grouping and analyzing the data in different ways 
>>> before
>>> graphing the results.
>>>
>>> So yes, I tend to have classes that accept their entire object state as
>>> parameters to the __init__ method (from the database values) and 
>>> then any
>>> other methods in the class are generally to do with the subsequent 
>>> analysis
>>> (including dunder methods for iteration, rendering and comparison etc).
>>
>> You may want to try designing your objects as namedtuples. That gives
>> you a lot of what you're looking for.
>>
>> ChrisA
>>
>
> Something like 
> https://docs.python.org/3/library/sqlite3.html#row-objects ?
>

Or something like https://docs.djangoproject.com/en/1.11/topics/db/models/

Sven

From srkunze at mail.de  Thu Apr 27 14:44:25 2017
From: srkunze at mail.de (Sven R. Kunze)
Date: Thu, 27 Apr 2017 20:44:25 +0200
Subject: [Python-ideas] Discourage operator.__dunder__ functions
In-Reply-To: <20170413182007.GG9464@ando.pearwood.info>
References: <20170413182007.GG9464@ando.pearwood.info>
Message-ID: <9cb3c59d-60fa-fc86-189f-f7c093e1ea5b@mail.de>

On 13.04.2017 20:20, Steven D'Aprano wrote:
> - And finally, I fail to see how having to type an extra four characters
> is a "convenience".

Just for the sake of completeness:

Re-usage of names is always a convenience. Developers can use a string 
variable to access dynamically both the real dunder method and the 
operator one. It's a frequently used pattern. Whether it's necessary 
here, is different matter.

Sven

From steve at pearwood.info  Thu Apr 27 18:43:32 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Apr 2017 08:43:32 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
Message-ID: <20170427224331.GA22525@ando.pearwood.info>

On Wed, Apr 26, 2017 at 11:29:19PM +0100, Erik wrote:

> But, if we're going to bikeshed and there is some weight behind the idea 
> that this "papercut" should be addressed, then given my previous 
> comparisons with importing, what about having 'import' as an operator:
> 
> def __init__(self, a, b, c):
>    self import a, b
>    self.foo = c * 100

[snarky]
If we're going to randomly choose arbitrary keywords with no connection 
to the operation being performed, can we use `del` instead of `import` 
because it's three characters less typing?
[/snarky]

*wink*

But seriously, I hate this idea. `import` has certain semantics that 
don't apply here:

- import goes through the full import system (sys.modules, reading 
  files from disk, sys.path, etc.); this "self import ..." does not;

- import creates local variables; this "self import ..." does not.

The semantics are very different and there's little or no connection 
between importing a module and setting an attribute on self.

If we're going to discuss pie-in-the-sky suggestions, here is an only 
semi-serious suggestion. A statement that injects identifiers into 
an explicitly given name space:

    inject spam, eggs as cackleberry, cheese into self

(If you don't like "inject", I'm okay with "load" or even "push".)

This would be equivalent to:

    self.spam = spam
    self.cackleberry = eggs
    self.cheese = cheese

Presumably it would be implemented a little more efficiently, e.g. only 
a single lookup for self, etc. Note that the target is completely 
general: you can inject into any object which will accept setting 
attributes:

    inject spam, eggs into namespace

If the "inject as" form is used, the value can be any expression:

    inject x + 1 as y into namespace

otherwise the value must be a valid identifier but not necessarily a 
local variable.

If I were designing a language from scratch, one with a maximalist 
approach to syntax, I'd take this seriously. But Python has relatively 
minimal syntax and keywords, and the problem this solves isn't big or 
important enough for the disruption of adding a new keyword.



-- 
Steve

From python at lucidity.plus.com  Thu Apr 27 19:18:19 2017
From: python at lucidity.plus.com (Erik)
Date: Fri, 28 Apr 2017 00:18:19 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170427224331.GA22525@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
Message-ID: <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>

On 27/04/17 23:43, Steven D'Aprano wrote:
> On Wed, Apr 26, 2017 at 11:29:19PM +0100, Erik wrote:
>> def __init__(self, a, b, c):
>>    self import a, b
>>    self.foo = c * 100
>
> [snarky]
> If we're going to randomly choose arbitrary keywords with no connection
> to the operation being performed,

The keyword I chose was not random or arbitrary and it _does_ have a 
connection to the operation being performed (bind a value in the source 
namespace to the target namespace using the same name it had in the 
source namespace - or rename it using the 'as' keyword).

> can we use `del` instead of `import`
> because it's three characters less typing?

Comments like this just serve to dismiss or trivialize the discussion. 
We acknowledged that we're bikeshedding so it was not a serious 
suggestion, just a "synapse prodder" ...

> But seriously, I hate this idea.

Good. It's not a proposal, but something that was supposed to generate 
constructive discussion.

> The semantics are very different and there's little or no connection
> between importing a module and setting an attribute on self.

At the technical level of what goes on under the covers, yes. At the 
higher level of what the words mean in spoken English, it's really not 
so different a concept.

> If we're going to discuss pie-in-the-sky suggestions,

That is just dismissing/trivializing the conversation again.

> (If you don't like "inject", I'm okay with "load" or even "push".)

No you're not, because that's a new keyword which might break existing 
code and that is even harder to justify than re-using an existing 
keyword in a different context.

> the problem this solves isn't big or
> important enough for the disruption of adding a new keyword.

So far, you are the only one to have suggested adding a new keyword, I 
think ;)

E.

From steve at pearwood.info  Thu Apr 27 19:21:34 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Apr 2017 09:21:34 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
Message-ID: <20170427232133.GB22525@ando.pearwood.info>

On Wed, Apr 26, 2017 at 03:54:22PM -0400, Jerry Hill wrote:
> On Tue, Apr 25, 2017 at 8:05 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > def ___init__(self, self.attr):
> 
> I'm not a python developer, I'm just a developer that uses python.
> That said, I really like this form.  It eliminates most of the
> redundancy, while still being explicit.  It's true that you have to
> repeat the 'self' name, but it feels like the right balance in my
> mind.  It seems like anyone who's familiar with python function
> definitions would immediately grasp what's going on.

I don't like it, not even a bit. It's not general, "self" is effectively 
a special case. Consider:

What happens if you use this syntax in a top-level function rather 
than a method? (Or a static method?)

def function(x, y, x.attr):
    ...

(And don't forget that behind the scenes, methods *are* functions.) What 
would this syntax even mean? 


Or if you use some other name other than the first parameter?

def method(self, spam, foo.eggs):
    ...


Conceptually, it is mixing up two distinct tasks:

- declaring the parameter list of the function/method;

- running part of the body of the method.

The parameter list of the method is the *interface* to the method: it 
tells you the public names and default values (and possibly types, if 
you use type annotations) of the method parameters. But this syntax 
overloads the parameter list to show part of the *implementation* of the 
method (namely, that some parameters are assigned directly to attributes 
of self). Every time the implementation changes, the parameter list will 
change too.

For instance, if you decide to add some argument validation to your 
arguments (checking that their values and types are correct), using the 
self.attr form is inappropriate.

And while it looks okay in the toy example shown by Ryan:

    def ___init__(self, self.attr):


it doesn't look so good in larger, more realistic cases, especially with 
other syntax. Here's a parameter list taken from some real code of mine, 
with the "self." syntax added:


class BinnedData(object):
    def __init__(self, self.number:int, 
                       self.start:float=None, 
                       self.end:float=None, 
                       self.width:float=None, 
                       *, 
                       self.policy=None, 
                       self.mark=False
                       ):

The repetition of "self" is not only tedious and verbose, it adds 
noise to the parameter list and pushes the reader's attention away from 
the important part (the name of the parameter, e.g. "width") and to 
something irrelevant to the interface ("self").

And I am not looking forward to having to explain to beginners to Python 
why this doesn't work:

data = BinnedData(self.number = 8, self.start = 0, self.end = 20)



-- 
Steve

From steve at pearwood.info  Thu Apr 27 19:28:55 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Apr 2017 09:28:55 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAN1YFWsFYGkWjN=Zn_xrBX97Jn8AYZaevcQzCz9nOaQ8dpSXhA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <701f3390-5f12-bf34-0874-8451d7f81add@mgmiller.net>
 <CAN1YFWsFYGkWjN=Zn_xrBX97Jn8AYZaevcQzCz9nOaQ8dpSXhA@mail.gmail.com>
Message-ID: <20170427232853.GC22525@ando.pearwood.info>

On Wed, Apr 26, 2017 at 08:52:46PM -0400, Juancarlo A?ez wrote:

> In my experience, what Python is lacking is a way to declare attributes
> outside of the constructor. Take a look at how it's done in C#, Swisft, or
> Go.

Since you apparently already know how they do it, how about telling us 
(1) how they do it, and (2) *why* they do it?


> Object attributes outside of the constructor would solve things more
> relevant than the vertical space used when assigning constructor parameters
> to attributes. 

Solve which things?


> For example, multiple inheritance is well designed in
> Python, except that it often requires constructors with no parameters,
> which leads to objects with no default attributes.

Can you elaborate?



-- 
Steve

From steve at pearwood.info  Thu Apr 27 19:34:11 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Apr 2017 09:34:11 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <02ad01d2be85$00c935c0$025ba140$@hotmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
 <20170426113231.GA31536@phdru.name>
 <02ad01d2be85$00c935c0$025ba140$@hotmail.com>
Message-ID: <20170427233410.GD22525@ando.pearwood.info>

On Wed, Apr 26, 2017 at 08:02:39AM -0400, tritium-list at sdamon.com wrote:

> >     self.__dict__.update(kwargs)
> 
> Touching __dict__ feels dirty to me.


Indeed. The right way is:

    vars(self).update(kwargs)

although that doesn't work if self is written to use __slots__ instead 
of having a __dict__.

You can even (almost) get this same effect and still have named 
parameters:

    def __init__(self, fe, fi, fo, fum, spam, eggs, foo, bar, baz):
        vars(self).update(locals())
        del self.self


-- 
Steve

From rosuav at gmail.com  Thu Apr 27 19:54:55 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Apr 2017 09:54:55 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170427232133.GB22525@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
Message-ID: <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>

On Fri, Apr 28, 2017 at 9:21 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> What happens if you use this syntax in a top-level function rather
> than a method? (Or a static method?)
>
> def function(x, y, x.attr):
>     ...
>
> (And don't forget that behind the scenes, methods *are* functions.) What
> would this syntax even mean?
>
>
> Or if you use some other name other than the first parameter?
>
> def method(self, spam, foo.eggs):

Exactly the same thing as:

def function(*args):
    x, y, x.attr = args

def method(*args):
    self, spam, foo.eggs = args

It's well-defined - at least for positional args. Not sure what
keyword arg handling would be though.

ChrisA

From steve at pearwood.info  Fri Apr 28 00:06:09 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Apr 2017 14:06:09 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
Message-ID: <20170428040609.GE22525@ando.pearwood.info>

On Fri, Apr 28, 2017 at 09:54:55AM +1000, Chris Angelico wrote:
> On Fri, Apr 28, 2017 at 9:21 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> > What happens if you use this syntax in a top-level function rather
> > than a method? (Or a static method?)
> >
> > def function(x, y, x.attr):
> >     ...
> >
> > (And don't forget that behind the scenes, methods *are* functions.) What
> > would this syntax even mean?
> >
> >
> > Or if you use some other name other than the first parameter?
> >
> > def method(self, spam, foo.eggs):
> 
> Exactly the same thing as:
> 
> def function(*args):
>     x, y, x.attr = args
> 
> def method(*args):
>     self, spam, foo.eggs = args


Obviously we can define syntax to do anything we like, but what is the 
logical connection between the syntax and the semantics? What part of 
"function parameter list" suggests "assign attributes to arbitrary 
objects"?

Do we support arbitrary targets in their full generality?

def function(arg, 
        (name if name is not None else other).the_object.attr[0].method(foo)['key'].argument
        ):
    ...

Of course, there are lots of things which are technically allowed but if 
someone is silly enough to do it, we say "don't do that". However, even 
the simple case is problematic:

def trigonometry(x, y, math.pi)

We could make that parameter declaration have the side-effect of 
assigning to math.pi, but that's a pretty surprising change to function 
declarations. Everything else declared inside a parameter list is local 
to the function: parameters themselves are local variables, default 
values and annotations are stored in the function not externally.

The only thing which is not local is that default values may be 
arbitrary expressions, which may have side-effects, but that is not a 
declaration, its an expression evaluated for its value.

What's the connection between parameter declarations and this sort of 
external side-effect? It seems pretty arbitrary to be a built-in part of 
the language, like having

def function(x)

automatically print x, or pickle it and upload the file to the cloud.

(Obviously the side-effect doesn't occur until the function is called, 
but still, it is part of the parameter list declaration, not the body of 
the function.)

I have grave doubts that people will find this obvious, let alone 
useful. Having a function declaration, the parameter list, have 
side-effects outside of the function seems like both a surprising and a 
bad idea to me.


> It's well-defined - at least for positional args. Not sure what
> keyword arg handling would be though.

Why would it be any different? If it makes sense to allow:

def function(spam, eggs, something.foo)

assign to something.foo, then why should it matter whether it was called 
by keyword or positional? Or declared as keyword only?

def function(spam, eggs, *, something.cheese)


I don't think it makes sense either way, but it isn't because it is a 
keyword arg. I think it is arbitrarily mixing two distinct things into 
one convenient but confusing syntax.

function(spam=1, eggs=2, something.cheese=3)  # won't work



-- 
Steve

From rosuav at gmail.com  Fri Apr 28 01:30:29 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Apr 2017 15:30:29 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170428040609.GE22525@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
Message-ID: <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>

On Fri, Apr 28, 2017 at 2:06 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Apr 28, 2017 at 09:54:55AM +1000, Chris Angelico wrote:
>> On Fri, Apr 28, 2017 at 9:21 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> > What happens if you use this syntax in a top-level function rather
>> > than a method? (Or a static method?)
>> >
>> > def function(x, y, x.attr):
>> >     ...
>> >
>> > (And don't forget that behind the scenes, methods *are* functions.) What
>> > would this syntax even mean?
>> >
>> >
>> > Or if you use some other name other than the first parameter?
>> >
>> > def method(self, spam, foo.eggs):
>>
>> Exactly the same thing as:
>>
>> def function(*args):
>>     x, y, x.attr = args
>>
>> def method(*args):
>>     self, spam, foo.eggs = args
>
>
> Obviously we can define syntax to do anything we like, but what is the
> logical connection between the syntax and the semantics? What part of
> "function parameter list" suggests "assign attributes to arbitrary
> objects"?
>
> Do we support arbitrary targets in their full generality?
>
> def function(arg,
>         (name if name is not None else other).the_object.attr[0].method(foo)['key'].argument
>         ):
>     ...
>
> Of course, there are lots of things which are technically allowed but if
> someone is silly enough to do it, we say "don't do that". However, even
> the simple case is problematic:
>
> def trigonometry(x, y, math.pi)
>
> We could make that parameter declaration have the side-effect of
> assigning to math.pi, but that's a pretty surprising change to function
> declarations. Everything else declared inside a parameter list is local
> to the function: parameters themselves are local variables, default
> values and annotations are stored in the function not externally.

What part of a 'for' loop suggests that you can do this?

odds = [0]*10; evens = [0]*10
for idx, (odds if idx%2 else evens)[idx//2] in stuff:
    ...

Nothing whatsoever says that this is a good idea, but it's perfectly
legal, because the for loop is defined in terms of assignment. If this
were to be accepted (which, fwiw, I'm not actually advocating, but
IF), it would also be defined in terms of assignment. You still
shouldn't be assigning to arbitrary objects, especially not randomly
rebinding module names, but it's easy to grok the assignment
equivalence.

And yes, it WOULD reinstate the argument unpacking removed by PEP
3113. So for this to go forward, the objections in that PEP would have
to be addressed.

ChrisA

From steve at pearwood.info  Fri Apr 28 02:57:23 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 28 Apr 2017 16:57:23 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
 <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
Message-ID: <20170428065723.GF22525@ando.pearwood.info>

On Fri, Apr 28, 2017 at 03:30:29PM +1000, Chris Angelico wrote:

> > Obviously we can define syntax to do anything we like, but what is the
> > logical connection between the syntax and the semantics? What part of
> > "function parameter list" suggests "assign attributes to arbitrary
> > objects"?
[...]
> What part of a 'for' loop suggests that you can do this?

I'm not sure what "this" is supposed to do. You've written some 
obscure but perfectly legal Python code:

> odds = [0]*10; evens = [0]*10
> for idx, (odds if idx%2 else evens)[idx//2] in stuff:
>     ...

My guess is that "this" refers to the side-effect that assigning to a 
list item updates the list item. Um, yeah, it does. What's your point? 
Of course it does. That's what its supposed to do.

Perhaps you think that there's something weird about using an arbitrary 
assignment target as the for-loop. I don't know why you think that's 
weird. Here's a simpler example:

for obj.attr in seq:
    ...

Yes, its a bit unusual to do that, but its not weird. The assignment 
target for a loop is just an ordinary assignment target, and the 
assignment occurs during the execution of code, just like any other 
assignment that occurs inside the body of the function.

What is weird is to have the function *declaration* have global side 
effects and perform assignments outside of the function. The parameter 
list is *not* a general assignment statement, it is a declaration of 
what local variables will be assigned to when you call the function.

In Python 3, function paramaters are plain (non-qualified) 
identifiers, not general assignment targets. Even in Python 2, the most 
that was supported were tuple-unpacking. Even that can be read as 
conceptually just a declaration:

    def func(a, (x,y)):

declares that the first argument is a, and the second argument is a pair 
of values x and y. But this proposal has to be read as a declaration 
plus a separate assignment:

    def func(a, spam.x):

declares that the second argument is called "x", (but not spam.x), and 
*in addition* there will be an assignment spam.x = x at some point, 
presumably before the function body gets entered.


> Nothing whatsoever says that this is a good idea, but it's perfectly
> legal, because the for loop is defined in terms of assignment. If this
> were to be accepted (which, fwiw, I'm not actually advocating, but
> IF), it would also be defined in terms of assignment.

Why should it be defined in terms of general assignment? That's the 
point I'm making. While function sigs are a form of assignment, they're 
more of a declaration than an executable statement like the other 
binding statements. There's a superficial connection, but they really 
are quite different things.

(For example: if you allow spam.foo as a parameter, that can call 
arbitrary Python code in spam.__setattr__, which assigning to foo as a 
parameter will not do.)


> You still
> shouldn't be assigning to arbitrary objects, especially not randomly
> rebinding module names, but it's easy to grok the assignment
> equivalence.

You need to speak to more beginners if you think the connection between 
spam.x and x is obvious:

    def func(spam.x):
        print(x)

Where is x declared? It looks like there's a local spam.x which isn't 
used, and a global x that is. But that's completely wrong. Without the 
context of somebody telling you "this is syntax for magically assigning 
to self.attributes in the constructor", I believe this will be 
unfathomable to the average non-expert.


> And yes, it WOULD reinstate the argument unpacking removed by PEP
> 3113. So for this to go forward, the objections in that PEP would have
> to be addressed.

Nicely remembered :-)



-- 
Steve

From random832 at fastmail.com  Fri Apr 28 03:02:24 2017
From: random832 at fastmail.com (Random832)
Date: Fri, 28 Apr 2017 03:02:24 -0400
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
 <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
Message-ID: <1493362944.2398068.959041040.3BBE546B@webmail.messagingengine.com>

On Fri, Apr 28, 2017, at 01:30, Chris Angelico wrote:
> Nothing whatsoever says that this is a good idea, but it's perfectly
> legal, because the for loop is defined in terms of assignment. If this
> were to be accepted (which, fwiw, I'm not actually advocating, but
> IF), it would also be defined in terms of assignment. You still
> shouldn't be assigning to arbitrary objects, especially not randomly
> rebinding module names, but it's easy to grok the assignment
> equivalence.

What's not clear is when the left side (an object whose attribute/item
is being assigned, and the item index) is evaluated, and why this should
be different from when default arguments are evaluated.

From rosuav at gmail.com  Fri Apr 28 03:23:59 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Apr 2017 17:23:59 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170428065723.GF22525@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
 <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
 <20170428065723.GF22525@ando.pearwood.info>
Message-ID: <CAPTjJmo+C50zsaeqUKv9riW7CKX8HjhVzhO5PtSG4ENOAZB=iA@mail.gmail.com>

On Fri, Apr 28, 2017 at 4:57 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Apr 28, 2017 at 03:30:29PM +1000, Chris Angelico wrote:
>
>> > Obviously we can define syntax to do anything we like, but what is the
>> > logical connection between the syntax and the semantics? What part of
>> > "function parameter list" suggests "assign attributes to arbitrary
>> > objects"?
> [...]
>> What part of a 'for' loop suggests that you can do this?
>
> I'm not sure what "this" is supposed to do. You've written some
> obscure but perfectly legal Python code:
>
>> odds = [0]*10; evens = [0]*10
>> for idx, (odds if idx%2 else evens)[idx//2] in stuff:
>>     ...
>
> My guess is that "this" refers to the side-effect that assigning to a
> list item updates the list item. Um, yeah, it does. What's your point?
> Of course it does. That's what its supposed to do.
>
> Perhaps you think that there's something weird about using an arbitrary
> assignment target as the for-loop. I don't know why you think that's
> weird. Here's a simpler example:
>
> for obj.attr in seq:
>     ...
>
> Yes, its a bit unusual to do that, but its not weird. The assignment
> target for a loop is just an ordinary assignment target, and the
> assignment occurs during the execution of code, just like any other
> assignment that occurs inside the body of the function.
>
> What is weird is to have the function *declaration* have global side
> effects and perform assignments outside of the function. The parameter
> list is *not* a general assignment statement, it is a declaration of
> what local variables will be assigned to when you call the function.

Waaaaaait a minute. Since when is the *declaration* doing this? No no
no. In every proposal, it's been a part of the function's *execution*.
The original example was an __init__ function that does the exact same
thing as my for loop example: it first sets "self" to the first
parameter, and then sets some attributes on self. As you say:

> Um, yeah, it does. What's your point?
> Of course it does. That's what its supposed to do.

There is nothing surprising in it (although my example was
deliberately convoluted and a bad example of code). It's assignment,
pure and simple.

When you call a function, the values you pass to it get assigned to
the names provided in the declaration. No assignments happen at
declaration other than the function's own name. (I suppose you could
say that default argument values are "assigned" at that point, but I
prefer to say "collected", since they're basically assembled into a
tuple and slapped into an attribute on the function.)

>> Nothing whatsoever says that this is a good idea, but it's perfectly
>> legal, because the for loop is defined in terms of assignment. If this
>> were to be accepted (which, fwiw, I'm not actually advocating, but
>> IF), it would also be defined in terms of assignment.
>
> Why should it be defined in terms of general assignment? That's the
> point I'm making. While function sigs are a form of assignment, they're
> more of a declaration than an executable statement like the other
> binding statements. There's a superficial connection, but they really
> are quite different things.

There's a lot more similarities than you might think. For example,
both of these will create "spam" as a local name, shadowing the
global:

spam = 1
def func1(spam):
    print(spam)
def func2():
    spam = 2
    print(spam)

As will many other forms of assignment, like "import spam" or "with x
as spam:". Some forms are more restricted than others ("import"
requires a NAME, not an arbitrary expression), but they all behave the
same way with regard to global/nonlocal/local naming - you can "global
numpy; import numpy" inside a function to do a delayed import, for
instance.

> (For example: if you allow spam.foo as a parameter, that can call
> arbitrary Python code in spam.__setattr__, which assigning to foo as a
> parameter will not do.)

Sure, but that's a difference between simple name assignment and
item/attribute assignment, not a difference between function
parameters and other types of variables. Are you next going to point
out that you can't "import spam as foo[0]"? Or that importing is
somehow special because of this? No. It's still assignment, just one
with a restricted syntax.

>> You still
>> shouldn't be assigning to arbitrary objects, especially not randomly
>> rebinding module names, but it's easy to grok the assignment
>> equivalence.
>
> You need to speak to more beginners if you think the connection between
> spam.x and x is obvious:
>
>     def func(spam.x):
>         print(x)
>
> Where is x declared? It looks like there's a local spam.x which isn't
> used, and a global x that is. But that's completely wrong. Without the
> context of somebody telling you "this is syntax for magically assigning
> to self.attributes in the constructor", I believe this will be
> unfathomable to the average non-expert.

Not sure what you mean. By my reading, that's exactly correct - there
IS a global x that is being used here, although "local spam.x" is not
accurate. I wouldn't call that "completely wrong". And if you find
that confusing, let's look at importing.

os = "Linux"
path = "/tmp"
def func():
    import os.path
    print(path)

What's local and what's global? What name is actually bound by "import
os.path"? What if you said "import os.path as path"? "import os.path
as os"? Does all this confuse people? IMO it's *less* confusing if you
can simply say "this is equivalent to <parameter> = <argument>".

>> And yes, it WOULD reinstate the argument unpacking removed by PEP
>> 3113. So for this to go forward, the objections in that PEP would have
>> to be addressed.
>
> Nicely remembered :-)
>

I didn't remember the PEP number, but the Google is helpful :)

ChrisA

From rosuav at gmail.com  Fri Apr 28 03:31:01 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Apr 2017 17:31:01 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <1493362944.2398068.959041040.3BBE546B@webmail.messagingengine.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
 <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
 <1493362944.2398068.959041040.3BBE546B@webmail.messagingengine.com>
Message-ID: <CAPTjJmq9xhOQ40ZA2=q+VCT9uFr824riqZ2ATgjsPtLVBFJ6OQ@mail.gmail.com>

On Fri, Apr 28, 2017 at 5:02 PM, Random832 <random832 at fastmail.com> wrote:
> On Fri, Apr 28, 2017, at 01:30, Chris Angelico wrote:
>> Nothing whatsoever says that this is a good idea, but it's perfectly
>> legal, because the for loop is defined in terms of assignment. If this
>> were to be accepted (which, fwiw, I'm not actually advocating, but
>> IF), it would also be defined in terms of assignment. You still
>> shouldn't be assigning to arbitrary objects, especially not randomly
>> rebinding module names, but it's easy to grok the assignment
>> equivalence.
>
> What's not clear is when the left side (an object whose attribute/item
> is being assigned, and the item index) is evaluated, and why this should
> be different from when default arguments are evaluated.

Now that is a very good, dare I say it, argument. It's easy enough to
explain to an expert, but it's not clear to a beginner.

Expert explanation (omitting kwargs):

def __init__(self, self.spam=[]):

is roughly equivalent to:

_defaults = ([],)
def __init__(*args):
    self, self.spam = (args + _defaults)[:2]

It's pretty straight-forward for default arguments to be evaluated at
declaration time, but assignment targets at execution time; but it
isn't clear.

I was never really in favour of the proposal, but this is a fairly big downside.

ChrisA

From contact at brice.xyz  Fri Apr 28 05:23:52 2017
From: contact at brice.xyz (Brice PARENT)
Date: Fri, 28 Apr 2017 11:23:52 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <9ba1a91b-42d8-d012-dfc3-c2a6f9ad5858@brice.xyz>
Message-ID: <8734b65b-c943-ee5d-3282-bc45f0cd2d42@brice.xyz>


Le 26/04/17 ? 13:12, Brice PARENT a ?crit :
> Why not simply do this :
>
> class MyClass:
>     def _set_multiple(self, **kwargs):
>         for key, value in kwargs.items():
>             setattr(self, key, value)
>
>     def __init__(self, a, b, c):
>         self._set_multiple(a=a, b=b, c=c)
>
> -> No new syntax, no inspection, simple to read, parse and understand, 
> and explicit.
>
> And you could directly do some simple cleanings or not define a 
> variable, like :
>
> class MyClass:
>     def _set_multiple(self, **kwargs):
>         for key, value in kwargs.items():
>             setattr(self, key, value)
>
>     def __init__(self, a, b, c):
>         self._set_multiple(a=a * 2, b=clean_field(b))
>
> - Brice
Also, a new syntax that could be useful in many other ways, while being 
easy to understand to any pythonista, would be :
class MyClass:
     def __init__(self, a, b, c):
         self.(a, b, c) = a, b, c

You still have to type twice the name of the variable instead of one 
though, if you don't want to use *args and **kwargs, but you still save 
the same vertical space with something efficient and way more useful. It 
is currently an invalid syntax, but that allows to be sure that it 
doesn't break old source code and doesn't require any deprecation. As a 
new syntax, it can probably be optimized a lot at the implementation level.

It can be used with *args and **kwargs also, ...
class MyClass:
     def __init__(self, *args, **kwargs):
         self.(a, b, c) = args
         self.(*kwargs.keys()) = kwargs.values()
         # or
         self.(kwargs.keys()) = kwargs.values()

... on instantiated objects (so not only in self), ...
background_color.('r', 'g', 'b') = 255, 127, 0

... update the values to be stored on-the-fly, ...
class MyClass:
     def __init__(self, a, b, c):
         self.(a, b, c) = a, b * 2.54, c

... make any validation you want, ...
class MyClass:
     def __init__(self, a, b, c):
         if c > 10:
             b = 0
         elif c < 0:
             raise ValidationError("c can't be lower than 0")

         self.(a, b, c) = a, b, c

... update the names of the attributes and insert names that weren't in 
the signature ...
class MyClass:
     def __init__(self, a, b, c):
         self.(a, b, _c, uuid) = a, b, c, str(uuid.uuid4())

Morality : This is a completely other thing to what's proposed here, 
even if it does solve more or less OP's initial problem. It's open to 
discussion whether it's useful or not, but it allows some kind of 
unpacking at the classes properties level, not exclusively into active 
scope.

- Brice


From p.f.moore at gmail.com  Fri Apr 28 05:47:34 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 28 Apr 2017 10:47:34 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
 <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
Message-ID: <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>

On 28 April 2017 at 00:18, Erik <python at lucidity.plus.com> wrote:
>> The semantics are very different and there's little or no connection
>> between importing a module and setting an attribute on self.
>
> At the technical level of what goes on under the covers, yes. At the higher
> level of what the words mean in spoken English, it's really not so different
> a concept.

I disagree. If you were importing into the *class* (instance?) I might
begin to see a connection, but importing into self? Also, if you try
to make the obvious generalisations (which you'd *have* to be able to
make due to the way Python works) things quickly get out of hand:

def __init__(self, a):
    self import a

OK, but self is just a variable name, so we can reasonably use a different name:

def __init__(foo, a):
    foo import a

So the syntax is <variable> import <var-list>

Presumably the following also works, because there's nothing special
about parameters?

def __init__(x, a):
    calc = a**2
    x import calc

And of course there's nothing special about __init__

def my_method(self, a):
    self import a

Or indeed about methods

def standalone(a, b):
    a import b

or statements inside functions:

if __name __ == '__main__:
    a = 12
    b = 13
    a import b

Hmm, I'd hope for a type error here. But what types would be allowed
for a? Class instances? Well, a is an int, which is a class instance.
See what I mean? Things get out of hand *very* fast.

>> (If you don't like "inject", I'm okay with "load" or even "push".)
>
> No you're not, because that's a new keyword which might break existing code
> and that is even harder to justify than re-using an existing keyword in a
> different context.

Well, picking an existing keyword simply[1] to avoid using a new one
is an equally bad justification.

[1] I know that's not what you did, but without the "it's sort of like
importing" justification, which Steven has said he doesn't agree with,
and nor do I, you're only left with "because it means we don't need a
new keyword" - at which point suggesting that we need to bite the
bullet and consider whether a new keyword is justified is perfectly
reasonable. (IMO, a new keyword is *not* justified - nor is reusing an
existing one unless you can find a *much* more compelling parallel
than you did with import).

To summarise:

1. There's some serious technical issues with your proposal, which as
far as I can see can only be solved by arbitrary restrictions on how
it can be used
2. The use of import as a keyword is at best controversial
3. Using a new keyword for this construct is pretty much a non-starter

But I do agree that the proposal was a reasonable way to reframe the
debate here. It's just that what it's done is confirm my belief that
there's nothing here that needs improvement compared to the status quo
(even though I agree that long lists of "self.x = x" statements can be
tedious boilerplate).

Paul

From tinchester at gmail.com  Fri Apr 28 07:55:12 2017
From: tinchester at gmail.com (=?UTF-8?Q?Tin_Tvrtkovi=C4=87?=)
Date: Fri, 28 Apr 2017 11:55:12 +0000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <mailman.5562.1493362652.2950.python-ideas@python.org>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
Message-ID: <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>

I'm gonna take a shot at elaborating this point.

We Python programmers often tout Python as a high-level, high-productivity
language where complex and useful things are easy to do. However, vanilla
Python classes are anything but high level; they're basically glorified
dictionaries that require an awful lot of boilerplate for basic features.
__init__ verbosity is just one of the symptoms of this.

I'm going to posit we need declarative classes. (This is what a library
like attrs provides, basically.) For a class to be declarative, it needs to
be possible to inspect the class for its attributes and more.

I'm putting forward three examples. These examples are based on attrs since
that's what I consider to be the best way of having declarative classes in
Python today.

First example. Here's a slide from a recent talk I've given to a local
Python meetup: http://tinche.github.io/talks/attrs/#/8/5 This is a class
decorator that, when applied to any attrs class, will generate a simple
shallow copy() method for that class. This copy method will be faster than
copy.copy() and IMO less surprising since it will actually invoke __init__
(and all logic inside). This isn't production code, just a proof of concept
for the talk.

Second example: I'm working on a library that can turn dictionaries into
attrs classes (i.e. turn unstructured data into structured data). Example:
http://cattrs.readthedocs.io/en/latest/structuring.html#attrs-classes. When
dealing with nested classes the library not only needs to know exactly
which fields a class has but also the types of those fields, recursively
(for which I'm planning to use attrs metadata). This functionality is
useful when preparing data to be stored or sent elsewhere, since
serialization libraries and clients in other languages expect json/msgpack
etc.

Third example: I work at a mobile games company. The backends are Python,
the games are written in other languages. The backends and the games share
data structures - requests, responses, the save game, game data. I want to
have a single source of truth for these structures, so I want to generate
classes/structs in other languages from the backend Python classes.

All of these examples require declarative classes to do properly. Other
approaches are, I suppose, possible in some cases - like inspecting
__init__ and whatnot. But I claim the underlying issue is that we should be
declaring classes differently, both to enable these new, high-level use
cases and because declarative classes are much more readable and contain
less boilerplate than what we have now, and this is (for me) Python is
about.

Message: 1
> Date: Fri, 28 Apr 2017 09:28:55 +1000
> From: Steven D'Aprano <steve at pearwood.info>
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] Augmented assignment syntax for objects.
> Message-ID: <20170427232853.GC22525 at ando.pearwood.info>
> Content-Type: text/plain; charset=iso-8859-1
>
> On Wed, Apr 26, 2017 at 08:52:46PM -0400, Juancarlo A?ez wrote:
>
> > In my experience, what Python is lacking is a way to declare attributes
> > outside of the constructor. Take a look at how it's done in C#, Swisft,
> or
> > Go.
>
> Since you apparently already know how they do it, how about telling us
> (1) how they do it, and (2) *why* they do it?
>
>
> > Object attributes outside of the constructor would solve things more
> > relevant than the vertical space used when assigning constructor
> parameters
> > to attributes.
>
> Solve which things?
>
>
> > For example, multiple inheritance is well designed in
> > Python, except that it often requires constructors with no parameters,
> > which leads to objects with no default attributes.
>
> Can you elaborate?
>
>
>
> --
> Steve
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170428/38bfb5d2/attachment.html>

From p.f.moore at gmail.com  Fri Apr 28 08:26:28 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 28 Apr 2017 13:26:28 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
Message-ID: <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>

On 28 April 2017 at 12:55, Tin Tvrtkovi? <tinchester at gmail.com> wrote:
> I'm putting forward three examples. These examples are based on attrs since
> that's what I consider to be the best way of having declarative classes in
> Python today.

Your comments and examples are interesting, but don't they just come
down to "attrs is a really good library"? I certainly intend to look
at it based on what's been said in this thread. But I don't see
anything much that suggests that anything *beyond* attrs is needed
(except maybe bringing attrs into the stdlib, if its release cycle
would make that reasonable and the author was interested, and honestly
"put it in the stdlib" could be said about any good library - in
practice though publishing on PyPI and accessing via pip is 99% of the
time the more reasonable option).

Am I missing some point?
Paul

From ncoghlan at gmail.com  Fri Apr 28 08:40:28 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Apr 2017 22:40:28 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
Message-ID: <CADiSq7c_3v0jXasHWPSwOaJHHUWp6e4+m+yFrhaW2rC7Mzq+Yg@mail.gmail.com>

On 28 April 2017 at 21:55, Tin Tvrtkovi? <tinchester at gmail.com> wrote:
> Third example: I work at a mobile games company. The backends are Python,
> the games are written in other languages. The backends and the games share
> data structures - requests, responses, the save game, game data. I want to
> have a single source of truth for these structures, so I want to generate
> classes/structs in other languages from the backend Python classes.

Some additional metaclass based prior art for declarative class definitions:

- Django ORM models
- SQL Alchemy ORM models
- JSL template definitions (http://jsl.readthedocs.io/en/latest/tutorial.html)

(I'm sure there's also plenty of prior art in Zope and Plone, but I'm
not personally all that familiar with either of those)

> All of these examples require declarative classes to do properly. Other
> approaches are, I suppose, possible in some cases - like inspecting __init__
> and whatnot. But I claim the underlying issue is that we should be declaring
> classes differently, both to enable these new, high-level use cases and
> because declarative classes are much more readable and contain less
> boilerplate than what we have now, and this is (for me) Python is about.

Exactly.

I don't think we should change anything about the basic semantics of
class or method definitions (since they're part of what makes it
possible to reason about Python as an inherently procedural language),
but I *do* think we should be considering the kinds of helpers we can
provide as class decorators to take the boilerplate and tedium out of
defining well-behaved classes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Fri Apr 28 09:07:51 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Apr 2017 23:07:51 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
 <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>
Message-ID: <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>

On 28 April 2017 at 22:26, Paul Moore <p.f.moore at gmail.com> wrote:
> On 28 April 2017 at 12:55, Tin Tvrtkovi? <tinchester at gmail.com> wrote:
>> I'm putting forward three examples. These examples are based on attrs since
>> that's what I consider to be the best way of having declarative classes in
>> Python today.
>
> Your comments and examples are interesting, but don't they just come
> down to "attrs is a really good library"? I certainly intend to look
> at it based on what's been said in this thread. But I don't see
> anything much that suggests that anything *beyond* attrs is needed
> (except maybe bringing attrs into the stdlib, if its release cycle
> would make that reasonable and the author was interested, and honestly
> "put it in the stdlib" could be said about any good library - in
> practice though publishing on PyPI and accessing via pip is 99% of the
> time the more reasonable option).
>
> Am I missing some point?

Yes, the point I attempted to raise earlier: at the language design
level, "How do we make __init__ methods easier to write?" is the
*wrong question* to be asking. It's treating the symptom (writing an
imperative initialiser is repetitive when it comes to field names)
rather than the root cause (writing imperative initialisers is still
part of the baseline recommendation for writing classes, and we don't
offer any supporting infrastructure for avoiding that directly in the
standard library)

Accordingly, a potentially better question to ask is "How do we make
it so that writing a custom __init__ method for each class definition
seems as strange and esoteric in 2025 as writing a custom metaclass
seems now?".

For a *lot* of classes, what we want to be able to define is:

- a set of data fields
- a basic initialiser to set those fields by name
- a repr based on those fields (or a subset)
- equality comparisons based on those fields (or a subset)
- ordering comparisons based on those fields (or a subset)
- conversion to a JSON-friendly dict
- pickling support
- copying support

Given the initial set of data fields, reasonable default behaviours
for all of the rest can be derived automatically, but we don't provide
the tools to do that derivation by default. This leaves teachers of
Python in a quandary: they can either teach "native Python classes"
(which are boilerplate heavy and error prone), or else they can pick a
third party library like attrs, and teach "Python-with-attrs", in the
same way that it's recommended to teach "Python-with-requests" rather
than the native urllib APIs when it comes to accessing resources over
HTTPS.

The difference between this case and requests is that HTTPS and
related protocols experience a high level of churn independently of
Python language updates, so there are real logistical benefits to
maintaining it as a third party module.

By contrast, if we add an "autoclass" module to the standard library
that adds more tools like functools.total_ordering to handle injecting
boilerplate functionality into class definitions at runtime, then
that's a pure win - folks can either delegate the details of their
instance behaviour to the standard library maintainers (in the same
way that most folks already delegate the behaviour of their
metaclasses), or else they can continue to write out all those
supported methods by hand if they really want or need the fine-grained
control.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Fri Apr 28 11:31:44 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Apr 2017 01:31:44 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAPTjJmo+C50zsaeqUKv9riW7CKX8HjhVzhO5PtSG4ENOAZB=iA@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
 <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
 <20170428065723.GF22525@ando.pearwood.info>
 <CAPTjJmo+C50zsaeqUKv9riW7CKX8HjhVzhO5PtSG4ENOAZB=iA@mail.gmail.com>
Message-ID: <20170428153144.GG22525@ando.pearwood.info>

On Fri, Apr 28, 2017 at 05:23:59PM +1000, Chris Angelico wrote:

> Waaaaaait a minute. Since when is the *declaration* doing this? 

That's what the suggested syntax says. You put the attribute assignment 
you want in the parameter list, which is part of the function/method 
declaration, not the body. If I recall correctly, the proposal was 
exactly:

    def __init__(self, self.attr):

That's not in the body of the function, it's in the parameter list, 
hence part of the declaration. When you call the constructor:

instance = MyClass(42)

__init__ is called, the argument 42 is bound to the formal parameter 
"attr", the assignment self.attr = attr is run, and THEN the body of the 
method is called. (Before you object, see below for justification for 
why there has to be both a local variable and an attribute.)

If there is an exception, what do you think the stack trace will point 
to? It cannot point to a line of source code that says "self.attr = 
attr", because there is no such line. At best, it can point to the 
declaration "def __init__(self, self.attr):" although it might not even 
do that.


> No no no. In every proposal, it's been a part of the function's *execution*.

I did explicitly state that the attribute assignment doesn't occur until 
the function is called, so I'm aware of this. Nevertheless, the source 
code being executed at function call time does not reside in the body of 
the function, but in the function signature.

Maybe I'd be less disturbed by this if we had late binding of defaults 
(defaults are re-evaluated each time they are needed), and a history of 
abusing that for the side-effects:

def function(a, b=print("using b default") or default):

But Python function defaults are early binding, and they're only 
evaluated once, so there's currently no analogy to this proposal.


[...]
> > Why should it be defined in terms of general assignment? That's the
> > point I'm making. While function sigs are a form of assignment, they're
> > more of a declaration than an executable statement like the other
> > binding statements. There's a superficial connection, but they really
> > are quite different things.
> 
> There's a lot more similarities than you might think. For example,
> both of these will create "spam" as a local name, shadowing the
> global:
> 
> spam = 1
> def func1(spam):
>     print(spam)
> def func2():
>     spam = 2
>     print(spam)

Right. Function parameters are local variables. But you can't write:

def func(global spam):

to make it assign to a global instead of a local, or nonlocal, and you 
can't (currently) write:

def func(math.spam):

to make it assign to an attribute of the math module. It's a big 
conceptual leap to go from "formal parameters are always bound to local 
variables of the function" to "formal parameters of the function are 
bound to anything, anywhere".

I wonder whether any other language allows this?


> As will many other forms of assignment, like "import spam" or "with x
> as spam:". Some forms are more restricted than others ("import"
> requires a NAME, not an arbitrary expression), 

Yes, we could restrict the syntax to simple identifiers with a maximum 
of a single dot:

    def __init__(self, self.attr,   # allowed
                       self[0].foo(1)['key'],  # SyntaxError
                       ):

which will avoid the worst of the horrors.


[...]
> > You need to speak to more beginners if you think the connection between
> > spam.x and x is obvious:
> >
> >     def func(spam.x):
> >         print(x)
> >
> > Where is x declared? It looks like there's a local spam.x which isn't
> > used, and a global x that is. But that's completely wrong. Without the
> > context of somebody telling you "this is syntax for magically assigning
> > to self.attributes in the constructor", I believe this will be
> > unfathomable to the average non-expert.
> 
> Not sure what you mean. By my reading, that's exactly correct - there
> IS a global x that is being used here, 

No there isn't. "x" is the parameter name, even though it is written 
with a "spam." prefix. Let's go back to the original:

class MyClass:
    def __init__(self, self.attr):
        ...

instance = MyClass(42)

That binds 42 to the local variable "attr", as well as doing the 
self.attr=attr assignment. After all, surely we want to be able to refer 
to the parameter by its local variable, for efficiency (avoiding 
redundant look-ups of self), or to by-pass any __getattr__ on self.

Or to really push the message, think about calling it by keyword:

instance = MyClass(self.attr=42)  # No.
instance = MyClass(attr=42)  # Yes.

In other words, even though the formal parameter is written as 
"self.attr", the argument is bound to the local "attr".

Quite frankly, I thought this was so self-evident that it didn't even 
occur to me that anyone would have imagined that no local variable 
"attr" would be created. As weird as it is to have a parameter declared 
as "self.attr" but bound to "attr", that's not as weird as having a 
parameter declared as "self.attr" but not bound to any local at all.

(If instance attributes and locals used the same notation, namely a bare 
identifier, we could gloss over this distinction. But we're not using 
Java :-)


-- 
Steve

From shoyer at gmail.com  Fri Apr 28 12:30:07 2017
From: shoyer at gmail.com (Stephan Hoyer)
Date: Fri, 28 Apr 2017 09:30:07 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
Message-ID: <CAEQ_TvchYOLQCNp+X-yeFOBtvQ6PTUYhoMh0Wy+mtfxFp8+5jA@mail.gmail.com>

On Fri, Apr 28, 2017 at 4:55 AM, Tin Tvrtkovi? <tinchester at gmail.com> wrote:

> I'm going to posit we need declarative classes. (This is what a library
> like attrs provides, basically.) For a class to be declarative, it needs to
> be possible to inspect the class for its attributes and more.
>

> I'm putting forward three examples. These examples are based on attrs
> since that's what I consider to be the best way of having declarative
> classes in Python today.
>

My favorite example of declarative classes in Python today is Python
3.6.1's typing.NamedTuple:

class Employee(NamedTuple):
    """Represents an employee."""
    name: str
    id: int = 3

NamedTuple fixes many of the warts of the original namedtuple:
1. It supports docstrings out of the box, not requiring a subclass (which
require setting __slots__ to get right)
2. It supports default values.
3. It has a sane declarative syntax.
4. It's fields can be type checked.

What's not to love? There are two valid complaints [1] about namedtuple
that still apply:
1. It's a tuple, which brings along extra baggage (numeric
indexing/iteration, comparing equal to tuples, etc.)
2. It's immutable. Sometimes it's convenient to have a mutable class.

It seems like a simple solution would be make variants of NamedTuple that
solve each of these issues. We might call these collections.Struct and
collections.FrozenStruct.

[1] e.g., see https://glyph.twistedmatrix.com/2016/08/attrs.html


> Third example: I work at a mobile games company. The backends are Python,
> the games are written in other languages. The backends and the games share
> data structures - requests, responses, the save game, game data. I want to
> have a single source of truth for these structures, so I want to generate
> classes/structs in other languages from the backend Python classes.
>

If you're willing to define these classes outside of Python in a separate
file (which is quite reasonable if you need cross-language support), then
this is exactly the problem solved by protocol buffers (
https://developers.google.com/protocol-buffers/). So I'm not sure we really
need this fix in Python.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170428/967540bc/attachment.html>

From python-ideas at mgmiller.net  Fri Apr 28 13:00:56 2017
From: python-ideas at mgmiller.net (Mike Miller)
Date: Fri, 28 Apr 2017 10:00:56 -0700
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
 <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>
 <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
Message-ID: <c69c3ede-6882-dd5e-7e7d-23f165048bd1@mgmiller.net>


On 2017-04-28 06:07, Nick Coghlan wrote:
> ?the root cause (writing imperative initialisers is still
> part of the baseline recommendation for writing classes, and we don't
> offer any supporting infrastructure for avoiding that directly in the
> standard library)
>
> For a *lot* of classes, what we want to be able to define is:
>
> - a set of data fields
> - a basic initialiser to set those fields by name
> - a repr based on those fields (or a subset)
> - equality comparisons based on those fields (or a subset)
> - ordering comparisons based on those fields (or a subset)
> - conversion to a JSON-friendly dict
> - pickling support
> - copying support

Nice!  I've implemented a higher-level base class in a few projects that does 
just this.  Things like pickling and JSON I might want to put into "mixins", but 
yes the majority of these would be nice to have implemented by default 
"batteries included" style.  I prefer a base class to decorators since they seem 
to stay out of the way more:

     class Vehicle(BaseObject, Picklable):
         ?

That still leaves the problem of how to define instance attributes concisely. 
Perhaps a "defaults" member or marker, or one of the other examples in the 
larger thread would do.

From rosuav at gmail.com  Fri Apr 28 14:01:18 2017
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 29 Apr 2017 04:01:18 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <20170428153144.GG22525@ando.pearwood.info>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <CAO41-mNvRoaUoHR4Li9BiNRcpmgooOGZJJt-TQ_bftQEVYDDdw@mail.gmail.com>
 <CADwdpyZAQprLrQmCT9UfN3LQFRdvuKj4o9_TD-RTZ6LwHMrJuA@mail.gmail.com>
 <20170427232133.GB22525@ando.pearwood.info>
 <CAPTjJmrmeCU_7Eo8gv18YBBz-+ewR8Twhr3HokxUzJGwKCc1ug@mail.gmail.com>
 <20170428040609.GE22525@ando.pearwood.info>
 <CAPTjJmqfW3ORFZZhSYOuwsHqOCSSyYBoWoFnTu613=rpYM9W2g@mail.gmail.com>
 <20170428065723.GF22525@ando.pearwood.info>
 <CAPTjJmo+C50zsaeqUKv9riW7CKX8HjhVzhO5PtSG4ENOAZB=iA@mail.gmail.com>
 <20170428153144.GG22525@ando.pearwood.info>
Message-ID: <CAPTjJmpzzMp8r+nVFHPce-sdAT4jMk5_UkNcMwHTvCqZMKAaDA@mail.gmail.com>

On Sat, Apr 29, 2017 at 1:31 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Apr 28, 2017 at 05:23:59PM +1000, Chris Angelico wrote:
> __init__ is called, the argument 42 is bound to the formal parameter
> "attr", the assignment self.attr = attr is run, and THEN the body of the
> method is called. (Before you object, see below for justification for
> why there has to be both a local variable and an attribute.)
>
> If there is an exception, what do you think the stack trace will point
> to? It cannot point to a line of source code that says "self.attr =
> attr", because there is no such line. At best, it can point to the
> declaration "def __init__(self, self.attr):" although it might not even
> do that.

I don't see why it couldn't. You'll get an AttributeError trying to
set "attr", or maybe it'll be an error from inside __setattr__, but
either way, it's going to come from the 'def' line.

>> > Why should it be defined in terms of general assignment? That's the
>> > point I'm making. While function sigs are a form of assignment, they're
>> > more of a declaration than an executable statement like the other
>> > binding statements. There's a superficial connection, but they really
>> > are quite different things.
>>
>> There's a lot more similarities than you might think. For example,
>> both of these will create "spam" as a local name, shadowing the
>> global:
>>
>> spam = 1
>> def func1(spam):
>>     print(spam)
>> def func2():
>>     spam = 2
>>     print(spam)
>
> Right. Function parameters are local variables. But you can't write:
>
> def func(global spam):
>
> to make it assign to a global instead of a local, or nonlocal, and you
> can't (currently) write:
>
> def func(math.spam):
>
> to make it assign to an attribute of the math module. It's a big
> conceptual leap to go from "formal parameters are always bound to local
> variables of the function" to "formal parameters of the function are
> bound to anything, anywhere".
>
> I wonder whether any other language allows this?

It's a big conceptual leap to go from "iteration steps through a
sequence, binding a variable to successive items in it" to the full
flexibility of Python's for loop. Just look at this:

for idx, val in enumerate(iterable):

It's completely intuitive to an expert Python programmer because it's
a common idiom, but think about it. Firstly, you create an iterable as
a derivative of another iterable. We're building not just lists out of
lists, but lazy iterables out of each other. And that iterable yields
two-item tuples. Finally, we do an unpacking assignment, stashing the
two items into two separate names. And nobody is suggesting that we
should restrict this syntax to simple examples like this; there is
absolutely nothing wrong with having incredibly complicated and
ridiculous assignments inside a for loop. Why? Because a for loop does
simple assignment, nothing more, nothing less.

>> As will many other forms of assignment, like "import spam" or "with x
>> as spam:". Some forms are more restricted than others ("import"
>> requires a NAME, not an arbitrary expression),
>
> Yes, we could restrict the syntax to simple identifiers with a maximum
> of a single dot:
>
>     def __init__(self, self.attr,   # allowed
>                        self[0].foo(1)['key'],  # SyntaxError
>                        ):
>
> which will avoid the worst of the horrors.

Yeah. In that sense, it actually is more like decorator syntax, which
is defined as a strict subset of expression syntax - you have an
absolute guarantee that any legal decorator is semantically equivalent
to the same expression elsewhere, but there are lots of expressions
that you can't use in a decorator. I'd be fine with that. It's still
defined in terms of assignment, but your example would indeed give a
SyntaxError.

>> > You need to speak to more beginners if you think the connection between
>> > spam.x and x is obvious:
>> >
>> >     def func(spam.x):
>> >         print(x)
>> >
>> > Where is x declared? It looks like there's a local spam.x which isn't
>> > used, and a global x that is. But that's completely wrong. Without the
>> > context of somebody telling you "this is syntax for magically assigning
>> > to self.attributes in the constructor", I believe this will be
>> > unfathomable to the average non-expert.
>>
>> Not sure what you mean. By my reading, that's exactly correct - there
>> IS a global x that is being used here,
>
> No there isn't. "x" is the parameter name, even though it is written
> with a "spam." prefix. Let's go back to the original:
>
> class MyClass:
>     def __init__(self, self.attr):
>         ...
>
> instance = MyClass(42)
>
> That binds 42 to the local variable "attr", as well as doing the
> self.attr=attr assignment. After all, surely we want to be able to refer
> to the parameter by its local variable, for efficiency (avoiding
> redundant look-ups of self), or to by-pass any __getattr__ on self.

Waaaaaaaaaait. Why? If you're slapping it in there, you should have no
guarantee that it exists under any other name.

> Or to really push the message, think about calling it by keyword:
>
> instance = MyClass(self.attr=42)  # No.
> instance = MyClass(attr=42)  # Yes.
>
> In other words, even though the formal parameter is written as
> "self.attr", the argument is bound to the local "attr".
>
> Quite frankly, I thought this was so self-evident that it didn't even
> occur to me that anyone would have imagined that no local variable
> "attr" would be created. As weird as it is to have a parameter declared
> as "self.attr" but bound to "attr", that's not as weird as having a
> parameter declared as "self.attr" but not bound to any local at all.

No way is that self-evident. If you want something in two places, you
put it there yourself. The parameter puts the value into exactly one
place. In all my examples of equivalence, there was not a single hint
of a local name "attr". Maybe that's why you consider this weird?
Because your imagined semantics are NOT equivalent to assignment?

> (If instance attributes and locals used the same notation, namely a bare
> identifier, we could gloss over this distinction. But we're not using
> Java :-)

Haha. Yeah, well, that actually doesn't work - or at least, not in C++
(not sure about Java but I expect it's the same). You can have locals
and instance attributes with the same names, and then you refer to the
latter as "this->foo". So it still wouldn't help. :)

ChrisA

From p.f.moore at gmail.com  Fri Apr 28 15:31:49 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 28 Apr 2017 20:31:49 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
 <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>
 <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
Message-ID: <CACac1F88cpCkAOtZvgu+6LNhBG=yXOgBBo4eKNu6p-z=Va_JtA@mail.gmail.com>

On 28 April 2017 at 14:07, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Am I missing some point?
>
> Yes, the point I attempted to raise earlier: at the language design
> level, "How do we make __init__ methods easier to write?" is the
> *wrong question* to be asking. It's treating the symptom (writing an
> imperative initialiser is repetitive when it comes to field names)
> rather than the root cause (writing imperative initialisers is still
> part of the baseline recommendation for writing classes, and we don't
> offer any supporting infrastructure for avoiding that directly in the
> standard library)
[...]

Ah, OK. I see what you're saying here. I agree, that's the direction
we should be looking in. I'd sort of lumped all of that side of things
in my mind under the header of "must take a look at attrs" and left it
at that for now. My mistake.

So basically yes, I agree we should have better means for writing
common class definition patterns in a declarative way, retaining the
current underlying mechanisms while making it so that people typically
don't need to interact with them. I suspect that those means will
probably take the form of stdlib/builtin support (decorators, custom
base classes, etc) but the design will need some thrashing out. Beyond
that, I don't really have much more to add until I've done some
research into the current state of the art, in the form of libraries
like attrs (and Stephan Hoyer mentioned typing.NamedTuple).

Paul

From python at lucidity.plus.com  Fri Apr 28 18:04:00 2017
From: python at lucidity.plus.com (Erik)
Date: Fri, 28 Apr 2017 23:04:00 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
 <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
 <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>
Message-ID: <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>

On 28/04/17 10:47, Paul Moore wrote:
> On 28 April 2017 at 00:18, Erik <python at lucidity.plus.com> wrote:
>>> The semantics are very different and there's little or no connection
>>> between importing a module and setting an attribute on self.
>>
>> At the technical level of what goes on under the covers, yes. At the higher
>> level of what the words mean in spoken English, it's really not so different
>> a concept.
>
> I disagree. If you were importing into the *class* (instance?) I might
> begin to see a connection, but importing into self?

I know you already understand the following, but I'll spell it out 
anyway. Here's a module:

-----------------
$ cat foo.py
def foo():
   global sys
   import sys

   current_namespace = set(globals().keys())
   print(initial_namespace ^ current_namespace)

def bar():
   before_import = set(locals().keys())
   import os
   after_import = set(locals().keys())
   print(before_import ^ after_import)

initial_namespace = set(globals().keys())
-----------------

Now, what happens when I run those functions:

$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
 >>> import foo
 >>> foo.foo()
{'sys', 'initial_namespace'}
 >>> foo.bar()
{'before_import', 'os'}
 >>>

... so the net effect of "import" is to bind an object into a namespace 
(a dict somewhere). In the case of 'foo()' it's binding the module 
object for "sys" into the dict of the module object that represents 
'foo.py'. In the case of 'bar()' it's binding the module object for "os" 
into the dict representing the local namespace of the current instance 
of the bar() call.

Isn't binding an object to a namespace the same operation that 
assignment performs?

So it's a type of assignment, and one that doesn't require the name to 
be spelled twice in the current syntax (and that's partly why I took 
offense at a suggestion - not by you - that I was picking "random or 
arbitrary" keywords. I picked it for that specific reason).

I realize that there are other semantic changes (importing a module 
twice doesn't do anything - and specifically repeated "import * from 
mod" will not do anything if the module mutates) - and perhaps this is 
your point.

> Also, if you try to make the obvious generalisations (which you'd *have* to be able to
> make due to the way Python works) things quickly get out of hand:
>
> def __init__(self, a):
>     self import a

self.a = a

>
> OK, but self is just a variable name, so we can reasonably use a different name:
>
> def __init__(foo, a):
>     foo import a

foo.a = a

> So the syntax is <variable> import <var-list>
>
> Presumably the following also works, because there's nothing special
> about parameters?
>
> def __init__(x, a):
>     calc = a**2
>     x import calc

x.calc = calc

> And of course there's nothing special about __init__
>
> def my_method(self, a):
>     self import a

self.a = a

> Or indeed about methods
>
> def standalone(a, b):
>     a import b

a.b = b

> or statements inside functions:
>
> if __name __ == '__main__:
>     a = 12
>     b = 13
>     a import b

a.b = b

> Hmm, I'd hope for a type error here. But what types would be allowed
> for a?

I think you're assuming I'm suggesting some sort of magic around "self" 
or some such thing. I'm not. I've written above exactly what I would 
expect the examples to be equivalent to. It's just an assignment which 
doesn't repeat the name (and in the comma-separated version allows 
several names to be assigned using compact syntax without spelling them 
twice, which is where this whole thing spawned from).

> See what I mean? Things get out of hand *very* fast.

I don't see how that's getting "out of hand". The proposal is nothing 
more complicated than a slightly-different spelling of assignment. It 
could be done today with a text-based preprocessor which converts the 
proposed form to an existing valid syntax. Therefore, if it's "out of 
hand" then so is the existing assignment syntax ;)

FWIW, I should probably state for the record that I'm not actually 
pushing for _anything_ right now. I'm replying to questions asked and 
also to statements made which I think have missed the point of what I 
was trying to say earlier. So I'm just engaging in the conversation at 
this point - if it appears confrontational then it's not meant to.

> To summarise:
>
> 1. There's some serious technical issues with your proposal, which as
> far as I can see can only be solved by arbitrary restrictions on how
> it can be used

To be honest, I still don't understand what the serious technical issues 
are (other than the parser probably doesn't handle this sort of 
keyword/operator hybrid!). Is it just that I'm seeing the word "import" 
in this context as a type of assignment and you're seeing any reference 
to the word "import" as being a completely different type of operation 
that comes with baggage?

Regards, E.

From levkivskyi at gmail.com  Fri Apr 28 19:51:04 2017
From: levkivskyi at gmail.com (Ivan Levkivskyi)
Date: Sat, 29 Apr 2017 01:51:04 +0200
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
 <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
 <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>
 <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
Message-ID: <CAOMjWk=fg5TTDgtdkv86x1-6dZjcad41M_22LUyX4+AuwTU4SQ@mail.gmail.com>

typing.NamedTuple was already mentioned in this discussion, I just would
like to add few comments:

1. I think that changing Python syntax to support declarative classes is
not a realistic option in nearby future.
In contrast, there is an existing syntax change - variable annotations -
that can be actively experimented with.
Variable annotations are designed to play well with metaclass machinery,
for example, they will respect
__prepare__ method, if it injects __annotations__ as a custom mapping, etc.

2. Possible strategy for declarative classes could be developing
independent *composable* mixins/traits:

  @comparable
  @pickleable
  class MyClass:
      x: int
      y: int

or

  class MyClass(Comparable, Pickleable):
      x: int
      y: int

The problem with the second approach is that it can lead to metaclass
conflicts. A possible solution will
be to provide a standard metaclass for mixins in stdlib.

3. Composability could be a problem if only normal variable assignments are
used: after one mixin transforms the class,
the original declarations are lost. Simple possible solution would be to
agree that mixins should *not* modify
__annotations__, only __dict__. Currently, PEP 484 and PEP 526 specify that
preferable use of annotations is type
declarations. However, I think this can be generalized to just
declarations. For example, by providing a special typing
construct that will allow additional info apart from type:

  def large(val):
      return val > 42

  @comparable
  @pickleable
  class MyClass
      x: Spec(int, validator=large)
      y: Spec(List[int], default_factory=list)


--
Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170429/109b8a11/attachment.html>

From p.f.moore at gmail.com  Sat Apr 29 07:02:43 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 29 Apr 2017 12:02:43 +0100
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
 <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
 <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>
 <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
Message-ID: <CACac1F9-nKZD331TnNto39qrM9eM96xmSfw3vCDOGvs-sjcKhg@mail.gmail.com>

On 28 April 2017 at 23:04, Erik <python at lucidity.plus.com> wrote:
>> See what I mean? Things get out of hand *very* fast.
>
> I don't see how that's getting "out of hand". The proposal is nothing more
> complicated than a slightly-different spelling of assignment. It could be
> done today with a text-based preprocessor which converts the proposed form
> to an existing valid syntax. Therefore, if it's "out of hand" then so is the
> existing assignment syntax ;)

Well with your clarification, it's now clear to me that your proposal is for

    <obj> import <var1>, <var2> ...

to be equivalent to

    _tmp = <obj>
    _tmp.var1 = var1
    _tmp.var2 = var2
    ...
    del _tmp

Please correct me if I'm wrong here - but that's what I get from your
comments. Note that the way I've described the proposal allows for an
*expression* on the left of import - that's simply because I see no
reason not to allow that (the whole "arbitrary restrictions" thing
again).

So based on the above, you're proposing reusing the import keyword for
a series of attribute assignments, on the basis that the current
meaning of import is to assign values to a number of attributes of the
current namespace. However, that misses the fact that the main point
of the current import statement is to do the *import* - the RHS is a
list of module names. The assignment is just how we get access to the
modules that have been imported. So by "reusing" the import keyword
purely by analogy with the assignment part of the current semantics,
you're ignoring the fundamental point of an import, which is to load a
module.

So based on your clarification, I'm now solidly against reusing the
"import" keyword, on the basis that your proposed syntax doesn't
import any modules.

On the other hand, the construct you describe above of repeated
attribute assignments is only really common in __init__ functions.
It's similar to the sort of "implied target" proposals that have been
raised before (based on the Pascal "with" statement and similar
constructs):

    SOME_KEYWORD <obj>:
        .var1 = var1
        .var2 = var2

Your construct is more limited in that it doesn't allow the RHS of the
assignments to be anything other than a variable with the same name as
the attribute (unless you extend it with "expr as var", which I can't
recall if you included in your proposal), nor does it allow for
statements other than assignments in the block. And the implied target
proposal doesn't really avoid the verbosity that prompted your
proposal. So while it's similar in principle, it may not actually help
much in your case.

So the reason I was saying things got "out of hand" was because the
final version I got to had absolutely nothing to do with an import (in
my view) - but on reflection, you're right it's no different than the
self version. It's just that the self version also has nothing to do
with an import! However, in the original context, it's easier to see
"name injection into a namespace" as a common factor, and miss
"importing a module" as the key point (at least it was easier for me
to miss that - you may have seen that and been happy that your
proposal was unrelated to modules even though it used the import
keyword).

I agree with Nick - the place we should be looking for improvements
here is in trying to find abstractions for common patterns of class
creation. Instead of looking at how to make the low-level mechanisms
like __init__ easier to write, we should be concentrating on making it
rare for people to *need* to use the low-level forms (while still
leaving them present for specialised cases).

Paul

From steve at pearwood.info  Sat Apr 29 08:34:39 2017
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Apr 2017 22:34:39 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
References: <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
 <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
 <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>
 <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
Message-ID: <20170429123438.GH22525@ando.pearwood.info>

On Fri, Apr 28, 2017 at 11:04:00PM +0100, Erik wrote:

> Isn't binding an object to a namespace the same operation that 
> assignment performs?

Yes.

> So it's a type of assignment, and one that doesn't require the name to 
> be spelled twice in the current syntax (and that's partly why I took 
> offense at a suggestion - not by you - that I was picking "random or 
> arbitrary" keywords. I picked it for that specific reason).

That was me.

You took *offense*?

For somebody who made it clear that you didn't care about this "self 
import name" suggestion, you sure are touchy to have it criticised.

Assignment is the least interesting and important thing that the import 
statement does. Many keywords do an name binding operation, not just 
import, but also:

    for, with, except, def, class 

and most relevant to my comment:

    del 

is actually defined by the documentation as a name binding operation. 

https://docs.python.org/3/reference/executionmodel.html

I'm sorry that the joke was too subtle, I did put a wink after it, but 
you deleted it from your reply.

The point is, import's name binding is the least interesting and least 
important thing that it does. We could have defined import to be a 
function that returns a module object, or a tuple of objects, and left 
the binding up to the caller:

    # What If import were a function instead of a statement?
    math = import("math")
    sin, cos = import("math", "sin", "cos")

That's rather similar to what __import__ does, but of course import as a 
statement is a much better design. Nevertheless, we could take away the 
name-binding aspect of import, and it would still be recognisable as 
performing an import: it would search the module cache, search the path, 
load modules, compile the code, create module objects, and all the other 
things needed to import a module. The name binding (assignment) is just 
an convenience.

But take away the *import functionality*, and "import" is no longer an 
appropriate name. Why is it called import when it doesn't import 
anything? Because of the name binding? Well, these keywords all do name 
binding too, why can't we use them?

    self for attr
    self except attr
    self def attr
    self class attr

etc. I'm sorry that my posting style rubbed you the wrong way and made 
you defensive, by I stand by my comment: chosing import as the keyword 
to do something completely unrelated to importing is an arbitrary 
choice, and we might as well choose del (or def, or for) and save some 
typing.

If Python were the sort of language to use more symbols rather than 
names, like Perl, we could invent a bind to the left arrow operator:

    self <- a, b, c

Well, that wouldn't work, because of operator precedence. But you get 
the idea. Or we could bind to the right:

    a, b, c -> self


Make of them what you will.


-- 
Steve

From tinchester at gmail.com  Sat Apr 29 08:53:26 2017
From: tinchester at gmail.com (=?UTF-8?Q?Tin_Tvrtkovi=C4=87?=)
Date: Sat, 29 Apr 2017 12:53:26 +0000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
 <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>
 <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
Message-ID: <CAEJ-mUrpbrjj6ZS_Btz6a8GY2uBeBGM=BSsuiRKod7XW704+aA@mail.gmail.com>

On Fri, Apr 28, 2017 at 3:07 PM Nick Coghlan <ncoghlan at gmail.com> wrote:

> Yes, the point I attempted to raise earlier: at the language design
> level, "How do we make __init__ methods easier to write?" is the
> *wrong question* to be asking. It's treating the symptom (writing an
> imperative initialiser is repetitive when it comes to field names)
> rather than the root cause (writing imperative initialisers is still
> part of the baseline recommendation for writing classes, and we don't
> offer any supporting infrastructure for avoiding that directly in the
> standard library) [...]
>


Very well put. I also can't help but hope these efforts lead to Python also
acquiring better tools for dealing with structured data (instances of
classes and enumerations) down the road. Like a Pythonic version of Rust's
match, for example. That would be really something.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170429/7030efd2/attachment.html>

From ncoghlan at gmail.com  Sat Apr 29 22:25:06 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Apr 2017 12:25:06 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <c69c3ede-6882-dd5e-7e7d-23f165048bd1@mgmiller.net>
References: <mailman.5562.1493362652.2950.python-ideas@python.org>
 <CAEJ-mUpFBGuNGGfH9u0hLWS86s9ytcJkDUgCQwEcxJBEP-oG1w@mail.gmail.com>
 <CACac1F8bY5qHS-BxBpFCL=XmRC2_KFwC=aZ4c-e=XJGMSw7m8A@mail.gmail.com>
 <CADiSq7fibwfkJx8XH2CaCOfVqvQ+yB75EVceyi5dRLbPVafp4g@mail.gmail.com>
 <c69c3ede-6882-dd5e-7e7d-23f165048bd1@mgmiller.net>
Message-ID: <CADiSq7d+w3yFtttcbwVQsenfTZoLLmqkX0dbZWy4+jK7b4iSEA@mail.gmail.com>

On 29 April 2017 at 03:00, Mike Miller <python-ideas at mgmiller.net> wrote:
> On 2017-04-28 06:07, Nick Coghlan wrote:
>> For a *lot* of classes, what we want to be able to define is:
>>
>> - a set of data fields
>> - a basic initialiser to set those fields by name
>> - a repr based on those fields (or a subset)
>> - equality comparisons based on those fields (or a subset)
>> - ordering comparisons based on those fields (or a subset)
>> - conversion to a JSON-friendly dict
>> - pickling support
>> - copying support
>
>
> Nice!  I've implemented a higher-level base class in a few projects that
> does just this.  Things like pickling and JSON I might want to put into
> "mixins", but yes the majority of these would be nice to have implemented by
> default "batteries included" style.  I prefer a base class to decorators
> since they seem to stay out of the way more:
>
>     class Vehicle(BaseObject, Picklable):
>         ?

Putting any questions of aesthetics aside, the main argument in favour
of using a base class and __init_subclass__ for autoclass definitions
would be the fact that further subclasses would automatically get the
magic method construction executed, rather than requiring that the
class decorator be repeated on the subclass.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Apr 29 22:36:34 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 30 Apr 2017 12:36:34 +1000
Subject: [Python-ideas] Augmented assignment syntax for objects.
In-Reply-To: <CAOMjWk=fg5TTDgtdkv86x1-6dZjcad41M_22LUyX4+AuwTU4SQ@mail.gmail.com>
References: <e4c1b637-4d43-aff2-1e59-931d483b443f@lucidity.plus.com>
 <20170425025305.GC20708@ando.pearwood.info>
 <8ecf5b9c-291d-5068-77eb-b50bdd822bd6@mgmiller.net>
 <f247b7ae-0b9b-8515-35a8-10696a918a17@brice.xyz>
 <CACac1F8VNjFzDG3Q1C0wHfuRGwye52sout0WXnMrV+uLyr8d4w@mail.gmail.com>
 <96e3f449-2b99-ff53-87e7-7a46c95a8506@mgmiller.net>
 <ec4aff27-798e-6312-4de7-852f43016cb2@lucidity.plus.com>
 <20170427224331.GA22525@ando.pearwood.info>
 <89690c09-6cdc-ad56-0d24-ab6b8a540268@lucidity.plus.com>
 <CACac1F_ocDJcUwtxjXSrEjq_MYiE5vBYNH2wEmenbk+DG0xuzw@mail.gmail.com>
 <46aad916-db9a-53ba-92c0-02355ca7d998@lucidity.plus.com>
 <CAOMjWk=fg5TTDgtdkv86x1-6dZjcad41M_22LUyX4+AuwTU4SQ@mail.gmail.com>
Message-ID: <CADiSq7dzVhBWvkOBy7bpr-mUXA2AGiexqo3=WPNHi4tUchkuig@mail.gmail.com>

On 29 April 2017 at 09:51, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:
> typing.NamedTuple was already mentioned in this discussion, I just would
> like to add few comments:
>
> 1. I think that changing Python syntax to support declarative classes is not
> a realistic option in nearby future.
> In contrast, there is an existing syntax change - variable annotations -
> that can be actively experimented with.
> Variable annotations are designed to play well with metaclass machinery, for
> example, they will respect
> __prepare__ method, if it injects __annotations__ as a custom mapping, etc.
>
> 2. Possible strategy for declarative classes could be developing independent
> *composable* mixins/traits:
>
>   @comparable
>   @pickleable
>   class MyClass:
>       x: int
>       y: int
>
> or
>
>   class MyClass(Comparable, Pickleable):
>       x: int
>       y: int
>
> The problem with the second approach is that it can lead to metaclass
> conflicts. A possible solution will
> be to provide a standard metaclass for mixins in stdlib.

Between __init_subclass__, __set_name__, __annotations__, and
guaranteed order preservation for class namespaces, Python 3.6 has
provided some pretty powerful tools for folks to experiment with new
approaches to declarative class definitions.

One point worth considering is that given standard library
implementations of the explicit class decorator forms (e.g in the
`types` module, or in the modules defining the relevant protocols), it
should be relatively straightforward to create mixin classes that use
__init_subclass__ to implicitly apply the explicit decorators.

The same doesn't hold the other way around - given a mixin, it's
tricky to convert that to a class decorator form.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guido at python.org  Sun Apr 30 13:07:09 2017
From: guido at python.org (Guido van Rossum)
Date: Sun, 30 Apr 2017 10:07:09 -0700
Subject: [Python-ideas] Proposed PEP 484 addition: describe a way of
 annotating decorated declarations
Message-ID: <CAP7+vJLCLW-g0aP3JpyRGVe0hx=Pd=Z5WKws6H61a10HC5jGhg@mail.gmail.com>

There's a PR to the peps proposal here:
https://github.com/python/peps/pull/242

The full text of the current proposal is below. The motivation for this is
that for complex decorators, even if the type checker can figure out what's
going on (by taking the signature of the decorator into account), it's
sometimes helpful to the human reader of the code to be reminded of the
type after applying the decorators (or a stack thereof). Much discussion
can be found in the PR. Note that we ended up having `Callable` in the type
because there's no rule that says a decorator returns a function type (e.g.
`property` doesn't).

This is a small thing but I'd like to run it by a larger audience than the
core mypy devs who have commented so far.

Here's the proposed text (wordsmithing suggestions in the PR please):

+Decorators
+----------
+
+Decorators can modify the types of the functions or classes they
+decorate. Use the ``decorated_type`` decorator to declare the type of
+the resulting item after all other decorators have been applied::
+
+ from typing import ContextManager, Iterator, decorated_type
+ from contextlib import contextmanager
+
+ class DatabaseSession: ...
+
+ @decorated_type(Callable[[str], ContextManager[DatabaseSession]])
+ @contextmanager
+ def session(url: str) -> Iterator[DatabaseSession]:
+ s = DatabaseSession(url)
+ try:
+ yield s
+ finally:
+ s.close()
+
+The argument of ``decorated_type`` is a type annotation on the name
+being declared (``session``, in the example above). If you have
+multiple decorators, ``decorated_type`` must be topmost. The
+``decorated_type`` decorator is invalid on a function declaration that
+is also decorated with ``overload``, but you can annotate the
+implementation of the overload series with ``decorated_type``.
+

-- 
--Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170430/92308f9d/attachment.html>