From stephen at xemacs.org  Thu Dec  1 02:38:21 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 01 Dec 2011 10:38:21 +0900
Subject: [Python-ideas] Is there a reason
	why	file.readlines()	doesn't/can't return an iterator?
In-Reply-To: <jb5qja$3k1$1@dough.gmane.org>
References: <CAFYqXL9bQxV7c3j9yecRTK-eRKpq2-B2f2_rEqnT8rxKZWOktQ@mail.gmail.com>
	<jb5crq$j0m$1@dough.gmane.org> <4ED650FF.9020705@netwok.org>
	<jb5m5r$vrl$1@dough.gmane.org> <4ED65DF6.3060900@netwok.org>
	<jb5qja$3k1$1@dough.gmane.org>
Message-ID: <87d3c9nkaa.fsf@uwakimon.sk.tsukuba.ac.jp>

Peter Otten writes:
 > ?ric Araujo wrote:
 > 
 > > Okay.  You can open a report on bugs.python.org to ask that the doc for
 > > readlines mention list(fp) as an alternative.
 > 
 > That would make sense to me only if readlines() were deprecated.

It makes sense to me.  Something like

    Note that "for line in fp:" (to iterate over the lines of the
    file) and "list(fp)" (to get a list of all lines in a file) are
    idiomatic Python.

to remind readers (especially new users) of the iterator concept,
which is even today not so familiar to beginning programmers.



From tjreedy at udel.edu  Fri Dec  2 04:13:03 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 01 Dec 2011 22:13:03 -0500
Subject: [Python-ideas] Is there a reason why file.readlines()
 doesn't/can't return an iterator?
In-Reply-To: <87d3c9nkaa.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAFYqXL9bQxV7c3j9yecRTK-eRKpq2-B2f2_rEqnT8rxKZWOktQ@mail.gmail.com>
	<jb5crq$j0m$1@dough.gmane.org> <4ED650FF.9020705@netwok.org>
	<jb5m5r$vrl$1@dough.gmane.org> <4ED65DF6.3060900@netwok.org>
	<jb5qja$3k1$1@dough.gmane.org>
	<87d3c9nkaa.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <jb9fo4$26m$1@dough.gmane.org>

On 11/30/2011 8:38 PM, Stephen J. Turnbull wrote:
> Peter Otten writes:
>   >  ?ric Araujo wrote:
>   >
>   >  >  Okay.  You can open a report on bugs.python.org to ask that the doc for
>   >  >  readlines mention list(fp) as an alternative.
>   >
>   >  That would make sense to me only if readlines() were deprecated.
>
> It makes sense to me.  Something like
>
>      Note that "for line in fp:" (to iterate over the lines of the
>      file) and "list(fp)" (to get a list of all lines in a file) are
>      idiomatic Python.
>
> to remind readers (especially new users) of the iterator concept,
> which is even today not so familiar to beginning programmers.

See http://bugs.python.org/issue13510 ,
where I suggested something like that.

-- 
Terry Jan Reedy




From matthew at woodcraft.me.uk  Fri Dec  2 22:02:05 2011
From: matthew at woodcraft.me.uk (Matthew Woodcraft)
Date: Fri, 02 Dec 2011 21:02:05 +0000
Subject: [Python-ideas] Is there a reason why file.readlines()
 doesn't/can't return an iterator?
In-Reply-To: <jb5crq$j0m$1@dough.gmane.org>
References: <CAFYqXL9bQxV7c3j9yecRTK-eRKpq2-B2f2_rEqnT8rxKZWOktQ@mail.gmail.com>
	<jb5crq$j0m$1@dough.gmane.org>
Message-ID: <jbbecd$n2q$1@dough.gmane.org>

On 2011-11-30 13:59, Peter Otten wrote:
> My observation on the Tutor mailing list is that there are no valid uses of 
> readlines(). It's just easier to discover the readlines() method than to 
> find out that you can iterate over the file directly.

In 2.x, iterating directly can behave unexpectedly if the file object is
something other than a regular file; see eg
http://bugs.python.org/issue1633941
http://bugs.python.org/issue3907
http://utcc.utoronto.ca/~cks/space/blog/python/FileIteratorProblems

readline() and readlines() don't have these problems.

-M-



From tjreedy at udel.edu  Fri Dec  2 22:55:23 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 02 Dec 2011 16:55:23 -0500
Subject: [Python-ideas] Is there a reason why file.readlines()
 doesn't/can't return an iterator?
In-Reply-To: <jbbecd$n2q$1@dough.gmane.org>
References: <CAFYqXL9bQxV7c3j9yecRTK-eRKpq2-B2f2_rEqnT8rxKZWOktQ@mail.gmail.com>
	<jb5crq$j0m$1@dough.gmane.org> <jbbecd$n2q$1@dough.gmane.org>
Message-ID: <jbbhgg$e7f$1@dough.gmane.org>

On 12/2/2011 4:02 PM, Matthew Woodcraft wrote:
> On 2011-11-30 13:59, Peter Otten wrote:
>> My observation on the Tutor mailing list is that there are no valid uses of
>> readlines(). It's just easier to discover the readlines() method than to
>> find out that you can iterate over the file directly.
>
> In 2.x, iterating directly can behave unexpectedly if the file object is
> something other than a regular file; see eg
> http://bugs.python.org/issue1633941
> http://bugs.python.org/issue3907

Both are fixed in 3.x

> http://utcc.utoronto.ca/~cks/space/blog/python/FileIteratorProblems
>
> readline() and readlines() don't have these problems.
>
> -M-


-- 
Terry Jan Reedy



From bauertomer at gmail.com  Sat Dec  3 01:12:58 2011
From: bauertomer at gmail.com (T.B.)
Date: Sat, 03 Dec 2011 02:12:58 +0200
Subject: [Python-ideas] Different bases format specification
Message-ID: <4ED9698A.6010609@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111203/7624e07f/attachment.html>

From python at mrabarnett.plus.com  Sat Dec  3 02:26:30 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 03 Dec 2011 01:26:30 +0000
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <4ED9698A.6010609@gmail.com>
References: <4ED9698A.6010609@gmail.com>
Message-ID: <4ED97AC6.9000209@mrabarnett.plus.com>

On 03/12/2011 00:12, T.B. wrote:
> I will start by stating that it's not my original idea, but taken from
> Erlang. See Erlang's io:format documentation here:
> http://www.erlang.org/doc/man/io.html#format-1 and notice the 'B'
> control sequence.
>
> I would like to have an easy built-in way to print integers in different
> bases (radices). There are so many half baked solutions out there:
> http://bugs.python.org/issue6783
> http://stackoverflow.com/questions/2267362/convert-integer-to-a-string-in-a-given-numeric-base-in-python
> http://stackoverflow.com/questions/2063425/python-elegant-inverse-function-of-intstring-base
> http://code.activestate.com/recipes/65212/
>
> I suggest using the precision field in the format specification for
> integers for that.
[snip]

I think that the precision field should be used only for the precision
and that sometimes using it for something completely different is a bad
idea.


From ncoghlan at gmail.com  Sat Dec  3 02:31:37 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Dec 2011 11:31:37 +1000
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <4ED9698A.6010609@gmail.com>
References: <4ED9698A.6010609@gmail.com>
Message-ID: <CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>

On Sat, Dec 3, 2011 at 10:12 AM, T.B. <bauertomer at gmail.com> wrote:
> I suggest using the precision field in the format specification for integers
> for that.

Supporting arbitrary bases for string formatting has been discussed
and rejected in the past (both in the context of PEP 3101's
introduction of new string formatting and on other occasions).

Nobody has ever produced convincing use cases for natively supporting
formatting with bases other than binary, octal, decimal and
hexadecimal. Accordingly, those 4 are supported explicitly via the
'b', 'o', 'd' and 'x'/'X' formatting codes, while other formats still
require an explicit conversion function.

As for "Why Not?"

1. 'd' stands for decimal. If support for arbitrary bases were added,
it would need to be as a separate format code (e.g. 'i' for integer)

2. The explicit 'b', 'o' and 'x' codes are related to integer literal
notation (0b10, 0o777, 0x1F), not to the second argument to int()

3. The use cases just aren't that strong. When you start dealing with
base36 and base64, you're not talking about formatting numbers for
human readers any more, you're talking about encoding numbers as short
pieces of text. Better to let people decide exactly the behaviour they
want by coding it themselves.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From bruce at leapyear.org  Sat Dec  3 03:19:24 2011
From: bruce at leapyear.org (Bruce Leban)
Date: Fri, 2 Dec 2011 18:19:24 -0800
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <4ED9698A.6010609@gmail.com>
References: <4ED9698A.6010609@gmail.com>
Message-ID: <CAGu0Ans+U83nM8vFYHEq6xsF9MRm3MCs=rockj5T6H5XPmo-iQ@mail.gmail.com>

On Fri, Dec 2, 2011 at 4:12 PM, T.B. <bauertomer at gmail.com> wrote:

>  I suggest using the precision field in the format specification for
> integers for that. Examples:
>
> "{:.16d}".format(31) #Prints '1f'
>
> "{:.2d}".format(-19) # Prints '-10011'
>
> <snip>
>
I don't think this belongs in format.

>  P.S. Bonus question: What "{:.-909d}".format(42) would print?
>
Any proposal which includes an inscrutable example doesn't bode well for
the usability of the feature. :-) Sure, negative bases are mathematically
meaningful but are they useful in Python? And why not complex bases then?
Or did you have something else strange in mind?

If there's enough need for encoding in different bases, including a
standard version of format_integer_in_base makes a lot more sense. We could
write format_integer_in_base(15, 16) to get "F" and
format_integer_in_base(64, "A23456789TJQK") to get "4K". But note that
standard base 64 encoding is not at all the same -- this function encodes
starting at LSB while base 64 encodes at word boundaries.

Finally, note that if you really want to mangle format strings you can do
it without changing the library. Just write it this
way "{:.16d}".format(arbitrarybase(31)) where you have defined

class arbitrarybase:
    def __format__(self, format_spec):
        return format_integer_in_base(parse format spec etc.)


--- Bruce
Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111202/ccdca990/attachment.html>

From bauertomer at gmail.com  Sat Dec  3 03:45:59 2011
From: bauertomer at gmail.com (T.B.)
Date: Sat, 03 Dec 2011 04:45:59 +0200
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <CAGu0Ans+U83nM8vFYHEq6xsF9MRm3MCs=rockj5T6H5XPmo-iQ@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CAGu0Ans+U83nM8vFYHEq6xsF9MRm3MCs=rockj5T6H5XPmo-iQ@mail.gmail.com>
Message-ID: <4ED98D67.8070006@gmail.com>

Tip for self: No more HTML e-mails.

On 2011-12-03 04:19, Bruce Leban wrote:
>
>>     P.S. Bonus question: What "{:.-909d}".format(42) would print?
>
> Any proposal which includes an inscrutable example doesn't bode well for
> the usability of the feature. :-) Sure, negative bases are
> mathematically meaningful but are they useful in Python? And why not
> complex bases then? Or did you have something else strange in mind?
>
My intention will be clear after reading 
http://bugs.python.org/issue2844. It will also ruin the surprise of 
figuring out alone.

Regards,
TB


From bauertomer at gmail.com  Sat Dec  3 04:16:00 2011
From: bauertomer at gmail.com (T.B.)
Date: Sat, 03 Dec 2011 05:16:00 +0200
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
Message-ID: <4ED99470.20205@gmail.com>


On 2011-12-03 03:31, Nick Coghlan wrote:
> On Sat, Dec 3, 2011 at 10:12 AM, T.B.<bauertomer at gmail.com>  wrote:
>> I suggest using the precision field in the format specification for integers
>> for that.
>
> Supporting arbitrary bases for string formatting has been discussed
> and rejected in the past (both in the context of PEP 3101's
> introduction of new string formatting and on other occasions).
>
> Nobody has ever produced convincing use cases for natively supporting
> formatting with bases other than binary, octal, decimal and
> hexadecimal. Accordingly, those 4 are supported explicitly via the
> 'b', 'o', 'd' and 'x'/'X' formatting codes, while other formats still
> require an explicit conversion function.

For weird math scenarios I know there are already many modules and 
packages. But what about ternary? 
en.wikipedia.org/wiki/Ternary_numeral_system has some points that 
include that base 9 and 27 are used [no citation].

> As for "Why Not?"
>
> 1. 'd' stands for decimal. If support for arbitrary bases were added,
> it would need to be as a separate format code (e.g. 'i' for integer)
>
> 2. The explicit 'b', 'o' and 'x' codes are related to integer literal
> notation (0b10, 0o777, 0x1F), not to the second argument to int()
>
That one reason I wrote: "It might be a nice mnemonic using 'b' instead, 
standing for 'base'. Then the default base will be 2."
Anyway, I think there should be 'B' and 'O' presentation types, that
will be used for outputting '0B' and '0O' prefixes.

Thanks for your reply,
TB


From ncoghlan at gmail.com  Sat Dec  3 05:31:51 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Dec 2011 14:31:51 +1000
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <4ED99470.20205@gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
Message-ID: <CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>

On Sat, Dec 3, 2011 at 1:16 PM, T.B. <bauertomer at gmail.com> wrote:
> For weird math scenarios I know there are already many modules and packages.
> But what about ternary? en.wikipedia.org/wiki/Ternary_numeral_system has
> some points that include that base 9 and 27 are used [no citation].

Weird math scenarios are no justification for changing the behaviour
of a builtin type. b/o/d/x/X cover all the common use cases,
everything else can be handled by libraries (including providing
custom string.Formatter subclasses).

Code and functionality are not free - we need solid gains in easier
(or otherwise improved) coding and maintenance for real world problems
before we add more of either.

>> 2. The explicit 'b', 'o' and 'x' codes are related to integer literal
>> notation (0b10, 0o777, 0x1F), not to the second argument to int()
>>
> That one reason I wrote: "It might be a nice mnemonic using 'b' instead,
> standing for 'base'. Then the default base will be 2."

And the symmetry with the integer literal codes will still be lost.

> Anyway, I think there should be 'B' and 'O' presentation types, that
> will be used for outputting '0B' and '0O' prefixes.

If you really want that (Why would you?) and so long as the numbers
aren't negative:

    "0B{:b}".format(number)
    "0O{:o}".format(number)

The only reason 'X' is provided for hexadecimal formatting is to
capitalize the letters that appear within the number itself.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From bauertomer at gmail.com  Sat Dec  3 06:06:42 2011
From: bauertomer at gmail.com (T.B.)
Date: Sat, 03 Dec 2011 07:06:42 +0200
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
Message-ID: <4ED9AE62.5030303@gmail.com>



On 2011-12-03 06:31, Nick Coghlan wrote:
>>> 2. The explicit 'b', 'o' and 'x' codes are related to integer literal
>>> notation (0b10, 0o777, 0x1F), not to the second argument to int()
>>>
>> That one reason I wrote: "It might be a nice mnemonic using 'b' instead,
>> standing for 'base'. Then the default base will be 2."
>
> And the symmetry with the integer literal codes will still be lost.

All is not lost. "{:b}".format(num) would still print numbers in binary 
notion. Only when using an optional field it will change the output. One 
far-reaching solution for the symmetry break is allowing ALL integer 
literal codes to have that optional field, and each literal code will 
have its own default base: b->2, o->8, d->10, x->16.

>> Anyway, I think there should be 'B' and 'O' presentation types, that
>> will be used for outputting '0B' and '0O' prefixes.

Horrors such as an uppercase-only file/shell/whatever still exist. They 
are rare and you'll usually call upper() before sending a 
message/writing to a file to them, but it might worth adding 'B'.

 > If you really want that (Why would you?) and so long as the numbers
 > aren't negative:
 >
 >      "0B{:b}".format(number)
 >      "0O{:o}".format(number)
 >
 > The only reason 'X' is provided for hexadecimal formatting is to
 > capitalize the letters that appear within the number itself.

0B111 and -0O755 are *current* valid python tokens. There is no 
"elegant" way of outputting those tokens. I suggested 'B' and 'O' as a 
side-effect, because bases > 10 have some digits as letters.

regards,
TB


From ncoghlan at gmail.com  Sat Dec  3 06:16:06 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Dec 2011 15:16:06 +1000
Subject: [Python-ideas] Different bases format specification
In-Reply-To: <4ED9AE62.5030303@gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<4ED9AE62.5030303@gmail.com>
Message-ID: <CADiSq7dvbqs5RViHi=US8v92KmHZ6RJdd2ooQGuUPRKFvyCLfg@mail.gmail.com>

On Sat, Dec 3, 2011 at 3:06 PM, T.B. <bauertomer at gmail.com> wrote:
>>> Anyway, I think there should be 'B' and 'O' presentation types, that
>>> will be used for outputting '0B' and '0O' prefixes.
>
> Horrors such as an uppercase-only file/shell/whatever still exist. They are
> rare and you'll usually call upper() before sending a message/writing to a
> file to them, but it might worth adding 'B'.

You're setting your bar for "hey, let's change the way a builtin type
works" *way* too low.

It's *OK* if obscure corner cases like bases outside the main four (2,
8, 10, 16), or "upper case only" environments require extra code.
"Make easy things easy and hard things possible" is the goal, not
"build in complex features to handle special cases that only arise on
rare occasions and can already be dealt with using the vast array of
general purpose programming tools Python provides".

It's not that supporting arbitrary bases is a terrible idea - it's
that it doesn't come up often enough as a general programming problem
to be worth going to the effort of making the change. There's no such
thing as a "trivial" change to a Python builtin - they *all* have
significant repercussions, as the update ripples out through the
Python ecosystem over the course of several years (see
http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From solipsis at pitrou.net  Sat Dec  3 17:45:00 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Dec 2011 17:45:00 +0100
Subject: [Python-ideas] Negative hexes
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
Message-ID: <20111203174500.49810bb1@pitrou.net>

On Sat, 3 Dec 2011 14:31:51 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> If you really want that (Why would you?) and so long as the numbers
> aren't negative:
> 
>     "0B{:b}".format(number)
>     "0O{:o}".format(number)
> 
> The only reason 'X' is provided for hexadecimal formatting is to
> capitalize the letters that appear within the number itself.

By the way, any reason why hex output represents negative number with a
negative sign (instead of the more usual 2s-complement representation)?

It's not too difficult to normalize by hand (e.g. add 2**32 if you know
the number is a 32-bit one) but it always irks me that Python doesn't
do it by default. I cannot think of a situation where the "sign" is
relevant when printing a hex number: hex is about the raw binary
representation of the number.

Regards

Antoine.




From guido at python.org  Sat Dec  3 18:05:02 2011
From: guido at python.org (Guido van Rossum)
Date: Sat, 3 Dec 2011 09:05:02 -0800
Subject: [Python-ideas] Negative hexes
In-Reply-To: <20111203174500.49810bb1@pitrou.net>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
Message-ID: <CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>

On Sat, Dec 3, 2011 at 8:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat, 3 Dec 2011 14:31:51 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > If you really want that (Why would you?) and so long as the numbers
> > aren't negative:
> >
> >     "0B{:b}".format(number)
> >     "0O{:o}".format(number)
> >
> > The only reason 'X' is provided for hexadecimal formatting is to
> > capitalize the letters that appear within the number itself.
>
> By the way, any reason why hex output represents negative number with a
> negative sign (instead of the more usual 2s-complement representation)?
>
> It's not too difficult to normalize by hand (e.g. add 2**32 if you know
> the number is a 32-bit one) but it always irks me that Python doesn't
> do it by default. I cannot think of a situation where the "sign" is
> relevant when printing a hex number: hex is about the raw binary
> representation of the number.
>

This is because Python's integers are not limited to 32 bits or 64 bits. If
you read PEP 237, you'll see that this was one of the hardest differences
between ints and longs to be resolved. You'd have to include an infinite
number of leading 'F' characters to format a negative long this way...

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111203/bf412482/attachment.html>

From solipsis at pitrou.net  Sat Dec  3 18:07:47 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Dec 2011 18:07:47 +0100
Subject: [Python-ideas] Negative hexes
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
Message-ID: <20111203180747.37c296e6@pitrou.net>

On Sat, 3 Dec 2011 09:05:02 -0800
Guido van Rossum <guido at python.org> wrote:
> On Sat, Dec 3, 2011 at 8:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> > On Sat, 3 Dec 2011 14:31:51 +1000
> > Nick Coghlan <ncoghlan at gmail.com> wrote:
> > >
> > > If you really want that (Why would you?) and so long as the numbers
> > > aren't negative:
> > >
> > >     "0B{:b}".format(number)
> > >     "0O{:o}".format(number)
> > >
> > > The only reason 'X' is provided for hexadecimal formatting is to
> > > capitalize the letters that appear within the number itself.
> >
> > By the way, any reason why hex output represents negative number with a
> > negative sign (instead of the more usual 2s-complement representation)?
> >
> > It's not too difficult to normalize by hand (e.g. add 2**32 if you know
> > the number is a 32-bit one) but it always irks me that Python doesn't
> > do it by default. I cannot think of a situation where the "sign" is
> > relevant when printing a hex number: hex is about the raw binary
> > representation of the number.
> >
> 
> This is because Python's integers are not limited to 32 bits or 64 bits. If
> you read PEP 237, you'll see that this was one of the hardest differences
> between ints and longs to be resolved. You'd have to include an infinite
> number of leading 'F' characters to format a negative long this way...

That's a fair point :)

Regards

Antoine.




From atmb4u at gmail.com  Sat Dec  3 20:12:30 2011
From: atmb4u at gmail.com (Anoop Thomas Mathew)
Date: Sun, 4 Dec 2011 00:42:30 +0530
Subject: [Python-ideas] Portable Continuations in Python
Message-ID: <CAENdBjQxmWJpYtdf_z=KAOg2e1cjgYDpSNW9S5ZX4rj8ChKcsw@mail.gmail.com>

Hi All,

I was wondering whether we can have a feature like, portable continuations
(as present in Scala 2.8) is possible in Python. It is a feature, by which
executions can be stopped in between, and transferred to another computer
and resume the process. This is a standard feature in scala 2.8. Something
similar is present in Stackless Python, but it doesn't support 'Portable'.
I am not sure how can we do that. But, if it is possible, and if some one
can throw some light how to achieve that, I am ready to volunteer to dig
deeper.

Awaiting comments,
Anoop Thomas Mathew


atm
___
Life is short, Live it hard.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111204/5f36d169/attachment.html>

From ncoghlan at gmail.com  Sun Dec  4 02:13:26 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Dec 2011 11:13:26 +1000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <20111203180747.37c296e6@pitrou.net>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
Message-ID: <CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>

On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> This is because Python's integers are not limited to 32 bits or 64 bits. If
>> you read PEP 237, you'll see that this was one of the hardest differences
>> between ints and longs to be resolved. You'd have to include an infinite
>> number of leading 'F' characters to format a negative long this way...
>
> That's a fair point :)

Random thought... could we use the integer precision field to fix
*that*, by having it indicate the intended number of bytes in the
integer?

That is, currently:

>>> "{:.4x}".format(31)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Precision not allowed in integer format specifier

What if instead that produced:

>>> "{:.4X}".format(31)
0000001F
>>> "{:.4X}".format(-31)
FFFFFFE1

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Sun Dec  4 02:16:48 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Dec 2011 11:16:48 +1000
Subject: [Python-ideas] Portable Continuations in Python
In-Reply-To: <CAENdBjQxmWJpYtdf_z=KAOg2e1cjgYDpSNW9S5ZX4rj8ChKcsw@mail.gmail.com>
References: <CAENdBjQxmWJpYtdf_z=KAOg2e1cjgYDpSNW9S5ZX4rj8ChKcsw@mail.gmail.com>
Message-ID: <CADiSq7fNzev9-e-3H+_FS2A+5Y6p6D5bOPYPf1ve=Ryfipt=LQ@mail.gmail.com>

On Sun, Dec 4, 2011 at 5:12 AM, Anoop Thomas Mathew <atmb4u at gmail.com> wrote:
> Hi All,
>
> I was wondering whether we can have a feature like, portable continuations
> (as present in Scala 2.8) is possible in Python. It is a feature, by which
> executions can be stopped in between, and transferred to another computer
> and resume the process. This is a standard feature in scala 2.8. Something
> similar is present in Stackless Python, but it doesn't support 'Portable'. I
> am not sure how can we do that. But, if it is possible, and if some one can
> throw some light how to achieve that, I am ready to volunteer to dig deeper.

Hi Anoop,

Take a look at the 'greenlets' extension module (which has the same
portability limitations as Stackless, since that's where it came
from), the 'cofunctions' PEP (PEP 3152), and the extensive discussions
regarding the latter in the archives of this list.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From guido at python.org  Sun Dec  4 02:17:56 2011
From: guido at python.org (Guido van Rossum)
Date: Sat, 3 Dec 2011 17:17:56 -0800
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
Message-ID: <CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>

On Sat, Dec 3, 2011 at 5:13 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >> This is because Python's integers are not limited to 32 bits or 64
> bits. If
> >> you read PEP 237, you'll see that this was one of the hardest
> differences
> >> between ints and longs to be resolved. You'd have to include an infinite
> >> number of leading 'F' characters to format a negative long this way...
> >
> > That's a fair point :)
>
> Random thought... could we use the integer precision field to fix
> *that*, by having it indicate the intended number of bytes in the
> integer?
>
> That is, currently:
>
> >>> "{:.4x}".format(31)
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> ValueError: Precision not allowed in integer format specifier
>
> What if instead that produced:
>
> >>> "{:.4X}".format(31)
> 0000001F
> >>> "{:.4X}".format(-31)
> FFFFFFE1
>

Usually that field is measured in characters/digits, so this should
probably produce FFE1; you'd need {:.8X} to produce FFFFFFE1. This would
then logically extend to binary and octal, in each case measuring
characters/digits in the indicated base.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111203/b5ddbfe3/attachment.html>

From python at mrabarnett.plus.com  Sun Dec  4 02:32:43 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 04 Dec 2011 01:32:43 +0000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
Message-ID: <4EDACDBB.1040607@mrabarnett.plus.com>

On 04/12/2011 01:17, Guido van Rossum wrote:
> On Sat, Dec 3, 2011 at 5:13 PM, Nick Coghlan <ncoghlan at gmail.com
> <mailto:ncoghlan at gmail.com>> wrote:
>
>     On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou <solipsis at pitrou.net
>     <mailto:solipsis at pitrou.net>> wrote:
>      >> This is because Python's integers are not limited to 32 bits or
>     64 bits. If
>      >> you read PEP 237, you'll see that this was one of the hardest
>     differences
>      >> between ints and longs to be resolved. You'd have to include an
>     infinite
>      >> number of leading 'F' characters to format a negative long this
>     way...
>      >
>      > That's a fair point :)
>
>     Random thought... could we use the integer precision field to fix
>     *that*, by having it indicate the intended number of bytes in the
>     integer?
>
>     That is, currently:
>
>      >>> "{:.4x}".format(31)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     ValueError: Precision not allowed in integer format specifier
>
>     What if instead that produced:
>
>      >>> "{:.4X}".format(31)
>     0000001F
>      >>> "{:.4X}".format(-31)
>     FFFFFFE1
>
> Usually that field is measured in characters/digits, so this should
> probably produce FFE1; you'd need {:.8X} to produce FFFFFFE1. This would
> then logically extend to binary and octal, in each case measuring
> characters/digits in the indicated base.
>
+1

Not only would that be more consistent, it would also have a use-case.


From guido at python.org  Sun Dec  4 02:51:48 2011
From: guido at python.org (Guido van Rossum)
Date: Sat, 3 Dec 2011 17:51:48 -0800
Subject: [Python-ideas] Negative hexes
In-Reply-To: <4EDACDBB.1040607@mrabarnett.plus.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
	<4EDACDBB.1040607@mrabarnett.plus.com>
Message-ID: <CAP7+vJJ+ZAMJ3j8Us25Ov6xUSv2qVKNkcxVLFATMbpsmWF4giA@mail.gmail.com>

On Sat, Dec 3, 2011 at 5:32 PM, MRAB <python at mrabarnett.plus.com> wrote:

> On 04/12/2011 01:17, Guido van Rossum wrote:
>
>> On Sat, Dec 3, 2011 at 5:13 PM, Nick Coghlan <ncoghlan at gmail.com
>> <mailto:ncoghlan at gmail.com>> wrote:
>>
>>    On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou <solipsis at pitrou.net
>>    <mailto:solipsis at pitrou.net>> wrote:
>>     >> This is because Python's integers are not limited to 32 bits or
>>    64 bits. If
>>     >> you read PEP 237, you'll see that this was one of the hardest
>>    differences
>>     >> between ints and longs to be resolved. You'd have to include an
>>    infinite
>>     >> number of leading 'F' characters to format a negative long this
>>    way...
>>     >
>>     > That's a fair point :)
>>
>>    Random thought... could we use the integer precision field to fix
>>    *that*, by having it indicate the intended number of bytes in the
>>    integer?
>>
>>    That is, currently:
>>
>>     >>> "{:.4x}".format(31)
>>    Traceback (most recent call last):
>>      File "<stdin>", line 1, in <module>
>>    ValueError: Precision not allowed in integer format specifier
>>
>>    What if instead that produced:
>>
>>     >>> "{:.4X}".format(31)
>>    0000001F
>>     >>> "{:.4X}".format(-31)
>>    FFFFFFE1
>>
>> Usually that field is measured in characters/digits, so this should
>> probably produce FFE1; you'd need {:.8X} to produce FFFFFFE1. This would
>> then logically extend to binary and octal, in each case measuring
>> characters/digits in the indicated base.
>>
>>  +1
>
> Not only would that be more consistent, it would also have a use-case.


OTOH I'm not sure what should happen if the number (negative or positive!)
doesn't fit in the precision.

How common is this use case? Personally I'm fine with writing x & (2**N -
1) where N is e.g. 32 or 64.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111203/7d1147d4/attachment.html>

From ncoghlan at gmail.com  Sun Dec  4 03:06:50 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Dec 2011 12:06:50 +1000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
Message-ID: <CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>

On Sun, Dec 4, 2011 at 11:17 AM, Guido van Rossum <guido at python.org> wrote:
> On Sat, Dec 3, 2011 at 5:13 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>> >> This is because Python's integers are not limited to 32 bits or 64
>> >> bits. If
>> >> you read PEP 237, you'll see that this was one of the hardest
>> >> differences
>> >> between ints and longs to be resolved. You'd have to include an
>> >> infinite
>> >> number of leading 'F' characters to format a negative long this way...
>> >
>> > That's a fair point :)
>>
>> Random thought... could we use the integer precision field to fix
>> *that*, by having it indicate the intended number of bytes in the
>> integer?
>>
>> That is, currently:
>>
>> >>> "{:.4x}".format(31)
>> Traceback (most recent call last):
>> ?File "<stdin>", line 1, in <module>
>> ValueError: Precision not allowed in integer format specifier
>>
>> What if instead that produced:
>>
>> >>> "{:.4X}".format(31)
>> 0000001F
>> >>> "{:.4X}".format(-31)
>> FFFFFFE1
>
>
> Usually that field is measured in characters/digits, so this should probably
> produce FFE1; you'd need {:.8X} to produce FFFFFFE1. This would then
> logically extend to binary and octal, in each case measuring
> characters/digits in the indicated base.

True, I guess it's just a matter of dividing the bit width by 2 for
binary, 3 for octal (rounding up) and 4 for binary. (My brain was
locked into bytes mode for some reason, so converting to character
counts seemed overly complicated - of course, if you go directly from
bits to characters, it's no more complicated than converting to a
bytes count).

".4d" would still raise an exception, though - I don't know of any
obvious way to make two's complement notation meaningful in base 10.

For numbers that didn't fit in the specified precision, I'd also
suggest raising ValueError.

This would be tinkering with the behaviour of builtin, so I guess it
would need a PEP? (I already have too many of those in train...
although I did just tidy that up a bit by officially deferring
consideration of the ImportEngine PEP until 3.4 at the earliest)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From python at mrabarnett.plus.com  Sun Dec  4 03:08:45 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 04 Dec 2011 02:08:45 +0000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CAP7+vJJ+ZAMJ3j8Us25Ov6xUSv2qVKNkcxVLFATMbpsmWF4giA@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
	<4EDACDBB.1040607@mrabarnett.plus.com>
	<CAP7+vJJ+ZAMJ3j8Us25Ov6xUSv2qVKNkcxVLFATMbpsmWF4giA@mail.gmail.com>
Message-ID: <4EDAD62D.8010307@mrabarnett.plus.com>

On 04/12/2011 01:51, Guido van Rossum wrote:
> On Sat, Dec 3, 2011 at 5:32 PM, MRAB <python at mrabarnett.plus.com
> <mailto:python at mrabarnett.plus.com>> wrote:
>
>     On 04/12/2011 01:17, Guido van Rossum wrote:
>
>         On Sat, Dec 3, 2011 at 5:13 PM, Nick Coghlan <ncoghlan at gmail.com
>         <mailto:ncoghlan at gmail.com>
>         <mailto:ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>>> wrote:
>
>             On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou
>         <solipsis at pitrou.net <mailto:solipsis at pitrou.net>
>         <mailto:solipsis at pitrou.net <mailto:solipsis at pitrou.net>>> wrote:
>          >> This is because Python's integers are not limited to 32 bits or
>             64 bits. If
>          >> you read PEP 237, you'll see that this was one of the hardest
>             differences
>          >> between ints and longs to be resolved. You'd have to include an
>             infinite
>          >> number of leading 'F' characters to format a negative long this
>             way...
>          >
>          > That's a fair point :)
>
>             Random thought... could we use the integer precision field
>         to fix
>             *that*, by having it indicate the intended number of bytes
>         in the
>             integer?
>
>             That is, currently:
>
>          >>> "{:.4x}".format(31)
>             Traceback (most recent call last):
>               File "<stdin>", line 1, in <module>
>             ValueError: Precision not allowed in integer format specifier
>
>             What if instead that produced:
>
>          >>> "{:.4X}".format(31)
>             0000001F
>          >>> "{:.4X}".format(-31)
>             FFFFFFE1
>
>         Usually that field is measured in characters/digits, so this should
>         probably produce FFE1; you'd need {:.8X} to produce FFFFFFE1.
>         This would
>         then logically extend to binary and octal, in each case measuring
>         characters/digits in the indicated base.
>
>     +1
>
>     Not only would that be more consistent, it would also have a use-case.
>
>
> OTOH I'm not sure what should happen if the number (negative or
> positive!) doesn't fit in the precision.
>
Well, the width is treated as the minimum width, so perhaps the
precision should be the minimum precision in this case.

> How common is this use case? Personally I'm fine with writing x & (2**N
> - 1) where N is e.g. 32 or 64.
>


From python at mrabarnett.plus.com  Sun Dec  4 03:14:56 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 04 Dec 2011 02:14:56 +0000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
	<CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
Message-ID: <4EDAD7A0.5020300@mrabarnett.plus.com>

On 04/12/2011 02:06, Nick Coghlan wrote:
> On Sun, Dec 4, 2011 at 11:17 AM, Guido van Rossum<guido at python.org>  wrote:
>> On Sat, Dec 3, 2011 at 5:13 PM, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>>>
>>> On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou<solipsis at pitrou.net>
>>> wrote:
>>>>> This is because Python's integers are not limited to 32 bits or 64
>>>>> bits. If
>>>>> you read PEP 237, you'll see that this was one of the hardest
>>>>> differences
>>>>> between ints and longs to be resolved. You'd have to include an
>>>>> infinite
>>>>> number of leading 'F' characters to format a negative long this way...
>>>>
>>>> That's a fair point :)
>>>
>>> Random thought... could we use the integer precision field to fix
>>> *that*, by having it indicate the intended number of bytes in the
>>> integer?
>>>
>>> That is, currently:
>>>
>>>>>> "{:.4x}".format(31)
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in<module>
>>> ValueError: Precision not allowed in integer format specifier
>>>
>>> What if instead that produced:
>>>
>>>>>> "{:.4X}".format(31)
>>> 0000001F
>>>>>> "{:.4X}".format(-31)
>>> FFFFFFE1
>>
>>
>> Usually that field is measured in characters/digits, so this should probably
>> produce FFE1; you'd need {:.8X} to produce FFFFFFE1. This would then
>> logically extend to binary and octal, in each case measuring
>> characters/digits in the indicated base.
>
> True, I guess it's just a matter of dividing the bit width by 2 for
> binary, 3 for octal (rounding up) and 4 for binary. (My brain was
> locked into bytes mode for some reason, so converting to character
> counts seemed overly complicated - of course, if you go directly from
> bits to characters, it's no more complicated than converting to a
> bytes count).
>
> ".4d" would still raise an exception, though - I don't know of any
> obvious way to make two's complement notation meaningful in base 10.
>
It wouldn't be two's complement in that case, it would be ten's
complement. (Ever used BCD arithmetic in assembly language? :-))

Actually, would it really be two's complement in octal or hexadecimal
either? Wouldn't it be eight's complement and sixteen's complement
respectively?

> For numbers that didn't fit in the specified precision, I'd also
> suggest raising ValueError.
>
> This would be tinkering with the behaviour of builtin, so I guess it
> would need a PEP? (I already have too many of those in train...
> although I did just tidy that up a bit by officially deferring
> consideration of the ImportEngine PEP until 3.4 at the earliest)
>


From tjreedy at udel.edu  Sun Dec  4 04:05:54 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 03 Dec 2011 22:05:54 -0500
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
	<CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
Message-ID: <jbeo2p$oag$1@dough.gmane.org>

On 12/3/2011 9:06 PM, Nick Coghlan wrote:

> This would be tinkering with the behaviour of builtin,

It is a backwards compatible augmentation:
(as I understand the proposal)
.n for ints means print negatives with n digits in baseNs-complement 
notation instead of raising an exception.

-- 
Terry Jan Reedy



From ncoghlan at gmail.com  Sun Dec  4 05:57:32 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 4 Dec 2011 14:57:32 +1000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <4EDAD7A0.5020300@mrabarnett.plus.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
	<CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
	<4EDAD7A0.5020300@mrabarnett.plus.com>
Message-ID: <CADiSq7eVQk5jY-M-AUerP1uuVejQQo8UNrBU_h_eML3tg26pZA@mail.gmail.com>

On Sun, Dec 4, 2011 at 12:14 PM, MRAB <python at mrabarnett.plus.com> wrote:
> On 04/12/2011 02:06, Nick Coghlan wrote:
>> ".4d" would still raise an exception, though - I don't know of any
>> obvious way to make two's complement notation meaningful in base 10.
>>
> It wouldn't be two's complement in that case, it would be ten's
> complement. (Ever used BCD arithmetic in assembly language? :-))
>
> Actually, would it really be two's complement in octal or hexadecimal
> either? Wouldn't it be eight's complement and sixteen's complement
> respectively?

For hexadecimal, it doesn't make much difference, since the number of
bits per digit is a power of 2: 2**32 == (2**4)**8

For octal, you're right, since it would be 2**33 == (2**3)**11

I quite like Terry's definition, which does extend cleanly to 'd' (as
well as giving the expected answer for 'b', 'o', 'x' and 'X'):

For integers, the precision field would be used to specify the
expected maximum number of digits in the answer, and to switch the
representation of negative values to the appropriate 'complement' form
(e.g. two's complement for binary numbers).

When a precision ".prec" is specified for an integer formatting code
(b, o, d, x or X), the value to be displayed would be calculated as
follows:

    _BASES = dict(b=2,o=8,d=10,x=16,X=16)
    _BASE_NAMES =
dict(b='binary',o='octal',d='decimal',x='hexadecimal',X='hexadecimal')
    _base = _BASES[format_code]
    _prec_bound = _base ** prec
    _max_value = _prec_bound / 2
    if value < -_max_value or value >= _max_value:
        _code = _BASE_NAMES[format_code]
        raise ValueError("Integer {} too large for {} precision of
{}".format(value, _code, n))
    _value = _prec_bound - value

However, I'm not sure that qualifies as *useful* behaviour - while the
bounds checking aspect could be useful for decimal, the complement
form of negative numbers is almost never going to be what anyone
wants. If we decide to improve things in terms of Python-level
handling of two's complement arithmetic, perhaps it would make more
sense to just provide a method on int objects that calculates the
two's complement of a number for a given bit length? Something like:

def as_twos_complement(self, bits):
    if self.bit_length() >= bits:
         raise ValueError("int {} too large for {}-bit signed
precision".format(self, bits))
    if self >= 0:
         return self
    return 2**bits + self # self is known to be negative at this point

As for whether this is worth doing or not... I think so. While Python
integers may be limited in size solely by available memory, it's going
to be a fact of computing life for quite some time that there are
going to be fixed size signed and unsigned integers under the hood
*somewhere*. We already provide a mechanism to find out how many bits
a given integer needs, this would be about providing a standard,
efficient, mechanism to convert negative integers to their two's
complement positive equivalents for a given number of bits.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From storchaka at gmail.com  Mon Dec  5 12:58:44 2011
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 05 Dec 2011 13:58:44 +0200
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<CAP7+vJKwsiZBnk_xn2FTsPv73+hpOhRnAMj2aQrXJUGdz6fKtQ@mail.gmail.com>
	<CADiSq7enfyJnZwHAvgT3qvX7Gkn9Syyjkh3DZ6V+exo=oa9fuQ@mail.gmail.com>
Message-ID: <jbibm3$lmb$1@dough.gmane.org>

04.12.11 04:06, Nick Coghlan ???????(??):
> ".4d" would still raise an exception, though - I don't know of any
> obvious way to make two's complement notation meaningful in base 10.

Obviously, '{0:.4d}'.format(-31) == '{0:4d}'.format((-31)%10**4) == '9969'.

> This would be tinkering with the behaviour of builtin, so I guess it
> would need a PEP?

No, no! Please, do not obfuscate Python formatting.



From solipsis at pitrou.net  Mon Dec  5 18:15:32 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 5 Dec 2011 18:15:32 +0100
Subject: [Python-ideas] Negative hexes
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
Message-ID: <20111205181532.1214a70f@pitrou.net>

On Sun, 4 Dec 2011 11:13:26 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Dec 4, 2011 at 3:07 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >> This is because Python's integers are not limited to 32 bits or 64 bits. If
> >> you read PEP 237, you'll see that this was one of the hardest differences
> >> between ints and longs to be resolved. You'd have to include an infinite
> >> number of leading 'F' characters to format a negative long this way...
> >
> > That's a fair point :)
> 
> Random thought... could we use the integer precision field to fix
> *that*, by having it indicate the intended number of bytes in the
> integer?
> 
> That is, currently:
> 
> >>> "{:.4x}".format(31)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: Precision not allowed in integer format specifier

This is so poorly discoverable that I don't think it's worth it.
Guido's approach looks sufficient to me.

Regards

Antoine.




From ncoghlan at gmail.com  Mon Dec  5 23:19:53 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Dec 2011 08:19:53 +1000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <20111205181532.1214a70f@pitrou.net>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<20111205181532.1214a70f@pitrou.net>
Message-ID: <CADiSq7dd-hHjynyMEvrb7vgdTkcWqxSVRQcgX064TgsuHXgEAg@mail.gmail.com>

On Tue, Dec 6, 2011 at 3:15 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> That is, currently:
>>
>> >>> "{:.4x}".format(31)
>> Traceback (most recent call last):
>> ? File "<stdin>", line 1, in <module>
>> ValueError: Precision not allowed in integer format specifier
>
> This is so poorly discoverable that I don't think it's worth it.
> Guido's approach looks sufficient to me.

I agree the formatting approach is way too obscure, but did you see my
later suggestion of an "as_twos_complement(bit_length)" conversion
method on int objects? (we could actually provide a more generic
version on numbers.Integer as well)

Cheers,
Nick.



-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From solipsis at pitrou.net  Tue Dec  6 00:10:54 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 6 Dec 2011 00:10:54 +0100
Subject: [Python-ideas] Negative hexes
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<20111205181532.1214a70f@pitrou.net>
	<CADiSq7dd-hHjynyMEvrb7vgdTkcWqxSVRQcgX064TgsuHXgEAg@mail.gmail.com>
Message-ID: <20111206001054.2cbba034@pitrou.net>

On Tue, 6 Dec 2011 08:19:53 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Dec 6, 2011 at 3:15 AM, Antoine Pitrou <solipsis-xNDA5Wrcr86sTnJN9+BGXg at public.gmane.org> wrote:
> >> That is, currently:
> >>
> >> >>> "{:.4x}".format(31)
> >> Traceback (most recent call last):
> >> ? File "<stdin>", line 1, in <module>
> >> ValueError: Precision not allowed in integer format specifier
> >
> > This is so poorly discoverable that I don't think it's worth it.
> > Guido's approach looks sufficient to me.
> 
> I agree the formatting approach is way too obscure, but did you see my
> later suggestion of an "as_twos_complement(bit_length)" conversion
> method on int objects? (we could actually provide a more generic
> version on numbers.Integer as well)

Ah, right. Yes, I think that would be useful, although a shorter name
would be nicer. to_unsigned() perhaps?

Regards

Antoine.




From ncoghlan at gmail.com  Tue Dec  6 01:14:03 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Dec 2011 10:14:03 +1000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <20111206001054.2cbba034@pitrou.net>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<20111205181532.1214a70f@pitrou.net>
	<CADiSq7dd-hHjynyMEvrb7vgdTkcWqxSVRQcgX064TgsuHXgEAg@mail.gmail.com>
	<20111206001054.2cbba034@pitrou.net>
Message-ID: <CADiSq7eUgZ=6EnNkgEYpG1Ou7-pGLsTaSkS6dJJm3L1tOMwcDg@mail.gmail.com>

On Tue, Dec 6, 2011 at 9:10 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 6 Dec 2011 08:19:53 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Tue, Dec 6, 2011 at 3:15 AM, Antoine Pitrou <solipsis-xNDA5Wrcr86sTnJN9+BGXg at public.gmane.org> wrote:
>> >> That is, currently:
>> >>
>> >> >>> "{:.4x}".format(31)
>> >> Traceback (most recent call last):
>> >> ? File "<stdin>", line 1, in <module>
>> >> ValueError: Precision not allowed in integer format specifier
>> >
>> > This is so poorly discoverable that I don't think it's worth it.
>> > Guido's approach looks sufficient to me.
>>
>> I agree the formatting approach is way too obscure, but did you see my
>> later suggestion of an "as_twos_complement(bit_length)" conversion
>> method on int objects? (we could actually provide a more generic
>> version on numbers.Integer as well)
>
> Ah, right. Yes, I think that would be useful, although a shorter name
> would be nicer. to_unsigned() perhaps?

Hmm, in the sense that the answer we're getting is the same answer you
would get with a cast to an unsigned type at the C level? I think
that's a little misleading - conceptually, the number is still signed,
we're just representing it differently (i.e. explicitly using the twos
complement form, rather than the the normal sign bit).

I'd be OK with dropping the explicit 'twos' qualifier, though - then
the method name could just be "to_complement()".

I guess we'd also want a "to_signed()" to reverse the process:

def to_complement(self, bits):
   "Convert this integer to its unsigned two's complement equivalent
for the given bit length"
   if self.bit_length() >= bits:
        raise ValueError("{} is too large for {}-bit two's complement
precision".format(self, bits))
   if self >= 0:
        return self
   return 2**bits + self # self is known to be negative at this point

def to_signed(self, bits):
   "Convert an integer in two's complement format to its signed
equivalent for the given bit length"
   if self < 0:
        raise ValueError("{} is already signed".format(self))
   if self.bit_length() > bits:
        raise ValueError("{} is too large for {}-bit two's complement
precision".format(self, bits))
   upper_bound = 2**bits
   if self < (upper_bound / 2):
      return self
   return upper_bound - self

Regards,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From guido at python.org  Tue Dec  6 01:16:59 2011
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Dec 2011 16:16:59 -0800
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CADiSq7eUgZ=6EnNkgEYpG1Ou7-pGLsTaSkS6dJJm3L1tOMwcDg@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<20111205181532.1214a70f@pitrou.net>
	<CADiSq7dd-hHjynyMEvrb7vgdTkcWqxSVRQcgX064TgsuHXgEAg@mail.gmail.com>
	<20111206001054.2cbba034@pitrou.net>
	<CADiSq7eUgZ=6EnNkgEYpG1Ou7-pGLsTaSkS6dJJm3L1tOMwcDg@mail.gmail.com>
Message-ID: <CAP7+vJK935bj9a8xptA_N5-qQYctX+PLZ0M64BnYU9aQ1pXhhA@mail.gmail.com>

On Mon, Dec 5, 2011 at 4:14 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Tue, Dec 6, 2011 at 9:10 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Tue, 6 Dec 2011 08:19:53 +1000
> > Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> On Tue, Dec 6, 2011 at 3:15 AM, Antoine Pitrou <
> solipsis-xNDA5Wrcr86sTnJN9+BGXg at public.gmane.org> wrote:
> >> >> That is, currently:
> >> >>
> >> >> >>> "{:.4x}".format(31)
> >> >> Traceback (most recent call last):
> >> >>   File "<stdin>", line 1, in <module>
> >> >> ValueError: Precision not allowed in integer format specifier
> >> >
> >> > This is so poorly discoverable that I don't think it's worth it.
> >> > Guido's approach looks sufficient to me.
> >>
> >> I agree the formatting approach is way too obscure, but did you see my
> >> later suggestion of an "as_twos_complement(bit_length)" conversion
> >> method on int objects? (we could actually provide a more generic
> >> version on numbers.Integer as well)
> >
> > Ah, right. Yes, I think that would be useful, although a shorter name
> > would be nicer. to_unsigned() perhaps?
>
> Hmm, in the sense that the answer we're getting is the same answer you
> would get with a cast to an unsigned type at the C level? I think
> that's a little misleading - conceptually, the number is still signed,
> we're just representing it differently (i.e. explicitly using the twos
> complement form, rather than the the normal sign bit).
>

I don't think there's a better term available. As long as the return value
of to_unsigned() is never negative I think it's a fine name.


> I'd be OK with dropping the explicit 'twos' qualifier, though - then
> the method name could just be "to_complement()".
>
> I guess we'd also want a "to_signed()" to reverse the process:
>

Sure.


> def to_complement(self, bits):
>   "Convert this integer to its unsigned two's complement equivalent
> for the given bit length"
>   if self.bit_length() >= bits:
>        raise ValueError("{} is too large for {}-bit two's complement
> precision".format(self, bits))
>   if self >= 0:
>        return self
>   return 2**bits + self # self is known to be negative at this point
>
> def to_signed(self, bits):
>   "Convert an integer in two's complement format to its signed
> equivalent for the given bit length"
>   if self < 0:
>        raise ValueError("{} is already signed".format(self))
>   if self.bit_length() > bits:
>        raise ValueError("{} is too large for {}-bit two's complement
> precision".format(self, bits))
>   upper_bound = 2**bits
>   if self < (upper_bound / 2):
>      return self
>   return upper_bound - self
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111205/a3a2531f/attachment.html>

From bruce at leapyear.org  Tue Dec  6 01:24:45 2011
From: bruce at leapyear.org (Bruce Leban)
Date: Mon, 5 Dec 2011 16:24:45 -0800
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CADiSq7eUgZ=6EnNkgEYpG1Ou7-pGLsTaSkS6dJJm3L1tOMwcDg@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<20111205181532.1214a70f@pitrou.net>
	<CADiSq7dd-hHjynyMEvrb7vgdTkcWqxSVRQcgX064TgsuHXgEAg@mail.gmail.com>
	<20111206001054.2cbba034@pitrou.net>
	<CADiSq7eUgZ=6EnNkgEYpG1Ou7-pGLsTaSkS6dJJm3L1tOMwcDg@mail.gmail.com>
Message-ID: <CAGu0AnuXn+-CsGejQczK+UurUzDpSp6CS2KyOgo-E_D8f-LO+Q@mail.gmail.com>

On Mon, Dec 5, 2011 at 4:14 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> I'd be OK with dropping the explicit 'twos' qualifier, though - then
> the method name could just be "to_complement()".
>

That name's ambiguous since there's also one's complement as well as ten's
and nine's complements.

Also, perhaps from_twos_complement might be a clearer name than to_signed.

--- Bruce
Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111205/79a4526f/attachment.html>

From ncoghlan at gmail.com  Tue Dec  6 01:52:14 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Dec 2011 10:52:14 +1000
Subject: [Python-ideas] Negative hexes
In-Reply-To: <CAP7+vJK935bj9a8xptA_N5-qQYctX+PLZ0M64BnYU9aQ1pXhhA@mail.gmail.com>
References: <4ED9698A.6010609@gmail.com>
	<CADiSq7e4ywOePrkWZTYqy8w_A0wNQVTL-vzD129D6uk2917sHg@mail.gmail.com>
	<4ED99470.20205@gmail.com>
	<CADiSq7f-pwuu4V=r26cXnjf1DbPCud5UxQOHTLENZqYvYeJkow@mail.gmail.com>
	<20111203174500.49810bb1@pitrou.net>
	<CAP7+vJKh6TOD+oHEWcVs5z9n7m5Y8H7bjoCPqXGCLDawohDfZA@mail.gmail.com>
	<20111203180747.37c296e6@pitrou.net>
	<CADiSq7dqHVEPsQ_tcKXCuALsp4KFzJ_NSyOVYaGsRMh8jqhm6w@mail.gmail.com>
	<20111205181532.1214a70f@pitrou.net>
	<CADiSq7dd-hHjynyMEvrb7vgdTkcWqxSVRQcgX064TgsuHXgEAg@mail.gmail.com>
	<20111206001054.2cbba034@pitrou.net>
	<CADiSq7eUgZ=6EnNkgEYpG1Ou7-pGLsTaSkS6dJJm3L1tOMwcDg@mail.gmail.com>
	<CAP7+vJK935bj9a8xptA_N5-qQYctX+PLZ0M64BnYU9aQ1pXhhA@mail.gmail.com>
Message-ID: <CADiSq7frkqDv0jiw3_ypqPFdxnE4epCdmJRx4qvughfDJjTB3Q@mail.gmail.com>

On Tue, Dec 6, 2011 at 10:16 AM, Guido van Rossum <guido at python.org> wrote:
>> Hmm, in the sense that the answer we're getting is the same answer you
>> would get with a cast to an unsigned type at the C level? I think
>> that's a little misleading - conceptually, the number is still signed,
>> we're just representing it differently (i.e. explicitly using the twos
>> complement form, rather than the the normal sign bit).
>
> I don't think there's a better term available. As long as the return value
> of to_unsigned() is never negative I think it's a fine name.

Recorded the RFE here (using "to_signed()/to_unsigned()"):
http://bugs.python.org/issue13535

I was sold on the name when I read my own docstring: "Convert this
integer to its unsigned two's complement equivalent
for the given bit length"

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Wed Dec  7 15:53:44 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Dec 2011 00:53:44 +1000
Subject: [Python-ideas] Exploring the 'strview' concept further
Message-ID: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>

With encouragement from me (and others) Armin Ronacher recently
attempted to articulate his problems in dealing with the migration to
Python 3 [1]. They're actually quite similar to the feelings I had
during my early attempts at restoring the ability of the URL parsing
APIs to deal directly with ASCII-encoded binary data, rather than
requiring that the application developer explicitly decode it to text
first [2].

Now, I clearly disagree with Armin on at least one point: there
already *is* "one true way" to have unified text processing code in
Python 3. That way is the way the Python 3.2 urllib.parse module
handles it: as soon as it is handed something that isn't a string, it
attempts to decode it using a default assumed encoding (specifically
'ascii', at least for now). It keeps track of whether or not the
arguments were decoded from bytes and, if they were, encodes the
return value on output [3]. If you're pipelining such interfaces, it's
obviously more efficiently to just decode once before invoking the
pipeline and then (optionally) encoding again at the end (just as is
the case in Python 2), but you can still make your APIs largely
polymorphic with respect to bytes and text without massive internal
code duplication.

So, that's always one of my first suggestions to people struggling
with Python 3's unicode model: I ask if they have tried putting aside
any concerns they may have about possible losses of efficiency, and
just tried the decode-on-input-and-return-an-output-coercion-function,
coerce-on-output approach. Python used to do this implicitly for you
at every string operation (minus the 'coerce on output' part), but now
it is asking that you do it manually, and decide for *yourself* on an
appropriate encoding, instead of the automatic assumption of ASCII
text that is present in Python 2 (we'll leave aside the issue of
platform-specific defaults in various contexts - that's a whole
different question and one I'm not at all equipped to answer. I don't
think I've ever even had to work on a system with any locale other
than en_US or en_GB).

Often this actually resolves their problem (since they're no longer
fighting the new Unicode model, and instead embracing it), and this is
why PEP 393 is going to be such a big deal when Python 3.3 is released
next year. Protocol developers are *right* to be worried about a
four-fold increase in memory usage (and the flow on effects on CPU
usage and cache misses) when going from bytes data to the UCS4
internal Unicode format used on most distro-provided Python builds for
Linux. With PEP 393's flexible internal representations, the amount of
memory used will be as little as possible while still allowing
straightforward O(1) lookup of individual code points.

However, that urllib.urlparse code also highlights another one of
Armin's complaints: like much of the stdlib (and core interpreter!),
it doesn't ducktype 'str'. Instead, it demands the real thing and
accepts no substitutes (not even collections.UserString). This kind of
behaviour is quite endemic - the coupling between the interpreter and
the details of the string implementation is, in general, even tighter
than that between the interpreter and the dict implementation used for
namespaces.

With PEP 3118, we introduced the concept of 'memoryview' to make
allowance for the fact that it is often useful to look at the same
chunk of memory in multiple ways, *without* incurring the costs of
making multiple copies. In a discussion back in June [4], I briefly
mentioned the idea of a 'strview' type that would extend those
concepts to providing a str-like view of a region of memory, *without*
necessarily making a copy of the entire thing.

DISCLAIMERS:
1. I don't know yet if this is a good idea. It may in fact be a
terrible idea. I think it is, at least, an idea worth discussing
further.
2. Making this concept work may require actually *classifying* our
codecs to some degree (for attributes like 'ASCII-compatible',
'stateless', 'fixed width', etc). That might be tedious, but doesn't
seem completely infeasible.
3. There are issues with memoryview itself that should be accounted
for if pursuing this idea [5]
4. There is an issue with CPython's operand coercion for sequence
concatenation and repetition that may affect attempts to implement
this idea, although you should be fine so long as you implement the
number methods in addition to the sequence ones (which happens
automatically for classes written in Python) [6]

So, how might a 'strview' object work?

1. The basic construction would be "strview(object, encoding,
errors)". For convenience, actual str objects would just be returned
unmodified (alternatively: a factory function could be provided with
that behaviour)
2. A 'strview' *wouldn't* try to pass itself off as a real string for
all purposes. Instead, it would support a new String ABC (more on that
below).
4. The encode() method would work like a string's normal encode()
method, decoding the original object to a str, then encoding that to
the desired encoding. If the encodings match, then an optimised fast
path of simply calling bytes() on the underlying object would be used.
5. If asked to index, slice or iterate over the underlying string, the
strview would use the incremental decoder for the relevant codec to
build an efficient mapping from code point indices to byte indices and
then return real strings (various strategies for doing this have been
posted to this list in the past). Alternatively, if codecs were
classified to explicitly indicate when they implemented stateless
fixed width encodings, then strview could simply be restricted to only
working with that subset of possible encodings. The latter strategy
might be needed to get around issues with stateful encodings like
ShiftJIS and ITA2 - those are hard (impossible?) to index and
interpret efficiently without fully decoding them and storing the
result.
6. The new type would implement the various binary operators supported
by strings, promoting itself to a real string type whenever needed
7. The new type would similarly support the full string API, returning
actual string objects rather than any kind of view.

What might a String ABC provide?

For a very long time, slice indices had to be real integers - we
didn't allow other "integer like" types. The reason was that floats
implemented __int__, so ducktyping on that method would have allowed
binary floating point numbers in functions where we didn't want to
permit them. The answer, ultimately, was to introduce __index__ (and,
eventually, numbers.Integral) to mark "true" integers, allowing things
like NumPy scalars to be used directly as slice indices without
inheriting from int.

An explicit String ABC, even if not supported for performance critical
core functionality like identifiers, would allow the implementation of
code like that in urllib.urlparse to be updated to avoid keying
behaviour on the concrete builtin str type - instead, it would check
against the String ABC, allowing for all the usual explicit type
registration goodies that ABCs support (and that make them much better
for type checking than concrete types).

Just as much of the old UserDict functionality is now available on
Mapping and MutableMapping, so much of the existing UserString
functionality could be moved to the hypothetical String ABC.

Hopefully-the-rambling-isn't-too-incoherent'ly-yours,
Nick.

[1] http://lucumr.pocoo.org/2011/12/7/thoughts-on-python3/
[2] http://bugs.python.org/issue9873
[3] http://hg.python.org/cpython/file/default/Lib/urllib/parse.py#l74
[4] http://mail.python.org/pipermail/python-ideas/2011-June/010439.html
[5] http://bugs.python.org/issue10181
[6] http://bugs.python.org/issue11477

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From solipsis at pitrou.net  Wed Dec  7 16:39:58 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 7 Dec 2011 16:39:58 +0100
Subject: [Python-ideas] Exploring the 'strview' concept further
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
Message-ID: <20111207163958.5ac4a9fe@pitrou.net>

On Thu, 8 Dec 2011 00:53:44 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> 3. There are issues with memoryview itself that should be accounted
> for if pursuing this idea [5]

These issues are related to complex buffer types (strided,
multi-dimensional, etc.). They wouldn't apply to a hypothetical
"linear unicode buffer".

> 1. The basic construction would be "strview(object, encoding,
> errors)". For convenience, actual str objects would just be returned
> unmodified (alternatively: a factory function could be provided with
> that behaviour)

The factory function is a better idea than silent pass-through, IMO.

> 5. If asked to index, slice or iterate over the underlying string, the
> strview would use the incremental decoder for the relevant codec to
> build an efficient mapping from code point indices to byte indices and
> then return real strings (various strategies for doing this have been
> posted to this list in the past).

Be careful, the incremental decoders use a layer of pure Python
wrappers. You want to call them on big blocks (at least 4 or 8KB, as
TextIOWrapper does) if you don't want to lose a lot of speed. So
building a mapping may not be easy.

Even bypassing the Python layer would still incur the overhead of
repeatedly calling a standalone function, instead of having a tight loop
such as the following:
http://hg.python.org/cpython/file/e49220f4c31f/Objects/unicodeobject.c#l4228

And of course, your mapping must be space-efficient enough that it's
much smaller than the full decoded string.

I think that for small strings (< 1024 bytes?), decoding and storing
the decoded string are not a big deal. Decoding once is *much* faster
(especially for optimized encodings such as latin-1 or utf-8, and only
them will be left in a few years) than trying to do it piecewise.
strview would only be a win for rather large strings. Which makes it
useless for URL parsing ;)

> Alternatively, if codecs were
> classified to explicitly indicate when they implemented stateless
> fixed width encodings, then strview could simply be restricted to only
> working with that subset of possible encodings.

From an usability POV this seems undesireable. On the other hand, if
complete decoding is required, calling str() is just as cheap.

> 7. The new type would similarly support the full string API, returning
> actual string objects rather than any kind of view.

Even for slicing?





From barry at python.org  Wed Dec  7 20:50:57 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 7 Dec 2011 14:50:57 -0500
Subject: [Python-ideas] Exploring the 'strview' concept further
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
Message-ID: <20111207145057.3522180f@resist.wooz.org>

On Dec 08, 2011, at 12:53 AM, Nick Coghlan wrote:

>With encouragement from me (and others) Armin Ronacher recently
>attempted to articulate his problems in dealing with the migration to
>Python 3 [1]. They're actually quite similar to the feelings I had
>during my early attempts at restoring the ability of the URL parsing
>APIs to deal directly with ASCII-encoded binary data, rather than
>requiring that the application developer explicitly decode it to text
>first [2].

I've just finished a port of dbus-python to Python 3, submitting the patches
upstream, although they haven't been reviewed yet.  It was an interesting
exercise for many reasons, and I have my own thoughts the state of porting
which I'll post at another time.  I agree with some of the issues that Armin
brought up, and disagree with others. ;)

At the C level, dbus defines its strings as UTF-8 encoded char*'s.  My first
crack at this (despite upstream's thinking otherwise) was to use bytes to
represent these objects.  That turned out to be completely infeasible for
several reasons.  The biggest problem is that some of the core callback
dispatch code was doing slicing and comparisons of these objects against
literals, or externally registered objects.  So where you might see something
like:

    >>> s = ':1/123'
    >>> s[:1] == ':'
    True

this doesn't work when `s` is a bytes object.  Given the number of places
internally that would have to be fixed, and the cost of imposing a potentially
huge number of changes to clients of the library, I ultimately decided that
upstream's suggestion to model these things as unicodes was right after all.
Once I made that change, the port went relatively easily, as judged by the
amount of time it took to get the test suite completing successfully. ;)

At the Python level, many of the interfaces which accept 8-bit strings in
Python 2, accept bytes or strs in Python 3, where the bytes must be utf-8
encoded.  Internally, I had to do many more type checks for PyUnicodes, and
then decode them to bytes before I could pass them to do the dbus C API.  In
almost all cases though, returning data from the dbus C API involved decoding
the char*'s to unicodes, not bytes.

This means rather than returning bytes if bytes were given, at the Python
layer, unicode is always returned.  This I think causes the least disruption
in user code.  Well, we'll see as I'm now going to be porting some dbus-based
applications.

>Now, I clearly disagree with Armin on at least one point: there
>already *is* "one true way" to have unified text processing code in
>Python 3. That way is the way the Python 3.2 urllib.parse module
>handles it: as soon as it is handed something that isn't a string, it
>attempts to decode it using a default assumed encoding (specifically
>'ascii', at least for now). It keeps track of whether or not the
>arguments were decoded from bytes and, if they were, encodes the
>return value on output [3]. If you're pipelining such interfaces, it's
>obviously more efficiently to just decode once before invoking the
>pipeline and then (optionally) encoding again at the end (just as is
>the case in Python 2), but you can still make your APIs largely
>polymorphic with respect to bytes and text without massive internal
>code duplication.

It's certainly an interesting idea, although I can't decide whether this is
more implicit or more explicit.  I'm not sure it would have helped me with
dbus-python since most of the porting work happens at the interface between
Python and an existing C API with clearly defined semantics.  There's also a
large body of existing code that uses the library, so an important goal is to
make *their* porting easier.  Have you had any experience porting applications
which use the new urllib and does it make your life easier?

>So, that's always one of my first suggestions to people struggling
>with Python 3's unicode model: I ask if they have tried putting aside
>any concerns they may have about possible losses of efficiency, and
>just tried the decode-on-input-and-return-an-output-coercion-function,
>coerce-on-output approach. Python used to do this implicitly for you
>at every string operation (minus the 'coerce on output' part), but now
>it is asking that you do it manually, and decide for *yourself* on an
>appropriate encoding, instead of the automatic assumption of ASCII
>text that is present in Python 2 (we'll leave aside the issue of
>platform-specific defaults in various contexts - that's a whole
>different question and one I'm not at all equipped to answer. I don't
>think I've ever even had to work on a system with any locale other
>than en_US or en_GB).

I also wonder if some of these ideas would help with RDM's re-imagining of the
email package.  email is notoriously quantum in its bytes/str duality -
sometimes the same email data needs to act like a str or a bytes at different
times.  The email-sig design attempts to address this with a layered approach,
although I think we're still wondering whether this is going to work in
practice.  (I'm not sure about the current state of the email package work.)

One difference here is that email usually tells you explicitly what the
encoding is.  Assuming the Content-Types don't lie or specify charsets that
are unknown to Python, I think it would be nice to pass them through the APIs
for better idempotency.

>However, that urllib.urlparse code also highlights another one of
>Armin's complaints: like much of the stdlib (and core interpreter!),
>it doesn't ducktype 'str'. Instead, it demands the real thing and
>accepts no substitutes (not even collections.UserString). This kind of
>behaviour is quite endemic - the coupling between the interpreter and
>the details of the string implementation is, in general, even tighter
>than that between the interpreter and the dict implementation used for
>namespaces.

In the dbus port, this is mostly not a problem, since the dbus-python data
types derive from PyBytes or PyUnicode.  There are one or two places that do
_CheckExacts() for various reasons, but I think mostly everything works
properly with _Check() calls.  I know that's not duck-typing though, and I
agree with you about the problem at the interpreter level.

(It turned out to be a bigger PITA with int/long.  An important requirement
was to not change the API for Python 2, so types which derived from PyInts in
Python 2 had to derive from PyLongs in Python 3.  Liberal use of #ifdefs
mostly handles the complexity, but porting clients to Python 3 will definitely
see API differences, e.g. in inheritance structure and such.)

I've rambled on enough, but I think you and Armin bring up some good points
that we really need to address in Python 3.3.  I'm fairly convinced that
there's little we could have done different before now, and most of these
issues are cropping up because people are doing actual real-world ports now,
which is a good thing!  I also strongly disagree with Armin that a Python 2.8
could help in any way.  Much better to expose these problems now and help make
Python 3.3 the best it can be.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111207/b710457c/attachment.pgp>

From ncoghlan at gmail.com  Thu Dec  8 00:51:45 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Dec 2011 09:51:45 +1000
Subject: [Python-ideas] Exploring the 'strview' concept further
In-Reply-To: <20111207163958.5ac4a9fe@pitrou.net>
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
	<20111207163958.5ac4a9fe@pitrou.net>
Message-ID: <CADiSq7d=vz-zWHZ+9xdTqyE_bmgD-8tGQ5T9sKLxbaZ7dOiqow@mail.gmail.com>

On Thu, Dec 8, 2011 at 1:39 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Alternatively, if codecs were
>> classified to explicitly indicate when they implemented stateless
>> fixed width encodings, then strview could simply be restricted to only
>> working with that subset of possible encodings.
>
> >From an usability POV this seems undesireable. On the other hand, if
> complete decoding is required, calling str() is just as cheap.

Yeah, that's kind of where I'm going with this. For stateless
encodings, views make sense - it's really just a memory view with a
particular way of interpreting the individual characters and providing
the string API rather than the bytes one. For the multitude of
ASCII-compatible single byte codings and the various fixed-width
encodings, that could be very useful. With some fiddling, you could
support BOM and signature encodings, too (just by offsetting your view
a bit and adjusting your interpretation of the individual code
points).

But for the fully general case of stateful encodings (including all
variable width encodings) it is basically impossible to do O(1)
indexing (which is the whole reason the Unicode model is the way it
is). Especially once PEP 393 is in place, you rapidly reach a point of
diminishing returns where converting the whole shebang to Unicode code
points and working directly on the code point array is the right
answer (and, if it isn't, you're clearly doing something sufficiently
sophisticated that you're going to be OK with rolling your own tools
to deal with the problem).

In those terms, I'm actually wondering if it might be appropriate to
extract some of the tools I created for the urllib.parse case and
publish them via the string module.

1. Provide a string.Text ABC (why *did* we put UserString in
collections, anyway?)
2. Provide a "coerce_to_str" helper:

    def coerce_to_str(*args, encoding, errors='strict'):
        # Invokes decode if necessary to create str args
        # and returns the coerced inputs along with
        # an appropriate result coercion function
        # - a noop for str inputs
        # - encoding function otherwise
        # False inputs (including None) are all coerced to the empty string
        args_are_text = isinstance(args[0], Text)
        if args_are_text:
            def _encode_result(obj):
                return obj
        else:
            def _encode_result(obj):
                return obj.encode(encoding, errors)
        def _decode(obj):
            if not obj:
                return ''
            if isinstance(obj, Text):
                return str(obj)
            return obj.decode(encoding, errors)
        def _decode_args(args):
            return tuple(map(_decode, args))
        for arg in args[1:]:
            # We special-case False values to support the relatively common
            # use of None and the empty string as default arguments
            if arg and args_are_text != isinstance(arg, Text):
                raise TypeError("Cannot mix text and non-text arguments")
        return _decode_args(args) + (_encode_result,)

Note the special-casing of None would be sufficient to support
arbitrary defaults in binary/text polymorphic APIs:

    def f(a, b=None):
        (a_str, b_str), _coerce_result = coerce_to_str(a, b, 'utf-8')
        if b is None:
            b_str = "Default text"

Cheers,
Nick.

>> 7. The new type would similarly support the full string API, returning
>> actual string objects rather than any kind of view.
>
> Even for slicing?

If we restricted strview to stateless encodings, then slicing could
also return views (there wouldn't be any point in returning a view for
iteration or indexing though - the view object would be bigger than
any single-character string. In fact, we could probably figure out a
cutoff whereby real strings are returned for sufficiently small
slices, too).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Thu Dec  8 01:22:32 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Dec 2011 10:22:32 +1000
Subject: [Python-ideas] Exploring the 'strview' concept further
In-Reply-To: <20111207145057.3522180f@resist.wooz.org>
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
	<20111207145057.3522180f@resist.wooz.org>
Message-ID: <CADiSq7fxBo5SCuPQsvUg0XvMmWKUWaKtMTUv6pUhLNZZHM8A0g@mail.gmail.com>

On Thu, Dec 8, 2011 at 5:50 AM, Barry Warsaw <barry at python.org> wrote:
> This means rather than returning bytes if bytes were given, at the Python
> layer, unicode is always returned. ?This I think causes the least disruption
> in user code. ?Well, we'll see as I'm now going to be porting some dbus-based
> applications.

If you look at the way the os APIs are defined, they mostly work on a
"bytes in, bytes out" model (e.g. os.listdir(), os.walk()). Where
there is no input type to reliably determine the expected output type,
then a 'b' variant is added (e.g. os.getcwdb(), os.environb). (And
yes, I do occasionally wonder if we should have a builtin "openb"
shorthand for "open(name, 'b')" with a signature that omits all the
parameters that are only valid for text mode files: "openb(file, mode,
buffering, closefd)")

An "always unicode out" model can work, too, but you need to be sure
your clients can cope with that.

One thing I like about my proposed string.coerce_to_str() API is that
you can use it to implement either approach. If you want bytes->bytes,
then you call the result coercion function, if you want to always emit
unicode, then you skip that step.

>>Now, I clearly disagree with Armin on at least one point: there
>>already *is* "one true way" to have unified text processing code in
>>Python 3. That way is the way the Python 3.2 urllib.parse module
>>handles it: as soon as it is handed something that isn't a string, it
>>attempts to decode it using a default assumed encoding (specifically
>>'ascii', at least for now). It keeps track of whether or not the
>>arguments were decoded from bytes and, if they were, encodes the
>>return value on output [3]. If you're pipelining such interfaces, it's
>>obviously more efficiently to just decode once before invoking the
>>pipeline and then (optionally) encoding again at the end (just as is
>>the case in Python 2), but you can still make your APIs largely
>>polymorphic with respect to bytes and text without massive internal
>>code duplication.
>
> It's certainly an interesting idea, although I can't decide whether this is
> more implicit or more explicit. ?I'm not sure it would have helped me with
> dbus-python since most of the porting work happens at the interface between
> Python and an existing C API with clearly defined semantics. ?There's also a
> large body of existing code that uses the library, so an important goal is to
> make *their* porting easier. ?Have you had any experience porting applications
> which use the new urllib and does it make your life easier?

No, the API updates were in response to user requests (and extended
discussions here and on python-dev). However, the use case of getting
URL components off the wire, manipulating them and putting them back
on the wire, all in an RFC compliant strict ASCII encoding made a lot
of sense, which is why we ended up going with this model (that and the
os module precedent).

> I also wonder if some of these ideas would help with RDM's re-imagining of the
> email package. ?email is notoriously quantum in its bytes/str duality -
> sometimes the same email data needs to act like a str or a bytes at different
> times. ?The email-sig design attempts to address this with a layered approach,
> although I think we're still wondering whether this is going to work in
> practice. ?(I'm not sure about the current state of the email package work.)

The email model in 3.2 (which actually makes it somewhat usable) is
quite similar to the way urllib.urlparse works (although I believe
there may be a bit more internal code duplication). The two API
updates actually co-evolved as part of the same set of discussions,
which is how they came to share the bytes->bytes, str->str philosophy.

I'm not sure about the status of email6 either, though - has anyone
heard from RDM lately?

> I've rambled on enough, but I think you and Armin bring up some good points
> that we really need to address in Python 3.3. ?I'm fairly convinced that
> there's little we could have done different before now, and most of these
> issues are cropping up because people are doing actual real-world ports now,
> which is a good thing! ?I also strongly disagree with Armin that a Python 2.8
> could help in any way. ?Much better to expose these problems now and help make
> Python 3.3 the best it can be.

Indeed - it's *already* the case that ports that can drop 2.5 support
have a much easier time of things, and those that can also drop 2.6
have it easier still (I believe the latter is uncommon though, since
most people want their stuff to run on platforms like the RHEL6 system
Python installation).

Whether or not 2.8 exists won't magically make the need to support 2.5
(or even earlier versions!) go away - the only thing that will make
that happen is time, as people upgrade their underlying OS
installations.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From barry at python.org  Thu Dec  8 04:22:37 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 7 Dec 2011 22:22:37 -0500
Subject: [Python-ideas] Exploring the 'strview' concept further
In-Reply-To: <CADiSq7fxBo5SCuPQsvUg0XvMmWKUWaKtMTUv6pUhLNZZHM8A0g@mail.gmail.com>
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
	<20111207145057.3522180f@resist.wooz.org>
	<CADiSq7fxBo5SCuPQsvUg0XvMmWKUWaKtMTUv6pUhLNZZHM8A0g@mail.gmail.com>
Message-ID: <20111207222237.42c88605@limelight.wooz.org>

On Dec 08, 2011, at 10:22 AM, Nick Coghlan wrote:

>Indeed - it's *already* the case that ports that can drop 2.5 support
>have a much easier time of things, and those that can also drop 2.6
>have it easier still (I believe the latter is uncommon though, since
>most people want their stuff to run on platforms like the RHEL6 system
>Python installation).

I think the ability to drop anything less than 2.6 is a huge win, but the
ability to drop 2.6 itself is a smaller win.  Yes, there are things in 2.7
that help make porting to Python 3 easier still, but at least IME so far, not
so much as to be overwhelmingly compelling.  Python 2.6 as a minimum *is* very
compelling.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111207/d6aca403/attachment.pgp>

From jimjjewett at gmail.com  Thu Dec  8 17:13:53 2011
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 8 Dec 2011 11:13:53 -0500
Subject: [Python-ideas] Exploring the 'strview' concept further
In-Reply-To: <CADiSq7d=vz-zWHZ+9xdTqyE_bmgD-8tGQ5T9sKLxbaZ7dOiqow@mail.gmail.com>
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
	<20111207163958.5ac4a9fe@pitrou.net>
	<CADiSq7d=vz-zWHZ+9xdTqyE_bmgD-8tGQ5T9sKLxbaZ7dOiqow@mail.gmail.com>
Message-ID: <CA+OGgf6NyNuj66Agn3AqyOZsQpFCLwM9_CjJPwSB3aikvKrngw@mail.gmail.com>

On Wed, Dec 7, 2011 at 6:51 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> For stateless
> encodings, views make sense - it's really just a memory view with a
> particular way of interpreting the individual characters and providing
> the string API rather than the bytes one. For the multitude of
> ASCII-compatible single byte codings and the various fixed-width
> encodings, that could be very useful. With some fiddling, you could
> support BOM and signature encodings, too (just by offsetting your view
> a bit and adjusting your interpretation of the individual code
> points).

I really like PEP 393, and it has gotten much better even since the
initial proposal, but this one objection has been bugging me the whole
time -- I just can't find a good way to explain it.

But with the concrete code, I will take a stab now...

I want the ability to use a more efficient string representation when
I know one exists -- such as when I could be using a single-byte
charset other than Latin-1, or when the underlying data is bytes, but
I want to treat it as text temporarily without copying the whole
buffer.

PyUnicode_Kind already supports the special case of
PyUnicode_WCHAR_KIND (also known as "legacy string, not ready" --
http://hg.python.org/cpython/file/174fbbed8747/Include/unicodeobject.h
around line 247).  I would like to see another option for "custom
subtype", and to accept that strings might stay in this state longer.

A custom type need not allow direct access to the buffer as an array,
so it would have to provide its own access functions.  I accept that
using these subtype-specific functions might be slower, but I think
the downside for "normal" strings can be limited to an extra case
statement in places like PyUnicode_WRITE (at
http://hg.python.org/cpython/file/174fbbed8747/Include/unicodeobject.h
lines 487-508; currently the default case asserts
PyUnicode_4BYTE_KIND).

Looing at Barry's example:
   >>> s = ':1/123'
   >>> s[:1] == ':'
   True

Modelling this as bytes with a unicode view on top, this would work
fine (so long as you sliced the view, rather than the original bytes
object), but creating that string view wouldn't require copying the
buffer.  (Of course, the subtype's implementation of
PyUnicode_Substring might well copy parts of the buffer.)

I would expect bytes in particular to grow an
as_string(encoding="Latin-1") method, which could be used to deprecate
the various string-related methods.

A type for an alternate one-byte encoding could be as simple as using
the 1Byte variants to create a string of the same type when large
strings are called for, and a translation function when individual
characters are requested.

-jJ


From solipsis at pitrou.net  Thu Dec  8 21:34:55 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 08 Dec 2011 21:34:55 +0100
Subject: [Python-ideas] Exploring the 'strview' concept further
In-Reply-To: <CA+OGgf6NyNuj66Agn3AqyOZsQpFCLwM9_CjJPwSB3aikvKrngw@mail.gmail.com>
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
	<20111207163958.5ac4a9fe@pitrou.net>
	<CADiSq7d=vz-zWHZ+9xdTqyE_bmgD-8tGQ5T9sKLxbaZ7dOiqow@mail.gmail.com>
	<CA+OGgf6NyNuj66Agn3AqyOZsQpFCLwM9_CjJPwSB3aikvKrngw@mail.gmail.com>
Message-ID: <1323376495.3274.16.camel@localhost.localdomain>


> But with the concrete code, I will take a stab now...
> 
> I want the ability to use a more efficient string representation when
> I know one exists -- such as when I could be using a single-byte
> charset other than Latin-1, or when the underlying data is bytes, but
> I want to treat it as text temporarily without copying the whole
> buffer.

How long is your buffer? Have you timed how long it takes to "copy" (or
decode) it?

> PyUnicode_Kind already supports the special case of
> PyUnicode_WCHAR_KIND (also known as "legacy string, not ready" --
> http://hg.python.org/cpython/file/174fbbed8747/Include/unicodeobject.h
> around line 247).  I would like to see another option for "custom
> subtype", and to accept that strings might stay in this state longer.

The unicode implementation is already complicated enough. I think adding
one further option will be a tough sell, if it doesn't exhibit major
benefits.

(note PyUnicode_WCHAR_KIND is deprecated and supposed to be removed some
day, perhaps Python 4 :-))

> I would expect bytes in particular to grow an
> as_string(encoding="Latin-1") method, which could be used to deprecate
> the various string-related methods.

Why deprecate useful functionality?

Regards

Antoine.




From stephen at xemacs.org  Fri Dec  9 05:41:41 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 09 Dec 2011 13:41:41 +0900
Subject: [Python-ideas] Exploring the 'strview' concept further
In-Reply-To: <CA+OGgf6NyNuj66Agn3AqyOZsQpFCLwM9_CjJPwSB3aikvKrngw@mail.gmail.com>
References: <CADiSq7dL1HKLQoA7Jb+3pyGO3kQ0dkcJTxV+x5zt-kKehDTxqw@mail.gmail.com>
	<20111207163958.5ac4a9fe@pitrou.net>
	<CADiSq7d=vz-zWHZ+9xdTqyE_bmgD-8tGQ5T9sKLxbaZ7dOiqow@mail.gmail.com>
	<CA+OGgf6NyNuj66Agn3AqyOZsQpFCLwM9_CjJPwSB3aikvKrngw@mail.gmail.com>
Message-ID: <878vmm1hmi.fsf@uwakimon.sk.tsukuba.ac.jp>

Jim Jewett writes:

 > I want the ability to use a more efficient string representation when
 > I know one exists -- such as when I could be using a single-byte
 > charset other than Latin-1,

For most people all of the time, and for almost all people most of the
time, this is a YAGNI, and gets more so every year.  As a facility of
the language, it is an attractive nuisance for developers many of whom
will undoubtedly go searching for truffles and end up consuming
Knuth's root of all error, and will attract lots of one-off RFEs to
deal with specific use cases that break with a minimal implementation.
N.B. Emacs has just given up on a 15-year experiment with such a
minimal facility (the execrable "string-as-unibyte" toggle).

Use of multiple internal text encodings is really a can of worms, as
the Emacs experience demonstrates (they were unable to even write
Latin-1 files properly, with repeated regressions of the so-called
"\201 bug" that I know of 1995-2008, mostly because of misuse of
string-as-unibyte).  XEmacs, with a proper character type, eliminated
the "\201 bug" *before* its multilingual version stopped crashing in
the codecs.  But even there, because the internal character type is
based on ISO-2022, it sucks, and we consistent have issues with bogus
decoding and the like that is hard to get around at the app level
because there's way too much generality at the underlying level that
we try to handle "transparently".

That's where you're going; maybe you can do better than XEmacs levels
of suckiness :-), but (for a general facility) it won't be easy.
Better to do that at the application level, which can decide for
itself what safeguards are needed.

 > or when the underlying data is bytes, but I want to treat it as
 > text temporarily without copying the whole buffer.

That ship has sailed AFAICS.  If the "copy-the-whole-buffer" style of
polymorphism isn't good enough, you have special knowledge of the data
and/or the application, and it's a layering violation to ask Python to
manage that data for you because Python's model of text is str.  It
will result in unexpected UnicodeErrors.

 > A custom type need not allow direct access to the buffer as an array,
 > so it would have to provide its own access functions.  I accept that
 > using these subtype-specific functions might be slower, but I think
 > the downside for "normal" strings can be limited to an extra case
 > statement in places like PyUnicode_WRITE

I expect that library code that must be robust against UnicodeError
(eg, email) will need to be prepared for gratuitous errors from custom
types.  Since that's at least a desideratum for all stdlib code, this
could be rather more expensive than you suggest.

 > A type for an alternate one-byte encoding could be as simple as using
 > the 1Byte variants to create a string of the same type when large
 > strings are called for, and a translation function when individual
 > characters are requested.

This would require substantial analysis in some cases I would expect
to be common to determine whether it wasn't a pessimization.  I
suppose that in many use cases, you will be implicitly creating many
strings, and the space overhead of the implicit strings may be greater
than the size of the single string.

In cases where the analysis is simple (eg, parsing an RFC 822 message
header out of the middle of a huge mbox file), the analysis that shows
that this could be done efficiently with a custom type can easily and
efficiently be converted to an implementation based on converting only
the bytes needed.

I understand the attraction of such facilities for simplifying user
code, but given my somewhat extensive experience with maintaining
them, I recommend that Python core Just Say No.  It's just too hard to
maintain "text invariants" when you might be processing a few million
bytes from /dev/urandom.  If one (as an application programmer) knows
better, and of course she does, then shouldn't she DTRT at the
application code level?


From dirkjan at ochtman.nl  Fri Dec  9 21:48:58 2011
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Fri, 9 Dec 2011 21:48:58 +0100
Subject: [Python-ideas] [Python-Dev] [PATCH] Adding braces to __future__
In-Reply-To: <20111209202629.GB2319@slate.Speedport_W_723V_Typ_A>
References: <20111209202629.GB2319@slate.Speedport_W_723V_Typ_A>
Message-ID: <CAKmKYaBeJRi=--hgBpHVr6=kmfAxPPhasn=fqfVag03L5zPk2g@mail.gmail.com>

On Fri, Dec 9, 2011 at 21:26, Cedric Sodhi <manday at gmx.net> wrote:
> (not proof-read)

A good sign that it's too long to inflict upon others...

Also, I scrolled and scrolled but failed to see any example of the
kind of change you seem to want.

Also, if you really really had read all the prior discussions, you'd
have known that this post belongs on python-ideas, not python-dev.
Redirecting there.

Cheers,

Dirkjan


From ncoghlan at gmail.com  Sat Dec 10 06:06:13 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 10 Dec 2011 15:06:13 +1000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For Python?
 (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
Message-ID: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>

A more appropriate way to phrase this suggestion would be one that
clearly states the problems with the status quo and explores them,
rather than assuming an answer, and indulging in a long rambling
diatribe against those that will shout you down for adding nothing new
to the discussion.

Step 1: post to the right mailing list (i.e. python-ideas, not python-dev)

Step 2: don't assume the conclusion in the subject line. Ask a
question, don't state an imperative. For example: "An Alternate Suite
Delineation Syntax For Python?"

Step 3: respect the time of others, by trimming your message to its core essence

For example:

An Alternate Suite Delineation Syntax For Python?
=================================================

Python's whitespace based delineation of suites is one of its greatest
strengths. It aligns what the human reader perceives with what the
computer is actually executing, reducing the frequency of semantic
errors due to mismatches between the use of separate block delimiters
and the human readable indentation.

However, this benefit comes at quite a high price: it is effectively
impossible to embed arbitrary Python statements into any environment
where leading whitespace is *not* significant, including Python's own
expression syntax.

It can be argued that this restriction has led directly to the
introduction of "expression friendly" variants of several Python top
level constructs (for example, lambda expressions, conditional
expressions and as a contributing factor in creating the various forms
of comprehension).

It is also one of the reasons Python-based templating languages almost
always create their own custom syntax - embedding Python's own
whitespace sensitive statement syntax into environments where leading
whitespace is either ignored or forms a significant part of the
template output is a formidable challenge.

In other languages, this kind of issue is handled by using explicit
suite and statement delimiters (often braces and semi-colons,
respectively) to allow full suites to be used as expressions.

While Python uses braces for another purpose (dictionary and set
definitions), it is already the case that semi-colons (';') can be
used as statement terminators, both optionally at the end of any
simple statement, and also to combine multiple simple statements into
a single larger statement (e.g. "x += y; print(x)").

It seems that this existing feature could be combined with a
brace-based notation to create an unambiguous "suite expression"
syntax that would enjoy the same semantics as ordinary Python suites
(i.e. doesn't create a new scope, doesn't directly affect control
flow), but allows *all* Python statements to be embedded inside
expressions.

Currently, the character sequence "{:" is a Syntax Error: you are
attempting to end a compound statement header line while an opening
brace remains unmatched, or else trying to build a dictionary without
specifying the key value. This creates an opportunity to re-use braces
for a suite expression syntax without conflicting with their use for
set and dictionary construction.

Specifically, it should be possible to create a variant of the
top-level Python syntax that:
  1. Explicitly delimits suites using the notation "{:" to open the
suite and "}" to end it
  2. Requires the use of ";" to separate simple statements (i.e.
newline characters would not end a statement, since we would be inside
an expression)
  3. Requires that all subordinate suites also be suite expressions
(i.e. leading whitespace would not be significant, since we would be
inside an expression)
  4. Returns the value of the last statement in the suite as the
result of the suite expression (since return statements would affect
the containing scope)
  5. Anonymous class and function definitions would be permitted in a
suite expression (but accepting only a suite expression instead of an
ordinary suite)

(Ruby block notation and C's comma expressions are pretty much direct
inspiration for the above list)

Some examples:

  Raise expressions:

    x = y if y is not None else {: raise ValueError("y must not be None!")}

  Try expressions:

    x = {: try {: y.hello} except AttributeError {: "world!"}}

  With expressions:

    data = {: with open(fname) as f {: f.read()}}

  Embedded assignments:
    if {: m = pat.search(data); m is not None}:
        # do something with m
    else:
        # No match!

  In-order conditional expressions:
    {: if a {:b} else {:c}}

  One-liner accumulator function:
    def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return s}}}

Cheers,
Nick.

P.S. I actually think the above idea is kinda cryptic and ugly. It is,
however, extraordinarily powerful, especially when it comes to
embedding Python code inside other environments that may not be
friendly to significant leading whitespace.

Applying the excellent set of criteria suggested by Mike Meyer:

1. What's the use case? Cleanly embedding arbitrary Python statements
inside environments that are not generally sensitive to leading
whitespace, such as templating languages, Python expressions and
Python strings.
2. Does it make such code more readable? Yes, I believe it does, by
providing "one obvious way to do it". Currently there are a mish-mash
of alternatives out there (or else people give up and use something
else entirely, like Javascript).
3. Does it encourage writing unreadable code? Perhaps. You certainly
end up with a lot of braces, colons and semi-colons floating around.
However, that's also something that can be addressed by style guides -
if people are using the embedded syntax when the indented syntax would
work fine, the problem isn't really due to the embedded syntax, it's
due to people not caring about maintainability.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From tjreedy at udel.edu  Sat Dec 10 06:08:57 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 10 Dec 2011 00:08:57 -0500
Subject: [Python-ideas] [Python-Dev] [PATCH] Adding braces to __future__
In-Reply-To: <CAKmKYaBeJRi=--hgBpHVr6=kmfAxPPhasn=fqfVag03L5zPk2g@mail.gmail.com>
References: <20111209202629.GB2319@slate.Speedport_W_723V_Typ_A>
	<CAKmKYaBeJRi=--hgBpHVr6=kmfAxPPhasn=fqfVag03L5zPk2g@mail.gmail.com>
Message-ID: <jbuphn$kcp$1@dough.gmane.org>

On 12/9/2011 3:48 PM, Dirkjan Ochtman wrote:
> On Fri, Dec 9, 2011 at 21:26, Cedric Sodhi<manday at gmx.net>  wrote:
>> (not proof-read)
>
> A good sign that it's too long to inflict upon others...
>
> Also, I scrolled and scrolled but failed to see any example of the
> kind of change you seem to want.
>
> Also, if you really really had read all the prior discussions, you'd
> have known that this post belongs on python-ideas, not python-dev.
> Redirecting there.

Rude trollish rants on something that isn't going to change do not 
belong here either. They belong on python-list or a personal blog.

FWIW I use Python *because* it uses indents and not braces.

-- 
Terry Jan Reedy



From cs at zip.com.au  Sat Dec 10 08:30:24 2011
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 10 Dec 2011 18:30:24 +1100
Subject: [Python-ideas] Adding braces to __future__
In-Reply-To: <4EE2A140.2010509@pearwood.info>
References: <4EE2A140.2010509@pearwood.info>
Message-ID: <20111210073023.GA31670@cskk.homeip.net>

On 10Dec2011 11:01, Steven D'Aprano <steve at pearwood.info> wrote:
| Guido van Rossum wrote:
| >Point of order (repeated), please move this thread to python-ideas.
| 
| Isn't that cruel to the people reading python-ideas?

That's what it's _for_!
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

A monkey has the right to copy what he sees other monkeys doing.
Shouldn't humans have equivalent rights?
- Alien Being <http://slashdot.org/~Alien%20Being>


From ron3200 at gmail.com  Sat Dec 10 08:33:01 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Sat, 10 Dec 2011 01:33:01 -0600
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
Message-ID: <1323502381.19113.36.camel@Gutsy>

On Sat, 2011-12-10 at 15:06 +1000, Nick Coghlan wrote:
>   One-liner accumulator function:
>     def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return
> s}}} 

>>> def acc(n):s=n; return lambda i, s=s: s+i
... 
>>> a = acc(2)
>>> a(1)
3
>>> a(3)
5
>>> a(7)
9

 ;-)

In your example returning an unnamed function is a new feature in
addition to the braces.  It would probably need to be more like this...

def acc(n){:s=n; def add(i){:nonlocal s; s+=i; return s};return add}

There are a couple of issues to work out.  Statements (def, class, for,
while, and with) don't work after simi-colons. There may be others.

Outside those issues, all you really need is an exit suit token.  The
colon can still be the open suit.  Two semi-colons could do it.

def acc(n):s=n; def add(i):nonlocal s; s+=i; return s;; return add

The real issues are in getting the statements to work.

Cheers,
   Ron




From p.f.moore at gmail.com  Sat Dec 10 12:32:46 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 10 Dec 2011 11:32:46 +0000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
Message-ID: <CACac1F9KtwJBJcwW09FCRyBgK28=jC=mOh9VmPuH+cBTYo90Eg@mail.gmail.com>

On 10 December 2011 05:06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> A more appropriate way to phrase this suggestion would be one that
> clearly states the problems with the status quo and explores them,
> rather than assuming an answer, and indulging in a long rambling
> diatribe against those that will shout you down for adding nothing new
> to the discussion.
>
> Step 1: post to the right mailing list (i.e. python-ideas, not python-dev)
>
> Step 2: don't assume the conclusion in the subject line. Ask a
> question, don't state an imperative. For example: "An Alternate Suite
> Delineation Syntax For Python?"
>
> Step 3: respect the time of others, by trimming your message to its core essence
>
> For example:
[...]

This, along with the original posting on python-dev, is a spectacular
example of how to propose a new feature, and how not to. It would make
an excellent example for a HOWTO document on proposing changes (with
the OP's permission - he might not like his posting being held op as
the canonical "how not to do this" example...) I barely read the
original posting, even though I tried to follow it - my mind was
screaming "ain't never going to happen" at me throughout. And yet
Nick's rewording is pretty persuasive, I can see the benefits and the
power of the idea. My only major objection is that the resulting code
is pretty ugly. I could easily imagine something along these lines
being accepted into Python.

Thanks, Nick, for doing this - and Cedric for the original idea, even
though I didn't appreciate it at the time :-) It certainly got me
thinking.

As regards the actual proposal, some comments:
- It's got serious power (if, try and with expressions, etc etc)
- It would answer a number of requests that keep coming up over and over again.
- I suspect the semantics in corner cases will be hard to pin down,
but the basic principle looks simple enough
- The code is pretty ugly, and certainly an attractive nuisance for
obfuscated code, but I can't think of a better syntax
- Given that it effectively replaces lambdas and if expressions, it
partly violates OOWTDI - and it does so in a way that could encourage
people to still propose try or with expressions ("they are as common
as conditionals, so why not?")

Paul.


From masklinn at masklinn.net  Sat Dec 10 12:50:08 2011
From: masklinn at masklinn.net (Masklinn)
Date: Sat, 10 Dec 2011 12:50:08 +0100
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
	Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CACac1F9KtwJBJcwW09FCRyBgK28=jC=mOh9VmPuH+cBTYo90Eg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<CACac1F9KtwJBJcwW09FCRyBgK28=jC=mOh9VmPuH+cBTYo90Eg@mail.gmail.com>
Message-ID: <0624B9B6-23B1-488A-BB80-D1EE4C578935@masklinn.net>

On 2011-12-10, at 12:32 , Paul Moore wrote:
> - The code is pretty ugly, and certainly an attractive nuisance for
> obfuscated code, but I can't think of a better syntax
There are always the possibilities of keeping the colon as a start
token and using an other un-matched token as an end.

An other possibility (with some precedent in generator comprehensions)
would be to declare current statements to be expressions (and define
their expressive semantics) and surround them with parens for
disambiguation purposes when necessary.

> - Given that it effectively replaces lambdas and if expressions, it
> partly violates OOWTDI
For lambdas, as far as I can tell `def` would still require `return`,
it is pretty common for languages to provide a shorter form for
single-expression bodies. Of course, this could also be resolved by
converting defs to implicit returns and removing `lambda` altogether.
This change, however, would break a lot of code.

> - and it does so in a way that could encourage
> people to still propose try or with expressions ("they are as common
> as conditionals, so why not?")
Considering these are both included in Nick's proposal (which is a
general transformation on existing statements, not a special-cased
expressive form of some), I don't think that is a valid objection:

On 2011-12-10, at 06:06 , Nick Coghlan wrote:
> Some examples:
> 
>  Try expressions:
> 
>    x = {: try {: y.hello} except AttributeError {: "world!"}}
> 
>  With expressions:
> 
>    data = {: with open(fname) as f {: f.read()}}



From richard.prosser at mail.com  Sat Dec 10 15:16:21 2011
From: richard.prosser at mail.com (Richard Prosser)
Date: Sat, 10 Dec 2011 14:16:21 +0000
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section to
	the tutorial
Message-ID: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>

Although I love Python there are some aspects of the language design which
are disappointing and which can even lead to problems in some cases.

A classic example is a mutable default argument having the potential to
produce unexpected side-effects, as a consequence of the non-intuitive
scoping rules.

Another awkward 'feature' is the requirement for a trailing comma in
singleton tuples, due I believe to the use of expression parentheses rather
than (say) the use of special brackets like chevrons.

Something that I personally wish for is the ability to declare variable
types 'up front' but that facility is missing from Python.


This is an important issue, so I propose that the Python tutorial be
updated to highlight such problems. I would be willing to write a draft
section myself but obviously it would need to be reviewed.

I am not sure if this is the appropriate place to make such a comment but
it seems to be a good starting point. Any advice on making a more formal
proposal would be welcome.


Cheers,

Richard Prosser
PS Is it too late to fix such warts in version 3?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111210/a4f4d507/attachment.html>

From masklinn at masklinn.net  Sat Dec 10 15:41:14 2011
From: masklinn at masklinn.net (Masklinn)
Date: Sat, 10 Dec 2011 15:41:14 +0100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
Message-ID: <28FB8C6C-D4B0-45E5-9535-6D8F6B37FCB5@masklinn.net>

On 2011-12-10, at 15:16 , Richard Prosser wrote:
> 
> A classic example is a mutable default argument having the potential to
> produce unexpected side-effects, as a consequence of the non-intuitive
> scoping rules.
As far as I know, mutable default arguments have nothing to do with
scoping, they have to do with the "toplevel" being evaluated fully, so
as the function declaration is evaluated (to create the function
object) so are its default arguments. This is independent from
Python's scoping issues unless I misunderstood what you meant by
"scoping".

But this is definitely something which trips people. However, there is
already a note (though a pretty low-key one, it should probably use an
actual warning directive instead of just bolding it, you should submit
a documentation patch) in the tutorial on that subject[0].

> Another awkward 'feature' is the requirement for a trailing comma in
> singleton tuples, due I believe to the use of expression parentheses rather
> than (say) the use of special brackets like chevrons.
For tuples, there are no matching operators left, as literal sets have
been added.

And technically, the irregularity with tuples is probably the empty
tuple `()` as parens in other tuple arities are only necessary for
disambiguation (much like parens around generator expressions): the
"tuple constructor" is the comma, not the parens,

    a = 1,
    b = 1, 2
    c = 1, 2, 3

are all valid and generate respectively a singleton, a pair and a
triple. In that context, the trailing comma for singletons makes
sense. If you want regularity, you can even add a trailing comma to
the pair and the triple (as you can in e.g. a list or a dict):

    a = 1,
    b = 1, 2,
    c = 1, 2, 3,

I'd rather have a lone comma (with or without parens, depending on the
context) create a null tuple.

> Something that I personally wish for is the ability to declare variable
> types 'up front' but that facility is missing from Python.
I fail to see how this is a "gotcha": since Python is dynamically
typed names don't have types (well technically Python 3 added
documentary type specs to arguments, but they're not used by any
implementation I know of though some third-party tools may already
have started using them)

[0] http://docs.python.org/tutorial/controlflow.html#default-argument-values


From fuzzyman at gmail.com  Sat Dec 10 16:27:20 2011
From: fuzzyman at gmail.com (Michael Foord)
Date: Sat, 10 Dec 2011 15:27:20 +0000
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
Message-ID: <CAKCKLWzM852jF9VxQKyxLKHE_enHePjQoARA3qEzR7AEN129SQ@mail.gmail.com>

On 10 December 2011 14:16, Richard Prosser <richard.prosser at mail.com> wrote:

> Although I love Python there are some aspects of the language design which
> are disappointing and which can even lead to problems in some cases.
>
> A classic example is a mutable default argument having the potential to
> produce unexpected side-effects, as a consequence of the non-intuitive
> scoping rules.
>

The default argument "problem" is not intuitive (and does need highlighting
to people new to Python). But it is still better than the alternatives. It
isn't to do with scoping but when the default arguments are evaluated. If
instead the default values were evaluated at call time, what would the
following code do:

a = 3
def function(arg=a):
    pass
del a
function()

In addition you have added extra overhead cost to each function call
(argument evaluation). Plus Python exposes the default arguments to
introspection. If the default arguments aren't evaluated until call time
you lose that capability.


>
> Another awkward 'feature' is the requirement for a trailing comma in
> singleton tuples, due I believe to the use of expression parentheses rather
> than (say) the use of special brackets like chevrons.
>
>
Well, parentheses are only part of the syntax for an empty tuple: ()

For non-empty tuple literals it is *only* the commas that are significant.

    single = 1,
    double = 2, 3

So although the syntax for a single member tuple does feel a bit awkward I
wonder what alternative you would suggest?



> Something that I personally wish for is the ability to declare variable
> types 'up front' but that facility is missing from Python.
>
>
That runs counter to the basic principles of Python where types are
determined at runtime. What effect would declaring types have and how would
that interact with the rest of the language? (Do you want every assignment
to do a runtime type check?)


>
> This is an important issue, so I propose that the Python tutorial be
> updated to highlight such problems. I would be willing to write a draft
> section myself but obviously it would need to be reviewed.
>
>
This is a good idea however. :-) Just do it. Create a documentation patch
(for Python 3) and attach it to an issue in the bug tracker:

    http://bugs.python.org/


> I am not sure if this is the appropriate place to make such a comment but
> it seems to be a good starting point. Any advice on making a more formal
> proposal would be welcome.
>
>
> Cheers,
>
> Richard Prosser
> PS Is it too late to fix such warts in version 3?
>

For the issues you've raised, yes. Unless you have specific proposals that
don't break backwards compatibility.

All the best,

Michael Foord



>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111210/a9f9b7c7/attachment.html>

From solipsis at pitrou.net  Sat Dec 10 18:45:54 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 10 Dec 2011 18:45:54 +0100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
Message-ID: <20111210184554.475bac43@pitrou.net>


Hello Richard,

On Sat, 10 Dec 2011 14:16:21 +0000
Richard Prosser <richard.prosser at mail.com> wrote:
> 
> This is an important issue, so I propose that the Python tutorial be
> updated to highlight such problems. I would be willing to write a draft
> section myself but obviously it would need to be reviewed.

I think documenting "gotchas" can be useful indeed. However, I'm not
sure the tutorial is the right place: it should present an easy to
grasp view of the language, not digress about edge cases.

So perhaps a FAQ, for example, would be more appropriate.

In any case, feel free to propose a draft on http://bugs.python.org
You can take a look at http://docs.python.org/dev/documenting/ if you
are not familiar with the process.

Regards

Antoine.




From ned at nedbatchelder.com  Sat Dec 10 19:34:15 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sat, 10 Dec 2011 13:34:15 -0500
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
Message-ID: <4EE3A627.8010002@nedbatchelder.com>

On 12/10/2011 9:16 AM, Richard Prosser wrote:
> Although I love Python there are some aspects of the language design 
> which are disappointing and which can even lead to problems in some cases.
>
> A classic example is a mutable default argument having the potential 
> to produce unexpected side-effects, as a consequence of the 
> non-intuitive scoping rules.
>
> Another awkward 'feature' is the requirement for a trailing comma in 
> singleton tuples, due I believe to the use of expression parentheses 
> rather than (say) the use of special brackets like chevrons.
>
> Something that I personally wish for is the ability to declare 
> variable types 'up front' but that facility is missing from Python.
>
One of these things is not like the others.  Mutable default arguments, 
and singleton tuples are surprises that make sense once you understand 
things on a deeper level.  It makes sense to call them out as a 
"gotcha": a common stumbling block for learners.  But "no type 
declarations" is not a wart, it's a fundamental feature of the language 
that is immediately apparent from the first lesson.

--Ned.


From greg.ewing at canterbury.ac.nz  Sat Dec 10 22:55:06 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 11 Dec 2011 10:55:06 +1300
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CACac1F9KtwJBJcwW09FCRyBgK28=jC=mOh9VmPuH+cBTYo90Eg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<CACac1F9KtwJBJcwW09FCRyBgK28=jC=mOh9VmPuH+cBTYo90Eg@mail.gmail.com>
Message-ID: <4EE3D53A.1030804@canterbury.ac.nz>

Paul Moore wrote:
> - The code is pretty ugly, and certainly an attractive nuisance for
> obfuscated code, but I can't think of a better syntax

Maybe that's actually an advantage? It would help to
discourage people from using it in code intended for
humans.

-- 
Greg


From mwm at mired.org  Sun Dec 11 01:16:08 2011
From: mwm at mired.org (Mike Meyer)
Date: Sat, 10 Dec 2011 16:16:08 -0800
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
Message-ID: <20111210161608.13313643@bhuda.mired.org>

On Sat, 10 Dec 2011 15:06:13 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> An Alternate Suite Delineation Syntax For Python?
> =================================================
>
> However, this benefit comes at quite a high price: it is effectively
> impossible to embed arbitrary Python statements into any environment
> where leading whitespace is *not* significant, including Python's own
> expression syntax.

Since you sent it to Python-ideas, I'm going to assume you were at
least semi-serious. So let me add the use case I came up with while
reading the original post, also often seen in template languages:

    Writing code to generate python.

With significant whitespace, you have to keep explicit track of the
current nesting level in order to generate the appropriate
indent. With real delimiters, you can drop that. Compare:

    def generate(self):
        yield '{keyword} {expr}{:'.format(keyword=self.name,
                                         expr=self.expresion)
        for s in self.suite:
	    for l in s.generate():
	        yield l

With:

    def generate(self, level):
        yield '{0}{keyword} {expr}:'.format(' ' * level,
                                            keyword=self.name,
                                            expr=self.expresion)
        for s in self.suite:
       	    for l in s.generate(level + 1):
	    	yield l


A minor improvement in just that one method, but it will show up in
every generate method of every class, making it significant in total.

> Applying the excellent set of criteria suggested by Mike Meyer:

"Suggested" may be a bit to much credit. I've observed the community
applying these criteria to pretty much every idea. Some ideas get
adopted in spite of failing one or more of them, but a *lot* of ideas
get dropped after failing one of them.

In this case:

1. What's the use case?

Writing code that's going to write python code.

> 2. Does it make such code more readable?

It makes the code generating code less complicated, which means more
readable.

> 3. Does it encourage writing unreadable code? Perhaps. You certainly
> end up with a lot of braces, colons and semi-colons floating around.
> However, that's also something that can be addressed by style guides -
> if people are using the embedded syntax when the indented syntax would
> work fine, the problem isn't really due to the embedded syntax, it's
> due to people not caring about maintainability.

I think not. As someone else observed, Haskell has both options
available - and the rules for the indented version are a lot less
straightforward than they are in Python (at least I haven't seen as
clean a statement of them). About the only use of the delimited
version in real code is turning a short suite into a one-line suite.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From tjreedy at udel.edu  Sun Dec 11 01:46:11 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 10 Dec 2011 19:46:11 -0500
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
Message-ID: <jc0uh3$i6$1@dough.gmane.org>

On 12/10/2011 12:06 AM, Nick Coghlan wrote:

>    4. Returns the value of the last statement in the suite as the
> result of the suite expression (since return statements would affect
> the containing scope)

I believe you mean return the value of the last statement that is 
executed, as in the try-except example or the conditional example:
     {: if a {:b} else {:c}}

I must congratulate you, Nick, for being the first person in my 14 years 
with Python to make a brace idea look half-way sensible. I can see that 
algorithmically generated code (which includes templates) might be a 
real use case.

-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Sun Dec 11 02:03:02 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 11 Dec 2011 14:03:02 +1300
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
Message-ID: <4EE40146.20904@canterbury.ac.nz>

Nick Coghlan suggested:

>     x = y if y is not None else {: raise ValueError("y must not be None!")}

I'd want to see {: closed with :} for symmetry.

-- 
Greg


From ncoghlan at gmail.com  Sun Dec 11 02:21:01 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Dec 2011 11:21:01 +1000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <jc0uh3$i6$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<jc0uh3$i6$1@dough.gmane.org>
Message-ID: <CADiSq7c8O1ZL0KVZVnh_0y7P6wX9AXZh-ANuWhk0yzECAG=k5w@mail.gmail.com>

On Sun, Dec 11, 2011 at 10:46 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/10/2011 12:06 AM, Nick Coghlan wrote:
>
>> ? 4. Returns the value of the last statement in the suite as the
>> result of the suite expression (since return statements would affect
>> the containing scope)
>
>
> I believe you mean return the value of the last statement that is executed,
> as in the try-except example or the conditional example:
>
> ? ?{: if a {:b} else {:c}}

Indeed, that was what I meant, albeit not what I wrote :)

> I must congratulate you, Nick, for being the first person in my 14 years
> with Python to make a brace idea look half-way sensible. I can see that
> algorithmically generated code (which includes templates) might be a real
> use case.

I actually surprised myself - I started writing that purely as a
devil's advocate "if you're going to rant about something to that
extent, you may as well *try* to be constructive" post, but the end
result was significantly less terrible than I originally expected :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From python at mrabarnett.plus.com  Sun Dec 11 02:33:27 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 11 Dec 2011 01:33:27 +0000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE40146.20904@canterbury.ac.nz>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<4EE40146.20904@canterbury.ac.nz>
Message-ID: <4EE40867.2000504@mrabarnett.plus.com>

On 11/12/2011 01:03, Greg Ewing wrote:
> Nick Coghlan suggested:
>
>> x = y if y is not None else {: raise ValueError("y must not be None!")}
>
> I'd want to see {: closed with :} for symmetry.
>
That's just what I was thinking. (Honest! :-))


From steve at pearwood.info  Sun Dec 11 02:55:51 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 11 Dec 2011 12:55:51 +1100
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <20111210161608.13313643@bhuda.mired.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
Message-ID: <4EE40DA7.4020702@pearwood.info>

Mike Meyer wrote:

> Since you sent it to Python-ideas, I'm going to assume you were at
> least semi-serious. So let me add the use case I came up with while
> reading the original post, also often seen in template languages:
> 
>     Writing code to generate python.
> 
> With significant whitespace, you have to keep explicit track of the
> current nesting level in order to generate the appropriate
> indent. With real delimiters, you can drop that. Compare:

That's hardly an onerous requirement. As you show, the two code snippets are 
almost identical, except that the indent version takes an explicit level 
argument, while your braces version neglects to close suites at all. A fair 
comparison should show the close-suite code, even if it is only a single line.

>     def generate(self):
>         yield '{keyword} {expr}{:'.format(keyword=self.name,
>                                          expr=self.expresion)
>         for s in self.suite:
> 	    for l in s.generate():
> 	        yield l

I guess you would need to yield '}' after the outer for loop.

> With:
> 
>     def generate(self, level):
>         yield '{0}{keyword} {expr}:'.format(' ' * level,
>                                             keyword=self.name,
>                                             expr=self.expresion)
>         for s in self.suite:
>        	    for l in s.generate(level + 1):
> 	    	yield l
> 
> 
> A minor improvement in just that one method, but it will show up in
> every generate method of every class, making it significant in total.

I wouldn't use the term "minor improvement" so much as "trivial difference".

But even if I grant that it is an improvement, how many generate methods do 
you expect there to be? Presumably most of them will inherit their generate 
method from one or two parent classes. Even if every block keyword gets an 
independent copy in violation of DRY, there's still fewer than a dozen classes 
needed: def class for while if elif else try except finally with. So the total 
significance is still likely to be small.

But wait... now you have your code generator, which generates ugly, unreadable 
code. Code needs to be read more often than written, even generated code, so 
now you need a code beautifier to reformat it using indentation for the human 
reader. Which means keeping track of the indent level. You're right back where 
you started, only worse, because now you have to parse the generated code to 
reformat it, instead of formatting it correctly the first time. So to save a 
dozen lines in your code generator, you end up with an extra hundred lines in 
a code beautifier. A classic example of code pessimation (of LOC rather than 
speed in this case).

I think it is suggestive that we will need to warn people not to use this 
hypothetical syntax. As Nick wrote:

>> However, that's also something that can be addressed by style guides -
>> if people are using the embedded syntax when the indented syntax would
>> work fine, the problem isn't really due to the embedded syntax, it's
>> due to people not caring about maintainability.

People who don't care about maintainability don't care about readability and 
they certainly don't read style guides unless they are forced to.

> I think not. As someone else observed, Haskell has both options
> available - and the rules for the indented version are a lot less
> straightforward than they are in Python (at least I haven't seen as
> clean a statement of them). About the only use of the delimited
> version in real code is turning a short suite into a one-line suite.

When you defend a feature on the basis that "people will hardly ever use it, 
except for trivial pieces of code that don't need it", that speaks volumes.




-- 
Steven


From julian at grayvines.com  Sun Dec 11 06:07:23 2011
From: julian at grayvines.com (Julian Berman)
Date: Sun, 11 Dec 2011 00:07:23 -0500
Subject: [Python-ideas] contextlib.maybe
Message-ID: <20111211050723.GA12197@air>

Twice recently I've found myself wanting to write the following code:

    def fn(a_file=None):
        responsible_for_closing = False

        if a_file is None:
            a_file = open(a_default_location)
            responsible_for_closing = True

        do_stuff(a_file)

        if responsible_for_closing:
            a_file.close()

which can be written slightly shorter I know, but it's still a tiny bit
messy and repetitive. What I'd prefer to write is something more like:

    def fn(a_file=None):
        with contextlib.maybe(a_file, open, default) as a_file:
            do_stuff(a_file)

where `maybe` takes an object and conditionally runs a context manager
if a check fails. Implementation would be:


    @contextlib.contextmanager
    def maybe(got, contextfactory, *args, checkif=bool, **kwargs):
        if checkif(got):
            yield got
        else:
            with contextfactory(*args, **kwargs) as got:
                yield got


It's hard to gauge utility for such simple functions (though contextlib
already has closing(), so I figured it'd be worth asking at least).
Would this be useful to others? Or perhaps I'm completely missing
something and you've got suggestions on how to better have an API where
an argument can be fetched if not provided but a context manager would
preferably need to be run to do so.


From tjreedy at udel.edu  Sun Dec 11 07:02:30 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 11 Dec 2011 01:02:30 -0500
Subject: [Python-ideas] contextlib.maybe
In-Reply-To: <20111211050723.GA12197@air>
References: <20111211050723.GA12197@air>
Message-ID: <jc1h1n$nfe$1@dough.gmane.org>

On 12/11/2011 12:07 AM, Julian Berman wrote:
> Twice recently I've found myself wanting to write the following code:
>
>      def fn(a_file=None):
>          responsible_for_closing = False
>
>          if a_file is None:
>              a_file = open(a_default_location)
>              responsible_for_closing = True
>
>          do_stuff(a_file)
>
>          if responsible_for_closing:
>              a_file.close()
>
> which can be written slightly shorter I know, but it's still a tiny bit
> messy and repetitive. What I'd prefer to write is something more like:
>
>      def fn(a_file=None):
>          with contextlib.maybe(a_file, open, default) as a_file:
>              do_stuff(a_file)

Expecting contextlib to have such a specialized context manager that 
does exactly what you want is perhaps too much. However, you should be 
able to write a class yourself that keeps the flag and does the 
conditional opening and closing in the __enter__ and __exit__ methods.

-- 
Terry Jan Reedy



From masklinn at masklinn.net  Sun Dec 11 09:44:17 2011
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 11 Dec 2011 09:44:17 +0100
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
	Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <20111210161608.13313643@bhuda.mired.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
Message-ID: <6048C036-E479-4BFA-A643-F79EECD9C2A7@masklinn.net>

On 2011-12-11, at 01:16 , Mike Meyer wrote:
> A minor improvement in just that one method, but it will show up in
> every generate method of every class, making it significant in total.
If users are not supposed to read the generated code, why not generate
pyc files directly? That would avoid the issue would it not?

And if users are supposed to edit (therefore read) the generated code,
wouldn't you want to make the file actually readable, therefore use
indentation-based blocks?

> I think not. As someone else observed, Haskell has both options
> available - and the rules for the indented version are a lot less
> straightforward than they are in Python (at least I haven't seen as
> clean a statement of them).

The short version is:
http://www.haskell.org/onlinereport/lexemes.html#lexemes-layout
* when a brace is omitted after a `where`, `let`, `of` or `do` keyword,
  a brace is inserted and the indentation of the next lexeme is
  remembered
* if the next lexeme is indented below the keyword, immediately insert
  a closing brace (before the lexeme itself)
* otherwise, for each subsequent line
  - if it contains only whitespace or is indented further than the
    current level, continue the current expression
  - if it is indented at the current level, insert a semicolon
  - if it is indented below the current level, insert a closing brace

You can find the complete and more formal definition at
http://www.haskell.org/onlinereport/syntax-iso.html#layout



From ncoghlan at gmail.com  Sun Dec 11 11:30:45 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 11 Dec 2011 20:30:45 +1000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE40DA7.4020702@pearwood.info>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info>
Message-ID: <CADiSq7cUSmeU-kQjGmh_jiGmy2B06Ogxx_w4WmBKxgX1iY_V+g@mail.gmail.com>

On Sun, Dec 11, 2011 at 11:55 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Mike Meyer wrote:
>
>> Since you sent it to Python-ideas, I'm going to assume you were at
>> least semi-serious. So let me add the use case I came up with while
>> reading the original post, also often seen in template languages:
>>
>> ? ?Writing code to generate python.
>>
>> With significant whitespace, you have to keep explicit track of the
>> current nesting level in order to generate the appropriate
>> indent. With real delimiters, you can drop that. Compare:
>
>
> That's hardly an onerous requirement. As you show, the two code snippets are
> almost identical, except that the indent version takes an explicit level
> argument, while your braces version neglects to close suites at all. A fair
> comparison should show the close-suite code, even if it is only a single
> line.

Indeed, that's a reason I didn't use straight up code generation as my
motivating use case - the significant whitespace simply isn't that
painful in that case. The reason significant leading whitespace can be
a PITA is due to two main circumstances:

1. Attempting to transport it through a channel that either strips
leading and trailing whitespace from lines, or else consolidates
certain whitespace sequences into a single whitespace character
(generally sequences of spaces and tabs becoming a single space).
Python source code simply cannot be passed through such channels
correctly - if they don't offer an escaping mechanism, or that
mechanism is not applied correctly, the code *will* be corrupted.
Explicitly delimited code, on the other hand, can be passed through
without semantic alteration (even if the details of the whitespace
change) and a pretty printer can fix it at the far end.

2. Attempting to transport it through a channel where leading
whitespace already has another meaning. This comes up primarily with
templating languages - your whitespace is generally part of the
template output, so it becomes problematic to break up your Python
code across multiple code snippets while keeping the suite groupings
clear. With explicit delimiters, though, you can just ignore the
template parts, and pretend the code snippets are all part of a single
string (the following is based on an example a colleague sent me [1],
with the closing delimiter adjusted to mirror the opening one as a
couple of folks suggested):

<% if danger_level > 3 {: %>
  <div class="alert"><% if danger_level == 5 {: %>EXTREME <% :}
%>DANGER ALERT!</div>

<% elif danger_level > 0 {: %>
  <div>Some chance of danger</div>

<% else {: %>
  <div>No danger</div>

<% :} %>

<% for a in ['cat', 'dog', 'rabbit'] {: %>
  <h2><%= a %></h2>
  <p><%= describe_animal(a) %></p>

<% :} %>

[1] https://gist.github.com/1455210

Is such code as beautiful and readable as normal Python code? No, of
course not. But it would serve a purpose in defining a *standard* way
to embed Python code in environments where the leading whitespace
causes problems.

In terms of how such a delimited syntax would relate to currently
legal Python code, the transformation is basically the reverse of the
Haskell one:

1. If you encounter a ":" to open a new suite, replace it and any
trailing whitespace with "{:".
2. All whitespace between statements in such a suite is replaced with
";" characters
3. Any trailing whitespace after the last statement in the suite is
replaced with ":};"

The rest of the changes (i.e. a suite's expression value being the
result of the last statement executed, anonymous function and class
definitions) then follow from the fact that you now have a suite
syntax that can potentially be embedded as an expression, so a range
of new possibilities open up.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From sven at marnach.net  Sun Dec 11 20:06:16 2011
From: sven at marnach.net (Sven Marnach)
Date: Sun, 11 Dec 2011 19:06:16 +0000
Subject: [Python-ideas] contextlib.maybe
In-Reply-To: <jc1h1n$nfe$1@dough.gmane.org>
References: <20111211050723.GA12197@air>
 <jc1h1n$nfe$1@dough.gmane.org>
Message-ID: <20111211190615.GB3760@pantoffel-wg.de>

Terry Reedy schrieb am So, 11. Dez 2011, um 01:02:30 -0500:
> Expecting contextlib to have such a specialized context manager that
> does exactly what you want is perhaps too much.

I don't think this is a very specialised need.  It would overcome a
fundamental limitation of context managers compared to some clean-up
constructs in other languages.  An example is Go's 'defer', which is
Go's replacement for try/finally blocks and context managers.  It
defers a function call to the time the current function returns, and
guarantees the deferred call will be executed no matter how control
leaves the current function -- just like a 'finally' clause.

The advantage of 'defer' over 'with' and try/finally blocks is that it
is not a compound statement, so you can "conditionally add a 'finally'
clause".  Returning to the example from the original post, let's see
how it would look in Go:

    func Fn(a_file *os.File) (err os.Error) {
        if a_file == nil {
            a_file, err = os.Open(a_default_location, os.O_RDONLY, 0)
            if err != nil { return }
            defer a_file.Close()
        }
        // do stuff
        return
    }

to achieve exactly what is desired.  The line 'defer a_file.Close()'
is only executed when we actually need to close the file.

Note that the disadvantage of 'defer' is also that it is not a
compound statement -- it is bound to the block defined by the current
function and in this regard less flexible than 'with' blocks.

We could add our own version of 'defer' to Python by offering a
context manager 'Deferrer' with a 'push()' method to push a clean-up
call-back on the Deferrer's stack.  This design would offer the
combined advantages of both approaches described above.  I, for one,
do think that this would make a very worthwhile addition to the
'contextlib' module.

We discussed such a context manager less than two weeks ago on this
list

    http://mail.python.org/pipermail/python-ideas/2011-October/012418.html

and Jan Kaliszewski even provided an implementation:

    http://mail.python.org/pipermail/python-ideas/2011-October/012463.html

What do you think?

Cheers,
    Sven


From richard.prosser at mail.com  Sun Dec 11 22:30:54 2011
From: richard.prosser at mail.com (Richard Prosser)
Date: Sun, 11 Dec 2011 21:30:54 +0000
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
Message-ID: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>

Ned,

I accept your comment about the tutorial - I wasn't sure about that. Thanks.

However I still feel that there are some aspects of the language which are
not in the true spirit of Python (i.e. 'intuitive').

The discussion about default mutable types is one of these. It seems to me
that the essential problem is that of assignment in general, which (I
believe) creates a reference on the LHS to the object on the RHS, rather
than having a copy operation to make the two objects completely separate.
That can be confusing in other contexts, not just with default parameters.

If I am to write a 'gotchas' FAQ or whatever then I would like to
understand the reasoning behind such design decisions but I can't find any
'deep' explanations at present - just several posts about people being
puzzled! A similar comment applies to the lack of type declarations.

So if you or anyone else can explain exactly why such odditties are
implemented I would be grateful.

Unfortunately it is almost certainly too late to propose fixes (if
appropriate) for such quirks in Python 3 but at least I should be able
provide arguments as to why things are done the way they are.


Richard


On 11 December 2011 01:21, <python-ideas-request at python.org> wrote:

> Message: 2
> Date: Sat, 10 Dec 2011 13:34:15 -0500
> From: Ned Batchelder <ned at nedbatchelder.com>
> To: Richard Prosser <richard.prosser at mail.com>
> Cc: python-ideas at python.org
> Subject: Re: [Python-ideas] Python Isn't Perfect: adding a 'gotchas'
>        section to the tutorial
> Message-ID: <4EE3A627.8010002 at nedbatchelder.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 12/10/2011 9:16 AM, Richard Prosser wrote:
> > Although I love Python there are some aspects of the language design
> > which are disappointing and which can even lead to problems in some
> cases.
> >
> > A classic example is a mutable default argument having the potential
> > to produce unexpected side-effects, as a consequence of the
> > non-intuitive scoping rules.
> >
> > Another awkward 'feature' is the requirement for a trailing comma in
> > singleton tuples, due I believe to the use of expression parentheses
> > rather than (say) the use of special brackets like chevrons.
> >
> > Something that I personally wish for is the ability to declare
> > variable types 'up front' but that facility is missing from Python.
> >
> One of these things is not like the others.  Mutable default arguments,
> and singleton tuples are surprises that make sense once you understand
> things on a deeper level.  It makes sense to call them out as a
> "gotcha": a common stumbling block for learners.  But "no type
> declarations" is not a wart, it's a fundamental feature of the language
> that is immediately apparent from the first lesson.
>
> --Ned.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111211/28c53824/attachment.html>

From mwm at mired.org  Sun Dec 11 22:40:56 2011
From: mwm at mired.org (Mike Meyer)
Date: Sun, 11 Dec 2011 13:40:56 -0800
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE40DA7.4020702@pearwood.info>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info>
Message-ID: <20111211134056.25d40c34@bhuda.mired.org>

On Sun, 11 Dec 2011 09:44:17 +0100
Masklinn <masklinn at masklinn.net> wrote:
> On 2011-12-11, at 01:16 , Mike Meyer wrote:
> > A minor improvement in just that one method, but it will show up in
> > every generate method of every class, making it significant in total.
> If users are not supposed to read the generated code, why not generate
> pyc files directly? That would avoid the issue would it not?

Users generally aren't expected to read generated code. It's pretty
ugly for systems that generate python now.

The reason you generate python instead of pyc is that backward
compatibility of python source is a priority for python development,
so with a little care your generated code will work across multiple
versions of python. The same is not true for byte code files (and if
that's changed with 3.x, I couldn't find anything saying so), so your
code generator is liable to depend on the version of the target python
interpreter.

On Sun, 11 Dec 2011 12:55:51 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> Mike Meyer wrote:
> 
> That's hardly an onerous requirement. As you show, the two code snippets are 
> almost identical, except that the indent version takes an explicit level 
> argument, while your braces version neglects to close suites at all. A fair 
> comparison should show the close-suite code, even if it is only a single line.

Correct. I thought it, but forgot to add it.

> > A minor improvement in just that one method, but it will show up in
> > every generate method of every class, making it significant in total.
> I wouldn't use the term "minor improvement" so much as "trivial difference".

I wouldn't refer to any change that involves adding a variable as
"trivial". Especially one that's part of the API. Part of the issue is
that I understated the problem - this change is ubiquitous, and large
enough that it adds up.

> But even if I grant that it is an improvement, how many generate methods do 
> you expect there to be?

Because it's in the API, not only does this affect every generate
method, but anything that might call them. You have to thread the
indent level to anything that might want to write code. It might be
better to make it a global (i.e. - instance or module variable) value
of some sort.

Basically, this "trivial difference" needs to be taken into
consideration when designing the system architecture.

You came close to the real reason for rejecting this here:

> When you defend a feature on the basis that "people will hardly ever use it, 
> except for trivial pieces of code that don't need it", that speaks volumes.

Two use cases are at or over the edge. Writing in something that lets
you embed python code? You arguably aren't writing python. Writing
code that generates python? *You* may not be writing python at all,
even if your code is.

That leaves the case of turning statements into expressions. While
suggestions/requests for similar things keep coming up, they aren't
all that frequent, and the tools we have for that have been acceptable
so far.

While Nick is to be congratulated for providing a solution to that
problem that's much better than it's predecessors, I don't think it's
worth the complexity it looks like it would require in the compiler.

   <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From masklinn at masklinn.net  Mon Dec 12 00:31:11 2011
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 12 Dec 2011 00:31:11 +0100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
Message-ID: <F41C1729-08BA-4660-8719-0F92C8272333@masklinn.net>

On 2011-12-11, at 22:30 , Richard Prosser wrote:
> However I still feel that there are some aspects of the language which are
> not in the true spirit of Python (i.e. 'intuitive').
I have to note that, as far as I know, "intuitive" is not quite part of the
"true spirit of Python". The Zen even has a stanza on the subject:

> Although that way may not be obvious at first unless you're Dutch.

> It seems to me
> that the essential problem is that of assignment in general, which (I
> believe) creates a reference on the LHS to the object on the RHS, rather
> than having a copy operation to make the two objects completely separate.
I don't see this as a problem, it's the normal semantics of languages using
reference-values for object types.

> A similar comment applies to the lack of type declarations.
> 
> So if you or anyone else can explain exactly why such odditties are
> implemented I would be grateful.
I don't understand, why do you consider dynamic typing to be "an
oddity"? There is nothing odd about it, and it's in fact older than
computer science itself.

The default arguments issue is an unfortunate interaction of Python's
core reference-value semantics and default arguments being implemented
as attributes to the function object, evaluated when said function
object is created (the feature itself is as old as Python, according to
the logs it was added for the 1.0.2 release back in 1994, the changelog
seems to peg it to April 14, so your only chance for an explanation of
why it was implemented with these semantics is hoping Guido has a very,
very good memory. Numerous post-hoc rationalization exist, but only him
may hold the true reason).

> Unfortunately it is almost certainly too late to propose fixes (if
> appropriate) for such quirks in Python 3
Most of these have been core semantic attributes of the language for
almost two decades now, so even if you had proposed these changes
during the Python 3 design cycle I think it's very unlikely they'd
have passed: they don't just change Python, they create a very
different language with a syntax similar to Python's.


From tjreedy at udel.edu  Mon Dec 12 01:13:23 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 11 Dec 2011 19:13:23 -0500
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
Message-ID: <jc3gv5$3hg$1@dough.gmane.org>

On 12/11/2011 4:30 PM, Richard Prosser wrote:

> However I still feel that there are some aspects of the language which
> are not in the true spirit of Python (i.e. 'intuitive').

While 'intuitive' may be part of the 'true spirit of Python', it is not 
included in the Zen of Python, perhaps because it it so slippery and 
person dependent. Python is more in tune with everyday life and 
therefore with naive intuition than some other languages.

> The discussion about default mutable types is one of these. It seems to
> me that the essential problem is that of assignment in general, which (I
> believe) creates a reference on the LHS to the object on the RHS, rather
> than having a copy operation to make the two objects completely
> separate. That can be confusing in other contexts, not just with default
> parameters.

When an organization reassign the role 'president' to a new person, they 
do not copy the person. Neither does Python. We use aliases (multiple 
ways to refer to the same entity) all the time in real life.

Python is quite consistent. Expressions evaluate to objects, either 
pre-existing or new. "Target = expression" binds the target to the 
object resulting from evaluating the expression. "f(expression)" binds 
the first parameter name of f to the expression object.

> If I am to write a 'gotchas' FAQ or whatever then I would like to
> understand the reasoning behind such design decisions but I can't find
> any 'deep' explanations at present - just several posts about people
> being puzzled! A similar comment applies to the lack of type declarations.

Both behaviors reflect the fact that Python is a named object language, 
rather than a named memory block language, and that in Python types are 
a property of objects rather than of names. This is similar to at least 
some uses of names in everyday life. For instance, the name Buddy could 
be bound by a particular person to a person, dog, other pet, story, 
boat, or even a rifle.

-- 
Terry Jan Reedy



From ned at nedbatchelder.com  Mon Dec 12 01:34:07 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 11 Dec 2011 19:34:07 -0500
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
Message-ID: <4EE54BFF.3050603@nedbatchelder.com>

Richard, I don't think I can provide you with a "why" for dynamic 
typing.  It's a choice Guido made early on, one that is central to the 
language, and one that I think pays off in the long run.  Values are 
objects, which are typed.  Names are untyped, and can refer to any 
value, regardless of what value they referred to in the past.  
Therefore, it doesn't make sense to talk about the type of a name, which 
is all that a type declaration could do.

This is very different than some other programming languages, but I 
don't know if it could be called unintuitive.  I'm not sure anything 
about programming could truly be called intuitive, I think the closest 
we can get is "familiar".  Certainly if you've worked with statically 
typed languages before, then dynamic typing is unfamiliar.

I worry that you are still placing dynamic typing into a category you 
call "gotchas" or "quirks", with the word "fixes" nearby.  Dynamic 
typing cannot be "fixed", it is central to the language.  I think it is 
great to write something to help those new to Python, but you should be 
sure that you fully understand Python before you undertake it.

--Ned.

On 12/11/2011 4:30 PM, Richard Prosser wrote:
> Ned,
>
> I accept your comment about the tutorial - I wasn't sure about that. 
> Thanks.
>
> However I still feel that there are some aspects of the language which 
> are not in the true spirit of Python (i.e. 'intuitive').
>
> The discussion about default mutable types is one of these. It seems 
> to me that the essential problem is that of assignment in general, 
> which (I believe) creates a reference on the LHS to the object on the 
> RHS, rather than having a copy operation to make the two objects 
> completely separate. That can be confusing in other contexts, not just 
> with default parameters.
>
> If I am to write a 'gotchas' FAQ or whatever then I would like to 
> understand the reasoning behind such design decisions but I can't find 
> any 'deep' explanations at present - just several posts about people 
> being puzzled! A similar comment applies to the lack of type declarations.
>
> So if you or anyone else can explain exactly why such odditties are 
> implemented I would be grateful.
>
> Unfortunately it is almost certainly too late to propose fixes (if 
> appropriate) for such quirks in Python 3 but at least I should be able 
> provide arguments as to why things are done the way they are.
>
>
> Richard
>
>
> On 11 December 2011 01:21, <python-ideas-request at python.org 
> <mailto:python-ideas-request at python.org>> wrote:
>
>     Message: 2
>     Date: Sat, 10 Dec 2011 13:34:15 -0500
>     From: Ned Batchelder <ned at nedbatchelder.com
>     <mailto:ned at nedbatchelder.com>>
>     To: Richard Prosser <richard.prosser at mail.com
>     <mailto:richard.prosser at mail.com>>
>     Cc: python-ideas at python.org <mailto:python-ideas at python.org>
>     Subject: Re: [Python-ideas] Python Isn't Perfect: adding a 'gotchas'
>            section to the tutorial
>     Message-ID: <4EE3A627.8010002 at nedbatchelder.com
>     <mailto:4EE3A627.8010002 at nedbatchelder.com>>
>     Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>     On 12/10/2011 9:16 AM, Richard Prosser wrote:
>     > Although I love Python there are some aspects of the language design
>     > which are disappointing and which can even lead to problems in
>     some cases.
>     >
>     > A classic example is a mutable default argument having the potential
>     > to produce unexpected side-effects, as a consequence of the
>     > non-intuitive scoping rules.
>     >
>     > Another awkward 'feature' is the requirement for a trailing comma in
>     > singleton tuples, due I believe to the use of expression parentheses
>     > rather than (say) the use of special brackets like chevrons.
>     >
>     > Something that I personally wish for is the ability to declare
>     > variable types 'up front' but that facility is missing from Python.
>     >
>     One of these things is not like the others.  Mutable default
>     arguments,
>     and singleton tuples are surprises that make sense once you understand
>     things on a deeper level.  It makes sense to call them out as a
>     "gotcha": a common stumbling block for learners.  But "no type
>     declarations" is not a wart, it's a fundamental feature of the
>     language
>     that is immediately apparent from the first lesson.
>
>     --Ned.
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111211/335de5cd/attachment.html>

From guido at python.org  Mon Dec 12 04:21:55 2011
From: guido at python.org (Guido van Rossum)
Date: Sun, 11 Dec 2011 19:21:55 -0800
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <20111211134056.25d40c34@bhuda.mired.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info>
	<20111211134056.25d40c34@bhuda.mired.org>
Message-ID: <CAP7+vJKCCnzANxVXWBpQVz4yF-Z4wQWOVnsZjYxAS6ZBe0pCKQ@mail.gmail.com>

On Sun, Dec 11, 2011 at 1:40 PM, Mike Meyer <mwm at mired.org> wrote:
> Users generally aren't expected to read generated code. It's pretty
> ugly for systems that generate python now.

If the point really is just generated code, we should just invent some
really ugly but easy-to-parse and 100%-guaranteed backwards compatible
set of brackets for just that purpose.

But I don't believe this is about generated code.

-- 
--Guido van Rossum (python.org/~guido)


From greg.ewing at canterbury.ac.nz  Mon Dec 12 06:21:26 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Dec 2011 18:21:26 +1300
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE40DA7.4020702@pearwood.info>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info>
Message-ID: <4EE58F56.10507@canterbury.ac.nz>

> Mike Meyer wrote:
> 
>> With significant whitespace, you have to keep explicit track of the
>> current nesting level in order to generate the appropriate
>> indent. With real delimiters, you can drop that.

Yes, but with an appropriate design you can arrange for that to
be done in only *one* place. There's no need for every method
that generates code to know about indentation levels.

For example, you can create a code-output-stream object that you
pass to your code-generating methods. You give it begin_block()
and end-block() methods that you call whenever you would
otherwise have written opening and closing delimiters. It
keeps track of the indentation level internally and emits the
appropriate amount of whitespace before each line.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Mon Dec 12 06:27:22 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Dec 2011 18:27:22 +1300
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <20111211134056.25d40c34@bhuda.mired.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info>
	<20111211134056.25d40c34@bhuda.mired.org>
Message-ID: <4EE590BA.6080305@canterbury.ac.nz>

Mike Meyer wrote:

> The reason you generate python instead of pyc is that backward
> compatibility of python source is a priority for python development,
> so with a little care your generated code will work across multiple
> versions of python.

Another approach would be to generate an AST (either an actual
Python AST or one of your own devising) and then traverse it
to generate indented source code. This is another way of
isolating knowledge of the indentation level to one part of
the code.

-- 
Greg


From mwm at mired.org  Mon Dec 12 06:45:29 2011
From: mwm at mired.org (Mike Meyer)
Date: Sun, 11 Dec 2011 21:45:29 -0800
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE58F56.10507@canterbury.ac.nz>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
Message-ID: <20111211214529.53d6dd34@bhuda.mired.org>

On Mon, 12 Dec 2011 18:21:26 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> > Mike Meyer wrote:
> >> With significant whitespace, you have to keep explicit track of the
> >> current nesting level in order to generate the appropriate
> >> indent. With real delimiters, you can drop that.
> Yes, but with an appropriate design you can arrange for that to
> be done in only *one* place.

Exactly. This change causes you to modify your architecture to deal
with it. In my book, changes that cause you to rethink the
architecture of your system aren't "trivial".

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From mwm at mired.org  Mon Dec 12 06:59:54 2011
From: mwm at mired.org (Mike Meyer)
Date: Sun, 11 Dec 2011 21:59:54 -0800
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <4EE54BFF.3050603@nedbatchelder.com>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
Message-ID: <20111211215954.79f33232@bhuda.mired.org>

On Sun, 11 Dec 2011 19:34:07 -0500
Ned Batchelder <ned at nedbatchelder.com> wrote:

> Richard, I don't think I can provide you with a "why" for dynamic 
> typing.

And this is the wrong place to ask. Dynamic typing and naming objects
dates back to the precursors to LISP in the mid 50s. You should be
asking the people who made that decision.

By the same token, have you asked anyone why C/Java/etc. have static
typing and name locations? It's an equally valid question.

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From bruce at leapyear.org  Mon Dec 12 07:09:06 2011
From: bruce at leapyear.org (Bruce Leban)
Date: Sun, 11 Dec 2011 22:09:06 -0800
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE58F56.10507@canterbury.ac.nz>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
Message-ID: <CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>

On Sun, Dec 11, 2011 at 9:21 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

>  Mike Meyer wrote:
>>
>>  With significant whitespace, you have to keep explicit track of the
>>> current nesting level in order to generate the appropriate
>>> indent. With real delimiters, you can drop that.
>>>
>>
> Yes, but with an appropriate design you can arrange for that to
> be done in only *one* place. There's no need for every method
> that generates code to know about indentation levels.
>

I've written this code (for HTML) and it's not only straightforward, it
makes the code easier to write and more robust. And you can make the
closing automatic with a context manager. Here's how you might write some
HTML:

    with html_tag('table'):
        with html_tag('tr'):
            for x in data:
                with html_tag('td'):
                     # do stuff with x

On the other hand, this might not help with template languages as Nick
points out:

On Sun, Dec 11, 2011 at 2:30 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> <% if danger_level > 3 {: %>
>  <div class="alert"><% if danger_level == 5 {: %>EXTREME <% :}
> %>DANGER ALERT!</div>
>
> <% elif danger_level > 0 {: %>
>  <div>Some chance of danger</div>
>
> <% else {: %>
>  <div>No danger</div>
>
> <% :} %>
>
> <% for a in ['cat', 'dog', 'rabbit'] {: %>
>  <h2><%= a %></h2>
>  <p><%= describe_animal(a) %></p>
>
> <% :} %>


 While I thought his example was compelling at first, after thinking it
through I realized that any existing template language that generates
Python has to be aware of indentation and it's going to have to modify the
indentation of the included Python code. So it would need to either read
the indentation or read the {: and :} tokens. So what if we think in terms
of modifying the template language instead of Python? In the code below,
I've added two extra tokens to the template language: %{> which increases
the indentation and <}% which decreases it. All other leading spaces would
be stripped. Here's that example rewritten:

<% if danger_level > 3: %{>
 <div class="alert"><% if danger_level == 5: %{>EXTREME <}%
%>DANGER ALERT!</div>

<% elif danger_level > 0: %{>
 <div>Some chance of danger</div>

<% else: %{>
 <div>No danger</div>

<}%>

<% for a in ['cat', 'dog', 'rabbit']: %{>
 <h2><%= a %></h2>
 <p><%= describe_animal(a) %></p>

<}%>

This isn't a perfect design. In particular %{> wouldn't be necessary if the
template language parses the embedded code, but it's sufficient to
illustrate the point.

--- Bruce
Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111211/0350525e/attachment.html>

From greg.ewing at canterbury.ac.nz  Mon Dec 12 07:40:00 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Dec 2011 19:40:00 +1300
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <20111211214529.53d6dd34@bhuda.mired.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<20111211214529.53d6dd34@bhuda.mired.org>
Message-ID: <4EE5A1C0.8090604@canterbury.ac.nz>

Mike Meyer wrote:

> Exactly. This change causes you to modify your architecture to deal
> with it.

It's not about "modifying" an architecture, it's about
considering the requirements and designing an appropriate
architecture from the beginning to accommodate them. If
you set out to generate Python code, write most of your
program and then say "Oh, bugger, what am I going to do
about indentation?", then you just haven't thought far
enough ahead.

Also, the particular architecture I suggested was only
an example. If you put your mind to it, you should be able
to come up with many ways of skinning this snake. If you
want to keep your generator style, you could yield special
"indent" and "dedent" tokens that the top level recognises
and does the appropriate thing with.

You might even use special strings such as "{:" and ":}"
embedded in the text to represent these. Then your code
generating methods wouldn't look all that much different
from how they would if you were generating the delimited
form of python that you are arguing for...

As a side note, I would argue that it's a good idea to
adopt an architecture that can manage indentation in any
case, whether you're generating indentation-sensitive code
or not.

I use this kind of architecture in Pyrex to generate C code,
even though indentation isn't required in C. I do it because
I want to generate properly-indented code anyway, for reasons
of readability. Even if the output is normally never seen,
it's a big help when debugging.

-- 
Greg


From ncoghlan at gmail.com  Mon Dec 12 07:42:09 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Dec 2011 16:42:09 +1000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
Message-ID: <CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>

On Mon, Dec 12, 2011 at 4:09 PM, Bruce Leban <bruce at leapyear.org> wrote:
> This isn't a perfect design. In particular %{> wouldn't be necessary if the
> template language parses the embedded code, but it's sufficient to
> illustrate the point.

Sure, but that's just perpetuating the status quo - you can fairly
easily create a Python *inspired* templating language right now by
building your suite delimeters into your template syntax (e.g. Django
and Jinja2 favour explicit suite closing snippets like "{% endfor
%}").

It isn't that Python *can't* be tunnelled through whitespace
insensitive environments - it's that there's no *standard* mechanism
for doing so, so everyone that has the need reinvents the wheel in
their own idiosyncratic way.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From greg.ewing at canterbury.ac.nz  Mon Dec 12 07:47:30 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Dec 2011 19:47:30 +1300
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
Message-ID: <4EE5A382.4030901@canterbury.ac.nz>

Nick Coghlan wrote:

> It isn't that Python *can't* be tunnelled through whitespace
> insensitive environments - it's that there's no *standard* mechanism
> for doing so, so everyone that has the need reinvents the wheel in
> their own idiosyncratic way.

So perhaps an appropriate solution would be a PEP setting out
such a standard, and maybe a library module implementing the
transformation to and from?

-- 
Greg


From ncoghlan at gmail.com  Mon Dec 12 07:56:03 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Dec 2011 16:56:03 +1000
Subject: [Python-ideas] An Alternate Suite Delineation Syntax For
 Python? (was Re: [Python-Dev] [PATCH] Adding braces to __future__)
In-Reply-To: <4EE5A382.4030901@canterbury.ac.nz>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<4EE5A382.4030901@canterbury.ac.nz>
Message-ID: <CADiSq7czk=mkJR271191uK61zzCH13ayuTpnJoparPZnHj1+9g@mail.gmail.com>

On Mon, Dec 12, 2011 at 4:47 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
>> It isn't that Python *can't* be tunnelled through whitespace
>> insensitive environments - it's that there's no *standard* mechanism
>> for doing so, so everyone that has the need reinvents the wheel in
>> their own idiosyncratic way.
>
>
> So perhaps an appropriate solution would be a PEP setting out
> such a standard, and maybe a library module implementing the
> transformation to and from?

Sure, that would be one way to go. It doesn't give you multi-line
lambda support or any of the other goodies, though.

Personally, I don't intend to take this idea any further than the
writeup I posted over on RTFD:
http://readthedocs.org/docs/ncoghlan_devs-python-notes/en/latest/pep_ideas/suite_expr.html

If someone else wants to pursue it, either as a Python reformatting
library or as an actual modification to the language grammar, more
power to 'em.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From greg.ewing at canterbury.ac.nz  Mon Dec 12 06:11:32 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Dec 2011 18:11:32 +1300
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
Message-ID: <4EE58D04.10403@canterbury.ac.nz>

Richard Prosser wrote:
> It seems to 
> me that the essential problem is that of assignment in general, which (I 
> believe) creates a reference on the LHS to the object on the RHS,
> 
> I would like to 
> understand the reasoning behind such design decisions but I can't find 
> any 'deep' explanations at present
> 
> So if you or anyone else can explain exactly why such odditties are 
> implemented I would be grateful.

Python's assignment semantics are only an "oddity" to people
whose prior exposure to programming languages is very limited.
To anyone familiar with almost any other dynamic language --
such as Lisp, Scheme, Smalltalk, or Javascript -- it's not
only unsurprising, it's the *obvious* thing to do. So I wouldn't
class it as a "gotcha" in the same sense as truly Python-specific
features like default argument evaluation and list comprehension
variable scope.

As for rationale, it comes down to something like this: Copying
large chunks of data is expensive, so it makes sense to do it
only when you really need to. And experience shows that most of
the time you *don't* need to copy things.

Furthermore, copying some kinds of things automatically and not
others (as some other languages such as VB and Java do) makes the
rules needlessly complicated and difficult to remember.

So Python does the simplest possible thing and doesn't copy
anything by default. If you want a copy, you need to do something
explicit to make it happen.

> Unfortunately it is almost certainly too late to propose fixes
 > (if appropriate) for such quirks in Python 3

Python's assignment behaviour is most definitely *not* something
that needs "fixing"!

-- 
Greg


From tjreedy at udel.edu  Mon Dec 12 09:55:57 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 12 Dec 2011 03:55:57 -0500
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <20111211215954.79f33232@bhuda.mired.org>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
	<20111211215954.79f33232@bhuda.mired.org>
Message-ID: <jc4fj0$2qs$1@dough.gmane.org>

On 12/12/2011 12:59 AM, Mike Meyer wrote:
> On Sun, 11 Dec 2011 19:34:07 -0500
> Ned Batchelder<ned at nedbatchelder.com>  wrote:
>
>> Richard, I don't think I can provide you with a "why" for dynamic
>> typing.
>
> And this is the wrong place to ask. Dynamic typing and naming objects
> dates back to the precursors to LISP in the mid 50s. You should be
> asking the people who made that decision.

List was designed for writing algorithms.

> By the same token, have you asked anyone why C/Java/etc. have static
> typing and name locations? It's an equally valid question.

C was designed for writing a computing machine operating system with 
mutable sequential memory slots numbered from 0 to 2**n - 1.

In this respect, Python is much more like List than C, even though its 
syntax is more like C.

-- 
Terry Jan Reedy



From masklinn at masklinn.net  Mon Dec 12 09:59:16 2011
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 12 Dec 2011 09:59:16 +0100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <4EE58D04.10403@canterbury.ac.nz>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE58D04.10403@canterbury.ac.nz>
Message-ID: <5E0C7E67-23DA-4F38-84ED-B3EB7137320F@masklinn.net>

On 2011-12-12, at 06:11 , Greg Ewing wrote:
> Richard Prosser wrote:
>> It seems to me that the essential problem is that of assignment in general, which (I believe) creates a reference on the LHS to the object on the RHS,
>> I would like to understand the reasoning behind such design decisions but I can't find any 'deep' explanations at present
>> So if you or anyone else can explain exactly why such odditties are implemented I would be grateful.
> 
> Python's assignment semantics are only an "oddity" to people
> whose prior exposure to programming languages is very limited.
> To anyone familiar with almost any other dynamic language --
> such as Lisp, Scheme, Smalltalk, or Javascript -- it's not
> only unsurprising
It's not even a question of "dynamic languages", Java and C#
reference types have exactly the same semantics (assignment copies
the value part of a type, which is the reference itself).

The only people I'd see confused by this are those with significant
C++ experience, where assignment of references does indeed go through
a copy of the object itself.

> As for rationale, it comes down to something like this: Copying
> large chunks of data is expensive, so it makes sense to do it
> only when you really need to. And experience shows that most of
> the time you *don't* need to copy things.
Of course technically that copy could very well be performed
"on write". This would significantly complexify the runtime as well.



From masklinn at masklinn.net  Mon Dec 12 10:00:59 2011
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 12 Dec 2011 10:00:59 +0100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <jc4fj0$2qs$1@dough.gmane.org>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
	<20111211215954.79f33232@bhuda.mired.org>
	<jc4fj0$2qs$1@dough.gmane.org>
Message-ID: <7D6FA2B6-E7CE-4025-9867-DD80C279AEA9@masklinn.net>

On 2011-12-12, at 09:55 , Terry Reedy wrote:
> On 12/12/2011 12:59 AM, Mike Meyer wrote:
>> On Sun, 11 Dec 2011 19:34:07 -0500
>> Ned Batchelder<ned at nedbatchelder.com>  wrote:
>>> Richard, I don't think I can provide you with a "why" for dynamic
>>> typing.
>> 
>> And this is the wrong place to ask. Dynamic typing and naming objects
>> dates back to the precursors to LISP in the mid 50s. You should be
>> asking the people who made that decision.
> List was designed for writing algorithms.
I think you meant "lisp" here ;)

>> By the same token, have you asked anyone why C/Java/etc. have static
>> typing and name locations? It's an equally valid question.
> C was designed for writing a computing machine operating system with mutable sequential memory slots numbered from 0 to 2**n - 1.
On the other hand, there are very few typed assemblies.

From phd at phdru.name  Sat Dec 10 15:42:30 2011
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 10 Dec 2011 18:42:30 +0400
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
Message-ID: <20111210144230.GA16801@iskra.aviel.ru>

On Sat, Dec 10, 2011 at 02:16:21PM +0000, Richard Prosser wrote:
> Although I love Python there are some aspects of the language design which
> are disappointing and which can even lead to problems in some cases.

   What really is disappointing is the number of people who criticize
Python without knowing it.

> Another awkward 'feature' is the requirement for a trailing comma in
> singleton tuples, due I believe to the use of expression parentheses rather
> than (say) the use of special brackets like chevrons.

   You do not understand the syntax. Parens do not construct tuples -
commas do. So for every tuple - even of length 1 - you must have a
comma. The only exception is an empty tuple (of length 0).

> Something that I personally wish for is the ability to declare variable
> types 'up front' but that facility is missing from Python.

   You can use annotations. See PEP 3107. Twas implemented in Python 3.0.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From masklinn at masklinn.net  Mon Dec 12 13:38:29 2011
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 12 Dec 2011 13:38:29 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <20111210144230.GA16801@iskra.aviel.ru>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
Message-ID: <5D96D1D2-ABE1-4C04-A0C2-6BC52AFE9D2D@masklinn.net>

On 2011-12-10, at 15:42 , Oleg Broytman wrote:
>   You can use annotations. See PEP 3107. Twas implemented in Python 3.0.
It's not used by the runtime though (I think IDEA/PyCharm uses it, but
I wouldn't bet on it), so it's mostly documentary at this point.


From ubershmekel at gmail.com  Mon Dec 12 13:41:26 2011
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Mon, 12 Dec 2011 14:41:26 +0200
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <5D96D1D2-ABE1-4C04-A0C2-6BC52AFE9D2D@masklinn.net>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<5D96D1D2-ABE1-4C04-A0C2-6BC52AFE9D2D@masklinn.net>
Message-ID: <CANSw7KzPJBAKrYHayWAX5HBPSR+v4m1fHKzx_nLuuVdvZXfBdw@mail.gmail.com>

On Mon, Dec 12, 2011 at 2:38 PM, Masklinn <masklinn at masklinn.net> wrote:

> On 2011-12-10, at 15:42 , Oleg Broytman wrote:
> >   You can use annotations. See PEP 3107. Twas implemented in Python 3.0.
> It's not used by the runtime though (I think IDEA/PyCharm uses it, but
> I wouldn't bet on it), so it's mostly documentary at this point.
>
>
<shameless>
pyopt uses annotations as types for parsing command line options if that
counts:

http://code.google.com/p/pyopt/

</shameless>

--Yuval
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111212/2169c72f/attachment.html>

From ned at nedbatchelder.com  Mon Dec 12 14:00:28 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Mon, 12 Dec 2011 08:00:28 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <20111210144230.GA16801@iskra.aviel.ru>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
Message-ID: <4EE5FAEC.1080907@nedbatchelder.com>

On 12/10/2011 9:42 AM, Oleg Broytman wrote:
> On Sat, Dec 10, 2011 at 02:16:21PM +0000, Richard Prosser wrote:
>> Although I love Python there are some aspects of the language design which
>> are disappointing and which can even lead to problems in some cases.
>     What really is disappointing is the number of people who criticize
> Python without knowing it.
>
>> Another awkward 'feature' is the requirement for a trailing comma in
>> singleton tuples, due I believe to the use of expression parentheses rather
>> than (say) the use of special brackets like chevrons.
>     You do not understand the syntax. Parens do not construct tuples -
> commas do. So for every tuple - even of length 1 - you must have a
> comma. The only exception is an empty tuple (of length 0).
I don't think we have to go as far as blaming the user.  Tuple syntax is 
a little tricky, people often trip up on (x,) as a single-item tuple.  
You and I understand why it is, and there isn't a better alternative, 
but that one-item syntax sticks out when compared to the others: (), (x, 
y), (x, y, z), etc.  This is a true "gotcha" as Richard originally 
expressed it.
> Oleg. 
--Ned.


From anacrolix at gmail.com  Mon Dec 12 14:06:07 2011
From: anacrolix at gmail.com (Matt Joiner)
Date: Tue, 13 Dec 2011 00:06:07 +1100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EE5FAEC.1080907@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
Message-ID: <CAB4yi1Pa1USmXw5ATNxNwtZcwP9toAYE1TGXEYuCL24sP0+1DQ@mail.gmail.com>

(a,) is the real gotcha, not ()

On Tue, Dec 13, 2011 at 12:00 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On 12/10/2011 9:42 AM, Oleg Broytman wrote:
>>
>> On Sat, Dec 10, 2011 at 02:16:21PM +0000, Richard Prosser wrote:
>>>
>>> Although I love Python there are some aspects of the language design
>>> which
>>> are disappointing and which can even lead to problems in some cases.
>>
>> ? ?What really is disappointing is the number of people who criticize
>> Python without knowing it.
>>
>>> Another awkward 'feature' is the requirement for a trailing comma in
>>> singleton tuples, due I believe to the use of expression parentheses
>>> rather
>>> than (say) the use of special brackets like chevrons.
>>
>> ? ?You do not understand the syntax. Parens do not construct tuples -
>> commas do. So for every tuple - even of length 1 - you must have a
>> comma. The only exception is an empty tuple (of length 0).
>
> I don't think we have to go as far as blaming the user. ?Tuple syntax is a
> little tricky, people often trip up on (x,) as a single-item tuple. ?You and
> I understand why it is, and there isn't a better alternative, but that
> one-item syntax sticks out when compared to the others: (), (x, y), (x, y,
> z), etc. ?This is a true "gotcha" as Richard originally expressed it.
>>
>> Oleg.
>
> --Ned.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
?_?


From solipsis at pitrou.net  Mon Dec 12 14:07:58 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 12 Dec 2011 14:07:58 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
Message-ID: <20111212140758.301e7431@pitrou.net>

On Mon, 12 Dec 2011 08:00:28 -0500
Ned Batchelder <ned at nedbatchelder.com>
wrote:
> On 12/10/2011 9:42 AM, Oleg Broytman wrote:
> > On Sat, Dec 10, 2011 at 02:16:21PM +0000, Richard Prosser wrote:
> >> Although I love Python there are some aspects of the language design which
> >> are disappointing and which can even lead to problems in some cases.
> >     What really is disappointing is the number of people who criticize
> > Python without knowing it.
> >
> >> Another awkward 'feature' is the requirement for a trailing comma in
> >> singleton tuples, due I believe to the use of expression parentheses rather
> >> than (say) the use of special brackets like chevrons.
> >     You do not understand the syntax. Parens do not construct tuples -
> > commas do. So for every tuple - even of length 1 - you must have a
> > comma. The only exception is an empty tuple (of length 0).
> I don't think we have to go as far as blaming the user.  Tuple syntax is 
> a little tricky, people often trip up on (x,) as a single-item tuple.  
> You and I understand why it is, and there isn't a better alternative, 
> but that one-item syntax sticks out when compared to the others: (), (x, 
> y), (x, y, z), etc.  This is a true "gotcha" as Richard originally 
> expressed it.

I think it would be more of a gotcha if parentheses were enough to
create a tuple, though. Parentheses are useful to group operations,
either for stylistic / syntactic support (think multi-line statements),
or to work around operator precedence. Creating a tuple by mistake
because you put some parentheses where not necessary would be really
annoying.

Regards

Antoine.




From ned at nedbatchelder.com  Mon Dec 12 14:15:45 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Mon, 12 Dec 2011 08:15:45 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <20111212140758.301e7431@pitrou.net>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
Message-ID: <4EE5FE81.3060304@nedbatchelder.com>

On 12/12/2011 8:07 AM, Antoine Pitrou wrote:
> On Mon, 12 Dec 2011 08:00:28 -0500
> Ned Batchelder<ned at nedbatchelder.com>
> wrote:
>> On 12/10/2011 9:42 AM, Oleg Broytman wrote:
>>> On Sat, Dec 10, 2011 at 02:16:21PM +0000, Richard Prosser wrote:
>>>> Although I love Python there are some aspects of the language design which
>>>> are disappointing and which can even lead to problems in some cases.
>>>      What really is disappointing is the number of people who criticize
>>> Python without knowing it.
>>>
>>>> Another awkward 'feature' is the requirement for a trailing comma in
>>>> singleton tuples, due I believe to the use of expression parentheses rather
>>>> than (say) the use of special brackets like chevrons.
>>>      You do not understand the syntax. Parens do not construct tuples -
>>> commas do. So for every tuple - even of length 1 - you must have a
>>> comma. The only exception is an empty tuple (of length 0).
>> I don't think we have to go as far as blaming the user.  Tuple syntax is
>> a little tricky, people often trip up on (x,) as a single-item tuple.
>> You and I understand why it is, and there isn't a better alternative,
>> but that one-item syntax sticks out when compared to the others: (), (x,
>> y), (x, y, z), etc.  This is a true "gotcha" as Richard originally
>> expressed it.
> I think it would be more of a gotcha if parentheses were enough to
> create a tuple, though. Parentheses are useful to group operations,
> either for stylistic / syntactic support (think multi-line statements),
> or to work around operator precedence. Creating a tuple by mistake
> because you put some parentheses where not necessary would be really
> annoying.
Believe me, I understand the issues.  It is true, though that the 
single-element tuple syntax is often a surprise to people, and often 
well into their Python learning experience.  We often repeat, "it isn't 
parens that make a tuple, but a comma."  Then why when displaying a 
tuple does Python insist on using parens around it?

 >>> 1, 2, 3
     (1, 2, 3)

I'm not saying it shouldn't, it's a rhetorical question.  The repr of a 
tuple always includes parens, even though "parens don't make a tuple."  
It's the best of all the options, but let's face it: it's confusing.

--Ned.
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From solipsis at pitrou.net  Mon Dec 12 14:19:19 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 12 Dec 2011 14:19:19 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EE5FE81.3060304@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
Message-ID: <1323695959.21067.21.camel@localhost.localdomain>

Le lundi 12 d?cembre 2011 ? 08:15 -0500, Ned Batchelder a ?crit :
> Believe me, I understand the issues.  It is true, though that the 
> single-element tuple syntax is often a surprise to people, and often 
> well into their Python learning experience.  We often repeat, "it isn't 
> parens that make a tuple, but a comma."  Then why when displaying a 
> tuple does Python insist on using parens around it?
> 
>  >>> 1, 2, 3
>      (1, 2, 3)

I would say:
- because it's easier to read (subjectively so, I guess)
- because it's easier to copy/paste into an expression without running
into precedence problems

Regards

Antoine.




From masklinn at masklinn.net  Mon Dec 12 14:25:42 2011
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 12 Dec 2011 14:25:42 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EE5FE81.3060304@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
Message-ID: <062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>

On 2011-12-12, at 14:15 , Ned Batchelder wrote:
> 
> Believe me, I understand the issues.  It is true, though that the single-element tuple syntax is often a surprise to people, and often well into their Python learning experience.  We often repeat, "it isn't parens that make a tuple, but a comma."  Then why when displaying a tuple does Python insist on using parens around it?
> 
> >>> 1, 2, 3
>    (1, 2, 3)
> 
> I'm not saying it shouldn't, it's a rhetorical question.  The repr of a tuple always includes parens, even though "parens don't make a tuple."  It's the best of all the options, but let's face it: it's confusing.
An alternative would be to just drop the literal unary tuple. I would be kind-of sad as I'd lose the ability to unpack singleton iterables, but it would "fix" the issue.

FWIW, Haskell does not have a literal singleton (the standard defines "unit" `()` and 2-tuple through 15-tuple)

From ned at nedbatchelder.com  Mon Dec 12 14:26:16 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Mon, 12 Dec 2011 08:26:16 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <1323695959.21067.21.camel@localhost.localdomain>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<1323695959.21067.21.camel@localhost.localdomain>
Message-ID: <4EE600F8.8010801@nedbatchelder.com>

On 12/12/2011 8:19 AM, Antoine Pitrou wrote:
> Le lundi 12 d?cembre 2011 ? 08:15 -0500, Ned Batchelder a ?crit :
>> Believe me, I understand the issues.  It is true, though that the
>> single-element tuple syntax is often a surprise to people, and often
>> well into their Python learning experience.  We often repeat, "it isn't
>> parens that make a tuple, but a comma."  Then why when displaying a
>> tuple does Python insist on using parens around it?
>>
>>   >>>  1, 2, 3
>>       (1, 2, 3)
> I would say:
> - because it's easier to read (subjectively so, I guess)
> - because it's easier to copy/paste into an expression without running
> into precedence problems
Yes, those are good reasons.  And you can see why Python's insistence on 
showing tuples with parens contributes to the gotcha that the parens are 
a red herring, and it's the commas that are important.  Also, who hasn't 
said this to a beginner:  "lists and tuples are very similar, lists use 
square brackets, tuples use parens"?  Somehow, in a list, the commas 
don't make a tuple...  It's complicated.

I welcome Richard's help in explaining this issue to beginners.

--Ned.
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From anacrolix at gmail.com  Mon Dec 12 14:41:41 2011
From: anacrolix at gmail.com (Matt Joiner)
Date: Tue, 13 Dec 2011 00:41:41 +1100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EE600F8.8010801@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<1323695959.21067.21.camel@localhost.localdomain>
	<4EE600F8.8010801@nedbatchelder.com>
Message-ID: <CAB4yi1O2Z-PODp19KF1Ar-xGRALZ9Zfvoa=54txOHgt4aDj4Gg@mail.gmail.com>

Here's some crazy ideas:
 * Don't allow iteration on tuples. for i in tuple
 * Make the singleton tuple the default object representation: (), a,
(a, b), (a, b, c), ...
 * Steal STM, Channels, and sparks from Haskell
 * Profit.

On Tue, Dec 13, 2011 at 12:26 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On 12/12/2011 8:19 AM, Antoine Pitrou wrote:
>>
>> Le lundi 12 d?cembre 2011 ? 08:15 -0500, Ned Batchelder a ?crit :
>>>
>>> Believe me, I understand the issues. ?It is true, though that the
>>> single-element tuple syntax is often a surprise to people, and often
>>> well into their Python learning experience. ?We often repeat, "it isn't
>>> parens that make a tuple, but a comma." ?Then why when displaying a
>>> tuple does Python insist on using parens around it?
>>>
>>> ?>>> ?1, 2, 3
>>> ? ? ?(1, 2, 3)
>>
>> I would say:
>> - because it's easier to read (subjectively so, I guess)
>> - because it's easier to copy/paste into an expression without running
>> into precedence problems
>
> Yes, those are good reasons. ?And you can see why Python's insistence on
> showing tuples with parens contributes to the gotcha that the parens are a
> red herring, and it's the commas that are important. ?Also, who hasn't said
> this to a beginner: ?"lists and tuples are very similar, lists use square
> brackets, tuples use parens"? ?Somehow, in a list, the commas don't make a
> tuple... ?It's complicated.
>
> I welcome Richard's help in explaining this issue to beginners.
>
> --Ned.
>
>> Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
?_?


From anacrolix at gmail.com  Mon Dec 12 14:44:46 2011
From: anacrolix at gmail.com (Matt Joiner)
Date: Tue, 13 Dec 2011 00:44:46 +1100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <5E0C7E67-23DA-4F38-84ED-B3EB7137320F@masklinn.net>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE58D04.10403@canterbury.ac.nz>
	<5E0C7E67-23DA-4F38-84ED-B3EB7137320F@masklinn.net>
Message-ID: <CAB4yi1P3cKh7M65sNyO_ZczCfgbzn5_H77CEc07ALSTJdhwQWg@mail.gmail.com>

> The only people I'd see confused by this are those with significant> C++ experience, where assignment of references does indeed go through> a copy of the object itself.

?_?

On Mon, Dec 12, 2011 at 7:59 PM, Masklinn <masklinn at masklinn.net> wrote:
> On 2011-12-12, at 06:11 , Greg Ewing wrote:
>> Richard Prosser wrote:
>>> It seems to me that the essential problem is that of assignment in general, which (I believe) creates a reference on the LHS to the object on the RHS,
>>> I would like to understand the reasoning behind such design decisions but I can't find any 'deep' explanations at present
>>> So if you or anyone else can explain exactly why such odditties are implemented I would be grateful.
>>
>> Python's assignment semantics are only an "oddity" to people
>> whose prior exposure to programming languages is very limited.
>> To anyone familiar with almost any other dynamic language --
>> such as Lisp, Scheme, Smalltalk, or Javascript -- it's not
>> only unsurprising
> It's not even a question of "dynamic languages", Java and C#
> reference types have exactly the same semantics (assignment copies
> the value part of a type, which is the reference itself).
>
> The only people I'd see confused by this are those with significant
> C++ experience, where assignment of references does indeed go through
> a copy of the object itself.
>
>> As for rationale, it comes down to something like this: Copying
>> large chunks of data is expensive, so it makes sense to do it
>> only when you really need to. And experience shows that most of
>> the time you *don't* need to copy things.
> Of course technically that copy could very well be performed
> "on write". This would significantly complexify the runtime as well.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
?_?


From tjreedy at udel.edu  Mon Dec 12 17:27:08 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 12 Dec 2011 11:27:08 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EE5FE81.3060304@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
Message-ID: <jc5a0u$3uv$1@dough.gmane.org>

On 12/12/2011 8:15 AM, Ned Batchelder wrote:

> parens that make a tuple, but a comma." Then why when displaying a tuple
> does Python insist on using parens around it?
>
>  >>> 1, 2, 3
> (1, 2, 3)
>
> I'm not saying it shouldn't, it's a rhetorical question.

Actually, I would agree that it would be better to not mislead by not 
printing the parens unless necessary. (If done from the beginning.)
For something like
 >>> t = 1,2
 >>> l = list(t)
 >>> t,l
((1, 2), [1, 2])

the outer parens are also unnecessary (while the inner ones are needed) 
and make the result less easy to read. But the easiest way to not print 
them would probably be to test the output string after it is constructed 
for beginning with '(' and ending with ')' and strip them off if so.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Mon Dec 12 17:40:03 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 12 Dec 2011 11:40:03 -0500
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <7D6FA2B6-E7CE-4025-9867-DD80C279AEA9@masklinn.net>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
	<20111211215954.79f33232@bhuda.mired.org>
	<jc4fj0$2qs$1@dough.gmane.org>
	<7D6FA2B6-E7CE-4025-9867-DD80C279AEA9@masklinn.net>
Message-ID: <jc5ap5$9ng$1@dough.gmane.org>

On 12/12/2011 4:00 AM, Masklinn wrote:
> On 2011-12-12, at 09:55 , Terry Reedy wrote:
>> On 12/12/2011 12:59 AM, Mike Meyer wrote:
>>> On Sun, 11 Dec 2011 19:34:07 -0500
>>> Ned Batchelder<ned at nedbatchelder.com>   wrote:
>>>> Richard, I don't think I can provide you with a "why" for dynamic
>>>> typing.
>>>
>>> And this is the wrong place to ask. Dynamic typing and naming objects
>>> dates back to the precursors to LISP in the mid 50s. You should be
>>> asking the people who made that decision.
>> List was designed for writing algorithms.
> I think you meant "lisp" here ;)

Definitely.

>>> By the same token, have you asked anyone why C/Java/etc. have static
>>> typing and name locations? It's an equally valid question.
>> C was designed for writing a computing machine operating system with mutable sequential memory slots numbered from 0 to 2**n - 1.
> On the other hand, there are very few typed assemblies.

The question of typing is somewhat orthogonal to that of naming value 
objects versus storage locations. There are strong stactic typed named 
value languages like ML. They even require different operators for int 
and float arithmetic, just like assembler.

Assemblers also type data versus address registers and may have signed 
versus unsigned int operations. But most typing has to be done by the 
programmer, just as C requires the programmer to do garbage collection.

-- 
Terry Jan Reedy



From masklinn at masklinn.net  Mon Dec 12 17:49:10 2011
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 12 Dec 2011 17:49:10 +0100
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <jc5ap5$9ng$1@dough.gmane.org>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
	<20111211215954.79f33232@bhuda.mired.org>
	<jc4fj0$2qs$1@dough.gmane.org>
	<7D6FA2B6-E7CE-4025-9867-DD80C279AEA9@masklinn.net>
	<jc5ap5$9ng$1@dough.gmane.org>
Message-ID: <7FBD2B97-5965-4D47-9256-DE22A5BEA7AC@masklinn.net>

On 2011-12-12, at 17:40 , Terry Reedy wrote:
>>>> By the same token, have you asked anyone why C/Java/etc. have static
>>>> typing and name locations? It's an equally valid question.
>>> C was designed for writing a computing machine operating system with mutable sequential memory slots numbered from 0 to 2**n - 1.
>> On the other hand, there are very few typed assemblies.
> The question of typing is somewhat orthogonal to that of naming value objects versus storage locations. There are strong stactic typed named value languages like ML.
I know, hence my quoting the part about typing, not the part about variable semantics.

> They even require different operators for int and float arithmetic, just like assembler.
That's got nothing to do with the typing or binding disciplines though, it's an issue of ML's type system. Haskell does not have that issue and is arguably more statically typed than MLs.

> Assemblers also type data versus address registers and may have signed versus unsigned int operations. But most typing has to be done by the programmer
Which is the point I was trying to make, a point which kind-of goes against the justification used for C being statically typed, as assemblies are also designed for driving systems with mutable sequential memory slots.

From tjreedy at udel.edu  Mon Dec 12 18:21:49 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 12 Dec 2011 12:21:49 -0500
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
	to the tutorial
In-Reply-To: <7FBD2B97-5965-4D47-9256-DE22A5BEA7AC@masklinn.net>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
	<20111211215954.79f33232@bhuda.mired.org>
	<jc4fj0$2qs$1@dough.gmane.org>
	<7D6FA2B6-E7CE-4025-9867-DD80C279AEA9@masklinn.net>
	<jc5ap5$9ng$1@dough.gmane.org>
	<7FBD2B97-5965-4D47-9256-DE22A5BEA7AC@masklinn.net>
Message-ID: <jc5d7f$6h3$1@dough.gmane.org>

On 12/12/2011 11:49 AM, Masklinn wrote:

> Which is the point I was trying to make, a point which kind-of goes
> against the justification used for C being statically typed, as
> assemblies are also designed for driving systems with mutable
> sequential memory slots.

C was designed to remove some, but just some, of the burden of managing 
blocks of memory. One of the unfortunate ironies of C is that C 
programmers tend to forget that they *are* working with contiguous 
mutable blocks and not with isolated Python-like objects. Hence 
buffer-run exploits by malware writers who *do* remember and study the 
exact order of memory blocks in particular binaries.

-- 
Terry Jan Reedy



From mwm at mired.org  Mon Dec 12 18:37:22 2011
From: mwm at mired.org (Mike Meyer)
Date: Mon, 12 Dec 2011 09:37:22 -0800
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <jc5ap5$9ng$1@dough.gmane.org>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE54BFF.3050603@nedbatchelder.com>
	<20111211215954.79f33232@bhuda.mired.org>
	<jc4fj0$2qs$1@dough.gmane.org>
	<7D6FA2B6-E7CE-4025-9867-DD80C279AEA9@masklinn.net>
	<jc5ap5$9ng$1@dough.gmane.org>
Message-ID: <20111212093722.36264852@mikmeyer-vm-fedora>

On Mon, 12 Dec 2011 11:40:03 -0500
Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/12/2011 4:00 AM, Masklinn wrote:
> > On 2011-12-12, at 09:55 , Terry Reedy wrote:
> >> On 12/12/2011 12:59 AM, Mike Meyer wrote:
> >>> By the same token, have you asked anyone why C/Java/etc. have
> >>> static typing and name locations? It's an equally valid question.
> >> C was designed for writing a computing machine operating system
> >> with mutable sequential memory slots numbered from 0 to 2**n - 1.
> > On the other hand, there are very few typed assemblies.
> The question of typing is somewhat orthogonal to that of naming value 
> objects versus storage locations. There are strong stactic typed
> named value languages like ML. They even require different operators
> for int and float arithmetic, just like assembler.

Yup. You can also find dynamically typed named locations languages,
like BCPL (though "untyped" might be more descriptive). It also has
different operators for int and float arithmetic, because the
locations don't have type information, so it has to come from the
operator.

    <mike


From Nikolaus at rath.org  Mon Dec 12 19:58:40 2011
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 12 Dec 2011 13:58:40 -0500
Subject: [Python-ideas] Avoiding nested for try..finally: atexit for
	functions?
In-Reply-To: <20111022002258.GB7138@chopin.edu.pl>
References: <87pqhtafrz.fsf@vostro.rath.org>
	<20111019040214.GA5524@flay.puzzling.org>
	<87lishccj3.fsf@inspiron.ap.columbia.edu>
	<CACac1F9beRi0+htqzLJVJNQaqRcR=ZtUCj3Rxh5deJKaXTqn-g@mail.gmail.com>
	<87fwipca9k.fsf@inspiron.ap.columbia.edu>
	<20111020001922.GA2191@chopin.edu.pl>
	<87mxcw1k2p.fsf@vostro.rath.org>
	<20111022002258.GB7138@chopin.edu.pl>
Message-ID: <4EE64EE0.10103@rath.org>

On 10/21/2011 08:22 PM, Jan Kaliszewski wrote:
> An improved (and at the same time simplified) implementation (being also
> a recipe for Python 2.x, though this list is about ideas for Py3.x):
> 
>     class CleanupManager(object):
> 
>         def __init__(self, initial_callbacks=()):
>             self.cleanup_callbacks = list(initial_callbacks)
> 
>         def register(self, callback, *args, **kwargs):
>             self.cleanup_callbacks.append((callback, args, kwargs))
> 
>         def __enter__(self):
>             return self
> 
>         def __exit__(self, exc_type, exc, tb):
>             self._next_callback()
> 
>         def _next_callback(self):
>             if self.cleanup_callbacks:
>                 callback, args, kwargs = self.cleanup_callbacks.pop()
>                 try:
>                     callback(*args, **kwargs)
>                 finally:
>                     # all cleanup callbacks to be used
>                     # Py3.x: all errors to be reported
>                     self._next_callback()
> 
> I hope it implements well what you explained... I'm not sure if it is
> worth to be added to the standard library (in the case of your primary
> example I'd rather prefer that try-finally nested structure) -- though
> in some cases it may become really useful:
> 
>     with CleanupMagager() as cm:
>         ...
>         cm.register(foo)
>         ...
>         if cond:
>             cm.register(bar)
>         else:
>             cm.register(spam)
>             for x in y:
>                 cm.register(baz, x)


I've opened a wishlist bug for this at http://bugs.python.org/issue13585.

Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From greg.ewing at canterbury.ac.nz  Mon Dec 12 22:13:22 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Dec 2011 10:13:22 +1300
Subject: [Python-ideas] Python Isn't Perfect: adding a 'gotchas' section
 to the tutorial
In-Reply-To: <5E0C7E67-23DA-4F38-84ED-B3EB7137320F@masklinn.net>
References: <CAJvrtnvP5u4nHe161xzLAAzB6Rz4iFshd0bZYU25EmgN-nMHSQ@mail.gmail.com>
	<4EE58D04.10403@canterbury.ac.nz>
	<5E0C7E67-23DA-4F38-84ED-B3EB7137320F@masklinn.net>
Message-ID: <4EE66E72.5080803@canterbury.ac.nz>

Masklinn wrote:

> Of course technically that copy could very well be performed
> "on write". This would significantly complexify the runtime as well.

It could also lead to a lot of gratuitous inefficiency
in programs, as people got into the habit of modifying
anything passed to them in the knowledge that it
wouldn't do any harm, but without considering the cost
of all the implicit copying that it caused.

-- 
Greg



From greg.ewing at canterbury.ac.nz  Mon Dec 12 22:21:11 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Dec 2011 10:21:11 +1300
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
Message-ID: <4EE67047.8050209@canterbury.ac.nz>

Masklinn wrote:

> FWIW, Haskell does not have a literal singleton (the standard defines 
> "unit" `()` and 2-tuple through 15-tuple)

That's because, due to its static typing, there is no
reason you would ever need to use a 1-tuple rather than
a bare value. We're not that lucky in Python, though.

-- 
Greg


From ron3200 at gmail.com  Mon Dec 12 23:07:13 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Mon, 12 Dec 2011 16:07:13 -0600
Subject: [Python-ideas] Transportable indent level markers.   /// ;;; \\\
In-Reply-To: <CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
Message-ID: <1323727633.29879.181.camel@Gutsy>


While typing this reply to the "Alternate Suite Delineation Syntax For
Python?" thread.  I realized that having things  all on a single line
isn't important or needed!!!  Yes, it's not needed for templates, or
code generation.  So this is an alternate to the alternate.

What is needed is a uniform way to represent indenting that is easy to
transport to (ie paste into) code with different indent levels.


INDENT LEVEL MARKERS:
----------------------
Offer a new way to spell <indent+1>, <indent+0>, and <indent-1>.  

To make that work, we need to use unique character sequences that aren't
already used.  After some thought, I came up with these indent-markers.

    ///   # indent level +1
    ;;;   # indent level +0
    \\\   # indent level -1

These are unique enough to be easy to see, and the tripled versions of
them aren't really that bad as we already have triple quotes, and they
would be less than the white space they are replacing in most cases.

So this isn't brackets, or braces. It would be just an alternate
spelling in already valid python. It should not effect the parser,
grammar, or interpreter in any way if it is done in pythons
tokenizer.  :-)


Reusing some previous examples.

#-----------------------

>    def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return
>  s}}}

#Standard python...

def acc(n):
    s=n
    def add(i):
        nonlocal s
        s += i
        return s
    return add


#Single line version...

  def acc(n): /// s=n ;;; def add(i): /// nonlocal s \ 
  ;;; s+=i ;;; return s \\\ return add

(The '\' character should still work.)


#Transportable indent level version...

This could also be valid... and could be transportable to any other
indent level without having to edit it.

(So would the above single line version by adding a ';;;' to the
beginning.)

;;; def acc(n):     # indent +0
/// s = n           # indent +1
;;; def add(i):     # indent +0
/// nonlocal s      # indent +1
;;; s += i          # indent +0
;;; return s        # indent +0
\\\ return add      # indent -1

It has the nice properties of being uniform, and being able to be
inserted easily into any existing code block without having to adjust
the indent levels.

I believe that these properties are more important than being able to
put things on a single line and could be a very useful in code
generating programs. ;-)


It may be easy to implement.

This could probably all be done in the tokinze.c get_tok() function,
where it detects and converts white space to indention levels.  Nothing
else in python would need to be changed.  The tokenizer already detects
line continuations and different line endings, so this wouldn't be that
out of place.

Cheers,
   Ron




From phd at phdru.name  Mon Dec 12 23:36:38 2011
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 13 Dec 2011 02:36:38 +0400
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
 \\\
In-Reply-To: <1323727633.29879.181.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
Message-ID: <20111212223638.GA32133@iskra.aviel.ru>

On Mon, Dec 12, 2011 at 04:07:13PM -0600, Ron Adam wrote:
>     ///   # indent level +1
>     ;;;   # indent level +0
>     \\\   # indent level -1
[skip]
>   def acc(n): /// s=n ;;; def add(i): /// nonlocal s \ 
>   ;;; s+=i ;;; return s \\\ return add

   from __future__ import brainfuck (I mean well-known esoteric
programming language)?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From massimo.dipierro at gmail.com  Mon Dec 12 23:40:31 2011
From: massimo.dipierro at gmail.com (Massimo Di Pierro)
Date: Mon, 12 Dec 2011 16:40:31 -0600
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
	\\\
In-Reply-To: <1323727633.29879.181.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
Message-ID: <2B666E0F-8B3D-4E66-9E3A-D5025393D432@gmail.com>

I do not like it and I am not convinced this belongs to Python 

Yet you may like this other solution:

exec(reindent("def f(a):; x=0; for i in range(a):; x+=i; pass; return x; print f(5)"))

here reindent is a modification of the web2py reindent function (used in templates).

- ; is converted to \n
- loops are closed with pass
- ifs are also closed with pass
- def can be closed with pass or return
- class must also end with pass

In web2py-ese templates we would not use it like this. we would write instead:

{{def f(a):}}{{x=0}}{{for i in range(a):}}{{x+=i}}{{pass}}{{return x}}{{=f(5)}}

or break it into

{{
def f(a):
   x=0
   for i in range(a):
      x+=i
      pass
   return x
= f(5)
}} (indentation is irrelevant even within the {{code}})


-------------- next part --------------
A non-text attachment was scrubbed...
Name: reindent.py
Type: text/x-python-script
Size: 4046 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111212/96a2c8d0/attachment.bin>
-------------- next part --------------




On Dec 12, 2011, at 4:07 PM, Ron Adam wrote:

> 
> While typing this reply to the "Alternate Suite Delineation Syntax For
> Python?" thread.  I realized that having things  all on a single line
> isn't important or needed!!!  Yes, it's not needed for templates, or
> code generation.  So this is an alternate to the alternate.
> 
> What is needed is a uniform way to represent indenting that is easy to
> transport to (ie paste into) code with different indent levels.
> 
> 
> INDENT LEVEL MARKERS:
> ----------------------
> Offer a new way to spell <indent+1>, <indent+0>, and <indent-1>.  
> 
> To make that work, we need to use unique character sequences that aren't
> already used.  After some thought, I came up with these indent-markers.
> 
>    ///   # indent level +1
>    ;;;   # indent level +0
>    \\\   # indent level -1
> 
> These are unique enough to be easy to see, and the tripled versions of
> them aren't really that bad as we already have triple quotes, and they
> would be less than the white space they are replacing in most cases.
> 
> So this isn't brackets, or braces. It would be just an alternate
> spelling in already valid python. It should not effect the parser,
> grammar, or interpreter in any way if it is done in pythons
> tokenizer.  :-)
> 
> 
> Reusing some previous examples.
> 
> #-----------------------
> 
>>   def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return
>> s}}}
> 
> #Standard python...
> 
> def acc(n):
>    s=n
>    def add(i):
>        nonlocal s
>        s += i
>        return s
>    return add
> 
> 
> #Single line version...
> 
>  def acc(n): /// s=n ;;; def add(i): /// nonlocal s \ 
>  ;;; s+=i ;;; return s \\\ return add
> 
> (The '\' character should still work.)
> 
> 
> #Transportable indent level version...
> 
> This could also be valid... and could be transportable to any other
> indent level without having to edit it.
> 
> (So would the above single line version by adding a ';;;' to the
> beginning.)
> 
> ;;; def acc(n):     # indent +0
> /// s = n           # indent +1
> ;;; def add(i):     # indent +0
> /// nonlocal s      # indent +1
> ;;; s += i          # indent +0
> ;;; return s        # indent +0
> \\\ return add      # indent -1
> 
> It has the nice properties of being uniform, and being able to be
> inserted easily into any existing code block without having to adjust
> the indent levels.
> 
> I believe that these properties are more important than being able to
> put things on a single line and could be a very useful in code
> generating programs. ;-)
> 
> 
> It may be easy to implement.
> 
> This could probably all be done in the tokinze.c get_tok() function,
> where it detects and converts white space to indention levels.  Nothing
> else in python would need to be changed.  The tokenizer already detects
> line continuations and different line endings, so this wouldn't be that
> out of place.
> 
> Cheers,
>   Ron
> 
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From anacrolix at gmail.com  Mon Dec 12 23:45:56 2011
From: anacrolix at gmail.com (Matt Joiner)
Date: Tue, 13 Dec 2011 09:45:56 +1100
Subject: [Python-ideas] Transportable indent level markers. /// ; ; ; \\\
In-Reply-To: <2B666E0F-8B3D-4E66-9E3A-D5025393D432@gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
	<2B666E0F-8B3D-4E66-9E3A-D5025393D432@gmail.com>
Message-ID: <CAB4yi1PN04z2dEiR1hXx7=pBtq90wZNpWh124qFhXDrjCKySkQ@mail.gmail.com>

?_?

On Tue, Dec 13, 2011 at 9:40 AM, Massimo Di Pierro
<massimo.dipierro at gmail.com> wrote:
> I do not like it and I am not convinced this belongs to Python
>
> Yet you may like this other solution:
>
> exec(reindent("def f(a):; x=0; for i in range(a):; x+=i; pass; return x; print f(5)"))
>
> here reindent is a modification of the web2py reindent function (used in templates).
>
> - ; is converted to \n
> - loops are closed with pass
> - ifs are also closed with pass
> - def can be closed with pass or return
> - class must also end with pass
>
> In web2py-ese templates we would not use it like this. we would write instead:
>
> {{def f(a):}}{{x=0}}{{for i in range(a):}}{{x+=i}}{{pass}}{{return x}}{{=f(5)}}
>
> or break it into
>
> {{
> def f(a):
> ? x=0
> ? for i in range(a):
> ? ? ?x+=i
> ? ? ?pass
> ? return x
> = f(5)
> }} (indentation is irrelevant even within the {{code}})
>
>
>
>
>
>
>
> On Dec 12, 2011, at 4:07 PM, Ron Adam wrote:
>
>>
>> While typing this reply to the "Alternate Suite Delineation Syntax For
>> Python?" thread. ?I realized that having things ?all on a single line
>> isn't important or needed!!! ?Yes, it's not needed for templates, or
>> code generation. ?So this is an alternate to the alternate.
>>
>> What is needed is a uniform way to represent indenting that is easy to
>> transport to (ie paste into) code with different indent levels.
>>
>>
>> INDENT LEVEL MARKERS:
>> ----------------------
>> Offer a new way to spell <indent+1>, <indent+0>, and <indent-1>.
>>
>> To make that work, we need to use unique character sequences that aren't
>> already used. ?After some thought, I came up with these indent-markers.
>>
>> ? ?/// ? # indent level +1
>> ? ?;;; ? # indent level +0
>> ? ?\\\ ? # indent level -1
>>
>> These are unique enough to be easy to see, and the tripled versions of
>> them aren't really that bad as we already have triple quotes, and they
>> would be less than the white space they are replacing in most cases.
>>
>> So this isn't brackets, or braces. It would be just an alternate
>> spelling in already valid python. It should not effect the parser,
>> grammar, or interpreter in any way if it is done in pythons
>> tokenizer. ?:-)
>>
>>
>> Reusing some previous examples.
>>
>> #-----------------------
>>
>>> ? def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return
>>> s}}}
>>
>> #Standard python...
>>
>> def acc(n):
>> ? ?s=n
>> ? ?def add(i):
>> ? ? ? ?nonlocal s
>> ? ? ? ?s += i
>> ? ? ? ?return s
>> ? ?return add
>>
>>
>> #Single line version...
>>
>> ?def acc(n): /// s=n ;;; def add(i): /// nonlocal s \
>> ?;;; s+=i ;;; return s \\\ return add
>>
>> (The '\' character should still work.)
>>
>>
>> #Transportable indent level version...
>>
>> This could also be valid... and could be transportable to any other
>> indent level without having to edit it.
>>
>> (So would the above single line version by adding a ';;;' to the
>> beginning.)
>>
>> ;;; def acc(n): ? ? # indent +0
>> /// s = n ? ? ? ? ? # indent +1
>> ;;; def add(i): ? ? # indent +0
>> /// nonlocal s ? ? ?# indent +1
>> ;;; s += i ? ? ? ? ?# indent +0
>> ;;; return s ? ? ? ?# indent +0
>> \\\ return add ? ? ?# indent -1
>>
>> It has the nice properties of being uniform, and being able to be
>> inserted easily into any existing code block without having to adjust
>> the indent levels.
>>
>> I believe that these properties are more important than being able to
>> put things on a single line and could be a very useful in code
>> generating programs. ;-)
>>
>>
>> It may be easy to implement.
>>
>> This could probably all be done in the tokinze.c get_tok() function,
>> where it detects and converts white space to indention levels. ?Nothing
>> else in python would need to be changed. ?The tokenizer already detects
>> line continuations and different line endings, so this wouldn't be that
>> out of place.
>>
>> Cheers,
>> ? Ron
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
?_?


From python at mrabarnett.plus.com  Tue Dec 13 00:15:39 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 12 Dec 2011 23:15:39 +0000
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
 \\\
In-Reply-To: <1323727633.29879.181.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
Message-ID: <4EE68B1B.6020406@mrabarnett.plus.com>

On 12/12/2011 22:07, Ron Adam wrote:
>
> While typing this reply to the "Alternate Suite Delineation Syntax For
> Python?" thread.  I realized that having things  all on a single line
> isn't important or needed!!!  Yes, it's not needed for templates, or
> code generation.  So this is an alternate to the alternate.
>
> What is needed is a uniform way to represent indenting that is easy to
> transport to (ie paste into) code with different indent levels.
>
>
> INDENT LEVEL MARKERS:
> ----------------------
> Offer a new way to spell<indent+1>,<indent+0>, and<indent-1>.
>
> To make that work, we need to use unique character sequences that aren't
> already used.  After some thought, I came up with these indent-markers.
>
>      ///   # indent level +1
>      ;;;   # indent level +0
>      \\\   # indent level -1
>
> These are unique enough to be easy to see, and the tripled versions of
> them aren't really that bad as we already have triple quotes, and they
> would be less than the white space they are replacing in most cases.
>
> So this isn't brackets, or braces. It would be just an alternate
> spelling in already valid python. It should not effect the parser,
> grammar, or interpreter in any way if it is done in pythons
> tokenizer.  :-)
>
[snip]
-1

If the problem is with leading whitespace being stripped, then what
about the non-breaking space ("\xA0")? We all use Unicode now, don't
we? :-)

The bonus is that it still looks like a space.


From ron3200 at gmail.com  Tue Dec 13 01:01:00 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Mon, 12 Dec 2011 18:01:00 -0600
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
 \\\
In-Reply-To: <4EE68B1B.6020406@mrabarnett.plus.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
	<4EE68B1B.6020406@mrabarnett.plus.com>
Message-ID: <1323734460.29879.231.camel@Gutsy>

On Mon, 2011-12-12 at 23:15 +0000, MRAB wrote:
> On 12/12/2011 22:07, Ron Adam wrote:
> >
> > While typing this reply to the "Alternate Suite Delineation Syntax For
> > Python?" thread.  I realized that having things  all on a single line
> > isn't important or needed!!!  Yes, it's not needed for templates, or
> > code generation.  So this is an alternate to the alternate.
> >
> > What is needed is a uniform way to represent indenting that is easy to
> > transport to (ie paste into) code with different indent levels.
> >
> >
> > INDENT LEVEL MARKERS:
> > ----------------------
> > Offer a new way to spell<indent+1>,<indent+0>, and<indent-1>.
> >
> > To make that work, we need to use unique character sequences that aren't
> > already used.  After some thought, I came up with these indent-markers.
> >
> >      ///   # indent level +1
> >      ;;;   # indent level +0
> >      \\\   # indent level -1
> >
> > These are unique enough to be easy to see, and the tripled versions of
> > them aren't really that bad as we already have triple quotes, and they
> > would be less than the white space they are replacing in most cases.
> >
> > So this isn't brackets, or braces. It would be just an alternate
> > spelling in already valid python. It should not effect the parser,
> > grammar, or interpreter in any way if it is done in pythons
> > tokenizer.  :-)
> >
> [snip]
> -1
> 
> If the problem is with leading whitespace being stripped, then what
> about the non-breaking space ("\xA0")? We all use Unicode now, don't
> we? :-)

Think bigger!  ie... easy automated program content and construction.

You have this string...

foo_bit.py
"""
def foo(*args):
    result = <do something with args>
    return result
x = foo(a, b, c)
"""

And you want it to be inserted in this string automatically with the
least amount of processing.  (with another program you have.)

bar_bit.py
"""
    ...
    def bar(a, b, c):
        ...
<insert foo_bit.py here>
        ...
        return x
    ...
"""
(where ... means more lines of python code.)


Now if you do a quick simple substitution you get.

"""
    ...
    def bar(a, b, c):
        ...
def foo(*args):
    result = <do something with args>
    return result
x = foo(a, b, c)
        ...
        return x
    ...
"""

Which won't work and is why template programs need to
do a lot of pre and post processing to get things right.

If foo_bit.py was like this...

"""
;;; def foo(*args):
;;;    result = <do something with args>
;;;    return result
;;; x = foo(a, b, c)
"""

Then you would get...

"""
    ...
    def bar(a, b, c):
        ...
;;; def foo(*args):
;;;    result = <do something with args>
;;;    return result
;;; x = foo(a, b, c)
        ...
        return x
    ...
"""

And the tokenizer in python could read that directly as ...

    ...
    def bar(a, b, c):
        ...
        def foo(*args):
            result = <do something with args>
            return result
        x = foo(a, b, c)
        ...
        return x
    ...

Which would make doing things like this really easy instead of really
tricky.

The indent level of bar.py won't matter because the indent marker ';;;'
says to use the current level of indent before this line.

Does that make more sense now?

Cheers,
   Ron




























From python at mrabarnett.plus.com  Tue Dec 13 01:40:44 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 13 Dec 2011 00:40:44 +0000
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
 \\\
In-Reply-To: <1323734460.29879.231.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
	<4EE68B1B.6020406@mrabarnett.plus.com>
	<1323734460.29879.231.camel@Gutsy>
Message-ID: <4EE69F0C.5060501@mrabarnett.plus.com>

On 13/12/2011 00:01, Ron Adam wrote:
> On Mon, 2011-12-12 at 23:15 +0000, MRAB wrote:
>> On 12/12/2011 22:07, Ron Adam wrote:
>>>
>>> While typing this reply to the "Alternate Suite Delineation Syntax For
>>> Python?" thread.  I realized that having things  all on a single line
>>> isn't important or needed!!!  Yes, it's not needed for templates, or
>>> code generation.  So this is an alternate to the alternate.
>>>
>>> What is needed is a uniform way to represent indenting that is easy to
>>> transport to (ie paste into) code with different indent levels.
>>>
>>>
>>> INDENT LEVEL MARKERS:
>>> ----------------------
>>> Offer a new way to spell<indent+1>,<indent+0>, and<indent-1>.
>>>
>>> To make that work, we need to use unique character sequences that aren't
>>> already used.  After some thought, I came up with these indent-markers.
>>>
>>>       ///   # indent level +1
>>>       ;;;   # indent level +0
>>>       \\\   # indent level -1
>>>
>>> These are unique enough to be easy to see, and the tripled versions of
>>> them aren't really that bad as we already have triple quotes, and they
>>> would be less than the white space they are replacing in most cases.
>>>
>>> So this isn't brackets, or braces. It would be just an alternate
>>> spelling in already valid python. It should not effect the parser,
>>> grammar, or interpreter in any way if it is done in pythons
>>> tokenizer.  :-)
>>>
>> [snip]
>> -1
>>
>> If the problem is with leading whitespace being stripped, then what
>> about the non-breaking space ("\xA0")? We all use Unicode now, don't
>> we? :-)
>
> Think bigger!  ie... easy automated program content and construction.
>
> You have this string...
>
> foo_bit.py
> """
> def foo(*args):
>      result =<do something with args>
>      return result
> x = foo(a, b, c)
> """
>
> And you want it to be inserted in this string automatically with the
> least amount of processing.  (with another program you have.)
>
> bar_bit.py
> """
>      ...
>      def bar(a, b, c):
>          ...
> <insert foo_bit.py here>
>          ...
>          return x
>      ...
> """
> (where ... means more lines of python code.)
>
>
> Now if you do a quick simple substitution you get.
>
> """
>      ...
>      def bar(a, b, c):
>          ...
> def foo(*args):
>      result =<do something with args>
>      return result
> x = foo(a, b, c)
>          ...
>          return x
>      ...
> """
>
OK, let's suppose that bar_bit.py is a template file with an insertion
point:

bar_bit.py
"""
      ...
      def bar(a, b, c):
          ...
          <insert here>
          ...
          return x
      ...
"""

I would expect the inserted code to be indented by however much the
"<insert here>" is indented:

"""
      ...
      def bar(a, b, c):
          ...
          def foo(*args):
               result =<do something with args>
               return result
          x = foo(a, b, c)
          ...
          return x
      ...
"""

> Which won't work and is why template programs need to
> do a lot of pre and post processing to get things right.
>
> If foo_bit.py was like this...
>
> """
> ;;; def foo(*args):
> ;;;    result =<do something with args>
> ;;;    return result
> ;;; x = foo(a, b, c)
> """
>
> Then you would get...
>
> """
>      ...
>      def bar(a, b, c):
>          ...
> ;;; def foo(*args):
> ;;;    result =<do something with args>
> ;;;    return result
> ;;; x = foo(a, b, c)
>          ...
>          return x
>      ...
> """
>
> And the tokenizer in python could read that directly as ...
>
>      ...
>      def bar(a, b, c):
>          ...
>          def foo(*args):
>              result =<do something with args>
>              return result
>          x = foo(a, b, c)
>          ...
>          return x
>      ...
>
> Which would make doing things like this really easy instead of really
> tricky.
>
> The indent level of bar.py won't matter because the indent marker ';;;'
> says to use the current level of indent before this line.
>
> Does that make more sense now?
>
I still think it looks messy and that it's the responsibility of the
templater to fix the indentation.


From stephen at xemacs.org  Tue Dec 13 03:44:29 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 13 Dec 2011 11:44:29 +0900
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EE67047.8050209@canterbury.ac.nz>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
Message-ID: <87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > Masklinn wrote:
 > 
 > > FWIW, Haskell does not have a literal singleton (the standard defines 
 > > "unit" `()` and 2-tuple through 15-tuple)
 > 
 > That's because, due to its static typing, there is no
 > reason you would ever need to use a 1-tuple rather than
 > a bare value. We're not that lucky in Python, though.

I think you have misstated your point?  That's not due to static
typing, that's because you may *always* identify 1-factor products
with the only factor, and Haskell made a deliberate decision to
consistently represent the isomorphism class by the factor rather than
the product.  Eg, Python uses the same strategy for characters and
strings (characters as a category are isomorphic to one-element
strings), but chose a different representative (the one-element string
rather than the character).

As for Python's relative lack of luck, I think that's unlikely to be
the reason why things are as they are.  I'm pretty sure that Python's
omission of character objects was deliberate<wink/>.  I think it was
the right choice, given that (1) despite category theory, I believe we
think of tuples as composite objects (at least I do), but strings seem
to be more "monolithic" in some sense, and (2) pretty much all
languages have chosen to leave identifiers unmarked (I guess Perl and
many template languages should be considered exceptions), and mark
strings and characters (often both with quotation marks, sometimes
characters with a prefix) -- may as well go with the familiar if
there's no good reason otherwise.





From tjreedy at udel.edu  Tue Dec 13 06:24:23 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 13 Dec 2011 00:24:23 -0500
Subject: [Python-ideas] Transportable indent level markers.   >>>===<<<
In-Reply-To: <1323727633.29879.181.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
Message-ID: <jc6ni9$f64$1@dough.gmane.org>

On 12/12/2011 5:07 PM, Ron Adam wrote:

> To make that work, we need to use unique character sequences that aren't
> already used.  After some thought, I came up with these indent-markers.
>
>      ///   # indent level +1
>      ;;;   # indent level +0
>      \\\   # indent level -1

 >>>
===
<<<
would work even better ;-).

-- 
Terry Jan Reedy



From guido at python.org  Tue Dec 13 16:53:35 2011
From: guido at python.org (Guido van Rossum)
Date: Tue, 13 Dec 2011 07:53:35 -0800
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <jc6ni9$f64$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
Message-ID: <CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>

On Mon, Dec 12, 2011 at 9:24 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/12/2011 5:07 PM, Ron Adam wrote:
>
>> To make that work, we need to use unique character sequences that aren't
>> already used. ?After some thought, I came up with these indent-markers.
>>
>> ? ? /// ? # indent level +1
>> ? ? ;;; ? # indent level +0
>> ? ? \\\ ? # indent level -1
>
>
> >>>
> ===
> <<<
> would work even better ;-).

No they wouldn't, these are too similar to the markers left behind by
merge tools for unmergable sections. Those markers were chosen because
they are unlikely to occur in regular code. Let's keep it that way.
:-)

-- 
--Guido van Rossum (python.org/~guido)


From tjreedy at udel.edu  Thu Dec 15 00:53:55 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 14 Dec 2011 18:53:55 -0500
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
Message-ID: <jcbcuo$man$1@dough.gmane.org>

On 12/13/2011 10:53 AM, Guido van Rossum wrote:
> On Mon, Dec 12, 2011 at 9:24 PM, Terry Reedy<tjreedy at udel.edu>  wrote:
>> On 12/12/2011 5:07 PM, Ron Adam wrote:
>>
>>> To make that work, we need to use unique character sequences that aren't
>>> already used.  After some thought, I came up with these indent-markers.
>>>
>>>      ///   # indent level +1
>>>      ;;;   # indent level +0
>>>      \\\   # indent level -1

My objection to these is that the interpretation is arbitrary; I happen 
to read them the wrong way.

>>
>>
>>>>>
>> ===
>> <<<
>> would work even better ;-).
>
> No they wouldn't, these are too similar to the markers left behind by
> merge tools for unmergable sections. Those markers were chosen because
> they are unlikely to occur in regular code. Let's keep it that way.
> :-)

Funny. I choose them for the same reason. >>> is also bad because some 
mail agents, like Thunderbird not only interpret initial runs of > as 
message quotes but replace them with various colored bars. So if this 
were a serious idea, I would propose something else like
->|
-|-
|<-
But a moot point unless we actually do want a standardized transport 
encoding for indents and dedents.

-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Thu Dec 15 01:18:10 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 15 Dec 2011 13:18:10 +1300
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <jcbcuo$man$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org>
Message-ID: <4EE93CC2.1070200@canterbury.ac.nz>

On 15/12/11 12:53, Terry Reedy wrote:

> Funny. I choose them for the same reason. >>> is also bad because some mail
> agents, like Thunderbird not only interpret initial runs of > as message
> quotes but replace them with various colored bars.

Does anyone know a way of turning that off, by the way?
It drives me batty when I'm looking at Python interactive
session transcripts.

-- 
Greg


From ron3200 at gmail.com  Thu Dec 15 02:08:19 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Wed, 14 Dec 2011 19:08:19 -0600
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <jcbcuo$man$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org>
Message-ID: <1323911299.9574.44.camel@Gutsy>

On Wed, 2011-12-14 at 18:53 -0500, Terry Reedy wrote:
> On 12/13/2011 10:53 AM, Guido van Rossum wrote:
> > On Mon, Dec 12, 2011 at 9:24 PM, Terry Reedy<tjreedy at udel.edu>  wrote:
> >> On 12/12/2011 5:07 PM, Ron Adam wrote:
> >>
> >>> To make that work, we need to use unique character sequences that aren't
> >>> already used.  After some thought, I came up with these indent-markers.
> >>>
> >>>      ///   # indent level +1
> >>>      ;;;   # indent level +0
> >>>      \\\   # indent level -1
> 
> My objection to these is that the interpretation is arbitrary; I happen 
> to read them the wrong way.
> 
> >>
> >>
> >>>>>
> >> ===
> >> <<<
> >> would work even better ;-).
> >
> > No they wouldn't, these are too similar to the markers left behind by
> > merge tools for unmergable sections. Those markers were chosen because
> > they are unlikely to occur in regular code. Let's keep it that way.
> > :-)
> 
> Funny. I choose them for the same reason. >>> is also bad because some 
> mail agents, like Thunderbird not only interpret initial runs of > as 
> message quotes but replace them with various colored bars. So if this 
> were a serious idea, I would propose something else like
> ->|
> -|-
> |<-
> But a moot point unless we actually do want a standardized transport 
> encoding for indents and dedents.

I think if there is enough interest in it, it could be done.

It only changes the pre-tokenized representation of the language.  To do
braces correctly, it would require much deeper changes.

My choices are based on both easy visibility and also what be nice for
single line input.  The ';;;' is consistent with the ';' in places where
it is used.  Triple characters are easier to type than odd sequences.

I suppose I'd switch the slashes though so they follow the top down flow
better.

;;; def acc(n):         # indent +0
\\\     s = n           # indent +1
;;;     def add(i):     # indent +0
\\\         nonlocal s  # indent +1
;;;         s += i      # indent +0
;;;         return s    # indent +0
///     return add      # indent -1

cheers,
   Ron



   
















From andre.roberge at gmail.com  Thu Dec 15 02:13:34 2011
From: andre.roberge at gmail.com (Andre Roberge)
Date: Wed, 14 Dec 2011 21:13:34 -0400
Subject: [Python-ideas] Fwd: Transportable indent level markers. >>>===<<<
In-Reply-To: <CAGMu_=pKO1aHy7PXE1b_B8UsZ+R=y--4y7HvRKBqK6SbgaRm3Q@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CAGMu_=pKO1aHy7PXE1b_B8UsZ+R=y--4y7HvRKBqK6SbgaRm3Q@mail.gmail.com>
Message-ID: <CAGMu_=rOn-zf1WNDxW6VbEkiykP4cLMaCvjPiPbQ3ODPqoH4sA@mail.gmail.com>

On Wed, Dec 14, 2011 at 9:08 PM, Ron Adam <ron3200 at gmail.com> wrote:

> On Wed, 2011-12-14 at 18:53 -0500, Terry Reedy wrote:
> > On 12/13/2011 10:53 AM, Guido van Rossum wrote:
> > > On Mon, Dec 12, 2011 at 9:24 PM, Terry Reedy<tjreedy at udel.edu>  wrote:
> > >> On 12/12/2011 5:07 PM, Ron Adam wrote:
> > >>
> > >>> To make that work, we need to use unique character sequences that
> aren't
> > >>> already used.  After some thought, I came up with these
> indent-markers.
> > >>>
> > >>>      ///   # indent level +1
> > >>>      ;;;   # indent level +0
> > >>>      \\\   # indent level -1
>

Why not use INDENT, ;, and DEDENT   (or   <INDENT>   <;>   and <DEDENT>) ?

;-)


Andr?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111214/de6a0c6b/attachment.html>

From python at mrabarnett.plus.com  Thu Dec 15 02:14:04 2011
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 15 Dec 2011 01:14:04 +0000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <4EE93CC2.1070200@canterbury.ac.nz>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <4EE93CC2.1070200@canterbury.ac.nz>
Message-ID: <4EE949DC.90200@mrabarnett.plus.com>

On 15/12/2011 00:18, Greg Ewing wrote:
> On 15/12/11 12:53, Terry Reedy wrote:
>
>>  Funny. I choose them for the same reason.>>>  is also bad because some mail
>>  agents, like Thunderbird not only interpret initial runs of>  as message
>>  quotes but replace them with various colored bars.
>
> Does anyone know a way of turning that off, by the way?
> It drives me batty when I'm looking at Python interactive
> session transcripts.
>
It annoys me too, so I went looking:

Quote bars
http://kb.mozillazine.org/Quote_bars

On my PC the prefs.js file was in a subfolder of:

C:\Documents and Settings\Administrator\Application 
Data\Thunderbird\Profiles

and it now shows the ">" as ">".


From ncoghlan at gmail.com  Thu Dec 15 02:21:34 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 11:21:34 +1000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <jcbcuo$man$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org>
Message-ID: <CADiSq7eUtW_rKiez-HN+9SS7aVmQzOhOEWcdRb-WbQgXK__PEw@mail.gmail.com>

On Thu, Dec 15, 2011 at 9:53 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> But a moot point unless we actually do want a standardized transport
> encoding for indents and dedents.

And if we *were* going to do that, it's far more sensible to use a
scheme that is at least *theoretically* open to the possibility of
eventually allowing suite expressions.

For the record, if anyone really wants to play with this, I'd start
with the tokenize module, and figure out what tweaks are needed to
create the following mappings between the source code and the token
stream:

"{:"  <-> token.OP(":"), token.NEWLINE, token.INDENT

":}"  <-> token.NEWLINE, token.DEDENT

Then you can do source-source transforms by taking the output of the
tweaked tokenizer and feeding it to the standard one and vice-versa.

That's should be all you really need to create a delimited syntax
variant. Things only get complex if you try to turn that delimited
syntax variant into multi-line lambda support.

P.S. See http://bugs.python.org/issue2134 in regard to why you can't
use token.COLON for this

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Thu Dec 15 02:29:48 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 11:29:48 +1000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <1323911299.9574.44.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
Message-ID: <CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>

On Thu, Dec 15, 2011 at 11:08 AM, Ron Adam <ron3200 at gmail.com> wrote:
> It only changes the pre-tokenized representation of the language. ?To do
> braces correctly, it would require much deeper changes.

This comment makes me think you may be working off a misunderstanding
of the way Python's tokenisation works.

Suites in Python's token stream are *already* explicitly delimited:
"INDENT" and "DEDENT" tokens respectively mark the beginning and end
of each suite. There's no such concept in the token stream as "don't
change the indent level" - that's the assumed behaviour.

So an explicitly delimited syntax that just offers an alternative way
to get INDENT and DEDENT tokens into the stream would be fairly
straightforward.

You'd probably also want an explicit ";;" token to force a
token.NEWLINE into the token stream.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ron3200 at gmail.com  Thu Dec 15 04:19:08 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Wed, 14 Dec 2011 21:19:08 -0600
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
Message-ID: <1323919148.10290.59.camel@Gutsy>

On Thu, 2011-12-15 at 11:29 +1000, Nick Coghlan wrote:
> On Thu, Dec 15, 2011 at 11:08 AM, Ron Adam <ron3200 at gmail.com> wrote:
> > It only changes the pre-tokenized representation of the language.  To do
> > braces correctly, it would require much deeper changes.
> 
> This comment makes me think you may be working off a misunderstanding
> of the way Python's tokenisation works.

In tokenizer.c there is the function tok_get() at line 1292.  The first
part of it counts spaces and/or tabs and checks for other things and
then returns either DEDENT, INDENT, or possibly an ERRORTOKEN.  

Since this would not add any new tokens, I referred to it as a
pretokenized representation.

The way it would work is to detects these characters and use it to
determine wheather to return DEDENT, or INDENT tokens in place of any
surrounding white space immediately before or after it.

If these are found in the middle of a line, it would backup with
tok_backup(), to just before the token, and then ruturn a NEWLINE.

On the next tok_get() call, it would repeat.  It's would not be hard to
do at all. 


> Suites in Python's token stream are *already* explicitly delimited:
> "INDENT" and "DEDENT" tokens respectively mark the beginning and end
> of each suite. There's no such concept in the token stream as "don't
> change the indent level" - that's the assumed behaviour.

You are thinking of later in the chain I think.  We doing the change
before that is decided.  Think of them as hints for the tokenizer.  The
"don't change the indent level" hint, ';;;' is just a hint to ignore the
white space here and accept the previous level.  Because of that, the
text source is able to be white space insensitive as far as indent
levels is concerned.

> So an explicitly delimited syntax that just offers an alternative way
> to get INDENT and DEDENT tokens into the stream would be fairly
> straightforward.

Right. :-)

> You'd probably also want an explicit ";;" token to force a
> token.NEWLINE into the token stream.

That isn't needed.  Any of these in the middle of a line will add a new
line and back up, so the next call to tok_get() will find it, and so on.

  ;;; x = 3 ;;; y = 2 ;;; def add(x,y): \\\ return x + y

So this would be converted as the tokenizer goes over it to..

  x = 3
  y = 2
  def add(x,y):
      return x + y


Line continuations '\' are handled in pretty much the same way, they
just eat the next newline, and continue.  There is no token for a line
continuation.  These would work on the same level,

Cheers,
  Ron




From ncoghlan at gmail.com  Thu Dec 15 04:40:13 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 13:40:13 +1000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <1323919148.10290.59.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
Message-ID: <CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>

On Thu, Dec 15, 2011 at 1:19 PM, Ron Adam <ron3200 at gmail.com> wrote:
>> You'd probably also want an explicit ";;" token to force a
>> token.NEWLINE into the token stream.
>
> That isn't needed. ?Any of these in the middle of a line will add a new
> line and back up, so the next call to tok_get() will find it, and so on.

OK, take the way you're thinking (indent +1, indent 0, indent -1) and
instead think in terms of starting a suite, terminating a statement
and terminating a suite:

/// -> {:
;;; -> ;;
\\\ -> :}

Now do you see why I'm saying you're needlessly complicating things?
Suite delimiters and statement terminators (or separators) are the way
full whitespace insensitivity is normally handled when designing a
language syntax. There's no reason to get creative here when the
standard terminology and conventions would work just fine.

For example, it shouldn't be difficult to create a variant of the
tokenize module's tokeniser that adds the following rules:

{: -> emits OP(':'), NEWLINE, INDENT and increments the parenlevel
;; -> emits NEWLINE
:} -> emits NEWLINE, DEDENT and decrements the parenlevel

and a variant of the untokenizer() that looks ahead and *emits* those
character sequences when applicable.

That should be enough to let you use Python code in whitespace
insensitive environments without changing the semantics:

Encoding for transport: tokenize() normally, untokenize() with suite delimiters
Decoding from transport: tokenize() with suite delimiter support,
untokenize() normally

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ron3200 at gmail.com  Thu Dec 15 06:09:17 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Wed, 14 Dec 2011 23:09:17 -0600
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
	<CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
Message-ID: <1323925757.11678.59.camel@Gutsy>

On Thu, 2011-12-15 at 13:40 +1000, Nick Coghlan wrote:
> On Thu, Dec 15, 2011 at 1:19 PM, Ron Adam <ron3200 at gmail.com> wrote:
> >> You'd probably also want an explicit ";;" token to force a
> >> token.NEWLINE into the token stream.
> >
> > That isn't needed.  Any of these in the middle of a line will add a new
> > line and back up, so the next call to tok_get() will find it, and so on.
> 
> OK, take the way you're thinking (indent +1, indent 0, indent -1) and
> instead think in terms of starting a suite, terminating a statement
> and terminating a suite:
> 
> /// -> {:
> ;;; -> ;;
> \\\ -> :}
> 
> Now do you see why I'm saying you're needlessly complicating things?

The complicated part is in getting across a new idea. ;-)


> Suite delimiters and statement terminators (or separators) are the way
> full whitespace insensitivity is normally handled when designing a
> language syntax. There's no reason to get creative here when the
> standard terminology and conventions would work just fine.
> 
> For example, it shouldn't be difficult to create a variant of the
> tokenize module's tokeniser that adds the following rules:

I'm not referring to a whole new variant, Just a small tweak to the
current one.


> {: -> emits OP(':'), NEWLINE, INDENT and increments the parenlevel
> ;; -> emits NEWLINE
> :} -> emits NEWLINE, DEDENT and decrements the parenlevel

This is the same idea with different spelling in this case.  I think
yours would be a bit harder to implement, but not all that hard to do.
The version I proposed would be even easier to do.

Unless you intend to enforce matching '{:' and ':}''s, then it become a
bit harder still.  I'm not sure that is needed, but it may seem strange
to users if you don't require them to match.  At what point should it be
a part of the grammar?


Ok how about some examples... how would you notate this code bit so it
can be pasted into an already existing file without having to re-edit
it?

  ;;; for x in items:
  \\\ sum = op(x)
  ;;; if sum > MAX:
  \\\ break

Lets say you are going to past in several locations in a file and those
locations may have different indent levels.  How would you write that
using your suggestion so you don't have to change it in anyway for it to
work.

With my suggestion, that would just work no matter the source file
indent level. (*)  All you need is the line numbers to insert it at.  I
think with your idea, you would also need a place holder of some type.

   * Assuming the required names are present. ;-)


> and a variant of the untokenizer() that looks ahead and *emits* those
> character sequences when applicable.

That shouldn't be a problem in either case.

> That should be enough to let you use Python code in whitespace
> insensitive environments without changing the semantics:

Right, except I'd be happier with something that doesn't alter the
visible characters, the colon included.

> Encoding for transport: tokenize() normally, untokenize() with suite delimiters
> Decoding from transport: tokenize() with suite delimiter support,
> untokenize() normally

I think one of the differences is I'm proposing a mode that doesn't
require the whole file to be in matching form.

So the "tokenize() with suite delimiter support" would just be the same
tokenize() that is normally used.  And the "untokenize() with suite
delimiters" would probably just be a file that you run on the source to
get a delimited version.  I'm not proposing any untokenize() support.
But I'm not against it either. <shrug>  

I also think indent markers are more consistent with how people think
when they are programming python.  It matches how the blocks are
defined.

If this was a language where we already had braces, then I'd definitely
be thinking the other way.  :-)

Cheers,
   Ron




From ncoghlan at gmail.com  Thu Dec 15 06:25:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 15:25:33 +1000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <1323925757.11678.59.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
	<CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
	<1323925757.11678.59.camel@Gutsy>
Message-ID: <CADiSq7f4qDKLDDHA4+2KUy8-uMjua978ZWAVpOY4X6fWahwv6g@mail.gmail.com>

On Thu, Dec 15, 2011 at 3:09 PM, Ron Adam <ron3200 at gmail.com> wrote:
> Lets say you are going to past in several locations in a file and those
> locations may have different indent levels. ?How would you write that
> using your suggestion so you don't have to change it in anyway for it to
> work.
>
> With my suggestion, that would just work no matter the source file
> indent level. (*) ?All you need is the line numbers to insert it at. ?I
> think with your idea, you would also need a place holder of some type.

OK, I at least understand what you're trying to do now. However, I
don't think you have a valid use case. What would a templating
language based on your notation look like? How would you extend it to
implement multi-line lambdas? If you can't cleanly do those two
things, who cares if it lets you copy and paste snippets at arbitrary
points in a Python file? What's the *functional* pay-off for the
ugliness?

(And I freely admit that my suggestion is also quite ugly. However,
the status quo, with everyone inventing assorted templating languages
for use with Python that are largely independent of Python itself
already sucks)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Thu Dec 15 06:31:15 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 15:31:15 +1000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <1323925757.11678.59.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
	<CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
	<1323925757.11678.59.camel@Gutsy>
Message-ID: <CADiSq7eAnNrvwGDFPxYjQUxYRK1mgshQNt-9NJDZW2Wdf+roNg@mail.gmail.com>

On Thu, Dec 15, 2011 at 3:09 PM, Ron Adam <ron3200 at gmail.com> wrote:
> Lets say you are going to past in several locations in a file and those
> locations may have different indent levels. ?How would you write that
> using your suggestion so you don't have to change it in anyway for it to
> work.

Forget to answer this bit: I wouldn't. I'd use a templating language,
that's what they're for.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ron3200 at gmail.com  Thu Dec 15 08:06:21 2011
From: ron3200 at gmail.com (Ron Adam)
Date: Thu, 15 Dec 2011 01:06:21 -0600
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <CADiSq7f4qDKLDDHA4+2KUy8-uMjua978ZWAVpOY4X6fWahwv6g@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
	<CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
	<1323925757.11678.59.camel@Gutsy>
	<CADiSq7f4qDKLDDHA4+2KUy8-uMjua978ZWAVpOY4X6fWahwv6g@mail.gmail.com>
Message-ID: <1323932781.11678.131.camel@Gutsy>

On Thu, 2011-12-15 at 15:25 +1000, Nick Coghlan wrote:
> On Thu, Dec 15, 2011 at 3:09 PM, Ron Adam <ron3200 at gmail.com> wrote:
> > Lets say you are going to past in several locations in a file and those
> > locations may have different indent levels.  How would you write that
> > using your suggestion so you don't have to change it in anyway for it to
> > work.
> >
> > With my suggestion, that would just work no matter the source file
> > indent level. (*)  All you need is the line numbers to insert it at.  I
> > think with your idea, you would also need a place holder of some type.
> 
> OK, I at least understand what you're trying to do now. However, I
> don't think you have a valid use case. What would a templating
> language based on your notation look like?

Any program that can patch text files together.  As to how it does it,
it probably would use a script of some sort.  This would just make it
easier to by offering a way around indenting issues.

Another use case is it makes transmitting python compatible code easier
in a web page or email.  The braces version would do that also.  I think
it could make python a bit more attractive for things like that.  Where
a app is transmitted and run at the destination.   There are still the
security issues to work out, but this would knock down one domino in
that direction.

For example they don't have to event ways to avoid the white space
munging.  Python could be represented as code blocks that might look
like...

 <pycode = ";;; for x in items: \\\ sum = op(x) ;;; if sum > MAX:
 \\\ break">

Except much longer.  In this case there isn't a need for it to be
pretty. But there is a need for it to work correctly.


I was thinking it might be possible to have the python shell echo the
indent markes in the left side as it scroll lines up. Replacing the
'>>>' and '...' with the respective markes as you enter the lines.  And
it could also take these if they are pasted in.  Cutting and pasting
from one shell to another could work in that case.

As far as unglyness goes... is it really that bad?  you can leave the
white space in.  The indent hints are used not the spaces in this case.

  ;;; for x in items:
  \\\     sum = op(x)
  ;;;     if sum > MAX:
  \\\         break

And keep in mind that this isn't something you will see in normal use.
It's just available for when its needed.  But if you run across code
like this it will still work.  Nothing special would need to be done to
make it work.

> How would you extend it to
> implement multi-line lambdas?

As I said, it doesn't change the language at all.  So extending lambda
is not in the scope of this topic.  That would be something new and
should be done separately in any case.

> If you can't cleanly do those two
> things, who cares if it lets you copy and paste snippets at arbitrary
> points in a Python file? What's the *functional* pay-off for the
> ugliness?

? See above.

> (And I freely admit that my suggestion is also quite ugly. However,
> the status quo, with everyone inventing assorted templating languages
> for use with Python that are largely independent of Python itself
> already sucks)

If it was a big patch to many files, I would say it isn't worth it.
But it would be a very small patch to only one file and wouldn't change
the grammer, ast, or anything else.  It's almost free.

I think it may have a nice benefit for very little effort. But I think
we should just wait a bit and see if there is more interest first.
;-)

Cheers,
    Ron




From ethan at stoneleaf.us  Thu Dec 15 07:51:01 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 14 Dec 2011 22:51:01 -0800
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <4EE949DC.90200@mrabarnett.plus.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>	<20111210161608.13313643@bhuda.mired.org>	<4EE40DA7.4020702@pearwood.info>
	<4EE58F56.10507@canterbury.ac.nz>	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>	<1323727633.29879.181.camel@Gutsy>
	<jc6ni9$f64$1@dough.gmane.org>	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>	<jcbcuo$man$1@dough.gmane.org>
	<4EE93CC2.1070200@canterbury.ac.nz>
	<4EE949DC.90200@mrabarnett.plus.com>
Message-ID: <4EE998D5.7070902@stoneleaf.us>

MRAB wrote:
> On 15/12/2011 00:18, Greg Ewing wrote:
>> On 15/12/11 12:53, Terry Reedy wrote:
>>
>>>  Funny. I choose them for the same reason.>>>  is also bad because 
>>> some mail
>>>  agents, like Thunderbird not only interpret initial runs of>  as 
>>> message
>>>  quotes but replace them with various colored bars.
>>
>> Does anyone know a way of turning that off, by the way?
>> It drives me batty when I'm looking at Python interactive
>> session transcripts.
>>
> It annoys me too, so I went looking:
> 
> Quote bars
> http://kb.mozillazine.org/Quote_bars
> 
> On my PC the prefs.js file was in a subfolder of:
> 
> C:\Documents and Settings\Administrator\Application 
> Data\Thunderbird\Profiles
> 
> and it now shows the ">" as ">".

I went the other way and changed my python prompt to '-->'.

~Ethan~


From ncoghlan at gmail.com  Thu Dec 15 08:57:47 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 17:57:47 +1000
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <1323932781.11678.131.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
	<CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
	<1323925757.11678.59.camel@Gutsy>
	<CADiSq7f4qDKLDDHA4+2KUy8-uMjua978ZWAVpOY4X6fWahwv6g@mail.gmail.com>
	<1323932781.11678.131.camel@Gutsy>
Message-ID: <CADiSq7fubTaXBuAgWdz6u5Q__siF_5q2zS1Z4qROEbekZfWm-Q@mail.gmail.com>

There's no way *any* variant of this idea (regardless of spelling) is
getting anywhere near the stdlib without a long history on PyPI as a
popular templating and code embedding tool.

--
Nick Coghlan (via Gmail on Android, so likely to be more terse than usual)
On Dec 15, 2011 5:06 PM, "Ron Adam" <ron3200 at gmail.com> wrote:

> On Thu, 2011-12-15 at 15:25 +1000, Nick Coghlan wrote:
> > On Thu, Dec 15, 2011 at 3:09 PM, Ron Adam <ron3200 at gmail.com> wrote:
> > > Lets say you are going to past in several locations in a file and those
> > > locations may have different indent levels.  How would you write that
> > > using your suggestion so you don't have to change it in anyway for it
> to
> > > work.
> > >
> > > With my suggestion, that would just work no matter the source file
> > > indent level. (*)  All you need is the line numbers to insert it at.  I
> > > think with your idea, you would also need a place holder of some type.
> >
> > OK, I at least understand what you're trying to do now. However, I
> > don't think you have a valid use case. What would a templating
> > language based on your notation look like?
>
> Any program that can patch text files together.  As to how it does it,
> it probably would use a script of some sort.  This would just make it
> easier to by offering a way around indenting issues.
>
> Another use case is it makes transmitting python compatible code easier
> in a web page or email.  The braces version would do that also.  I think
> it could make python a bit more attractive for things like that.  Where
> a app is transmitted and run at the destination.   There are still the
> security issues to work out, but this would knock down one domino in
> that direction.
>
> For example they don't have to event ways to avoid the white space
> munging.  Python could be represented as code blocks that might look
> like...
>
>  <pycode = ";;; for x in items: \\\ sum = op(x) ;;; if sum > MAX:
>  \\\ break">
>
> Except much longer.  In this case there isn't a need for it to be
> pretty. But there is a need for it to work correctly.
>
>
> I was thinking it might be possible to have the python shell echo the
> indent markes in the left side as it scroll lines up. Replacing the
> '>>>' and '...' with the respective markes as you enter the lines.  And
> it could also take these if they are pasted in.  Cutting and pasting
> from one shell to another could work in that case.
>
> As far as unglyness goes... is it really that bad?  you can leave the
> white space in.  The indent hints are used not the spaces in this case.
>
>  ;;; for x in items:
>  \\\     sum = op(x)
>  ;;;     if sum > MAX:
>  \\\         break
>
> And keep in mind that this isn't something you will see in normal use.
> It's just available for when its needed.  But if you run across code
> like this it will still work.  Nothing special would need to be done to
> make it work.
>
> > How would you extend it to
> > implement multi-line lambdas?
>
> As I said, it doesn't change the language at all.  So extending lambda
> is not in the scope of this topic.  That would be something new and
> should be done separately in any case.
>
> > If you can't cleanly do those two
> > things, who cares if it lets you copy and paste snippets at arbitrary
> > points in a Python file? What's the *functional* pay-off for the
> > ugliness?
>
> ? See above.
>
> > (And I freely admit that my suggestion is also quite ugly. However,
> > the status quo, with everyone inventing assorted templating languages
> > for use with Python that are largely independent of Python itself
> > already sucks)
>
> If it was a big patch to many files, I would say it isn't worth it.
> But it would be a very small patch to only one file and wouldn't change
> the grammer, ast, or anything else.  It's almost free.
>
> I think it may have a nice benefit for very little effort. But I think
> we should just wait a bit and see if there is more interest first.
> ;-)
>
> Cheers,
>    Ron
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/b5a559ee/attachment.html>

From jeremy at jeremysanders.net  Thu Dec 15 11:11:01 2011
From: jeremy at jeremysanders.net (Jeremy Sanders)
Date: Thu, 15 Dec 2011 10:11:01 +0000
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
	\\\
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
Message-ID: <jcch3l$fo$1@dough.gmane.org>

Ron Adam wrote:
> 
> ;;; def acc(n):     # indent +0
> /// s = n           # indent +1
> ;;; def add(i):     # indent +0
> /// nonlocal s      # indent +1
> ;;; s += i          # indent +0
> ;;; return s        # indent +0
> \\\ return add      # indent -1
> 

To allow easy use of python in environments which aren't space-sensitive, 
wouldn't another solution be to have some sort of non-space character which 
could act as leading spaces, e.g.

def foo():
;;a = 0
;;for i in range(10):
;;;;a += i
;;return a

Of course this wouldn't work for enviroments which lose or cannot contain 
line breaks, however.

Jeremy




From ubershmekel at gmail.com  Thu Dec 15 11:18:17 2011
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Thu, 15 Dec 2011 12:18:17 +0200
Subject: [Python-ideas] Transportable indent level markers. /// ; ; ; \\\
In-Reply-To: <jcch3l$fo$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jcch3l$fo$1@dough.gmane.org>
Message-ID: <CANSw7KyNiQZi_k-ANFh9HrA9S8CsKh55__QwdF=hXmQ+pQiKtg@mail.gmail.com>

On Thu, Dec 15, 2011 at 12:11 PM, Jeremy Sanders
<jeremy at jeremysanders.net>wrote:

> Ron Adam wrote:To allow easy use of python in environments which aren't
> space-sensitive,
> wouldn't another solution be to have some sort of non-space character which
> could act as leading spaces, e.g.
>
>
Another solution - the templating language for bottle.py requires every
block opened to be closed with an "end". Eg:

if name:
  print name
else:
  input("your name please:")
__end__

Or we could use a special character for that...

http://bottlepy.org/docs/dev/stpl.html#embedded-python-code

--Yuval
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/23dd7fa4/attachment.html>

From phd at phdru.name  Thu Dec 15 11:20:38 2011
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 15 Dec 2011 14:20:38 +0400
Subject: [Python-ideas] Transportable indent level markers
In-Reply-To: <jcch3l$fo$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jcch3l$fo$1@dough.gmane.org>
Message-ID: <20111215102038.GA31413@iskra.aviel.ru>

On Thu, Dec 15, 2011 at 10:11:01AM +0000, Jeremy Sanders wrote:
> To allow easy use of python in environments which aren't space-sensitive, 
> wouldn't another solution be to have some sort of non-space character which 
> could act as leading spaces, e.g.
> 
> def foo():
> ;;a = 0
> ;;for i in range(10):
> ;;;;a += i
> ;;return a
> 
> Of course this wouldn't work for enviroments which lose or cannot contain 
> line breaks, however.

<code>
<line indent="0">def foo():</line>
<line indent="1">a = 0</line>
<line indent="1">for i in range(10):</line>
<line indent="2">a += i</line>
<line indent="1">return a</line>
</code>

   ;-)

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From jeremy at jeremysanders.net  Thu Dec 15 11:31:36 2011
From: jeremy at jeremysanders.net (Jeremy Sanders)
Date: Thu, 15 Dec 2011 10:31:36 +0000
Subject: [Python-ideas] Transportable indent level markers
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jcch3l$fo$1@dough.gmane.org>
	<20111215102038.GA31413@iskra.aviel.ru>
Message-ID: <jccia8$cno$1@dough.gmane.org>

Oleg Broytman wrote:

> <code>
> <line indent="0">def foo():</line>
> <line indent="1">a = 0</line>
> <line indent="1">for i in range(10):</line>
> <line indent="2">a += i</line>
> <line indent="1">return a</line>
> </code>
> 
>    ;-)

Nice. I'd like the option of JSON too, please, though the brackets might be 
a bit confusing :-)

Jeremy




From ncoghlan at gmail.com  Thu Dec 15 12:41:04 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 15 Dec 2011 21:41:04 +1000
Subject: [Python-ideas] Inconsistent script/console behaviour
In-Reply-To: <CAPkN8x+OEU9_npBfRwScgfFzShvEbFLxMwZTRq63-hLre3MYFA@mail.gmail.com>
References: <CAPkN8x+AcLJUQEvWVc8WtR8+MecfiNRQoA-6vAcBbuX2_BM0DQ@mail.gmail.com>
	<CAP7+vJKS4WgyqRNrtWQCfVeGa0d7A2TUxM=JOhxLJGfHJDW39g@mail.gmail.com>
	<j5k46t$dc6$1@dough.gmane.org>
	<CAPkN8x+OEU9_npBfRwScgfFzShvEbFLxMwZTRq63-hLre3MYFA@mail.gmail.com>
Message-ID: <CADiSq7f_UAuY7VsL9uviWFXTQm4ESN00VOy-RaQRYjr1B-zdUA@mail.gmail.com>

(moved thread to python-ideas, where it belongs. Messing with 20 years
of precedent is *not* a topic for python-dev)

On Thu, Dec 15, 2011 at 6:58 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>>?You can't fix this without completely changing the way the interactive
>> console treats blank lines.
>
> But the fix doesn't require changing the way interactive console treats
> blank lines at all. It only requires to finish current block when a dedented
> line is encountered and not throwing obviously confusing SyntaxError. At the
> very least it should not say it is SyntaxError, because the code is pretty
> valid Python code. If it appears to be invalid "Python Console code" - the
> error message should say that explicitly. That would be a correct
> user-friendly fix for this UX issue, but I'd still like the behavior to be
> fixed - i.e. "allow dedented lines end current block in console without
> SyntaxError". Right now I don't see the reasons why it is not possible.
>
> Please speak code when replying about use cases/examples that will be broken
> - I didn't quite get the problem with "global scope if" statements.

There are two steps to what the interactive console does:

Step 1: accept a block of code.
Step 2: compile and execute it in single statement mode (in order to
get the result of any expressions for display)

It is the *first* step that is terminated by a blank line. That step
understands almost nothing about Python syntax - just enough to figure
out whether or not it should be asking for a continuation line. (IIRC,
it pretty much just does bracket matching and "if I have seen a ':'
character, ask for continuation lines until one of them is blank".

So yes, fixing example 1 *does* require significant changes to the way
the interactive interpreter works. The interactive interpreter
currently operates on individual statements. For compound statements,
those statements are terminated by a blank line. If you don't
terminate them correctly, then it will attempt to interpret two
statements as one, and hence you will get UnexpectedDedentError.

To fix example 1, the interactive interpreter would have to be
redesigned to use exec mode instead, so it can accept multiple
statements at once. That's not impossible, but is a lot of effort to
address a relatively minor inconvenience (the inability to tolerate
blank lines *within* a suite significantly more annoying in my
experience, but even harder to fix than example 1 would be).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jimjjewett at gmail.com  Thu Dec 15 16:33:08 2011
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 15 Dec 2011 10:33:08 -0500
Subject: [Python-ideas] Transportable indent level markers. >>>===<<<
In-Reply-To: <CADiSq7f4qDKLDDHA4+2KUy8-uMjua978ZWAVpOY4X6fWahwv6g@mail.gmail.com>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jc6ni9$f64$1@dough.gmane.org>
	<CAP7+vJK0u7aDcjBrOQFGJHuAFGRqwfOuOzEMQY04ayVmDXXKNQ@mail.gmail.com>
	<jcbcuo$man$1@dough.gmane.org> <1323911299.9574.44.camel@Gutsy>
	<CADiSq7cUkzkbXJaFJBF-KfWvijPYL3u97MotJFybCW5=rufKeg@mail.gmail.com>
	<1323919148.10290.59.camel@Gutsy>
	<CADiSq7eHY6CqwnRCk-OaUWo3rOiu4jRNAHhE7TNRtJKkXoWD+Q@mail.gmail.com>
	<1323925757.11678.59.camel@Gutsy>
	<CADiSq7f4qDKLDDHA4+2KUy8-uMjua978ZWAVpOY4X6fWahwv6g@mail.gmail.com>
Message-ID: <CA+OGgf6jKVm36rcJ_=zh+D8e_SW6TvER8J86ER-Ov9DYQ+Mz8w@mail.gmail.com>

On Thu, Dec 15, 2011 at 12:25 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> (And I freely admit that my suggestion is also quite ugly. However,
> the status quo, with everyone inventing assorted templating languages
> for use with Python that are largely independent of Python itself
> already sucks)

Guido actually endorsed ugly if it is just used for transport.  That
said, for embedding in a template, I would prefer to avoid ugly.  But
why does it have to be all punctuation?


:indent 4             if is_frobbable(user, request):
:indent 5                 frob(request)
:indent 4             else:
:indent 5                  spit_error("Can't frob here, mate!")

The grammar would be that at the start of a (line, string, whatever
record marker the embedding environment has),

    the literal ":indent"
      followed by whitespace
    followed by a decimal number <count>
      followed by whitespace
    followed by a line of python code

would be equivalent to that same line of python code indented <count>
times.  Ideally, the actual leading whitespace would also demonstrate
proper indenting, but it wouldn't *need* to, so the code would be
robust against transmission.

-jJ


From jimjjewett at gmail.com  Thu Dec 15 16:39:56 2011
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 15 Dec 2011 10:39:56 -0500
Subject: [Python-ideas] Transportable indent level markers
In-Reply-To: <jccia8$cno$1@dough.gmane.org>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy> <jcch3l$fo$1@dough.gmane.org>
	<20111215102038.GA31413@iskra.aviel.ru> <jccia8$cno$1@dough.gmane.org>
Message-ID: <CA+OGgf7GN63ebx8hEJUjP9Cx=uzjUVsaxAW3nSb4CfeM42p3dQ@mail.gmail.com>

On Thu, Dec 15, 2011 at 5:31 AM, Jeremy Sanders
<jeremy at jeremysanders.net> wrote:
> Oleg Broytman wrote:
>
>> <code>
>> <line indent="0">def foo():</line>
>> <line indent="1">a = 0</line>
>> <line indent="1">for i in range(10):</line>
>> <line indent="2">a += i</line>
>> <line indent="1">return a</line>
>> </code>

This would be awkward when the code is supposed to produce an
attribute value, so that entire elements are inside an element-open
tag.

> Nice. I'd like the option of JSON too, please, though the brackets
> might be a bit confusing :-)

If you're willing to wrap each line in a string, then leading
whitespace will already be preserved.

If you're not willing to wrap each line in a string, then it isn't
clear how JSON would represent the code part, regardless of how
indention is conveyed.

-jJ


From python at 2sn.net  Thu Dec 15 17:26:50 2011
From: python at 2sn.net (Alexander Heger)
Date: Thu, 15 Dec 2011 10:26:50 -0600
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA110C.4050507@2sn.net>
References: <4EEA110C.4050507@2sn.net>
Message-ID: <4EEA1FCA.4080000@2sn.net>

Hi,

in IDL/GDL, if at have an array
x = [1,2,3]
?I can simply construct a new array by
y = [0,x]
which would give
[0,1,2,3]

I know you can do this case with
x[0:0] = [0]
but obviously more complicated cases are conceivable, common for me, in fact
x=[1,2,3]
y=[5,6,7]
z=[0,x,4,y,8]
result
[0,1,2,3,4,5,6,7,8]

or maybe even
z=[0,x[0:2],5]
and so forth.

On the other hand, if you have a function you can pass the list of
arguments to another function
def f(*args):
? g(*args)
which expands the list args. ?So, would it be possible - not even
reasonable to have something similar for list and arrays, maybe
dictionaries, in comprehensions as well?

x = [1,2,3]
y = [0,*x]
to obtain [0,1,2,3]

or similar for lists
x = (1,2,3)
y = (0,*x)
to obtain (0,1,2,3)

(for dictionaries, which are not sorted, the extend function seems fine)

Or is there a way of doing this that in a similarly compact and
obvious way I did not yet discover?

-Alexander


From masklinn at masklinn.net  Thu Dec 15 18:27:40 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 15 Dec 2011 18:27:40 +0100
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA1FCA.4080000@2sn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
Message-ID: <BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>

On 2011-12-15, at 17:26 , Alexander Heger wrote:
> 
> Or is there a way of doing this that in a similarly compact and
> obvious way I did not yet discover?

If the list is uniform, you can flatten a single level by using `itertools.chain`:

    >>> import itertools
    >>> x = [1,2,3]
    >>> y = itertools.chain.from_iterable([[0], x])
    >>> list(y)
    [0, 1, 2, 3]
    >>> # alternatively
    ... y = list(itertools.chain([0], x))
    >>> y
    [0, 1, 2, 3]
    >>> 

I know of no "better" way to do it at the moment, apart from using slice-assignment with a *stop* bound of 0:

    >>> y = [0, 0]
    >>> y[1:0] = x
    >>> y
    [0, 1, 2, 3, 0]



From ubershmekel at gmail.com  Thu Dec 15 18:30:57 2011
From: ubershmekel at gmail.com (Yuval Greenfield)
Date: Thu, 15 Dec 2011 19:30:57 +0200
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA1FCA.4080000@2sn.net>
References: <4EEA110C.4050507@2sn.net>
	<4EEA1FCA.4080000@2sn.net>
Message-ID: <CANSw7KzVu+5jeEatcBRApff4TtLC_JodDgKi8X-cXJAv3AWS9g@mail.gmail.com>

On Thu, Dec 15, 2011 at 6:26 PM, Alexander Heger <python at 2sn.net> wrote:

> Hi,
>
> in IDL/GDL, if at have an array
> x = [1,2,3]
> ?I can simply construct a new array by
> y = [0,x]
> which would give
> [0,1,2,3]
>
> I know you can do this case with
> x[0:0] = [0]
> but obviously more complicated cases are conceivable, common for me, in
> fact
> x=[1,2,3]
> y=[5,6,7]
> z=[0,x,4,y,8]
> result
> [0,1,2,3,4,5,6,7,8]
>
> or maybe even
> z=[0,x[0:2],5]
> and so forth.
>
> On the other hand, if you have a function you can pass the list of
> arguments to another function
> def f(*args):
> ? g(*args)
> which expands the list args. ?So, would it be possible - not even
> reasonable to have something similar for list and arrays, maybe
> dictionaries, in comprehensions as well?
>
> x = [1,2,3]
> y = [0,*x]
> to obtain [0,1,2,3]
>
> or similar for lists
> x = (1,2,3)
> y = (0,*x)
> to obtain (0,1,2,3)
>
> (for dictionaries, which are not sorted, the extend function seems fine)
>
> Or is there a way of doing this that in a similarly compact and
> obvious way I did not yet discover?
>
> -Alexander
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>


This is the wrong list for such a question. Python-ideas is a place to
discuss ideas for the python language itself. You can use
stackoverflow.comin the future.

In any case, is this good enough?

>>> x = [1,2,3]
>>> y = [0] + x
>>> y
[0, 1, 2, 3]



Yuval Greenfield
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/eadacdaf/attachment.html>

From phd at phdru.name  Thu Dec 15 18:33:11 2011
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 15 Dec 2011 21:33:11 +0400
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA1FCA.4080000@2sn.net>
References: <4EEA110C.4050507@2sn.net>
 <4EEA1FCA.4080000@2sn.net>
Message-ID: <20111215173311.GA15847@iskra.aviel.ru>

On Thu, Dec 15, 2011 at 10:26:50AM -0600, Alexander Heger wrote:
> x = [1,2,3]
> y = [0,*x]
> to obtain [0,1,2,3]

y = [0] + x

> or similar for lists
> x = (1,2,3)
> y = (0,*x)
> to obtain (0,1,2,3)

y = (0,) + x

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From python at 2sn.net  Thu Dec 15 18:35:12 2011
From: python at 2sn.net (Alexander Heger)
Date: Thu, 15 Dec 2011 11:35:12 -0600
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
Message-ID: <4EEA2FD0.6020504@2sn.net>

Dear Masklinn,

thanks for your suggested solution.

I know all of these, but
1) it is not as elegant or short
2) why does unpacking not work syntactically the same as for the 
function parameters?
It seems a natural extension that appears not to have a syntactic 
conflict.  If it is not even a necessity for consistency.

So, the point is not that something like
[0,*x,0,*y,0]
can't be done in other ways, but that it can't be done in a neat way.

-Alexander

On 12/15/2011 11:27 AM, Masklinn wrote:
> On 2011-12-15, at 17:26 , Alexander Heger wrote:
>>
>> Or is there a way of doing this that in a similarly compact and
>> obvious way I did not yet discover?
>
> If the list is uniform, you can flatten a single level by using `itertools.chain`:
>
>      >>>  import itertools
>      >>>  x = [1,2,3]
>      >>>  y = itertools.chain.from_iterable([[0], x])
>      >>>  list(y)
>      [0, 1, 2, 3]
>      >>>  # alternatively
>      ... y = list(itertools.chain([0], x))
>      >>>  y
>      [0, 1, 2, 3]
>      >>>
>
> I know of no "better" way to do it at the moment, apart from using slice-assignment with a *stop* bound of 0:
>
>      >>>  y = [0, 0]
>      >>>  y[1:0] = x
>      >>>  y
>      [0, 1, 2, 3, 0]
>
>


From python at 2sn.net  Thu Dec 15 18:41:09 2011
From: python at 2sn.net (Alexander Heger)
Date: Thu, 15 Dec 2011 11:41:09 -0600
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <20111215173311.GA15847@iskra.aviel.ru>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru>
Message-ID: <4EEA3135.20107@2sn.net>

Dear Oleg,

OK, these work for the examples I listed, I had not tried that. Thanks!

I still think the element extraction would be nice, though.
what does not work your way is
x = [1,2,3]
y = (0,) + x

but
y = (0,*x)
could do ... ?

-Alexander

On 12/15/2011 11:33 AM, Oleg Broytman wrote:
> On Thu, Dec 15, 2011 at 10:26:50AM -0600, Alexander Heger wrote:
>> x = [1,2,3]
>> y = [0,*x]
>> to obtain [0,1,2,3]
>
> y = [0] + x
>
>> or similar for lists
>> x = (1,2,3)
>> y = (0,*x)
>> to obtain (0,1,2,3)
>
> y = (0,) + x
>
> Oleg.


From andre.roberge at gmail.com  Thu Dec 15 18:47:32 2011
From: andre.roberge at gmail.com (Andre Roberge)
Date: Thu, 15 Dec 2011 13:47:32 -0400
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA3135.20107@2sn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru> <4EEA3135.20107@2sn.net>
Message-ID: <CAGMu_=qS3A20UMzNO4z69Fh9fFoGeTWGSoixs8oZnpOHgA+=Rw@mail.gmail.com>

On Thu, Dec 15, 2011 at 1:41 PM, Alexander Heger <python at 2sn.net> wrote:

> Dear Oleg,
>
> OK, these work for the examples I listed, I had not tried that. Thanks!
>
> I still think the element extraction would be nice, though.
> what does not work your way is
> x = [1,2,3]
> y = (0,) + x
>
> The following works
--> x = [1,2,3]
--> y = (0,) + tuple(x)
--> y
(0, 1, 2, 3)

Explicit (i.e. tuple conversion) is better than implicit   (The Zen of
Python)
Andr?



> but
> y = (0,*x)
> could do ... ?
>
> -Alexander
>
> On 12/15/2011 11:33 AM, Oleg Broytman wrote:
>
>> On Thu, Dec 15, 2011 at 10:26:50AM -0600, Alexander Heger wrote:
>>
>>> x = [1,2,3]
>>> y = [0,*x]
>>> to obtain [0,1,2,3]
>>>
>>
>> y = [0] + x
>>
>>  or similar for lists
>>> x = (1,2,3)
>>> y = (0,*x)
>>> to obtain (0,1,2,3)
>>>
>>
>> y = (0,) + x
>>
>> Oleg.
>>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/e8d72222/attachment.html>

From fuzzyman at gmail.com  Thu Dec 15 18:48:28 2011
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 15 Dec 2011 17:48:28 +0000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA2FD0.6020504@2sn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
	<4EEA2FD0.6020504@2sn.net>
Message-ID: <CAKCKLWx9njnaiVLeee0fYh8f4oGyxr2iSkj9LWDayObRucH7eg@mail.gmail.com>

On 15 December 2011 17:35, Alexander Heger <python at 2sn.net> wrote:

> Dear Masklinn,
>
> thanks for your suggested solution.
>
> I know all of these, but
> 1) it is not as elegant or short
> 2) why does unpacking not work syntactically the same as for the function
> parameters?
> It seems a natural extension that appears not to have a syntactic
> conflict.  If it is not even a necessity for consistency.
>
> So, the point is not that something like
> [0,*x,0,*y,0]
> can't be done in other ways, but that it can't be done in a neat way.
>


I quite like that (suggested) syntax.

Michael



>
> -Alexander
>
>
> On 12/15/2011 11:27 AM, Masklinn wrote:
>
>> On 2011-12-15, at 17:26 , Alexander Heger wrote:
>>
>>>
>>> Or is there a way of doing this that in a similarly compact and
>>> obvious way I did not yet discover?
>>>
>>
>> If the list is uniform, you can flatten a single level by using
>> `itertools.chain`:
>>
>>     >>>  import itertools
>>     >>>  x = [1,2,3]
>>     >>>  y = itertools.chain.from_iterable(**[[0], x])
>>     >>>  list(y)
>>     [0, 1, 2, 3]
>>     >>>  # alternatively
>>     ... y = list(itertools.chain([0], x))
>>     >>>  y
>>     [0, 1, 2, 3]
>>     >>>
>>
>> I know of no "better" way to do it at the moment, apart from using
>> slice-assignment with a *stop* bound of 0:
>>
>>     >>>  y = [0, 0]
>>     >>>  y[1:0] = x
>>     >>>  y
>>     [0, 1, 2, 3, 0]
>>
>>
>>  ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/385ba0e9/attachment.html>

From steve at pearwood.info  Thu Dec 15 18:53:02 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 16 Dec 2011 04:53:02 +1100
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CAKCKLWx9njnaiVLeee0fYh8f4oGyxr2iSkj9LWDayObRucH7eg@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net>
	<4EEA1FCA.4080000@2sn.net>	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>	<4EEA2FD0.6020504@2sn.net>
	<CAKCKLWx9njnaiVLeee0fYh8f4oGyxr2iSkj9LWDayObRucH7eg@mail.gmail.com>
Message-ID: <4EEA33FE.8060200@pearwood.info>

Michael Foord wrote:
> On 15 December 2011 17:35, Alexander Heger <python at 2sn.net> wrote:
> 
>> Dear Masklinn,
>>
>> thanks for your suggested solution.
>>
>> I know all of these, but
>> 1) it is not as elegant or short
>> 2) why does unpacking not work syntactically the same as for the function
>> parameters?
>> It seems a natural extension that appears not to have a syntactic
>> conflict.  If it is not even a necessity for consistency.
>>
>> So, the point is not that something like
>> [0,*x,0,*y,0]
>> can't be done in other ways, but that it can't be done in a neat way.
>>
> 
> 
> I quite like that (suggested) syntax.

So do I. But I question the need for it.

+0.


-- 
Steven


From phd at phdru.name  Thu Dec 15 18:53:41 2011
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 15 Dec 2011 21:53:41 +0400
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CANSw7KzVu+5jeEatcBRApff4TtLC_JodDgKi8X-cXJAv3AWS9g@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<CANSw7KzVu+5jeEatcBRApff4TtLC_JodDgKi8X-cXJAv3AWS9g@mail.gmail.com>
Message-ID: <20111215175341.GA16314@iskra.aviel.ru>

On Thu, Dec 15, 2011 at 07:30:57PM +0200, Yuval Greenfield wrote:
> This is the wrong list for such a question. Python-ideas is a place to
> discuss ideas for the python language itself.

   IMO it's ok because the OP proposed a new feature:

> On Thu, Dec 15, 2011 at 6:26 PM, Alexander Heger <python at 2sn.net> wrote:
> > y = (0,*x)

   and we're going to discuss why it will never happens.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.


From jeanpierreda at gmail.com  Thu Dec 15 18:53:33 2011
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Thu, 15 Dec 2011 12:53:33 -0500
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA2FD0.6020504@2sn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
	<4EEA2FD0.6020504@2sn.net>
Message-ID: <CABicbJJk607PB9agXt14u80GR93yo_sYBdnuThqMmHTXCjfX+Q@mail.gmail.com>

This would behave differently than function calls! I can't call foo(a, b, *x, c)

I'm actually +1 on unpacking in literals, though.

-- Devin

On Thu, Dec 15, 2011 at 12:35 PM, Alexander Heger <python at 2sn.net> wrote:
> Dear Masklinn,
>
> thanks for your suggested solution.
>
> I know all of these, but
> 1) it is not as elegant or short
> 2) why does unpacking not work syntactically the same as for the function
> parameters?
> It seems a natural extension that appears not to have a syntactic conflict.
> ?If it is not even a necessity for consistency.
>
> So, the point is not that something like
> [0,*x,0,*y,0]
> can't be done in other ways, but that it can't be done in a neat way.
>
> -Alexander
>
>
> On 12/15/2011 11:27 AM, Masklinn wrote:
>>
>> On 2011-12-15, at 17:26 , Alexander Heger wrote:
>>>
>>>
>>> Or is there a way of doing this that in a similarly compact and
>>> obvious way I did not yet discover?
>>
>>
>> If the list is uniform, you can flatten a single level by using
>> `itertools.chain`:
>>
>> ? ? >>> ?import itertools
>> ? ? >>> ?x = [1,2,3]
>> ? ? >>> ?y = itertools.chain.from_iterable([[0], x])
>> ? ? >>> ?list(y)
>> ? ? [0, 1, 2, 3]
>> ? ? >>> ?# alternatively
>> ? ? ... y = list(itertools.chain([0], x))
>> ? ? >>> ?y
>> ? ? [0, 1, 2, 3]
>> ? ? >>>
>>
>> I know of no "better" way to do it at the moment, apart from using
>> slice-assignment with a *stop* bound of 0:
>>
>> ? ? >>> ?y = [0, 0]
>> ? ? >>> ?y[1:0] = x
>> ? ? >>> ?y
>> ? ? [0, 1, 2, 3, 0]
>>
>>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From fuzzyman at gmail.com  Thu Dec 15 19:02:24 2011
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 15 Dec 2011 18:02:24 +0000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA33FE.8060200@pearwood.info>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
	<4EEA2FD0.6020504@2sn.net>
	<CAKCKLWx9njnaiVLeee0fYh8f4oGyxr2iSkj9LWDayObRucH7eg@mail.gmail.com>
	<4EEA33FE.8060200@pearwood.info>
Message-ID: <CAKCKLWxmJ6nyVZ=nhGa0a4YHqmS1gri6Ds8Ou87_xwo5OPd+hg@mail.gmail.com>

On 15 December 2011 17:53, Steven D'Aprano <steve at pearwood.info> wrote:

> Michael Foord wrote:
>
>> On 15 December 2011 17:35, Alexander Heger <python at 2sn.net> wrote:
>>
>>  Dear Masklinn,
>>>
>>> thanks for your suggested solution.
>>>
>>> I know all of these, but
>>> 1) it is not as elegant or short
>>> 2) why does unpacking not work syntactically the same as for the function
>>> parameters?
>>> It seems a natural extension that appears not to have a syntactic
>>> conflict.  If it is not even a necessity for consistency.
>>>
>>> So, the point is not that something like
>>> [0,*x,0,*y,0]
>>> can't be done in other ways, but that it can't be done in a neat way.
>>>
>>>
>>
>> I quite like that (suggested) syntax.
>>
>
> So do I. But I question the need for it.
>
> +0.
>
>
Well, the new tuple unpacking syntax makes it easy to work with the head
and tail of a sequence:

    head, *rest = some_sequence
   *rest, tail = some_sequence

This syntax provides a corollary:

    some_sequence = (head, *rest)
    some_sequence = (*rest, tail)

Tuple unpacking like this being the equivalent of car and cdr, with the
corollary being the equivalent of cons. (Given my limited understanding of
these operations.)

Michael


>
> --
> Steven
>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/82c0633c/attachment.html>

From masklinn at masklinn.net  Thu Dec 15 19:25:41 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 15 Dec 2011 19:25:41 +0100
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA2FD0.6020504@2sn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
	<4EEA2FD0.6020504@2sn.net>
Message-ID: <82DCC16A-AF24-488D-AECB-9B4AD23FFF9E@masklinn.net>

On 2011-12-15, at 18:35 , Alexander Heger wrote:
> 
> I know all of these, but
> 1) it is not as elegant or short
Sure, I was just answering the rest of of the question
> 
> 2) why does unpacking not work syntactically the same as for the function parameters?
Because nobody's thought of it so far.

> So, the point is not that something like
> [0,*x,0,*y,0]
> can't be done in other ways, but that it can't be done in a neat way.
Well, you *did* ask:

> Or is there a way of doing this that in a similarly compact and
> obvious way I did not yet discover?


From masklinn at masklinn.net  Thu Dec 15 19:28:51 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 15 Dec 2011 19:28:51 +0100
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CAKCKLWxmJ6nyVZ=nhGa0a4YHqmS1gri6Ds8Ou87_xwo5OPd+hg@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<BA28C3D7-44AF-4362-9527-BA2444F6EFEE@masklinn.net>
	<4EEA2FD0.6020504@2sn.net>
	<CAKCKLWx9njnaiVLeee0fYh8f4oGyxr2iSkj9LWDayObRucH7eg@mail.gmail.com>
	<4EEA33FE.8060200@pearwood.info>
	<CAKCKLWxmJ6nyVZ=nhGa0a4YHqmS1gri6Ds8Ou87_xwo5OPd+hg@mail.gmail.com>
Message-ID: <B824AB7F-7D6B-48A0-BB47-431599DAA67A@masklinn.net>

On 2011-12-15, at 19:02 , Michael Foord wrote:
> Well, the new tuple unpacking syntax makes it easy to work with the head
> and tail of a sequence:
> 
>    head, *rest = some_sequence
>   *rest, tail = some_sequence
> 
> This syntax provides a corollary:
> 
>    some_sequence = (head, *rest)
>    some_sequence = (*rest, tail)
> 
> Tuple unpacking like this being the equivalent of car and cdr, with the
> corollary being the equivalent of cons. (Given my limited understanding of
> these operations.)
Even more so with Py3's unpacking supporting more than just head and tail:

    >>> a, b, *c, d, e = range(15)
    >>> a
    0
    >>> b
    1
    >>> d
    13
    >>> e
    14
    >>> c
    [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Alexander's syntax would be nicely parallel to this.

On the other hand, much as Steven I'm not sure how useful it is, especially for tuples.

From ethan at stoneleaf.us  Thu Dec 15 18:44:01 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 15 Dec 2011 09:44:01 -0800
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CANSw7KzVu+5jeEatcBRApff4TtLC_JodDgKi8X-cXJAv3AWS9g@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net>	<4EEA1FCA.4080000@2sn.net>
	<CANSw7KzVu+5jeEatcBRApff4TtLC_JodDgKi8X-cXJAv3AWS9g@mail.gmail.com>
Message-ID: <4EEA31E1.3040104@stoneleaf.us>

Yuval Greenfield wrote:
> This is the wrong list for such a question. Python-ideas is a place to 
> discuss ideas for the python language itself.

python-ideas is exactly the place to ask about syntax changes to Python.

~Ethan~


From guido at python.org  Thu Dec 15 20:07:07 2011
From: guido at python.org (Guido van Rossum)
Date: Thu, 15 Dec 2011 11:07:07 -0800
Subject: [Python-ideas] Inconsistent script/console behaviour
In-Reply-To: <CADiSq7f_UAuY7VsL9uviWFXTQm4ESN00VOy-RaQRYjr1B-zdUA@mail.gmail.com>
References: <CAPkN8x+AcLJUQEvWVc8WtR8+MecfiNRQoA-6vAcBbuX2_BM0DQ@mail.gmail.com>
	<CAP7+vJKS4WgyqRNrtWQCfVeGa0d7A2TUxM=JOhxLJGfHJDW39g@mail.gmail.com>
	<j5k46t$dc6$1@dough.gmane.org>
	<CAPkN8x+OEU9_npBfRwScgfFzShvEbFLxMwZTRq63-hLre3MYFA@mail.gmail.com>
	<CADiSq7f_UAuY7VsL9uviWFXTQm4ESN00VOy-RaQRYjr1B-zdUA@mail.gmail.com>
Message-ID: <CAP7+vJ+N0PKXODaXZGCahQ0YVJoadHMK3JcKnYeDQmnAZ9_huQ@mail.gmail.com>

On Thu, Dec 15, 2011 at 3:41 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> There are two steps to what the interactive console does:
>
> Step 1: accept a block of code.
> Step 2: compile and execute it in single statement mode (in order to
> get the result of any expressions for display)
>
> It is the *first* step that is terminated by a blank line. That step
> understands almost nothing about Python syntax - just enough to figure
> out whether or not it should be asking for a continuation line. (IIRC,
> it pretty much just does bracket matching and "if I have seen a ':'
> character, ask for continuation lines until one of them is blank".
>

Actually, in CPython at least, the first step uses a special feature of the
parser that can tell whether what has been entered so far is complete or
not. Try entering a subtle syntax error:

>>> def foo():
...   x = 1
...   y x
  File "<stdin>", line 3
    y x
      ^
SyntaxError: invalid syntax
>>>

It does treat an entirely blank line (with no comment or whitespace) as a
dedent all the way back to zero.

The problem with allowing what Anatoly asks for is that the added line
could introduce another indented block instead of just a statement.

Almost all languages with an interactive mode have some kind of syntactic
awkwardness or differences. E.g. I recently had problems with Haskell's
interactive mode, which won't let you define functions directly -- you must
use 'let'; and sqlite3 reads anything until you enter a semicolon. Trying
to mess with Python's approach is likely to make things worse in some other
way. If you think otherwise, show us the patch.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/20bdab5e/attachment.html>

From g.brandl at gmx.net  Thu Dec 15 21:24:27 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 15 Dec 2011 21:24:27 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <jcdl1o$hma$1@dough.gmane.org>

On 12/13/2011 03:44 AM, Stephen J. Turnbull wrote:
> Greg Ewing writes:
>  > Masklinn wrote:
>  > 
>  > > FWIW, Haskell does not have a literal singleton (the standard defines 
>  > > "unit" `()` and 2-tuple through 15-tuple)
>  > 
>  > That's because, due to its static typing, there is no
>  > reason you would ever need to use a 1-tuple rather than
>  > a bare value. We're not that lucky in Python, though.
> 
> I think you have misstated your point?  That's not due to static
> typing, that's because you may *always* identify 1-factor products
> with the only factor, and Haskell made a deliberate decision to
> consistently represent the isomorphism class by the factor rather than
> the product.

Well, I would say the reason is that the type "tuple of any length" does
not exist in Haskell.  So there's no way you will have to pass a 1-tuple
to a function that operates on tuples only.

But of course, if we all used tuples as tuples only, we wouldn't have to do
that either.  It's only because we use tuples as sequences every so often.

Georg



From ned at nedbatchelder.com  Thu Dec 15 21:42:42 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 15:42:42 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <jcdl1o$hma$1@dough.gmane.org>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org>
Message-ID: <4EEA5BC2.4070402@nedbatchelder.com>

On 12/15/2011 3:24 PM, Georg Brandl wrote:
> On 12/13/2011 03:44 AM, Stephen J. Turnbull wrote:
>> Greg Ewing writes:
>>   >  Masklinn wrote:
>>   >
>>   >  >  FWIW, Haskell does not have a literal singleton (the standard defines
>>   >  >  "unit" `()` and 2-tuple through 15-tuple)
>>   >
>>   >  That's because, due to its static typing, there is no
>>   >  reason you would ever need to use a 1-tuple rather than
>>   >  a bare value. We're not that lucky in Python, though.
>>
>> I think you have misstated your point?  That's not due to static
>> typing, that's because you may *always* identify 1-factor products
>> with the only factor, and Haskell made a deliberate decision to
>> consistently represent the isomorphism class by the factor rather than
>> the product.
> Well, I would say the reason is that the type "tuple of any length" does
> not exist in Haskell.  So there's no way you will have to pass a 1-tuple
> to a function that operates on tuples only.
>
> But of course, if we all used tuples as tuples only, we wouldn't have to do
> that either.  It's only because we use tuples as sequences every so often.
This is another place where Python is inconsistent.  We're told, "lists 
are for homogenous sequences of varying length, like a C array; tuples 
are for heterogenous aggregations of known length, like a C struct."   
Then we define a function foo(*args), and Python gives us a tuple!  :-(

--Ned.
> Georg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From g.brandl at gmx.net  Thu Dec 15 21:51:01 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 15 Dec 2011 21:51:01 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA5BC2.4070402@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
Message-ID: <jcdmjj$32g$1@dough.gmane.org>

On 12/15/2011 09:42 PM, Ned Batchelder wrote:
> On 12/15/2011 3:24 PM, Georg Brandl wrote:
>> On 12/13/2011 03:44 AM, Stephen J. Turnbull wrote:
>>> Greg Ewing writes:
>>>   >  Masklinn wrote:
>>>   >
>>>   >  >  FWIW, Haskell does not have a literal singleton (the standard defines
>>>   >  >  "unit" `()` and 2-tuple through 15-tuple)
>>>   >
>>>   >  That's because, due to its static typing, there is no
>>>   >  reason you would ever need to use a 1-tuple rather than
>>>   >  a bare value. We're not that lucky in Python, though.
>>>
>>> I think you have misstated your point?  That's not due to static
>>> typing, that's because you may *always* identify 1-factor products
>>> with the only factor, and Haskell made a deliberate decision to
>>> consistently represent the isomorphism class by the factor rather than
>>> the product.
>> Well, I would say the reason is that the type "tuple of any length" does
>> not exist in Haskell.  So there's no way you will have to pass a 1-tuple
>> to a function that operates on tuples only.
>>
>> But of course, if we all used tuples as tuples only, we wouldn't have to do
>> that either.  It's only because we use tuples as sequences every so often.
> This is another place where Python is inconsistent.  We're told, "lists 
> are for homogenous sequences of varying length, like a C array; tuples 
> are for heterogenous aggregations of known length, like a C struct."   
> Then we define a function foo(*args), and Python gives us a tuple!  :-(

Yep. To be consistent, we'd need an "immutable list" type...
another thing that Haskell has no need for :)

Georg



From solipsis at pitrou.net  Thu Dec 15 21:51:47 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 15 Dec 2011 21:51:47 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
Message-ID: <20111215215147.22f41c80@pitrou.net>

On Thu, 15 Dec 2011 15:42:42 -0500
Ned Batchelder <ned at nedbatchelder.com>
wrote:
> On 12/15/2011 3:24 PM, Georg Brandl wrote:
> > On 12/13/2011 03:44 AM, Stephen J. Turnbull wrote:
> >> Greg Ewing writes:
> >>   >  Masklinn wrote:
> >>   >
> >>   >  >  FWIW, Haskell does not have a literal singleton (the standard defines
> >>   >  >  "unit" `()` and 2-tuple through 15-tuple)
> >>   >
> >>   >  That's because, due to its static typing, there is no
> >>   >  reason you would ever need to use a 1-tuple rather than
> >>   >  a bare value. We're not that lucky in Python, though.
> >>
> >> I think you have misstated your point?  That's not due to static
> >> typing, that's because you may *always* identify 1-factor products
> >> with the only factor, and Haskell made a deliberate decision to
> >> consistently represent the isomorphism class by the factor rather than
> >> the product.
> > Well, I would say the reason is that the type "tuple of any length" does
> > not exist in Haskell.  So there's no way you will have to pass a 1-tuple
> > to a function that operates on tuples only.
> >
> > But of course, if we all used tuples as tuples only, we wouldn't have to do
> > that either.  It's only because we use tuples as sequences every so often.
> This is another place where Python is inconsistent.  We're told, "lists 
> are for homogenous sequences of varying length, like a C array; tuples 
> are for heterogenous aggregations of known length, like a C struct."   
> Then we define a function foo(*args), and Python gives us a tuple!  :-(

How is it inconsistent? Function signatures generally have a fixed (or
mostly fixed) number of heterogenous arguments.

Regards

Antoine.




From timothy.c.delaney at gmail.com  Thu Dec 15 21:59:26 2011
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Fri, 16 Dec 2011 07:59:26 +1100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA5BC2.4070402@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
Message-ID: <CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>

On 16 December 2011 07:42, Ned Batchelder <ned at nedbatchelder.com> wrote:

> This is another place where Python is inconsistent.  We're told, "lists
> are for homogenous sequences of varying length, like a C array; tuples are
> for heterogenous aggregations of known length, like a C struct."   Then we
> define a function foo(*args), and Python gives us a tuple!  :-(
>

How is that inconsistent? At the point where the tuple is constructed, it
has a known length. And it's definitely a heterogenous aggregation.

I think where you're getting confused is that you're thinking of a *single*
struct definition for every tuple. But the concept you should have is that
each tuple has its own struct definition. And with functions, the structure
is defined at function call time.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111216/622cb362/attachment.html>

From ned at nedbatchelder.com  Thu Dec 15 22:00:03 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 16:00:03 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <20111215215147.22f41c80@pitrou.net>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<20111215215147.22f41c80@pitrou.net>
Message-ID: <4EEA5FD3.2070603@nedbatchelder.com>

On 12/15/2011 3:51 PM, Antoine Pitrou wrote:
> On Thu, 15 Dec 2011 15:42:42 -0500
> Ned Batchelder<ned at nedbatchelder.com>
> wrote:
>> On 12/15/2011 3:24 PM, Georg Brandl wrote:
>>> On 12/13/2011 03:44 AM, Stephen J. Turnbull wrote:
>>>> Greg Ewing writes:
>>>>    >   Masklinn wrote:
>>>>    >
>>>>    >   >   FWIW, Haskell does not have a literal singleton (the standard defines
>>>>    >   >   "unit" `()` and 2-tuple through 15-tuple)
>>>>    >
>>>>    >   That's because, due to its static typing, there is no
>>>>    >   reason you would ever need to use a 1-tuple rather than
>>>>    >   a bare value. We're not that lucky in Python, though.
>>>>
>>>> I think you have misstated your point?  That's not due to static
>>>> typing, that's because you may *always* identify 1-factor products
>>>> with the only factor, and Haskell made a deliberate decision to
>>>> consistently represent the isomorphism class by the factor rather than
>>>> the product.
>>> Well, I would say the reason is that the type "tuple of any length" does
>>> not exist in Haskell.  So there's no way you will have to pass a 1-tuple
>>> to a function that operates on tuples only.
>>>
>>> But of course, if we all used tuples as tuples only, we wouldn't have to do
>>> that either.  It's only because we use tuples as sequences every so often.
>> This is another place where Python is inconsistent.  We're told, "lists
>> are for homogenous sequences of varying length, like a C array; tuples
>> are for heterogenous aggregations of known length, like a C struct."
>> Then we define a function foo(*args), and Python gives us a tuple!  :-(
> How is it inconsistent? Function signatures generally have a fixed (or
> mostly fixed) number of heterogenous arguments.
What?  The whole point of the "def foo(*args)" syntax is so a function 
can take an unknown-length list of arguments, which will be treated 
uniformly.

--Ned.
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From masklinn at masklinn.net  Thu Dec 15 22:05:26 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 15 Dec 2011 22:05:26 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <20111215215147.22f41c80@pitrou.net>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<20111215215147.22f41c80@pitrou.net>
Message-ID: <354A8605-6CDF-40C5-AEA9-BFF7FF0DBB05@masklinn.net>

On 2011-12-15, at 21:51 , Antoine Pitrou wrote:
> On Thu, 15 Dec 2011 15:42:42 -0500
> Ned Batchelder <ned at nedbatchelder.com>
> wrote:
>> This is another place where Python is inconsistent.  We're told, "lists 
>> are for homogenous sequences of varying length, like a C array; tuples 
>> are for heterogenous aggregations of known length, like a C struct."   
>> Then we define a function foo(*args), and Python gives us a tuple!  :-(
> 
> How is it inconsistent? Function signatures generally have a fixed (or
> mostly fixed) number of heterogenous arguments.
But *args is mostly dedicated to an arbitrary (non-fixed) number of homogenous arguments. In statically typed languages with varargs, they're generally represented as an array-type collection.

It's either that or a "rest of that crap" catchall used to forward arguments without caring much for what they are.

From masklinn at masklinn.net  Thu Dec 15 22:06:43 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 15 Dec 2011 22:06:43 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA5FD3.2070603@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<20111215215147.22f41c80@pitrou.net>
	<4EEA5FD3.2070603@nedbatchelder.com>
Message-ID: <A1FB1E82-B6CD-4291-A27F-3B4C2CF48EA9@masklinn.net>

On 2011-12-15, at 22:00 , Ned Batchelder wrote:
> 
> What?  The whole point of the "def foo(*args)" syntax is so a function can take an unknown-length list of arguments, which will be treated uniformly.
That's not *entirely* true, it's also used for proxy functions, in order to just pass a bunch of arguments (positional or keywords) along to the next level without having to unpack and repack them one by one.

From ned at nedbatchelder.com  Thu Dec 15 22:06:59 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 16:06:59 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA5FD3.1020907@stoneleaf.us>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>	<20111210144230.GA16801@iskra.aviel.ru>	<4EE5FAEC.1080907@nedbatchelder.com>	<20111212140758.301e7431@pitrou.net>	<4EE5FE81.3060304@nedbatchelder.com>	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>	<4EE67047.8050209@canterbury.ac.nz>	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com>
	<4EEA5FD3.1020907@stoneleaf.us>
Message-ID: <4EEA6173.9040402@nedbatchelder.com>

On 12/15/2011 4:00 PM, Ethan Furman wrote:
> Ned Batchelder wrote:
>> This is another place where Python is inconsistent.  We're told, 
>> "lists are for homogenous sequences of varying length, like a C 
>> array; tuples are for heterogenous aggregations of known length, like 
>> a C struct."   Then we define a function foo(*args), and Python gives 
>> us a tuple!  :-(
>
> Where is that in the docs?  Sounds like a patch is needed:
>
> "lists are for sequences where items need to be added/removed; tuples 
> are for sequences/aggregations where items will not be added/removed 
> once the tuple is created"
>
I don't know if it appears in the docs, but I hear it all the time, and 
Guido has said it 
(http://mail.python.org/pipermail/python-dev/2003-March/033964.html):

    Tuples are for heterogeneous data, list are for homogeneous data.
    Tuples are *not* read-only lists.

I don't want to get too far off the original point, which was: Python 
isn't as simple as we'd like to thing, and even smart beginners can be 
tripped up by things we've provided to them.

--Ned.

> ~Ethan~
>
> PS
> As Antoine noted, a tuple for 'args' is appropriate, as once args is 
> created at function call time, we won't be adding or removing from it.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/427b6443/attachment.html>

From ned at nedbatchelder.com  Thu Dec 15 22:09:31 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 16:09:31 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
Message-ID: <4EEA620B.1000300@nedbatchelder.com>

On 12/15/2011 3:59 PM, Tim Delaney wrote:
> On 16 December 2011 07:42, Ned Batchelder <ned at nedbatchelder.com 
> <mailto:ned at nedbatchelder.com>> wrote:
>
>     This is another place where Python is inconsistent.  We're told,
>     "lists are for homogenous sequences of varying length, like a C
>     array; tuples are for heterogenous aggregations of known length,
>     like a C struct."   Then we define a function foo(*args), and
>     Python gives us a tuple!  :-(
>
>
> How is that inconsistent? At the point where the tuple is constructed, 
> it has a known length. And it's definitely a heterogenous aggregation.
>
> I think where you're getting confused is that you're thinking of a 
> *single* struct definition for every tuple. But the concept you should 
> have is that each tuple has its own struct definition. And with 
> functions, the structure is defined at function call time.
>
Tim, this seems misguided to me.  Finish that foo function definition: 
it will *have* to have  "for a in args:"   Since I don't know the length 
of args when I write the function, I have to treat it as an unknown 
length.  What good is a "structure" that changes length and definition 
with every instance?  I think you're trying too hard to fit the reality 
into the theory.

--Ned.

> Tim Delaney
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/ba54281c/attachment.html>

From ned at nedbatchelder.com  Thu Dec 15 22:11:10 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 16:11:10 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <A1FB1E82-B6CD-4291-A27F-3B4C2CF48EA9@masklinn.net>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<20111215215147.22f41c80@pitrou.net>
	<4EEA5FD3.2070603@nedbatchelder.com>
	<A1FB1E82-B6CD-4291-A27F-3B4C2CF48EA9@masklinn.net>
Message-ID: <4EEA626E.8040101@nedbatchelder.com>

On 12/15/2011 4:06 PM, Masklinn wrote:
> On 2011-12-15, at 22:00 , Ned Batchelder wrote:
>> What?  The whole point of the "def foo(*args)" syntax is so a function can take an unknown-length list of arguments, which will be treated uniformly.
> That's not *entirely* true, it's also used for proxy functions, in order to just pass a bunch of arguments (positional or keywords) along to the next level without having to unpack and repack them one by one.
We are in agreement: I don't know the length, and I will treat them 
uniformly (pass them all through to the next function).

--Ned.


From ethan at stoneleaf.us  Thu Dec 15 22:00:03 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 15 Dec 2011 13:00:03 -0800
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA5BC2.4070402@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>	<20111210144230.GA16801@iskra.aviel.ru>	<4EE5FAEC.1080907@nedbatchelder.com>	<20111212140758.301e7431@pitrou.net>	<4EE5FE81.3060304@nedbatchelder.com>	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>	<4EE67047.8050209@canterbury.ac.nz>	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com>
Message-ID: <4EEA5FD3.1020907@stoneleaf.us>

Ned Batchelder wrote:
> This is another place where Python is inconsistent.  We're told, "lists 
> are for homogenous sequences of varying length, like a C array; tuples 
> are for heterogenous aggregations of known length, like a C struct."   
> Then we define a function foo(*args), and Python gives us a tuple!  :-(

Where is that in the docs?  Sounds like a patch is needed:

"lists are for sequences where items need to be added/removed; tuples 
are for sequences/aggregations where items will not be added/removed 
once the tuple is created"

~Ethan~

PS
As Antoine noted, a tuple for 'args' is appropriate, as once args is 
created at function call time, we won't be adding or removing from it.


From ncoghlan at gmail.com  Thu Dec 15 22:52:04 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 16 Dec 2011 07:52:04 +1000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA31E1.3040104@stoneleaf.us>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<CANSw7KzVu+5jeEatcBRApff4TtLC_JodDgKi8X-cXJAv3AWS9g@mail.gmail.com>
	<4EEA31E1.3040104@stoneleaf.us>
Message-ID: <CADiSq7dbPW9x2N+wwTF_OZsLkeq9WuyLk+q-+3KFU512mZnpkA@mail.gmail.com>

This won't happen because it is a lot of work for no real gain over the
more explicit "tuple(x)"

--
Nick Coghlan (via Gmail on Android, so likely to be more terse than usual)
On Dec 16, 2011 4:45 AM, "Ethan Furman" <ethan at stoneleaf.us> wrote:

> Yuval Greenfield wrote:
>
>> This is the wrong list for such a question. Python-ideas is a place to
>> discuss ideas for the python language itself.
>>
>
> python-ideas is exactly the place to ask about syntax changes to Python.
>
> ~Ethan~
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111216/5a70c308/attachment.html>

From ethan at stoneleaf.us  Thu Dec 15 22:24:26 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 15 Dec 2011 13:24:26 -0800
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA620B.1000300@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>	<20111210144230.GA16801@iskra.aviel.ru>	<4EE5FAEC.1080907@nedbatchelder.com>	<20111212140758.301e7431@pitrou.net>	<4EE5FE81.3060304@nedbatchelder.com>	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>	<4EE67047.8050209@canterbury.ac.nz>	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com>	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
	<4EEA620B.1000300@nedbatchelder.com>
Message-ID: <4EEA658A.3010509@stoneleaf.us>

Ned Batchelder wrote:
>   On 12/15/2011 3:59 PM, Tim Delaney wrote:
>> On 16 December 2011 07:42, Ned Batchelder wrote:
>>
>>     This is another place where Python is inconsistent.  We're told,
>>     "lists are for homogenous sequences of varying length, like a C
>>     array; tuples are for heterogenous aggregations of known length,
>>     like a C struct."   Then we define a function foo(*args), and
>>     Python gives us a tuple!  :-(
>>
>>
>> How is that inconsistent? At the point where the tuple is constructed, 
>> it has a known length. And it's definitely a heterogenous aggregation.
>>
>> I think where you're getting confused is that you're thinking of a 
>> *single* struct definition for every tuple. But the concept you should 
>> have is that each tuple has its own struct definition. And with 
>> functions, the structure is defined at function call time.
>>
> Tim, this seems misguided to me.  Finish that foo function definition: 
> it will *have* to have  "for a in args:"   Since I don't know the length 
> of args when I write the function, I have to treat it as an unknown 
> length.  What good is a "structure" that changes length and definition 
> with every instance?  I think you're trying too hard to fit the reality 
> into the theory.

Python is a dynamic language -- why can't it have dynamic structs?

~Ethan~


From fuzzyman at gmail.com  Thu Dec 15 22:53:56 2011
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 15 Dec 2011 21:53:56 +0000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <4EEA3135.20107@2sn.net>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru> <4EEA3135.20107@2sn.net>
Message-ID: <CAKCKLWwKS65FWnRNa8-Z5EoZW72WguwsEzjLPAbfFP=o0mJjhg@mail.gmail.com>

On 15 December 2011 17:41, Alexander Heger <python at 2sn.net> wrote:

> Dear Oleg,
>
> OK, these work for the examples I listed, I had not tried that. Thanks!
>
> I still think the element extraction would be nice, though.
> what does not work your way is
> x = [1,2,3]
> y = (0,) + x
>
> but
> y = (0,*x)
> could do ... ?
>


In fact there's already an issue for this, with patch:

http://bugs.python.org/issue2292

Actually it goes a lot further than just the syntax you suggest... It
extends tuple unpacking into a few more places as well.

All the best,

Michael Foord


>
> -Alexander
>
>
> On 12/15/2011 11:33 AM, Oleg Broytman wrote:
>
>> On Thu, Dec 15, 2011 at 10:26:50AM -0600, Alexander Heger wrote:
>>
>>> x = [1,2,3]
>>> y = [0,*x]
>>> to obtain [0,1,2,3]
>>>
>>
>> y = [0] + x
>>
>>  or similar for lists
>>> x = (1,2,3)
>>> y = (0,*x)
>>> to obtain (0,1,2,3)
>>>
>>
>> y = (0,) + x
>>
>> Oleg.
>>
> ______________________________**_________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/db677c56/attachment.html>

From ned at nedbatchelder.com  Thu Dec 15 22:54:16 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 16:54:16 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA658A.3010509@stoneleaf.us>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>	<20111210144230.GA16801@iskra.aviel.ru>	<4EE5FAEC.1080907@nedbatchelder.com>	<20111212140758.301e7431@pitrou.net>	<4EE5FE81.3060304@nedbatchelder.com>	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>	<4EE67047.8050209@canterbury.ac.nz>	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com>	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
	<4EEA620B.1000300@nedbatchelder.com>
	<4EEA658A.3010509@stoneleaf.us>
Message-ID: <4EEA6C88.1030203@nedbatchelder.com>

On 12/15/2011 4:24 PM, Ethan Furman wrote:
> Ned Batchelder wrote:
>>   On 12/15/2011 3:59 PM, Tim Delaney wrote:
>>> On 16 December 2011 07:42, Ned Batchelder wrote:
>>>
>>>     This is another place where Python is inconsistent.  We're told,
>>>     "lists are for homogenous sequences of varying length, like a C
>>>     array; tuples are for heterogenous aggregations of known length,
>>>     like a C struct."   Then we define a function foo(*args), and
>>>     Python gives us a tuple!  :-(
>>>
>>>
>>> How is that inconsistent? At the point where the tuple is 
>>> constructed, it has a known length. And it's definitely a 
>>> heterogenous aggregation.
>>>
>>> I think where you're getting confused is that you're thinking of a 
>>> *single* struct definition for every tuple. But the concept you 
>>> should have is that each tuple has its own struct definition. And 
>>> with functions, the structure is defined at function call time.
>>>
>> Tim, this seems misguided to me.  Finish that foo function 
>> definition: it will *have* to have  "for a in args:"   Since I don't 
>> know the length of args when I write the function, I have to treat it 
>> as an unknown length.  What good is a "structure" that changes length 
>> and definition with every instance?  I think you're trying too hard 
>> to fit the reality into the theory.
>
> Python is a dynamic language -- why can't it have dynamic structs?
>
I honestly don't know how to interpret this.  In what way is it a 
"struct" if I have no idea how many fields it has, or how they might 
differ from each other?   I think we've lost the utility of this discussion.

--Ned.
> ~Ethan~
>


From masklinn at masklinn.net  Thu Dec 15 22:55:54 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 15 Dec 2011 22:55:54 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA626E.8040101@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<20111215215147.22f41c80@pitrou.net>
	<4EEA5FD3.2070603@nedbatchelder.com>
	<A1FB1E82-B6CD-4291-A27F-3B4C2CF48EA9@masklinn.net>
	<4EEA626E.8040101@nedbatchelder.com>
Message-ID: <72413090-E197-40A6-A49E-D9D56B38AFAA@masklinn.net>

On 15 d?c. 2011, at 22:11, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On 12/15/2011 4:06 PM, Masklinn wrote:
>> On 2011-12-15, at 22:00 , Ned Batchelder wrote:
>>> What?  The whole point of the "def foo(*args)" syntax is so a function can take an unknown-length list of arguments, which will be treated uniformly.
>> That's not *entirely* true, it's also used for proxy functions, in order to just pass a bunch of arguments (positional or keywords) along to the next level without having to unpack and repack them one by one.
> We are in agreement: I don't know the length, and I will treat them uniformly (pass them all through to the next function).
Ah yes, you're right, I had not considered this use case under that angle. 

From grosser.meister.morti at gmx.net  Thu Dec 15 23:14:45 2011
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Thu, 15 Dec 2011 23:14:45 +0100
Subject: [Python-ideas] Transportable indent level markers.   /// ; ; ;
 \\\
In-Reply-To: <1323727633.29879.181.camel@Gutsy>
References: <CADiSq7dSMHFG4MQeaMzJhjvWAGQxVvRvvdON=VeRQz2U+DsqRg@mail.gmail.com>
	<20111210161608.13313643@bhuda.mired.org>
	<4EE40DA7.4020702@pearwood.info> <4EE58F56.10507@canterbury.ac.nz>
	<CAGu0AnvX_LiocE8EyA+Csd-U8eqJm25WStOpuCN+E1GKBQscYQ@mail.gmail.com>
	<CADiSq7eLBpOoqJu7yqAZqgBy7JDaHuQdJsKS0-f=JXuoJ0J2jQ@mail.gmail.com>
	<1323727633.29879.181.camel@Gutsy>
Message-ID: <4EEA7155.5090405@gmx.net>

Need a new why of indentation?

(V) (;,,;) (V)

Why not Zoidberg??


Seriously though, I don't think something like this will be ever accepted. It totally goes against 
pythons idea behind indentation.

On 12/12/2011 11:07 PM, Ron Adam wrote:
>
> While typing this reply to the "Alternate Suite Delineation Syntax For
> Python?" thread.  I realized that having things  all on a single line
> isn't important or needed!!!  Yes, it's not needed for templates, or
> code generation.  So this is an alternate to the alternate.
>
> What is needed is a uniform way to represent indenting that is easy to
> transport to (ie paste into) code with different indent levels.
>
>
> INDENT LEVEL MARKERS:
> ----------------------
> Offer a new way to spell<indent+1>,<indent+0>, and<indent-1>.
>
> To make that work, we need to use unique character sequences that aren't
> already used.  After some thought, I came up with these indent-markers.
>
>      ///   # indent level +1
>      ;;;   # indent level +0
>      \\\   # indent level -1
>
> These are unique enough to be easy to see, and the tripled versions of
> them aren't really that bad as we already have triple quotes, and they
> would be less than the white space they are replacing in most cases.
>
> So this isn't brackets, or braces. It would be just an alternate
> spelling in already valid python. It should not effect the parser,
> grammar, or interpreter in any way if it is done in pythons
> tokenizer.  :-)
>
>
> Reusing some previous examples.
>
> #-----------------------
>
>>     def acc(n) {: s=n; return {: def (i) {: nonlocal s; s += i; return
>>   s}}}
>
> #Standard python...
>
> def acc(n):
>      s=n
>      def add(i):
>          nonlocal s
>          s += i
>          return s
>      return add
>
>
> #Single line version...
>
>    def acc(n): /// s=n ;;; def add(i): /// nonlocal s \
>    ;;; s+=i ;;; return s \\\ return add
>
> (The '\' character should still work.)
>
>
> #Transportable indent level version...
>
> This could also be valid... and could be transportable to any other
> indent level without having to edit it.
>
> (So would the above single line version by adding a ';;;' to the
> beginning.)
>
> ;;; def acc(n):     # indent +0
> /// s = n           # indent +1
> ;;; def add(i):     # indent +0
> /// nonlocal s      # indent +1
> ;;; s += i          # indent +0
> ;;; return s        # indent +0
> \\\ return add      # indent -1
>
> It has the nice properties of being uniform, and being able to be
> inserted easily into any existing code block without having to adjust
> the indent levels.
>
> I believe that these properties are more important than being able to
> put things on a single line and could be a very useful in code
> generating programs. ;-)
>
>
> It may be easy to implement.
>
> This could probably all be done in the tokinze.c get_tok() function,
> where it detects and converts white space to indention levels.  Nothing
> else in python would need to be changed.  The tokenizer already detects
> line continuations and different line endings, so this wouldn't be that
> out of place.
>
> Cheers,
>     Ron
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



From ncoghlan at gmail.com  Fri Dec 16 00:06:01 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 16 Dec 2011 09:06:01 +1000
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA6173.9040402@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<4EEA5FD3.1020907@stoneleaf.us>
	<4EEA6173.9040402@nedbatchelder.com>
Message-ID: <CADiSq7d-S42HyZGXD+Me-rMjHszQ6sApqE73gQ0-exPGBgVTzA@mail.gmail.com>

On Fri, Dec 16, 2011 at 7:06 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> Where is that in the docs?? Sounds like a patch is needed:
>
> "lists are for sequences where items need to be added/removed; tuples are
> for sequences/aggregations where items will not be added/removed once the
> tuple is created"
>
> I don't know if it appears in the docs, but I hear it all the time, and
> Guido has said it
> (http://mail.python.org/pipermail/python-dev/2003-March/033964.html):
>
> Tuples are for heterogeneous data, list are for homogeneous data.
> Tuples are *not* read-only lists.
>
> I don't want to get too far off the original point, which was: Python isn't
> as simple as we'd like to thing, and even smart beginners can be tripped up
> by things we've provided to them.

Guido has softened his stance on that point over the years (IIRC, the
question came up explicitly in the discussion over making tuple()
fully conform to the Sequence ABC - you can guess the outcome from the
fact that tuple these days in fact *does* fully conform to that ABC,
including the previously missing index() and count() methods).

So tuples have two valid use cases: as read-only arbitrary-length
sequences of homogeneous data and as fixed-length sequences of
heterogeneous data.

These days, the latter use case is often better served by creating a
collections.namedtuple() definition rather than using a bare tuple
directly.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From p.f.moore at gmail.com  Fri Dec 16 00:13:01 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 15 Dec 2011 23:13:01 +0000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CAKCKLWwKS65FWnRNa8-Z5EoZW72WguwsEzjLPAbfFP=o0mJjhg@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru> <4EEA3135.20107@2sn.net>
	<CAKCKLWwKS65FWnRNa8-Z5EoZW72WguwsEzjLPAbfFP=o0mJjhg@mail.gmail.com>
Message-ID: <CACac1F_fTq4ub0-Wt1eogghytuXidpUafAN6LOMBuziGv6pytg@mail.gmail.com>

On 15 December 2011 21:53, Michael Foord <fuzzyman at gmail.com> wrote:
> In fact there's already an issue for this, with patch:
>
> http://bugs.python.org/issue2292
>
> Actually it goes a lot further than just the syntax you suggest... It
> extends tuple unpacking into a few more places as well.

Nice. It seemed to have general support, plus BDFL approval. What's
needed to get it moving for 3.3? The issue mentions a PEP, but I don't
know if that was just when it was in the "Python 3000" timeframe.

Paul.


From fuzzyman at gmail.com  Fri Dec 16 00:28:37 2011
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 15 Dec 2011 23:28:37 +0000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CACac1F_fTq4ub0-Wt1eogghytuXidpUafAN6LOMBuziGv6pytg@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru> <4EEA3135.20107@2sn.net>
	<CAKCKLWwKS65FWnRNa8-Z5EoZW72WguwsEzjLPAbfFP=o0mJjhg@mail.gmail.com>
	<CACac1F_fTq4ub0-Wt1eogghytuXidpUafAN6LOMBuziGv6pytg@mail.gmail.com>
Message-ID: <CAKCKLWzeuVqtPPDfJObTU2gbZGeuf+M=eKYTamjkrgvn+0VYAw@mail.gmail.com>

On 15 December 2011 23:13, Paul Moore <p.f.moore at gmail.com> wrote:

> On 15 December 2011 21:53, Michael Foord <fuzzyman at gmail.com> wrote:
> > In fact there's already an issue for this, with patch:
> >
> > http://bugs.python.org/issue2292
> >
> > Actually it goes a lot further than just the syntax you suggest... It
> > extends tuple unpacking into a few more places as well.
>
> Nice. It seemed to have general support, plus BDFL approval. What's
> needed to get it moving for 3.3? The issue mentions a PEP, but I don't
> know if that was just when it was in the "Python 3000" timeframe.
>


Guido was -0 in the email linked to from the issue. Not sure if that counts
as approval...

Michael


>
> Paul.
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/4c3eac79/attachment.html>

From guido at python.org  Fri Dec 16 00:33:24 2011
From: guido at python.org (Guido van Rossum)
Date: Thu, 15 Dec 2011 15:33:24 -0800
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CAKCKLWzeuVqtPPDfJObTU2gbZGeuf+M=eKYTamjkrgvn+0VYAw@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru> <4EEA3135.20107@2sn.net>
	<CAKCKLWwKS65FWnRNa8-Z5EoZW72WguwsEzjLPAbfFP=o0mJjhg@mail.gmail.com>
	<CACac1F_fTq4ub0-Wt1eogghytuXidpUafAN6LOMBuziGv6pytg@mail.gmail.com>
	<CAKCKLWzeuVqtPPDfJObTU2gbZGeuf+M=eKYTamjkrgvn+0VYAw@mail.gmail.com>
Message-ID: <CAP7+vJKqS7HqrC3b45XuDduCj51kCa5soU99dJLkTgMGq2VwRA@mail.gmail.com>

On Thu, Dec 15, 2011 at 3:28 PM, Michael Foord <fuzzyman at gmail.com> wrote:

>
>
> On 15 December 2011 23:13, Paul Moore <p.f.moore at gmail.com> wrote:
>
>> On 15 December 2011 21:53, Michael Foord <fuzzyman at gmail.com> wrote:
>> > In fact there's already an issue for this, with patch:
>> >
>> > http://bugs.python.org/issue2292
>> >
>> > Actually it goes a lot further than just the syntax you suggest... It
>> > extends tuple unpacking into a few more places as well.
>>
>> Nice. It seemed to have general support, plus BDFL approval. What's
>> needed to get it moving for 3.3? The issue mentions a PEP, but I don't
>> know if that was just when it was in the "Python 3000" timeframe.
>>
>
>
> Guido was -0 in the email linked to from the issue. Not sure if that
> counts as approval...
>
> Michael
>

I think that -0 was contextual (too many moving parts for the original Py3k
release). Today I am +1.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/bd93dcc2/attachment.html>

From ncoghlan at gmail.com  Fri Dec 16 00:48:43 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 16 Dec 2011 09:48:43 +1000
Subject: [Python-ideas] list / array comprehensions extension
In-Reply-To: <CAP7+vJKqS7HqrC3b45XuDduCj51kCa5soU99dJLkTgMGq2VwRA@mail.gmail.com>
References: <4EEA110C.4050507@2sn.net> <4EEA1FCA.4080000@2sn.net>
	<20111215173311.GA15847@iskra.aviel.ru> <4EEA3135.20107@2sn.net>
	<CAKCKLWwKS65FWnRNa8-Z5EoZW72WguwsEzjLPAbfFP=o0mJjhg@mail.gmail.com>
	<CACac1F_fTq4ub0-Wt1eogghytuXidpUafAN6LOMBuziGv6pytg@mail.gmail.com>
	<CAKCKLWzeuVqtPPDfJObTU2gbZGeuf+M=eKYTamjkrgvn+0VYAw@mail.gmail.com>
	<CAP7+vJKqS7HqrC3b45XuDduCj51kCa5soU99dJLkTgMGq2VwRA@mail.gmail.com>
Message-ID: <CADiSq7cBkj6+pEKWZMCjGx6_XtqGdJRg4DkK0Bnox7hxG-_pxQ@mail.gmail.com>

On Fri, Dec 16, 2011 at 9:33 AM, Guido van Rossum <guido at python.org> wrote:
> I think that -0 was contextual (too many moving parts for the original Py3k
> release). Today I am +1.

I stand corrected :)

(And after skimming #2292, I'm also +1 on the course of action
outlined there - the idea is sound in principle, but needs a PEP to
thrash out all the niggling little details)

Cheers,
Nick.


-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From tjreedy at udel.edu  Fri Dec 16 02:16:40 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 15 Dec 2011 20:16:40 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA5BC2.4070402@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
Message-ID: <jce65v$c4p$1@dough.gmane.org>

On 12/15/2011 3:42 PM, Ned Batchelder wrote:

> This is another place where Python is inconsistent. We're told, "lists
> are for homogenous sequences of varying length, like a C array; tuples
> are for heterogenous aggregations of known length, like a C struct."

I have not been told that for several years, and I am pretty sure you 
will not find any such thing in the current docs. I consider it pretty 
much obsolete, as the differences that flowed from that idea are gone. 
In Python 3, tuples have all the non-mutating sequence methods that list 
does. The situation was much different in 1.4.

-- 
Terry Jan Reedy



From guido at python.org  Fri Dec 16 02:39:04 2011
From: guido at python.org (Guido van Rossum)
Date: Thu, 15 Dec 2011 17:39:04 -0800
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <jce65v$c4p$1@dough.gmane.org>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com> <jce65v$c4p$1@dough.gmane.org>
Message-ID: <CAP7+vJJyjvxkKYSbzZ3AAD=cC31n=ha+wwpfybKeLtRRXCzuFQ@mail.gmail.com>

On Thu, Dec 15, 2011 at 5:16 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 12/15/2011 3:42 PM, Ned Batchelder wrote:
>
>  This is another place where Python is inconsistent. We're told, "lists
>> are for homogenous sequences of varying length, like a C array; tuples
>> are for heterogenous aggregations of known length, like a C struct."
>>
>
> I have not been told that for several years, and I am pretty sure you will
> not find any such thing in the current docs. I consider it pretty much
> obsolete, as the differences that flowed from that idea are gone. In Python
> 3, tuples have all the non-mutating sequence methods that list does. The
> situation was much different in 1.4.
>

I strongly disagree. Being immutable sequences (i.e. homogeneous) is a
minor secondary role for tuples. Their primary role remains to hold a small
bunch of heterogeneous values -- like namedtuple, but without needing
forethought. A good example are dictionary items -- these are (key, value)
pairs where for a given dict, the keys are all of the same type (or of a
small set of related types) and ditto for the values, but the key type and
the value types are unrelated.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/dcff1eaf/attachment.html>

From ben+python at benfinney.id.au  Fri Dec 16 03:23:17 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 16 Dec 2011 13:23:17 +1100
Subject: [Python-ideas] Python Users Aren't Perfect
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
Message-ID: <87sjklz23u.fsf@benfinney.id.au>

Ned Batchelder <ned at nedbatchelder.com>
writes:

> This is another place where Python is inconsistent. We're told, "lists
> are for homogenous sequences of varying length, like a C array; tuples
> are for heterogenous aggregations of known length, like a C struct."

I think that's a poor rendition of the distinction. Rather, what I
advise is: if the *meaning* of an item depends on its position in the
sequence (like with a C struct), use a tuple. If the meaning of an item
is unaffected by its position in the sequence, use a list.

> Then we define a function foo(*args), and Python gives us a tuple! :-(

Yes, exactly: the positional arguments to the function are *not* a
homogeneous sequence, so a list doesn't connote the right thing. The
position is important ? we call them ?positional arguments? ? so a tuple
makes sense.

-- 
 \     ?I am the product of millions of generations of individuals who |
  `\      each fought against a hostile universe and won, and I aim to |
_o__)              maintain the tradition.? ?Paul Z. Myers, 2009-09-12 |
Ben Finney



From ned at nedbatchelder.com  Fri Dec 16 03:55:16 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 21:55:16 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <87sjklz23u.fsf@benfinney.id.au>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<87sjklz23u.fsf@benfinney.id.au>
Message-ID: <4EEAB314.1030207@nedbatchelder.com>



On 12/15/2011 9:23 PM, Ben Finney wrote:
> Ned Batchelder<ned at nedbatchelder.com>
> writes:
>
>> Then we define a function foo(*args), and Python gives us a tuple! :-(
> Yes, exactly: the positional arguments to the function are *not* a
> homogeneous sequence, so a list doesn't connote the right thing. The
> position is important ? we call them ?positional arguments? ? so a tuple
> makes sense.
In general the positional arguments to a function are not homogenous, 
but the foo(*args) syntax is for precisely when you don't know how many 
args you will get, and you will of necessity treat them homogenously, 
no?  We've been through this in another branch of this thread...

--Ned.


From ned at nedbatchelder.com  Fri Dec 16 03:56:27 2011
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 15 Dec 2011 21:56:27 -0500
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <CAP7+vJJyjvxkKYSbzZ3AAD=cC31n=ha+wwpfybKeLtRRXCzuFQ@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<jce65v$c4p$1@dough.gmane.org>
	<CAP7+vJJyjvxkKYSbzZ3AAD=cC31n=ha+wwpfybKeLtRRXCzuFQ@mail.gmail.com>
Message-ID: <4EEAB35B.907@nedbatchelder.com>

On 12/15/2011 8:39 PM, Guido van Rossum wrote:
> On Thu, Dec 15, 2011 at 5:16 PM, Terry Reedy <tjreedy at udel.edu 
> <mailto:tjreedy at udel.edu>> wrote:
>
>     On 12/15/2011 3:42 PM, Ned Batchelder wrote:
>
>         This is another place where Python is inconsistent. We're
>         told, "lists
>         are for homogenous sequences of varying length, like a C
>         array; tuples
>         are for heterogenous aggregations of known length, like a C
>         struct."
>
>
>     I have not been told that for several years, and I am pretty sure
>     you will not find any such thing in the current docs. I consider
>     it pretty much obsolete, as the differences that flowed from that
>     idea are gone. In Python 3, tuples have all the non-mutating
>     sequence methods that list does. The situation was much different
>     in 1.4.
>
>
> I strongly disagree. Being immutable sequences (i.e. homogeneous) is a 
> minor secondary role for tuples. Their primary role remains to hold a 
> small bunch of heterogeneous values -- like namedtuple, but without 
> needing forethought. A good example are dictionary items -- these are 
> (key, value) pairs where for a given dict, the keys are all of the 
> same type (or of a small set of related types) and ditto for the 
> values, but the key type and the value types are unrelated.
Could you explain why the foo(*args) syntax creates args as a tuple 
rather than a list?

--Ned.
>
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/3a637a32/attachment.html>

From stephen at xemacs.org  Fri Dec 16 04:08:55 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 16 Dec 2011 12:08:55 +0900
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA626E.8040101@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<20111215215147.22f41c80@pitrou.net>
	<4EEA5FD3.2070603@nedbatchelder.com>
	<A1FB1E82-B6CD-4291-A27F-3B4C2CF48EA9@masklinn.net>
	<4EEA626E.8040101@nedbatchelder.com>
Message-ID: <87fwglxlfc.fsf@uwakimon.sk.tsukuba.ac.jp>

Ned Batchelder writes:
 > On 12/15/2011 4:06 PM, Masklinn wrote:
 > > On 2011-12-15, at 22:00 , Ned Batchelder wrote:
 > >> What?  The whole point of the "def foo(*args)" syntax is so a function can take an unknown-length list of arguments, which will be treated uniformly.
 > > That's not *entirely* true, it's also used for proxy functions, in order to just pass a bunch of arguments (positional or keywords) along to the next level without having to unpack and repack them one by one.
 > We are in agreement: I don't know the length, and I will treat them 
 > uniformly (pass them all through to the next function).

That's way too high a level of abstraction.

The discussion I found persuasive is the varargs analogy.

/* "Hello Ned" in C */
#include <stdio.h>
char *s = "string";
int i = 1;
double x = 3.1415926;

int main (int argc, char *argv[]) {
  printf ("string %s, int %d, double %f\nHello Ned!  Are we homogenized yet?");
  exit(-1);
}

Of course it still demands "correct perception" on the part of the
user, but I don't see this as "inconsistent".


From guido at python.org  Fri Dec 16 04:35:21 2011
From: guido at python.org (Guido van Rossum)
Date: Thu, 15 Dec 2011 19:35:21 -0800
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEAB35B.907@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com> <jce65v$c4p$1@dough.gmane.org>
	<CAP7+vJJyjvxkKYSbzZ3AAD=cC31n=ha+wwpfybKeLtRRXCzuFQ@mail.gmail.com>
	<4EEAB35B.907@nedbatchelder.com>
Message-ID: <CAP7+vJKArncMq_npP06ssaiEpZDDM1opf4qNz0cEifgYgy4z0A@mail.gmail.com>

On Thu, Dec 15, 2011 at 6:56 PM, Ned Batchelder <ned at nedbatchelder.com>wrote:

>  On 12/15/2011 8:39 PM, Guido van Rossum wrote:
>
> On Thu, Dec 15, 2011 at 5:16 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>> On 12/15/2011 3:42 PM, Ned Batchelder wrote:
>>
>>  This is another place where Python is inconsistent. We're told, "lists
>>> are for homogenous sequences of varying length, like a C array; tuples
>>> are for heterogenous aggregations of known length, like a C struct."
>>>
>>
>>  I have not been told that for several years, and I am pretty sure you
>> will not find any such thing in the current docs. I consider it pretty much
>> obsolete, as the differences that flowed from that idea are gone. In Python
>> 3, tuples have all the non-mutating sequence methods that list does. The
>> situation was much different in 1.4.
>>
>
> I strongly disagree. Being immutable sequences (i.e. homogeneous) is a
> minor secondary role for tuples. Their primary role remains to hold a small
> bunch of heterogeneous values -- like namedtuple, but without needing
> forethought. A good example are dictionary items -- these are (key, value)
> pairs where for a given dict, the keys are all of the same type (or of a
> small set of related types) and ditto for the values, but the key type and
> the value types are unrelated.
>
> Could you explain why the foo(*args) syntax creates args as a tuple rather
> than a list?
>

It's a historical accident. In ABC, functions didn't have multiple
arguments -- instead there was a single argument that was a tuple. If you
go back way, way in Python's early history (before 1.0 I think) you'll find
that arguments kind of worked the same way. Then an idiom developed to
accept a variable number of arguments, in order to support default argument
values (because Python's tuples, unlike ABC's, were sequences even then).
This turned out to be awkward if you wanted 1 or 2 arguments (the 1-arg
case had to be dealt with special because you'd receive a plain value
instead of a tuple) and then the *args and arg=value syntax was invented.
But because previously varargs had been tuples, we kept them tuples. (I
think this was also in the days that you couldn't unpack a list -- but I'm
not actually sure of that. I do know that tuple unpacking was always in the
language.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111215/5152850b/attachment.html>

From grosser.meister.morti at gmx.net  Fri Dec 16 05:21:52 2011
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Fri, 16 Dec 2011 05:21:52 +0100
Subject: [Python-ideas] contextlib.maybe
In-Reply-To: <20111211050723.GA12197@air>
References: <20111211050723.GA12197@air>
Message-ID: <4EEAC760.1000109@gmx.net>

On 12/11/2011 06:07 AM, Julian Berman wrote:
> Twice recently I've found myself wanting to write the following code:
>
>      def fn(a_file=None):
>          responsible_for_closing = False
>
>          if a_file is None:
>              a_file = open(a_default_location)
>              responsible_for_closing = True
>
>          do_stuff(a_file)
>
>          if responsible_for_closing:
>              a_file.close()
>

What about this?

	def fn(a_file=None):
		if a_file is None:
			with open(a_default_location) as a_file:
				do_stuff(a_file)
		else:
			do_stuff(a_file)


That's how I do it when this comes up.

> which can be written slightly shorter I know, but it's still a tiny bit
> messy and repetitive. What I'd prefer to write is something more like:
>
>      def fn(a_file=None):
>          with contextlib.maybe(a_file, open, default) as a_file:
>              do_stuff(a_file)
>
> where `maybe` takes an object and conditionally runs a context manager
> if a check fails. Implementation would be:
>
>
>      @contextlib.contextmanager
>      def maybe(got, contextfactory, *args, checkif=bool, **kwargs):
>          if checkif(got):
>              yield got
>          else:
>              with contextfactory(*args, **kwargs) as got:
>                  yield got
>
>
> It's hard to gauge utility for such simple functions (though contextlib
> already has closing(), so I figured it'd be worth asking at least).
> Would this be useful to others? Or perhaps I'm completely missing
> something and you've got suggestions on how to better have an API where
> an argument can be fetched if not provided but a context manager would
> preferably need to be run to do so.


From mwm at mired.org  Fri Dec 16 05:34:28 2011
From: mwm at mired.org (Mike Meyer)
Date: Thu, 15 Dec 2011 20:34:28 -0800
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA6C88.1030203@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
	<4EEA620B.1000300@nedbatchelder.com>
	<4EEA658A.3010509@stoneleaf.us>
	<4EEA6C88.1030203@nedbatchelder.com>
Message-ID: <20111215203428.77c0c821@bhuda.mired.org>

On Thu, 15 Dec 2011 16:54:16 -0500
Ned Batchelder <ned at nedbatchelder.com> wrote:
> On 12/15/2011 4:24 PM, Ethan Furman wrote:
> > Python is a dynamic language -- why can't it have dynamic structs?
> I honestly don't know how to interpret this.

I do. Ethan is incorrectly trying to turn a similarity - that both
Python tuples and C structs are collections of heterogeneous data -
into an analogy. They aren't analogs, as that's pretty much the only
similarity.

If you really want a dynamic tuple - a dynamic collection of
heterogeneous data accessed by an index - use a list. That was
standard practice in LISP-like languages for a long time. But Python
(and most modern LISPs) has better tools for such.

For instance, the proper analog for a C struct in Python is the
class. You provide definitions of them both along with names for their
heterogeneous components. You create instances of them both that can
assign different values to those components. The components are
properly accessed by name. There are probably other similarities.

There are a number of differences as well. Most notably, instances of
classes are dynamic. You can change, and even add and delete,
components dynamically.

So Python already has a dynamic struct. You just have to get the
analogies right.

	<mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From greg.ewing at canterbury.ac.nz  Fri Dec 16 06:50:39 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Dec 2011 18:50:39 +1300
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEA620B.1000300@nedbatchelder.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
	<4EEA620B.1000300@nedbatchelder.com>
Message-ID: <4EEADC2F.9050009@canterbury.ac.nz>

Ned Batchelder wrote:
> Finish that foo function definition: 
> it will *have* to have  "for a in args:"

Not necessarily. It might be something like this:

    def foo(*args):
       if len(args) == 2:
          x, y = args
          fizzle2d(x, y)
       else:
          x, y, z = args
          fizzle3d(x, y, z)

In other words, it might be what is effectively an
overloaded function with a number of different possible
calling signatures. Once you've decided which signature
is being invoked, each member of the argument tuple
takes on a specific role.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Dec 16 07:06:43 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Dec 2011 19:06:43 +1300
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <CAP7+vJKArncMq_npP06ssaiEpZDDM1opf4qNz0cEifgYgy4z0A@mail.gmail.com>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<jce65v$c4p$1@dough.gmane.org>
	<CAP7+vJJyjvxkKYSbzZ3AAD=cC31n=ha+wwpfybKeLtRRXCzuFQ@mail.gmail.com>
	<4EEAB35B.907@nedbatchelder.com>
	<CAP7+vJKArncMq_npP06ssaiEpZDDM1opf4qNz0cEifgYgy4z0A@mail.gmail.com>
Message-ID: <4EEADFF3.3060105@canterbury.ac.nz>

Guido van Rossum wrote:
> (I think this was also in the days that you 
> couldn't unpack a list -- but I'm not actually sure of that. I do know 
> that tuple unpacking was always in the language.)

I can definitely remember a time when you had to use
(...) to unpack tuples and [...] to unpack lists. My
Python memory doesn't go back further than that, though.

-- 
Greg



From ethan at stoneleaf.us  Fri Dec 16 08:50:11 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 15 Dec 2011 23:50:11 -0800
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEADC2F.9050009@canterbury.ac.nz>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>	<20111210144230.GA16801@iskra.aviel.ru>	<4EE5FAEC.1080907@nedbatchelder.com>	<20111212140758.301e7431@pitrou.net>	<4EE5FE81.3060304@nedbatchelder.com>	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>	<4EE67047.8050209@canterbury.ac.nz>	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>	<jcdl1o$hma$1@dough.gmane.org>
	<4EEA5BC2.4070402@nedbatchelder.com>	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>	<4EEA620B.1000300@nedbatchelder.com>
	<4EEADC2F.9050009@canterbury.ac.nz>
Message-ID: <4EEAF833.1080206@stoneleaf.us>

Greg Ewing wrote:
> Ned Batchelder wrote:
>> Finish that foo function definition: it will *have* to have  "for a in 
>> args:"
> 
> Not necessarily. It might be something like this:
> 
>    def foo(*args):
>       if len(args) == 2:
>          x, y = args
>          fizzle2d(x, y)
>       else:
>          x, y, z = args
>          fizzle3d(x, y, z)
> 
> In other words, it might be what is effectively an
> overloaded function with a number of different possible
> calling signatures. Once you've decided which signature
> is being invoked, each member of the argument tuple
> takes on a specific role.
> 

Thanks, Greg -- this is pretty much what I meant about dynamic structs.  :)

~Ethan~


From ncoghlan at gmail.com  Fri Dec 16 09:34:54 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 16 Dec 2011 18:34:54 +1000
Subject: [Python-ideas] contextlib.maybe
In-Reply-To: <4EEAC760.1000109@gmx.net>
References: <20111211050723.GA12197@air>
	<4EEAC760.1000109@gmx.net>
Message-ID: <CADiSq7d5kQkakY4SMHMb_9Li0bSAXVq_DXm4vuYiWjWTH6KdHg@mail.gmail.com>

On Fri, Dec 16, 2011 at 2:21 PM, Mathias Panzenb?ck
<grosser.meister.morti at gmx.net> wrote:
> On 12/11/2011 06:07 AM, Julian Berman wrote:
>>
>> Twice recently I've found myself wanting to write the following code:
>>
>> ? ? def fn(a_file=None):
>> ? ? ? ? responsible_for_closing = False
>>
>> ? ? ? ? if a_file is None:
>> ? ? ? ? ? ? a_file = open(a_default_location)
>> ? ? ? ? ? ? responsible_for_closing = True
>>
>> ? ? ? ? do_stuff(a_file)
>>
>> ? ? ? ? if responsible_for_closing:
>> ? ? ? ? ? ? a_file.close()
>>
>
> What about this?
>
> ? ? ? ?def fn(a_file=None):
> ? ? ? ? ? ? ? ?if a_file is None:
> ? ? ? ? ? ? ? ? ? ? ? ?with open(a_default_location) as a_file:
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?do_stuff(a_file)
> ? ? ? ? ? ? ? ?else:
> ? ? ? ? ? ? ? ? ? ? ? ?do_stuff(a_file)
>
>
> That's how I do it when this comes up.

With contextlib2.ContextStack [1] the problem of conditional release
of resources can be handled as follows:

    def fn(a_file=None):
        with ContextStack() as stack:
            if a_file is None:
                # The stack will release the file when we're done
                a_file = stack.enter_context(open(a_default_location))
            do_stuff(a_file)

If you're interested in seeing this (or something like it) in the
standard library for 3.3, grab the module of PyPI, play around with it
and send me feedback on the BitBucket issue tracker [2].

[1] http://contextlib2.readthedocs.org/en/latest/index.html#contextlib2.ContextStack
[2] https://bitbucket.org/ncoghlan/contextlib2/issues?status=new&status=open

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From masklinn at masklinn.net  Fri Dec 16 09:37:11 2011
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 16 Dec 2011 09:37:11 +0100
Subject: [Python-ideas] Python Users Aren't Perfect
In-Reply-To: <4EEADC2F.9050009@canterbury.ac.nz>
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<CAN8CLg==syWhJcOHKyyrhaoRKW8UQ7nTB1e6kTuOGo5Psjou3Q@mail.gmail.com>
	<4EEA620B.1000300@nedbatchelder.com>
	<4EEADC2F.9050009@canterbury.ac.nz>
Message-ID: <F0E7D462-D389-4720-BC21-F7BE3C246049@masklinn.net>

On 2011-12-16, at 06:50 , Greg Ewing wrote:
> Ned Batchelder wrote:
>> Finish that foo function definition: it will *have* to have  "for a in args:"
> 
> Not necessarily. It might be something like this:
> 
>   def foo(*args):
>      if len(args) == 2:
>         x, y = args
>         fizzle2d(x, y)
>      else:
>         x, y, z = args
>         fizzle3d(x, y, z)
> 
> In other words, it might be what is effectively an
> overloaded function with a number of different possible
> calling signatures. Once you've decided which signature
> is being invoked, each member of the argument tuple
> takes on a specific role.
Is that really used instead of optional arguments with a placeholder value e.g.

    def foo(x, y, z=None):
        if z is None:
            fizzle2d(x, y)
        else:
            fizzle3d(x, y, z)

?

From ben+python at benfinney.id.au  Fri Dec 16 10:08:17 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 16 Dec 2011 20:08:17 +1100
Subject: [Python-ideas] Python Users Aren't Perfect
References: <CAJvrtnuMgdpoauYXu1Enk8hRVUxsSG=t7bp4hsMxGFsigyvrLw@mail.gmail.com>
	<20111210144230.GA16801@iskra.aviel.ru>
	<4EE5FAEC.1080907@nedbatchelder.com>
	<20111212140758.301e7431@pitrou.net>
	<4EE5FE81.3060304@nedbatchelder.com>
	<062FD665-B80A-401E-A3C2-75CA2ABE4424@masklinn.net>
	<4EE67047.8050209@canterbury.ac.nz>
	<87d3btyyuq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<jcdl1o$hma$1@dough.gmane.org> <4EEA5BC2.4070402@nedbatchelder.com>
	<87sjklz23u.fsf@benfinney.id.au> <4EEAB314.1030207@nedbatchelder.com>
Message-ID: <87obv8zxxa.fsf@benfinney.id.au>

Ned Batchelder <ned at nedbatchelder.com>
writes:

> On 12/15/2011 9:23 PM, Ben Finney wrote:

> > the positional arguments to the function are *not* a homogeneous
> > sequence, so a list doesn't connote the right thing. The position is
> > important ? we call them ?positional arguments? ? so a tuple makes
> > sense.

> In general the positional arguments to a function are not homogenous,
> but the foo(*args) syntax is for precisely when you don't know how
> many args you will get

It's valuable for the case where the arguments need to be passed
verbatim to a superclass. At some point, something in the call chain
should know the meaning of each positional argument.

> and you will of necessity treat them homogenously, no?

If the function consumes the positional arguments as a homogeneous list,
I'd say that's a poor design (the function should instead specify a
single argument which is a homogeneous sequence).

-- 
 \       ?If you define cowardice as running away at the first sign of |
  `\   danger, screaming and tripping and begging for mercy, then yes, |
_o__)               Mr. Brave man, I guess I'm a coward.? ?Jack Handey |
Ben Finney



From alon at horev.net  Sat Dec 17 02:00:09 2011
From: alon at horev.net (Alon Horev)
Date: Sat, 17 Dec 2011 03:00:09 +0200
Subject: [Python-ideas] context managers used to allocate resources can be
	abused - a solution
Message-ID: <CAKZkVDsfXqHU2SKBfNRicLm5=sMSOC2wBx9KYagrrW3dNjydcw@mail.gmail.com>

Hello,

while I was implementing a connection pool i've noticed a pitfall of our
beloved with statements:

with pool.get_connection() as conn:
....conn.execute(...)
conn.execute(...) # the connection has been returned to the pool and does
not belong to the user!

a proposed solution:

with protected_context(pool.get_connection()) as conn:
....conn.execute(...)
conn.execute(...) # raises OutOfContextError()

with protected_context(file("/tmp/bla.txt", "w")) as f:
....file.write("blo")
file.write("blu") # raises OutOfContextError()

the solution is to basically proxy all methods to the real object until the
context ends and then the proxy expires.
what do you think about adding it to contextlib?

        Thanks, Alon Horev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111217/bc60cb92/attachment.html>

From ncoghlan at gmail.com  Sat Dec 17 03:07:55 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 17 Dec 2011 12:07:55 +1000
Subject: [Python-ideas] context managers used to allocate resources can
 be abused - a solution
In-Reply-To: <CAKZkVDsfXqHU2SKBfNRicLm5=sMSOC2wBx9KYagrrW3dNjydcw@mail.gmail.com>
References: <CAKZkVDsfXqHU2SKBfNRicLm5=sMSOC2wBx9KYagrrW3dNjydcw@mail.gmail.com>
Message-ID: <CADiSq7fzg0PBxQs-ivLQMekONPoeg40tg+8p_Fz+7chXnYsS_g@mail.gmail.com>

On Sat, Dec 17, 2011 at 11:00 AM, Alon Horev <alon at horev.net> wrote:
> Hello,
>
> while I was implementing a connection pool?i've noticed a pitfall of our
> beloved with statements:
>
> with pool.get_connection() as conn:
> ....conn.execute(...)
> conn.execute(...) # the connection has been returned to the pool and?does
> not belong to the user!

That's a bug in the connection pool implementation. If the underlying
connection has been returned to the pool, the proxy returned by the
context manager shouldn't work any more.

Try the above code structure with a file object and it will give you
"IO operation on closed file".

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From tjreedy at udel.edu  Sun Dec 18 04:23:13 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 17 Dec 2011 22:23:13 -0500
Subject: [Python-ideas] context managers used to allocate resources can
 be abused - a solution
In-Reply-To: <CADiSq7fzg0PBxQs-ivLQMekONPoeg40tg+8p_Fz+7chXnYsS_g@mail.gmail.com>
References: <CAKZkVDsfXqHU2SKBfNRicLm5=sMSOC2wBx9KYagrrW3dNjydcw@mail.gmail.com>
	<CADiSq7fzg0PBxQs-ivLQMekONPoeg40tg+8p_Fz+7chXnYsS_g@mail.gmail.com>
Message-ID: <jcjmbb$e7n$1@dough.gmane.org>

On 12/16/2011 9:07 PM, Nick Coghlan wrote:
> On Sat, Dec 17, 2011 at 11:00 AM, Alon Horev<alon at horev.net>  wrote:
>> Hello,
>>
>> while I was implementing a connection pool i've noticed a pitfall of our
>> beloved with statements:
>>
>> with pool.get_connection() as conn:
>> ....conn.execute(...)
>> conn.execute(...) # the connection has been returned to the pool and does
>> not belong to the user!

Good catch!

> That's a bug in the connection pool implementation. If the underlying
> connection has been returned to the pool, the proxy returned by the
> context manager shouldn't work any more.

I believe Nick is saying that conn.__exit__ should close the connection 
and release the resource, that being the point of with statements and 
context managers. Can you open a tracker issue?

> Try the above code structure with a file object and it will give you
> "IO operation on closed file".

Because file_object.__exit__ closes the connection to the OS resource.

-- 
Terry Jan Reedy



From ncoghlan at gmail.com  Sun Dec 18 07:41:26 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 18 Dec 2011 16:41:26 +1000
Subject: [Python-ideas] context managers used to allocate resources can
 be abused - a solution
In-Reply-To: <jcjmbb$e7n$1@dough.gmane.org>
References: <CAKZkVDsfXqHU2SKBfNRicLm5=sMSOC2wBx9KYagrrW3dNjydcw@mail.gmail.com>
	<CADiSq7fzg0PBxQs-ivLQMekONPoeg40tg+8p_Fz+7chXnYsS_g@mail.gmail.com>
	<jcjmbb$e7n$1@dough.gmane.org>
Message-ID: <CADiSq7dUHufTYYgccfkDcjmtGHq=mYrY8eH1Dc6sr-kKcZTYFA@mail.gmail.com>

On Sun, Dec 18, 2011 at 1:23 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> That's a bug in the connection pool implementation. If the underlying
>> connection has been returned to the pool, the proxy returned by the
>> context manager shouldn't work any more.
>
>
> I believe Nick is saying that conn.__exit__ should close the connection and
> release the resource, that being the point of with statements and context
> managers. Can you open a tracker issue?

No, that's not what I'm saying at all - the whole purpose of using a
connection pool is that you *don't* release the underlying OS resource
when you return a resource to the pool.

What I'm saying is that if a pool supports the context management
protocol for *temporary* access to one of the pool resources, then
*that* context manager is responsible for ensuring that the resource
reference returned is correctly invalidated in __exit__ (just as
closing a file object prevents further IO operations).

I'm not aware of any such pools in the stdlib (multiprocessing has
some pool APIs, but I don't believe they support the context
management protocol for pooled resources), so I'm suggesting that the
only problem here is in the design of the OP's own pool API.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From nathan.alexander.rice at gmail.com  Mon Dec 19 00:28:27 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sun, 18 Dec 2011 18:28:27 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
Message-ID: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>

I believe it would be a good idea in instances where it is known that a
collection of a single type is going to be returned, to return a subclass
with type information and type specific methods "mixed in".  You could
provide member methods as collection methods that operate in a
vectorized manner, returning a new collection or iterator with the results
much like the mathematical functions in NumPy.  This would also give
people a reliable method to make functions operate on both scalar and
vector values.  I believe this could be implemented without needing
subclasses for everything under the sun with a generic collection "type
contract" mix-in.  If a developer wanted to provide additional type specific
collection/iterator methods they would of course need to subclass that.

To avoid handcuffing people with types (which is definitely un-pythonic)
and maintain backwards compatibility, the standard collection
modification methods could be hooked so that if an object of an
incorrect type is added, a warning is raised and the collection
gracefully degrades by removing mixed-in type information and
methods.  Additionally, a method could be provided that lets the user
"terminate the contract" causing the collection to degrade without a
warning.

I have several motivations for this:

-- Performing a series of operations using comprehensions or map
tends to be highly verbose in an uninformative way.  Compare the
current method with what would be possible using "typed" collections:

L2 = [X(e) for e in L1]
L3 = [Y(e) for e in L2]
vs
L2 = X(L1) # assuming X has been updated to work in both vector/scalar
L3 = Y(L2) # context...

L2 = [Z(Y(X(e))) for e in L1]
vs
L2 = Z(Y(X(L1)))

L2 = [e.X().Y().Z() for e in L1]
vs
L2 = L1.X().Y().Z() # assuming vectorized versions of member methods
#are folded into the collection via the mixin.

--  Because collections are type agnostic, it is not possible to place
methods on them that are type specific.  This leads to a lot of cases
where python forces you to read inside out or a the syntax gets
very disjoint in general.  A good example of this is:

"\n".join(l.capitalize() for l in my_string.split("\n"))

which could reduce to something far more readable, such as:

my_string.split("\n").capitalize().join_items("\n")

Besides the benefits to basic language usability (in my opinion) there
are tool and development benefits:

-- The additional type information would simplify static analysis and
provide cues for optimization (I'm looking at pypy here; their list
strategies play to this perfectly)

-- The warning on "violating the contract" and without first terminating
it would be a helpful tool in catching and debugging errors.

I have some thoughts on syntax and specifics that I think would work well,
however I wanted to solicit more feedback before I go too far down that path.


Nathan


From ncoghlan at gmail.com  Mon Dec 19 00:45:08 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Dec 2011 09:45:08 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
Message-ID: <CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>

On Mon, Dec 19, 2011 at 9:28 AM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> -- Performing a series of operations using comprehensions or map
> tends to be highly verbose in an uninformative way. ?Compare the
> current method with what would be possible using "typed" collections:
>
> L2 = [X(e) for e in L1]
> L3 = [Y(e) for e in L2]
> vs
> L2 = X(L1) # assuming X has been updated to work in both vector/scalar
> L3 = Y(L2) # context...

This use case is why map() remains a builtin, even in Python 3:

L2 = map(X, L1)
L3 = map(Y, L2)

Short, but explicit (no under-the-hood guessing about whether or not
something should be treated as a scalar or vector value - in the
general case, this distinction isn't as clear as you might think, just
look at strings).

> L2 = [Z(Y(X(e))) for e in L1]
> vs
> L2 = Z(Y(X(L1)))

def XYZ(arg):
   """Look, I can document what this means!"""
   return Z(Y(X(arg)))

L2 = map(XYZ, L1)

> L2 = [e.X().Y().Z() for e in L1]
> vs
> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods
> #are folded into the collection via the mixin.

def XYZ_methods(arg):
   """I can also document what *this* means"""
   return arg.X().Y().Z()

L2 = map(XYZ_methods, L1)

> -- ?Because collections are type agnostic, it is not possible to place
> methods on them that are type specific. ?This leads to a lot of cases
> where python forces you to read inside out or a the syntax gets
> very disjoint in general. ?A good example of this is:
>
> "\n".join(l.capitalize() for l in my_string.split("\n"))
>
> which could reduce to something far more readable, such as:
>
> my_string.split("\n").capitalize().join_items("\n")

Another bad example, since that's just a really verbose way of writing
my_string.capitalize().

Short answer: what advantage does your proposal really offer over
simply extracting the repetitive operation out to a use case specific
function, and making effective use of the existing vectorisation
utilities (i.e. map() and itertools)?

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From tjreedy at udel.edu  Mon Dec 19 01:18:02 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 18 Dec 2011 19:18:02 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
Message-ID: <jclvs4$oft$1@dough.gmane.org>

On 12/18/2011 6:28 PM, Nathan Rice wrote:
> I believe it would be a good idea in instances where it is known that a
> collection of a single type is going to be returned, to return a subclass
> with type information and type specific methods "mixed in".

Are you familiar with numpy, which has typed multi-dimensional arrays 
and array operations? And the option to add user functions to work with 
such?

-- 
Terry Jan Reedy



From nathan.alexander.rice at gmail.com  Mon Dec 19 03:02:38 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sun, 18 Dec 2011 21:02:38 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jclvs4$oft$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<jclvs4$oft$1@dough.gmane.org>
Message-ID: <CAOFbRmJXc61HDUWznOvB7VJgLXdReutxiNdHrH1WMr9qrNgUrw@mail.gmail.com>

>> I believe it would be a good idea in instances where it is known that a
>> collection of a single type is going to be returned, to return a subclass
>> with type information and type specific methods "mixed in".
>
> Are you familiar with numpy, which has typed multi-dimensional arrays and
> array operations? And the option to add user functions to work with such?

I am, and vectorized functions/typed arrays from numpy are a source of
inspiration for me in this proposal.  The problem with having typed
arrays/etc as an add-on is that you lose the benefits as soon as you
step into code that isn't meant to interact with with the specific
library.  This isn't such an issue with NumPy since it has a healthy
ecosystem, but there are lots of other instances where having type
aware collections would be nice.


From nathan.alexander.rice at gmail.com  Mon Dec 19 03:24:12 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sun, 18 Dec 2011 21:24:12 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
Message-ID: <CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>

On Sun, Dec 18, 2011 at 6:45 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Mon, Dec 19, 2011 at 9:28 AM, Nathan Rice
> <nathan.alexander.rice at gmail.com> wrote:
>> -- Performing a series of operations using comprehensions or map
>> tends to be highly verbose in an uninformative way. ?Compare the
>> current method with what would be possible using "typed" collections:
>>
>> L2 = [X(e) for e in L1]
>> L3 = [Y(e) for e in L2]
>> vs
>> L2 = X(L1) # assuming X has been updated to work in both vector/scalar
>> L3 = Y(L2) # context...
>
> This use case is why map() remains a builtin, even in Python 3:

Yes, but map(lambda x: getattr(x, "method")(), thing) is ugly, and
map(lambda x: x.method_2(param), map(lambda x: x.method(param),
thing)) is really ugly.  On top of that, it is asking more of code
analysis tools to verify that code, and IDEs aren't going to be able
to tell you the methods on the ambiguous x in the lambda. Sure if the
only argument is self, you could call class.method, but I don't think
that is the majority use case.

> L2 = map(X, L1)
> L3 = map(Y, L2)
>
> Short, but explicit (no under-the-hood guessing about whether or not
> something should be treated as a scalar or vector value - in the
> general case, this distinction isn't as clear as you might think, just
> look at strings).

Yes, I love that feature of strings, it is a source of lots of bugs,
but I digress.  The reason this partially solves that problem is that
instead of having to do a bunch of guesswork on an iterable to see if
you should do the vectorized version of the function, you just check
to see if it is an instance of a certain TypedCollectionContract.  If
so, vectorize.  No "isinstance(foo, Iterable) and not isinstance(foo,
basestr)" silliness here.

>> L2 = [Z(Y(X(e))) for e in L1]
>> vs
>> L2 = Z(Y(X(L1)))
>
> def XYZ(arg):
> ? """Look, I can document what this means!"""
> ? return Z(Y(X(arg)))
>
> L2 = map(XYZ, L1)

What about WXY, and XZ, and WYZ, and YZ, and...

>> L2 = [e.X().Y().Z() for e in L1]
>> vs
>> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods
>> #are folded into the collection via the mixin.
>
> def XYZ_methods(arg):
> ? """I can also document what *this* means"""
> ? return arg.X().Y().Z()
>
> L2 = map(XYZ_methods, L1)
>
>> -- ?Because collections are type agnostic, it is not possible to place
>> methods on them that are type specific. ?This leads to a lot of cases
>> where python forces you to read inside out or a the syntax gets
>> very disjoint in general. ?A good example of this is:
>>
>> "\n".join(l.capitalize() for l in my_string.split("\n"))
>>
>> which could reduce to something far more readable, such as:
>>
>> my_string.split("\n").capitalize().join_items("\n")
>
> Another bad example, since that's just a really verbose way of writing
> my_string.capitalize().

The python interpreter says otherwise...

>>> foo = "line 1\nline 2\nline 3"
>>> foo.capitalize()
'Line 1\nline 2\nline 3'
>>> "\n".join(s.capitalize() for s in foo.split("\n"))
'Line 1\nLine 2\nLine 3'
>>>

> Short answer: what advantage does your proposal really offer over
> simply extracting the repetitive operation out to a use case specific
> function, and making effective use of the existing vectorisation
> utilities (i.e. map() and itertools)?

IDEs can provide context hints, interpreters can use the contract to
change how they treat the collection to improve performance, lint
style code analysis will be easier and it is across the board less
verbose than the currently available options, while reading left to
right with fewer non letter characters I.E. more understandable.

Nathan


From wuwei23 at gmail.com  Mon Dec 19 04:39:19 2011
From: wuwei23 at gmail.com (alex23)
Date: Sun, 18 Dec 2011 19:39:19 -0800 (PST)
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
Message-ID: <1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>

On Dec 19, 12:24?pm, Nathan Rice <nathan.alexander.r... at gmail.com>
wrote:
> Yes, but map(lambda x: getattr(x, "method")(), thing) is ugly

    from operator import methodcaller

    method = methodcaller('method')
    result = map(method, thing)

> map(lambda x: x.method_2(param), map(lambda x: x.method(param),
> thing)) is really ugly.

    method = methodcaller('method', param)
    method2 = methodcaller('method_2', param)
    result = map(method2, map(method, thing))

If your code is ugly, stop writing ugly code. :)


From wuwei23 at gmail.com  Mon Dec 19 05:14:28 2011
From: wuwei23 at gmail.com (alex23)
Date: Sun, 18 Dec 2011 20:14:28 -0800 (PST)
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
Message-ID: <5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>

Received via email, this isn't a private discussion:

> Nathan Rice <nathan.alexander.r... at gmail.com> wrote:
> I fail to see how the clunky act of importing a function most python
> users probably don't know exists then creating another new function is
> better than chaining one or two method calls.

If you consider importing functionality 'clunky', then we're already
at an impasse over what we consider to be 'elegant'. I also don't
believe that user ignorance of standard library features mandates the
inclusion of _everything_ into the base langauge.

> Just because you CAN do something already doesn't mean you can do it in
> an elegant way.

And just because YOU find it inelegant doesn't make it so. I find
using operator & functools _far_ clearer in intent than using lambda,
_and it works right now_, which was the point I was trying to make
here. You wrote unreadable code and then tried to use that as an
argument for your idea. Breaking down complex statements into clearer
parts isn't a radical notion; cramming more and more functionality
onto a single line isn't something to which Python needs to aspire.

> If the only metric was "can we do this?" we would all still be using Fortran.

And if it's "I don't want to have to code to deal with this, let the
language change to do it", you end up with PHP.


From ncoghlan at gmail.com  Mon Dec 19 05:29:59 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Dec 2011 14:29:59 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
Message-ID: <CADiSq7dSbBTu1p=qmQ+R5=7ejDPHjji49XO8Ey0u5cQQJWcAkA@mail.gmail.com>

On Mon, Dec 19, 2011 at 12:24 PM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> On Sun, Dec 18, 2011 at 6:45 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Another bad example, since that's just a really verbose way of writing
>> my_string.capitalize().
>
> The python interpreter says otherwise...
>
>>>> foo = "line 1\nline 2\nline 3"
>>>> foo.capitalize()
> 'Line 1\nline 2\nline 3'
>>>> "\n".join(s.capitalize() for s in foo.split("\n"))
> 'Line 1\nLine 2\nLine 3'

Ah, my mistake, I was thinking of title() rather than capitalize().

Still:

def capitalize_lines(s):
    return "\n".join(s.capitalize() for line in s.split("\n"))

There comes a time when the contortions people go to to avoid naming a
frequently repeated operation just get silly. If you do something a
lot, pull it out into a function and name it. The named function can
even be a local closure if the usage is sufficiently specific to one
operation - then it can still reference local variables without
requiring a lot of additional parameters.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From nathan.alexander.rice at gmail.com  Mon Dec 19 05:31:31 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sun, 18 Dec 2011 23:31:31 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
Message-ID: <CAOFbRmKhYoM70ajTUcaaJRP3XqpkyCWVh479vTN65sm5Bs+otQ@mail.gmail.com>

My apologies for the direct email.  Gmail is somewhat retarded when it
comes to the python lists, and I don't always remember to change the
to:...

>> Nathan Rice <nathan.alexander.r... at gmail.com> wrote:
>> I fail to see how the clunky act of importing a function most python
>> users probably don't know exists then creating another new function is
>> better than chaining one or two method calls.
>
> If you consider importing functionality 'clunky', then we're already
> at an impasse over what we consider to be 'elegant'. I also don't
> believe that user ignorance of standard library features mandates the
> inclusion of _everything_ into the base langauge.

There are a lot of warts in the standard library, (thus Py3).  I'm all
for importing things that deserve to be in their own namespace with
other, similar things.  I'm not for having to import something that is
needlessly more verbose than solutions that are available without an
import (see comprehensions).  If you were just trying to map a single
argument function I'd say "yeah, go for it" but from my perspective,
that code is square peg, meet round hole.

>> Just because you CAN do something already doesn't mean you can do it in
>> an elegant way.
>
> And just because YOU find it inelegant doesn't make it so. I find
> using operator & functools _far_ clearer in intent than using lambda,
> _and it works right now_, which was the point I was trying to make
> here. You wrote unreadable code and then tried to use that as an
> argument for your idea. Breaking down complex statements into clearer
> parts isn't a radical notion; cramming more and more functionality
> onto a single line isn't something to which Python needs to aspire.

What makes a solution elegant?  I'd say having access to a cross
cutting paradigm that is backwards compatible, reduces code while
making it more readable and provides opportunities to improve the
toolchain is more elegant.  At any rate, I could care less about the
lambda, that is a strawman.

>> If the only metric was "can we do this?" we would all still be using Fortran.
>
> And if it's "I don't want to have to code to deal with this, let the
> language change to do it", you end up with PHP.

Well, that would be true if you also added "I don't care if the
language is internally consistent in any way" and a few others, but
I'm not here to bash PHP.


From ncoghlan at gmail.com  Mon Dec 19 05:33:17 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Dec 2011 14:33:17 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
Message-ID: <CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>

On Mon, Dec 19, 2011 at 12:24 PM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> Yes, I love that feature of strings, it is a source of lots of bugs,
> but I digress. ?The reason this partially solves that problem is that
> instead of having to do a bunch of guesswork on an iterable to see if
> you should do the vectorized version of the function, you just check
> to see if it is an instance of a certain TypedCollectionContract. ?If
> so, vectorize. ?No "isinstance(foo, Iterable) and not isinstance(foo,
> basestr)" silliness here.

If you plan to introduce a new ABC to drive this, then I have a simple proposal:

1. Write a module that implements your "broadcast API for collections"
2. Publish it on PyPI
3. When it has popular uptake to indicate widespread user demand, then
come back to us

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From nathan.alexander.rice at gmail.com  Mon Dec 19 05:47:04 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Sun, 18 Dec 2011 23:47:04 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
Message-ID: <CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>

> Ah, my mistake, I was thinking of title() rather than capitalize().>> Still:>> def capitalize_lines(s):> ? ?return "\n".join(s.capitalize() for line in s.split("\n"))>> There comes a time when the contortions people go to to avoid naming a> frequently repeated operation just get silly. If you do something a> lot, pull it out into a function and name it. The named function can> even be a local closure if the usage is sufficiently specific to one> operation - then it can still reference local variables without> requiring a lot of additional parameters.
Oh, I agree that it is best just to make it a function, and I have
(several times!). ?So have a lot of other people.
>> Yes, I love that feature of strings, it is a source of lots of bugs,
>> but I digress. ?The reason this partially solves that problem is that
>> instead of having to do a bunch of guesswork on an iterable to see if
>> you should do the vectorized version of the function, you just check
>> to see if it is an instance of a certain TypedCollectionContract. ?If
>> so, vectorize. ?No "isinstance(foo, Iterable) and not isinstance(foo,
>> basestr)" silliness here.
>
> If you plan to introduce a new ABC to drive this, then I have a simple proposal:
>
> 1. Write a module that implements your "broadcast API for collections"
> 2. Publish it on PyPI
> 3. When it has popular uptake to indicate widespread user demand, then
> come back to us

I certainly plan to make an implementation for example purposes.  And
if this was Ruby and not Python, that module would actually be
universally useful, because they can rewire their internals better
than we can.  Unfortunately, in Python land it is very difficult to
hook into list/iterator/etc creation on a global scale.  As a result,
a user would have to be very careful about calling into other people's
code, and the type information would still get blown away pretty
frequently.  Kind of like if you subclass string; all the string
methods still return strings, so the language is working against you
there.  Regardless, I'm sure everything that gets added to the
language was first a hugely popular PyPI project, that is clearly the
way the language evolves.


From ncoghlan at gmail.com  Mon Dec 19 06:44:52 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Dec 2011 15:44:52 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
Message-ID: <CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>

On Mon, Dec 19, 2011 at 2:47 PM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> Regardless, I'm sure everything that gets added to the
> language was first a hugely popular PyPI project, that is clearly the
> way the language evolves.

Not everything, no. The real criteria is to have solid use cases where
a proposal clearly improves the language. Sometimes that's just
obvious (e.g. supporting a popular new compression protocol),
sometimes a PEP is enough to explain, other times real world
experience on PyPI is the best option. You don't have *any* of those
at this point (just some vague hand-waving), so publishing a PyPI
package is the most obvious way to start acquiring more data (and to
prove that there's even a version of the idea that can be taken beyond
the hand-waving stage).

What you seem to be asking for is a general purpose typed container
factory along the following lines:

    def typed_container(container_type, data_type):
        class TypedContainer(container_type):
            def __getattr__(self, attr):
                data_type_attr = getattribute(data_type, attr)
                if callable(data_type_attr):
                    _result_type = type(self)
                    def _broadcast(*args, **kwds):
                        _result_type(data_type_attr(x, *args, **kwds)
for x in self)
                    return _broadcast
                return data_type_attr
        return TypedContainer

I think it will have a lot of problems in practice (note that NumPy
doesn't try to solve the broadcasting problem in general, just for a
single specific data type), but, if the concept has any merit at all,
that's certainly something that can be demonstrated quite adequately
on PyPI.

To get a better idea of the level of evidence you're trying to reach
if your suggestion is ever going to get anywhere, try taking a look at
http://www.boredomandlaziness.org/2011/02/justifying-python-language-changes.html
and http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From greg.ewing at canterbury.ac.nz  Mon Dec 19 07:48:32 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 19 Dec 2011 19:48:32 +1300
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
Message-ID: <4EEEDE40.9090402@canterbury.ac.nz>

Nick Coghlan wrote:

> What you seem to be asking for is a general purpose typed container
> factory along the following lines:
> 
>     def typed_container(container_type, data_type):
>         class TypedContainer(container_type):
>             def __getattr__(self, attr):
>                 data_type_attr = getattribute(data_type, attr)
>                 if callable(data_type_attr):
>                     _result_type = type(self)
>                     def _broadcast(*args, **kwds):
>                         _result_type(data_type_attr(x, *args, **kwds) for x in self)
>                     return _broadcast
>                 return data_type_attr
>         return TypedContainer

Seems to me this would be better described as a "broadcasting container"
than a "typed container". The only use it makes of the element type is
to speed things up a bit by extracting bound methods. So the typed-ness
is not an essential feature of its functionality, just something
required to support an optimisation.

Extended to handle the various operator methods, this might be a useful
thing to have around. In conjunction with array.array, it could provide
a kind of "numpy lite" for when depending on full-blown numpy would seem
like overkill.

-- 
Greg


From ncoghlan at gmail.com  Mon Dec 19 09:08:10 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 19 Dec 2011 18:08:10 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <4EEEDE40.9090402@canterbury.ac.nz>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<4EEEDE40.9090402@canterbury.ac.nz>
Message-ID: <CADiSq7dWuaOm8JDZ3P6Fhg143gmCEffzYu-Mh67CMJpcSujY7w@mail.gmail.com>

On Mon, Dec 19, 2011 at 4:48 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Extended to handle the various operator methods, this might be a useful
> thing to have around. In conjunction with array.array, it could provide
> a kind of "numpy lite" for when depending on full-blown numpy would seem
> like overkill.

Agreed, but that "might" is the killer - hence why PyPI is the
appropriate place for this idea. Perhaps when it's been refined for a
couple of years, a case might be made for standard lib inclusion.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From storchaka at gmail.com  Mon Dec 19 09:33:40 2011
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 19 Dec 2011 10:33:40 +0200
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
Message-ID: <jcmstb$p7a$1@dough.gmane.org>

19.12.11 01:28, Nathan Rice ???????(??):
> L2 = [X(e) for e in L1]
> L3 = [Y(e) for e in L2]
> vs
> L2 = X(L1) # assuming X has been updated to work in both vector/scalar
> L3 = Y(L2) # context...

L = ['a', 'bc', ['ada', 'a']]

What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?

> L2 = [Z(Y(X(e))) for e in L1]
> vs
> L2 = Z(Y(X(L1)))
>
> L2 = [e.X().Y().Z() for e in L1]
> vs
> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods
> #are folded into the collection via the mixin.

What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]?



From storchaka at gmail.com  Mon Dec 19 09:39:08 2011
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 19 Dec 2011 10:39:08 +0200
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
Message-ID: <jcmt7i$ras$1@dough.gmane.org>

19.12.11 06:14, alex23 ???????(??):
>  I find
> using operator&  functools _far_ clearer in intent than using lambda,
> _and it works right now_, which was the point I was trying to make
> here.

I find using list comprehensions and generator expressions even more 
clearer.



From nathan.alexander.rice at gmail.com  Mon Dec 19 14:52:33 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Mon, 19 Dec 2011 08:52:33 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
Message-ID: <CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>

>> L2 = [X(e) for e in L1]>> L3 = [Y(e) for e in L2]>> vs>> L2 = X(L1) # assuming X has been updated to work in both vector/scalar>> L3 = Y(L2) # context...>>> L = ['a', 'bc', ['ada', 'a']]>> What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?>>>> L2 = [Z(Y(X(e))) for e in L1]>> vs>> L2 = Z(Y(X(L1)))>>>> L2 = [e.X().Y().Z() for e in L1]>> vs>> L2 = L1.X().Y().Z() # assuming vectorized versions of member methods>> #are folded into the collection via the mixin.>>> What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]?

A fair concern; if the vectorized version of the child method were
given the same name as the child method, I agree that this could
result in ambiguity.

There are multiple ways that member methods could be made available on
the collection, including a proxy attribute, renaming, etc.
...
>>  I find
>> using operator&  functools _far_ clearer in intent than using lambda,
>>
>> _and it works right now_, which was the point I was trying to make
>> here.
>
>
> I find using list comprehensions and generator expressions even more
> clearer.

In general I think comprehensions are superior to map because they are
more intuitive.  Based on my experience reading code in the community
I think that is well supported.

Sadly, even though I have issues with the way they are used in many
cases, lambdas are superior to a lot of other options because they are
simple and cover many use cases.  When used inline with simple
expressions they provide a lot of mileage.


From nathan.alexander.rice at gmail.com  Mon Dec 19 15:30:37 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Mon, 19 Dec 2011 09:30:37 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
Message-ID: <CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>

> Not everything, no. The real criteria is to have solid use cases where
> a proposal clearly improves the language. Sometimes that's just
> obvious (e.g. supporting a popular new compression protocol),
> sometimes a PEP is enough to explain, other times real world
> experience on PyPI is the best option. You don't have *any* of those
> at this point (just some vague hand-waving), so publishing a PyPI
> package is the most obvious way to start acquiring more data (and to
> prove that there's even a version of the idea that can be taken beyond
> the hand-waving stage).

Couple things.

1. The "broadcasting" that people seemed to have latched on to is only
part of what I put forward, and I agree it is something that would
have to be done *correctly* to be beneficial.  I would have no issues
with providing a userspace lib to do this if type decorations were
included in homogeneous collections/iterables, as long as the
implementation of the decoration didn't suffer from some form of
"string failure" (string subclasses are basically worthless as methods
return strings, not an instance of the class).

2. A "type decorator" on homogeneous collections and iterables has a
lot of nice little benefits throughout the toolchain.

3. Being able to place methods on a "type decorator" is useful, it
solves issues like "foo".join() which really wants to be a method on
string collections.

4. I wanted to gauge people's feelings before I went through the steps
involved in writing a PEP.  I believe that is the right thing to do,
so I don't feel the "hand waving" comment is warranted.  I've already
learned people view collections that provide child object methods in
vector form as a very big change even if it is backwards compatible;
that is fine, I'm willing to shelve that if consensus is that people
aren't comfortable with it.

> What you seem to be asking for is a general purpose typed container
> factory along the following lines:

> ? ?def typed_container(container_type, data_type):
> ? ? ? ?class TypedContainer(container_type):
> ? ? ? ? ? ?def __getattr__(self, attr):
> ? ? ? ? ? ? ? ?data_type_attr = getattribute(data_type, attr)
> ? ? ? ? ? ? ? ?if callable(data_type_attr):
> ? ? ? ? ? ? ? ? ? ?_result_type = type(self)
> ? ? ? ? ? ? ? ? ? ?def _broadcast(*args, **kwds):
> ? ? ? ? ? ? ? ? ? ? ? ?_result_type(data_type_attr(x, *args, **kwds)
> for x in self)
> ? ? ? ? ? ? ? ? ? ?return _broadcast
> ? ? ? ? ? ? ? ?return data_type_attr
> ? ? ? ?return TypedContainer

Something along those lines. Again I feel people have latched on to
one element of what I proposed here, to the detriment of the proposal
as a whole.

> I think it will have a lot of problems in practice (note that NumPy
> doesn't try to solve the broadcasting problem in general, just for a
> single specific data type), but, if the concept has any merit at all,
> that's certainly something that can be demonstrated quite adequately
> on PyPI.

This may be the case, thus my request for input.  I agree that a
thoughtful approach is prudent, however as I have stated, as it
currently stands the language would not be very supportive of an
add-on module that does this.  With PyPy, I could probably get my
hooks in deeply enough that I could make something really useful,
however given CPython is the current flavor du jour, I doubt it would
get much traction.  Kind of hard to sell someone on syntactic sugar
when they have to constantly wrap return values.

> To get a better idea of the level of evidence you're trying to reach
> if your suggestion is ever going to get anywhere, try taking a look at
> http://www.boredomandlaziness.org/2011/02/justifying-python-language-changes.html
> and http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html.

I agree that changes to syntax and commonly used modules that impact
how people interface with them should be carefully vetted.  Type
decorations on homogeneous collections/iterators are effectively
invisible in that perspective though; the main problem with them as I
see it is that it involves touching a lot of code to implement, even
if the actual implementation would be simple.

I appreciate your feedback,

Nathan


From ethan at stoneleaf.us  Mon Dec 19 15:10:09 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 19 Dec 2011 06:10:09 -0800
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
Message-ID: <4EEF45C1.7030201@stoneleaf.us>

Nathan Rice wrote:
> To avoid handcuffing people with types (which is definitely un-pythonic)
> and maintain backwards compatibility, the standard collection
> modification methods could be hooked so that if an object of an
> incorrect type is added, a warning is raised and the collection
> gracefully degrades by removing mixed-in type information and
> methods.  

-1 to automatic downgrading.  Imagine in step C that the collection is 
auto-downgraded and later in step Q a method is called that no longer 
exists -- Exception raised.  (I find a Warning less than useful -- 
either you aren't going to use one of the special methods, in which case 
you didn't need the special container and the warning is just noise, or 
you do need the special methods and you'll get an exception further on 
when you try to use it and it's no longer there.)

> Additionally, a method could be provided that lets the user
> "terminate the contract" causing the collection to degrade without a
> warning.

+1  Either it was not needed to begin with, or the need has been 
satisfied and now we need to add in objects with a different type



Do you have some examples of functions where you pass in a container 
object and get the same type of object back, but it's a different object?

~Ethan~


From nathan.alexander.rice at gmail.com  Mon Dec 19 16:47:05 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Mon, 19 Dec 2011 10:47:05 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <4EEF45C1.7030201@stoneleaf.us>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<4EEF45C1.7030201@stoneleaf.us>
Message-ID: <CAOFbRmK-t3mBQjgWnz-1qfmRtTY9+BEQTbGtiDtqezAFn24fdg@mail.gmail.com>

>> To avoid handcuffing people with types (which is definitely un-pythonic)
>> and maintain backwards compatibility, the standard collection
>> modification methods could be hooked so that if an object of an
>> incorrect type is added, a warning is raised and the collection
>> gracefully degrades by removing mixed-in type information and
>> methods.
>
>
> -1 to automatic downgrading. ?Imagine in step C that the collection is
> auto-downgraded and later in step Q a method is called that no longer exists
> -- Exception raised. ?(I find a Warning less than useful -- either you
> aren't going to use one of the special methods, in which case you didn't
> need the special container and the warning is just noise, or you do need the
> special methods and you'll get an exception further on when you try to use
> it and it's no longer there.)

Personally, I like warnings.  When my code passes tests, and I see a
warning, often times it clues me in to an additional edge case or
subtle bug.

What are the other options?  Throw an exception immediately, or don't
downgrade at all, then throw an exception when someone tries to use a
method that doesn't exist on a child?  Throwing an exception
immediately forces people to be explicit which is good, but it isn't
backwards compatible.  Not downgrading at all seems like it could be a
source of annoyingly subtle bugs, since it is possible that you could
put something in the list that fulfills part of the contract, but not
all of it, or fulfills the contract in a subtly incorrect way.

>> Additionally, a method could be provided that lets the user
>> "terminate the contract" causing the collection to degrade without a
>> warning.
>
>
> +1 ?Either it was not needed to begin with, or the need has been satisfied
> and now we need to add in objects with a different type
>
>
>
> Do you have some examples of functions where you pass in a container object
> and get the same type of object back, but it's a different object?

The restriction on being the same type is not needed.  If a list is
homogeneous, an iterator or set derived from the list would be
homogeneous, and keeping that information in tact would probably be a
good thing.

With the slight adjustment, list, tuple, set, filter, iter (+most of
itertools), sorted, reversed and heapq off the top of my head.
Looking at the list, you could give type decorations legs just by
having iterables of decorated objects retain the decoration, and
having the basic collection types inherit type decorations at
construction.


From storchaka at gmail.com  Mon Dec 19 18:12:03 2011
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 19 Dec 2011 19:12:03 +0200
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
Message-ID: <jcnr9a$oef$1@dough.gmane.org>

19.12.11 15:52, Nathan Rice ???????(??):
>>> L2 = [X(e) for e in L1]>>  L3 = [Y(e) for e in L2]>>  vs>>  L2 = X(L1) # assuming X has been updated to work in both vector/scalar>>  L3 = Y(L2) # context...>>>  L = ['a', 'bc', ['ada', 'a']]>>  What is len(L)? 3 or [1, 2, 2] or [1, 2, [3, 1]]?>>>>  L2 = [Z(Y(X(e))) for e in L1]>>  vs>>  L2 = Z(Y(X(L1)))>>>>  L2 = [e.X().Y().Z() for e in L1]>>  vs>>  L2 = L1.X().Y().Z() # assuming vectorized versions of member methods>>  #are folded into the collection via the mixin.>>>  What is L.count('a')? 1 or [1, 0, 1] or [1, 0, [2, 1]]?
>
> A fair concern; if the vectorized version of the child method were
> given the same name as the child method, I agree that this could
> result in ambiguity.
>
> There are multiple ways that member methods could be made available on
> the collection, including a proxy attribute, renaming, etc.

len is not method, it is function (although it uses method __len__). 
There are many functions applicable to list and to its elements (in 
particular when members of list are lists).



From nathan.alexander.rice at gmail.com  Mon Dec 19 18:47:55 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Mon, 19 Dec 2011 12:47:55 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jcnr9a$oef$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
	<jcnr9a$oef$1@dough.gmane.org>
Message-ID: <CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>

>>>> L2 = [X(e) for e in L1]>> ?L3 = [Y(e) for e in L2]>> ?vs>> ?L2 = X(L1) #
>>>> assuming X has been updated to work in both vector/scalar>> ?L3 = Y(L2) #
>>>> context...>>> ?L = ['a', 'bc', ['ada', 'a']]>> ?What is len(L)? 3 or [1, 2,
>>>> 2] or [1, 2, [3, 1]]?>>>> ?L2 = [Z(Y(X(e))) for e in L1]>> ?vs>> ?L2 =
>>>> Z(Y(X(L1)))>>>> ?L2 = [e.X().Y().Z() for e in L1]>> ?vs>> ?L2 =
>>>> L1.X().Y().Z() # assuming vectorized versions of member methods>> ?#are
>>>> folded into the collection via the mixin.>>> ?What is L.count('a')? 1 or [1,
>>>> 0, 1] or [1, 0, [2, 1]]?
>>
>>
>> A fair concern; if the vectorized version of the child method were
>> given the same name as the child method, I agree that this could
>> result in ambiguity.
>>
>> There are multiple ways that member methods could be made available on
>> the collection, including a proxy attribute, renaming, etc.
>
>
> len is not method, it is function (although it uses method __len__). There
> are many functions applicable to list and to its elements (in particular
> when members of list are lists).

I am aware of this.  I was talking about the "collection inside a
collection" example after that.  Additionally, now I realize my
initial reading of your example was too hasty. You violate the
contract in the mixed list example with strings and lists, and that
should rightfully not provide vectorized methods; providing type
decorators for very abstract notions such as "iterable" is a bad idea
imho.  Of course, [[1, 2], [3, 4], [5, 6]] could serve as an alternate
example of why you can't directly map child methods with the same
names.


From ethan at stoneleaf.us  Mon Dec 19 19:01:06 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 19 Dec 2011 10:01:06 -0800
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>	<jcmt7i$ras$1@dough.gmane.org>	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>	<jcnr9a$oef$1@dough.gmane.org>
	<CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>
Message-ID: <4EEF7BE2.4030203@stoneleaf.us>

Nathan Rice wrote:
> [[1, 2], [3, 4], [5, 6]] could serve as an alternate
> example of why you can't directly map child methods with the same
> names.

If you have to change the names, doesn't that negate the ability of "a 
reliable method to make functions operate on both scalar and
vector values"?

~Ethan~


From nathan.alexander.rice at gmail.com  Mon Dec 19 20:26:34 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Mon, 19 Dec 2011 14:26:34 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <4EEF7BE2.4030203@stoneleaf.us>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
	<jcnr9a$oef$1@dough.gmane.org>
	<CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>
	<4EEF7BE2.4030203@stoneleaf.us>
Message-ID: <CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>

On Mon, Dec 19, 2011 at 1:01 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> Nathan Rice wrote:
>>
>> [[1, 2], [3, 4], [5, 6]] could serve as an alternate
>> example of why you can't directly map child methods with the same
>> names.
>
>
> If you have to change the names, doesn't that negate the ability of "a
> reliable method to make functions operate on both scalar and
> vector values"?

To clarify...

Functions could reliably support both vector and scalar context, by
checking that a type contract exists.

Because of the name clash between parent and elementwise member
methods when dealing with collections of collections (and some other
classes), if the type contract provided "broadcast" versions of child
methods, they would have to be provided with under an alias, perhaps X
-> elementwise_X.  I do not think people should be able to be
oblivious of when they are calling collection and member methods, that
encourages sloppy programming.  I like the broadcast feature because
it makes it easier to write code in a clear left to right narrative
style, and that seems very pythonic to me compared with an
"inside-out" style.


From ethan at stoneleaf.us  Mon Dec 19 21:09:25 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 19 Dec 2011 12:09:25 -0800
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>	<jcmt7i$ras$1@dough.gmane.org>	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>	<jcnr9a$oef$1@dough.gmane.org>	<CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>	<4EEF7BE2.4030203@stoneleaf.us>
	<CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>
Message-ID: <4EEF99F5.9050300@stoneleaf.us>

Nathan Rice wrote:
> On Mon, Dec 19, 2011 at 1:01 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
>> Nathan Rice wrote:
>>> [[1, 2], [3, 4], [5, 6]] could serve as an alternate
>>> example of why you can't directly map child methods with the same
>>> names.
>>
>> If you have to change the names, doesn't that negate the ability of "a
>> reliable method to make functions operate on both scalar and
>> vector values"?
> 
> To clarify...
> 
> Functions could reliably support both vector and scalar context, by
> checking that a type contract exists.

That sounds like type checking, which is not Pythonic.  Duck-typing, on 
the other hand, *is*.

def capitalize_me(some_obj):
     some_obj = some_obj.capitalize()

--> capitalize_me("a simple string")
"A simple string"

--> capitalize_me(["a", "list", "of", "strings"])
["A", "List", "Of", "Strings"]


> Because of the name clash between parent and elementwise member
> methods when dealing with collections of collections (and some other
> classes), if the type contract provided "broadcast" versions of child
> methods, they would have to be provided with under an alias, perhaps X
> -> elementwise_X.  I do not think people should be able to be
> oblivious of when they are calling collection and member methods, that
> encourages sloppy programming.  I like the broadcast feature because
> it makes it easier to write code in a clear left to right narrative
> style, and that seems very pythonic to me compared with an
> "inside-out" style.

Duck-typing is neither oblivious, nor sloppy -- at least, not 
inherently.  Is my sample code above not what you had in mind?

~Ethan~


From joshua.landau.ws at gmail.com  Mon Dec 19 21:37:58 2011
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Mon, 19 Dec 2011 20:37:58 +0000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
	<jcnr9a$oef$1@dough.gmane.org>
	<CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>
	<4EEF7BE2.4030203@stoneleaf.us>
	<CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>
Message-ID: <CAN1F8qXaF8isLrP5Fb24=kjFx9H0D2QKwOdUT+bOv+PF=igY1g@mail.gmail.com>

On 19 December 2011 19:26, Nathan Rice <nathan.alexander.rice at gmail.com>wrote:

> On Mon, Dec 19, 2011 at 1:01 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> > Nathan Rice wrote:
> >>
> >> [[1, 2], [3, 4], [5, 6]] could serve as an alternate
> >> example of why you can't directly map child methods with the same
> >> names.
> >
> >
> > If you have to change the names, doesn't that negate the ability of "a
> > reliable method to make functions operate on both scalar and
> > vector values"?
>
> To clarify...
>
> Functions could reliably support both vector and scalar context, by
> checking that a type contract exists.
>
> Because of the name clash between parent and elementwise member
> methods when dealing with collections of collections (and some other
> classes), if the type contract provided "broadcast" versions of child
> methods, they would have to be provided with under an alias, perhaps X
> -> elementwise_X.  I do not think people should be able to be
> oblivious of when they are calling collection and member methods, that
> encourages sloppy programming.  I like the broadcast feature because
> it makes it easier to write code in a clear left to right narrative
> style, and that seems very pythonic to me compared with an
> "inside-out" style.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>

...or we could just extend Pep
225<http://www.python.org/dev/peps/pep-0225/> (deferred) with
"~." so we have "['a', 'b', 'c']~.upper()" [syntax debatable]. You don't
get the type-checking but that seemed more of a problem to me, as I'm not a
real duck.

Your example:
my_string.split("\n").capitalize().join_items("\n")
to
"\n".join(my_string.split("\n")~.capitalize())

And personally I like "string.join(lst)" over "lst.join_with(string)"...

-- Joshua, pointlessly interjecting once again
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111219/16a48116/attachment.html>

From nathan.alexander.rice at gmail.com  Mon Dec 19 23:28:09 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Mon, 19 Dec 2011 17:28:09 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAN1F8qXaF8isLrP5Fb24=kjFx9H0D2QKwOdUT+bOv+PF=igY1g@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
	<jcnr9a$oef$1@dough.gmane.org>
	<CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>
	<4EEF7BE2.4030203@stoneleaf.us>
	<CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>
	<CAN1F8qXaF8isLrP5Fb24=kjFx9H0D2QKwOdUT+bOv+PF=igY1g@mail.gmail.com>
Message-ID: <CAOFbRmL42mqb+pvm1ueXGx4R8kxwEaWED=Y1mWpCDLNW=4zdPw@mail.gmail.com>

>> Functions could reliably support both vector and scalar context, by>> checking that a type contract exists.>>> That sounds like type checking, which is not Pythonic. ?Duck-typing, on the> other hand, *is*.
While you'll hear no argument from me on duck typing, in this instance
that collides head-on with having clear names for elementwise versions
of member methods.  I personally would feel better erring on the side
of clearer names at the cost of some kind of *very cursory*
examination of what you are being passed.  Exactly what form that
examination might take is an open question that deserves further
thought.

> def capitalize_me(some_obj):> ? ?some_obj = some_obj.capitalize()>> --> capitalize_me("a simple string")> "A simple string">> --> capitalize_me(["a", "list", "of", "strings"])> ["A", "List", "Of", "Strings"]
Though I don't want to comment on the definition of capitalize_me, in
terms of function behavior across the string and list, that is pretty
much what I am talking about.

>> Because of the name clash between parent and elementwise member>> methods when dealing with collections of collections (and some other>> classes), if the type contract provided "broadcast" versions of child>> methods, they would have to be provided with under an alias, perhaps X>> -> elementwise_X. ?I do not think people should be able to be>> oblivious of when they are calling collection and member methods, that>> encourages sloppy programming. ?I like the broadcast feature because>> it makes it easier to write code in a clear left to right narrative>> style, and that seems very pythonic to me compared with an>> "inside-out" style.>>> Duck-typing is neither oblivious, nor sloppy -- at least, not inherently.> ?Is my sample code above not what you had in mind?
I don't think duck typing is sloppy, with the caveat that the thing
you expect to behave like a duck is some kind of animal, reasonably
crafted animatronic robot, or operated puppet :).  I feel like when
you move from a single object to a collection that is basically like
having a second function, which has been integrated in with the first
function for the purposes of API simplicity.  Then, it becomes an
outer function that has the job of returning the correct inner
function and executing it with the given input.

---

On Mon, Dec 19, 2011 at 3:37 PM, Joshua Landau
<joshua.landau.ws at gmail.com> wrote:
> ...or we could just extend Pep 225?(deferred)?with "~." so we have "['a',
> 'b', 'c']~.upper()" [syntax debatable]. You don't get the type-checking but
> that seemed more of a problem to me, as I'm not a real duck.

That does run into some of the same problems w.r.t duck typing, and
solutions to problems with overloaded operators are hard to google
for.  If the method calls were aliased, google searches would be
easier and less experienced users wouldn't have to worry about
wrestling with operator overloading.  That being said, I think
elementwise operators are a great idea in a numeric/scientific
context.

There are of course both pros and cons to having type declarations.  I
know that people do not like to be limited by a lack of foresight in
their predecessors; I have run into unreasonable interface/type
specifications and annoyances with  private/protected variables in
Java and I would never want to see that infiltrate Python.  I think
what I am trying to sell here is more akin to metadata than static
types.  People should be able to basically keep doing what they are
already doing, but have a reasonable mechanism to provide additional
object metadata in a standard way that is easily accessible by
downstream consumers.  That gives you a lot of the upsides of type
declarations while staying Pythonic.

Maybe a better option than having a "type contract" base for
homogeneous collections is to give objects a metadata namespace with a
loose schema, and declare it there?  As long as you could provide the
metadata at object creation time you would still get all the same
benefits, and it would avoid potentially overworking the class/mixin
concept.

> Your example:
> my_string.split("\n").capitalize().join_items("\n")
> to
> "\n".join(my_string.split("\n")~.capitalize())
>
> And personally I like "string.join(lst)" over "lst.join_with(string)"...

Of course, everyone has their own preferences; having a very small set
of internally consistent ways to do something rather than just one way
is good for this reason.


Nathan


From ncoghlan at gmail.com  Tue Dec 20 00:07:55 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Dec 2011 09:07:55 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
Message-ID: <CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>

On Tue, Dec 20, 2011 at 12:30 AM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> 1. The "broadcasting" that people seemed to have latched on to is only
> part of what I put forward, and I agree it is something that would
> have to be done *correctly* to be beneficial. ?I would have no issues
> with providing a userspace lib to do this if type decorations were
> included in homogeneous collections/iterables, as long as the
> implementation of the decoration didn't suffer from some form of
> "string failure" (string subclasses are basically worthless as methods
> return strings, not an instance of the class).

I deliberately ignored everything else, because the broadcasting
aspect is the only part that doesn't reek of "Python, why u no use
static typing?" and hence the only part I find particularly
interesting.

If you reframe the typing ideas in terms of Abstract Base Classes and
generation of typed proxies, the rest of it may become interesting.

> 4. I wanted to gauge people's feelings before I went through the steps
> involved in writing a PEP. ?I believe that is the right thing to do,
> so I don't feel the "hand waving" comment is warranted. ?I've already
> learned people view collections that provide child object methods in
> vector form as a very big change even if it is backwards compatible;
> that is fine, I'm willing to shelve that if consensus is that people
> aren't comfortable with it.

The "Hand waving" comment is absolutely warranted, because without
code, we don't really know what you mean.

> This may be the case, thus my request for input. ?I agree that a
> thoughtful approach is prudent, however as I have stated, as it
> currently stands the language would not be very supportive of an
> add-on module that does this. ?With PyPy, I could probably get my
> hooks in deeply enough that I could make something really useful,
> however given CPython is the current flavor du jour, I doubt it would
> get much traction. ?Kind of hard to sell someone on syntactic sugar
> when they have to constantly wrap return values.

Everything the numpy folks have done, they have done with the type
system as it currently stands. It would be perfectly possible to
create proxies for objects that implement the container ABCs that
provide the type constraints that you describe, all without touching
the language core or standard library.

If you take full advantage of the power that descriptors and
metaclasses offer, then you can do a *lot* with this concept as a PyPI
module. As a specific suggestion, I'd advise exploring a
"container.broadcast" descriptor on a mixin type that let you do
things like:

L2 = L1.broadcast(X, *args, **kwds)
L3 = L2.broadcast(Y, *args, **kwds)

L2 = L1.broadcast(X).broadcast(Y).broadcast(Z)
L2 = L1.broadcast_chain(X)(Y)(Z).apply()

L2 = L1.broadcast.X().broadcast.Y().broadcast.Z()
L2 = L1.broadcast_chain.X().Y().Z().apply()

my_string.split("\n").broadcast.capitalize().join_items("\n")

> I agree that changes to syntax and commonly used modules that impact
> how people interface with them should be carefully vetted. ?Type
> decorations on homogeneous collections/iterators are effectively
> invisible in that perspective though; the main problem with them as I
> see it is that it involves touching a lot of code to implement, even
> if the actual implementation would be simple.

No, they're not invisible at all - they're a massive conceptual
addition, on a similar scale to Abstract Base Classes themselves.
Syntax matters for readability purposes, but it's the *semantics* that
is at issue here (and also in many other PEPs).

We're talking years of exploration and debate here, not something that
can be resolved in a few weeks or months. There's zero chance of
anything like this making it into 3.3., so think 3.4 at the earliest,
and more likely 3.5 (and that's only if coded explorations of the
concept prove useful in practice - it's entirely possible that the
real outcome will be "never").

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From tjreedy at udel.edu  Tue Dec 20 00:10:55 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 19 Dec 2011 18:10:55 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
Message-ID: <jcogac$i6o$1@dough.gmane.org>

On 12/18/2011 9:24 PM, Nathan Rice wrote:

>> Another bad example, since that's just a really verbose way of writing
>> my_string.capitalize().
>
> The python interpreter says otherwise...
>
>>>> foo = "line 1\nline 2\nline 3"
>>>> foo.capitalize()
> 'Line 1\nline 2\nline 3'
>>>> "\n".join(s.capitalize() for s in foo.split("\n"))
> 'Line 1\nLine 2\nLine 3'

Using the right method (as Nick corrected himself):
 >>> foo.title()
'Line 1\nLine 2\nLine 3'

-- 
Terry Jan Reedy



From ncoghlan at gmail.com  Tue Dec 20 00:26:49 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Dec 2011 09:26:49 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jcogac$i6o$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<jcogac$i6o$1@dough.gmane.org>
Message-ID: <CADiSq7cj-VSMFd9uPx+KKt-32MV93SWdMJbdB3KM2QWGRdo=wQ@mail.gmail.com>

On Tue, Dec 20, 2011 at 9:10 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/18/2011 9:24 PM, Nathan Rice wrote:
>
>>> Another bad example, since that's just a really verbose way of writing
>>> my_string.capitalize().
>>
>>
>> The python interpreter says otherwise...
>>
>>>>> foo = "line 1\nline 2\nline 3"
>>>>> foo.capitalize()
>>
>> 'Line 1\nline 2\nline 3'
>>>>>
>>>>> "\n".join(s.capitalize() for s in foo.split("\n"))
>>
>> 'Line 1\nLine 2\nLine 3'
>
>
> Using the right method (as Nick corrected himself):
>>>> foo.title()
>
> 'Line 1\nLine 2\nLine 3'

Nope, foo.title() isn't exactly the same - Nathan's variant
capitalises the first letter of each line, foo.title() would
capitalise the first letter of each word.

Not a common operation, but split + capitalise + join is likely the
simplest way to spell it.

</tangent>

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From joshua.landau.ws at gmail.com  Tue Dec 20 02:05:14 2011
From: joshua.landau.ws at gmail.com (Joshua Landau)
Date: Tue, 20 Dec 2011 01:05:14 +0000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmL42mqb+pvm1ueXGx4R8kxwEaWED=Y1mWpCDLNW=4zdPw@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<1ec1a14b-f521-4e01-b3f5-f60e1ed0bab9@q11g2000vbq.googlegroups.com>
	<5e760100-547c-4208-b47e-26277ec94943@z17g2000vbe.googlegroups.com>
	<jcmt7i$ras$1@dough.gmane.org>
	<CAOFbRmJ7CxdB_omQRDoaLOzw-Ez2a0uhA2isGYxAKbhY86rLOw@mail.gmail.com>
	<CAOFbRmJ0YO7ghwUbcd5tz8eUAphbpFsVP2Mn+d0s29S7F9gh_A@mail.gmail.com>
	<jcnr9a$oef$1@dough.gmane.org>
	<CAOFbRm+ghMjY3Yy5aHTWWT_e=OoR6x4dgGvGi962BBoQbkrRFw@mail.gmail.com>
	<4EEF7BE2.4030203@stoneleaf.us>
	<CAOFbRmLo8rzp3BmOvH3AR0NmkhUtZJ4WaKh_uAWv9Bpeei7mXQ@mail.gmail.com>
	<CAN1F8qXaF8isLrP5Fb24=kjFx9H0D2QKwOdUT+bOv+PF=igY1g@mail.gmail.com>
	<CAOFbRmL42mqb+pvm1ueXGx4R8kxwEaWED=Y1mWpCDLNW=4zdPw@mail.gmail.com>
Message-ID: <CAN1F8qV_HTwucs6CqfSwmarJAKpmJ=cdVGTw44ZmdBCCgZnN8g@mail.gmail.com>

On 19 December 2011 22:28, Nathan Rice <nathan.alexander.rice at gmail.com>wrote:

> On Mon, Dec 19, 2011 at 3:37 PM, Joshua Landau
> <joshua.landau.ws at gmail.com> wrote:
> > ...or we could just extend Pep 225 (deferred) with "~." so we have "['a',
> > 'b', 'c']~.upper()" [syntax debatable]. You don't get the type-checking
> but
> > that seemed more of a problem to me, as I'm not a real duck.
>
> That does run into some of the same problems w.r.t duck typing, and
> solutions to problems with overloaded operators are hard to google
> for.  If the method calls were aliased, google searches would be
> easier and less experienced users wouldn't have to worry about
> wrestling with operator overloading.


The thing is, "listofstrings.method()" is less obviously elementwise than
"listofstrings~.method()", *especially* if "~" becomes the default
elementwise character. So I have to disagree. Additionally,
"listofstrings~.method()" is easy to paste into an interpreter, and should
quickly yield enlightenment. I don't see that as a valid argument.


> That being said, I think
> elementwise operators are a great idea in a numeric/scientific
> context.
>

But not always in the same case as your broadcasting idea? The elementwise
operator is far more general, so I'd say it has a more general use-case in
response. Only a fraction of my code is of the "numeric/scientific" type,
and I'd lap up elementwise operators.


> There are of course both pros and cons to having type declarations.  I
> know that people do not like to be limited by a lack of foresight in
> their predecessors; I have run into unreasonable interface/type
> specifications and annoyances with  private/protected variables in
> Java and I would never want to see that infiltrate Python.  I think
> what I am trying to sell here is more akin to metadata than static
> types.  People should be able to basically keep doing what they are
> already doing, but have a reasonable mechanism to provide additional
> object metadata in a standard way that is easily accessible by
> downstream consumers.  That gives you a lot of the upsides of type
> declarations while staying Pythonic.
>

The thing is... what? This isn't a criticism. I'm genuinely lost as to what
you mean. I lost you on the "metadata" part. In what way is this metadata,
and in turn not typing? Can you do:
foo = "a b".split()
foo[1] = 1

If you can, then this whole idea should really be generalised more,
preferably to the elementwise operator level, as it's got no real type
constraints. If you can't, then how is it not typing? Again, this is a
genuine question. I'm home-taught, so please don't expect me to know the
lingo :P.

I actually love the idea of typing when done well. I've just started
Haskell (quite a shock to my Python'd brain) and I've
thoroughly enjoyed its type system. I just think that it has no place in
python. It only works in Haskell because Haskell is built on it, and types
can't go a-changin' (nor can values, for that matter). In python, this sort
of system will mean that typed code/arrays/iterators will severely grind
with untyped code/values/arrays/iterators.

Someone defines a 'typed collection or iterator', to paraphrase the title.
Someone else makes a goose-that-looks-like-a-duck, but doesn't have a beak.
With an untyped system - such as the current method or an elementwise
operator - that's fine as you just want it swim, but with your typed
container something has to go wrong! Either it crashed, which is
anti-duck-typing, or it falls back down, in which case you end up reverting
everything to the old method anyway.

Haskell gets around this by having both types and 'things that implement
things' and so could do this because it knows what everything implements.
But in python we don't know, and won't until all our dusty code is
rewritten. I apologise for using a language I've just started to learn as a
reference, but that's the best reference I've had.

*I really should check this, but it's late and I'm tired. If I've said the
same thing twice or rambled on about hexagonal motorised paintings for too
long, blame my cat...*

 Maybe a better option than having a "type contract" base for
> homogeneous collections is to give objects a metadata namespace with a
> loose schema, and declare it there?  As long as you could provide the
> metadata at object creation time you would still get all the same
> benefits, and it would avoid potentially overworking the class/mixin
> concept.
>
> > Your example:
> > my_string.split("\n").capitalize().join_items("\n")
> > to
> > "\n".join(my_string.split("\n")~.capitalize())
> >
> > And personally I like "string.join(lst)" over "lst.join_with(string)"...
>
> Of course, everyone has their own preferences; having a very small set
> of internally consistent ways to do something rather than just one way
> is good for this reason.
>
>
> Nathan
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>

Before anyone gets too many ideas about what is and isn't:
 >>> str.capitalize("ABCDEFGH")
'Abcdefgh'

Remember that these are predominantly lowercasing actions :P
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111220/7ff500af/attachment.html>

From tjreedy at udel.edu  Tue Dec 20 02:14:49 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 19 Dec 2011 20:14:49 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
Message-ID: <jconin$vl1$1@dough.gmane.org>

On 12/19/2011 9:30 AM, Nathan Rice wrote:

> Couple things.
>
> 1. The "broadcasting" that people seemed to have latched on to is only
> part of what I put forward,

Perhaps because it is the most understandable.

> and I agree it is something that would
> have to be done *correctly* to be beneficial.  I would have no issues
> with providing a userspace lib to do this if type decorations were
> included in homogeneous collections/iterables,

The meaning of 'homogeneous' depends on the context -- the purpose and 
use of the collection. For some purposes -- str(o), len(c), o in c, 
c.index(o), and others,  -- all objects, collections, or seqeuences 
*are* 'homogeneous' as instances or subclasses of 'object'. On the other 
hand, even [-1, 0, 1] is heterogeneous with respect to both sqrt and 
log, with the divide different for each. So I do not consider 
'homogeneous' to be a property of collections as such.

Python's current restricted-type mutable sequence factory is 
array.array. The types do not even have to be Python types, just machine 
storage types. The typecode is part of the object and exposed as an 
attribute. Such sequences cannot be 'degraded' because type-checking is 
done with all operations. It would not be difficult to make a TypedList 
class that did the same, either subclassing or wrapping list.

What you have noticed is that iter(array(tc,init)) does not get the 
typecode information, so potentially useful information is lost. Your 
first concrete proposal might be that the information be kept and that 
arrayiterators get a type attribute corresponding to the Python type 
that the produced values are converted to. Also, array could expose the 
mapping to typecodes to Python types. These changes would allow 
experiments that would show the value of your basic idea.

> as long as the
> implementation of the decoration didn't suffer from some form of
> "string failure" (string subclasses are basically worthless as methods
> return strings, not an instance of the class).

This problem is generic to subclassing built-in classes. List would be a 
better example here since strings already are specialized sequences.

> 2. A "type decorator" on homogeneous collections and iterables has a
> lot of nice little benefits throughout the toolchain.

That is what you need to demonstrate, because it does not seem clear 
yet. What would you do with an arrayiterator with a type attribute.

By the way, a 'decorator' in Python is a specific category of callable 
used in a specific way. Perhaps you mean 'type attribute'?

> 3. Being able to place methods on a "type decorator" is useful,

'Placing methods' on an attribute or even a callable does not mean much.
You can only concretely add methods to concrete classes, not abstract 
categories.

> it solves issues like "foo".join() which really wants to be a method on
> string collections.

No it does not. 'String collection' is a category, not a class. Nor can 
it be a class without drastically revising Python. It is a category that 
cuts across all generic collection classes. So .join has to be a method 
of the joiner class.

> 4. I wanted to gauge people's feelings before I went through the steps
> involved in writing a PEP.  I believe that is the right thing to do,
> so I don't feel the "hand waving" comment is warranted.

To the extent one does not understand what you say, and to the extent 
that it seems disconnected from concrete reality, it is easy to see it 
as hand waving. That you perhaps did not understand why .join is a 
string method points in that direction.

>  I've already
> learned people view collections that provide child object methods in
> vector form as a very big change

Because we understand that non-method functions have virtues, and Python 
already has collection functions.

 > even if it is backwards compatible; that is fine.

Backwards compatible duplication needs justification.
...
> I agree that changes to syntax and commonly used modules that impact
> how people interface with them should be carefully vetted.  Type
> decorations on homogeneous collections/iterators are effectively

I am still not sure what you are really proposing. You may have the germ 
of a useful idea, but I think it needs clarification and a demonstration.

> invisible in that perspective though;

Slowdowns are not invisible. Requiring a type check on every addition to 
every built-in collection might result in such.

> the main problem with them as I
> see it is that it involves touching a lot of code to implement, even
> if the actual implementation would be simple.

Changes that touch a lot of code are fairly rare and require major 
benefits. One was the switch to new-style classes started in 2.2 and 
ended in 3.0. Several people contributed patches. They must have thought 
that unifying types and classes into one system was worth it.

In 3.3, the two unicode implementations (one per build) are effectively 
combined with a third with a new C-level API. Adding and tweaking the 
new API (which continues today) and converting the entire C core and 
stdlib codebase to the new API has required something on the order of 50 
patches over 3 months, so far. But it improves performance (overall) and 
removed the inherent bugs in representing 3-bytes chars with 2 2-byte 
chars and in having different Python builds respond differently to the 
same code. Note that the PEP concretely lays out the new C structures 
and API and that there was a prototype implementation showing benefits 
before it was approved.

-- 
Terry Jan Reedy



From pytom at yahoo.cn  Tue Dec 20 04:07:20 2011
From: pytom at yahoo.cn (Tom Zhou)
Date: Tue, 20 Dec 2011 11:07:20 +0800 (CST)
Subject: [Python-ideas] Columnize in module "cmd"
In-Reply-To: <1324349593.51908.YahooMailNeo@web92411.mail.cnh.yahoo.com>
References: <1324349593.51908.YahooMailNeo@web92411.mail.cnh.yahoo.com>
Message-ID: <1324350440.30993.YahooMailNeo@web92413.mail.cnh.yahoo.com>



Hi~ alls

recently, i focus on the module "cmd", and find some confused things-- the function named "columnize". Why we need a multiloop as
"for nrows ..
? ? ? ? for col ..
? ? ? ? ? ? ? ? ? ?for row.."
?? i think we can make a easier method, for example, first, find out the maxlen str in list, and use its length as the standard size to format the list.
Ok, maybe i ignore something, so please give me some hints.


---
thanks
tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111220/bfc4856f/attachment.html>

From storchaka at gmail.com  Tue Dec 20 10:25:41 2011
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 20 Dec 2011 11:25:41 +0200
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
Message-ID: <jcpkal$rav$1@dough.gmane.org>

20.12.11 01:07, Nick Coghlan ???????(??):
> L2 = L1.broadcast(X, *args, **kwds)
> L3 = L2.broadcast(Y, *args, **kwds)
>
> L2 = L1.broadcast(X).broadcast(Y).broadcast(Z)
> L2 = L1.broadcast_chain(X)(Y)(Z).apply()
>
> L2 = L1.broadcast.X().broadcast.Y().broadcast.Z()
> L2 = L1.broadcast_chain.X().Y().Z().apply()
>
> my_string.split("\n").broadcast.capitalize().join_items("\n")

from idontlikecomprehensions import broadcast, broadcast_chain
L2 = broadcast(L1).X(*args, **kwds)
L2 = broadcast_chain(L1).X().Y().Z().apply()



From guido at python.org  Tue Dec 20 16:14:38 2011
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Dec 2011 08:14:38 -0700
Subject: [Python-ideas] Columnize in module "cmd"
In-Reply-To: <1324350440.30993.YahooMailNeo@web92413.mail.cnh.yahoo.com>
References: <1324349593.51908.YahooMailNeo@web92411.mail.cnh.yahoo.com>
	<1324350440.30993.YahooMailNeo@web92413.mail.cnh.yahoo.com>
Message-ID: <CAP7+vJL9RRUCKhuqA7T6OLSJOsjtc9p74Yk7fwHrxwh=VJDfuw@mail.gmail.com>

That code is trying to find an optimal solution where the columns may be of
different width.

Good luck!

--Guido

On Mon, Dec 19, 2011 at 8:07 PM, Tom Zhou <pytom at yahoo.cn> wrote:

>
> Hi~ alls
>  recently, i focus on the module "cmd", and find some confused things--
> the function named "columnize". Why we need a multiloop as
> "for nrows ..
>         for col ..
>                    for row.."
> ?? i think we can make a easier method, for example, first, find out the
> maxlen str in list, and use its length as the standard size to format the
> list.
> Ok, maybe i ignore something, so please give me some hints.
>
>
> ---
> thanks
> tom
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111220/bf7c2dfb/attachment.html>

From nathan.alexander.rice at gmail.com  Tue Dec 20 19:50:23 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Tue, 20 Dec 2011 13:50:23 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jcpkal$rav$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
Message-ID: <CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>

Nick:  My main issue with anything you've said is that I felt your
original statement about going off, creating a library, evangelizing
it vigorously until it is popular and coming back in several years as
a prerequisite to even engaging in a thoughtful discussion on the
subject was overly dismissive and phrased in a combative manner.  I do
appreciate the strong challenges you've presented though.

Terry:  My apologies for being unclear.  I do not take enough care in
using terminology in the standard way. In my case, I think code is a
better communication mechanism, so that is where I will be focusing
most of my efforts moving forward.


***Broadcasting/Elementwise operations***

I agree code speaks more clearly than words:

I put a module called "elementwise" on pypi
(http://pypi.python.org/pypi/elementwise/0.111220) that implements my
idea of what a nice broadcast proxy should do.  The proxy itself could
be leaner, I started out trying to make it in a very surgical manner,
but 1.) There are serious dragons in the way python handles lookup of
operator overloading special methods 2.) there are serious dragons in
how python handles complex inheritance graphs that result in
"object.__new__() takes no parameters", despite not having any builtin
bases and having no base class overriding __new__ or __init__ 3.)
Proxying EVERYTHING is pythonic in some sense.  I am aware that there
are probably issues around having some combination of special methods
defined that will confuse the interpreter.

The syntax is really simple:

your_iterable_elementwise = ElementwiseProxy(your_iterable)

your_iterable_elementwise.method_1().method_2().method_3()

(your_iterable_elementwise + 1) * 2 + 10

The __iter__ method is how you break the cycle,  so when you are done
working in an elementwise manner, just call
{list|set|tuple|...}(your_iterable_elementwise).  In place operator
modification should work as well.  All operations on an
ElementwiseProxy return another ElementwiseProxy with a parent
attribute that you can use to backtrack in the operation history, if
for some reason you were brought into the loop partway through.

I also included an ElementwiseProxyMixin that gives you a ruby-esque
class.each property for iterables, which is basically just returns a
"ElementwiseProxy(self)".  I think the syntax is nicer.


***I can haz moar Metadata (was "typed" collections)***

Because "typed" is sort of a dirty word in the python community, and
some people have had issues with other terms I have used, I am going
to settle on addressing things in term of the problem.  The problem is
metadata - languages with declared types have a lot of it, python has
a lot less.  Metadata is useful, and we should try to pack as much if
it in as we can, because it lets us do things in a intelligent ways on
a case by case basis.

I understand annotations are designed for this, but they are
absolutely pointless without some sort of schema or strong
conventions.  I see from the archived discussions that people wanted
to avoid a design by committee situation, but people working in the
tool-chain aren't going to mess with annotations unless there is one
metadata standard that is pretty much ubiquitous (at which point it
should be included in the stdlib anyhow).  I think it is a decent idea
to have it incubate outside the stdlib, but I don't see any progress
on that front at all.  With that being said, I would like to reopen
the discussion about an metadata annotation schema, so we can get an
incubation implementation going and begin evangelism.


From tjreedy at udel.edu  Wed Dec 21 00:01:26 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 20 Dec 2011 18:01:26 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
Message-ID: <jcr44j$onv$1@dough.gmane.org>

On 12/20/2011 1:50 PM, Nathan Rice wrote:

> I put a module called "elementwise" on pypi
> (http://pypi.python.org/pypi/elementwise/0.111220) that implements my
> idea of what a nice broadcast proxy should do.

I downloaded and took a brief look. I hope to get back to it later.

2.) there are serious dragons in
> how python handles complex inheritance graphs that result in
> "object.__new__() takes no parameters", despite not having any builtin
> bases and having no base class overriding __new__ or __init__

Best not to use object as the apex of multiple inheritance.

> Because "typed" is sort of a dirty word in the python community,

Not exactly true, and unnecessarily combative. More true is that 
careless use of 'typed' has gotten tiresome. Python is strongly 
dynamically typed. But people occasionally post -- again the same day 
you posted to python list -- that Python is weakly typed. I am tired of 
explaining that 'typed' is not synonymous with 'statically typed'.

Or consider your subject line. Python collections are typed both as to 
collection object and the contents. Python has narrow-content typed 
sequences. So just saying you want 'typed collections' does not say 
anything. We already have them. We even have propagation of 
narrow-content typing for operations on bytes and strings.

But we do not have anything similar for numbers or user classes. And 
that might be worthwhile. So your subject seems more like 'adding 
generic narrowly typed sequences to Python'.

-- 
Terry Jan Reedy



From steve at pearwood.info  Wed Dec 21 01:09:36 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 21 Dec 2011 11:09:36 +1100
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jcr44j$onv$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>	<jcpkal$rav$1@dough.gmane.org>	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
Message-ID: <4EF123C0.2010002@pearwood.info>

Terry Reedy wrote:

> 2.) there are serious dragons in
>> how python handles complex inheritance graphs that result in
>> "object.__new__() takes no parameters", despite not having any builtin
>> bases and having no base class overriding __new__ or __init__
> 
> Best not to use object as the apex of multiple inheritance.

Is that possible in Python 3?



>> Because "typed" is sort of a dirty word in the python community,
> 
> Not exactly true, and unnecessarily combative. More true is that 
> careless use of 'typed' has gotten tiresome. Python is strongly 
> dynamically typed. But people occasionally post -- again the same day 
> you posted to python list -- that Python is weakly typed. I am tired of 
> explaining that 'typed' is not synonymous with 'statically typed'.

+1



-- 
Steven



From jeanpierreda at gmail.com  Wed Dec 21 01:51:43 2011
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 20 Dec 2011 19:51:43 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jcr44j$onv$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
Message-ID: <CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>

> Not exactly true, and unnecessarily combative. More true is that careless
> use of 'typed' has gotten tiresome. Python is strongly dynamically typed.
> But people occasionally post -- again the same day you posted to python list
> -- that Python is weakly typed. I am tired of explaining that 'typed' is not
> synonymous with 'statically typed'.

I don't find this much less careless. How do you differentiate between
the "strong typing" of Python and the "strong typing" of Agda? It
isn't a binary quantity.

Perhaps, instead, we should stop claiming things are "strong" or
"weak". If I said that, relatively speaking, Python is weakly typed,
people would get offended -- not because I made any technically
incorrect statement (on the spectrum, Python is far closer to assembly
than Agda), but because to call it "weak" is insulting.

-- Devin

On Tue, Dec 20, 2011 at 6:01 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/20/2011 1:50 PM, Nathan Rice wrote:
>
>> I put a module called "elementwise" on pypi
>> (http://pypi.python.org/pypi/elementwise/0.111220) that implements my
>> idea of what a nice broadcast proxy should do.
>
>
> I downloaded and took a brief look. I hope to get back to it later.
>
>
> 2.) there are serious dragons in
>>
>> how python handles complex inheritance graphs that result in
>> "object.__new__() takes no parameters", despite not having any builtin
>> bases and having no base class overriding __new__ or __init__
>
>
> Best not to use object as the apex of multiple inheritance.
>
>
>> Because "typed" is sort of a dirty word in the python community,
>
>
> Not exactly true, and unnecessarily combative. More true is that careless
> use of 'typed' has gotten tiresome. Python is strongly dynamically typed.
> But people occasionally post -- again the same day you posted to python list
> -- that Python is weakly typed. I am tired of explaining that 'typed' is not
> synonymous with 'statically typed'.
>
> Or consider your subject line. Python collections are typed both as to
> collection object and the contents. Python has narrow-content typed
> sequences. So just saying you want 'typed collections' does not say
> anything. We already have them. We even have propagation of narrow-content
> typing for operations on bytes and strings.
>
> But we do not have anything similar for numbers or user classes. And that
> might be worthwhile. So your subject seems more like 'adding generic
> narrowly typed sequences to Python'.
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From tjreedy at udel.edu  Wed Dec 21 01:59:29 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 20 Dec 2011 19:59:29 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
Message-ID: <jcrb1v$1vg$1@dough.gmane.org>

On 12/20/2011 7:51 PM, Devin Jeanpierre wrote:
>> Not exactly true, and unnecessarily combative. More true is that careless
>> use of 'typed' has gotten tiresome. Python is strongly dynamically typed.
>> But people occasionally post -- again the same day you posted to python list
>> -- that Python is weakly typed. I am tired of explaining that 'typed' is not
>> synonymous with 'statically typed'.
>
> I don't find this much less careless. How do you differentiate between
> the "strong typing" of Python and the "strong typing" of Agda? It
> isn't a binary quantity.
>
> Perhaps, instead, we should stop claiming things are "strong" or
> "weak". If I said that, relatively speaking, Python is weakly typed,
> people would get offended -- not because I made any technically
> incorrect statement (on the spectrum, Python is far closer to assembly
> than Agda), but because to call it "weak" is insulting.

If you are going to use term idiosyncratically, then consider giving you 
definition along with it. See
https://en.wikipedia.org/wiki/Strongly_typed
for a common usage, by which Python is strongly typed.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Wed Dec 21 02:09:12 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 20 Dec 2011 20:09:12 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <4EF123C0.2010002@pearwood.info>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>	<jcpkal$rav$1@dough.gmane.org>	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org> <4EF123C0.2010002@pearwood.info>
Message-ID: <jcrbk6$5bn$1@dough.gmane.org>

On 12/20/2011 7:09 PM, Steven D'Aprano wrote:
> Terry Reedy wrote:
>
>> 2.) there are serious dragons in
>>> how python handles complex inheritance graphs that result in
>>> "object.__new__() takes no parameters", despite not having any builtin
>>> bases and having no base class overriding __new__ or __init__
>>
>> Best not to use object as the apex of multiple inheritance.
>
> Is that possible in Python 3?

I believe the recommendation I have seen is something like this

 >>> class C():
	def __init__(self, *arg, **kwds): pass
	
 >>> C(1)
<__main__.C object at 0x00000000034C0F98>

whereas
 >>> object(1)
Traceback (most recent call last):
   File "<pyshell#2>", line 1, in <module>
     object(1)
TypeError: object.__new__() takes no parameters

Now, so others have posted, define A and B with super, inheriting from C 
instead of object, and D with super inheriting from A and B, and all 
should go well instead of crashing. But I have not tested this myself. 
Same for ordinary single inheritance chain.

-- 
Terry Jan Reedy



From ncoghlan at gmail.com  Wed Dec 21 02:14:52 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Dec 2011 11:14:52 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
Message-ID: <CADiSq7dF+MS9orYsX_-EJ4qO=eNXd65=A4_M+gsTkQVw7RbdBg@mail.gmail.com>

On Wed, Dec 21, 2011 at 10:51 AM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> Perhaps, instead, we should stop claiming things are "strong" or
> "weak". If I said that, relatively speaking, Python is weakly typed,
> people would get offended -- not because I made any technically
> incorrect statement (on the spectrum, Python is far closer to assembly
> than Agda), but because to call it "weak" is insulting.

When you can mutate a str object into an int object (or vice-versa),
then you can claim Python is weakly typed without being technically
incorrect.

Weak typing has a very specific meaning: objects can change their type
without changing their identity (e.g. via pointer casting in C and
C++). Python lets objects *lie* about their types to some degree (by
altering __class__), but type(obj) will always reveal the true
underling type (indeed, "obj.__class__ != type(obj)" is one of the
ways to detect when you've been given a proxy object, if the
distinction matters for a particular use case). (For CPython,
extension module authors can actually use C code to get around the
strong typing if they really try, but the authors of such code get no
sympathy when it inevitably blows up in obscure and hard to debug
ways. Old-style classes in 2.x can also be legitimately described as
weakly typed, since *all* instances of such classes share a single
underlying type, and __class__ is the true determinant of their
behaviour)

Weak vs strong typing and dynamic vs static typing are well-defined
concepts - it's just all too common that folks that initially learn to
program with a static language confuse the two spectra and think that
"static typing" and "strong typing" are the same thing. They're not
only not the same, they're actually completely orthogonal. CPython,
for example, uses the weak static typing of C to implement Python's
strong dynamic typing mechanisms. IronPython and Jython have strong
typing at both levels, but retain the static vs dynamic split. I
believe PyPy uses strong dynamic typing throughout (although RPython
has a type inference mechanism and a few other tricks to support
translation to machine code)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From ncoghlan at gmail.com  Wed Dec 21 02:38:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Dec 2011 11:38:33 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7dF+MS9orYsX_-EJ4qO=eNXd65=A4_M+gsTkQVw7RbdBg@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
	<CADiSq7dF+MS9orYsX_-EJ4qO=eNXd65=A4_M+gsTkQVw7RbdBg@mail.gmail.com>
Message-ID: <CADiSq7ff4DL3xUa6Tx+oy83wVh2r_T2Go+Qvk7c+B5bDzhhv8g@mail.gmail.com>

I realised I gave a quick definition of strong vs weak typing, but not
dynamic vs static. Here's explanations for all four:

Strong typing: an object's type is immutable. To change the type, you
must change the object's identity (i.e. create a new object).
vs
Weak typing: an object's type (and hence its behaviour) can be changed
while leaving its identity untouched.

Python allows weak typing at the __class__ level (to support proxy
objects and similar metaprogramming tools), but the underlying object
model of the language is strongly typed.

Static typing: types are assigned not only to objects, but also to
labels that refer to objects. The type of the label and the type of
the object must match. Accordingly, variables must be explicitly
associated with a type via variable declarations.
vs
Dynamic typing: types are assigned only to objects, and labels
themselves are untyped. Accordingly, variables can be defined
implicitly just by assigning a value to them. (Some otherwise
statically typed languages include explicit support for dynamically
typed references)

(An interest hybrid variant for static vs dynamic is an approach where
labels *are* typed, but they acquire their type from the first value
assigned to them. Automatic type inferencing can make static typing
significantly less painful to work with, while still picking up most
type errors at compile time. C++11 has gone that way with its
introduction of "auto" type declarations for initialised types, where
you explicitly tell the compiler "this variable is of the same type as
the result of the initialiser").

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jeanpierreda at gmail.com  Wed Dec 21 02:41:24 2011
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 20 Dec 2011 20:41:24 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <jcrb1v$1vg$1@dough.gmane.org>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
	<jcrb1v$1vg$1@dough.gmane.org>
Message-ID: <CABicbJLvdD3emVxgKbp=8-P=3TPJchwVCrDYD04SOAP0f+r7=g@mail.gmail.com>

> If you are going to use term idiosyncratically, then consider giving you
> definition along with it. See
> https://en.wikipedia.org/wiki/Strongly_typed
> for a common usage, by which Python is strongly typed.

The list of "strongly typed" languages is prefixed with the following warning:

> Note that some of these definitions are contradictory, others are merely
> orthogonal, and still others are special cases (with additional constraints)
> of other, more "liberal" (less strong) definitions. Because of the wide
> divergence among these definitions, it is possible to defend claims about
> most programming languages that they are either strongly or weakly typed.

That is the point I was trying to make.

-- Devin

On Tue, Dec 20, 2011 at 7:59 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/20/2011 7:51 PM, Devin Jeanpierre wrote:
>>>
>>> Not exactly true, and unnecessarily combative. More true is that careless
>>> use of 'typed' has gotten tiresome. Python is strongly dynamically typed.
>>> But people occasionally post -- again the same day you posted to python
>>> list
>>> -- that Python is weakly typed. I am tired of explaining that 'typed' is
>>> not
>>> synonymous with 'statically typed'.
>>
>>
>> I don't find this much less careless. How do you differentiate between
>> the "strong typing" of Python and the "strong typing" of Agda? It
>> isn't a binary quantity.
>>
>> Perhaps, instead, we should stop claiming things are "strong" or
>> "weak". If I said that, relatively speaking, Python is weakly typed,
>> people would get offended -- not because I made any technically
>> incorrect statement (on the spectrum, Python is far closer to assembly
>> than Agda), but because to call it "weak" is insulting.
>
>
> If you are going to use term idiosyncratically, then consider giving you
> definition along with it. See
> https://en.wikipedia.org/wiki/Strongly_typed
> for a common usage, by which Python is strongly typed.
>
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From jeanpierreda at gmail.com  Wed Dec 21 02:46:43 2011
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 20 Dec 2011 20:46:43 -0500
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CABicbJLvdD3emVxgKbp=8-P=3TPJchwVCrDYD04SOAP0f+r7=g@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
	<jcrb1v$1vg$1@dough.gmane.org>
	<CABicbJLvdD3emVxgKbp=8-P=3TPJchwVCrDYD04SOAP0f+r7=g@mail.gmail.com>
Message-ID: <CABicbJKJoSEwu6b509uPjSRbU710oeQo78sJXROdoiTHeg=VJg@mail.gmail.com>

Hm, I made a lie. That isn't the point I was trying to make. Or
rather, I made two points at once originally, and the second one
contradicted the first one :)

(I stated that strong vs weak is dumb, and then I said Python was
weakly typed. I meant to say "in the sense that Agda is strong", but I
guess I screwed that one up)

*sigh*.

It's perfectly possible to say that Python is strongly typed, with a
self-consistent definition of "strongly typed". It's also possible to
say that it's weakly typed, with another self-consistent definition.
There is no _standard_ definition. The definition that Agda uses is
not the definition that Python uses.

I think people tend to choose the one that is least likely to call
their favorite language "weak".

Devin

On Tue, Dec 20, 2011 at 8:41 PM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
>> If you are going to use term idiosyncratically, then consider giving you
>> definition along with it. See
>> https://en.wikipedia.org/wiki/Strongly_typed
>> for a common usage, by which Python is strongly typed.
>
> The list of "strongly typed" languages is prefixed with the following warning:
>
>> Note that some of these definitions are contradictory, others are merely
>> orthogonal, and still others are special cases (with additional constraints)
>> of other, more "liberal" (less strong) definitions. Because of the wide
>> divergence among these definitions, it is possible to defend claims about
>> most programming languages that they are either strongly or weakly typed.
>
> That is the point I was trying to make.
>
> -- Devin
>
> On Tue, Dec 20, 2011 at 7:59 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 12/20/2011 7:51 PM, Devin Jeanpierre wrote:
>>>>
>>>> Not exactly true, and unnecessarily combative. More true is that careless
>>>> use of 'typed' has gotten tiresome. Python is strongly dynamically typed.
>>>> But people occasionally post -- again the same day you posted to python
>>>> list
>>>> -- that Python is weakly typed. I am tired of explaining that 'typed' is
>>>> not
>>>> synonymous with 'statically typed'.
>>>
>>>
>>> I don't find this much less careless. How do you differentiate between
>>> the "strong typing" of Python and the "strong typing" of Agda? It
>>> isn't a binary quantity.
>>>
>>> Perhaps, instead, we should stop claiming things are "strong" or
>>> "weak". If I said that, relatively speaking, Python is weakly typed,
>>> people would get offended -- not because I made any technically
>>> incorrect statement (on the spectrum, Python is far closer to assembly
>>> than Agda), but because to call it "weak" is insulting.
>>
>>
>> If you are going to use term idiosyncratically, then consider giving you
>> definition along with it. See
>> https://en.wikipedia.org/wiki/Strongly_typed
>> for a common usage, by which Python is strongly typed.
>>
>>
>> --
>> Terry Jan Reedy
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas


From pytom at yahoo.cn  Wed Dec 21 04:28:54 2011
From: pytom at yahoo.cn (Tom Zhou)
Date: Wed, 21 Dec 2011 11:28:54 +0800 (CST)
Subject: [Python-ideas] =?utf-8?b?5Zue5aSN77yaICBDb2x1bW5pemUgaW4gbW9kdWxl?=
 =?utf-8?b?ICJjbWQi?=
In-Reply-To: <CAP7+vJL9RRUCKhuqA7T6OLSJOsjtc9p74Yk7fwHrxwh=VJDfuw@mail.gmail.com>
References: <1324349593.51908.YahooMailNeo@web92411.mail.cnh.yahoo.com>
	<1324350440.30993.YahooMailNeo@web92413.mail.cnh.yahoo.com>
	<CAP7+vJL9RRUCKhuqA7T6OLSJOsjtc9p74Yk7fwHrxwh=VJDfuw@mail.gmail.com>
Message-ID: <1324438134.11446.YahooMailNeo@web92408.mail.cnh.yahoo.com>

Thanks ?a lot for your reply!?
I've reservations, ?maybe code looks like:?
"""
colwidth = max(map(len, list))
ncols = displaywidth/(colwidth+2)+1
nrows = size/ncols+1
for nrows
? ? ?for ncols
? ? ? ? ? texts = list[...]
? ? ? ? ? output texts
"""
is clear, and just waste some space of stdout.

---
Best Regards
tom


________________________________
 ???? Guido van Rossum <guido at python.org>
???? Tom Zhou <pytom at yahoo.cn> 
??? "python-ideas at python.org" <python-ideas at python.org> 
????? 2011?12?20?, ???, ?? 7:14
??: Re: [Python-ideas] Columnize in module "cmd"
 

That code is trying to find an optimal solution where the columns may be of different width.

Good luck!

--Guido



On Mon, Dec 19, 2011 at 8:07 PM, Tom Zhou <pytom at yahoo.cn> wrote:


>
>Hi~ alls
>
>recently, i focus on the module "cmd", and find some confused things-- the function named "columnize". Why we need a multiloop as
>"for nrows ..
>? ? ? ? for col ..
>? ? ? ? ? ? ? ? ? ?for row.."
>?? i think we can make a easier method, for example, first, find out the maxlen str in list, and use its length as the standard size to format the list.
>Ok, maybe i ignore something, so please give me some hints.
>
>
>
>
>---
>thankstom
>
>
>_______________________________________________
>Python-ideas mailing list
>Python-ideas at python.org
>http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111221/45978a9e/attachment.html>

From steve at pearwood.info  Wed Dec 21 12:16:19 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 21 Dec 2011 22:16:19 +1100
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <CADiSq7dF+MS9orYsX_-EJ4qO=eNXd65=A4_M+gsTkQVw7RbdBg@mail.gmail.com>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>	<jcpkal$rav$1@dough.gmane.org>	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>	<jcr44j$onv$1@dough.gmane.org>	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
	<CADiSq7dF+MS9orYsX_-EJ4qO=eNXd65=A4_M+gsTkQVw7RbdBg@mail.gmail.com>
Message-ID: <4EF1C003.4000208@pearwood.info>

Nick Coghlan wrote:

> Weak vs strong typing and dynamic vs static typing are well-defined
> concepts - it's just all too common that folks that initially learn to
> program with a static language confuse the two spectra and think that
> "static typing" and "strong typing" are the same thing.
[...]

While I like the definitions Nick has given, I think he's a tad optimistic to 
claim that the various foo-typing are "well-defined".

I think that weak and strong typing aren't dichotomies, but extremes in a
continuum. Most languages include elements of both weak and strong typing, 
particularly coercion of ints to floats.

Chris Smith's influential article "What To Know Before Debating Type
Systems" goes further, suggesting that weak and strong typing are
meaningless terms. I don't go that far, but you should read his article:

http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/

See also http://en.wikipedia.org/wiki/Type_system



-- 
Steven


From ncoghlan at gmail.com  Wed Dec 21 15:04:02 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Dec 2011 00:04:02 +1000
Subject: [Python-ideas] Adding "Typed" collections/iterators to Python
In-Reply-To: <4EF1C003.4000208@pearwood.info>
References: <CAOFbRm+F4c6b0Vxc79=VF4fgKA1a2mZzucqwF6Aba_tBi0WVqA@mail.gmail.com>
	<CADiSq7fy6zEm02FD6WDYmdLiMibsFn=hqzDOE8-=krWBH_ENbA@mail.gmail.com>
	<CAOFbRmK8XmX8imVQv+3EqqXFAhfpmWOCZZuVH45onbYAbTBmbw@mail.gmail.com>
	<CADiSq7eXmsvaa4FtRuNAr43u2V+-AGw6BiB5g=RpWNOFCRj0Aw@mail.gmail.com>
	<CAOFbRmJGqDz+rdyd27Q_X-TJubRaVzuQijgsn0_cF6sokUn6GQ@mail.gmail.com>
	<CADiSq7eUGP1Jf+XsOjKFFm=j6RBeSyktDQtq305+PpbfKFb0kg@mail.gmail.com>
	<CAOFbRmJCsHRimL1E=BUKsB9RzY4N694kBA6DS6PSpeFfZ78ZKg@mail.gmail.com>
	<CADiSq7diESA1UHurBKOqAJucTCJo7B-o8MJtWT3TUWcjdMXC8g@mail.gmail.com>
	<jcpkal$rav$1@dough.gmane.org>
	<CAOFbRmJoY8=wpFYSi1qcwVzvXZ=SbZXxTA90c2XEienaDhxSjQ@mail.gmail.com>
	<jcr44j$onv$1@dough.gmane.org>
	<CABicbJJS0JOgM9avMaW9xNM9nVHUpCpQ66s_o4_fSuuH9nL=Ag@mail.gmail.com>
	<CADiSq7dF+MS9orYsX_-EJ4qO=eNXd65=A4_M+gsTkQVw7RbdBg@mail.gmail.com>
	<4EF1C003.4000208@pearwood.info>
Message-ID: <CADiSq7d3gx4sJhiDE2NM5m5nF_QVkC89JCMDtsuM_TgDU5x2nA@mail.gmail.com>

On Wed, Dec 21, 2011 at 9:16 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> I think that weak and strong typing aren't dichotomies, but extremes in a
> continuum. Most languages include elements of both weak and strong typing,
> particularly coercion of ints to floats.
>
> Chris Smith's influential article "What To Know Before Debating Type
> Systems" goes further, suggesting that weak and strong typing are
> meaningless terms. I don't go that far, but you should read his article:

Yeah, I'd only ever encountered weak/strong under the meanings I gave,
but the Wikipedia article Terry linked was eye-opening (and does
indeed suggest that if you're going to use weak/strong as terms, it's
necessary to define them in situ).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From jxo6948 at rit.edu  Thu Dec 22 00:06:51 2011
From: jxo6948 at rit.edu (John O'Connor)
Date: Wed, 21 Dec 2011 18:06:51 -0500
Subject: [Python-ideas] defaultattrgetter
Message-ID: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>

Pythons:

What are your thoughts on the concept of a `defaultattrgetter`? It
would be to operator.attrgetter what getattr(foo, x, default) is to
getattr(foo, x). I dont like that attrgetter requires the attribute to
exist or else the getter be wrapped in a try catch with respect to how
defaultdict solves roughly the same problem for dictionary keys.

The semantics could be something such as:

from operator import defaultattrgetter
_x = defaultattrgetter({'x': 0})
_y = defaultattrgetter({'y': 1})

- or -

_xy = defaultattrgetter({'x': 0, 'y': 1})

One use case I am thinking of is functions that may be decorated with
attributes x and/or y. Obviously a python implementation of
defaultattrgetter would be trivial to implement but one of the
benefits of these functions is the speed. It also seems like it would
fit in well with the rest of the operator module.

 Generally speaking, would anyone else have a use for this?

- John


From steve at pearwood.info  Thu Dec 22 00:33:00 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 22 Dec 2011 10:33:00 +1100
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
Message-ID: <4EF26CAC.2030602@pearwood.info>

John O'Connor wrote:
> Pythons:
> 
> What are your thoughts on the concept of a `defaultattrgetter`? 

Why create a new function for it? Why not just give attrgetter a keyword only 
argument default?


-- 
Steven



From jxo6948 at rit.edu  Thu Dec 22 08:05:01 2011
From: jxo6948 at rit.edu (John O'Connor)
Date: Thu, 22 Dec 2011 02:05:01 -0500
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <4EF26CAC.2030602@pearwood.info>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
	<4EF26CAC.2030602@pearwood.info>
Message-ID: <CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>

On Wed, Dec 21, 2011 at 6:33 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Why create a new function for it? Why not just give attrgetter a keyword
> only argument default?

I need to revise my example. It should be:
_x = defaultattrgetter(('x', 0))
_xy = defaultattrgetter(('x', 0), ('y', 1))

Which is what I had originally but I was too quick to change it in the
course of writing thinking the dict notation looked cleaner. But,
since the argument order matters a dict wont work. The same applies to
using keyword arguments. I'm not sure if there is a clean way to add
this type of functionally to attrgetter without strings being a
special case.


From stefan_ml at behnel.de  Thu Dec 22 08:23:03 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Dec 2011 08:23:03 +0100
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
Message-ID: <jculso$v0n$1@dough.gmane.org>

John O'Connor, 22.12.2011 08:05:
> On Wed, Dec 21, 2011 at 6:33 PM, Steven D'Aprano wrote:
>> Why create a new function for it? Why not just give attrgetter a keyword
>> only argument default?
>
> I need to revise my example. It should be:
> _x = defaultattrgetter(('x', 0))
> _xy = defaultattrgetter(('x', 0), ('y', 1))
>
> Which is what I had originally but I was too quick to change it in the
> course of writing thinking the dict notation looked cleaner. But,
> since the argument order matters a dict wont work. The same applies to
> using keyword arguments. I'm not sure if there is a clean way to add
> this type of functionally to attrgetter without strings being a
> special case.

I don't consider it a major use case to be able to use different default 
return values for different steps in the lookup process. If you want that, 
write your own lookup function, that's trivial enough.

If such a feature gets added (which would be for Python 3.3 or later), I 
second Steven's proposal of making it a keyword argument, i.e.

     lookup_a_b_c = operator.attrgetter('a', 'b', 'c', default=123)

Stefan



From masklinn at masklinn.net  Thu Dec 22 08:57:30 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 22 Dec 2011 08:57:30 +0100
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <jculso$v0n$1@dough.gmane.org>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
	<jculso$v0n$1@dough.gmane.org>
Message-ID: <EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>

On 2011-12-22, at 08:23 , Stefan Behnel wrote:
> John O'Connor, 22.12.2011 08:05:
>> On Wed, Dec 21, 2011 at 6:33 PM, Steven D'Aprano wrote:
>>> Why create a new function for it? Why not just give attrgetter a keyword
>>> only argument default?
>> 
>> I need to revise my example. It should be:
>> _x = defaultattrgetter(('x', 0))
>> _xy = defaultattrgetter(('x', 0), ('y', 1))
>> 
>> Which is what I had originally but I was too quick to change it in the
>> course of writing thinking the dict notation looked cleaner. But,
>> since the argument order matters a dict wont work. The same applies to
>> using keyword arguments. I'm not sure if there is a clean way to add
>> this type of functionally to attrgetter without strings being a
>> special case.
> 
> I don't consider it a major use case to be able to use different default return values for different steps in the lookup process. If you want that, write your own lookup function, that's trivial enough.
> 
> If such a feature gets added (which would be for Python 3.3 or later), I second Steven's proposal of making it a keyword argument, i.e.
> 
>    lookup_a_b_c = operator.attrgetter('a', 'b', 'c', default=123)
The problem with that is ? does the default value apply to all three attributes? What if you need a default value for one but not the others, or different values for all three? Does this behavior really make sense?

Maybe a dict of default values, at least when extracting more than one attribute?

From stefan_ml at behnel.de  Thu Dec 22 09:09:28 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Dec 2011 09:09:28 +0100
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>	<4EF26CAC.2030602@pearwood.info>	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
Message-ID: <jcuojo$f1c$1@dough.gmane.org>

Masklinn, 22.12.2011 08:57:
> On 2011-12-22, at 08:23 , Stefan Behnel wrote:
>> John O'Connor, 22.12.2011 08:05:
>>> On Wed, Dec 21, 2011 at 6:33 PM, Steven D'Aprano wrote:
>>>> Why create a new function for it? Why not just give attrgetter a keyword
>>>> only argument default?
>>>
>>> I need to revise my example. It should be:
>>> _x = defaultattrgetter(('x', 0))
>>> _xy = defaultattrgetter(('x', 0), ('y', 1))
>>>
>>> Which is what I had originally but I was too quick to change it in the
>>> course of writing thinking the dict notation looked cleaner. But,
>>> since the argument order matters a dict wont work. The same applies to
>>> using keyword arguments. I'm not sure if there is a clean way to add
>>> this type of functionally to attrgetter without strings being a
>>> special case.
>>
>> I don't consider it a major use case to be able to use different default return values for different steps in the lookup process. If you want that, write your own lookup function, that's trivial enough.
>>
>> If such a feature gets added (which would be for Python 3.3 or later), I second Steven's proposal of making it a keyword argument, i.e.
>>
>>     lookup_a_b_c = operator.attrgetter('a', 'b', 'c', default=123)
> The problem with that is ? does the default value apply to all three attributes?

That's what I meant: it should hit the most common case and leave the rest 
to the user to handle.


> What if you need a default value for one but not the others, or different values for all three? Does this behavior really make sense?

What would be the use case? Would you want the lookup process to continue 
on the default argument if an intermediate attribute is not found? And if a 
subsequent lookup fails? Take the corresponding default value again and 
keep looking up on it? That feels rather unpredictable from a users' point 
of view.

I think the idea is that you ask for an attribute, which (in the less 
common cases) happens to be a multi-step lookup, and if the attribute is 
not found, you want it to return a default value *for the attribute you 
requested*, i.e. not a different value for any of the intermediate 
attributes, only a specific value that corresponds to the last attribute in 
the lookup chain. In the 99.9% case, that will be something like None or a 
"keep going, I don't care" value, not something that depends on the lookup 
path in any way.

I don't think the 0.1% case where you want more than that is worth a 
substantially more complicated API.

Stefan



From masklinn at masklinn.net  Thu Dec 22 09:25:05 2011
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 22 Dec 2011 09:25:05 +0100
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <jcuojo$f1c$1@dough.gmane.org>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>	<4EF26CAC.2030602@pearwood.info>	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
	<jcuojo$f1c$1@dough.gmane.org>
Message-ID: <C1199458-F6A3-4AF2-998C-9AB564F30434@masklinn.net>

On 2011-12-22, at 09:09 , Stefan Behnel wrote:
> 
>> What if you need a default value for one but not the others, or different values for all three? Does this behavior really make sense?
> 
> What would be the use case? Would you want the lookup process to continue on the default argument if an intermediate attribute is not found? And if a subsequent lookup fails? Take the corresponding default value again and keep looking up on it? That feels rather unpredictable from a users' point of view.
A multiple-*args attrgetter fetches different arguments, for a "lookup process" you need a dotted path (attrgetter('foo.bar.baz')).

For that one a default value returned after any lookup failure makes sense, but it's not what attrgetter('foo', 'bar', 'baz') does:

>>> from collections import namedtuple
>>> from operator import attrgetter
>>> r = namedtuple('T', 'a b c d e')(1, 2, 3, 4, 5)
>>> attrgetter('a.__index__')(r)
<method-wrapper '__index__' of int object at 0x10020b1e8>
>>> attrgetter('a', 'b', 'e')(r)
(1, 2, 5)

And for the second case, you'd need to be able to specify a default value for each of a, b and e.

> I think the idea is that you ask for an attribute, which (in the less common cases) happens to be a multi-step lookup, and if the attribute is not found, you want it to return a default value *for the attribute you requested*, i.e. not a different value for any of the intermediate attributes
Right, we're talking about different things and as noted above I think you're figuring the behavior of attrgetter(arg0, arg1, ?) wrong.

> I don't think the 0.1% case where you want more than that is worth a substantially more complicated API.
Thing is, I don't know about others but my main use of attrgetter *by far* is to fetch multiple attributes simultanously. When fetching a single attribute (even with multiple steps), it's often (if not always) shorter to use a lambda with an attribute lookup.

From stefan_ml at behnel.de  Thu Dec 22 09:35:42 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Dec 2011 09:35:42 +0100
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <C1199458-F6A3-4AF2-998C-9AB564F30434@masklinn.net>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>	<4EF26CAC.2030602@pearwood.info>	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>	<jculso$v0n$1@dough.gmane.org>	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>	<jcuojo$f1c$1@dough.gmane.org>
	<C1199458-F6A3-4AF2-998C-9AB564F30434@masklinn.net>
Message-ID: <jcuq4v$ogb$1@dough.gmane.org>

Masklinn, 22.12.2011 09:25:
> On 2011-12-22, at 09:09 , Stefan Behnel wrote:
>>
>>> What if you need a default value for one but not the others, or different values for all three? Does this behavior really make sense?
>>
>> What would be the use case? Would you want the lookup process to continue on the default argument if an intermediate attribute is not found? And if a subsequent lookup fails? Take the corresponding default value again and keep looking up on it? That feels rather unpredictable from a users' point of view.
> A multiple-*args attrgetter fetches different arguments, for a "lookup process" you need a dotted path (attrgetter('foo.bar.baz')).

Ah, thanks, I remembered that incorrectly then.

In that case, I vote -1 for adding the feature in the first place, because 
it would complicate an otherwise simple API too much and the current syntax 
proposals are too verbose and too hard to read.

Stefan



From __peter__ at web.de  Thu Dec 22 10:27:49 2011
From: __peter__ at web.de (Peter Otten)
Date: Thu, 22 Dec 2011 10:27:49 +0100
Subject: [Python-ideas] defaultattrgetter
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
	<jcuojo$f1c$1@dough.gmane.org>
	<C1199458-F6A3-4AF2-998C-9AB564F30434@masklinn.net>
	<jcuq4v$ogb$1@dough.gmane.org>
Message-ID: <jcut6m$ci8$1@dough.gmane.org>

Stefan Behnel wrote:

> Masklinn, 22.12.2011 09:25:
>> On 2011-12-22, at 09:09 , Stefan Behnel wrote:
>>>
>>>> What if you need a default value for one but not the others, or
>>>> different values for all three? Does this behavior really make sense?
>>>
>>> What would be the use case? Would you want the lookup process to
>>> continue on the default argument if an intermediate attribute is not
>>> found? And if a subsequent lookup fails? Take the corresponding default
>>> value again and keep looking up on it? That feels rather unpredictable
>>> from a users' point of view.
>> A multiple-*args attrgetter fetches different arguments, for a "lookup
>> process" you need a dotted path (attrgetter('foo.bar.baz')).
> 
> Ah, thanks, I remembered that incorrectly then.
> 
> In that case, I vote -1 for adding the feature in the first place, because
> it would complicate an otherwise simple API too much and the current
> syntax proposals are too verbose and too hard to read.

I still like the addition, I think

>>> attrgetter("last_name", default="unknown")(None)
'unknown'

would be the most common use case by far and

>>> location = None
>>> attrgetter("x", "y", "z", default=0)(location)
(0, 0, 0)
>>> attrgetter("x.unit", default=None)(location)
None # don't know if there's no location.x or no location.x.unit

are useful and easy enough to grasp.





From jxo6948 at rit.edu  Thu Dec 22 10:55:20 2011
From: jxo6948 at rit.edu (John O'Connor)
Date: Thu, 22 Dec 2011 04:55:20 -0500
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <jcuojo$f1c$1@dough.gmane.org>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
	<jcuojo$f1c$1@dough.gmane.org>
Message-ID: <CABCbifVD=mXyEayiN7dv7a3Q18ME9YUN56K6aL5K3CR4et9Arw@mail.gmail.com>

> I think the idea is that you ask for an attribute, which (in the less common
> cases) happens to be a multi-step lookup, and if the attribute is not found,
> you want it to return a default value *for the attribute you requested*,
> i.e. not a different value for any of the intermediate attributes, only a
> specific value that corresponds to the last attribute in the lookup chain.
> In the 99.9% case, that will be something like None or a "keep going, I
> don't care" value, not something that depends on the lookup path in any way.
>
> I don't think the 0.1% case where you want more than that is worth a
> substantially more complicated API.
>

I could settle for the default= but I think it is too simple and thus
incomplete. I think it is extreme to say that one default argument
covers 99.9% use case. I do think you are right about the ignored path
but that path may use more than one condition.

_xy = defaultattrgetter(('x', True), ('y', False))
x, y = _xy(foo)
# usually do x unless told otherwise
if x: ...
# usually dont do y
if y: ...


I guess one alternative could be:
_xy = attrgetter('x', 'y', defaults={'x': True, 'y': False})

but that just looks like DRY without the D.


From arnodel at gmail.com  Thu Dec 22 11:21:46 2011
From: arnodel at gmail.com (Arnaud Delobelle)
Date: Thu, 22 Dec 2011 10:21:46 +0000
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <CABCbifVD=mXyEayiN7dv7a3Q18ME9YUN56K6aL5K3CR4et9Arw@mail.gmail.com>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
	<jcuojo$f1c$1@dough.gmane.org>
	<CABCbifVD=mXyEayiN7dv7a3Q18ME9YUN56K6aL5K3CR4et9Arw@mail.gmail.com>
Message-ID: <CAJ6cK1bOidXdpZi0_24ZsarUBxvEF-0O4YgTL-BJA2uO46j1ow@mail.gmail.com>

On Dec 22, 2011 9:56 AM, "John O&apos;Connor" <jxo6948 at rit.edu> wrote:
>
> > I think the idea is that you ask for an attribute, which (in the less
common
> > cases) happens to be a multi-step lookup, and if the attribute is not
found,
> > you want it to return a default value *for the attribute you requested*,
> > i.e. not a different value for any of the intermediate attributes, only
a
> > specific value that corresponds to the last attribute in the lookup
chain.
> > In the 99.9% case, that will be something like None or a "keep going, I
> > don't care" value, not something that depends on the lookup path in any
way.
> >
> > I don't think the 0.1% case where you want more than that is worth a
> > substantially more complicated API.
> >
>
> I could settle for the default= but I think it is too simple and thus
> incomplete. I think it is extreme to say that one default argument
> covers 99.9% use case. I do think you are right about the ignored path
> but that path may use more than one condition.
>
> _xy = defaultattrgetter(('x', True), ('y', False))
> x, y = _xy(foo)
> # usually do x unless told otherwise
> if x: ...
> # usually dont do y
> if y: ...
>
>
> I guess one alternative could be:
> _xy = attrgetter('x', 'y', defaults={'x': True, 'y': False})
>
> but that just looks like DRY without the D.

You could have attrgetter('x', 'y', defaults=(True, False)) to put the D
back in.

Arnaud
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111222/43e86ebb/attachment.html>

From storchaka at gmail.com  Thu Dec 22 11:32:17 2011
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 22 Dec 2011 12:32:17 +0200
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <CABCbifVD=mXyEayiN7dv7a3Q18ME9YUN56K6aL5K3CR4et9Arw@mail.gmail.com>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
	<jcuojo$f1c$1@dough.gmane.org>
	<CABCbifVD=mXyEayiN7dv7a3Q18ME9YUN56K6aL5K3CR4et9Arw@mail.gmail.com>
Message-ID: <jcv0vn$65k$1@dough.gmane.org>

22.12.11 11:55, John O'Connor ???????(??):
> I do think you are right about the ignored path
> but that path may use more than one condition.
>
> _xy = defaultattrgetter(('x', True), ('y', False))
> x, y = _xy(foo)
> # usually do x unless told otherwise
> if x: ...
> # usually dont do y
> if y: ...
>
>
> I guess one alternative could be:
> _xy = attrgetter('x', 'y', defaults={'x': True, 'y': False})

Or
_xy = attrgetter('x', 'y', defaults=(True, False))


Sometimes, however, will be better to specify not value, but factory 
(when value is list, dictionary, or complex object), as for defauldict.

_xyz = defaultattrgetter('x', ('y', int, 123), ('z', list))
x has not default value,
y has default value int(123),
z has default value list().



From ncoghlan at gmail.com  Thu Dec 22 13:14:12 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Dec 2011 22:14:12 +1000
Subject: [Python-ideas] defaultattrgetter
In-Reply-To: <jcv0vn$65k$1@dough.gmane.org>
References: <CABCbifVkN8By7-_NPCQB9n3EQHqGiPib8kaZ6QzHPkMH5S3VVg@mail.gmail.com>
	<4EF26CAC.2030602@pearwood.info>
	<CABCbifXuu=TM_mtkW0He+t3kZ9Ck0kLav3cnbjkicxLPBqN2Cg@mail.gmail.com>
	<jculso$v0n$1@dough.gmane.org>
	<EB0C79AD-55BB-40BD-9C5F-1C1624E4635E@masklinn.net>
	<jcuojo$f1c$1@dough.gmane.org>
	<CABCbifVD=mXyEayiN7dv7a3Q18ME9YUN56K6aL5K3CR4et9Arw@mail.gmail.com>
	<jcv0vn$65k$1@dough.gmane.org>
Message-ID: <CADiSq7fUWpYPxpJXqv1Dm6jC7rNYdJNPHjOG6=TFe=P7MzWr4A@mail.gmail.com>

Once again, please don't get so enamoured of higher order functions
that you miss the obvious solution: *just write a new function that
does exactly what you want*.

"Swiss Army APIs" are not a good thing. Sometimes they're fairly
unavoidable because they're exposing a complex underlying operation
with a lot of moving parts (e.g. subprocess.Popen), but other times
they're useless cruft that is so hard to remember that most people
never bother with them, as just writing the custom function is
significantly easier.

So, for the 3 examples given:

def _x(arg): return getattr(arg, 'x', 0)
def _y(arg): return getattr(arg, 'y', 1)
def _xy(arg): return _x(arg), _y(arg)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia


From g.rodola at gmail.com  Tue Dec 27 18:57:29 2011
From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=)
Date: Tue, 27 Dec 2011 18:57:29 +0100
Subject: [Python-ideas] Should stdlib provide something to properly handle
	SIGINT/SIGTERM signals?
Message-ID: <CAFYqXL9MjAQ0843EE723QvN5gaxnQfGEJpHCbQ+CwcgX__R4DQ@mail.gmail.com>

Since this is a problem which occurs very often, I'd like to hear your
opinion as to whether something like this might find some place in the
python stdlib (signal module maybe?).
Please read the discussion included in this recipe:
http://code.activestate.com/recipes/577997/
It should provide a description of the problem and the general use case.

Thanks in advance for your comments,

--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/


From techtonik at gmail.com  Tue Dec 27 18:01:50 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 27 Dec 2011 09:01:50 -0800 (PST)
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
Message-ID: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>

As you may know, the python-ideas list is opened only to subscribers. This 
is inconvenient, because:
1. it requires three step subscription process
2. it is impossible to post a reply to existing thread/idea

There is a web-interface in Google Groups at 
https://groups.google.com/forum/#!forum/python-ideas that can solve 
problems above and provide some more nifty features such as embedded 
search.  But there comes another problem that messages posted through the 
group doesn't end in list, because list requires subscription.  I've 
already tried to find a solution, but run out of time, so I summarized the 
proposal at http://wiki.python.org/moin/MailmanWithGoogleGroups

I may or may not be able to publish outcomes of my research, so it would be 
nice to get some help in investigating the problem and publishing a 
solution on aforementioned wiki page. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111227/eb783889/attachment.html>

From tjreedy at udel.edu  Tue Dec 27 22:52:49 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 27 Dec 2011 16:52:49 -0500
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
Message-ID: <jddenk$gr1$1@dough.gmane.org>

On 12/27/2011 12:01 PM, anatoly techtonik wrote:
> As you may know, the python-ideas list is opened only to subscribers.
> This is inconvenient, because:
> 1. it requires three step subscription process
> 2. it is impossible to post a reply to existing thread/idea
>
> There is a web-interface in Google Groups at
> https://groups.google.com/forum/#!forum/python-ideas
> <https://groups.google.com/forum/#!forum/python-ideas> that can solve
> problems above and provide some more nifty features such as embedded
> search. But there comes another problem that messages posted through the
> group doesn't end in list, because list requires subscription. I've
> already tried to find a solution, but run out of time, so I summarized
> the proposal at http://wiki.python.org/moin/MailmanWithGoogleGroups

I interact with this list as newsgroup gmane.comp.python.ideas via 
news.gmane.org. The first post, which can be (and usually should be) a 
reply to an existing post, requires that one respond to an email to the 
from address. This is the miminum necessary to protect from spam. No big 
deal.

Google mail and google groups are main gateways for spam on python list. 
We do not need that here.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Tue Dec 27 22:58:31 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 27 Dec 2011 16:58:31 -0500
Subject: [Python-ideas] Should stdlib provide something to properly
 handle SIGINT/SIGTERM signals?
In-Reply-To: <CAFYqXL9MjAQ0843EE723QvN5gaxnQfGEJpHCbQ+CwcgX__R4DQ@mail.gmail.com>
References: <CAFYqXL9MjAQ0843EE723QvN5gaxnQfGEJpHCbQ+CwcgX__R4DQ@mail.gmail.com>
Message-ID: <jddf2a$ijr$1@dough.gmane.org>

On 12/27/2011 12:57 PM, Giampaolo Rodol? wrote:
> Since this is a problem which occurs very often, I'd like to hear your
> opinion as to whether something like this might find some place in the
> python stdlib (signal module maybe?).
> Please read the discussion included in this recipe:
> http://code.activestate.com/recipes/577997/
> It should provide a description of the problem and the general use case.

It seems to me that your problem is this

"From http://docs.python.org/library/atexit.html:
     the functions registered via this module are not called when the 
program is killed by a signal not handled by Python
"

So it seems to me that the solution should be an option to change that, 
or rather, change the handling of exit signals.

-- 
Terry Jan Reedy




From ethan at stoneleaf.us  Wed Dec 28 06:40:23 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 27 Dec 2011 21:40:23 -0800
Subject: [Python-ideas] Should stdlib provide something to properly
 handle SIGINT/SIGTERM signals?
In-Reply-To: <CAFYqXL9MjAQ0843EE723QvN5gaxnQfGEJpHCbQ+CwcgX__R4DQ@mail.gmail.com>
References: <CAFYqXL9MjAQ0843EE723QvN5gaxnQfGEJpHCbQ+CwcgX__R4DQ@mail.gmail.com>
Message-ID: <4EFAABC7.6090203@stoneleaf.us>

Giampaolo Rodol? wrote:
 > The behavior I would expect when facing such a problem is the
 > following:
 >
 >     * I want my exit function to always be called, no matter what
 >           o I'm aware that it won't be called in case of SIGKILL or
 >             os._exit() though
 >     * I do not want to override previously registered signal handlers
 >      (if any)
 >           o Instead I want the old handler to be called fist and my
 >             exit function to be called last

Aren't nested handlers usually called in a last-in/first-out manner?

~Ethan~


From techtonik at gmail.com  Wed Dec 28 11:00:55 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 28 Dec 2011 02:00:55 -0800 (PST)
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <CANSw7Ky18H6gdo=Tpiotxmzpm6tu+6V+VyukKqv5YDSSk6XDWQ@mail.gmail.com>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<CANSw7Ky18H6gdo=Tpiotxmzpm6tu+6V+VyukKqv5YDSSk6XDWQ@mail.gmail.com>
Message-ID: <30503727.947.1325066456024.JavaMail.geo-discussion-forums@yqai28>

On Tuesday, December 27, 2011 10:53:56 PM UTC+3, RunThePun wrote:
>
> On Tue, Dec 27, 2011 at 7:01 PM, anatoly techtonik <tech... at gmail.com>wrote:
>
>> As you may know, the python-ideas list is opened only to subscribers. 
>> This is inconvenient, because:
>> 1. it requires three step subscription process
>> 2. it is impossible to post a reply to existing thread/idea
>>
>> There is a web-interface in Google Groups at https://groups.google.com/**
>> forum/#!forum/python-ideas<https://groups.google.com/forum/#!forum/python-ideas> that 
>> can solve problems above and provide some more nifty features such as 
>> embedded search.  But there comes another problem that messages posted 
>> through the group doesn't end in list, because list requires subscription. 
>>  I've already tried to find a solution, but run out of time, so I 
>> summarized the proposal at 
>> http://wiki.python.org/moin/MailmanWithGoogleGroups
>>
>> I may or may not be able to publish outcomes of my research, so it would 
>> be nice to get some help in investigating the problem and publishing a 
>> solution on aforementioned wiki page. Thanks.
>>
>>
>>
>
> Concerning the search problem I've used google queries as such:
>
>     list comprehensions site:
> http://mail.python.org/pipermail/python-ideas/
>
> I agree that having a "nosey" or "star" feature like issue trackers could 
> be nice, though I'm not sure Google Groups is the most modern 
> infrastructure to solve all our problems. I remember hearing that open 
> google groups get a lot of spam for example.
>

Over last six months I found only 3 spam messages sent from 
https://groups.google.com/forum/#!forum/python-ideas and for some reason I 
think that my mailbox filter would be smart enough to put them into 
appropriate folder even they came from Mailman.

Maybe mailman can be improved?
>   eg would it help if the PyPI login cookie allowed you to post on 
> mailman? If mailman allowed starring threads?
>
 
Certainly. But the threads should be stacked in different order that just 
by month, because some threads can span over several months.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111228/7e3547c4/attachment.html>

From techtonik at gmail.com  Wed Dec 28 11:11:03 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 28 Dec 2011 02:11:03 -0800 (PST)
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <jddenk$gr1$1@dough.gmane.org>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<jddenk$gr1$1@dough.gmane.org>
Message-ID: <14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>

On Wednesday, December 28, 2011 12:52:49 AM UTC+3, Terry Reedy wrote:
>
> On 12/27/2011 12:01 PM, anatoly techtonik wrote:
> > As you may know, the python-ideas list is opened only to subscribers.
> > This is inconvenient, because:
> > 1. it requires three step subscription process
> > 2. it is impossible to post a reply to existing thread/idea
> >
> > There is a web-interface in Google Groups at
> > https://groups.google.com/forum/#!forum/python-ideas
> > <https://groups.google.com/forum/#!forum/python-ideas> that can solve
> > problems above and provide some more nifty features such as embedded
> > search. But there comes another problem that messages posted through the
> > group doesn't end in list, because list requires subscription. I've
> > already tried to find a solution, but run out of time, so I summarized
> > the proposal at http://wiki.python.org/moin/MailmanWithGoogleGroups
>
> I interact with this list as newsgroup gmane.comp.python.ideas via 
> news.gmane.org. The first post, which can be (and usually should be) a 
> reply to an existing post, requires that one respond to an email to the 
> from address. This is the miminum necessary to protect from spam. No big 
> deal.
>
> Google mail and google groups are main gateways for spam on python list. 
> We do not need that here.
>
Does Mailman support confirmation for messages with high SPAM ratio?

There is already an Groups interface 
at https://groups.google.com/forum/#!topic/python-ideas/ and if a person is 
not subscribed in Mailman, the message appears in Group, but doesn't appear 
in Mailman even after the person is finally subscribed (if that persons 
notices and understands Mailman bounce message instructing him to 
subscribe).

Somebody needs to put a notice about obligatory subscription to Mailman to 
https://groups.google.com/forum/#!topic/python-ideas/ if nobody wants more 
tight groups integration.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111228/39f7df0f/attachment.html>

From techtonik at gmail.com  Wed Dec 28 11:15:24 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 28 Dec 2011 02:15:24 -0800 (PST)
Subject: [Python-ideas] PEP x: Static module/package inspection
In-Reply-To: <29228470.233.1324982829840.JavaMail.geo-discussion-forums@yqbl25>
References: <29228470.233.1324982829840.JavaMail.geo-discussion-forums@yqbl25>
Message-ID: <20544069.58.1325067324546.JavaMail.geo-discussion-forums@yqiz15>

(reposting this from Google Group once more as the previous post missed 
Mailing List, because I was not subscribed in Mailman)

*Static module/package inspection*

Abstract:
 - static: without execution (as opposed to dynamic)
 - module/package: .py or __init__.py file
 - inspection: get an overview of the contents

*What should this do?*
*
*
The proposal to add a mechanism to Python interpreter to get an outline of 
module/package contents without importing or executing module/package. The 
outline includes names of classes, functions, variables. It also should 
contain values for variables that could be provided without sophisticated 
calculations (e.g. a string, integer, but probably not expressions as it 
may lead to security leaks).

*Why?*
*
*
*user story PEPx.001:*
As a Python package maintainer, I find it bothersome to repeatedly write 
bolierplate code (e.g. setup.py) to package my single file module. The 
reason I should write setup.py is to provide version and description info. 
This info is already available in my module source code. So I need to 
either copy/paste the info from the module manually, or to import (and 
hence execute) my module during packaging and installation, which I don't 
want either, because modules are often installed with root privileges.

With this PEP, packing tool will be able to extract meta information from 
my module without executing it or without me manually copying version 
fields into some 'package configuration file'.

*user story PEPx.002:*
As a Python Application developer, I find it really complicated to provide 
plugin extension subsystem for my users. Users need a mechanism to switch 
between different versions of the plugin, and this mechanism is usually 
provided by external tool such as setuptools to manage and install multiple 
versions of plugins in local Python package repository. It is rather hard 
to create an alternative approach, because you are forced to maintain 
external meta-data about your plugin modules even in case it is already 
available inside the module.

With this PEP, Python Application will be able to inspect 
meta-data embedded inside of plugins before choosing which version to load. 
This will also provide a standard mechanism for applications to check 
modules returned by packaging tools without executing them. This will 
greatly simplify writing and debugging custom plugins loaders on different 
platforms.


*Feedback goal*
At this stage I'd like to a community response to two separate questions:
1. If everybody feels this functionality will be useful for Python
2. If the solution is technically feasible

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111228/086ba63f/attachment.html>

From brian at python.org  Wed Dec 28 15:32:40 2011
From: brian at python.org (Brian Curtin)
Date: Wed, 28 Dec 2011 08:32:40 -0600
Subject: [Python-ideas] Idea: Google Groups web interface for Ideas
In-Reply-To: <14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>
References: <11616830.383.1325005310654.JavaMail.geo-discussion-forums@yqlp13>
	<jddenk$gr1$1@dough.gmane.org>
	<14270238.69.1325067063613.JavaMail.geo-discussion-forums@yqip20>
Message-ID: <CAD+XWwoyAFTi1oC68QEzZUmcPZdf9eNL2qfWYB94pCm0f8tpiQ@mail.gmail.com>

On Wed, Dec 28, 2011 at 04:11, anatoly techtonik <techtonik at gmail.com> wrote:
> Somebody needs to put a notice about obligatory subscription to Mailman to
> https://groups.google.com/forum/#!topic/python-ideas/ if nobody wants more
> tight groups integration.

Whoever created that can put up the notice. I've never even heard of
this Google Groups mirror to know who would be the one to do that.

I really do not think anything is going to happen here - it works fine
for everyone else and the hundreds of other mailing lists on
python.org as well as the thousands of mailing lists around the world.
Responding to old threads seems to be your biggest point for this, but
you can always start a new thread and say "bringing up this topic from
6 months ago", link the original thread, then restart discussion -
that's acceptable.


From fuzzyman at gmail.com  Wed Dec 28 16:28:44 2011
From: fuzzyman at gmail.com (Michael Foord)
Date: Wed, 28 Dec 2011 15:28:44 +0000
Subject: [Python-ideas] PEP x: Static module/package inspection
In-Reply-To: <20544069.58.1325067324546.JavaMail.geo-discussion-forums@yqiz15>
References: <29228470.233.1324982829840.JavaMail.geo-discussion-forums@yqbl25>
	<20544069.58.1325067324546.JavaMail.geo-discussion-forums@yqiz15>
Message-ID: <CAKCKLWx5b1zkUw97+6k4R1EwU3hydj_852gUkfb31bzM7h9L9g@mail.gmail.com>

On 28 December 2011 10:15, anatoly techtonik <techtonik at gmail.com> wrote:

> (reposting this from Google Group once more as the previous post missed
> Mailing List, because I was not subscribed in Mailman)
>
> *Static module/package inspection*
>
> Abstract:
>  - static: without execution (as opposed to dynamic)
>  - module/package: .py or __init__.py file
>  - inspection: get an overview of the contents
>
> *What should this do?*
> *
> *
> The proposal to add a mechanism to Python interpreter to get an outline of
> module/package contents without importing or executing module/package. The
> outline includes names of classes, functions, variables. It also should
> contain values for variables that could be provided without sophisticated
> calculations (e.g. a string, integer, but probably not expressions as it
> may lead to security leaks).
>
> *Why?*
> *
> *
> *user story PEPx.001:*
> As a Python package maintainer, I find it bothersome to repeatedly write
> bolierplate code (e.g. setup.py) to package my single file module. The
> reason I should write setup.py is to provide version and description info.
> This info is already available in my module source code. So I need to
> either copy/paste the info from the module manually, or to import (and
> hence execute) my module during packaging and installation, which I don't
> want either, because modules are often installed with root privileges.
>
> With this PEP, packing tool will be able to extract meta information from
> my module without executing it or without me manually copying version
> fields into some 'package configuration file'.
>
> *user story PEPx.002:*
> As a Python Application developer, I find it really complicated to provide
> plugin extension subsystem for my users. Users need a mechanism to switch
> between different versions of the plugin, and this mechanism is usually
> provided by external tool such as setuptools to manage and install multiple
> versions of plugins in local Python package repository. It is rather hard
> to create an alternative approach, because you are forced to maintain
> external meta-data about your plugin modules even in case it is already
> available inside the module.
>
> With this PEP, Python Application will be able to inspect
> meta-data embedded inside of plugins before choosing which version to load.
> This will also provide a standard mechanism for applications to check
> modules returned by packaging tools without executing them. This will
> greatly simplify writing and debugging custom plugins loaders on different
> platforms.
>
>
> *Feedback goal*
> At this stage I'd like to a community response to two separate questions:
> 1. If everybody feels this functionality will be useful for Python
> 2. If the solution is technically feasible
>
>

On a simple level, all of this is already "obtainable" by using the ast
module that can parse Python code. I would love to see a "python-object"
layer on top of this that will take an ast for a module (or other object)
and return something that represents the same object as the ast.

So all module level objects will have corresponding objects - where they
are Python objects (builtin-literals) then they will represented exactly.
For classes and functions you'll get an object back that has the same
attributes plus some metadata (e.g. for functions /  methods what arguments
they take etc).

That is certainly doable and would make introspecting-without-executing a
lot simpler.

I think your specific use cases are better served by adding functionality
to the packaging (distutils2) package however. I'd particularly like to see
plugin support in packaging (a cutdown version of setuptools entry points).

All the best,

Michael


>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111228/6f2bca25/attachment.html>

From nathan.alexander.rice at gmail.com  Wed Dec 28 22:18:38 2011
From: nathan.alexander.rice at gmail.com (Nathan Rice)
Date: Wed, 28 Dec 2011 16:18:38 -0500
Subject: [Python-ideas] PEP x: Static module/package inspection
In-Reply-To: <20544069.58.1325067324546.JavaMail.geo-discussion-forums@yqiz15>
References: <29228470.233.1324982829840.JavaMail.geo-discussion-forums@yqbl25>
	<20544069.58.1325067324546.JavaMail.geo-discussion-forums@yqiz15>
Message-ID: <CAOFbRmLZQvUdf=C+ga8SVkbwhp-eYmy3v3h9HXny3cB1w=hMWQ@mail.gmail.com>

On Wed, Dec 28, 2011 at 5:15 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> user story PEPx.001:
> As a Python package maintainer, I find it bothersome to repeatedly write
> bolierplate code (e.g. setup.py) to package my single file module. The
> reason I should write setup.py is to provide version and description info.
> This info is already available in my module source code. So I need to either
> copy/paste the info from the module manually, or to import (and hence
> execute) my module during packaging and installation, which I don't want
> either, because modules are often installed with root privileges.

I agree this is a pain.  I also agree with Micheal that this is moreso
a packager issue.  Part of the problem is that I don't believe there
is a strong enough convention around writing modules with an eye to
being package tools accessible.  If there was a PEP on module metadata
for packaging tools to use for introspection, that might motivate
package tool authors to support automated packaging :) *HINT HINT*
Sphinx could also take advantage of some of it too.

> With this PEP, packing tool will be able to extract meta information from my
> module without executing it or without me manually copying version fields
> into some 'package configuration file'.
>
> user story PEPx.002:
> As a Python Application developer, I find it really complicated to provide
> plugin extension subsystem for my users. Users need a mechanism to switch
> between different versions of the plugin, and this mechanism is usually
> provided by external tool such as setuptools to manage and install multiple
> versions of plugins in local Python package repository. It is rather hard to
> create an alternative approach, because you are forced to maintain external
> meta-data about your plugin modules even in case it is already available
> inside the module.

See above.  Maintaining the same information twice is definitely a bad
thing, but we already have the ability to do everything required.
What is missing is good, strong conventions on module metadata
annotation that tool creators write to.

> With this PEP, Python Application will be able to inspect meta-data embedded
> inside of plugins before choosing which version to load. This will also
> provide a standard mechanism for applications to check modules returned by
> packaging tools without executing them. This will greatly simplify writing
> and debugging custom plugins loaders on different platforms.

Having more nuanced import behavior is something I can get behind.
Sure, I can wrap an import in a try except, and check the __version__
if is defined (after determining if it is a string/tuple/etc, and
possibly parsing it), but more nuanced behavior would certainly be
nice.  Being able to specify version in the import line (and have
multiple versions installed), being able to get fine grained exception
beyond ImportError (ParseError, anyone?), not having to worry that the
same file is being imported twice, that sort of stuff.

I'm +1 getting a module level metadata conventions PEP draft started.
I'm also +1 on taking a look at import behavior (though that is
tangential here).

Nathan


From guido at python.org  Fri Dec 30 17:40:06 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Dec 2011 09:40:06 -0700
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
Message-ID: <CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>

Hi Julien,

Don't despair! I have tried to get people to warm up to dict addition too
-- in fact it was my counter-proposal at the time when we were considering
adding sets to the language. I will look at your proposal, but I have a
point of order first: this should be discussed on python-ideas, not on
python-dev. I have added python-ideas to the thread and moved python-dev to
Bcc, so followups will hopefully all go to python-ideas.

--Guido

On Fri, Dec 30, 2011 at 7:26 AM, julien tayon <julien at tayon.net> wrote:

> Hello,
> Sorry to annoy the very busy core devs :) out of the blue
>
> I quite noticed people were
> 1) wanting to have a new dict for Xmas
> 2) strongly resenting dict addition.
>
> Even though I am not a good developper, I have come to a definition of
> addition that would follow algebraic rules, and not something of a
> dutch logic. (it is a jest, not a troll)
>
> I propose the following code to validate my point of view regarding
> the dictionnatry addition as a proof of concept :
> https://github.com/jul/ADictAdd_iction/blob/master/test.py
>
> It follows all my dusty math books regarding addition + it has the
> amability to have rules of conservation.
>
> I pretty much see a real advantage in this behaviour in functional
> programming (map/reduce). (see the demonstrate.py), and it has a sense
> (if dict can be seen has vectors).
>
> I have been told to be a troll, but I am pretty serious.
>
> Since, I coded with luck, no internet, intuition, and a complete
> ignorance of the real meaning of the magic methods most of the time,
> thus the actual implementation of the addition surely needs a complete
> refactoring.
>
> Sheers,
> Bonne f?tes
> Julien
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111230/a6b172a6/attachment.html>

From guido at python.org  Fri Dec 30 18:02:41 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Dec 2011 10:02:41 -0700
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
Message-ID: <CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>

Hi Julien,

Having now looked at your code, I realize we are not talking about the same
concepts at all.

What I meant is similar to set union on the keys, where if a key exists in
both dicts, the value in the result is equal to one of the values in the
operands (and if the value is the same for both operands, that value is
also the result value).

But looking at your code, you are more interested in *elementwise*
addition, so that {'x': 1} + {'x': 2} results in {'x': 3}, and missing keys
are considered to be zero. (For disjoint key sets, your result is the same
as mine. :-)

I think this would be a nice thing for some applications, but I'm doubtful
that it's of enough interest to add to the standard library, let alone make
it a standard feature of the built-in dict class. One problem is that the
dict API is already very rich (dozens of methods and operations) Another
problem is that your proposed class solves only part of the puzzle of
elementwise operations -- if you have elementwise addition for dicts, why
not for lists? But l1+l2 already has a meaning for list: concatenation.
We've discussed this before. NumPy has got its own solution (a1+a2 does
elementwise addition if a1 and a2 are NumPy arrays), but for the stdlib
we've always decided that it's better to leave the choice up to the user,
the app or the framework (like NumPy), instead of making a choice in the
stdlib that causes backward incompatibility.

(Had I known all this 22 years ago, maybe I could have chosen a different
operator for list concatenation. But I don't think the issue is important
enough to fix, even if we ever did a Python 4.)

--Guido

On Fri, Dec 30, 2011 at 9:40 AM, Guido van Rossum <guido at python.org> wrote:

> Hi Julien,
>
> Don't despair! I have tried to get people to warm up to dict addition too
> -- in fact it was my counter-proposal at the time when we were considering
> adding sets to the language. I will look at your proposal, but I have a
> point of order first: this should be discussed on python-ideas, not on
> python-dev. I have added python-ideas to the thread and moved python-dev to
> Bcc, so followups will hopefully all go to python-ideas.
>
> --Guido
>
>
> On Fri, Dec 30, 2011 at 7:26 AM, julien tayon <julien at tayon.net> wrote:
>
>> Hello,
>> Sorry to annoy the very busy core devs :) out of the blue
>>
>> I quite noticed people were
>> 1) wanting to have a new dict for Xmas
>> 2) strongly resenting dict addition.
>>
>> Even though I am not a good developper, I have come to a definition of
>> addition that would follow algebraic rules, and not something of a
>> dutch logic. (it is a jest, not a troll)
>>
>> I propose the following code to validate my point of view regarding
>> the dictionnatry addition as a proof of concept :
>> https://github.com/jul/ADictAdd_iction/blob/master/test.py
>>
>> It follows all my dusty math books regarding addition + it has the
>> amability to have rules of conservation.
>>
>> I pretty much see a real advantage in this behaviour in functional
>> programming (map/reduce). (see the demonstrate.py), and it has a sense
>> (if dict can be seen has vectors).
>>
>> I have been told to be a troll, but I am pretty serious.
>>
>> Since, I coded with luck, no internet, intuition, and a complete
>> ignorance of the real meaning of the magic methods most of the time,
>> thus the actual implementation of the addition surely needs a complete
>> refactoring.
>>
>> Sheers,
>> Bonne f?tes
>> Julien
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111230/84bc958f/attachment.html>

From ericsnowcurrently at gmail.com  Fri Dec 30 18:12:34 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 30 Dec 2011 10:12:34 -0700
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
Message-ID: <CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>

On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido at python.org> wrote:
> What I meant is similar to set union on the keys, where if a key exists in
> both dicts, the value in the result is equal to one of the values in the
> operands (and if the value is the same for both operands, that value is also
> the result value).

+1

This is the one I was thinking of too.

-eric


From bruce at leapyear.org  Fri Dec 30 20:37:21 2011
From: bruce at leapyear.org (Bruce Leban)
Date: Fri, 30 Dec 2011 11:37:21 -0800
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
Message-ID: <CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>

On Fri, Dec 30, 2011 at 9:12 AM, Eric Snow <ericsnowcurrently at gmail.com>wrote:

> On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido at python.org>
> wrote:
> > What I meant is similar to set union on the keys, where if a key exists
> in
> > both dicts, the value in the result is equal to one of the values in the
> > operands (and if the value is the same for both operands, that value is
> also
> > the result value).
>
> +1
>
> This is the one I was thinking of too.
>
> -eric
>

My first thought on reading Julien's message was that {1:2} + {1:3} =>
{1:5} would be useful when doing reduction,  There already is an operation
that combines two dictionaries and does not combine values: update. Guido's
version of + is a shorthand for:

def __add__(self, other):
  result = self.copy()
  return result.update(other)


Is saving one line of code be enough of a benefit?

I think dicts are complex enough that if new functionality is added, it
would be better to use named methods which can have meaningful names and
are easier look up in the documentation. Furthermore, methods are more
flexible as this example shows:

def merge(self, other, merger=lambda x, y: x + y):
  """Merges another dictionary into this one, combining values."""
  for k in other:
    if k in self:
      self[k] = merger(self[k], other[k])
    else:
      self[k] = other[k]


--- Bruce
Follow me: http://www.twitter.com/Vroo http://www.vroospeak.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111230/c3a243b8/attachment.html>

From jkbbwr at gmail.com  Fri Dec 30 20:39:13 2011
From: jkbbwr at gmail.com (Jakob Bowyer)
Date: Fri, 30 Dec 2011 19:39:13 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
Message-ID: <63E20FB6-D6CC-407E-92A0-AD8D55A625C4@gmail.com>

Not that I'm smart enough to really contribute to these discussions but wouldn't it be an idea to have dictionary addition turn the values from d2 into d1 so that

d1 = {1:1,2:2,3:3} 
d2 = {1:2,2:2,4:4}
d3 = d1 + d2 
print d3
# {1:2,2:2,3:3,4:4}
# but 
d4 = d2 + d1
print d4
# {1:1,2:2,3:3,4:4}

Meaning that replacement is conditional on which side the other dictionary is on with the + operator. This might be harder to understand. So it's not a great idea... Still throwing it in the ring is the only way to learn

Sent from my iPad

On 30 Dec 2011, at 17:12, Eric Snow <ericsnowcurrently at gmail.com> wrote:

> On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido at python.org> wrote:
>> What I meant is similar to set union on the keys, where if a key exists in
>> both dicts, the value in the result is equal to one of the values in the
>> operands (and if the value is the same for both operands, that value is also
>> the result value).
> 
> +1
> 
> This is the one I was thinking of too.
> 
> -eric
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas


From nathan at cmu.edu  Fri Dec 30 21:01:54 2011
From: nathan at cmu.edu (Nathan Schneider)
Date: Fri, 30 Dec 2011 15:01:54 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
Message-ID: <CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>

On Fri, Dec 30, 2011 at 2:37 PM, Bruce Leban <bruce at leapyear.org> wrote:
>
> On Fri, Dec 30, 2011 at 9:12 AM, Eric Snow <ericsnowcurrently at gmail.com>
> wrote:
>>
>> On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido at python.org>
>> wrote:
>> > What I meant is similar to set union on the keys, where if a key exists
>> > in
>> > both dicts, the value in the result is equal to one of the values in the
>> > operands (and if the value is the same for both operands, that value is
>> > also
>> > the result value).
>>
>> +1
>>
>> This is the one I was thinking of too.
>>
>> -eric
>
>
> My first thought on reading Julien's message was that {1:2} + {1:3} => {1:5}
> would be useful when doing reduction, ?There already is an operation that
> combines two dictionaries and does not combine values: update. Guido's
> version of + is a shorthand for:
>
> def __add__(self, other):
> ? result = self.copy()
> ? return result.update(other)

The Counter class already supports this with the + operator. (It also
supports element-wise max with the | operator.) Perhaps it would be
useful to extend this support to dict, especially if there is a use
case for element-wise string concatenation.

As for a non-mutating alternative to update(): I have encountered a
number of scenarios in which this is desirable. A previous suggestion
[1] was to add a replace() method along these lines. But I think an
operator would make it clearer which is the mutating vs. non-mutating
one.

Instead of +, I think << would be a better choice for this sort of
modified concatenation/asymmetric union operation. For example, we can
write:

class Xdict(dict):
    def __lshift__(x, y):
        return Xdict(x.items() + y.items())

d = Xdict({'a': 9, 'b': 12})
d << {'a': 3, 'c': 8} # result: {'a': 3, 'b': 12, 'c': 8}
d['a']==9 # True

The rationale for <<:
 *  __add__ (+) is quite common, and thus banning the addition of two
dicts probably catches a lot of bugs. Moreover, when dict values are
numeric, a user might expect element-wise addition (as in Counter).
 *  __or__ (|) is classically a commutative operation, and is already
overloaded by set and Counter.
 * Counter does not already overload <<, and thus could be extended to
support the same behavior.
 * As far as I am aware, the only present use of << is for integer
shifting, so this new idiom would not pose a conflict.

Cheers,
Nathan

[1] http://groups.google.com/group/python-ideas/browse_thread/thread/275bbd1acfa8d433/6c87b56021ca5f44


>
> Is saving one line of code be enough of a benefit?
>
> I think dicts are complex enough that if new functionality is added, it
> would be better to use named methods which can have meaningful names and are
> easier look up in the documentation. Furthermore, methods are more flexible
> as this example shows:
>
> def merge(self, other, merger=lambda x, y: x + y):
> ? """Merges another dictionary into this one, combining values."""
> ? for k in other:
> ? ? if k in self:
> ? ? ? self[k] = merger(self[k], other[k])
> ? ? else:
> ? ? ? self[k] = other[k]
>
>
> --- Bruce
> Follow me:?http://www.twitter.com/Vroo?http://www.vroospeak.com
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From tjreedy at udel.edu  Fri Dec 30 21:29:01 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 30 Dec 2011 15:29:01 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
Message-ID: <jdl6ul$b5b$1@dough.gmane.org>


> On Fri, Dec 30, 2011 at 7:26 AM, julien tayon
> <julien at tayon.net
> <mailto:julien at tayon.net>> wrote:

>     I quite noticed people were
>     1) wanting to have a new dict for Xmas

Someone has proposed a modified version of the current dict type. The 
questions are whether a) it will behave correctly in all situations; b) 
give better overall performance; and c) be as easily maintainable.

>     2) strongly resenting dict addition.

This strikes me as a nonsensical statement. As noted below, we already 
have a type of 'dict addition' -- and even 'dict subtraction'. 
Counterfactual negative statements like this lead to the impression that 
you are trolling, even if you do not intend to.

>     I propose the following code to validate my point of view regarding
>     the dictionnatry addition as a proof of concept :
>     https://github.com/jul/ADictAdd_iction/blob/master/test.py

This link any gives the tests, not the accu_dict itself. So there is no 
way to know from the above exactly what you are proposing. From what 
Guido wrote in his next response, I am guessing that it is something 
like collections.Counter, which is an implementation of the multiset idea.

 >>> c = Counter(a=3, b=1)
 >>> d = Counter(a=1, b=2)
 >>> c + d                       # add two counters together:  c[x] + d[x]
Counter({'a': 4, 'b': 3})

 >>> c - d                       # subtract (keeping only positive counts)
Counter({'a': 2})

Tagging numbers by keys is a generalization of tagging them by position 
(index).  So this is generalized vector arithmetic and that is what your 
tests test.

------------------------
You have mixed up some of the terminology.

     def test_add_permut(self):
         """( a + b ) + c = a + ( b + c )"""

This is the associative law, not 'permutative'.

     def test_permut_scal_mul(self):
         """ ( an_int * other_scalar ) * a = an_int ( other_scalar * a )"""

Again, associativity (of scalar multiplication).
---

     def test_other_mul_scal_commut(self):
         """ ( an_int + other_scalar ) a = an_int * a + other_scalar * a """

This is a distributive law.

     def test_scal_lin_combo(self):
         """ an_int ( a + b) = an_int *a + an_int * b """

So is this.
---
     def test_permutation(self):
         """a + b = b + a """

This is commutivity, which you had right earlier in the file:

     def test_commutativity(self):
         """ a + b = b + a """
---

     def test_neutral(self):
         """ a + neutral = a """

Not exactly incorrect, but this is a test of the additive identify, 
which is usually called 'zero' rather than 'neutral'.

     def test_neutral_mul_scal(self):
         """ 1 * a = a """

while this tests the scalar multiplicative identify

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Fri Dec 30 21:33:43 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 30 Dec 2011 15:33:43 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
Message-ID: <jdl77e$cji$1@dough.gmane.org>

On 12/30/2011 12:02 PM, Guido van Rossum wrote:

> But looking at your code, you are more interested in *elementwise*
> addition, so that {'x': 1} + {'x': 2} results in {'x': 3}, and missing
> keys are considered to be zero.

This is part of what collections.Counter does.

> (Had I known all this 22 years ago, maybe I could have chosen a
> different operator for list concatenation.

In base 1 notation, count addition amounts to sequence concatenation. So 
I think you intuitively made the right choice.

-- 
Terry Jan Reedy



From arnodel at gmail.com  Fri Dec 30 21:45:50 2011
From: arnodel at gmail.com (Arnaud Delobelle)
Date: Fri, 30 Dec 2011 20:45:50 +0000
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdl6ul$b5b$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<jdl6ul$b5b$1@dough.gmane.org>
Message-ID: <CAJ6cK1aqSFdEgEdef1ahGG_=OAcvi=AOkHfNDOu9=8pNo4jRoQ@mail.gmail.com>

On 30 December 2011 20:29, Terry Reedy <tjreedy at udel.edu> wrote:
[...]
> You have mixed up some of the terminology.
[...]
> ---
> ? ?def test_permutation(self):
> ? ? ? ?"""a + b = b + a """
>
> This is commutivity, which you had right earlier in the file:

You mean commutativity :)

[...]
> ? ?def test_neutral(self):
> ? ? ? ?""" a + neutral = a """
>
> Not exactly incorrect, but this is a test of the additive identify, which is
> usually called 'zero' rather than 'neutral'.

You mean additive identity (by the way, I think Julien used the word
'neutral' because in French we call it '?l?ment neutre')

> ? ?def test_neutral_mul_scal(self):
> ? ? ? ?""" 1 * a = a """
>
> while this tests the scalar multiplicative identify

You mean multiplicative identity :)

Nit-picklingly yours,

-- 
Arnaud


From dreamingforward at gmail.com  Fri Dec 30 23:00:14 2011
From: dreamingforward at gmail.com (Mark Janssen)
Date: Fri, 30 Dec 2011 16:00:14 -0600
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
Message-ID: <CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>

On Fri, Dec 30, 2011 at 2:01 PM, Nathan Schneider <nathan at cmu.edu> wrote:
> On Fri, Dec 30, 2011 at 2:37 PM, Bruce Leban <bruce at leapyear.org> wrote:
>>
>> On Fri, Dec 30, 2011 at 9:12 AM, Eric Snow <ericsnowcurrently at gmail.com>
>> wrote:
>>>
>>> On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido at python.org>
>>> wrote:
>>> > What I meant is similar to set union on the keys, where if a key exists
>>> > in
>>> > both dicts, the value in the result is equal to one of the values in the
>>> > operands (and if the value is the same for both operands, that value is
>>> > also
>>> > the result value).
>>>
>>> +1

Far more powerful is to allow an optional "collision" function that is
called *only* when a key collision occurs.   Special case the default
behavior so that little performance hit is made if no such function is
given upon initialization.  Add the functionality to defaultdict and
see if it gets more traction.

I think you'll find a lot of code can be refactored with this simple
abstraction.  It becomes trivial and canonical to create a graph class
for example (at least in implementation if not in interface.)  To add
that feature to the interface, you need to refactor the set/dict
relationship so that dict inherits from set.  Then you'll really
cooking with gas....

Winks,

mark


From guido at python.org  Sat Dec 31 16:52:09 2011
From: guido at python.org (Guido van Rossum)
Date: Sat, 31 Dec 2011 08:52:09 -0700
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAFpLVkx+jukfKBN8PkOqsFMcP07tMvM7H31-bZBh+_cCEOXwYQ@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAFpLVkx+jukfKBN8PkOqsFMcP07tMvM7H31-bZBh+_cCEOXwYQ@mail.gmail.com>
Message-ID: <CAP7+vJKDDZRgbAyjamQ=ygUN9QTzpTvqt6-dGyKL6MeyPOd9GQ@mail.gmail.com>

On Sat, Dec 31, 2011 at 2:16 AM, julien tayon <julien at tayon.net> wrote:

> Dear All :)
>
> 2011/12/30 Eric Snow <ericsnowcurrently at gmail.com>:
> > On Fri, Dec 30, 2011 at 10:02 AM, Guido van Rossum <guido at python.org>
> wrote:
> >> What I meant is similar to set union on the keys, where if a key exists
> in
> >> both dicts, the value in the result is equal to one of the values in the
> >> operands (and if the value is the same for both operands, that value is
> also
> >> the result value).
> >
> > +1
> >
> > This is the one I was thinking of too.
> >
> Well, since I have coded way too much in Perl, my altered sense of
> reality has come to a concept I may be introducing too early which is
> : algebrae.
>
> strings, lists, ... have a record algebrae.
> ndarray, accudict have a linear algebrae
> sets ... have sets algebrae.
>
> And much more algebrae exists wich all exists not only in my
> imagination, but also in math (wich I quite dislike). (Abelian
> stuff/Group/matrix/hilbert)
> All of these algebrae are consistent as long as any object in the
> chain of algebrae are following the same rules.
>
> And each of these are very legitimate (even though of course my dict
> addition is the best without trying to be obnoxious).
>
>
> I was kind of thinking of
> 1) giving a property to object called .. __algebrae__,
> 2) and through some magic being able to change the algebrae of an
> object on the fly.
>
> My twisted sense of reality inherited from Perl (but a little less
> than my math books) tells me There Is More Than One Way To
> Consistently Add/Mul/Div/Sub It.
>
>
> As a Proof of Concept I could deliver a monkeypatching of list() that
> makes it behave like an numpy array.
>

Please don't present this in terms of modifications of existing
functions/types/methods. Please use subclasses, new modules, new functions,
etc.


> But, at first I wish to concentrate on dict addition, since I can only
> steal a few hours connectivity per day ... So I will try to answer to
> everyone since I saw some spoilers of what I had hidden in my mind  :)
>
> Cheers
> Thanks
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111231/f6e45d40/attachment.html>

From julien at tayon.net  Sat Dec 31 17:07:46 2011
From: julien at tayon.net (julien tayon)
Date: Sat, 31 Dec 2011 17:07:46 +0100
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
Message-ID: <CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>

Hello,

I had tough time finding a connection to internet :)

@Arnaud + Terry
Thx, code corrected to have the right terminology.
at least my code really test for the good rules :)
But being precise matter, so that I changed all the terms to fit in.

@nathan

<< is a symbol indicating an asymetry for left and right value, and I
carefully tried to have a symetrical behaviour with left & right :
associativity, distribution, commutativity.

The whole point about using a symbol, is that it follow rules.

That is the reason why consistent_addition class is all about testing
linear algebraic rules (hopefully the right way, and now, I hope with
the right terminology).

But I have reason to think that there are more than way to do
addition. Linear algebrae is the most used and intuitive one for non
developer, and developer think of sets operations as more intuitive.
I guess, it all come down to one point. Which is what is a dict as a
mathematical object. If there is more than one choice, which has the
most benefits. Should there be more than one definition ?



@Guido  & Eric

In my book there are rules for sets.
The question is a dict the same as a vector or as a set ?

since sets are using logical operators for operations that are roughly
sets operations why not use & ^ | for sets operation on dict ?
it would be pretty consistent with sets operations.

( @jakkob )
In this case I would expect inconsistencies error when doing
dict( a = 1 ) | dict( a = 2 , b = 2 ) (thus key-> value are seen like
an atomic element of the set, unless a "value collision" operator is
given, and this collision operator would recursively and naturely
apply to the descendants)
because I expect both a | b == b | a and a + b = b + a

The problem would be indeed
list + list
vectorish behaviour is my prefered of course.
but if we shift the record algebrae addition to another symbol (let's
assume << ) we resolve the conflict. But then we have a problem of
retro compatibility.

We then may be tempted to use & | ^  (since it would be used for dict)
and then we have a problem :
should [ 1 , 2 ] & [ 3 , 2 ] mean set addition  or applying & to all
the element of the same rank ?

I'm pretty stucked myself in trying to be consistent.

Just for the record, why dit you not use "." (dot) for concatenation ?
  I know it is typographically unreadable on a messy screen, but is
there a better reason ?

@terry & Nathan Regarding Counter.
on
https://github.com/jul/ADictAdd_iction/blob/master/demonstrate.py#L120
you may notice that I do have all the property of counter, but counter
does not as -this dict- aggregate values and (sub) totals at the same
time.
It is very convenient for map reduce since it does some re-reduce
operations at reduce time. smart, no ? <:o)

@terry
Sorry for making assertive and carefree statements regarding strong
resentment. My bad.



@bruce,

truth is aim at similarity cosinus for dict, and I imagine a map
reduce on dict representing values of a model you want ideal = dict(
blue = 1 , height = 180, weight = 70, wage = 500 )
in the filter before the map reduce you would want to filter all
records close to this one by using similarity cosinus in this way :
filter( lambda model : cos( ideal, model) > .7 , all_model )
http://en.wikipedia.org/wiki/Cosine_similarity

of course I  will try to advocate dot product, and metrics.

As a result my real goal is to make people consider dict as vectors of
path to value => value not sets of key => value

To solve issues such as weighted decisions and non linear choices
(based on trigger or a value belonging to a set) I can fairly easily
concevie a projector (which would be a class transforming a vector in
a vector, but not with a a matrix, but with computed rules ).



@Mark Jansen

+1 are you telepath ?

Yes a key collision operator would be nice :

given for instance an apache log,  I will have segments of path, and I
may want keys to be the accumulated graph of a user on a website. so
my collision rule might be :
(key  being the referer value the page visited afterwards)
dict( a = b, a = c ) + dict( b = e ) = dict( a = dict( b = e ), a = c)

I thought of it, but I really loved the conservation rule. And I
feared people would think of it as too complicated. Keep It Simple I
learnt :)

@all
Wrapping everything to logicial sense.

I was thinking of supersets of object (thus out of the stdlib) that
would have different algebrae and could be used as casts on object to
redifine + - * / cos & | ^
and for different objects would behave consistently ex :
vector( dict ) vector( list ) vector( string )
would make dict & list & string behave like vectors  thus mainly
supporting elementwise operations

sets( dict ) , sets( string ) ...
would make dict string be sets of elements ... (with the original dict
addition design of GvR et al)

Each superset would have a unit test  for the operation verifying that
behaviour is consistent.  (associativity, distributivity, ...) what
about this solution ?

-- 
Friendly
jul


From tjreedy at udel.edu  Sat Dec 31 22:55:55 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 31 Dec 2011 16:55:55 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
Message-ID: <jdo0dk$7q3$1@dough.gmane.org>

On 12/31/2011 11:07 AM, julien tayon wrote:

> The question is a dict the same as a vector or as a set ?

A dict is a mapping, which is to say, a functional set of key,value 
pairs. 'Functional' means that the keys are unique. This is a key 
property (pun intended). A vector can be regarded as a functional set of 
count,value pairs. Some languages use dicts for vectors. Even in Python, 
people use dicts for sparse arrays of various dimensions, with 
unspecified pairs assumed to have a 0 value.

So it makes the most sense to me for d1 + d2 to be interpreted as set 
union, with some resolution of key conflicts to restore the functional 
property. Your vector interpretation of adding values only works for 
values that can be added.

> since sets are using logical operators for operations that are roughly
> sets operations why not use &  ^ | for sets operation on dict ?
> it would be pretty consistent with sets operations.

I agree, except that dicts are not just sets of pairs, but functional 
sets. If one wants to view a dict simply as a set of keys, values, or 
pairs, ignoring the functional property for the pairs, that is what the 
.keys(), .values(), and .items() set view methods are for. All the set 
operations are available on the resulting set view objects.

-- 
Terry Jan Reedy



From nathan at cmu.edu  Sat Dec 31 23:57:02 2011
From: nathan at cmu.edu (Nathan Schneider)
Date: Sat, 31 Dec 2011 17:57:02 -0500
Subject: [Python-ideas] [Python-Dev] hello,
	new dict addition for new eve ?
In-Reply-To: <jdo0dk$7q3$1@dough.gmane.org>
References: <CAFpLVkwzE0e8XTFLpc7BvdF3sSaeCYJJfTRVqHZsdw=FXSiEfA@mail.gmail.com>
	<CAP7+vJ+9M_CiepXVPyJRX3rDtk4wi+=gux5Ts3m62H-aTR=OwQ@mail.gmail.com>
	<CAP7+vJJB0u2MRV0-wX5mYWVbb_gSTYkPdq+6E-A294_rgrSk+Q@mail.gmail.com>
	<CALFfu7Bk3GS4A5aL+QL4+--wa_u1OJ_9p7EdxokVD-kAuWfneg@mail.gmail.com>
	<CAGu0Anuq1Vyh2FOLjSP3Et0+8aHbvZjqmpoRFjhi-yrCgzXiXw@mail.gmail.com>
	<CADQLQrU1AES_g4txnfOExWpk6VH5Utan-7E_EGn=3q7GkB_muw@mail.gmail.com>
	<CAMjeLr_AxWSyzCX73X6gVKRY=vmccEcPiEE5NmV+bfdqgFR12Q@mail.gmail.com>
	<CAFpLVkzX3HSie1J5LTuG6RFt-kQ7yMyyDiSiy-hLU-ceujMP4Q@mail.gmail.com>
	<CAFpLVkwMT=N+A_0ZiwD7fmiT2VMPq4XSoFnaG7=1bX8tmCt3Ng@mail.gmail.com>
	<jdo0dk$7q3$1@dough.gmane.org>
Message-ID: <CADQLQrUwjV5bZQVQpX-iR+z_HVoqhYeUG+LaZ7t7OvTXwj2=4w@mail.gmail.com>

On Sat, Dec 31, 2011 at 4:55 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 12/31/2011 11:07 AM, julien tayon wrote:
>
>> The question is a dict the same as a vector or as a set ?
>
>
> A dict is a mapping, which is to say, a functional set of key,value pairs.
> 'Functional' means that the keys are unique. This is a key property (pun
> intended). A vector can be regarded as a functional set of count,value
> pairs. Some languages use dicts for vectors. Even in Python, people use
> dicts for sparse arrays of various dimensions, with unspecified pairs
> assumed to have a 0 value.
>
> So it makes the most sense to me for d1 + d2 to be interpreted as set union,
> with some resolution of key conflicts to restore the functional property.
> Your vector interpretation of adding values only works for values that can
> be added.

It occurs to me that any argument for a vector interpretation of
dicts, with element-wise operations, should apply equally well to
lists and tuples, which are perhaps even more natural as
representations for vectors. But of course, the + operator for
sequences is interpreted as concatenation, not element-wise addition;
it would be strange if the reverse were true for dicts.

The necessary resolution of key conflicts is what makes "+" feel less
natural to me for dicts than it does for sequences, where the order of
the operands is transparently reflected in the output. For dicts, in
contrast, we are talking about a "lossy" union/combination where what
is lost depends on the order of the operands?this is less transparent
in the output. However it is spelled, though, I would be thrilled if
such an operation were built in. :)

Nathan

>
>> since sets are using logical operators for operations that are roughly
>> sets operations why not use & ?^ | for sets operation on dict ?
>> it would be pretty consistent with sets operations.
>
>
> I agree, except that dicts are not just sets of pairs, but functional sets.
> If one wants to view a dict simply as a set of keys, values, or pairs,
> ignoring the functional property for the pairs, that is what the .keys(),
> .values(), and .items() set view methods are for. All the set operations are
> available on the resulting set view objects.
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas